Full Code of ssssssss-team/spider-flow for AI

master c799cca99c7d cached
213 files
4.0 MB
1.1M tokens
1942 symbols
1 requests
Download .txt
Showing preview only (4,276K chars total). Download the full file or copy to clipboard to get everything.
Repository: ssssssss-team/spider-flow
Branch: master
Commit: c799cca99c7d
Files: 213
Total size: 4.0 MB

Directory structure:
gitextract_2yt2vx0u/

├── .gitattributes
├── .gitignore
├── Dockerfile
├── LICENSE
├── README.md
├── db/
│   └── spiderflow.sql
├── pom.xml
├── spider-flow-api/
│   ├── pom.xml
│   └── src/
│       └── main/
│           └── java/
│               └── org/
│                   └── spiderflow/
│                       ├── ExpressionEngine.java
│                       ├── Grammerable.java
│                       ├── annotation/
│                       │   ├── Comment.java
│                       │   ├── Example.java
│                       │   └── Return.java
│                       ├── common/
│                       │   └── CURDController.java
│                       ├── concurrent/
│                       │   ├── ChildPriorThreadSubmitStrategy.java
│                       │   ├── LinkedThreadSubmitStrategy.java
│                       │   ├── ParentPriorThreadSubmitStrategy.java
│                       │   ├── RandomThreadSubmitStrategy.java
│                       │   ├── SpiderFlowThreadPoolExecutor.java
│                       │   ├── SpiderFutureTask.java
│                       │   └── ThreadSubmitStrategy.java
│                       ├── context/
│                       │   ├── CookieContext.java
│                       │   ├── SpiderContext.java
│                       │   └── SpiderContextHolder.java
│                       ├── enums/
│                       │   ├── FlowNoticeType.java
│                       │   └── FlowNoticeWay.java
│                       ├── executor/
│                       │   ├── FunctionExecutor.java
│                       │   ├── FunctionExtension.java
│                       │   ├── PluginConfig.java
│                       │   └── ShapeExecutor.java
│                       ├── expression/
│                       │   └── DynamicMethod.java
│                       ├── io/
│                       │   ├── Line.java
│                       │   ├── RandomAccessFileReader.java
│                       │   └── SpiderResponse.java
│                       ├── listener/
│                       │   └── SpiderListener.java
│                       ├── model/
│                       │   ├── Grammer.java
│                       │   ├── JsonBean.java
│                       │   ├── Plugin.java
│                       │   ├── Shape.java
│                       │   ├── SpiderLog.java
│                       │   ├── SpiderNode.java
│                       │   └── SpiderOutput.java
│                       └── utils/
│                           └── Maps.java
├── spider-flow-core/
│   ├── pom.xml
│   └── src/
│       └── main/
│           └── java/
│               └── org/
│                   └── spiderflow/
│                       └── core/
│                           ├── Spider.java
│                           ├── executor/
│                           │   ├── function/
│                           │   │   ├── Base64FunctionExecutor.java
│                           │   │   ├── DateFunctionExecutor.java
│                           │   │   ├── ExtractFunctionExecutor.java
│                           │   │   ├── FileFunctionExecutor.java
│                           │   │   ├── JsonFunctionExecutor.java
│                           │   │   ├── ListFunctionExecutor.java
│                           │   │   ├── MD5FunctionExecutor.java
│                           │   │   ├── RandomFunctionExecutor.java
│                           │   │   ├── StringFunctionExecutor.java
│                           │   │   ├── ThreadFunctionExecutor.java
│                           │   │   ├── UrlFunctionExecutor.java
│                           │   │   └── extension/
│                           │   │       ├── ArrayFunctionExtension.java
│                           │   │       ├── DateFunctionExtension.java
│                           │   │       ├── ElementFunctionExtension.java
│                           │   │       ├── ElementsFunctionExtension.java
│                           │   │       ├── ListFunctionExtension.java
│                           │   │       ├── MapFunctionExtension.java
│                           │   │       ├── ObjectFunctionExtension.java
│                           │   │       ├── ResponseFunctionExtension.java
│                           │   │       ├── SqlRowSetExtension.java
│                           │   │       └── StringFunctionExtension.java
│                           │   └── shape/
│                           │       ├── CommentExecutor.java
│                           │       ├── ExecuteSQLExecutor.java
│                           │       ├── ForkJoinExecutor.java
│                           │       ├── FunctionExecutor.java
│                           │       ├── LoopExecutor.java
│                           │       ├── OutputExecutor.java
│                           │       ├── ProcessExecutor.java
│                           │       ├── RequestExecutor.java
│                           │       ├── StartExecutor.java
│                           │       └── VariableExecutor.java
│                           ├── expression/
│                           │   ├── DefaultExpressionEngine.java
│                           │   ├── ExpressionError.java
│                           │   ├── ExpressionGlobalVariables.java
│                           │   ├── ExpressionTemplate.java
│                           │   ├── ExpressionTemplateContext.java
│                           │   ├── interpreter/
│                           │   │   ├── AstInterpreter.java
│                           │   │   ├── JavaReflection.java
│                           │   │   └── Reflection.java
│                           │   └── parsing/
│                           │       ├── Ast.java
│                           │       ├── CharacterStream.java
│                           │       ├── Parser.java
│                           │       ├── Span.java
│                           │       ├── Token.java
│                           │       ├── TokenStream.java
│                           │       ├── TokenType.java
│                           │       └── Tokenizer.java
│                           ├── io/
│                           │   ├── HttpRequest.java
│                           │   └── HttpResponse.java
│                           ├── job/
│                           │   ├── SpiderJob.java
│                           │   ├── SpiderJobContext.java
│                           │   └── SpiderJobManager.java
│                           ├── mapper/
│                           │   ├── DataSourceMapper.java
│                           │   ├── FlowNoticeMapper.java
│                           │   ├── FunctionMapper.java
│                           │   ├── SpiderFlowMapper.java
│                           │   ├── TaskMapper.java
│                           │   └── VariableMapper.java
│                           ├── model/
│                           │   ├── DataSource.java
│                           │   ├── FlowNotice.java
│                           │   ├── Function.java
│                           │   ├── SpiderFlow.java
│                           │   ├── Task.java
│                           │   └── Variable.java
│                           ├── script/
│                           │   └── ScriptManager.java
│                           ├── serializer/
│                           │   └── FastJsonSerializer.java
│                           ├── service/
│                           │   ├── DataSourceService.java
│                           │   ├── FlowNoticeService.java
│                           │   ├── FunctionService.java
│                           │   ├── SpiderFlowService.java
│                           │   ├── TaskService.java
│                           │   └── VariableService.java
│                           └── utils/
│                               ├── DataSourceUtils.java
│                               ├── EmailUtils.java
│                               ├── ExecutorsUtils.java
│                               ├── ExpressionUtils.java
│                               ├── ExtractUtils.java
│                               ├── FileUtils.java
│                               └── SpiderFlowUtils.java
└── spider-flow-web/
    ├── pom.xml
    └── src/
        └── main/
            ├── java/
            │   └── org/
            │       └── spiderflow/
            │           ├── SpiderApplication.java
            │           ├── configuration/
            │           │   ├── ResourcesConfiguration.java
            │           │   └── WebSocketConfiguration.java
            │           ├── controller/
            │           │   ├── DataSourceController.java
            │           │   ├── FlowNoticeController.java
            │           │   ├── FunctionController.java
            │           │   ├── SpiderFlowController.java
            │           │   ├── SpiderRestController.java
            │           │   ├── TaskController.java
            │           │   └── VariableController.java
            │           ├── logback/
            │           │   ├── SpiderFlowFileAppender.java
            │           │   └── SpiderFlowWebSocketAppender.java
            │           ├── model/
            │           │   ├── SpiderWebSocketContext.java
            │           │   └── WebSocketEvent.java
            │           └── websocket/
            │               └── WebSocketEditorServer.java
            └── resources/
                ├── application.properties
                ├── logback-spring.xml
                └── static/
                    ├── css/
                    │   ├── easyui.css
                    │   ├── editor.css
                    │   ├── index.css
                    │   ├── layui-black-gray.css
                    │   └── layui-blue.css
                    ├── datasource-edit.html
                    ├── datasources.html
                    ├── editCron.html
                    ├── editor.html
                    ├── function-edit.html
                    ├── functions.html
                    ├── index.html
                    ├── js/
                    │   ├── canvas-viewer.js
                    │   ├── codemirror/
                    │   │   ├── codemirror.css
                    │   │   ├── codemirror.js
                    │   │   ├── dracula.css
                    │   │   ├── idea.css
                    │   │   ├── javascript.js
                    │   │   ├── placeholder.js
                    │   │   ├── show-hint.css
                    │   │   ├── show-hint.js
                    │   │   ├── spiderflow-hint.js
                    │   │   ├── spiderflow.js
                    │   │   └── sql.js
                    │   ├── common.js
                    │   ├── cron/
                    │   │   └── cron.js
                    │   ├── editor.js
                    │   ├── index.js
                    │   ├── jsontree/
                    │   │   ├── jsontree.css
                    │   │   └── jsontree.js
                    │   ├── layui/
                    │   │   ├── css/
                    │   │   │   ├── layui.css
                    │   │   │   ├── layui.mobile.css
                    │   │   │   └── modules/
                    │   │   │       ├── code.css
                    │   │   │       ├── laydate/
                    │   │   │       │   └── default/
                    │   │   │       │       └── laydate.css
                    │   │   │       └── layer/
                    │   │   │           └── default/
                    │   │   │               └── layer.css
                    │   │   ├── ext/
                    │   │   │   ├── eleTree/
                    │   │   │   │   ├── eleTree.css
                    │   │   │   │   └── eleTree.js
                    │   │   │   └── treeselect/
                    │   │   │       └── treeselect.js
                    │   │   ├── extends/
                    │   │   │   ├── formSelects-v4.css
                    │   │   │   ├── formSelects-v4.js
                    │   │   │   └── treeGrid.js
                    │   │   └── layui.all.js
                    │   ├── log-viewer.js
                    │   ├── mxgraph/
                    │   │   ├── css/
                    │   │   │   ├── common.css
                    │   │   │   └── explorer.css
                    │   │   ├── mxgraph.js
                    │   │   └── resources/
                    │   │       ├── editor.txt
                    │   │       ├── editor_de.txt
                    │   │       ├── editor_zh.txt
                    │   │       ├── graph.txt
                    │   │       ├── graph_de.txt
                    │   │       └── graph_zh.txt
                    │   └── spider-editor.js
                    ├── log.html
                    ├── resources/
                    │   └── templates/
                    │       ├── comment.html
                    │       ├── edge.html
                    │       ├── executeSql.html
                    │       ├── forkJoin.html
                    │       ├── function.html
                    │       ├── loop.html
                    │       ├── output.html
                    │       ├── process.html
                    │       ├── request.html
                    │       ├── root.html
                    │       ├── start.html
                    │       └── variable.html
                    ├── spiderList-notice.html
                    ├── spiderList.html
                    ├── task.html
                    ├── variable-edit.html
                    └── variables.html

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitattributes
================================================
*.js linguist-language=java
*.css linguist-language=java
*.html linguist-language=java

================================================
FILE: .gitignore
================================================
target
*.iml
out/
.idea
.classpath
.project
.settings
bin/
.myeclipse

================================================
FILE: Dockerfile
================================================
FROM java:8

MAINTAINER octopus

RUN mkdir -p /spider-flow

WORKDIR /spider-flow

EXPOSE 8088

ADD ./spider-flow-web/target/spider-flow.jar ./

CMD sleep 30;java -Djava.security.egd=file:/dev/./urandom -jar spider-flow.jar

================================================
FILE: LICENSE
================================================
MIT License

Copyright (c) 2019 小东

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


================================================
FILE: README.md
================================================
<p align="center">
    <img src="https://www.spiderflow.org/images/logo.svg" width="600">
</p>
<p align="center">
    <a target="_blank" href="https://www.oracle.com/technetwork/java/javase/downloads/index.html"><img src="https://img.shields.io/badge/JDK-1.8+-green.svg" /></a>
    <a target="_blank" href="https://www.spiderflow.org"><img src="https://img.shields.io/badge/Docs-latest-blue.svg"/></a>
    <a target="_blank" href="https://github.com/ssssssss-team/spider-flow/releases"><img src="https://img.shields.io/github/v/release/ssssssss-team/spider-flow?logo=github"></a>
    <a target="_blank" href='https://gitee.com/ssssssss-team/spider-flow'><img src="https://gitee.com/ssssssss-team/spider-flow/badge/star.svg?theme=white" /></a>
    <a target="_blank" href='https://github.com/ssssssss-team/spider-flow'><img src="https://img.shields.io/github/stars/ssssssss-team/spider-flow.svg?style=social"/></a>
    <a target="_blank" href="LICENSE"><img src="https://img.shields.io/:license-MIT-blue.svg"></a>
    <a target="_blank" href="https://shang.qq.com/wpa/qunwpa?idkey=10faa4cf9743e0aa379a72f2ad12a9e576c81462742143c8f3391b52e8c3ed8d"><img src="https://img.shields.io/badge/Join-QQGroup-blue"></a>
</p>

[介绍](#介绍) | [特性](#特性) | [插件](#插件) | <a target="_blank" href="http://demo.spiderflow.org">DEMO站点</a> | <a target="_blank" href="https://www.spiderflow.org">文档</a> | <a target="_blank" href="https://www.spiderflow.org/changelog.html">更新日志</a> | [截图](#项目部分截图) | [其它开源](#其它开源项目) | [免责声明](#免责声明)

## 介绍
平台以流程图的方式定义爬虫,是一个高度灵活可配置的爬虫平台

## 特性
- [x] 支持Xpath/JsonPath/css选择器/正则提取/混搭提取
- [x] 支持JSON/XML/二进制格式
- [x] 支持多数据源、SQL select/selectInt/selectOne/insert/update/delete
- [x] 支持爬取JS动态渲染(或ajax)的页面
- [x] 支持代理
- [x] 支持自动保存至数据库/文件
- [x] 常用字符串、日期、文件、加解密等函数
- [x] 支持插件扩展(自定义执行器,自定义方法)
- [x] 任务监控,任务日志
- [x] 支持HTTP接口
- [x] 支持Cookie自动管理
- [x] 支持自定义函数

## 插件
- [x] [Selenium插件](https://gitee.com/ssssssss-team/spider-flow-selenium)
- [x] [Redis插件](https://gitee.com/ssssssss-team/spider-flow-redis)
- [x] [OSS插件](https://gitee.com/ssssssss-team/spider-flow-oss)
- [x] [Mongodb插件](https://gitee.com/ssssssss-team/spider-flow-mongodb)
- [x] [IP代理池插件](https://gitee.com/ssssssss-team/spider-flow-proxypool)
- [x] [OCR识别插件](https://gitee.com/ssssssss-team/spider-flow-ocr)
- [x] [电子邮箱插件](https://gitee.com/ssssssss-team/spider-flow-mailbox)

## 项目部分截图
### 爬虫列表
![爬虫列表](https://images.gitee.com/uploads/images/2020/0412/104521_e1eb3fbb_297689.png "list.png")
### 爬虫测试
![爬虫测试](https://images.gitee.com/uploads/images/2020/0412/104659_b06dfbf0_297689.gif "test.gif")
### Debug
![Debug](https://images.gitee.com/uploads/images/2020/0412/104741_f9e1190e_297689.png "debug.png")
### 日志
![日志](https://images.gitee.com/uploads/images/2020/0412/104800_a757f569_297689.png "logo.png")

## 其它开源项目
- [spider-flow-vue,spider-flow的前端](https://gitee.com/ssssssss-team/spider-flow-vue)
- [magic-api,一个以XML为基础自动映射为HTTP接口的框架](https://gitee.com/ssssssss-team/magic-api)
- [magic-api-spring-boot-starter](https://gitee.com/ssssssss-team/magic-api-spring-boot-starter)


## 免责声明
请勿将`spider-flow`应用到任何可能会违反法律规定和道德约束的工作中,请友善使用`spider-flow`,遵守蜘蛛协议,不要将`spider-flow`用于任何非法用途。如您选择使用`spider-flow`即代表您遵守此协议,作者不承担任何由于您违反此协议带来任何的法律风险和损失,一切后果由您承担。


================================================
FILE: db/spiderflow.sql
================================================
SET FOREIGN_KEY_CHECKS=0;

CREATE DATABASE spiderflow;
USE spiderflow;

DROP TABLE IF EXISTS `sp_flow`;
CREATE TABLE `sp_flow` (
  `id` varchar(32) NOT NULL,
  `name` varchar(64) DEFAULT NULL COMMENT '任务名字',
  `xml` longtext DEFAULT NULL COMMENT 'xml表达式',
  `cron` varchar(255) DEFAULT NULL COMMENT 'corn表达式',
  `enabled` char(1) DEFAULT '0' COMMENT '任务是否启动,默认未启动',
  `create_date` datetime DEFAULT CURRENT_TIMESTAMP  COMMENT '创建时间',
  `last_execute_time` datetime DEFAULT NULL  COMMENT '上一次执行时间',
  `next_execute_time` datetime DEFAULT NULL   COMMENT '下一次执行时间',
  `execute_count` int(8) DEFAULT NULL  COMMENT '定时执行的已执行次数',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT '爬虫任务表';

INSERT INTO `sp_flow` VALUES ('b45fb98d2a564c23ba623a377d5e12e9', '爬取码云GVP', '<mxGraphModel>\n  <root>\n    <mxCell id=\"0\">\n      <JsonProperty as=\"data\">\n        {&quot;spiderName&quot;:&quot;爬取码云GVP&quot;,&quot;threadCount&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"1\" parent=\"0\"/>\n    <mxCell id=\"2\" value=\"开始\" style=\"start\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"80\" y=\"80\" width=\"24\" height=\"24\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;shape&quot;:&quot;start&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"5\" value=\"抓取首页\" style=\"request\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"180\" y=\"80\" width=\"24\" height=\"24\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;抓取首页&quot;,&quot;loopVariableName&quot;:&quot;&quot;,&quot;sleep&quot;:&quot;&quot;,&quot;timeout&quot;:&quot;&quot;,&quot;response-charset&quot;:&quot;&quot;,&quot;method&quot;:&quot;GET&quot;,&quot;body-type&quot;:&quot;none&quot;,&quot;body-content-type&quot;:&quot;text/plain&quot;,&quot;loopCount&quot;:&quot;&quot;,&quot;url&quot;:&quot;https://gitee.com/gvp/all&quot;,&quot;proxy&quot;:&quot;&quot;,&quot;request-body&quot;:[&quot;&quot;],&quot;follow-redirect&quot;:&quot;1&quot;,&quot;shape&quot;:&quot;request&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"6\" value=\"\" parent=\"1\" source=\"2\" target=\"5\" edge=\"1\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"7\" value=\"提取项目名、地址\" style=\"variable\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"330\" y=\"80\" width=\"24\" height=\"24\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;提取项目名、地址&quot;,&quot;loopVariableName&quot;:&quot;&quot;,&quot;variable-name&quot;:[&quot;projectUrls&quot;,&quot;projectNames&quot;],&quot;loopCount&quot;:&quot;&quot;,&quot;variable-value&quot;:[&quot;${extract.selectors(resp.html,&#39;.categorical-project-card a&#39;,&#39;attr&#39;,&#39;href&#39;)}&quot;,&quot;${extract.selectors(resp.html,&#39;.project-name&#39;)}&quot;],&quot;shape&quot;:&quot;variable&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"8\" value=\"\" parent=\"1\" source=\"5\" target=\"7\" edge=\"1\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"9\" value=\"抓取详情页\" style=\"request\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"450.16668701171875\" y=\"80\" width=\"24\" height=\"24\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;抓取详情页&quot;,&quot;loopVariableName&quot;:&quot;projectIndex&quot;,&quot;sleep&quot;:&quot;&quot;,&quot;timeout&quot;:&quot;&quot;,&quot;response-charset&quot;:&quot;&quot;,&quot;method&quot;:&quot;GET&quot;,&quot;body-type&quot;:&quot;none&quot;,&quot;body-content-type&quot;:&quot;text/plain&quot;,&quot;loopCount&quot;:&quot;10&quot;,&quot;url&quot;:&quot;https://gitee.com/${projectUrls[projectIndex]}&quot;,&quot;proxy&quot;:&quot;&quot;,&quot;request-body&quot;:[&quot;&quot;],&quot;follow-redirect&quot;:&quot;1&quot;,&quot;shape&quot;:&quot;request&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"10\" value=\"\" parent=\"1\" source=\"7\" target=\"9\" edge=\"1\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"12\" value=\"提取项目描述\" style=\"variable\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"550\" y=\"80\" width=\"24\" height=\"24\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;提取项目描述&quot;,&quot;loopVariableName&quot;:&quot;&quot;,&quot;variable-name&quot;:[&quot;projectDesc&quot;],&quot;loopCount&quot;:&quot;&quot;,&quot;variable-value&quot;:[&quot;${extract.selector(resp.html,&#39;.git-project-desc-text&#39;)}&quot;],&quot;shape&quot;:&quot;variable&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"13\" value=\"\" parent=\"1\" source=\"9\" target=\"12\" edge=\"1\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"14\" value=\"输出\" style=\"output\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"660.1666870117188\" y=\"80\" width=\"24\" height=\"24\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;输出&quot;,&quot;output-name&quot;:[&quot;项目名&quot;,&quot;项目地址&quot;,&quot;项目描述&quot;],&quot;output-value&quot;:[&quot;${projectNames[projectIndex]}&quot;,&quot;https://gitee.com${projectUrls[projectIndex]}&quot;,&quot;${projectDesc}&quot;],&quot;shape&quot;:&quot;output&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"15\" value=\"\" parent=\"1\" source=\"12\" target=\"14\" edge=\"1\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n  </root>\n</mxGraphModel>\n', null, '0', '2019-08-22 13:46:54', null, null, null);
INSERT INTO `sp_flow` VALUES ('f0a67f17ee1a498a9b2f4ca30556f3c3', '抓取每日菜价', '<mxGraphModel>\n  <root>\n    <mxCell id=\"0\">\n      <JsonProperty as=\"data\">\n        {&quot;spiderName&quot;:&quot;抓取每日菜价&quot;,&quot;threadCount&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"1\" parent=\"0\"/>\n    <mxCell id=\"2\" value=\"开始\" style=\"start\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"80\" y=\"80\" width=\"24\" height=\"24\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;shape&quot;:&quot;start&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"3\" value=\"开始抓取\" style=\"request\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"219.83334350585938\" y=\"80\" width=\"24\" height=\"24\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;开始抓取&quot;,&quot;loopVariableName&quot;:&quot;&quot;,&quot;sleep&quot;:&quot;&quot;,&quot;timeout&quot;:&quot;&quot;,&quot;response-charset&quot;:&quot;&quot;,&quot;method&quot;:&quot;GET&quot;,&quot;body-type&quot;:&quot;none&quot;,&quot;body-content-type&quot;:&quot;text/plain&quot;,&quot;loopCount&quot;:&quot;&quot;,&quot;url&quot;:&quot;http://www.beijingprice.cn:8086/price/priceToday/PageLoad/LoadPrice?jsoncallback=1&quot;,&quot;proxy&quot;:&quot;&quot;,&quot;request-body&quot;:[&quot;&quot;],&quot;follow-redirect&quot;:&quot;1&quot;,&quot;shape&quot;:&quot;request&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"4\" value=\"\" parent=\"1\" source=\"2\" target=\"3\" edge=\"1\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"5\" value=\"解析JSON\" style=\"variable\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"350\" y=\"80\" width=\"24\" height=\"24\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;解析JSON&quot;,&quot;loopVariableName&quot;:&quot;&quot;,&quot;variable-name&quot;:[&quot;jsonstr&quot;,&quot;jsondata&quot;,&quot;data&quot;],&quot;loopCount&quot;:&quot;&quot;,&quot;variable-value&quot;:[&quot;${string.substring(resp.html,2,resp.html.length()-1)}&quot;,&quot;${json.parse(jsonstr)}&quot;,&quot;${extract.jsonpath(jsondata[0],&#39;data&#39;)}&quot;],&quot;shape&quot;:&quot;variable&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"6\" value=\"\" parent=\"1\" source=\"3\" target=\"5\" edge=\"1\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"7\" value=\"输出\" style=\"output\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"480.16668701171875\" y=\"80\" width=\"24\" height=\"24\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;输出&quot;,&quot;loopVariableName&quot;:&quot;i&quot;,&quot;output-name&quot;:[&quot;菜名&quot;,&quot;菜价&quot;,&quot;单位&quot;],&quot;loopCount&quot;:&quot;${list.length(data)}&quot;,&quot;output-value&quot;:[&quot;${data[i].ItemName}&quot;,&quot;${data[i].Price04}&quot;,&quot;${data[i].ItemUnit}&quot;],&quot;shape&quot;:&quot;output&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"8\" value=\"\" parent=\"1\" source=\"5\" target=\"7\" edge=\"1\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n  </root>\n</mxGraphModel>\n', null, '0', '2019-08-22 13:48:22', null, null, null);
INSERT INTO `sp_flow` VALUES ('b4430885ba8349588d1220d37eac831d', '爬取开源中国动弹', '<mxGraphModel>\n  <root>\n    <mxCell id=\"0\">\n      <JsonProperty as=\"data\">\n        {&quot;spiderName&quot;:&quot;爬取开源中国动弹&quot;,&quot;threadCount&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"1\" parent=\"0\"/>\n    <mxCell id=\"2\" value=\"开始\" style=\"start\" vertex=\"1\" parent=\"1\">\n      <mxGeometry x=\"80\" y=\"80\" width=\"32\" height=\"32\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;shape&quot;:&quot;start&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"3\" value=\"爬取动弹\" style=\"request\" vertex=\"1\" parent=\"1\">\n      <mxGeometry x=\"220\" y=\"80\" width=\"32\" height=\"32\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;爬取动弹&quot;,&quot;loopVariableName&quot;:&quot;&quot;,&quot;sleep&quot;:&quot;&quot;,&quot;timeout&quot;:&quot;&quot;,&quot;response-charset&quot;:&quot;&quot;,&quot;method&quot;:&quot;GET&quot;,&quot;parameter-name&quot;:[&quot;type&quot;,&quot;lastLogId&quot;],&quot;body-type&quot;:&quot;none&quot;,&quot;body-content-type&quot;:&quot;text/plain&quot;,&quot;loopCount&quot;:&quot;&quot;,&quot;url&quot;:&quot;https://www.oschina.net/tweets/widgets/_tweet_index_list &quot;,&quot;proxy&quot;:&quot;&quot;,&quot;parameter-value&quot;:[&quot;ajax&quot;,&quot;${lastLogId}&quot;],&quot;request-body&quot;:&quot;&quot;,&quot;follow-redirect&quot;:&quot;1&quot;,&quot;tls-validate&quot;:&quot;1&quot;,&quot;shape&quot;:&quot;request&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"4\" value=\"\" edge=\"1\" parent=\"1\" source=\"2\" target=\"3\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"5\" value=\"提取lastLogId以及tweets\" style=\"variable\" vertex=\"1\" parent=\"1\">\n      <mxGeometry x=\"340\" y=\"80\" width=\"32\" height=\"32\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;提取lastLogId以及tweets&quot;,&quot;loopVariableName&quot;:&quot;&quot;,&quot;variable-name&quot;:[&quot;lastLogId&quot;,&quot;tweets&quot;,&quot;fetchCount&quot;],&quot;loopCount&quot;:&quot;&quot;,&quot;variable-value&quot;:[&quot;${resp.selector(&#39;.tweet-item:last-child&#39;).attr(&#39;data-tweet-id&#39;)}&quot;,&quot;${resp.selectors(&#39;.tweet-item[data-tweet-id]&#39;)}&quot;,&quot;${fetchCount == null ? 0 : fetchCount + 1}&quot;],&quot;shape&quot;:&quot;variable&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"6\" value=\"\" edge=\"1\" parent=\"1\" source=\"3\" target=\"5\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"7\" value=\"循环\" style=\"loop\" vertex=\"1\" parent=\"1\">\n      <mxGeometry x=\"340\" y=\"250\" width=\"32\" height=\"32\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;循环&quot;,&quot;loopVariableName&quot;:&quot;index&quot;,&quot;loopCount&quot;:&quot;${list.length(tweets)}&quot;,&quot;shape&quot;:&quot;loop&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"8\" value=\"\" edge=\"1\" parent=\"1\" source=\"5\" target=\"7\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"9\" value=\"提取详细信息\" style=\"variable\" vertex=\"1\" parent=\"1\">\n      <mxGeometry x=\"340\" y=\"340\" width=\"32\" height=\"32\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;提取详细信息&quot;,&quot;loopVariableName&quot;:&quot;&quot;,&quot;variable-name&quot;:[&quot;content&quot;,&quot;author&quot;,&quot;like&quot;,&quot;reply&quot;,&quot;publishTime&quot;],&quot;loopCount&quot;:&quot;&quot;,&quot;variable-value&quot;:[&quot;${tweets[index].selector(&#39;.text&#39;).text()}&quot;,&quot;${tweets[index].selector(&#39;.user&#39;).text()}&quot;,&quot;${tweets[index].selector(&#39;.like span&#39;).text()}&quot;,&quot;${tweets[index].selector(&#39;.reply span&#39;).text()}&quot;,&quot;${tweets[index].selector(&#39;.date&#39;).regx(&#39;(.*?)&amp;nbsp&#39;)}&quot;],&quot;shape&quot;:&quot;variable&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"10\" value=\"\" edge=\"1\" parent=\"1\" source=\"7\" target=\"9\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"11\" value=\"输出\" style=\"output\" vertex=\"1\" parent=\"1\">\n      <mxGeometry x=\"340\" y=\"430\" width=\"32\" height=\"32\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;输出&quot;,&quot;loopVariableName&quot;:&quot;&quot;,&quot;output-name&quot;:[&quot;作者&quot;,&quot;内容&quot;,&quot;点赞数&quot;,&quot;评论数&quot;,&quot;发布时间&quot;],&quot;loopCount&quot;:&quot;&quot;,&quot;output-value&quot;:[&quot;${author}&quot;,&quot;${content}&quot;,&quot;${like}&quot;,&quot;${reply}&quot;,&quot;${publishTime}&quot;],&quot;shape&quot;:&quot;output&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"12\" value=\"\" edge=\"1\" parent=\"1\" source=\"9\" target=\"11\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"13\" value=\"爬取3页\" edge=\"1\" parent=\"1\" source=\"5\" target=\"3\">\n      <mxGeometry x=\"-0.0312\" y=\"-20\" relative=\"1\" as=\"geometry\">\n        <Array as=\"points\">\n          <mxPoint x=\"356\" y=\"180\"/>\n          <mxPoint x=\"236\" y=\"180\"/>\n        </Array>\n        <mxPoint as=\"offset\"/>\n      </mxGeometry>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;爬取5页&quot;,&quot;condition&quot;:&quot;${fetchCount &lt; 3}&quot;}\n      </JsonProperty>\n    </mxCell>\n  </root>\n</mxGraphModel>\n', '', '0', '2019-11-03 17:02:49', '2019-11-04 10:11:31', '2019-11-03 17:30:56', '3');
INSERT INTO `sp_flow` VALUES ('663aaa5e36a84c9594ef3cfd6738e9a7', '百度热点', '<mxGraphModel>\n  <root>\n    <mxCell id=\"0\">\n      <JsonProperty as=\"data\">\n        {&quot;spiderName&quot;:&quot;百度热点&quot;,&quot;threadCount&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"1\" parent=\"0\"/>\n    <mxCell id=\"2\" value=\"开始\" style=\"start\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"80\" y=\"80\" width=\"32\" height=\"32\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;shape&quot;:&quot;start&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"3\" value=\"开始抓取\" style=\"request\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"220\" y=\"80\" width=\"32\" height=\"32\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;开始抓取&quot;,&quot;loopVariableName&quot;:&quot;&quot;,&quot;sleep&quot;:&quot;&quot;,&quot;timeout&quot;:&quot;&quot;,&quot;response-charset&quot;:&quot;gbk&quot;,&quot;method&quot;:&quot;GET&quot;,&quot;body-type&quot;:&quot;none&quot;,&quot;body-content-type&quot;:&quot;text/plain&quot;,&quot;loopCount&quot;:&quot;&quot;,&quot;url&quot;:&quot;https://top.baidu.com/buzz?b=1&amp;fr=topindex&quot;,&quot;proxy&quot;:&quot;&quot;,&quot;request-body&quot;:&quot;&quot;,&quot;follow-redirect&quot;:&quot;1&quot;,&quot;tls-validate&quot;:&quot;1&quot;,&quot;shape&quot;:&quot;request&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"4\" value=\"定义变量\" style=\"variable\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"360\" y=\"80\" width=\"32\" height=\"32\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;定义变量&quot;,&quot;loopVariableName&quot;:&quot;&quot;,&quot;variable-name&quot;:[&quot;elementbd&quot;],&quot;loopCount&quot;:&quot;&quot;,&quot;variable-value&quot;:[&quot;${resp.xpaths(&#39;//*[@id=\\&quot;main\\&quot;]/div[2]/div/table/tbody/tr&#39;)}&quot;],&quot;shape&quot;:&quot;variable&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"5\" value=\"输出\" style=\"output\" parent=\"1\" vertex=\"1\">\n      <mxGeometry x=\"480\" y=\"80\" width=\"32\" height=\"32\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;输出&quot;,&quot;loopVariableName&quot;:&quot;i&quot;,&quot;output-name&quot;:[&quot;名称&quot;,&quot;地址&quot;,&quot;百度指数&quot;,&quot;2&quot;],&quot;loopCount&quot;:&quot;${elementbd.size()-1}&quot;,&quot;output-value&quot;:[&quot;${elementbd[i+1].xpath(&#39;//td[2]/a[1]/text()&#39;)}&quot;,&quot;${elementbd[i+1].xpath(&#39;//td[2]/a[1]/@href&#39;)}&quot;,&quot;${elementbd[i+1].xpath(&#39;//td[4]/span/text()&#39;)}&quot;,&quot;${elementbd[i+1].xpath(&#39;//td[3]/a[2]/text()&#39;)}&quot;],&quot;shape&quot;:&quot;output&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"6\" value=\"\" parent=\"1\" source=\"2\" target=\"3\" edge=\"1\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"7\" value=\"\" parent=\"1\" source=\"3\" target=\"4\" edge=\"1\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n    <mxCell id=\"8\" value=\"\" parent=\"1\" source=\"4\" target=\"5\" edge=\"1\">\n      <mxGeometry relative=\"1\" as=\"geometry\"/>\n      <JsonProperty as=\"data\">\n        {&quot;value&quot;:&quot;&quot;,&quot;condition&quot;:&quot;&quot;}\n      </JsonProperty>\n    </mxCell>\n  </root>\n</mxGraphModel>\n', '0 0/30 * * * ? *', '1', '2019-10-20 17:24:21', '2019-11-04 08:52:05', '2019-10-30 14:52:39', '45');

DROP TABLE IF EXISTS `sp_datasource`;
CREATE TABLE `sp_datasource` (
  `id` varchar(32) NOT NULL,
  `name` varchar(255) DEFAULT NULL,
  `driver_class_name` varchar(255) DEFAULT NULL,
  `jdbc_url` varchar(255) DEFAULT NULL,
  `username` varchar(64) DEFAULT NULL,
  `password` varchar(32) DEFAULT NULL,
  `create_date` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

DROP TABLE IF EXISTS `sp_variable`;
CREATE TABLE `sp_variable` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `name` varchar(32) DEFAULT NULL COMMENT '变量名',
  `value` varchar(512) DEFAULT NULL COMMENT '变量值',
  `description` varchar(255) DEFAULT NULL COMMENT '变量描述',
  `create_date` datetime DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=utf8mb4;

/* v0.3.0 新增 */
DROP TABLE IF EXISTS `sp_task`;
CREATE TABLE `sp_task` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `flow_id` varchar(32) NOT NULL,
  `begin_time` datetime DEFAULT NULL,
  `end_time` datetime DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=utf8mb4;

/* v0.4.0 新增 */
DROP TABLE IF EXISTS `sp_function`;
CREATE TABLE `sp_function`  (
  `id` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT '函数名',
  `parameter` varchar(512) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL COMMENT '参数',
  `script` text CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL COMMENT 'js脚本',
  `create_date` datetime(0) NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

/* v0.5.0 新增 */
DROP TABLE IF EXISTS `sp_flow_notice`;
CREATE TABLE `sp_flow_notice` (
  `id` varchar(32) NOT NULL,
  `recipients` varchar(200) DEFAULT NULL COMMENT '收件人',
  `notice_way` char(10) DEFAULT NULL COMMENT '通知方式',
  `start_notice` char(1) DEFAULT '0' COMMENT '流程开始通知:1:开启通知,0:关闭通知',
  `exception_notice` char(1) DEFAULT '0' COMMENT '流程异常通知:1:开启通知,0:关闭通知',
  `end_notice` char(1) DEFAULT '0' COMMENT '流程结束通知:1:开启通知,0:关闭通知',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT '爬虫任务通知表';

================================================
FILE: pom.xml
================================================
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>org.spiderflow</groupId>
	<artifactId>spider-flow</artifactId>
	<version>0.5.0</version>
	<packaging>pom</packaging>
	<name>spider-flow</name>
	<url>https://gitee.com/jmxd/spider-flow</url>

	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.0.7.RELEASE</version>
	</parent>
	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<spider-flow.version>${project.version}</spider-flow.version>
		<alibaba.fastjson.version>1.2.83</alibaba.fastjson.version>
		<alibaba.druid.version>1.1.16</alibaba.druid.version>
		<alibaba.transmittable.version>2.11.5</alibaba.transmittable.version>
		<mybatis.plus.version>3.1.0</mybatis.plus.version>
		<apache.commons.text.verion>1.6</apache.commons.text.verion>
		<apache.commons.csv.verion>1.8</apache.commons.csv.verion>
		<commons.io.version>2.7</commons.io.version>
		<guava.version>28.2-jre</guava.version>
		<jsoup.version>1.11.3</jsoup.version>
		<xsoup.version>0.3.1</xsoup.version>
	</properties>

	<dependencies>
		<!-- spring-boot相关配置 -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-quartz</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-mail</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-jdbc</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-websocket</artifactId>
		</dependency>
		<!-- 数据库相关 -->
		<dependency>
			<groupId>com.baomidou</groupId>
			<artifactId>mybatis-plus-boot-starter</artifactId>
			<version>${mybatis.plus.version}</version>
		</dependency>
		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
		</dependency>
		<!-- alibaba相关包 -->
		<dependency>
			<groupId>com.alibaba</groupId>
			<artifactId>fastjson</artifactId>
			<version>${alibaba.fastjson.version}</version>
		</dependency>
		<dependency>
			<groupId>com.alibaba</groupId>
			<artifactId>druid-spring-boot-starter</artifactId>
			<version>${alibaba.druid.version}</version>
		</dependency>
		<dependency>
			<groupId>com.alibaba</groupId>
			<artifactId>transmittable-thread-local</artifactId>
			<version>${alibaba.transmittable.version}</version>
		</dependency>
		<!-- apache commons相关 -->
		<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-text</artifactId>
			<version>${apache.commons.text.verion}</version>
		</dependency>
		<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-csv</artifactId>
			<version>${apache.commons.csv.verion}</version>
		</dependency>
		<!-- commons包 -->
		<dependency>
			<groupId>commons-io</groupId>
			<artifactId>commons-io</artifactId>
			<version>${commons.io.version}</version>
		</dependency>
		<dependency>
			<groupId>commons-codec</groupId>
			<artifactId>commons-codec</artifactId>
		</dependency>
		<!-- 其它包 -->
		<dependency>
			<groupId>com.google.guava</groupId>
			<artifactId>guava</artifactId>
			<version>${guava.version}</version>
		</dependency>
		<dependency>
			<groupId>org.jsoup</groupId>
			<artifactId>jsoup</artifactId>
			<version>${jsoup.version}</version>
		</dependency>
		<dependency>
			<groupId>us.codecraft</groupId>
			<artifactId>xsoup</artifactId>
			<version>${xsoup.version}</version>
		</dependency>
	</dependencies>

	<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>org.spiderflow</groupId>
				<artifactId>spider-flow-api</artifactId>
				<version>${spider-flow.version}</version>
			</dependency>
			<dependency>
				<groupId>org.spiderflow</groupId>
				<artifactId>spider-flow-core</artifactId>
				<version>${spider-flow.version}</version>
			</dependency>
			<dependency>
				<groupId>org.spiderflow</groupId>
				<artifactId>spider-flow-selenium</artifactId>
				<version>${spider-flow.version}</version>
			</dependency>
			<dependency>
				<groupId>org.spiderflow</groupId>
				<artifactId>spider-flow-proxypool</artifactId>
				<version>${spider-flow.version}</version>
			</dependency>
			<dependency>
				<groupId>org.spiderflow</groupId>
				<artifactId>spider-flow-mongodb</artifactId>
				<version>${spider-flow.version}</version>
			</dependency>
			<dependency>
				<groupId>org.spiderflow</groupId>
				<artifactId>spider-flow-redis</artifactId>
				<version>${spider-flow.version}</version>
			</dependency>
			<dependency>
				<groupId>org.spiderflow</groupId>
				<artifactId>spider-flow-ocr</artifactId>
				<version>${spider-flow.version}</version>
			</dependency>
			<dependency>
				<groupId>org.spiderflow</groupId>
				<artifactId>spider-flow-oss</artifactId>
				<version>${spider-flow.version}</version>
			</dependency>
			<dependency>
				<groupId>org.spiderflow</groupId>
				<artifactId>spider-flow-mailbox</artifactId>
				<version>${spider-flow.version}</version>
			</dependency>
		</dependencies>
	</dependencyManagement>
	<modules>
		<module>spider-flow-api</module>
		<module>spider-flow-core</module>
		<module>spider-flow-web</module>
	</modules>
</project>

================================================
FILE: spider-flow-api/pom.xml
================================================
<?xml version="1.0"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.spiderflow</groupId>
		<artifactId>spider-flow</artifactId>
		<version>0.5.0</version>
	</parent>
	<artifactId>spider-flow-api</artifactId>
	<name>spider-flow-api</name>
	<url>https://gitee.com/jmxd/spider-flow/tree/master/spider-flow-api</url>
	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	</properties>
</project>


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/ExpressionEngine.java
================================================
package org.spiderflow;

import java.util.Map;

/**
 * 表达式引擎
 */
public interface ExpressionEngine {

	/**
	 * 执行表达式
	 * @param expression	表达式
	 * @param variables	变量
	 * @return
	 */
	Object execute(String expression, Map<String, Object> variables);

}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/Grammerable.java
================================================
package org.spiderflow;

import java.util.List;
import org.spiderflow.model.Grammer;

public interface Grammerable {

	List<Grammer> grammers();
	
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/annotation/Comment.java
================================================
package org.spiderflow.annotation;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
 * 该注解用来标注自定义的方法注释,用来页面代码提示
 */
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD,ElementType.TYPE})
public @interface Comment {
	String value();
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/annotation/Example.java
================================================
package org.spiderflow.annotation;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
 * 该注解用来标注自定义的方法注释,用来页面代码案例
 */
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface Example {
	String value();
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/annotation/Return.java
================================================
package org.spiderflow.annotation;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
 * 该注解用来标注自定义的方法注释,用来页面提示返回值类型
 */
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface Return {
	Class<?>[] value();
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/common/CURDController.java
================================================
package org.spiderflow.common;

import org.spiderflow.model.JsonBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;

import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;

public abstract class CURDController<S extends ServiceImpl<M, T>,M extends BaseMapper<T>, T> {
	
	@Autowired
	private S service;
	
	@RequestMapping("/list")
	public IPage<T> list(@RequestParam(name = "page",defaultValue = "1")Integer page, @RequestParam(name = "limit",defaultValue = "1")Integer size){
		return service.page(new Page<T>(page, size), new QueryWrapper<T>().orderByDesc("create_date"));
	}
	
	@RequestMapping("get")
	public JsonBean<T> get(String id) {
		return new JsonBean<T>(service.getById(id));
	}
	
	@RequestMapping("delete")
	public JsonBean<Boolean> delete(String id){
		return new JsonBean<Boolean>(service.removeById(id));
	}
	
	@RequestMapping("save")
	public JsonBean<Boolean> save(T t){
		return new JsonBean<Boolean>(service.saveOrUpdate(t));
	}
	
}

================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/ChildPriorThreadSubmitStrategy.java
================================================
package org.spiderflow.concurrent;

import org.spiderflow.model.SpiderNode;

import java.util.Comparator;
import java.util.PriorityQueue;

public class ChildPriorThreadSubmitStrategy implements ThreadSubmitStrategy{

    private Object mutex = this;

    private Comparator<SpiderNode> comparator = (o1, o2) -> {
        if(o1.hasLeftNode(o2.getNodeId())){
            return -1;
        }
        return 1;
    };

    private PriorityQueue<SpiderFutureTask<?>> priorityQueue = new PriorityQueue<>((o1, o2) -> comparator.compare(o1.getNode(),o2.getNode()));

    @Override
    public Comparator<SpiderNode> comparator() {
        return comparator;
    }

    @Override
    public void add(SpiderFutureTask<?> task) {
        synchronized (mutex){
            priorityQueue.add(task);
        }
    }

    @Override
    public boolean isEmpty() {
        synchronized (mutex){
            return priorityQueue.isEmpty();
        }
    }

    @Override
    public SpiderFutureTask<?> get() {
        synchronized (mutex){
            return priorityQueue.poll();
        }
    }
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/LinkedThreadSubmitStrategy.java
================================================
package org.spiderflow.concurrent;

import org.spiderflow.model.SpiderNode;

import java.util.Comparator;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;

public class LinkedThreadSubmitStrategy implements ThreadSubmitStrategy{

    private List<SpiderFutureTask<?>> taskList = new CopyOnWriteArrayList<>();

    @Override
    public Comparator<SpiderNode> comparator() {
        return (o1, o2) -> -1;
    }

    @Override
    public void add(SpiderFutureTask<?> task) {
        taskList.add(task);
    }

    @Override
    public boolean isEmpty() {
        return taskList.isEmpty();
    }

    @Override
    public SpiderFutureTask<?> get() {
        return taskList.remove(0);
    }
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/ParentPriorThreadSubmitStrategy.java
================================================
package org.spiderflow.concurrent;

import org.spiderflow.model.SpiderNode;

import java.util.Comparator;
import java.util.PriorityQueue;

public class ParentPriorThreadSubmitStrategy implements ThreadSubmitStrategy {

    private Object mutex = this;

    private Comparator<SpiderNode> comparator = (o1, o2) -> {
        if (o1.hasLeftNode(o2.getNodeId())) {
            return 1;
        }
        return -1;
    };

    private PriorityQueue<SpiderFutureTask<?>> priorityQueue = new PriorityQueue<>((o1, o2) -> comparator.compare(o1.getNode(), o2.getNode()));

    @Override
    public Comparator<SpiderNode> comparator() {
        return comparator;
    }

    @Override
    public void add(SpiderFutureTask<?> task) {
        synchronized (mutex) {
            priorityQueue.add(task);
        }
    }

    @Override
    public boolean isEmpty() {
        synchronized (mutex) {
            return priorityQueue.isEmpty();
        }
    }

    @Override
    public SpiderFutureTask<?> get() {
        synchronized (mutex) {
            return priorityQueue.poll();
        }
    }

}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/RandomThreadSubmitStrategy.java
================================================
package org.spiderflow.concurrent;

import org.apache.commons.lang3.RandomUtils;
import org.spiderflow.model.SpiderNode;

import java.util.Comparator;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;

public class RandomThreadSubmitStrategy implements ThreadSubmitStrategy{

    private List<SpiderFutureTask<?>> taskList = new CopyOnWriteArrayList<>();

    @Override
    public Comparator<SpiderNode> comparator() {
        return (o1, o2) -> RandomUtils.nextInt(0,3) - 1;
    }

    @Override
    public void add(SpiderFutureTask<?> task) {
        taskList.add(task);
    }

    @Override
    public boolean isEmpty() {
        return taskList.isEmpty();
    }

    @Override
    public SpiderFutureTask<?> get() {
        return taskList.remove(RandomUtils.nextInt(0, taskList.size()));
    }
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/SpiderFlowThreadPoolExecutor.java
================================================
package org.spiderflow.concurrent;

import org.spiderflow.model.SpiderNode;

import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;

public class SpiderFlowThreadPoolExecutor {

	/**
	 * 最大线程数
	 */
	private int maxThreads;

	/**
	 * 真正线程池
	 */
	private ThreadPoolExecutor executor;

	/**
	 * 线程number计数器
	 */
	private final AtomicInteger poolNumber = new AtomicInteger(1);

	/**
	 * ThreadGroup
	 */
	private static final ThreadGroup SPIDER_FLOW_THREAD_GROUP = new ThreadGroup("spider-flow-group");

	/**
	 * 线程名称前缀
	 */
	private static final String THREAD_POOL_NAME_PREFIX = "spider-flow-";

	public SpiderFlowThreadPoolExecutor(int maxThreads) {
		super();
		this.maxThreads = maxThreads;
		//创建线程池实例
		this.executor = new ThreadPoolExecutor(maxThreads, maxThreads, 10, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>(), runnable -> {
			//重写线程名称
			return new Thread(SPIDER_FLOW_THREAD_GROUP, runnable, THREAD_POOL_NAME_PREFIX + poolNumber.getAndIncrement());
		});
	}

	public Future<?> submit(Runnable runnable){
		return this.executor.submit(runnable);
	}


	/**
	 * 创建子线程池
	 * @param threads	线程池大小
	 * @return
	 */
	public SubThreadPoolExecutor createSubThreadPoolExecutor(int threads,ThreadSubmitStrategy submitStrategy){
		return new SubThreadPoolExecutor(Math.min(maxThreads, threads),submitStrategy);
	}

	/**
	 * 子线程池
	 */
	public class SubThreadPoolExecutor{

		/**
		 * 线程池大小
		 */
		private int threads;

		/**
		 * 正在执行中的任务
		 */
		private Future<?>[] futures;

		/**
		 * 执行中的数量
		 */
		private AtomicInteger executing = new AtomicInteger(0);

		/**
		 * 是否运行中
		 */
		private volatile boolean running = true;

		/**
		 * 是否提交任务中
		 */
		private volatile boolean submitting = false;

		private ThreadSubmitStrategy submitStrategy;

		public SubThreadPoolExecutor(int threads,ThreadSubmitStrategy submitStrategy) {
			super();
			this.threads = threads;
			this.futures = new Future[threads];
			this.submitStrategy = submitStrategy;
		}
		
		/**
		 * 等待所有线程执行完毕
		 */
		public void awaitTermination(){
			while(executing.get() > 0){
				removeDoneFuture();
			}
			running = false;
			//当停止时,唤醒提交任务线程使其结束
			synchronized (submitStrategy){
				submitStrategy.notifyAll();
			}
		}
		
		private int index(){
			for (int i = 0; i < threads; i++) {
				if(futures[i] == null || futures[i].isDone()){
					return i;
				}
			}
			return -1;
		}

		/**
		 * 清除已完成的任务
		 */
		private void removeDoneFuture(){
			for (int i = 0; i < threads; i++) {
				try {
					if(futures[i] != null && futures[i].get(10,TimeUnit.MILLISECONDS) == null){
						futures[i] = null;
					}
				} catch (Throwable t) {
					//忽略异常
				} 
			}
		}

		/**
		 * 等待有空闲线程
		 */
		private void await(){
			while(index() == -1){
				removeDoneFuture();
			}
		}

		/**
		 * 异步提交任务
		 */
		public <T> Future<T> submitAsync(Runnable runnable, T value, SpiderNode node){
			SpiderFutureTask<T> future = new SpiderFutureTask<>(()-> {
				try {
					//执行任务
					runnable.run();
				} finally {
					//正在执行的线程数-1
					executing.decrementAndGet();
				}
			}, value,node,this);

			submitStrategy.add(future);
			//如果是第一次调用submitSync方法,则启动提交任务线程
			if(!submitting){
				submitting = true;
				CompletableFuture.runAsync(this::submit);
			}
			synchronized (submitStrategy){
				//通知继续从集合中取任务提交到线程池中
				submitStrategy.notifyAll();

			}
			return future;
		}

		private void submit(){
			while(running){
				try {
					synchronized (submitStrategy){
						//如果集合是空的,则等待提交
						if(submitStrategy.isEmpty()){
							submitStrategy.wait();	//等待唤醒
						}
					}
					//当该线程被唤醒时,把集合中所有任务都提交到线程池中
					while(!submitStrategy.isEmpty()){
						//从提交策略中获取任务提交到线程池中
						SpiderFutureTask<?> futureTask = submitStrategy.get();
						//如果没有空闲线程且在线程池中提交,则直接运行
						if(index() == -1 && Thread.currentThread().getThreadGroup() == SPIDER_FLOW_THREAD_GROUP){
							futureTask.run();
						}else{
							//等待有空闲线程时在提交
							await();
							//提交任务至线程池中
							futures[index()] = executor.submit(futureTask);
						}
					}
				} catch (InterruptedException ignored) {
				}
			}
		}
	}
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/SpiderFutureTask.java
================================================
package org.spiderflow.concurrent;

import java.util.concurrent.FutureTask;
import org.spiderflow.concurrent.SpiderFlowThreadPoolExecutor.SubThreadPoolExecutor;
import org.spiderflow.model.SpiderNode;

public class SpiderFutureTask<V> extends FutureTask {

    private SubThreadPoolExecutor executor;

    private SpiderNode node;

    public SpiderFutureTask(Runnable runnable, V result, SpiderNode node,SubThreadPoolExecutor executor) {
        super(runnable,result);
        this.executor = executor;
        this.node = node;
    }

    public SubThreadPoolExecutor getExecutor() {
        return executor;
    }

    public SpiderNode getNode() {
        return node;
    }
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/ThreadSubmitStrategy.java
================================================
package org.spiderflow.concurrent;

import org.spiderflow.model.SpiderNode;

import java.util.Comparator;

public interface ThreadSubmitStrategy {

    Comparator<SpiderNode> comparator();

    void add(SpiderFutureTask<?> task);

    boolean isEmpty();

    SpiderFutureTask<?> get();
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/context/CookieContext.java
================================================
package org.spiderflow.context;

import java.util.HashMap;

/**
 * Cookie上下文
 */
public class CookieContext extends HashMap<String, String> {
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/context/SpiderContext.java
================================================
package org.spiderflow.context;

import org.spiderflow.concurrent.SpiderFlowThreadPoolExecutor.SubThreadPoolExecutor;
import org.spiderflow.model.SpiderNode;
import org.spiderflow.model.SpiderOutput;

import java.util.*;
import java.util.concurrent.Future;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.locks.ReentrantLock;

/**
 * 爬虫上下文
 * @author jmxd
 *
 */
public class SpiderContext extends HashMap<String, Object>{
	
	private String id = UUID.randomUUID().toString().replace("-", "");

	/**
	 * 流程ID
	 */
	private String flowId;
	
	private static final long serialVersionUID = 8379177178417619790L;

	/**
	 * 流程执行线程
	 */
	private SubThreadPoolExecutor threadPool;

	/**
	 * 根节点
	 */
	private SpiderNode rootNode;

	/**
	 * 爬虫是否运行中
	 */
	private volatile boolean running = true;

	/**
	 * Future队列
	 */
	private LinkedBlockingQueue<Future<?>> futureQueue = new LinkedBlockingQueue<>();

	/**
	 * Cookie上下文
	 */
	private CookieContext cookieContext = new CookieContext();

	public List<SpiderOutput> getOutputs() {
		return Collections.emptyList();
	}

	public <T> T get(String key){
		return (T) super.get(key);
	}

	public <T> T get(String key,T defaultValue){
		T value = this.get(key);
		return value == null ? defaultValue : value;
	}

	public String getFlowId() {
		return flowId;
	}

	public void setFlowId(String flowId) {
		this.flowId = flowId;
	}

	public LinkedBlockingQueue<Future<?>> getFutureQueue() {
		return futureQueue;
	}

	public boolean isRunning() {
		return running;
	}

	public void setRunning(boolean running) {
		this.running = running;
	}

	public void addOutput(SpiderOutput output){

	}

	public SubThreadPoolExecutor getThreadPool() {
		return threadPool;
	}

	public void setThreadPool(SubThreadPoolExecutor threadPool) {
		this.threadPool = threadPool;
	}

	public SpiderNode getRootNode() {
		return rootNode;
	}

	public void setRootNode(SpiderNode rootNode) {
		this.rootNode = rootNode;
	}
	
	public String getId() {
		return id;
	}
	
	public CookieContext getCookieContext() {
		return cookieContext;
	}

	public void pause(String nodeId,String event,String key,Object value){}

	public void resume(){}

	public void stop(){}

}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/context/SpiderContextHolder.java
================================================
package org.spiderflow.context;

import com.alibaba.ttl.TransmittableThreadLocal;

public class SpiderContextHolder {

	private static final ThreadLocal<SpiderContext> THREAD_LOCAL = new TransmittableThreadLocal<>();

	public static SpiderContext get() {
		return THREAD_LOCAL.get();
	}

	public static void set(SpiderContext context) {
		THREAD_LOCAL.set(context);
	}

	public static void remove() {
		THREAD_LOCAL.remove();
	}

}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/enums/FlowNoticeType.java
================================================
package org.spiderflow.enums;

/**
 * 流程通知类型
 * 
 * @author BillDowney
 * @date 2020年4月4日 上午1:32:53
 */
public enum FlowNoticeType {
	/**
	 * 流程开始通知
	 */
	startNotice,
	/**
	 * 流程异常通知
	 */
	exceptionNotice,
	/**
	 * 流程结束通知
	 */
	endNotice

}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/enums/FlowNoticeWay.java
================================================
package org.spiderflow.enums;

import java.util.LinkedHashMap;
import java.util.Map;

/**
 * 流程通知方式
 * 
 * @author BillDowney
 * @date 2020年4月3日 下午3:26:18
 */
public enum FlowNoticeWay {

	email("邮件通知");

	private FlowNoticeWay(String title) {
		this.title = title;
	}

	private String title;

	@Override
	public String toString() {
		return this.name() + ":" + this.title;
	}

	public static Map<String, String> getMap() {
		Map<String, String> map = new LinkedHashMap<String, String>();
		for (FlowNoticeWay type : FlowNoticeWay.values()) {
			map.put(type.name(), type.toString());
		}
		return map;
	}

}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/executor/FunctionExecutor.java
================================================
package org.spiderflow.executor;

public interface FunctionExecutor {
	
	String getFunctionPrefix();

}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/executor/FunctionExtension.java
================================================
package org.spiderflow.executor;

public interface FunctionExtension {
	
	Class<?> support();
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/executor/PluginConfig.java
================================================
package org.spiderflow.executor;

import org.spiderflow.model.Plugin;

public interface PluginConfig {
	
	Plugin plugin();
	
}

================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/executor/ShapeExecutor.java
================================================
package org.spiderflow.executor;

import java.util.Map;

import org.spiderflow.context.SpiderContext;
import org.spiderflow.model.Shape;
import org.spiderflow.model.SpiderNode;
/**
 * 执行器接口
 * @author jmxd
 *
 */
public interface ShapeExecutor {
	
	String LOOP_VARIABLE_NAME = "loopVariableName";
	
	String LOOP_COUNT = "loopCount";
	
	String THREAD_COUNT = "threadCount";

	default Shape shape(){
		return null;
	}
	
	/**
	 * 节点形状
	 * @return 节点形状名称
	 */
	String supportShape();
	
	/**
	 * 执行器具体的功能实现
	 * @param node 当前要执行的爬虫节点
	 * @param context 爬虫上下文
	 * @param variables 节点流程的全部变量的集合
	 */
	void execute(SpiderNode node, SpiderContext context, Map<String, Object> variables);
	
	default boolean allowExecuteNext(SpiderNode node, SpiderContext context, Map<String, Object> variables){
		return true;
	}
	
	default boolean isThread(){
		return true;
	}
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/expression/DynamicMethod.java
================================================
package org.spiderflow.expression;

import java.util.List;

public interface DynamicMethod {
	
	Object execute(String methodName, List<Object> parameters);

}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/io/Line.java
================================================
package org.spiderflow.io;

public class Line {

	private long from;

	private String text;

	private long to;

	public Line(long from, String text, long to) {
		this.from = from;
		this.text = text;
		this.to = to;
	}

	public long getFrom() {
		return from;
	}

	public void setFrom(long from) {
		this.from = from;
	}

	public String getText() {
		return text;
	}

	public void setText(String text) {
		this.text = text;
	}

	public long getTo() {
		return to;
	}

	public void setTo(long to) {
		this.to = to;
	}

	@Override
	public String toString() {
		return "Line{" +
				"from=" + from +
				", text='" + text + '\'' +
				", to=" + to +
				'}';
	}
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/io/RandomAccessFileReader.java
================================================
package org.spiderflow.io;

import java.io.Closeable;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.util.ArrayList;
import java.util.List;
import java.util.regex.Pattern;

public class RandomAccessFileReader implements Closeable {

	private RandomAccessFile raf;

	/**
	 * 从index位置开始读取
	 */
	private long index;

	/**
	 * 读取顺序,默认倒叙
	 */
	private boolean reversed;

	/**
	 * 缓冲区大小
	 */
	private int bufSize;

	public RandomAccessFileReader(RandomAccessFile raf, long index, boolean reversed) throws IOException {
		this(raf, index, 1024, reversed);
	}

	public RandomAccessFileReader(RandomAccessFile raf, long index, int bufSize, boolean reversed) throws IOException {
		if (raf == null) {
			throw new NullPointerException("file is null");
		}
		this.raf = raf;
		this.reversed = reversed;
		this.bufSize = bufSize;
		this.index = index;
		this.init();
	}

	private void init() throws IOException {
		if (reversed) {
			this.index = this.index == -1 ? this.raf.length() : Math.min(this.index, this.raf.length());
		} else {
			this.index = Math.min(Math.max(this.index, 0), this.raf.length());
		}
		if (this.index > 0) {
			this.raf.seek(this.index);
		}
	}

	/**
	 * 读取n行
	 *
	 * @param n        要读取的行数
	 * @param keywords 搜索的关键词
	 * @param matchcase 是否区分大小写
	 * @param regx 是否是正则搜索
	 * @return 返回Line对象,包含行的起始位置与终止位置
	 */
	public List<Line> readLine(int n, String keywords, boolean matchcase, boolean regx) throws IOException {
		List<Line> lines = new ArrayList<>(n);
		long lastCRLFIndex = reversed ? this.index : (this.index > 0 ? this.index + 1 : -1);
		boolean find = keywords == null || keywords.isEmpty();
		Pattern pattern = regx && !find ? Pattern.compile(keywords) : null;
		while (n > 0) {
			byte[] buf = reversed ? new byte[(int) Math.min(this.bufSize, this.index)] : new byte[this.bufSize];
			if (this.reversed) {
				if (this.index == 0) {
					break;
				}
				this.raf.seek(this.index -= buf.length);
			}
			int len = this.raf.read(buf, 0, buf.length);
			if (len == -1) {    //已读完
				break;
			}
			for (int i = 0; i < len && n > 0; i++) {
				int readIndex = reversed ? len - i - 1 : i;
				if (isCRLF(buf[readIndex])) {    //如果读取到\r或\n
					if (Math.abs(this.index + readIndex - lastCRLFIndex) > 1) { //两行之间的间距,当=1时则代表有\r\n,\n\r,\r\r,\n\n四种情况之一
						long fromIndex = reversed ? this.index + readIndex : lastCRLFIndex;    //计算起止位置
						long endIndex = reversed ? lastCRLFIndex : this.index + readIndex;    //计算终止位置
						Line line = readLine(fromIndex + 1, endIndex);    //取出文本
						if (find || (find = (pattern == null ? find(line.getText(), keywords, matchcase) : find(line.getText(), pattern)))) {    //定位查找,使被查找的行始终在第一行
							if (reversed) {
								lines.add(0, line);    //反向查找时,插入到List头部
							} else {
								lines.add(line);
							}
							n--;
						}
					}
					lastCRLFIndex = this.index + readIndex;    //记录上次读取到的\r或\n位置
				}
			}
			if (!reversed) {
				this.index += buf.length;
			}
		}
		if (reversed && n > 0 && lastCRLFIndex > 1 && (find || lines.size() > 0)) {
			lines.add(0, readLine(0, lastCRLFIndex));
		}
		return lines;
	}

	private boolean find(String text, String keywords, boolean matchcase) {
		return matchcase ? text.contains(keywords) : text.toLowerCase().contains(keywords.toLowerCase());
	}

	private boolean find(String text, Pattern pattern) {
		return pattern.matcher(text).find();
	}

	/**
	 * 从指定位置读取一行
	 *
	 * @param fromIndex 开始位置
	 * @param endIndex  结束位置
	 * @return 返回Line对象
	 * @throws IOException
	 */
	private Line readLine(long fromIndex, long endIndex) throws IOException {
		long index = this.raf.getFilePointer();
		this.raf.seek(fromIndex);
		byte[] buf = new byte[(int) (endIndex - fromIndex)];
		this.raf.read(buf, 0, buf.length);
		Line line = new Line(fromIndex, new String(buf), endIndex);
		this.raf.seek(index);
		return line;
	}

	private boolean isCRLF(byte b) {
		return b == 13 || b == 10;
	}

	@Override
	public void close() throws IOException {
		if (this.raf != null) {
			this.raf.close();
		}
	}
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/io/SpiderResponse.java
================================================
package org.spiderflow.io;

import java.io.InputStream;
import java.util.Map;

import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;

import com.alibaba.fastjson.JSON;

public interface SpiderResponse {

	@Comment("获取返回状态码")
	@Example("${resp.statusCode}")
	int getStatusCode();

	@Comment("获取网页标题")
	@Example("${resp.title}")
	String getTitle();

	@Comment("获取网页html")
	@Example("${resp.html}")
	String getHtml();

	@Comment("获取json")
	@Example("${resp.json}")
	default Object getJson(){
		return JSON.parse(getHtml());
	}
	@Comment("获取cookies")
	@Example("${resp.cookies}")
	Map<String,String> getCookies();

	@Comment("获取headers")
	@Example("${resp.headers}")
	Map<String,String> getHeaders();

	@Comment("获取byte[]")
	@Example("${resp.bytes}")
	byte[] getBytes();

	@Comment("获取ContentType")
	@Example("${resp.contentType}")
	String getContentType();

	@Comment("获取当前url")
	@Example("${resp.url}")
	String getUrl();

	@Example("${resp.setCharset('UTF-8')}")
	default void setCharset(String charset){

	}
	@Example("${resp.stream}")
	default InputStream getStream(){
		return null;
	}
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/listener/SpiderListener.java
================================================
package org.spiderflow.listener;

import org.spiderflow.context.SpiderContext;

public interface SpiderListener {

	/**
	 * 开始执行之前
	 */
	void beforeStart(SpiderContext context);
	
	/**
	 * 执行完毕之后
	 */
	void afterEnd(SpiderContext context);
	
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/model/Grammer.java
================================================
package org.spiderflow.model;

import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;

import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.annotation.Return;

public class Grammer {
	
	private String owner;

	private String method;
	
	private String comment;
	
	private String example;
	
	private String function;
	
	private List<String> returns;
	
	public String getOwner() {
		return owner;
	}

	public void setOwner(String owner) {
		this.owner = owner;
	}

	public String getMethod() {
		return method;
	}

	public void setMethod(String method) {
		this.method = method;
	}
	
	public String getFunction() {
		return function;
	}

	public void setFunction(String function) {
		this.function = function;
	}

	public String getComment() {
		return comment;
	}

	public void setComment(String comment) {
		this.comment = comment;
	}

	public String getExample() {
		return example;
	}

	public void setExample(String example) {
		this.example = example;
	}

	public List<String> getReturns() {
		return returns;
	}
	
	public void setReturns(List<String> returns) {
		this.returns = returns;
	}
	
	public static List<Grammer> findGrammers(Class<?> clazz,String function,String owner,boolean mustStatic){
		Method[] methods = clazz.getDeclaredMethods();
		List<Grammer> grammers = new ArrayList<>();
		for (Method method : methods) {
			if(Modifier.isPublic(method.getModifiers()) && (Modifier.isStatic(method.getModifiers())||!mustStatic)){
				Grammer grammer = new Grammer();
				grammer.setMethod(method.getName());
				Comment comment = method.getAnnotation(Comment.class);
				if(comment != null){
					grammer.setComment(comment.value());
				}
				Example example = method.getAnnotation(Example.class);
				if(example != null){
					grammer.setExample(example.value());
				}
				Return returns = method.getAnnotation(Return.class);
				if(returns != null){
					Class<?>[] clazzs = returns.value();
					List<String> returnTypes = new ArrayList<>();
					for (int i = 0; i < clazzs.length; i++) {
						returnTypes.add(clazzs[i].getSimpleName());
					}
					grammer.setReturns(returnTypes);
				}else{
					grammer.setReturns(Collections.singletonList(method.getReturnType().getSimpleName()));
				}
				grammer.setFunction(function);
				grammer.setOwner(owner);
				grammers.add(grammer);
			}
		}
		return grammers;
	}
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/model/JsonBean.java
================================================
package org.spiderflow.model;

public class JsonBean<T> {

	private Integer code = 1;
	
	private String message = "执行成功";
	
	private T data;

	public JsonBean(Integer code, String message, T data) {
		this.code = code;
		this.message = message;
		this.data = data;
	}

	public JsonBean(Integer code, String message) {
		this.code = code;
		this.message = message;
	}

	public JsonBean(T data) {
		this.data = data;
	}

	public Integer getCode() {
		return code;
	}

	public void setCode(Integer code) {
		this.code = code;
	}

	public String getMessage() {
		return message;
	}

	public void setMessage(String message) {
		this.message = message;
	}

	public T getData() {
		return data;
	}

	public void setData(T data) {
		this.data = data;
	}
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/model/Plugin.java
================================================
package org.spiderflow.model;

public class Plugin {

	private String name;
	
	private String url;
	
	public Plugin(String name, String url) {
		this.name = name;
		this.url = url;
	}

	public String getName() {
		return name;
	}

	public void setName(String name) {
		this.name = name;
	}

	public String getUrl() {
		return url;
	}

	public void setUrl(String url) {
		this.url = url;
	}
	
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/model/Shape.java
================================================
package org.spiderflow.model;

public class Shape {
	
	private String name;
	
	private String label;
	
	private String title;
	
	private String image;

	private String desc;

	public String getName() {
		return name;
	}

	public void setName(String name) {
		this.name = name;
	}

	public String getLabel() {
		return label;
	}

	public void setLabel(String label) {
		this.label = label;
	}

	public String getTitle() {
		return title;
	}

	public void setTitle(String title) {
		this.title = title;
	}

	public String getImage() {
		return image;
	}

	public void setImage(String image) {
		this.image = image;
	}

	public String getDesc() {
		return desc;
	}

	public void setDesc(String desc) {
		this.desc = desc;
	}
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/model/SpiderLog.java
================================================
package org.spiderflow.model;

import java.util.ArrayList;
import java.util.List;

import org.apache.commons.lang3.exception.ExceptionUtils;

public class SpiderLog {
	
	private String level;
	
	private String message;
	
	private List<Object> variables;

	public SpiderLog(String level,String message, List<Object> variables) {
		if(variables != null && variables.size() > 0){
			List<Object> nVariables = new ArrayList<>(variables.size());
			for (Object object : variables) {
				if(object instanceof Throwable){
					nVariables.add(ExceptionUtils.getStackTrace((Throwable) object));
				}else{
					nVariables.add(object);
				}
			}
			this.variables = nVariables;
		}
		this.level = level;
		this.message = message;
	}

	public String getLevel() {
		return level;
	}

	public void setLevel(String level) {
		this.level = level;
	}

	public String getMessage() {
		return message;
	}

	public void setMessage(String message) {
		this.message = message;
	}

	public List<Object> getVariables() {
		return variables;
	}

	public void setVariables(List<Object> variables) {
		this.variables = variables;
	}
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/model/SpiderNode.java
================================================
package org.spiderflow.model;

import java.util.*;
import java.util.concurrent.atomic.AtomicInteger;

import org.apache.commons.lang3.StringUtils;
import org.apache.commons.text.StringEscapeUtils;

import com.alibaba.fastjson.JSONArray;


/**
 * 爬虫节点
 * @author jmxd
 *
 */
public class SpiderNode {
	/**
	 * 节点的Json属性
	 */
	private Map<String,Object> jsonProperty = new HashMap<>();
	/**
	 * 节点列表中的下一个节点
	 */
	private List<SpiderNode> nextNodes = new ArrayList<>();

	/**
	 * 节点列表中的上一个节点
	 */
	private List<SpiderNode> prevNodes = new ArrayList<>();

	/**
	 * 父级节点ID
	 */
	private Set<String> parentNodes;

	/**
	 * 节点流转条件
	 */
	private Map<String,String> condition = new HashMap<>();

	/**
	 * 异常流转
	 */
	private Map<String,String> exception = new HashMap<>();

	/**
	 * 传递变量
	 */
	private Map<String,String> transmitVariable = new HashMap<>();
	/**
	 * 节点名称
	 */
	private String nodeName;
	/**
	 * 节点ID
	 */
	private String nodeId;

	/**
	 * 计数器,用来计算当前节点执行中的个数
	 */
	private AtomicInteger counter = new AtomicInteger();

	public String getNodeId() {
		return nodeId;
	}

	public void setNodeId(String nodeId) {
		this.nodeId = nodeId;
	}

	public String getNodeName() {
		return nodeName;
	}

	public void setNodeName(String nodeName) {
		this.nodeName = nodeName;
	}

	public String getStringJsonValue(String key){
		String value = (String) this.jsonProperty.get(key);
		if(value != null){
			value = StringEscapeUtils.unescapeHtml4(value);
		}
		return value;
	}

	public String getStringJsonValue(String key,String defaultValue){
		String value = getStringJsonValue(key);
		return StringUtils.isNotBlank(value) ? value : defaultValue;
	}
	
	public List<Map<String,String>> getListJsonValue(String ... keys){
		List<JSONArray> arrays = new ArrayList<>();
		int size = -1;
		List<Map<String,String>> result = new ArrayList<>();
		for (int i = 0; i < keys.length; i++) {
			JSONArray jsonArray = (JSONArray) this.jsonProperty.get(keys[i]);
			if(jsonArray != null){
				if(size == -1){
					size = jsonArray.size();
				}else if(size != jsonArray.size()){
					throw new ArrayIndexOutOfBoundsException();
				}
				arrays.add(jsonArray);
			}
		}
		for (int i = 0;i < size;i++) {
			Map<String,String> item = new HashMap<>();
			for (int j = 0; j < keys.length; j++) {
				String val = arrays.get(j).getString(i);
				if(val != null){
					val = StringEscapeUtils.unescapeHtml4(val);
				}
				item.put(keys[j],val);
			}
			result.add(item);
		}
		return result;
	}
	public void setJsonProperty(Map<String, Object> jsonProperty) {
		this.jsonProperty = jsonProperty;
	}

	public void addNextNode(SpiderNode nextNode){
		nextNode.prevNodes.add(this);
		this.nextNodes.add(nextNode);
	}

	public String getExceptionFlow(String fromNodeId) {
		return exception.get(fromNodeId);
	}

	public boolean isTransmitVariable(String fromNodeId) {
		String value = transmitVariable.get(fromNodeId);
		return value == null || "1".equalsIgnoreCase(value);
	}

	public void setTransmitVariable(String fromNodeId,String value){
		this.transmitVariable.put(fromNodeId,value);
	}

	public void setExceptionFlow(String fromNodeId,String value){
		this.exception.put(fromNodeId,value);
	}

	public List<SpiderNode> getNextNodes() {
		return nextNodes;
	}

	public String getCondition(String fromNodeId) {
		return condition.get(fromNodeId);
	}

	public void setCondition(String fromNodeId,String condition) {
		this.condition.put(fromNodeId, condition);
	}

	public void increment(){
		counter.incrementAndGet();
	}

	public void decrement(){
		counter.decrementAndGet();
	}

	public boolean hasLeftNode(String nodeId){
		if(parentNodes == null){
			Set<String> parents = new HashSet<>();
			generateParents(parents);
			this.parentNodes = parents;
		}
		return this.parentNodes.contains(nodeId);
	}

	private void generateParents(Set<String> parents){
		for (SpiderNode prevNode : prevNodes) {
			if(parents.add(prevNode.nodeId)){
				prevNode.generateParents(parents);
			}
		}
	}

	public boolean isDone(){
		return isDone(new HashSet<>());
	}
	public boolean isDone(Set<String> visited){
		if(this.counter.get() == 0){
			for (SpiderNode prevNode : prevNodes) {
				if(visited.add(nodeId)&&!prevNode.isDone(visited)){
					return false;
				}
			}
			return true;
		}
		return false;
	}

	@Override
	public String toString() {
		return "SpiderNode [jsonProperty=" + jsonProperty + ", nextNodes=" + nextNodes + ", condition=" + condition
				+ ", nodeName=" + nodeName + ", nodeId=" + nodeId + "]";
	}
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/model/SpiderOutput.java
================================================
package org.spiderflow.model;

import java.util.ArrayList;
import java.util.List;

public class SpiderOutput {
	
	/**
	 * 节点名称
	 */
	private String nodeName;
	
	/**
	 * 节点Id
	 */
	private String nodeId;
	
	/**
	 * 输出项的名
	 */
	private List<String> outputNames = new ArrayList<>();
	
	/**
	 * 输出项的值
	 */
	private List<Object> values = new ArrayList<>();

	public String getNodeName() {
		return nodeName;
	}

	public void setNodeName(String nodeName) {
		this.nodeName = nodeName;
	}

	public List<String> getOutputNames() {
		return outputNames;
	}

	public void setOutputNames(List<String> outputNames) {
		this.outputNames = outputNames;
	}

	public List<Object> getValues() {
		return values;
	}

	public void setValues(List<Object> values) {
		this.values = values;
	}
	
	public void addOutput(String name,Object value){
		this.outputNames.add(name);
		this.values.add(value);
	}

	public String getNodeId() {
		return nodeId;
	}

	public void setNodeId(String nodeId) {
		this.nodeId = nodeId;
	}

	@Override
	public String toString() {
		return "SpiderOutput [nodeName=" + nodeName + ", nodeId=" + nodeId + ", outputNames=" + outputNames
				+ ", values=" + values + "]";
	}
}


================================================
FILE: spider-flow-api/src/main/java/org/spiderflow/utils/Maps.java
================================================
package org.spiderflow.utils;

import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class Maps {

	public static <K,V> Map<K,V> add(Map<K,V> srcMap,K k,V v){
		HashMap<K, V> destMap = new HashMap<>(srcMap);
		destMap.put(k, v);
		return destMap;
	}
	
	public static <K,V> Map<K,V> newMap(K key,V value){
		HashMap<K, V> map = new HashMap<>();
		map.put(key, value);
		return map;
	}
	
	public static <K,V> Map<K,V> add(Map<K,V> srcMap,List<K> ks,List<V> vs){
		HashMap<K, V> destMap = new HashMap<>(srcMap);
		if(ks != null && vs != null && ks.size() == vs.size()){
			int size = ks.size();
			for (int i = 0; i < size; i++) {
				destMap.put(ks.get(0), vs.get(0));
			}
		}
		return destMap;
	}
}


================================================
FILE: spider-flow-core/pom.xml
================================================
<?xml version="1.0"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.spiderflow</groupId>
		<artifactId>spider-flow</artifactId>
		<version>0.5.0</version>
	</parent>
	<artifactId>spider-flow-core</artifactId>
	<name>spider-flow-core</name>
	<url>https://gitee.com/jmxd/spider-flow/tree/master/spider-flow-core</url>
	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	</properties>
	<dependencies>
		<dependency>
			<groupId>org.spiderflow</groupId>
			<artifactId>spider-flow-api</artifactId>
		</dependency>
	</dependencies>
</project>


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/Spider.java
================================================
package org.spiderflow.core;

import com.alibaba.ttl.TtlRunnable;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.math.NumberUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.spiderflow.concurrent.*;
import org.spiderflow.concurrent.SpiderFlowThreadPoolExecutor.SubThreadPoolExecutor;
import org.spiderflow.context.SpiderContext;
import org.spiderflow.context.SpiderContextHolder;
import org.spiderflow.core.executor.shape.LoopExecutor;
import org.spiderflow.core.model.SpiderFlow;
import org.spiderflow.core.service.FlowNoticeService;
import org.spiderflow.core.utils.ExecutorsUtils;
import org.spiderflow.core.utils.ExpressionUtils;
import org.spiderflow.core.utils.SpiderFlowUtils;
import org.spiderflow.enums.FlowNoticeType;
import org.spiderflow.executor.ShapeExecutor;
import org.spiderflow.listener.SpiderListener;
import org.spiderflow.model.SpiderNode;
import org.spiderflow.model.SpiderOutput;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;
import java.lang.reflect.Array;
import java.util.*;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Future;
import java.util.concurrent.FutureTask;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.atomic.AtomicInteger;

/**
 * 爬虫的核心类
 *
 * @author jmxd
 */
@Component
public class Spider {

	@Autowired(required = false)
	private List<SpiderListener> listeners;

	@Value("${spider.thread.max:64}")
	private Integer totalThreads;

	@Value("${spider.thread.default:8}")
	private Integer defaultThreads;

	@Value("${spider.detect.dead-cycle:5000}")
	private Integer deadCycle;
	
	@Autowired
	private FlowNoticeService flowNoticeService;

	public static SpiderFlowThreadPoolExecutor executorInstance;

	private static final String ATOMIC_DEAD_CYCLE = "__atomic_dead_cycle";

	private static Logger logger = LoggerFactory.getLogger(Spider.class);

	@PostConstruct
	private void init() {
		executorInstance = new SpiderFlowThreadPoolExecutor(totalThreads);
	}

	public List<SpiderOutput> run(SpiderFlow spiderFlow, SpiderContext context, Map<String, Object> variables) {
		if (variables == null) {
			variables = new HashMap<>();
		}
		SpiderNode root = SpiderFlowUtils.loadXMLFromString(spiderFlow.getXml());
		// 流程开始通知
		flowNoticeService.sendFlowNotice(spiderFlow, FlowNoticeType.startNotice);
		executeRoot(root, context, variables);
		// 流程结束通知
		flowNoticeService.sendFlowNotice(spiderFlow, FlowNoticeType.endNotice);
		return context.getOutputs();
	}

	public List<SpiderOutput> run(SpiderFlow spiderFlow, SpiderContext context) {
		return run(spiderFlow, context, new HashMap<>());
	}

	public void runWithTest(SpiderNode root, SpiderContext context) {
		//将上下文存到ThreadLocal里,以便后续使用
		SpiderContextHolder.set(context);
		//死循环检测的计数器(死循环检测只在测试时有效)
		AtomicInteger executeCount = new AtomicInteger(0);
		//存入到上下文中,以供后续检测
		context.put(ATOMIC_DEAD_CYCLE, executeCount);
		//执行根节点
		executeRoot(root, context, new HashMap<>());
		//当爬虫任务执行完毕时,判断是否超过预期
		if (executeCount.get() > deadCycle) {
			logger.error("检测到可能出现死循环,测试终止");
		} else {
			logger.info("测试完毕!");
		}
		//将上下文从ThreadLocal移除,防止内存泄漏
		SpiderContextHolder.remove();
	}

	/**
	 * 执行根节点
	 */
	private void executeRoot(SpiderNode root, SpiderContext context, Map<String, Object> variables) {
		//获取当前流程执行线程数
		int nThreads = NumberUtils.toInt(root.getStringJsonValue(ShapeExecutor.THREAD_COUNT), defaultThreads);
		String strategy = root.getStringJsonValue("submit-strategy");
		ThreadSubmitStrategy submitStrategy;
		//选择提交策略,这里一定要使用new,不能与其他实例共享
		if("linked".equalsIgnoreCase(strategy)){
			submitStrategy = new LinkedThreadSubmitStrategy();
		}else if("child".equalsIgnoreCase(strategy)){
			submitStrategy = new ChildPriorThreadSubmitStrategy();
		}else if("parent".equalsIgnoreCase(strategy)){
			submitStrategy = new ParentPriorThreadSubmitStrategy();
		}else{
			submitStrategy = new RandomThreadSubmitStrategy();
		}
		//创建子线程池,采用一父多子的线程池,子线程数不能超过总线程数(超过时进入队列等待),+1是因为会占用一个线程用来调度执行下一级
		SubThreadPoolExecutor pool = executorInstance.createSubThreadPoolExecutor(Math.max(nThreads,1) + 1,submitStrategy);
		context.setRootNode(root);
		context.setThreadPool(pool);
		//触发监听器
		if (listeners != null) {
			listeners.forEach(listener -> listener.beforeStart(context));
		}
		Comparator<SpiderNode> comparator = submitStrategy.comparator();
		//启动一个线程开始执行任务,并监听其结束并执行下一级
		Future<?> f = pool.submitAsync(TtlRunnable.get(() -> {
			try {
				//执行具体节点
				Spider.this.executeNode(null, root, context, variables);
				Queue<Future<?>> queue = context.getFutureQueue();
				//循环从队列中获取Future,直到队列为空结束,当任务完成时,则执行下一级
				while (!queue.isEmpty()) {
					try {
						//TODO 这里应该是取出最先执行完毕的任务
						Optional<Future<?>> max = queue.stream().filter(Future::isDone).max((o1, o2) -> {
							try {
								return comparator.compare(((SpiderTask) o1.get()).node, ((SpiderTask) o2.get()).node);
							} catch (InterruptedException | ExecutionException e) {
							}
							return 0;

						});
						if (max.isPresent()) {	//判断任务是否完成
							queue.remove(max.get());
							if (context.isRunning()) {	//检测是否运行中(当在页面中点击"停止"时,此值为false,其余为true)
								SpiderTask task = (SpiderTask) max.get().get();
								task.node.decrement();	//任务执行完毕,计数器减一(该计数器是给Join节点使用)
								if (task.executor.allowExecuteNext(task.node, context, task.variables)) {	//判断是否允许执行下一级
									logger.debug("执行节点[{}:{}]完毕", task.node.getNodeName(), task.node.getNodeId());
									//执行下一级
									Spider.this.executeNextNodes(task.node, context, task.variables);
								} else {
									logger.debug("执行节点[{}:{}]完毕,忽略执行下一节点", task.node.getNodeName(), task.node.getNodeId());
								}
							}
						}
						//睡眠1ms,让出cpu
						Thread.sleep(1);
					} catch (InterruptedException ignored) {
					} catch (Throwable t){
						logger.error("程序发生异常",t);
					}
				}
				//等待线程池结束
				pool.awaitTermination();
			} finally {
				//触发监听器
				if (listeners != null) {
					listeners.forEach(listener -> listener.afterEnd(context));
				}
			}
		}), null, root);
		try {
			f.get();	//阻塞等待所有任务执行完毕
		} catch (InterruptedException | ExecutionException ignored) {}
	}

	/**
	 * 执行下一级节点
	 */
	private void executeNextNodes(SpiderNode node, SpiderContext context, Map<String, Object> variables) {
		List<SpiderNode> nextNodes = node.getNextNodes();
		if (nextNodes != null) {
			for (SpiderNode nextNode : nextNodes) {
				executeNode(node, nextNode, context, variables);
			}
		}
	}

	/**
	 * 执行节点
	 */
	public void executeNode(SpiderNode fromNode, SpiderNode node, SpiderContext context, Map<String, Object> variables) {
		String shape = node.getStringJsonValue("shape");
		if (StringUtils.isBlank(shape)) {
			executeNextNodes(node, context, variables);
			return;
		}
		//判断箭头上的条件,如果不成立则不执行
		if (!executeCondition(fromNode, node, variables, context)) {
			return;
		}
		logger.debug("执行节点[{}:{}]", node.getNodeName(), node.getNodeId());
		//找到对应的执行器
		ShapeExecutor executor = ExecutorsUtils.get(shape);
		if (executor == null) {
			logger.error("执行失败,找不到对应的执行器:{}", shape);
			context.setRunning(false);
		}
		int loopCount = 1;	//循环次数默认为1,如果节点有循环属性且填了循环次数/集合,则取出循环次数
		int loopStart = 0;	//循环起始位置
		int loopEnd = 1;	//循环结束位置
		String loopCountStr = node.getStringJsonValue(ShapeExecutor.LOOP_COUNT);
		Object loopArray = null;
		boolean isLoop = false;
		if (isLoop = StringUtils.isNotBlank(loopCountStr)) {
			try {
				loopArray = ExpressionUtils.execute(loopCountStr, variables);
				if(loopArray == null){
					loopCount = 0;
				}else if(loopArray instanceof Collection){
					loopCount = ((Collection)loopArray).size();
					loopArray = ((Collection)loopArray).toArray();
				}else if(loopArray.getClass().isArray()){
					loopCount = Array.getLength(loopArray);
				}else{
					loopCount = NumberUtils.toInt(loopArray.toString(),0);
					loopArray = null;
				}
				loopEnd = loopCount;
				if(loopCount > 0){
					loopStart = Math.max(NumberUtils.toInt(node.getStringJsonValue(LoopExecutor.LOOP_START), 0),0);
					int end = NumberUtils.toInt(node.getStringJsonValue(LoopExecutor.LOOP_END), -1);
					if(end >=0){
						loopEnd = Math.min(end,loopEnd);
					}else{
						loopEnd = Math.max(loopEnd + end + 1,0);
					}
				}
				logger.info("获取循环次数{}={}", loopCountStr, loopCount);
			} catch (Throwable t) {
				loopCount = 0;
				logger.error("获取循环次数失败,异常信息:{}", t);
			}
		}
		if (loopCount > 0) {
			//获取循环下标的变量名称
			String loopVariableName = node.getStringJsonValue(ShapeExecutor.LOOP_VARIABLE_NAME);
			String loopItem = node.getStringJsonValue(LoopExecutor.LOOP_ITEM,"item");
			List<SpiderTask> tasks = new ArrayList<>();
			for (int i = loopStart; i < loopEnd; i++) {
				node.increment();	//节点执行次数+1(后续Join节点使用)
				if (context.isRunning()) {
					Map<String, Object> nVariables = new HashMap<>();
					// 判断是否需要传递变量
					if(fromNode == null || node.isTransmitVariable(fromNode.getNodeId())){
						nVariables.putAll(variables);
					}
					if(isLoop){
						// 存入下标变量
						if (!StringUtils.isBlank(loopVariableName)) {
							nVariables.put(loopVariableName, i);
						}
						// 存入item
						nVariables.put(loopItem,loopArray == null ? i : Array.get(loopArray, i));
					}
					tasks.add(new SpiderTask(TtlRunnable.get(() -> {
						if (context.isRunning()) {
							try {
								//死循环检测,当执行节点次数大于阈值时,结束本次测试
								AtomicInteger executeCount = context.get(ATOMIC_DEAD_CYCLE);
								if (executeCount != null && executeCount.incrementAndGet() > deadCycle) {
									context.setRunning(false);
									return;
								}
								//执行节点具体逻辑
								executor.execute(node, context, nVariables);
								//当未发生异常时,移除ex变量
								nVariables.remove("ex");
							} catch (Throwable t) {
								nVariables.put("ex", t);
								logger.error("执行节点[{}:{}]出错,异常信息:{}", node.getNodeName(), node.getNodeId(), t);
							}
						}
					}), node, nVariables, executor));
				}
			}
			LinkedBlockingQueue<Future<?>> futureQueue = context.getFutureQueue();
			for (SpiderTask task : tasks) {
				if(executor.isThread()){	//判断节点是否是异步运行
					//提交任务至线程池中,并将Future添加到队列末尾
					futureQueue.add(context.getThreadPool().submitAsync(task.runnable, task, node));
				}else{
					FutureTask<SpiderTask> futureTask = new FutureTask<>(task.runnable, task);
					futureTask.run();
					futureQueue.add(futureTask);
				}
			}
		}
	}

	/**
	 *	判断箭头上的表达式是否成立
	 */
	private boolean executeCondition(SpiderNode fromNode, SpiderNode node, Map<String, Object> variables, SpiderContext context) {
		if (fromNode != null) {
			boolean hasException = variables.get("ex") != null;
			String exceptionFlow = node.getExceptionFlow(fromNode.getNodeId());
			//当出现异常流转 : 1
			//未出现异常流转 : 2
			if(("1".equalsIgnoreCase(exceptionFlow) && !hasException) || ("2".equalsIgnoreCase(exceptionFlow) && hasException)){
				return false;
			}
			String condition = node.getCondition(fromNode.getNodeId());
			if (StringUtils.isNotBlank(condition)) { // 判断是否有条件
				Object result = null;
				try {
					result = ExpressionUtils.execute(condition, variables);
				} catch (Exception e) {
					logger.error("判断{}出错,异常信息:{}", condition, e);
				}
				if (result != null) {
					boolean isContinue = "true".equals(result) || Objects.equals(result, true);
					logger.debug("判断{}={}", condition, isContinue);
					return isContinue;
				}
				return false;
			}
		}
		return true;
	}

	class SpiderTask{

		Runnable runnable;

		SpiderNode node;

		Map<String,Object> variables;

		ShapeExecutor executor;

		public SpiderTask(Runnable runnable, SpiderNode node, Map<String, Object> variables,ShapeExecutor executor) {
			this.runnable = runnable;
			this.node = node;
			this.variables = variables;
			this.executor = executor;
		}
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/Base64FunctionExecutor.java
================================================
package org.spiderflow.core.executor.function;

import org.apache.commons.codec.binary.Base64;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExecutor;
import org.springframework.stereotype.Component;

/**
 * 字符串内容和Base64互相转换 工具类 防止NPE
 * @author Administrator
 *
 */
@Component
@Comment("base64常用方法")
public class Base64FunctionExecutor implements FunctionExecutor{
	
	@Override
	public String getFunctionPrefix() {
		return "base64";
	}
	
	@Comment("根据byte[]进行base64加密")
	@Example("${base64.encode(resp.bytes)}")
	public static String encode(byte[] bytes){
		return bytes != null ? Base64.encodeBase64String(bytes) : null;
	}
	
	@Comment("根据String进行base64加密")
	@Example("${base64.encode(resp.bytes,'UTF-8')}")
	public static String encode(String content,String charset){
		return encode(StringFunctionExecutor.bytes(content,charset));
	}
	
	@Comment("根据String进行base64加密")
	@Example("${base64.encode(resp.html)}")
	public static String encode(String content){
		return encode(StringFunctionExecutor.bytes(content));
	}
	
	@Comment("根据byte[]进行base64加密")
	@Example("${base64.encodeBytes(resp.bytes)}")
	public static byte[] encodeBytes(byte[] bytes){
		return bytes != null ? Base64.encodeBase64(bytes) : null;
	}
	
	@Comment("根据String进行base64加密")
	@Example("${base64.encodeBytes(resp.html,'UTF-8')}")
	public static byte[] encodeBytes(String content,String charset){
		return encodeBytes(StringFunctionExecutor.bytes(content,charset));
	}
	
	@Comment("根据String进行base64加密")
	@Example("${base64.encodeBytes(resp.html)}")
	public static byte[] encodeBytes(String content){
		return encodeBytes(StringFunctionExecutor.bytes(content));
	}
	
	@Comment("根据String进行base64解密")
	@Example("${base64.decode(resp.html)}")
	public static byte[] decode(String base64){
		return base64  != null ? Base64.decodeBase64(base64) :null;
	}
	
	@Comment("根据byte[]进行base64解密")
	@Example("${base64.decode(resp.bytes)}")
	public static byte[] decode(byte[] base64){
		return base64  != null ? Base64.decodeBase64(base64) :null;
	}
	
	@Comment("根据String进行base64解密")
	@Example("${base64.decodeString(resp.html)}")
	public static String decodeString(String base64){
		return base64  != null ? new String(Base64.decodeBase64(base64)) :null;
	}
	
	@Comment("根据byte[]进行base64解密")
	@Example("${base64.decodeString(resp.bytes)}")
	public static String decodeString(byte[] base64){
		return base64  != null ? new String(Base64.decodeBase64(base64)) :null;
	}
	
	@Comment("根据byte[]进行base64解密")
	@Example("${base64.decodeString(resp.bytes,'UTF-8')}")
	public static String decodeString(byte[] base64,String charset){
		return base64  != null ? StringFunctionExecutor.newString(Base64.decodeBase64(base64),charset) :null;
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/DateFunctionExecutor.java
================================================
package org.spiderflow.core.executor.function;

import java.text.ParseException;
import java.util.Date;

import org.apache.commons.lang3.time.DateFormatUtils;
import org.apache.commons.lang3.time.DateUtils;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExecutor;
import org.springframework.stereotype.Component;

/**
 * 时间获取/格式化 工具类 防止NPE 默认格式(yyyy-MM-dd HH:mm:ss)
 * @author Administrator
 *
 */
@Component
@Comment("日期常用方法")
public class DateFunctionExecutor implements FunctionExecutor{
	
	@Override
	public String getFunctionPrefix() {
		return "date";
	}
	
	private static final String DEFAULT_PATTERN = "yyyy-MM-dd HH:mm:ss";

	@Comment("格式化日期")
	@Example("${date.format(date.now())}")
	public static String format(Date date) {
		return format(date, DEFAULT_PATTERN);
	}

	@Comment("格式化日期")
	@Example("${date.format(1569059534000l)}")
	public static String format(Long millis) {
		return format(millis, DEFAULT_PATTERN);
	}

	@Comment("格式化日期")
	@Example("${date.format(date.now(),'yyyy-MM-dd')}")
	public static String format(Date date, String pattern) {
		return date != null ? DateFormatUtils.format(date, pattern) : null;
	}

	@Comment("格式化日期")
	@Example("${date.format(1569059534000l,'yyyy-MM-dd')}")
	public static String format(Long millis, String pattern) {
		return millis != null ? DateFormatUtils.format(millis, pattern) : null;
	}
	
	@Comment("字符串转为日期类型")
	@Example("${date.parse('2019-01-01 00:00:00')}")
	public static Date parse(String date) throws ParseException{
		return date != null ? DateUtils.parseDate(date, DEFAULT_PATTERN) : null;
	}
	
	@Comment("字符串转为日期类型")
	@Example("${date.parse('2019-01-01','yyyy-MM-dd')}")
	public static Date parse(String date,String pattern) throws ParseException{
		return date != null ? DateUtils.parseDate(date, pattern) : null;
	}

	@Comment("数字为日期类型")
	@Example("${date.parse(1569059534000l)}")
	public static Date parse(Long millis){
		return new Date(millis);
	}
	
	@Comment("获取当前时间")
	@Example("${date.now()}")
	public static Date now(){
		return new Date();
	}
	
	@Comment("获取指定日期n年后的日期")
	@Example("${date.addYears(date.now(),2)}")
	public static Date addYears(Date date,int amount){
		return DateUtils.addYears(date, amount);
	}
	
	@Comment("获取指定日期n月后的日期")
	@Example("${date.addMonths(date.now(),2)}")
	public static Date addMonths(Date date,int amount){
		return DateUtils.addMonths(date, amount);
	}
	
	@Comment("获取指定日期n天后的日期")
	@Example("${date.addDays(date.now(),2)}")
	public static Date addDays(Date date,int amount){
		return DateUtils.addDays(date, amount);
	}
	
	@Comment("获取指定日期n小时后的日期")
	@Example("${date.addHours(date.now(),2)}")
	public static Date addHours(Date date,int amount){
		return DateUtils.addHours(date, amount);
	}
	
	@Comment("获取指定日期n分钟后的日期")
	@Example("${date.addMinutes(date.now(),2)}")
	public static Date addMinutes(Date date,int amount){
		return DateUtils.addMinutes(date, amount);
	}
	
	@Comment("获取指定日期n秒后的日期")
	@Example("${date.addSeconds(date.now(),2)}")
	public static Date addSeconds(Date date,int amount){
		return DateUtils.addSeconds(date, amount);
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/ExtractFunctionExecutor.java
================================================
package org.spiderflow.core.executor.function;

import java.util.List;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Element;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.core.utils.ExtractUtils;
import org.spiderflow.executor.FunctionExecutor;
import org.springframework.stereotype.Component;

@Component
@Comment("数据抽取常用方法")
public class ExtractFunctionExecutor implements FunctionExecutor{

	@Override
	public String getFunctionPrefix() {
		return "extract";
	}
	
	@Comment("根据jsonpath提取内容")
	@Example("${extract.jsonpath(resp.json,'$.code')}")
	public static Object jsonpath(Object root,String jsonpath){
		return ExtractUtils.getValueByJsonPath(root, jsonpath);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${extract.regx(resp.html,'<title>(.*?)</title>')}")
	public static String regx(String content,String pattern){
		return ExtractUtils.getFirstMatcher(content, pattern, true);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${extract.regx(resp.html,'<title>(.*?)</title>',1)}")
	public static String regx(String content,String pattern,int groupIndex){
		return ExtractUtils.getFirstMatcher(content, pattern, groupIndex);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${extract.regx(resp.html,'<a href=\"(.*?)\">(.*?)</a>',[1,2])}")
	public static List<String> regx(String content,String pattern,List<Integer> groups){
		return ExtractUtils.getFirstMatcher(content, pattern, groups);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${extract.regxs(resp.html,'<h2>(.*?)</h2>')}")
	public static List<String> regxs(String content,String pattern){
		return ExtractUtils.getMatchers(content, pattern, true);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${extract.regxs(resp.html,'<h2>(.*?)</h2>',1)}")
	public static List<String> regxs(String content,String pattern,int groupIndex){
		return ExtractUtils.getMatchers(content, pattern, groupIndex);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${extract.regxs(resp.html,'<a href=\"(.*?)\">(.*?)</a>',[1,2])}")
	public static List<List<String>> regxs(String content,String pattern,List<Integer> groups){
		return ExtractUtils.getMatchers(content, pattern, groups);
	}
	
	@Comment("根据xpath提取内容")
	@Example("${extract.xpath(resp.element(),'//title/text()')}")
	public static String xpath(Element element,String xpath){
		return ExtractUtils.getValueByXPath(element, xpath);
	}
	
	@Comment("根据xpath提取内容")
	@Example("${extract.xpath(resp.html,'//title/text()')}")
	public static String xpath(String content,String xpath){
		return xpath(Jsoup.parse(content),xpath);
	}
	
	@Comment("根据xpaths提取内容")
	@Example("${extract.xpaths(resp.element(),'//h2/text()')}")
	public static List<String> xpaths(Element element,String xpath){
		return ExtractUtils.getValuesByXPath(element, xpath);
	}
	
	@Comment("根据xpaths提取内容")
	@Example("${extract.xpaths(resp.html,'//h2/text()')}")
	public static List<String> xpaths(String content,String xpath){
		return xpaths(Jsoup.parse(content),xpath);
	}
	
	@Comment("根据css选择器提取内容")
	@Example("${extract.selectors(resp.html,'div > a')}")
	public static List<String> selectors(Object object,String selector){
		return ExtractUtils.getHTMLBySelector(getElement(object), selector);
	}
	
	@Comment("根据css选择器提取内容")
	@Example("${extract.selector(resp.html,'div > a','text')}")
	public static Object selector(Object object,String selector,String type){
		if("element".equals(type)){
			return ExtractUtils.getFirstElement(getElement(object), selector);
		}else if("text".equals(type)){
			return ExtractUtils.getFirstTextBySelector(getElement(object), selector);
		}else if("outerhtml".equals(type)){
			return ExtractUtils.getFirstOuterHTMLBySelector(getElement(object), selector);
		}
		return null;
	}
	
	@Comment("根据css选择器提取内容")
	@Example("${extract.selector(resp.html,'div > a','attr','href')}")
	public static String selector(Object object,String selector,String type,String attrValue){
		if("attr".equals(type)){
			return ExtractUtils.getFirstAttrBySelector(getElement(object), selector,attrValue);
		}
		return null;
	}
	
	@Comment("根据css选择器提取内容")
	@Example("${extract.selector(resp.html,'div > a')}")
	public static String selector(Object object,String selector){
		return ExtractUtils.getFirstHTMLBySelector(getElement(object), selector);
	}
	
	@Comment("根据css选择器提取内容")
	@Example("${extract.selectors(resp.html,'div > a','element')}")
	public static Object selectors(Object object,String selector,String type){
		if("element".equals(type)){
			return ExtractUtils.getElements(getElement(object), selector);
		}else if("text".equals(type)){
			return ExtractUtils.getTextBySelector(getElement(object), selector);
		}else if("outerhtml".equals(type)){
			return ExtractUtils.getOuterHTMLBySelector(getElement(object), selector);
		}
		return null;
	}
	
	@Comment("根据css选择器提取内容")
	@Example("${extract.selectors(resp.html,'div > a','attr','href')}")
	public static Object selectors(Object object,String selector,String type,String attrValue){
		if("attr".equals(type)){
			return ExtractUtils.getAttrBySelector(getElement(object), selector,attrValue);
		}
		return null;
	}
	
	private static Element getElement(Object object){
		if(object != null){
			return object instanceof Element ? (Element)object:Jsoup.parse((String) object);
		}
		return null;
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/FileFunctionExecutor.java
================================================
package org.spiderflow.core.executor.function;

import java.io.*;
import java.nio.charset.Charset;
import java.util.List;

import org.apache.commons.io.IOUtils;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.core.utils.FileUtils;
import org.spiderflow.executor.FunctionExecutor;
import org.springframework.stereotype.Component;
import org.springframework.util.CollectionUtils;

/**
 * 文件读写 工具类 防止NPE 
 * @author Administrator
 *
 */
@Component
@Comment("file常用方法")
public class FileFunctionExecutor implements FunctionExecutor{
	
	@Override
	public String getFunctionPrefix() {
		return "file";
	}
	
	/**
	 * 
	 * @param path 文件路径/名
	 * @param createDirectory 是否需要创建
	 * @return File 文件
	 */
	private static File getFile(String path,boolean createDirectory){
		File f = new File(path);
		if(createDirectory&&!f.getParentFile().exists()){
			f.getParentFile().mkdirs();
		}
		return f;
	}

	@Comment("写出文件")
	@Example("${file.write('e:/result.html',resp.html,false)}")
	public static void write(String path,String content,boolean append) throws IOException{
		write(path,content,Charset.defaultCharset().name(),append);
	}
	
	@Comment("写出文件")
	@Example("${file.write('e:/result.html',resp.html,'UTF-8',false)}")
	public static void write(String path,String content,String charset,boolean append) throws IOException{
		write(path,StringFunctionExecutor.bytes(content, charset),append);
	}
	
	@Comment("写出文件")
	@Example("${file.write('e:/result.html',resp.bytes,false)}")
	public static void write(String path,byte[] bytes,boolean append) throws IOException{
		write(path, new ByteArrayInputStream(bytes),append);
	}

	@Comment("写出文件")
	@Example("${file.write('e:/result.html',resp.stream,false)}")
	public static void write(String path, InputStream stream, boolean append) throws IOException {
		try(FileOutputStream fos = new FileOutputStream(getFile(path,true),append)){
			IOUtils.copyLarge(stream, fos);
		}
	}

	@Comment("写出文件")
	@Example("${file.write('e:/result.html',resp.bytes,false)}")
	public static void write(String path, InputStream stream) throws IOException {
		write(path, stream,false);
	}
	
	@Comment("写出文件")
	@Example("${file.write('e:/result.html',resp.html)}")
	public static void write(String path,String content) throws IOException{
		write(path,content,false);
	}
	
	@Comment("写出文件")
	@Example("${file.write('e:/result.html',resp.html,'UTF-8')}")
	public static void write(String path,String content,String charset) throws IOException{
		write(path,content,charset,false);
	}
	
	@Comment("写出文件")
	@Example("${file.write('e:/result.html',resp.bytes)}")
	public static void write(String path,byte[] bytes) throws IOException{
		write(path,bytes,false);
	}

	@Comment("下载Url资源")
	@Example("${file.download('e:/downloadPath',urls)}")
	public static void download(String path, List<String> urls) throws IOException{
		if(!CollectionUtils.isEmpty(urls)) {
			for (String url : urls) {
				FileUtils.downloadFile(path, url, true);
			}
		}
	}

	@Comment("下载Url资源")
	@Example("${file.download('e:/downloadPath',urls)}")
	public static void download(String path, String url) throws IOException {
		if (url != null) {
			FileUtils.downloadFile(path, url, true);
		}
	}
	
	@Comment("读取文件")
	@Example("${file.bytes('e:/result.html')}")
	public static byte[] bytes(String path) throws IOException{
		try(FileInputStream fis = new FileInputStream(getFile(path, false))){
			return IOUtils.toByteArray(fis);	
		}
	}
	
	@Comment("读取文件")
	@Example("${file.string('e:/result.html','UTF-8')}")
	public static String string(String path,String charset) throws IOException{
		return StringFunctionExecutor.newString(bytes(path), charset);
	}
	
	@Comment("读取文件")
	@Example("${file.string('e:/result.html')}")
	public static String string(String path) throws IOException{
		return StringFunctionExecutor.newString(bytes(path), Charset.defaultCharset().name());
	}
	
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/JsonFunctionExecutor.java
================================================
package org.spiderflow.core.executor.function;

import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExecutor;
import org.springframework.stereotype.Component;

import com.alibaba.fastjson.JSON;

/**
 * Json和String互相转换 工具类 防止NPE 
 * @author Administrator
 *
 */
@Component
@Comment("json常用方法")
public class JsonFunctionExecutor implements FunctionExecutor{
	
	@Override
	public String getFunctionPrefix() {
		return "json";
	}

	@Comment("将字符串转为json对象")
	@Example("${json.parse('{code : 1}')}")
	public static Object parse(String jsonString){
		return jsonString != null ? JSON.parse(jsonString) : null;
	}
	
	@Comment("将对象转为json字符串")
	@Example("${json.stringify(objVar)}")
	public static String stringify(Object object){
		return object != null ? JSON.toJSONString(object) : null;
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/ListFunctionExecutor.java
================================================
package org.spiderflow.core.executor.function;

import java.util.ArrayList;
import java.util.List;
import java.util.regex.Pattern;

import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExecutor;
import org.springframework.stereotype.Component;

/**
 * List 工具类 防止NPE 添加了类似python的split()方法 
 * @author Administrator
 *
 */
@Component
@Comment("list常用方法")
public class ListFunctionExecutor implements FunctionExecutor{
	
	@Override
	public String getFunctionPrefix() {
		return "list";
	}

	@Comment("获取list的长度")
	@Example("${list.length(listVar)}")
	public static int length(List<?> list){
		return list != null ? list.size() : 0;
	}
	
	/**
	 * 
	 * @param list 原List
	 * @param len 按多长进行分割
	 * @return List<List<?>> 分割后的数组
	 */
	@Comment("分割List")
	@Example("${list.split(listVar,10)}")
	public static List<List<?>> split(List<?> list,int len){
		List<List<?>> result = new ArrayList<>();
		if (list == null || list.size() == 0 || len < 1) {
			return result;
		}
		int size = list.size();
		int count = (size + len - 1) / len;
		for (int i = 0; i < count; i++) {
			List<?> subList = list.subList(i * len, ((i + 1) * len > size ? size : len * (i + 1)));
			result.add(subList);
		}
		return result;
	}
	
	@Comment("截取List")
	@Example("${list.sublist(listVar,fromIndex,toIndex)}")
	public static List<?> sublist(List<?> list,int fromIndex,int toIndex){
		return list!= null ? list.subList(fromIndex, toIndex) : new ArrayList<>();
	}

	@Comment("过滤字符串list元素")
	@Example("${listVar.filterStr(pattern)}")
	public static List<String> filterStr(List<String> list, String pattern) {
		if (list == null || list.isEmpty()) {
			return null;
		}
		List<String> result = new ArrayList<>(list.size());
		for (String item : list) {
			if (Pattern.matches(pattern, item)) {
				result.add(item);
			}
		}
		return result;
	}
		
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/MD5FunctionExecutor.java
================================================
package org.spiderflow.core.executor.function;

import org.apache.commons.codec.digest.DigestUtils;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExecutor;
import org.springframework.stereotype.Component;

import java.io.IOException;
import java.io.InputStream;

@Component
@Comment("MD5常用方法")
public class MD5FunctionExecutor implements FunctionExecutor {

    @Override
    public String getFunctionPrefix() {
        return "md5";
    }

    @Comment("md5加密")
    @Example("${md5.string(resp.html)}")
    public static String string(String str){
        return DigestUtils.md5Hex(str);
    }

    @Comment("md5加密")
    @Example("${md5.string(resp.bytes)}")
    public static String string(byte[] bytes){
        return DigestUtils.md5Hex(bytes);
    }

    @Comment("md5加密")
    @Example("${md5.string(resp.stream)}")
    public static String string(InputStream stream) throws IOException {
        return DigestUtils.md5Hex(stream);
    }

    @Comment("md5加密")
    @Example("${md5.bytes(resp.html)}")
    public static byte[] bytes(String str){
        return DigestUtils.md5(str);
    }

    @Comment("md5加密")
    @Example("${md5.bytes(resp.bytes)}")
    public static byte[] bytes(byte[] bytes){
        return DigestUtils.md5(bytes);
    }

    @Comment("md5加密")
    @Example("${md5.bytes(resp.stream)}")
    public static byte[] bytes(InputStream stream) throws IOException {
        return DigestUtils.md5(stream);
    }
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/RandomFunctionExecutor.java
================================================
package org.spiderflow.core.executor.function;

import org.apache.commons.lang3.RandomUtils;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExecutor;
import org.springframework.stereotype.Component;

/**
 * 随机数/字符串 生成方法 
 * @author Administrator
 *
 */
@Component
public class RandomFunctionExecutor implements FunctionExecutor{
	
	@Override
	public String getFunctionPrefix() {
		return "random";
	}
	
	@Comment("随机获取int")
	@Example("${random.randomInt(1,10)}")
	public static int randomInt(int min,int max){
		return RandomUtils.nextInt(min, max);
	}
	
	@Comment("随机获取double")
	@Example("${random.randomDouble(1,10)}")
	public static double randomDouble(double min,double max){
		return RandomUtils.nextDouble(min, max);
	}
	
	@Comment("随机获取long")
	@Example("${random.randomLong(1,10)}")
	public static long randomLong(long min,long max){
		return RandomUtils.nextLong(min, max);
	}
	
	/**
	 * 
	 * @param chars 字符个数
	 * @param length 字符范围
	 * @return String 随机字符串
	 */
	@Comment("随机获取字符串")
	@Example("${random.string('abcde',10)}")
	public static String string(String chars,int length){
		if (chars != null) {
			char[] newChars = new char[length];
			int len = chars.length();
			for (int i = 0; i < length; i++) {
				newChars[i] = chars.charAt(randomInt(0,len));
			}
			return new String(newChars);
		}
		return null;
	}
	
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/StringFunctionExecutor.java
================================================
package org.spiderflow.core.executor.function;

import java.io.UnsupportedEncodingException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.UUID;

import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExecutor;
import org.springframework.stereotype.Component;

/**
 * String 工具类 防止NPE 
 * @author Administrator
 *
 */
@Component
@Comment("string常用方法")
public class StringFunctionExecutor implements FunctionExecutor{
	
	@Override
	public String getFunctionPrefix() {
		return "string";
	}

	@Comment("截取字符串方法")
	@Example("${string.substring(str,5)}")
	public static String substring(String content, int beginIndex) {
		return content != null ? content.substring(beginIndex) : null;
	}

	@Comment("截取字符串方法")
	@Example("${string.substring(str,0,str.length() - 1)}")
	public static String substring(String content, int beginIndex, int endIndex) {
		return content != null ? content.substring(beginIndex, endIndex) : null;
	}

	@Comment("将字符串转为小写")
	@Example("${string.lower(str)}")
	public static String lower(String content) {
		return content != null ? content.toLowerCase() : null;
	}

	@Comment("将字符串转为大写")
	@Example("${string.upper(str)}")
	public static String upper(String content) {
		return content != null ? content.toUpperCase() : null;
	}

	@Comment("查找指定字符在字符串在中的位置")
	@Example("${string.indexOf(content,str)}")
	public static int indexOf(String content, String str) {
		return content != null ? content.indexOf(str) : -1;
	}

	@Comment("查找指定字符在字符串中最后出现的位置")
	@Example("${string.lastIndexOf(content,str)}")
	public static int lastIndexOf(String content, String str) {
		return content != null ? content.lastIndexOf(str) : -1;
	}

	@Comment("查找指定字符在字符串在中的位置")
	@Example("${string.indexOf(content,str,fromIndex)}")
	public static int indexOf(String content, String str, int fromIndex) {
		return content != null ? content.indexOf(str, fromIndex) : -1;
	}
	
	@Comment("将字符串转为int")
	@Example("${string.toInt(value)}")
	public static int toInt(String value){
		return Integer.parseInt(value);
	}
	
	@Comment("将字符串转为Integer")
	@Example("${string.toInt(value,defaultValue)}")
	public static Integer toInt(String value,Integer defaultValue){
		try {
			return Integer.parseInt(value);
		} catch (Exception e) {
			return defaultValue;
		}
	}
	
	@Comment("字符串替换")
	@Example("${string.replace(content,source,target)}")
	public static String replace(String content,String source,String target){
		return content != null ? content.replace(source, target): null;
	}
	
	@Comment("正则替换字符串")
	@Example("${string.replaceAll(content,regx,target)}")
	public static String replaceAll(String content,String regx,String target){
		return content != null ? content.replaceAll(regx, target): null;
	}
	
	@Comment("正则替换字符串")
	@Example("${string.replaceFirst(content,regx,target)}")
	public static String replaceFirst(String content,String regx,String target){
		return content != null ? content.replaceFirst(regx, target): null;
	}
	
	@Comment("正则替换字符串")
	@Example("${string.length(content)}")
	public static int length(String content){
		return content != null ? content.length() : -1;
	}
	
	@Comment("去除字符串两边的空格")
	@Example("${string.trim(content)}")
	public static String trim(String content){
		return content != null ? content.trim() : null;
	}
	
	@Comment("分割字符串")
	@Example("${string.split(content,regx)}")
	public static List<String> split(String content,String regx){
		return content != null ? Arrays.asList(content.split(regx)) : new ArrayList<>(0);
	}
	
	@Comment("获取字符串的byte[]")
	@Example("${string.bytes(content)}")
	public static byte[] bytes(String content){
		return content != null ? content.getBytes() : null;
	}
	
	@Comment("获取字符串的byte[]")
	@Example("${string.bytes(content,charset)}")
	public static byte[] bytes(String content,String charset){
		try {
			return content != null ? content.getBytes(charset) : null;
		} catch (UnsupportedEncodingException e) {
			return null;
		}
	}
	
	@Comment("byte[]转String")
	@Example("${string.newString(bytes)}")
	public static String newString(byte[] bytes){
		return bytes != null ? new String(bytes) : null;
	}
	
	@Comment("byte[]转String")
	@Example("${string.newString(bytes,charset)}")
	public static String newString(byte[] bytes,String charset){
		try {
			return bytes != null ? new String(bytes,charset) : null;
		} catch (UnsupportedEncodingException e) {
			return null;
		}
	}
	
	@Comment("判断两个字符串是否相同")
	@Example("${string.newString(bytes,charset)}")
	public static boolean equals(String str1,String str2){
		return str1 == null ? str2 == null : str1.equals(str2);
	}
	
	@Comment("生成UUID")
	@Example("${string.uuid()}")
	public static String uuid() {
		return UUID.randomUUID().toString().replace("-", "");
	}
	
	@Comment("生成多个UUID")
	@Example("${string.uuid(size)}")
	public static List<String> uuids(Integer size) {
		List<String> ids = new ArrayList<String>();
		for (int i = 0; i < size; i++) {
			ids.add(UUID.randomUUID().toString().replace("-", ""));
		}
		return ids;
	}

}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/ThreadFunctionExecutor.java
================================================
package org.spiderflow.core.executor.function;

import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExecutor;
import org.springframework.stereotype.Component;

/**
 * Created on 2019-12-06
 *
 * @author Octopus
 */
@Component
@Comment("thread常用方法")
public class ThreadFunctionExecutor implements FunctionExecutor {
    @Override
    public String getFunctionPrefix() {
        return "thread";
    }

    @Comment("线程休眠")
    @Example("${thread.sleep(1000L)}")
    public static void sleep(Long sleepTime){
        try {
            Thread.sleep(sleepTime);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/UrlFunctionExecutor.java
================================================
package org.spiderflow.core.executor.function;

import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.net.URLEncoder;
import java.nio.charset.Charset;
import java.util.HashMap;
import java.util.Map;

import org.apache.commons.lang3.StringUtils;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExecutor;
import org.springframework.stereotype.Component;

/**
 * url 按指定字符集进行编码/解码 默认字符集(UTF-8) 工具类 防止NPE 
 */
@Component
public class UrlFunctionExecutor implements FunctionExecutor{
	
	@Override
	public String getFunctionPrefix() {
		return "url";
	}
	
	@Comment("获取url参数")
	@Example("${url.parameter('http://www.baidu.com/s?wd=spider-flow','wd')}")
	public static String parameter(String url,String key){
		return parameterMap(url).get(key);
	}
	
	@Comment("获取url全部参数")
	@Example("${url.parameterMap('http://www.baidu.com/s?wd=spider-flow&abbr=sf')}")
	public static Map<String,String> parameterMap(String url){
		Map<String,String> map = new HashMap<String,String>();
		int index = url.indexOf("?");
		if(index != -1) {
	        String param = url.substring(index+1);
	        if(StringUtils.isNotBlank(param)) {
		        String[] params = param.split("&");
		        for (String item : params) {
		            String[] kv = item.split("=");
		            if(kv.length > 0) {
		            	if(StringUtils.isNotBlank(kv[0])) {
		            		String value = "";
		            		if(StringUtils.isNotBlank(kv[1])) {
		            			int kv1Index = kv[1].indexOf("#");
		            			if(kv1Index != -1) {
		            				value = kv[1].substring(0,kv1Index);
		            			}else {
		            				value = kv[1];
		            			}
		            		}
		            		map.put(kv[0],value);
		            	}
		            }
		        }
	        }
		}
		return map;
	}
	
	@Comment("url编码")
	@Example("${url.encode('http://www.baidu.com/s?wd=spider-flow')}")
	public static String encode(String url){
		return encode(url,Charset.defaultCharset().name());
	}
	
	@Comment("url编码")
	@Example("${url.encode('http://www.baidu.com/s?wd=spider-flow','UTF-8')}")
	public static String encode(String url,String charset){
		try {
			return url != null ? URLEncoder.encode(url,charset) : null;
		} catch (UnsupportedEncodingException e) {
			return null;
		}
	}
	
	@Comment("url解码")
	@Example("${url.decode(strVar)}")
	public static String decode(String url){
		return decode(url,Charset.defaultCharset().name());
	}
	
	@Comment("url解码")
	@Example("${url.decode(strVar,'UTF-8')}")
	public static String decode(String url,String charset){
		try {
			return url != null ? URLDecoder.decode(url, charset) : null;
		} catch (UnsupportedEncodingException e) {
			return null;
		}
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ArrayFunctionExtension.java
================================================
package org.spiderflow.core.executor.function.extension;

import java.util.Arrays;
import java.util.List;

import org.apache.commons.lang3.StringUtils;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExtension;
import org.springframework.stereotype.Component;

@Component
public class ArrayFunctionExtension implements FunctionExtension{
	
	@Override
	public Class<?> support() {
		return Object[].class;
	}
	
	@Comment("获取数组的长度")
	@Example("${arrayVar.size()}")
	public static int size(Object[] objs){
		return objs.length;
	}
	
	@Comment("将数组拼接起来")
	@Example("${arrayVar.join()}")
	public static String join(Object[] objs,String separator){
		return StringUtils.join(objs,separator);
	}
	
	@Comment("将数组用separator拼接起来")
	@Example("${arrayVar.join('-')}")
	public static String join(Object[] objs){
		return StringUtils.join(objs);
	}
	
	@Comment("将数组转为List")
	@Example("${arrayVar.toList()}")
	public static List<?> toList(Object[] objs){
		return Arrays.asList(objs);
	}

}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/DateFunctionExtension.java
================================================
package org.spiderflow.core.executor.function.extension;

import java.util.Date;

import org.apache.commons.lang3.time.DateFormatUtils;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExtension;
import org.springframework.stereotype.Component;

@Component
public class DateFunctionExtension implements FunctionExtension{

	@Override
	public Class<?> support() {
		return Date.class;
	}
	
	@Comment("格式化日期")
	@Example("${dateVar.format()}")
	public static String format(Date date){
		return format(date, "yyyy-MM-dd HH:mm:ss");
	}
	
	@Comment("格式化日期")
	@Example("${dateVar.format('yyyy-MM-dd HH:mm:ss')}")
	public static String format(Date date,String pattern){
		return DateFormatUtils.format(date,pattern);
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ElementFunctionExtension.java
================================================
package org.spiderflow.core.executor.function.extension;

import java.util.List;

import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.annotation.Return;
import org.spiderflow.core.utils.ExtractUtils;
import org.spiderflow.executor.FunctionExtension;
import org.springframework.stereotype.Component;

@Component
public class ElementFunctionExtension implements FunctionExtension{

	@Override
	public Class<?> support() {
		return Element.class;
	}
	
	@Comment("根据xpath提取内容")
	@Example("${elementVar.xpath('//title/text()')}")
	@Return({Element.class,String.class})
	public static String xpath(Element element,String xpath){
		return ExtractUtils.getValueByXPath(element, xpath);
	}
	

	@Comment("根据xpath提取内容")
	@Example("${elementVar.xpaths('//h2/text()')}")
	@Return({Element.class,String.class})
	public static List<String> xpaths(Element element,String xpath){
		return ExtractUtils.getValuesByXPath(element, xpath);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${elementVar.regx('<title>(.*?)</title>')}")
	public static String regx(Element element,String regx){
		return ExtractUtils.getFirstMatcher(element.html(), regx, true);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${elementVar.regx('<title>(.*?)</title>',1)}")
	public static String regx(Element element,String regx,int groupIndex){
		return ExtractUtils.getFirstMatcher(element.html(), regx, groupIndex);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${elementVar.regx('<a href=\"(.*?)\">(.*?)</a>',[1,2])}")
	public static List<String> regx(Element element,String regx,List<Integer> groups){
		return ExtractUtils.getFirstMatcher(element.html(), regx, groups);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${elementVar.regxs('<h2>(.*?)</h2>')}")
	public static List<String> regxs(Element element,String regx){
		return ExtractUtils.getMatchers(element.html(), regx, true);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${elementVar.regxs('<h2>(.*?)</h2>',1)}")
	public static List<String> regxs(Element element,String regx,int groupIndex){
		return ExtractUtils.getMatchers(element.html(), regx, groupIndex);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${elementVar.regxs('<a href=\"(.*?)\">(.*?)</a>',[1,2])}")
	public static List<List<String>> regxs(Element element,String regx,List<Integer> groups){
		return ExtractUtils.getMatchers(element.html(), regx, groups);
	}
	
	@Comment("根据css选择器提取内容")
	@Example("${elementVar.selector('div > a')}")
	public static Element selector(Element element,String cssQuery){
		return element.selectFirst(cssQuery);
	}
	
	@Comment("根据css选择器提取内容")
	@Example("${elementVar.selectors('div > a')}")
	public static Elements selectors(Element element,String cssQuery){
		return element.select(cssQuery);
	}

	@Comment("获取同级节点")
	@Example("${elementVar.subling()}")
	public static Elements subling(Element element){
		return element.siblingElements();
	}

	@Comment("获取上级节点")
	@Example("${elementVar.parent()}")
	public static Element parent(Element element){
		return element.parent();
	}

	@Comment("获取上级节点")
	@Example("${elementVar.parents()}")
	public static Elements parents(Element element){
		return element.parents();
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ElementsFunctionExtension.java
================================================
package org.spiderflow.core.executor.function.extension;

import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.core.utils.ExtractUtils;
import org.spiderflow.executor.FunctionExtension;
import org.springframework.stereotype.Component;

import java.util.ArrayList;
import java.util.List;

@Component
public class ElementsFunctionExtension implements FunctionExtension{

	@Override
	public Class<?> support() {
		return Elements.class;
	}
	
	@Comment("根据xpath提取内容")
	@Example("${elementsVar.xpath('//title/text()')}")
	public static String xpath(Elements elements,String xpath){
		return ExtractUtils.getValueByXPath(elements, xpath);
	}
	
	@Comment("根据xpath提取内容")
	@Example("${elementsVar.xpaths('//h2/text()')}")
	public static List<String> xpaths(Elements elements,String xpath){
		return ExtractUtils.getValuesByXPath(elements, xpath);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${elementsVar.regx('<title>(.*?)</title>')}")
	public static String regx(Elements elements,String regx){
		return ExtractUtils.getFirstMatcher(elements.html(), regx, true);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${elementsVar.regx('<title>(.*?)</title>',1)}")
	public static String regx(Elements elements,String regx,int groupIndex){
		return ExtractUtils.getFirstMatcher(elements.html(), regx, groupIndex);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${elementsVar.regx('<a href=\"(.*?)\">(.*?)</a>',[1,2])}")
	public static List<String> regx(Elements elements,String regx,List<Integer> groups){
		return ExtractUtils.getFirstMatcher(elements.html(), regx, groups);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${elementsVar.regxs('<h2>(.*?)</h2>')}")
	public static List<String> regxs(Elements elements,String regx){
		return ExtractUtils.getMatchers(elements.html(), regx, true);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${elementsVar.regxs('<h2>(.*?)</h2>',1)}")
	public static List<String> regxs(Elements elements,String regx,int groupIndex){
		return ExtractUtils.getMatchers(elements.html(), regx, groupIndex);
	}
	
	@Comment("根据正则表达式提取内容")
	@Example("${elementsVar.regxs('<a href=\"(.*?)\">(.*?)</a>',[1,2])}")
	public static List<List<String>> regxs(Elements elements,String regx,List<Integer> groups){
		return ExtractUtils.getMatchers(elements.html(), regx, groups);
	}
	
	@Comment("根据css选择器提取内容")
	@Example("${elementsVar.selector('div > a')}")
	public static Element selector(Elements elements,String selector){
		Elements foundElements = elements.select(selector);
		if(foundElements.size() > 0){
			return foundElements.get(0);
		}
		return null;
	}

	@Comment("返回所有attr")
	@Example("${elementsVar.attrs('href')}")
	public static List<String> attrs(Elements elements,String key){
		List<String> list = new ArrayList<>(elements.size());
		for (Element element : elements) {
			list.add(element.attr(key));
		}
		return list;
	}

	@Comment("返回所有value")
	@Example("${elementsVar.vals()}")
	public static List<String> vals(Elements elements){
		List<String> list = new ArrayList<>(elements.size());
		for (Element element : elements) {
			list.add(element.val());
		}
		return list;
	}

	@Comment("返回所有text")
	@Example("${elementsVar.texts()}")
	public static List<String> texts(Elements elements){
		List<String> list = new ArrayList<>(elements.size());
		for (Element element : elements) {
			list.add(element.text());
		}
		return list;
	}

	@Comment("返回所有html")
	@Example("${elementsVar.htmls()}")
	public static List<String> htmls(Elements elements){
		List<String> list = new ArrayList<>(elements.size());
		for (Element element : elements) {
			list.add(element.html());
		}
		return list;
	}

	@Comment("返回所有outerHtml")
	@Example("${elementsVar.outerHtmls()}")
	public static List<String> outerHtmls(Elements elements){
		List<String> list = new ArrayList<>(elements.size());
		for (Element element : elements) {
			list.add(element.outerHtml());
		}
		return list;
	}

	@Comment("返回所有ownTexts")
	@Example("${elementsVar.ownTexts()}")
	public static List<String> ownTexts(Elements elements){
		List<String> list = new ArrayList<>(elements.size());
		for (Element element : elements) {
			list.add(element.ownText());
		}
		return list;
	}

	@Comment("返回所有wholeText")
	@Example("${elementsVar.wholeTexts()}")
	public static List<String> wholeTexts(Elements elements){
		List<String> list = new ArrayList<>(elements.size());
		for (Element element : elements) {
			list.add(element.wholeText());
		}
		return list;
	}
	
	@Comment("根据css选择器提取内容")
	@Example("${elementsVar.selectors('div > a')}")
	public static Elements selectors(Elements elements,String selector){
		return elements.select(selector);
	}

	@Comment("获取上级节点")
	@Example("${elementsVar.parents()}")
	public static Elements parents(Elements elements){
		return elements.parents();
	}

}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ListFunctionExtension.java
================================================
package org.spiderflow.core.executor.function.extension;

import org.apache.commons.lang3.StringUtils;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExtension;
import org.springframework.stereotype.Component;

import java.util.Collections;
import java.util.List;


@Component
public class ListFunctionExtension implements FunctionExtension{

	@Override
	public Class<?> support() {
		return List.class;
	}
	
	@Comment("获取list的长度")
	@Example("${listVar.length()}")
	public static int length(List<?> list){
		return list.size();
	}
	
	@Comment("将list拼接起来")
	@Example("${listVar.join()}")
	public static String join(List<?> list){
		return StringUtils.join(list.toArray());
	}
	
	@Comment("将list用separator拼接起来")
	@Example("${listVar.join('-')}")
	public static String join(List<?> list,String separator){
		if(list.size() == 1){
			return list.get(0).toString();
		}else{
			return StringUtils.join(list.toArray(),separator);
		}
	}

	@Comment("将list<String>排序")
	@Example("${listVar.sort()}")
	public static List<String> sort(List<String> list){
		Collections.sort(list);
		return list;
	}

	@Comment("将list打乱顺序")
	@Example("${listVar.shuffle()}")
	public static List<?> shuffle(List<?> list){
		Collections.shuffle(list);
		return list;
	}
	
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/MapFunctionExtension.java
================================================
package org.spiderflow.core.executor.function.extension;

import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExtension;
import org.springframework.stereotype.Component;

import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

@Component
public class MapFunctionExtension implements FunctionExtension {

	@Override
	public Class<?> support() {
		return Map.class;
	}

	@Comment("将map转换为List")
	@Example("${mapmVar.toList('=')}")
	public static List<String> toList(Map<?,?> map,String separator){
		return map.entrySet().stream().map(entry-> entry.getKey() + separator + entry.getValue()).collect(Collectors.toList());
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ObjectFunctionExtension.java
================================================
package org.spiderflow.core.executor.function.extension;

import java.util.Objects;

import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.core.utils.ExtractUtils;
import org.spiderflow.executor.FunctionExtension;
import org.springframework.stereotype.Component;

import com.alibaba.fastjson.JSON;

@Component
public class ObjectFunctionExtension implements FunctionExtension{
	
	@Override
	public Class<?> support() {
		return Object.class;
	}
	
	@Comment("将对象转为string类型")
	@Example("${objVar.string()}")
	public static String string(Object obj){
		if (obj instanceof String) {
			return (String) obj;
		}
		return Objects.toString(obj);
	}
	
	@Comment("根据jsonpath提取内容")
	@Example("${objVar.jsonpath('$.code')}")
	public static Object jsonpath(Object obj,String path){
		if(obj instanceof String){
			return ExtractUtils.getValueByJsonPath(JSON.parse((String)obj), path);
		}
		return ExtractUtils.getValueByJsonPath(obj, path);
	}

	@Comment("睡眠等待一段时间")
	@Example("${objVar.sleep(1000)}")
	public static Object sleep(Object obj, int millis) {
		try {
			Thread.sleep(millis);
		} catch (InterruptedException ignored) {
		}
		return obj;
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ResponseFunctionExtension.java
================================================
package org.spiderflow.core.executor.function.extension;

import java.util.List;
import java.util.regex.Pattern;
import java.util.stream.Collectors;

import org.apache.commons.lang3.StringUtils;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.annotation.Return;
import org.spiderflow.core.utils.ExtractUtils;
import org.spiderflow.executor.FunctionExtension;
import org.spiderflow.io.SpiderResponse;
import org.springframework.stereotype.Component;

@Component
public class ResponseFunctionExtension implements FunctionExtension {

    @Override
    public Class<?> support() {
        return SpiderResponse.class;
    }

    @Comment("将请求结果转为Element对象")
    @Example("${resp.element()}")
    public static Element element(SpiderResponse response) {
        return Jsoup.parse(response.getHtml(),response.getUrl());
    }

    @Comment("根据xpath在请求结果中查找")
    @Example("${resp.xpath('//title/text()')}")
    @Return({Element.class, String.class})
    public static String xpath(SpiderResponse response, String xpath) {
        return ExtractUtils.getValueByXPath(element(response), xpath);
    }

    @Comment("根据xpath在请求结果中查找")
    @Example("${resp.xpaths('//a/@href')}")
    public static List<String> xpaths(SpiderResponse response, String xpath) {
        return ExtractUtils.getValuesByXPath(element(response), xpath);
    }

    @Comment("根据正则表达式提取请求结果中的内容")
    @Example("${resp.regx('<title>(.*?)</title>')}")
    public static String regx(SpiderResponse response, String pattern) {
        return ExtractUtils.getFirstMatcher(response.getHtml(), pattern, true);
    }

    @Comment("根据正则表达式提取请求结果中的内容")
    @Example("${resp.regx('<title>(.*?)</title>',1)}")
    public static String regx(SpiderResponse response, String pattern, int groupIndex) {
        return ExtractUtils.getFirstMatcher(response.getHtml(), pattern, groupIndex);
    }

    @Comment("根据正则表达式提取请求结果中的内容")
    @Example("${resp.regx('<a href=\"(.*?)\">(.*?)</a>',[1,2])}")
    public static List<String> regx(SpiderResponse response, String pattern, List<Integer> groups) {
        return ExtractUtils.getFirstMatcher(response.getHtml(), pattern, groups);
    }

    @Comment("根据正则表达式提取请求结果中的内容")
    @Example("${resp.regxs('<h2>(.*?)</h2>')}")
    public static List<String> regxs(SpiderResponse response, String pattern) {
        return ExtractUtils.getMatchers(response.getHtml(), pattern, true);
    }

    @Comment("根据正则表达式提取请求结果中的内容")
    @Example("${resp.regxs('<h2>(.*?)</h2>',1)}")
    public static List<String> regxs(SpiderResponse response, String pattern, int groupIndex) {
        return ExtractUtils.getMatchers(response.getHtml(), pattern, groupIndex);
    }

    @Comment("根据正则表达式提取请求结果中的内容")
    @Example("${resp.regxs('<a href=\"(.*?)\">(.*?)</a>',[1,2])}")
    public static List<List<String>> regxs(SpiderResponse response, String pattern, List<Integer> groups) {
        return ExtractUtils.getMatchers(response.getHtml(), pattern, groups);
    }

    @Comment("根据css选择器提取请求结果")
    @Example("${resp.selector('div > a')}")
    public static Element selector(SpiderResponse response, String selector) {
        return ElementFunctionExtension.selector(element(response), selector);
    }

    @Comment("根据css选择器提取请求结果")
    @Example("${resp.selectors('div > a')}")
    public static Elements selectors(SpiderResponse response, String selector) {
        return ElementFunctionExtension.selectors(element(response), selector);
    }

    @Comment("根据jsonpath提取请求结果")
    @Example("${resp.jsonpath('$.code')}")
    public static Object jsonpath(SpiderResponse response, String path) {
        return ExtractUtils.getValueByJsonPath(response.getJson(), path);
    }

    @Comment("获取页面上的链接")
    @Example("${resp.links()}")
    public static List<String> links(SpiderResponse response) {
        return ExtractUtils.getAttrBySelector(element(response), "a", "abs:href")
                .stream()
                .filter(link -> StringUtils.isNotBlank(link))
                .collect(Collectors.toList());
    }

    @Comment("获取页面上的链接")
    @Example("${resp.links('https://www\\.xxx\\.com/xxxx/(.*?)')}")
    public static List<String> links(SpiderResponse response, String regx) {
        Pattern pattern = Pattern.compile(regx);
        return links(response)
				.stream()
                .filter(link -> pattern.matcher(link).matches())
                .collect(Collectors.toList());
    }

    @Comment("获取当前页面所有图片链接")
    @Example("${resp.images()}")
    public static List<String> images(SpiderResponse response) {
        return ExtractUtils.getAttrBySelector(element(response), "img", "src")
                .stream()
                .filter(link -> StringUtils.isNotBlank(link))
                .collect(Collectors.toList());
    }
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/SqlRowSetExtension.java
================================================
package org.spiderflow.core.executor.function.extension;

import org.apache.commons.lang3.exception.ExceptionUtils;
import org.spiderflow.annotation.Example;
import org.spiderflow.executor.FunctionExtension;
import org.springframework.jdbc.support.rowset.SqlRowSet;
import org.springframework.stereotype.Component;

import java.util.HashMap;
import java.util.Map;

@Component
public class SqlRowSetExtension implements FunctionExtension {
    public static Map<String, String[]> tableMetaMap = new HashMap<>();

    @Override
    public Class<?> support() {
        return SqlRowSet.class;
    }

    @Example("${rs.nextToMap()}")
    public static Map<String, Object> nextToMap(SqlRowSet sqlRowSet) {
        try {
            if (!sqlRowSet.next()) {
                return null;
            }
            String[] columnNames = sqlRowSet.getMetaData().getColumnNames();
            Map<String, Object> result = new HashMap<>();
            for (String columnName : columnNames) {
                result.put(columnName, sqlRowSet.getObject(columnName));
            }
            return result;
        } catch (Exception e) {
            ExceptionUtils.wrapAndThrow(e);
        }
        return null;
    }


}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/StringFunctionExtension.java
================================================
package org.spiderflow.core.executor.function.extension;

import com.alibaba.fastjson.JSON;
import org.apache.commons.lang3.math.NumberUtils;
import org.apache.commons.text.StringEscapeUtils;
import org.jsoup.nodes.Element;
import org.jsoup.parser.Parser;
import org.jsoup.select.Elements;
import org.spiderflow.annotation.Comment;
import org.spiderflow.annotation.Example;
import org.spiderflow.annotation.Return;
import org.spiderflow.core.executor.function.DateFunctionExecutor;
import org.spiderflow.core.utils.ExtractUtils;
import org.spiderflow.executor.FunctionExtension;
import org.springframework.stereotype.Component;

import java.text.ParseException;
import java.util.Date;
import java.util.List;

@Component
public class StringFunctionExtension implements FunctionExtension{

	@Override
	public Class<?> support() {
		return String.class;
	}	
	
	@Comment("根据正则表达式提取String中的内容")
	@Example("${strVar.regx('<title>(.*?)</title>')}")
	public static String regx(String source,String pattern){
		return ExtractUtils.getFirstMatcher(source, pattern, true);
	}
	
	@Comment("根据正则表达式提取String中的内容")
	@Example("${strVar.regx('<title>(.*?)</title>',1)}")
	public static String regx(String source,String pattern,int groupIndex){
		return ExtractUtils.getFirstMatcher(source, pattern, groupIndex);
	}
	
	@Comment("根据正则表达式提取String中的内容")
	@Example("${strVar.regx('<a href=\"(.*?)\">(.*?)</a>',[1,2])}")
	public static List<String> regx(String source,String pattern,List<Integer> groups){
		return ExtractUtils.getFirstMatcher(source, pattern, groups);
	}
	
	@Comment("根据正则表达式提取String中的内容")
	@Example("${strVar.regxs('<h2>(.*?)</h2>')}")
	public static List<String> regxs(String source,String pattern){
		return ExtractUtils.getMatchers(source, pattern, true);
	}
	
	@Comment("根据正则表达式提取String中的内容")
	@Example("${strVar.regxs('<h2>(.*?)</h2>',1)}")
	public static List<String> regxs(String source,String pattern,int groupIndex){
		return ExtractUtils.getMatchers(source, pattern, groupIndex);
	}
	
	@Comment("根据正则表达式提取String中的内容")
	@Example("${strVar.regxs('<a href=\"(.*?)\">(.*?)</a>',[1,2])}")
	public static List<List<String>> regxs(String source,String pattern,List<Integer> groups){
		return ExtractUtils.getMatchers(source, pattern, groups);
	}
	
	@Comment("根据xpath在String变量中查找")
	@Example("${strVar.xpath('//title/text()')}")
	@Return({Element.class,String.class})
	public static String xpath(String source,String xpath){
		return ExtractUtils.getValueByXPath(element(source), xpath);
	}
	
	@Comment("根据xpath在String变量中查找")
	@Example("${strVar.xpaths('//a/@href')}")
	public static List<String> xpaths(String source,String xpath){
		return ExtractUtils.getValuesByXPath(element(source), xpath);
	}
	
	@Comment("将String变量转为Element对象")
	@Example("${strVar.element()}")
	public static Element element(String source){
		return Parser.xmlParser().parseInput(source,"");
	}
	
	@Comment("根据css选择器提取")
	@Example("${strVar.selector('div > a')}")
	public static Element selector(String source,String cssQuery){
		return element(source).selectFirst(cssQuery);
	}
	
	@Comment("根据css选择器提取")
	@Example("${strVar.selector('div > a')}")
	public static Elements selectors(String source,String cssQuery){
		return element(source).select(cssQuery);
	}

	@Comment("将string转为json对象")
	@Example("${strVar.json()}")
	public static Object json(String source){
		return JSON.parse(source);
	}
	
	@Comment("根据jsonpath提取内容")
	@Example("${strVar.jsonpath('$.code')}")
	public static Object jsonpath(String source,String jsonPath){
		return ExtractUtils.getValueByJsonPath(json(source), jsonPath);
	}
	
	@Comment("将字符串转为int类型")
	@Example("${strVar.toInt(0)}")
	public static Integer toInt(String source,int defaultValue){
		return NumberUtils.toInt(source, defaultValue);
	}
	
	@Comment("将字符串转为int类型")
	@Example("${strVar.toInt()}")
	public static Integer toInt(String source){
		return NumberUtils.toInt(source);
	}
	
	@Comment("将字符串转为double类型")
	@Example("${strVar.toDouble()}")
	public static Double toDouble(String source){
		return NumberUtils.toDouble(source);
	}
	
	@Comment("将字符串转为long类型")
	@Example("${strVar.toLong()}")
	public static Long toLong(String source){
		return NumberUtils.toLong(source);
	}
	
	@Comment("将字符串转为date类型")
	@Example("${strVar.toDate('yyyy-MM-dd HH:mm:ss')}")
	public static Date toDate(String source,String pattern) throws ParseException{
		return DateFunctionExecutor.parse(source, pattern);
	}

	@Comment("反转义字符串")
	@Example("${strVar.unescape()}")
	public static String unescape(String source){
		return StringEscapeUtils.unescapeJava(source);
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/CommentExecutor.java
================================================
package org.spiderflow.core.executor.shape;

import org.spiderflow.context.SpiderContext;
import org.spiderflow.executor.ShapeExecutor;
import org.spiderflow.model.SpiderNode;
import org.springframework.stereotype.Component;

import java.util.Map;

@Component
public class CommentExecutor implements ShapeExecutor{

	@Override
	public void execute(SpiderNode node, SpiderContext context, Map<String,Object> variables) {
		
	}

	@Override
	public String supportShape() {
		return "comment";
	}

}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/ExecuteSQLExecutor.java
================================================
package org.spiderflow.core.executor.shape;

import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.exception.ExceptionUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.spiderflow.Grammerable;
import org.spiderflow.context.SpiderContext;
import org.spiderflow.core.utils.DataSourceUtils;
import org.spiderflow.core.utils.ExpressionUtils;
import org.spiderflow.core.utils.ExtractUtils;
import org.spiderflow.executor.ShapeExecutor;
import org.spiderflow.model.Grammer;
import org.spiderflow.model.SpiderNode;
import org.springframework.jdbc.core.ArgumentPreparedStatementSetter;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.support.GeneratedKeyHolder;
import org.springframework.jdbc.support.KeyHolder;
import org.springframework.stereotype.Component;

import java.lang.reflect.Array;
import java.sql.PreparedStatement;
import java.sql.Statement;
import java.util.*;

/**
 * SQL执行器
 *
 * @author jmxd
 */
@Component
public class ExecuteSQLExecutor implements ShapeExecutor, Grammerable {

	public static final String DATASOURCE_ID = "datasourceId";

	public static final String SQL = "sql";

	public static final String STATEMENT_TYPE = "statementType";

	public static final String STATEMENT_SELECT = "select";

	public static final String STATEMENT_SELECT_ONE = "selectOne";

	public static final String STATEMENT_SELECT_INT = "selectInt";

	public static final String STATEMENT_INSERT = "insert";

	public static final String STATEMENT_UPDATE = "update";

	public static final String STATEMENT_DELETE = "delete";
	public static final String SELECT_RESULT_STREAM = "isStream";
	public static final String STATEMENT_INSERT_PK = "insertofPk";
	
	private static final Logger logger = LoggerFactory.getLogger(ExecuteSQLExecutor.class);

	@Override
	public void execute(SpiderNode node, SpiderContext context, Map<String, Object> variables) {
		String dsId = node.getStringJsonValue(DATASOURCE_ID);
		String sql = node.getStringJsonValue(SQL);
		if (StringUtils.isBlank(dsId)) {
			logger.warn("数据源ID为空!");
		} else if (StringUtils.isBlank(sql)) {
			logger.warn("sql为空!");
		} else {
			JdbcTemplate template = new JdbcTemplate(DataSourceUtils.getDataSource(dsId));
			//把变量替换成占位符
			List<String> parameters = ExtractUtils.getMatchers(sql, "#(.*?)#", true);
			sql = sql.replaceAll("#(.*?)#", "?");
			try {
				Object sqlObject = ExpressionUtils.execute(sql, variables);
				if(sqlObject == null){
					logger.warn("获取的sql为空!");
					return;
				}
				sql = sqlObject.toString();
				context.pause(node.getNodeId(),"common",SQL,sql);
			} catch (Exception e) {
				logger.error("获取sql出错,异常信息:{}", e.getMessage(), e);
				ExceptionUtils.wrapAndThrow(e);
			}
			int size = parameters.size();
			Object[] params = new Object[size];
			boolean hasList = false;
			int parameterSize = 0;
			//当参数中存在List或者数组时,认为是批量操作
			for (int i = 0; i < size; i++) {
				Object parameter = ExpressionUtils.execute(parameters.get(i), variables);
				if (parameter != null) {
					if (parameter instanceof List) {
						hasList = true;
						parameterSize = Math.max(parameterSize, ((List<?>) parameter).size());
					} else if (parameter.getClass().isArray()) {
						hasList = true;
						parameterSize = Math.max(parameterSize, Array.getLength(parameter));
					}
				}
				params[i] = parameter;
			}
			String statementType = node.getStringJsonValue(STATEMENT_TYPE);
			logger.debug("执行sql:{}", sql);
			if (STATEMENT_SELECT.equals(statementType)) {
				boolean isStream = "1".equals(node.getStringJsonValue(SELECT_RESULT_STREAM));
				try {
					if (isStream) {
						variables.put("rs", template.queryForRowSet(sql, params));
					} else {
						variables.put("rs", template.queryForList(sql, params));
					}
				} catch (Exception e) {
					variables.put("rs", null);
					logger.error("执行sql出错,异常信息:{}", e.getMessage(), e);
					ExceptionUtils.wrapAndThrow(e);
				}
			} else if (STATEMENT_SELECT_ONE.equals(statementType)) {
				Map<String, Object> rs;
				try {
					rs = template.queryForMap(sql, params);
					variables.put("rs", rs);
				} catch (Exception e) {
					variables.put("rs", null);
					logger.error("执行sql出错,异常信息:{}", e.getMessage(), e);
					ExceptionUtils.wrapAndThrow(e);
				}
			} else if (STATEMENT_SELECT_INT.equals(statementType)) {
				Integer rs;
				try {
					rs = template.queryForObject(sql, params, Integer.class);
					rs = rs == null ? 0 : rs;
					variables.put("rs", rs);
				} catch (Exception e) {
					variables.put("rs", 0);
					logger.error("执行sql出错,异常信息:{}", e.getMessage(), e);
					ExceptionUtils.wrapAndThrow(e);
				}
			} else if (STATEMENT_UPDATE.equals(statementType) || STATEMENT_INSERT.equals(statementType) || STATEMENT_DELETE.equals(statementType)) {
				try {
					int updateCount = 0;
					if (hasList) {
						/*
						  批量操作时,将参数Object[]转化为List<Object[]>
						  当参数不为数组或List时,自动转为Object[]
						  当数组或List长度不足时,自动取最后一项补齐
						 */
						int[] rs = template.batchUpdate(sql, convertParameters(params, parameterSize));
						if (rs.length > 0) {
							updateCount = Arrays.stream(rs).sum();
						}
					} else {
						updateCount = template.update(sql, params);
					}
					variables.put("rs", updateCount);
				} catch (Exception e) {
					logger.error("执行sql出错,异常信息:{}", e.getMessage(), e);
					variables.put("rs", -1);
					ExceptionUtils.wrapAndThrow(e);
				}
			} else if(STATEMENT_INSERT_PK.equals(statementType)) {
				try {
					KeyHolder keyHolder = new GeneratedKeyHolder();
					final String insertSQL = sql;
					template.update(con -> {
						PreparedStatement ps = con.prepareStatement(insertSQL, Statement.RETURN_GENERATED_KEYS);
						new ArgumentPreparedStatementSetter(params).setValues(ps);
						return ps;
					}, keyHolder);
					variables.put("rs", keyHolder.getKey().intValue());
				} catch (Exception e) {
					logger.error("执行sql出错,异常信息:{}", e.getMessage(), e);
					variables.put("rs", -1);
					ExceptionUtils.wrapAndThrow(e);
				}
			}
		}
	}
	private List<Object[]> convertParameters(Object[] params, int length) {
		List<Object[]> result = new ArrayList<>(length);
		int size = params.length;
		for (int i = 0; i < length; i++) {
			Object[] parameters = new Object[size];
			for (int j = 0; j < size; j++) {
				parameters[j] = getValue(params[j], i);
			}
			result.add(parameters);
		}
		return result;
	}

	private Object getValue(Object object, int index) {
		if (object == null) {
			return null;
		} else if (object instanceof List) {
			List<?> list = (List<?>) object;
			int size = list.size();
			if (size > 0) {
				return list.get(Math.min(list.size() - 1, index));
			}
		} else if (object.getClass().isArray()) {
			int size = Array.getLength(object);
			if (size > 0) {
				Array.get(object, Math.min(-1, index));
			}
		} else {
			return object;
		}
		return null;
	}

	@Override
	public String supportShape() {
		return "executeSql";
	}

	@Override
	public List<Grammer> grammers() {
		Grammer grammer = new Grammer();
		grammer.setComment("执行SQL结果");
		grammer.setFunction("rs");
		grammer.setReturns(Arrays.asList("List<Map<String,Object>>", "int"));
		return Collections.singletonList(grammer);
	}


}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/ForkJoinExecutor.java
================================================
package org.spiderflow.core.executor.shape;

import org.spiderflow.context.SpiderContext;
import org.spiderflow.executor.ShapeExecutor;
import org.spiderflow.model.SpiderNode;
import org.springframework.stereotype.Component;

import java.util.HashMap;
import java.util.Map;

/**
 * 等待执行结束执行器
 * 
 */
@Component
public class ForkJoinExecutor implements ShapeExecutor {

	/**
	 * 缓存已完成节点的变量
	 */
	private Map<String, Map<String, Object>> cachedVariables = new HashMap<>();
	
	@Override
	public void execute(SpiderNode node, SpiderContext context, Map<String, Object> variables) {
	}

	@Override
	public String supportShape() {
		return "forkJoin";
	}

	@Override
	public boolean allowExecuteNext(SpiderNode node, SpiderContext context, Map<String, Object> variables) {
		String key = context.getId() + "-" + node.getNodeId();
		synchronized (node){
			boolean isDone = node.isDone();
			Map<String, Object> cached = cachedVariables.get(key);
			if(!isDone){
				if(cached == null){
					cached = new HashMap<>();
					cachedVariables.put(key, cached);
				}
				cached.putAll(variables);
			}else if(cached != null){
				//将缓存的变量存入到当前变量中,传递给下一级
				variables.putAll(cached);
				cachedVariables.remove(key);
			}
			return isDone;
		}
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/FunctionExecutor.java
================================================
package org.spiderflow.core.executor.shape;

import java.util.List;
import java.util.Map;

import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.exception.ExceptionUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.spiderflow.context.SpiderContext;
import org.spiderflow.core.utils.ExpressionUtils;
import org.spiderflow.executor.ShapeExecutor;
import org.spiderflow.model.SpiderNode;
import org.springframework.stereotype.Component;

/**
 * 函数执行器
 * @author Administrator
 *
 */
@Component
public class FunctionExecutor implements ShapeExecutor{
	
	public static final String FUNCTION = "function";

	private static final Logger logger = LoggerFactory.getLogger(FunctionExecutor.class);
	
	@Override
	public void execute(SpiderNode node, SpiderContext context, Map<String,Object> variables) {
		List<Map<String, String>> functions = node.getListJsonValue(FUNCTION);
		for (Map<String, String> item : functions) {
			String function = item.get(FUNCTION);
			if(StringUtils.isNotBlank(function)){
				try {
					logger.debug("执行函数{}",function);
					ExpressionUtils.execute(function, variables);
				} catch (Exception e) {
					logger.error("执行函数{}失败,异常信息:{}",function,e);
					ExceptionUtils.wrapAndThrow(e);
				}
			}
		}
	}

	@Override
	public String supportShape() {
		return "function";
	}

}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/LoopExecutor.java
================================================
package org.spiderflow.core.executor.shape;

import java.util.Map;

import org.spiderflow.context.SpiderContext;
import org.spiderflow.executor.ShapeExecutor;
import org.spiderflow.model.SpiderNode;
import org.springframework.stereotype.Component;

/**
 * 循环执行器
 * @author Administrator
 *
 */
@Component
public class LoopExecutor implements ShapeExecutor{
	
	public static final String LOOP_ITEM = "loopItem";
	
	public static final String LOOP_START = "loopStart";

	public static final String LOOP_END = "loopEnd";
	
	@Override
	public void execute(SpiderNode node, SpiderContext context, Map<String,Object> variables) {
	}

	@Override
	public String supportShape() {
		return "loop";
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/OutputExecutor.java
================================================
package org.spiderflow.core.executor.shape;

import com.alibaba.fastjson.JSON;
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVPrinter;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.exception.ExceptionUtils;
import org.apache.ibatis.jdbc.SQL;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.spiderflow.context.SpiderContext;
import org.spiderflow.core.serializer.FastJsonSerializer;
import org.spiderflow.core.utils.DataSourceUtils;
import org.spiderflow.core.utils.ExpressionUtils;
import org.spiderflow.executor.ShapeExecutor;
import org.spiderflow.io.SpiderResponse;
import org.spiderflow.listener.SpiderListener;
import org.spiderflow.model.SpiderNode;
import org.spiderflow.model.SpiderOutput;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Component;

import java.io.*;
import java.util.*;

/**
 * 输出执行器
 * @author Administrator
 *
 */
@Component
public class OutputExecutor implements ShapeExecutor, SpiderListener {

	public static final String OUTPUT_ALL = "output-all";

	public static final String OUTPUT_NAME = "output-name";

	public static final String OUTPUT_VALUE = "output-value";

	public static final String DATASOURCE_ID = "datasourceId";

	public static final String OUTPUT_DATABASE = "output-database";

	public static final String OUTPUT_CSV = "output-csv";

	public static final String TABLE_NAME = "tableName";

	public static final String CSV_NAME = "csvName";

	public static final String CSV_ENCODING = "csvEncoding";

	private static Logger logger = LoggerFactory.getLogger(OutputExecutor.class);

	/**
	 * 输出CSVPrinter节点变量
	 */
	private Map<String, CSVPrinter> cachePrinter = new HashMap<>();

	@Override
	public void execute(SpiderNode node, SpiderContext context, Map<String,Object> variables) {
		SpiderOutput output = new SpiderOutput();
		output.setNodeName(node.getNodeName());
		output.setNodeId(node.getNodeId());
		boolean outputAll = "1".equals(node.getStringJsonValue(OUTPUT_ALL));
		boolean databaseFlag = "1".equals(node.getStringJsonValue(OUTPUT_DATABASE));
		boolean csvFlag = "1".equals(node.getStringJsonValue(OUTPUT_CSV));
		if (outputAll) {
			outputAll(output, variables);
		}
		List<Map<String, String>> outputs = node.getListJsonValue(OUTPUT_NAME, OUTPUT_VALUE);
		Map<String, Object> outputData = null;
		if (databaseFlag || csvFlag) {
			outputData = new HashMap<>(outputs.size());
		}
		for (Map<String, String> item : outputs) {
			Object value = null;
			String outputValue = item.get(OUTPUT_VALUE);
			String outputName = item.get(OUTPUT_NAME);
			try {
				value = ExpressionUtils.execute(outputValue, variables);
				context.pause(node.getNodeId(),"common",outputName,value);
				logger.debug("输出{}={}", outputName,value);
			} catch (Exception e) {
				logger.error("输出{}出错,异常信息:{}", outputName,e);
			}
			output.addOutput(outputName, value);
			if ((databaseFlag || csvFlag) && value != null) {
				outputData.put(outputName, value.toString());
			}
		}
		if(databaseFlag){
			String dsId = node.getStringJsonValue(DATASOURCE_ID);
			String tableName = node.getStringJsonValue(TABLE_NAME);
			if (StringUtils.isBlank(dsId)) {
				logger.warn("数据源ID为空!");
			} else if (StringUtils.isBlank(tableName)) {
				logger.warn("表名为空!");
			} else {
				outputDB(dsId, tableName, outputData);
			}
		}
		if (csvFlag) {
			String csvName = node.getStringJsonValue(CSV_NAME);
			outputCSV(node, context, csvName, outputData);
		}
		context.addOutput(output);
	}

	/**
	 * 输出所有参数
	 * @param output
	 * @param variables
	 */
	private void outputAll(SpiderOutput output,Map<String,Object> variables){
		for (Map.Entry<String, Object> item : variables.entrySet()) {
			Object value = item.getValue();
			if (value instanceof SpiderResponse) {
				SpiderResponse resp = (SpiderResponse) value;
				output.addOutput(item.getKey() + ".html", resp.getHtml());
				continue;
			}
			//去除不输出的信息
			if ("ex".equals(item.getKey())) {
				continue;
			}
			//去除不能序列化的参数
			try {
				JSON.toJSONString(value, FastJsonSerializer.serializeConfig);
			} catch (Exception e) {
				e.printStackTrace();
				continue;
			}
			//输出信息
			output.addOutput(item.getKey(), item.getValue());
		}
	}

	private void outputDB(String databaseId, String tableName, Map<String, Object> data) {
		if (data == null || data.isEmpty()) {
			return;
		}
		JdbcTemplate template = new JdbcTemplate(DataSourceUtils.getDataSource(databaseId));
		Set<String> keySet = data.keySet();
		Object[] params = new Object[data.size()];
		SQL sql = new SQL();
		//设置表名
		sql.INSERT_INTO(tableName);
		int index = 0;
		//设置字段名
		for (String key : keySet) {
			sql.VALUES(key, "?");
			params[index] = data.get(key);
			index++;
		}
		try {
			//执行sql
			template.update(sql.toString(), params);
		} catch (Exception e) {
			logger.error("执行sql出错,异常信息:{}", e.getMessage(), e);
			ExceptionUtils.wrapAndThrow(e);
		}
	}

	private void outputCSV(SpiderNode node, SpiderContext context, String csvName, Map<String, Object> data) {
		if (data == null || data.isEmpty()) {
			return;
		}
		String key = context.getId() + "-" + node.getNodeId();
		CSVPrinter printer = cachePrinter.get(key);
		List<String> records = new ArrayList<>(data.size());
		String[] headers = data.keySet().toArray(new String[data.size()]);
		try {
			if (printer == null) {
				synchronized (cachePrinter) {
					printer = cachePrinter.get(key);
					if (printer == null) {
						CSVFormat format = CSVFormat.DEFAULT.withHeader(headers);
						FileOutputStream os = new FileOutputStream(csvName);
						String csvEncoding = node.getStringJsonValue(CSV_ENCODING);
						if ("UTF-8BOM".equals(csvEncoding)) {
							csvEncoding = csvEncoding.substring(0, 5);
							byte[] bom = {(byte) 0xEF, (byte) 0xBB, (byte) 0xBF};
							os.write(bom);
							os.flush();
						}
						OutputStreamWriter osw = new OutputStreamWriter(os, csvEncoding);
						printer = new CSVPrinter(osw, format);
						cachePrinter.put(key, printer);
					}
				}
			}
			for (int i = 0; i < headers.length; i++) {
				records.add(data.get(headers[i]).toString());
			}
			synchronized (cachePrinter) {
				printer.printRecord(records);
			}
		} catch (IOException e) {
			logger.error("文件输出错误,异常信息:{}", e.getMessage(), e);
			ExceptionUtils.wrapAndThrow(e);
		}
	}

	@Override
	public String supportShape() {
		return "output";
	}

	@Override
	public void beforeStart(SpiderContext context) {

	}

	@Override
	public void afterEnd(SpiderContext context) {
		this.releasePrinters();
	}

	private void releasePrinters() {
		for (Iterator<Map.Entry<String, CSVPrinter>> iterator = this.cachePrinter.entrySet().iterator(); iterator.hasNext(); ) {
			Map.Entry<String, CSVPrinter> entry = iterator.next();
			CSVPrinter printer = entry.getValue();
			if (printer != null) {
				try {
					printer.flush();
					printer.close();
					this.cachePrinter.remove(entry.getKey());
				} catch (IOException e) {
					logger.error("文件输出错误,异常信息:{}", e.getMessage(), e);
					ExceptionUtils.wrapAndThrow(e);
				}
			}
		}
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/ProcessExecutor.java
================================================
package org.spiderflow.core.executor.shape;

import java.util.Map;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.spiderflow.context.SpiderContext;
import org.spiderflow.core.Spider;
import org.spiderflow.core.model.SpiderFlow;
import org.spiderflow.core.service.SpiderFlowService;
import org.spiderflow.core.utils.SpiderFlowUtils;
import org.spiderflow.executor.ShapeExecutor;
import org.spiderflow.model.SpiderNode;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

/**
 * 子流程执行器
 * @author Administrator
 *
 */
@Component
public class ProcessExecutor implements ShapeExecutor{
	
	public static final String FLOW_ID = "flowId";

	private static Logger logger = LoggerFactory.getLogger(ProcessExecutor.class);
	
	@Autowired
	private SpiderFlowService spiderFlowService;
	
	@Autowired
	private Spider spider;
	
	@Override
	public void execute(SpiderNode node, SpiderContext context, Map<String,Object> variables) {
		String flowId = node.getStringJsonValue("flowId");
		SpiderFlow spiderFlow = spiderFlowService.getById(flowId);
		if(spiderFlow != null){
			logger.info("执行子流程:{}", spiderFlow.getName());
			SpiderNode root = SpiderFlowUtils.loadXMLFromString(spiderFlow.getXml());
			spider.executeNode(null,root,context,variables);
		}else{
			logger.info("执行子流程:{}失败,找不到该子流程", flowId);
		}
	}

	@Override
	public String supportShape() {
		return "process";
	}

}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/RequestExecutor.java
================================================
package org.spiderflow.core.executor.shape;

import com.google.common.hash.BloomFilter;
import com.google.common.hash.Funnel;
import com.google.common.hash.Funnels;
import org.apache.commons.io.FileUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.exception.ExceptionUtils;
import org.apache.commons.lang3.math.NumberUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.spiderflow.Grammerable;
import org.spiderflow.context.CookieContext;
import org.spiderflow.context.SpiderContext;
import org.spiderflow.core.executor.function.MD5FunctionExecutor;
import org.spiderflow.core.io.HttpRequest;
import org.spiderflow.core.io.HttpResponse;
import org.spiderflow.core.utils.ExpressionUtils;
import org.spiderflow.executor.ShapeExecutor;
import org.spiderflow.io.SpiderResponse;
import org.spiderflow.listener.SpiderListener;
import org.spiderflow.model.Grammer;
import org.spiderflow.model.SpiderNode;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;
import java.io.*;
import java.nio.charset.Charset;
import java.util.*;

/**
 * 请求执行器
 * @author Administrator
 *
 */
@Component
public class RequestExecutor implements ShapeExecutor,Grammerable, SpiderListener {
	
	public static final String SLEEP = "sleep";
	
	public static final String URL = "url";
	
	public static final String PROXY = "proxy";
	
	public static final String REQUEST_METHOD = "method";
	
	public static final String PARAMETER_NAME = "parameter-name";
	
	public static final String PARAMETER_VALUE = "parameter-value";

	public static final String COOKIE_NAME = "cookie-name";

	public static final String COOKIE_VALUE = "cookie-value";

	public static final String PARAMETER_FORM_NAME = "parameter-form-name";
	
	public static final String PARAMETER_FORM_VALUE = "parameter-form-value";
	
	public static final String PARAMETER_FORM_FILENAME = "parameter-form-filename";
	
	public static final String PARAMETER_FORM_TYPE = "parameter-form-type";
	
	public static final String BODY_TYPE = "body-type";
	
	public static final String BODY_CONTENT_TYPE = "body-content-type";
	
	public static final String REQUEST_BODY = "request-body";
	
	public static final String HEADER_NAME = "header-name";
	
	public static final String HEADER_VALUE = "header-value";
	
	public static final String TIMEOUT = "timeout";

	public static final String RETRY_COUNT = "retryCount";

	public static final String RETRY_INTERVAL = "retryInterval";

	public static final String RESPONSE_CHARSET = "response-charset";
	
	public static final String FOLLOW_REDIRECT = "follow-redirect";
	
	public static final String TLS_VALIDATE = "tls-validate";

	public static final String LAST_EXECUTE_TIME = "__last_execute_time_";

	public static final String COOKIE_AUTO_SET = "cookie-auto-set";

	public static final String REPEAT_ENABLE = "repeat-enable";

	public static final String BLOOM_FILTER_KEY = "_bloomfilter";

	@Value("${spider.workspace}")
	private String workspcace;

	@Value("${spider.bloomfilter.capacity:5000000}")
	private Integer capacity;

	@Value("${spider.bloomfilter.error-rate:0.00001}")
	private Double errorRate;

	private static final Logger logger = LoggerFactory.getLogger(RequestExecutor.class);
	
	@Override
	public String supportShape() {
		return "request";
	}

	@PostConstruct
	void init(){
		//允许设置被限制的请求头
		System.setProperty("sun.net.http.allowRestrictedHeaders", "true");
	}

	@Override
	public void execute(SpiderNode node, SpiderContext context, Map<String,Object> variables) {
		CookieContext cookieContext = context.getCookieContext();
		String sleepCondition = node.getStringJsonValue(SLEEP);
		if(StringUtils.isNotBlank(sleepCondition)){
			try {
				Object value = ExpressionUtils.execute(sleepCondition, variables);
				if(value != null){
					long sleepTime = NumberUtils.toLong(value.toString(), 0L);
					synchronized (node.getNodeId().intern()) {
						//实际等待时间 = 上次执行时间 + 睡眠时间 - 当前时间
						Long lastExecuteTime = context.get(LAST_EXECUTE_TIME + node.getNodeId(), 0L);
						if (lastExecuteTime != 0) {
							sleepTime = lastExecuteTime + sleepTime - System.currentTimeMillis();
						}
						if (sleepTime > 0) {
							context.pause(node.getNodeId(),"common",SLEEP,sleepTime);
							logger.debug("设置延迟时间:{}ms", sleepTime);
							Thread.sleep(sleepTime);
						}
						context.put(LAST_EXECUTE_TIME + node.getNodeId(), System.currentTimeMillis());
					}
				}
			} catch (Throwable t) {
				logger.error("设置延迟时间失败", t);
			}
		}
		BloomFilter<String> bloomFilter = null;
		//重试次数
		int retryCount = NumberUtils.toInt(node.getStringJsonValue(RETRY_COUNT), 0) + 1;
		//重试间隔时间,单位毫秒
		int retryInterval = NumberUtils.toInt(node.getStringJsonValue(RETRY_INTERVAL), 0);
        boolean successed = false;
		for (int i = 0; i < retryCount && !successed; i++) {
			HttpRequest request = HttpRequest.create();
			//设置请求url
			String url = null;
			try {
				url = ExpressionUtils.execute(node.getStringJsonValue(URL), variables).toString();
			} catch (Exception e) {
				logger.error("设置请求url出错,异常信息", e);
				ExceptionUtils.wrapAndThrow(e);
			}
			if("1".equalsIgnoreCase(node.getStringJsonValue(REPEAT_ENABLE,"0"))){
				bloomFilter = createBloomFilter(context);
				synchronized (bloomFilter){
					if(bloomFilter.mightContain(MD5FunctionExecutor.string(url))){
						logger.info("过滤重复URL:{}",url);
						return;
					}
				}
			}
			context.pause(node.getNodeId(),"common",URL,url);
			logger.info("设置请求url:{}", url);
			request.url(url);
			//设置请求超时时间
			int timeout = NumberUtils.toInt(node.getStringJsonValue(TIMEOUT), 60000);
			logger.debug("设置请求超时时间:{}", timeout);
			request.timeout(timeout);

			String method = Objects.toString(node.getStringJsonValue(REQUEST_METHOD), "GET");
			//设置请求方法
			request.method(method);
			logger.debug("设置请求方法:{}", method);

			//是否跟随重定向
			boolean followRedirects = !"0".equals(node.getStringJsonValue(FOLLOW_REDIRECT));
			request.followRedirect(followRedirects);
			logger.debug("设置跟随重定向:{}", followRedirects);

			//是否验证TLS证书,默认是验证
			if("0".equals(node.getStringJsonValue(TLS_VALIDATE))){
				request.validateTLSCertificates(false);
				logger.debug("设置TLS证书验证:{}", false);
			}
			SpiderNode root = context.getRootNode();
			//设置请求header
			setRequestHeader(root, request, root.getListJsonValue(HEADER_NAME,HEADER_VALUE), context, variables);
			setRequestHeader(node, request, node.getListJsonValue(HEADER_NAME,HEADER_VALUE), context, variables);

			//设置全局Cookie
			Map<String, String> cookies = getRequestCookie(root, root.getListJsonValue(COOKIE_NAME, COOKIE_VALUE), context, variables);
			if(!cookies.isEmpty()){
				logger.info("设置全局Cookie:{}", cookies);
				request.cookies(cookies);
			}
			//设置自动管理的Cookie
			boolean cookieAutoSet = !"0".equals(node.getStringJsonValue(COOKIE_AUTO_SET));
			if(cookieAutoSet && !cookieContext.isEmpty()){
				context.pause(node.getNodeId(),COOKIE_AUTO_SET,COOKIE_AUTO_SET,cookieContext);
				request.cookies(cookieContext);
				logger.info("自动设置Cookie:{}", cookieContext);
			}
			//设置本节点Cookie
			cookies = getRequestCookie(node, node.getListJsonValue(COOKIE_NAME, COOKIE_VALUE), context, variables);
			if(!cookies.isEmpty()){
				request.cookies(cookies);
				logger.debug("设置Cookie:{}", cookies);
			}
			if(cookieAutoSet){
				cookieContext.putAll(cookies);
			}

			String bodyType = node.getStringJsonValue(BODY_TYPE);
			List<InputStream> streams = null;
			if("raw".equals(bodyType)){
				String contentType = node.getStringJsonValue(BODY_CONTENT_TYPE);
				request.contentType(contentType);
				try {
					Object requestBody = ExpressionUtils.execute(node.getStringJsonValue(REQUEST_BODY), variables);
					context.pause(node.getNodeId(),"request-body",REQUEST_BODY,requestBody);
					request.data(requestBody);
					logger.info("设置请求Body:{}", requestBody);
				} catch (Exception e) {
					logger.debug("设置请求Body出错", e);
				}
			}else if("form-data".equals(bodyType)){
				List<Map<String, String>> formParameters = node.getListJsonValue(PARAMETER_FORM_NAME,PARAMETER_FORM_VALUE,PARAMETER_FORM_TYPE,PARAMETER_FORM_FILENAME);
				streams = setRequestFormParameter(node,request,formParameters,context,variables);
			}else{
				//设置请求参数
				setRequestParameter(root, request, root.getListJsonValue(PARAMETER_NAME,PARAMETER_VALUE), context, variables);
				setRequestParameter(node, request, node.getListJsonValue(PARAMETER_NAME,PARAMETER_VALUE), context, variables);
			}
			//设置代理
			String proxy = node.getStringJsonValue(PROXY);
			if(StringUtils.isNotBlank(proxy)){
				try {
					Object value = ExpressionUtils.execute(proxy, variables);
					context.pause(node.getNodeId(),"common",PROXY,value);
					if(value != null){
						String[] proxyArr = value.toString().split(":");
						if(proxyArr.length == 2){
							request.proxy(proxyArr[0], Integer.parseInt(proxyArr[1]));
							logger.info("设置代理:{}",proxy);
						}
					}
				} catch (Exception e) {
					logger.error("设置代理出错,异常信息:{}",e);
				}
			}
			Throwable exception = null;
			try {
				HttpResponse response = request.execute();
                successed = response.getStatusCode() == 200;
                if(successed){
                	if(bloomFilter != null){
                		synchronized (bloomFilter){
							bloomFilter.put(MD5FunctionExecutor.string(url));
						}
					}
                    String charset = node.getStringJsonValue(RESPONSE_CHARSET);
                    if(StringUtils.isNotBlank(charset)){
                        response.setCharset(charset);
                        logger.debug("设置response charset:{}",charset);
                    }
                    //cookie存入cookieContext
                    cookieContext.putAll(response.getCookies());
                    //结果存入变量
                    variables.put("resp", response);
                }
			} catch (IOException e) {
				successed = false;
                exception = e;
			} finally{
                if(streams != null){
                    for (InputStream is : streams) {
                        try {
                            is.close();
                        } catch (Exception e) {
                        }
                    }
                }
                if(!successed){
                    if(i + 1 < retryCount){
                        if(retryInterval > 0){
                            try {
                                Thread.sleep(retryInterval);
                            } catch (InterruptedException ignored) {
                            }
                        }
                        logger.info("第{}次重试:{}",i + 1,url);
                    }else{
                        //记录访问失败的日志
						if(context.getFlowId() != null){ //测试环境
							//TODO 需增加记录请求参数
							File file = new File(workspcace, context.getFlowId() + File.separator + "logs" + File.separator + "access_error.log");
							try {
								File directory = file.getParentFile();
								if(!directory.exists()){
									directory.mkdirs();
								}
								FileUtils.write(file,url + "\r\n","UTF-8",true);
							} catch (IOException ignored) {
							}
						}
                        logger.error("请求{}出错,异常信息:{}",url,exception);
                    }
                }
			}
		}
	}
	
	private List<InputStream> setRequestFormParameter(SpiderNode node, HttpRequest request,List<Map<String, String>> parameters,SpiderContext context,Map<String,Object> variables){
		List<InputStream> streams = new ArrayList<>();
		if(parameters != null){
			for (Map<String,String> nameValue : parameters) {
				Object value;
				String parameterName = nameValue.get(PARAMETER_FORM_NAME);
				if(StringUtils.isNotBlank(parameterName)){
					String parameterValue = nameValue.get(PARAMETER_FORM_VALUE);
					String parameterType = nameValue.get(PARAMETER_FORM_TYPE);
					String parameterFilename = nameValue.get(PARAMETER_FORM_FILENAME);
					boolean hasFile = "file".equals(parameterType);
					try {
						value = ExpressionUtils.execute(parameterValue, variables);
						if(hasFile){
							InputStream stream = null;
							if(value instanceof byte[]){
								stream = new ByteArrayInputStream((byte[]) value);
							}else if(value instanceof String){
								stream = new ByteArrayInputStream(((String)value).getBytes());
							}else if(value instanceof InputStream){
								stream = (InputStream) value;
							}
							if(stream != null){
								streams.add(stream);
								request.data(parameterName, parameterFilename, stream);
								context.pause(node.getNodeId(),"request-body",parameterName,parameterFilename);
								logger.info("设置请求参数:{}={}",parameterName,parameterFilename);
							}else{
								logger.warn("设置请求参数:{}失败,无二进制内容",parameterName);
							}
						}else{
							request.data(parameterName, value);
							context.pause(node.getNodeId(),"request-body",parameterName,value);
							logger.info("设置请求参数:{}={}",parameterName,value);
						}
						
					} catch (Exception e) {
						logger.error("设置请求参数:{}出错,异常信息:{}",parameterName,e);
					}
				}
			}
		}
		return streams;
	}

	private Map<String,String> getRequestCookie(SpiderNode node, List<Map<String, String>> cookies, SpiderContext context, Map<String, Object> variables) {
		Map<String,String> cookieMap = new HashMap<>();
		if (cookies != null) {
			for (Map<String, String> nameValue : cookies) {
				Object value;
				String cookieName = nameValue.get(COOKIE_NAME);
				if (StringUtils.isNotBlank(cookieName)) {
					String cookieValue = nameValue.get(COOKIE_VALUE);
					try {
						value = ExpressionUtils.execute(cookieValue, variables);
						if (value != null) {
							cookieMap.put(cookieName, value.toString());
							context.pause(node.getNodeId(),"request-cookie",cookieName,value.toString());
							logger.info("设置请求Cookie:{}={}", cookieName, value);
						}
					} catch (Exception e) {
						logger.error("设置请求Cookie:{}出错,异常信息:{}", cookieName, e);
					}
				}
			}
		}
		return cookieMap;
	}

	private void setRequestParameter(SpiderNode node, HttpRequest request, List<Map<String, String>> parameters, SpiderContext context, Map<String, Object> variables) {
		if (parameters != null) {
			for (Map<String, String> nameValue : parameters) {
				Object value = null;
				String parameterName = nameValue.get(PARAMETER_NAME);
				if (StringUtils.isNotBlank(parameterName)) {
					String parameterValue = nameValue.get(PARAMETER_VALUE);
					try {
						value = ExpressionUtils.execute(parameterValue, variables);
						context.pause(node.getNodeId(),"request-parameter",parameterName,value);
						logger.info("设置请求参数:{}={}", parameterName, value);
					} catch (Exception e) {
						logger.error("设置请求参数:{}出错,异常信息:{}", parameterName, e);
					}
					request.data(parameterName, value);
				}
			}
		}
	}

	private void setRequestHeader(SpiderNode node,HttpRequest request, List<Map<String, String>> headers, SpiderContext context, Map<String, Object> variables) {
		if (headers != null) {
			for (Map<String, String> nameValue : headers) {
				Object value = null;
				String headerName = nameValue.get(HEADER_NAME);
				if (StringUtils.isNotBlank(headerName)) {
					String headerValue = nameValue.get(HEADER_VALUE);
					try {
						value = ExpressionUtils.execute(headerValue, variables);
						context.pause(node.getNodeId(),"request-header",headerName,value);
						logger.info("设置请求Header:{}={}", headerName, value);
					} catch (Exception e) {
						logger.error("设置请求Header:{}出错,异常信息:{}", headerName, e);
					}
					request.header(headerName, value);
				}
			}
		}
	}

	@Override
	public List<Grammer> grammers() {
		List<Grammer> grammers = Grammer.findGrammers(SpiderResponse.class,"resp" , "SpiderResponse", false);
		Grammer grammer = new Grammer();
		grammer.setFunction("resp");
		grammer.setComment("抓取结果");
		grammer.setOwner("SpiderResponse");
		grammers.add(grammer);
		return grammers;
	}

	@Override
	public void beforeStart(SpiderContext context) {

	}

	private BloomFilter<String> createBloomFilter(SpiderContext context){
		BloomFilter<String> filter = context.get(BLOOM_FILTER_KEY);
		if(filter == null){
			Funnel<CharSequence> funnel = Funnels.stringFunnel(Charset.forName("UTF-8"));
			String fileName = context.getFlowId() + File.separator + "url.bf";
			File file = new File(workspcace,fileName);
			if(file.exists()){
				try(FileInputStream fis = new FileInputStream(file)){
					filter = BloomFilter.readFrom(fis,funnel);
				} catch (IOException e) {
					logger.error("读取布隆过滤器出错",e);
				}

			}else{
				filter = BloomFilter.create(funnel,capacity,errorRate);
			}
			context.put(BLOOM_FILTER_KEY,filter);
		}
		return filter;
	}

	@Override
	public void afterEnd(SpiderContext context) {
		BloomFilter<String> filter = context.get(BLOOM_FILTER_KEY);
		if(filter != null){
			File file = new File(workspcace,context.getFlowId() + File.separator + "url.bf");
			if(!file.getParentFile().exists()){
				file.getParentFile().mkdirs();
			}
			try(FileOutputStream fos = new FileOutputStream(file)){
				filter.writeTo(fos);
				fos.flush();
			}catch(IOException e){
				logger.error("保存布隆过滤器出错",e);
			}
		}
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/StartExecutor.java
================================================
package org.spiderflow.core.executor.shape;

import java.util.Map;

import org.spiderflow.context.SpiderContext;
import org.spiderflow.executor.ShapeExecutor;
import org.spiderflow.model.SpiderNode;
import org.springframework.stereotype.Component;

/**
 * 开始执行器
 * @author Administrator
 *
 */
@Component
public class StartExecutor implements ShapeExecutor{

	@Override
	public void execute(SpiderNode node, SpiderContext context, Map<String,Object> variables) {
		
	}

	@Override
	public String supportShape() {
		return "start";
	}

}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/VariableExecutor.java
================================================
package org.spiderflow.core.executor.shape;

import java.util.List;
import java.util.Map;

import org.apache.commons.lang3.exception.ExceptionUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.spiderflow.context.SpiderContext;
import org.spiderflow.core.utils.ExpressionUtils;
import org.spiderflow.executor.ShapeExecutor;
import org.spiderflow.model.SpiderNode;
import org.springframework.stereotype.Component;

/**
 * 定义变量执行器
 * @author Administrator
 *
 */
@Component
public class VariableExecutor implements ShapeExecutor{
	
	private static final String VARIABLE_NAME = "variable-name";
	
	private static final String VARIABLE_VALUE = "variable-value";

	private static final Logger logger = LoggerFactory.getLogger(VariableExecutor.class);
	
	@Override
	public void execute(SpiderNode node, SpiderContext context, Map<String,Object> variables) {
		List<Map<String, String>> variableList = node.getListJsonValue(VARIABLE_NAME,VARIABLE_VALUE);
		for (Map<String, String> nameValue : variableList) {
			Object value = null;
			String variableName = nameValue.get(VARIABLE_NAME);
			String variableValue = nameValue.get(VARIABLE_VALUE);
			try {
				value = ExpressionUtils.execute(variableValue, variables);
				logger.debug("设置变量{}={}",variableName,value);
				context.pause(node.getNodeId(),"common",variableName,value);
			} catch (Exception e) {
				logger.error("设置变量{}出错,异常信息:{}",variableName,e);
				ExceptionUtils.wrapAndThrow(e);
			}
			variables.put(variableName, value);
		}
	}

	@Override
	public String supportShape() {
		return "variable";
	}
	
	@Override
	public boolean isThread() {
		return false;
	}

}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/DefaultExpressionEngine.java
================================================
package org.spiderflow.core.expression;

import java.util.List;
import java.util.Map;

import javax.annotation.PostConstruct;

import org.apache.commons.lang3.StringUtils;
import org.spiderflow.ExpressionEngine;
import org.spiderflow.core.expression.interpreter.Reflection;
import org.spiderflow.executor.FunctionExecutor;
import org.spiderflow.executor.FunctionExtension;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

@Component
public class DefaultExpressionEngine implements ExpressionEngine{
	
	@Autowired
	private List<FunctionExecutor> functionExecutors;
	
	@Autowired
	private List<FunctionExtension> functionExtensions;
	
	@PostConstruct
	private void init(){
		for (FunctionExtension extension : functionExtensions) {
			Reflection.getInstance().registerExtensionClass(extension.support(), extension.getClass());
		}
	}
	
	@Override
	public Object execute(String expression, Map<String, Object> variables) {
		if(StringUtils.isBlank(expression)){
			return expression;
		}
		ExpressionTemplateContext context = new ExpressionTemplateContext(variables);
		for (FunctionExecutor executor : functionExecutors) {
			context.set(executor.getFunctionPrefix(), executor);
		}
		ExpressionGlobalVariables.getVariables().entrySet().forEach(entry->{
			context.set(entry.getKey(),ExpressionTemplate.create(entry.getValue()).render(context));
		});
		try {
			ExpressionTemplateContext.set(context);
			return ExpressionTemplate.create(expression).render(context);
		} finally {
			ExpressionTemplateContext.remove();
		}
	}
	
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/ExpressionError.java
================================================

package org.spiderflow.core.expression;

import org.spiderflow.core.expression.parsing.Span;
import org.spiderflow.core.expression.parsing.Span.Line;
import org.spiderflow.core.expression.parsing.TokenStream;

/** All errors reported by the library go through the static functions of this class. */
public class ExpressionError {

	/**
	 * <p>
	 * Create an error message based on the provided message and stream, highlighting the line on which the error happened. If the
	 * stream has more tokens, the next token will be highlighted. Otherwise the end of the source of the stream will be
	 * highlighted.
	 * </p>
	 *
	 * <p>
	 * Throws a {@link RuntimeException}
	 * </p>
	 */
	public static void error (String message, TokenStream stream) {
		if (stream.hasMore())
			error(message, stream.consume().getSpan());
		else {
			String source = stream.getSource();
			if (source == null)
				error(message, new Span(" ", 0, 1));
			else
				error(message, new Span(source, source.length() - 1, source.length()));
		}
	}

	/** Create an error message based on the provided message and location, highlighting the location in the line on which the
	 * error happened. Throws a {@link TemplateException} **/
	public static void error (String message, Span location, Throwable cause) {

		Line line = location.getLine();
		message = "Error (" + line.getLineNumber() + "): " + message + "\n\n";
		message += line.getText();
		message += "\n";

		int errorStart = location.getStart() - line.getStart();
		int errorEnd = errorStart + location.getText().length() - 1;
		for (int i = 0, n = line.getText().length(); i < n; i++) {
			boolean useTab = line.getText().charAt(i) == '\t';
			message += i >= errorStart && i <= errorEnd ? "^" : useTab ? "\t" : " ";
		}

		if (cause == null)
			throw new TemplateException(message, location);
		else
			throw new TemplateException(message, location, cause);
	}

	/** Create an error message based on the provided message and location, highlighting the location in the line on which the
	 * error happened. Throws a {@link TemplateException} **/
	public static void error (String message, Span location) {
		error(message, location, null);
	}

	/** Exception thrown by all basis-template code via {@link ExpressionError#error(String, Span)}. In case an error happens deep inside a
	 * list of included templates, the {@link #getMessage()} method will return a condensed error message. **/
	public static class TemplateException extends RuntimeException {
		private static final long serialVersionUID = 1L;
		private final Span location;
		private final String errorMessage;

		private TemplateException (String message, Span location) {
			super(message);
			this.errorMessage = message;
			this.location = location;
		}

		public TemplateException (String message, Span location, Throwable cause) {
			super(message, cause);
			this.errorMessage = message;
			this.location = location;
		}

		/** Returns the location in the template at which the error happened. **/
		public Span getLocation () {
			return location;
		}

		@Override
		public String getMessage () {
			StringBuilder builder = new StringBuilder();

			if (getCause() == null || getCause() == this) {
				return super.getMessage();
			}

			builder.append(errorMessage.substring(0, errorMessage.indexOf('\n')));
			builder.append("\n");

			Throwable cause = getCause();
			while (cause != null && cause != this) {
				if (cause instanceof TemplateException) {
					TemplateException ex = (TemplateException)cause;
					if (ex.getCause() == null || ex.getCause() == ex)
						builder.append(ex.errorMessage);
					else
						builder.append(ex.errorMessage.substring(0, ex.errorMessage.indexOf('\n')));
					builder.append("\n");
				}
				cause = cause.getCause();
			}
			return builder.toString();
		}
	}
	
	public static class StringLiteralException extends RuntimeException {

		private static final long serialVersionUID = 1L;
		
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/ExpressionGlobalVariables.java
================================================
package org.spiderflow.core.expression;

import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantReadWriteLock;

public class ExpressionGlobalVariables {

	private static Map<String, String> variables = new HashMap<>();

	private static ReentrantReadWriteLock readWriteLock = new ReentrantReadWriteLock();

	public static void reset(Map<String, String> map){
		Lock lock = readWriteLock.writeLock();
		lock.lock();
		try {
			variables.clear();
			variables.putAll(map);
		} finally {
			lock.unlock();
		}
	}

	public static Map<String, String> getVariables(){
		Lock lock = readWriteLock.readLock();
		lock.lock();
		try {
			return variables;
		} finally {
			lock.unlock();
		}
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/ExpressionTemplate.java
================================================

package org.spiderflow.core.expression;

import java.io.OutputStream;
import java.util.List;

import org.spiderflow.core.expression.interpreter.AstInterpreter;
import org.spiderflow.core.expression.parsing.Ast;
import org.spiderflow.core.expression.parsing.Ast.Node;
import org.spiderflow.core.expression.parsing.Parser;


/** A template is loaded by a {@link TemplateLoader} from a file marked up with the basis-template language. The template can be
 * rendered to a {@link String} or {@link OutputStream} by calling one of the <code>render()</code> methods. The
 * {@link ExpressionTemplateContext} passed to the <code>render()</code> methods is used to look up variable values referenced in the
 * template. */
public class ExpressionTemplate {
	private final List<Node> nodes;

	/** Internal. Created by {@link Parser}. **/
	private ExpressionTemplate (List<Node> nodes) {
		this.nodes = nodes;
	}
	
	public static ExpressionTemplate create(String source){
		return new ExpressionTemplate(Parser.parse(source));
	}
	
	/** Internal. The AST nodes representing this template after parsing. See {@link Ast}. Used by {@link AstInterpreter}. **/
	public List<Node> getNodes () {
		return nodes;
	}

	/** Renders the template using the TemplateContext to resolve variable values referenced in the template. **/
	public Object render (ExpressionTemplateContext context) {
		return AstInterpreter.interpret(this, context);
	}
}


================================================
FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/ExpressionTemplateContext.java
================================================

package org.spiderflow.core.expression;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;

import org.spiderflow.core.expression.interpreter.AstInterpreter;


/**
 * <p>
 * A template context stores mappings from variable names to user provided variable values. A {@link ExpressionTemplate} is given a context
 * for rendering to resolve variable values it references in template expressions.
 * </p>
 *
 * <p>
 * Internally, a template context is a stack of these mappings, similar to scopes in a programming language, and used as such by
 * the {@link AstInterpreter}.
 * </p>
 */
public class ExpressionTemplateContext {
	private final List<Map<String, Object>> scopes = new ArrayList<Map<String, Object>>();

	/** Keeps track of previously allocated, unused scopes. New scopes are first tried to be retrieved from this pool to avoid
	 * generating garbage. **/
	private final List<Map<String, Object>> freeScopes = new ArrayList<Map<String, Object>>();

	private final static ThreadLocal<ExpressionTemplateContext> CONTEXT_THREAD_LOCAL = new ThreadLocal<>();

	public static ExpressionTemplateContext get(){
		return CONTEXT_THREAD_LOCAL.get();
	}

	public static void remove(){
		CONTEXT_THREAD_LOCAL.remove();
	}

	public static void set(ExpressionTemplateContext context){
		CONTEXT_THREAD_LOCAL.set(context);
	}

	public ExpressionTemplateContext () {
		push();
	}
	
	public ExpressionTemplateContext(Map<String,Object> variables) {
		this();
		if(variables != null){
			variables.forEach(this::set);
		}
	}

	/** Sets the value of the variable with the given name. If the variable already exists in one of the scopes, that variable is
	 * set. Otherwise the variable is set on the last pushed scope. */
	public ExpressionTemplateContext set (String name, Object value) {
		for (int i = scopes.size() - 1; i >= 0; i--) {
			Map<String, Object> ctx = scopes.get(i);
			if (ctx.isEmpty()) c
Download .txt
gitextract_2yt2vx0u/

├── .gitattributes
├── .gitignore
├── Dockerfile
├── LICENSE
├── README.md
├── db/
│   └── spiderflow.sql
├── pom.xml
├── spider-flow-api/
│   ├── pom.xml
│   └── src/
│       └── main/
│           └── java/
│               └── org/
│                   └── spiderflow/
│                       ├── ExpressionEngine.java
│                       ├── Grammerable.java
│                       ├── annotation/
│                       │   ├── Comment.java
│                       │   ├── Example.java
│                       │   └── Return.java
│                       ├── common/
│                       │   └── CURDController.java
│                       ├── concurrent/
│                       │   ├── ChildPriorThreadSubmitStrategy.java
│                       │   ├── LinkedThreadSubmitStrategy.java
│                       │   ├── ParentPriorThreadSubmitStrategy.java
│                       │   ├── RandomThreadSubmitStrategy.java
│                       │   ├── SpiderFlowThreadPoolExecutor.java
│                       │   ├── SpiderFutureTask.java
│                       │   └── ThreadSubmitStrategy.java
│                       ├── context/
│                       │   ├── CookieContext.java
│                       │   ├── SpiderContext.java
│                       │   └── SpiderContextHolder.java
│                       ├── enums/
│                       │   ├── FlowNoticeType.java
│                       │   └── FlowNoticeWay.java
│                       ├── executor/
│                       │   ├── FunctionExecutor.java
│                       │   ├── FunctionExtension.java
│                       │   ├── PluginConfig.java
│                       │   └── ShapeExecutor.java
│                       ├── expression/
│                       │   └── DynamicMethod.java
│                       ├── io/
│                       │   ├── Line.java
│                       │   ├── RandomAccessFileReader.java
│                       │   └── SpiderResponse.java
│                       ├── listener/
│                       │   └── SpiderListener.java
│                       ├── model/
│                       │   ├── Grammer.java
│                       │   ├── JsonBean.java
│                       │   ├── Plugin.java
│                       │   ├── Shape.java
│                       │   ├── SpiderLog.java
│                       │   ├── SpiderNode.java
│                       │   └── SpiderOutput.java
│                       └── utils/
│                           └── Maps.java
├── spider-flow-core/
│   ├── pom.xml
│   └── src/
│       └── main/
│           └── java/
│               └── org/
│                   └── spiderflow/
│                       └── core/
│                           ├── Spider.java
│                           ├── executor/
│                           │   ├── function/
│                           │   │   ├── Base64FunctionExecutor.java
│                           │   │   ├── DateFunctionExecutor.java
│                           │   │   ├── ExtractFunctionExecutor.java
│                           │   │   ├── FileFunctionExecutor.java
│                           │   │   ├── JsonFunctionExecutor.java
│                           │   │   ├── ListFunctionExecutor.java
│                           │   │   ├── MD5FunctionExecutor.java
│                           │   │   ├── RandomFunctionExecutor.java
│                           │   │   ├── StringFunctionExecutor.java
│                           │   │   ├── ThreadFunctionExecutor.java
│                           │   │   ├── UrlFunctionExecutor.java
│                           │   │   └── extension/
│                           │   │       ├── ArrayFunctionExtension.java
│                           │   │       ├── DateFunctionExtension.java
│                           │   │       ├── ElementFunctionExtension.java
│                           │   │       ├── ElementsFunctionExtension.java
│                           │   │       ├── ListFunctionExtension.java
│                           │   │       ├── MapFunctionExtension.java
│                           │   │       ├── ObjectFunctionExtension.java
│                           │   │       ├── ResponseFunctionExtension.java
│                           │   │       ├── SqlRowSetExtension.java
│                           │   │       └── StringFunctionExtension.java
│                           │   └── shape/
│                           │       ├── CommentExecutor.java
│                           │       ├── ExecuteSQLExecutor.java
│                           │       ├── ForkJoinExecutor.java
│                           │       ├── FunctionExecutor.java
│                           │       ├── LoopExecutor.java
│                           │       ├── OutputExecutor.java
│                           │       ├── ProcessExecutor.java
│                           │       ├── RequestExecutor.java
│                           │       ├── StartExecutor.java
│                           │       └── VariableExecutor.java
│                           ├── expression/
│                           │   ├── DefaultExpressionEngine.java
│                           │   ├── ExpressionError.java
│                           │   ├── ExpressionGlobalVariables.java
│                           │   ├── ExpressionTemplate.java
│                           │   ├── ExpressionTemplateContext.java
│                           │   ├── interpreter/
│                           │   │   ├── AstInterpreter.java
│                           │   │   ├── JavaReflection.java
│                           │   │   └── Reflection.java
│                           │   └── parsing/
│                           │       ├── Ast.java
│                           │       ├── CharacterStream.java
│                           │       ├── Parser.java
│                           │       ├── Span.java
│                           │       ├── Token.java
│                           │       ├── TokenStream.java
│                           │       ├── TokenType.java
│                           │       └── Tokenizer.java
│                           ├── io/
│                           │   ├── HttpRequest.java
│                           │   └── HttpResponse.java
│                           ├── job/
│                           │   ├── SpiderJob.java
│                           │   ├── SpiderJobContext.java
│                           │   └── SpiderJobManager.java
│                           ├── mapper/
│                           │   ├── DataSourceMapper.java
│                           │   ├── FlowNoticeMapper.java
│                           │   ├── FunctionMapper.java
│                           │   ├── SpiderFlowMapper.java
│                           │   ├── TaskMapper.java
│                           │   └── VariableMapper.java
│                           ├── model/
│                           │   ├── DataSource.java
│                           │   ├── FlowNotice.java
│                           │   ├── Function.java
│                           │   ├── SpiderFlow.java
│                           │   ├── Task.java
│                           │   └── Variable.java
│                           ├── script/
│                           │   └── ScriptManager.java
│                           ├── serializer/
│                           │   └── FastJsonSerializer.java
│                           ├── service/
│                           │   ├── DataSourceService.java
│                           │   ├── FlowNoticeService.java
│                           │   ├── FunctionService.java
│                           │   ├── SpiderFlowService.java
│                           │   ├── TaskService.java
│                           │   └── VariableService.java
│                           └── utils/
│                               ├── DataSourceUtils.java
│                               ├── EmailUtils.java
│                               ├── ExecutorsUtils.java
│                               ├── ExpressionUtils.java
│                               ├── ExtractUtils.java
│                               ├── FileUtils.java
│                               └── SpiderFlowUtils.java
└── spider-flow-web/
    ├── pom.xml
    └── src/
        └── main/
            ├── java/
            │   └── org/
            │       └── spiderflow/
            │           ├── SpiderApplication.java
            │           ├── configuration/
            │           │   ├── ResourcesConfiguration.java
            │           │   └── WebSocketConfiguration.java
            │           ├── controller/
            │           │   ├── DataSourceController.java
            │           │   ├── FlowNoticeController.java
            │           │   ├── FunctionController.java
            │           │   ├── SpiderFlowController.java
            │           │   ├── SpiderRestController.java
            │           │   ├── TaskController.java
            │           │   └── VariableController.java
            │           ├── logback/
            │           │   ├── SpiderFlowFileAppender.java
            │           │   └── SpiderFlowWebSocketAppender.java
            │           ├── model/
            │           │   ├── SpiderWebSocketContext.java
            │           │   └── WebSocketEvent.java
            │           └── websocket/
            │               └── WebSocketEditorServer.java
            └── resources/
                ├── application.properties
                ├── logback-spring.xml
                └── static/
                    ├── css/
                    │   ├── easyui.css
                    │   ├── editor.css
                    │   ├── index.css
                    │   ├── layui-black-gray.css
                    │   └── layui-blue.css
                    ├── datasource-edit.html
                    ├── datasources.html
                    ├── editCron.html
                    ├── editor.html
                    ├── function-edit.html
                    ├── functions.html
                    ├── index.html
                    ├── js/
                    │   ├── canvas-viewer.js
                    │   ├── codemirror/
                    │   │   ├── codemirror.css
                    │   │   ├── codemirror.js
                    │   │   ├── dracula.css
                    │   │   ├── idea.css
                    │   │   ├── javascript.js
                    │   │   ├── placeholder.js
                    │   │   ├── show-hint.css
                    │   │   ├── show-hint.js
                    │   │   ├── spiderflow-hint.js
                    │   │   ├── spiderflow.js
                    │   │   └── sql.js
                    │   ├── common.js
                    │   ├── cron/
                    │   │   └── cron.js
                    │   ├── editor.js
                    │   ├── index.js
                    │   ├── jsontree/
                    │   │   ├── jsontree.css
                    │   │   └── jsontree.js
                    │   ├── layui/
                    │   │   ├── css/
                    │   │   │   ├── layui.css
                    │   │   │   ├── layui.mobile.css
                    │   │   │   └── modules/
                    │   │   │       ├── code.css
                    │   │   │       ├── laydate/
                    │   │   │       │   └── default/
                    │   │   │       │       └── laydate.css
                    │   │   │       └── layer/
                    │   │   │           └── default/
                    │   │   │               └── layer.css
                    │   │   ├── ext/
                    │   │   │   ├── eleTree/
                    │   │   │   │   ├── eleTree.css
                    │   │   │   │   └── eleTree.js
                    │   │   │   └── treeselect/
                    │   │   │       └── treeselect.js
                    │   │   ├── extends/
                    │   │   │   ├── formSelects-v4.css
                    │   │   │   ├── formSelects-v4.js
                    │   │   │   └── treeGrid.js
                    │   │   └── layui.all.js
                    │   ├── log-viewer.js
                    │   ├── mxgraph/
                    │   │   ├── css/
                    │   │   │   ├── common.css
                    │   │   │   └── explorer.css
                    │   │   ├── mxgraph.js
                    │   │   └── resources/
                    │   │       ├── editor.txt
                    │   │       ├── editor_de.txt
                    │   │       ├── editor_zh.txt
                    │   │       ├── graph.txt
                    │   │       ├── graph_de.txt
                    │   │       └── graph_zh.txt
                    │   └── spider-editor.js
                    ├── log.html
                    ├── resources/
                    │   └── templates/
                    │       ├── comment.html
                    │       ├── edge.html
                    │       ├── executeSql.html
                    │       ├── forkJoin.html
                    │       ├── function.html
                    │       ├── loop.html
                    │       ├── output.html
                    │       ├── process.html
                    │       ├── request.html
                    │       ├── root.html
                    │       ├── start.html
                    │       └── variable.html
                    ├── spiderList-notice.html
                    ├── spiderList.html
                    ├── task.html
                    ├── variable-edit.html
                    └── variables.html
Download .txt
SYMBOL INDEX (1942 symbols across 147 files)

FILE: db/spiderflow.sql
  type `sp_flow` (line 7) | CREATE TABLE `sp_flow` (
  type `sp_datasource` (line 26) | CREATE TABLE `sp_datasource` (
  type `sp_variable` (line 38) | CREATE TABLE `sp_variable` (
  type `sp_task` (line 49) | CREATE TABLE `sp_task` (
  type `sp_function` (line 59) | CREATE TABLE `sp_function`  (
  type `sp_flow_notice` (line 70) | CREATE TABLE `sp_flow_notice` (

FILE: spider-flow-api/src/main/java/org/spiderflow/ExpressionEngine.java
  type ExpressionEngine (line 8) | public interface ExpressionEngine {
    method execute (line 16) | Object execute(String expression, Map<String, Object> variables);

FILE: spider-flow-api/src/main/java/org/spiderflow/Grammerable.java
  type Grammerable (line 6) | public interface Grammerable {
    method grammers (line 8) | List<Grammer> grammers();

FILE: spider-flow-api/src/main/java/org/spiderflow/common/CURDController.java
  class CURDController (line 14) | public abstract class CURDController<S extends ServiceImpl<M, T>,M exten...
    method list (line 19) | @RequestMapping("/list")
    method get (line 24) | @RequestMapping("get")
    method delete (line 29) | @RequestMapping("delete")
    method save (line 34) | @RequestMapping("save")

FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/ChildPriorThreadSubmitStrategy.java
  class ChildPriorThreadSubmitStrategy (line 8) | public class ChildPriorThreadSubmitStrategy implements ThreadSubmitStrat...
    method comparator (line 21) | @Override
    method add (line 26) | @Override
    method isEmpty (line 33) | @Override
    method get (line 40) | @Override

FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/LinkedThreadSubmitStrategy.java
  class LinkedThreadSubmitStrategy (line 9) | public class LinkedThreadSubmitStrategy implements ThreadSubmitStrategy{
    method comparator (line 13) | @Override
    method add (line 18) | @Override
    method isEmpty (line 23) | @Override
    method get (line 28) | @Override

FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/ParentPriorThreadSubmitStrategy.java
  class ParentPriorThreadSubmitStrategy (line 8) | public class ParentPriorThreadSubmitStrategy implements ThreadSubmitStra...
    method comparator (line 21) | @Override
    method add (line 26) | @Override
    method isEmpty (line 33) | @Override
    method get (line 40) | @Override

FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/RandomThreadSubmitStrategy.java
  class RandomThreadSubmitStrategy (line 10) | public class RandomThreadSubmitStrategy implements ThreadSubmitStrategy{
    method comparator (line 14) | @Override
    method add (line 19) | @Override
    method isEmpty (line 24) | @Override
    method get (line 29) | @Override

FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/SpiderFlowThreadPoolExecutor.java
  class SpiderFlowThreadPoolExecutor (line 8) | public class SpiderFlowThreadPoolExecutor {
    method SpiderFlowThreadPoolExecutor (line 35) | public SpiderFlowThreadPoolExecutor(int maxThreads) {
    method submit (line 45) | public Future<?> submit(Runnable runnable){
    method createSubThreadPoolExecutor (line 55) | public SubThreadPoolExecutor createSubThreadPoolExecutor(int threads,T...
    class SubThreadPoolExecutor (line 62) | public class SubThreadPoolExecutor{
      method SubThreadPoolExecutor (line 91) | public SubThreadPoolExecutor(int threads,ThreadSubmitStrategy submit...
      method awaitTermination (line 101) | public void awaitTermination(){
      method index (line 112) | private int index(){
      method removeDoneFuture (line 124) | private void removeDoneFuture(){
      method await (line 139) | private void await(){
      method submitAsync (line 148) | public <T> Future<T> submitAsync(Runnable runnable, T value, SpiderN...
      method submit (line 173) | private void submit(){

FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/SpiderFutureTask.java
  class SpiderFutureTask (line 7) | public class SpiderFutureTask<V> extends FutureTask {
    method SpiderFutureTask (line 13) | public SpiderFutureTask(Runnable runnable, V result, SpiderNode node,S...
    method getExecutor (line 19) | public SubThreadPoolExecutor getExecutor() {
    method getNode (line 23) | public SpiderNode getNode() {

FILE: spider-flow-api/src/main/java/org/spiderflow/concurrent/ThreadSubmitStrategy.java
  type ThreadSubmitStrategy (line 7) | public interface ThreadSubmitStrategy {
    method comparator (line 9) | Comparator<SpiderNode> comparator();
    method add (line 11) | void add(SpiderFutureTask<?> task);
    method isEmpty (line 13) | boolean isEmpty();
    method get (line 15) | SpiderFutureTask<?> get();

FILE: spider-flow-api/src/main/java/org/spiderflow/context/CookieContext.java
  class CookieContext (line 8) | public class CookieContext extends HashMap<String, String> {

FILE: spider-flow-api/src/main/java/org/spiderflow/context/SpiderContext.java
  class SpiderContext (line 17) | public class SpiderContext extends HashMap<String, Object>{
    method getOutputs (line 53) | public List<SpiderOutput> getOutputs() {
    method get (line 57) | public <T> T get(String key){
    method get (line 61) | public <T> T get(String key,T defaultValue){
    method getFlowId (line 66) | public String getFlowId() {
    method setFlowId (line 70) | public void setFlowId(String flowId) {
    method getFutureQueue (line 74) | public LinkedBlockingQueue<Future<?>> getFutureQueue() {
    method isRunning (line 78) | public boolean isRunning() {
    method setRunning (line 82) | public void setRunning(boolean running) {
    method addOutput (line 86) | public void addOutput(SpiderOutput output){
    method getThreadPool (line 90) | public SubThreadPoolExecutor getThreadPool() {
    method setThreadPool (line 94) | public void setThreadPool(SubThreadPoolExecutor threadPool) {
    method getRootNode (line 98) | public SpiderNode getRootNode() {
    method setRootNode (line 102) | public void setRootNode(SpiderNode rootNode) {
    method getId (line 106) | public String getId() {
    method getCookieContext (line 110) | public CookieContext getCookieContext() {
    method pause (line 114) | public void pause(String nodeId,String event,String key,Object value){}
    method resume (line 116) | public void resume(){}
    method stop (line 118) | public void stop(){}

FILE: spider-flow-api/src/main/java/org/spiderflow/context/SpiderContextHolder.java
  class SpiderContextHolder (line 5) | public class SpiderContextHolder {
    method get (line 9) | public static SpiderContext get() {
    method set (line 13) | public static void set(SpiderContext context) {
    method remove (line 17) | public static void remove() {

FILE: spider-flow-api/src/main/java/org/spiderflow/enums/FlowNoticeType.java
  type FlowNoticeType (line 9) | public enum FlowNoticeType {

FILE: spider-flow-api/src/main/java/org/spiderflow/enums/FlowNoticeWay.java
  type FlowNoticeWay (line 12) | public enum FlowNoticeWay {
    method FlowNoticeWay (line 16) | private FlowNoticeWay(String title) {
    method toString (line 22) | @Override
    method getMap (line 27) | public static Map<String, String> getMap() {

FILE: spider-flow-api/src/main/java/org/spiderflow/executor/FunctionExecutor.java
  type FunctionExecutor (line 3) | public interface FunctionExecutor {
    method getFunctionPrefix (line 5) | String getFunctionPrefix();

FILE: spider-flow-api/src/main/java/org/spiderflow/executor/FunctionExtension.java
  type FunctionExtension (line 3) | public interface FunctionExtension {
    method support (line 5) | Class<?> support();

FILE: spider-flow-api/src/main/java/org/spiderflow/executor/PluginConfig.java
  type PluginConfig (line 5) | public interface PluginConfig {
    method plugin (line 7) | Plugin plugin();

FILE: spider-flow-api/src/main/java/org/spiderflow/executor/ShapeExecutor.java
  type ShapeExecutor (line 13) | public interface ShapeExecutor {
    method shape (line 21) | default Shape shape(){
    method supportShape (line 29) | String supportShape();
    method execute (line 37) | void execute(SpiderNode node, SpiderContext context, Map<String, Objec...
    method allowExecuteNext (line 39) | default boolean allowExecuteNext(SpiderNode node, SpiderContext contex...
    method isThread (line 43) | default boolean isThread(){

FILE: spider-flow-api/src/main/java/org/spiderflow/expression/DynamicMethod.java
  type DynamicMethod (line 5) | public interface DynamicMethod {
    method execute (line 7) | Object execute(String methodName, List<Object> parameters);

FILE: spider-flow-api/src/main/java/org/spiderflow/io/Line.java
  class Line (line 3) | public class Line {
    method Line (line 11) | public Line(long from, String text, long to) {
    method getFrom (line 17) | public long getFrom() {
    method setFrom (line 21) | public void setFrom(long from) {
    method getText (line 25) | public String getText() {
    method setText (line 29) | public void setText(String text) {
    method getTo (line 33) | public long getTo() {
    method setTo (line 37) | public void setTo(long to) {
    method toString (line 41) | @Override

FILE: spider-flow-api/src/main/java/org/spiderflow/io/RandomAccessFileReader.java
  class RandomAccessFileReader (line 10) | public class RandomAccessFileReader implements Closeable {
    method RandomAccessFileReader (line 29) | public RandomAccessFileReader(RandomAccessFile raf, long index, boolea...
    method RandomAccessFileReader (line 33) | public RandomAccessFileReader(RandomAccessFile raf, long index, int bu...
    method init (line 44) | private void init() throws IOException {
    method readLine (line 64) | public List<Line> readLine(int n, String keywords, boolean matchcase, ...
    method find (line 110) | private boolean find(String text, String keywords, boolean matchcase) {
    method find (line 114) | private boolean find(String text, Pattern pattern) {
    method readLine (line 126) | private Line readLine(long fromIndex, long endIndex) throws IOException {
    method isCRLF (line 136) | private boolean isCRLF(byte b) {
    method close (line 140) | @Override

FILE: spider-flow-api/src/main/java/org/spiderflow/io/SpiderResponse.java
  type SpiderResponse (line 11) | public interface SpiderResponse {
    method getStatusCode (line 13) | @Comment("获取返回状态码")
    method getTitle (line 17) | @Comment("获取网页标题")
    method getHtml (line 21) | @Comment("获取网页html")
    method getJson (line 25) | @Comment("获取json")
    method getCookies (line 30) | @Comment("获取cookies")
    method getHeaders (line 34) | @Comment("获取headers")
    method getBytes (line 38) | @Comment("获取byte[]")
    method getContentType (line 42) | @Comment("获取ContentType")
    method getUrl (line 46) | @Comment("获取当前url")
    method setCharset (line 50) | @Example("${resp.setCharset('UTF-8')}")
    method getStream (line 54) | @Example("${resp.stream}")

FILE: spider-flow-api/src/main/java/org/spiderflow/listener/SpiderListener.java
  type SpiderListener (line 5) | public interface SpiderListener {
    method beforeStart (line 10) | void beforeStart(SpiderContext context);
    method afterEnd (line 15) | void afterEnd(SpiderContext context);

FILE: spider-flow-api/src/main/java/org/spiderflow/model/Grammer.java
  class Grammer (line 13) | public class Grammer {
    method getOwner (line 27) | public String getOwner() {
    method setOwner (line 31) | public void setOwner(String owner) {
    method getMethod (line 35) | public String getMethod() {
    method setMethod (line 39) | public void setMethod(String method) {
    method getFunction (line 43) | public String getFunction() {
    method setFunction (line 47) | public void setFunction(String function) {
    method getComment (line 51) | public String getComment() {
    method setComment (line 55) | public void setComment(String comment) {
    method getExample (line 59) | public String getExample() {
    method setExample (line 63) | public void setExample(String example) {
    method getReturns (line 67) | public List<String> getReturns() {
    method setReturns (line 71) | public void setReturns(List<String> returns) {
    method findGrammers (line 75) | public static List<Grammer> findGrammers(Class<?> clazz,String functio...

FILE: spider-flow-api/src/main/java/org/spiderflow/model/JsonBean.java
  class JsonBean (line 3) | public class JsonBean<T> {
    method JsonBean (line 11) | public JsonBean(Integer code, String message, T data) {
    method JsonBean (line 17) | public JsonBean(Integer code, String message) {
    method JsonBean (line 22) | public JsonBean(T data) {
    method getCode (line 26) | public Integer getCode() {
    method setCode (line 30) | public void setCode(Integer code) {
    method getMessage (line 34) | public String getMessage() {
    method setMessage (line 38) | public void setMessage(String message) {
    method getData (line 42) | public T getData() {
    method setData (line 46) | public void setData(T data) {

FILE: spider-flow-api/src/main/java/org/spiderflow/model/Plugin.java
  class Plugin (line 3) | public class Plugin {
    method Plugin (line 9) | public Plugin(String name, String url) {
    method getName (line 14) | public String getName() {
    method setName (line 18) | public void setName(String name) {
    method getUrl (line 22) | public String getUrl() {
    method setUrl (line 26) | public void setUrl(String url) {

FILE: spider-flow-api/src/main/java/org/spiderflow/model/Shape.java
  class Shape (line 3) | public class Shape {
    method getName (line 15) | public String getName() {
    method setName (line 19) | public void setName(String name) {
    method getLabel (line 23) | public String getLabel() {
    method setLabel (line 27) | public void setLabel(String label) {
    method getTitle (line 31) | public String getTitle() {
    method setTitle (line 35) | public void setTitle(String title) {
    method getImage (line 39) | public String getImage() {
    method setImage (line 43) | public void setImage(String image) {
    method getDesc (line 47) | public String getDesc() {
    method setDesc (line 51) | public void setDesc(String desc) {

FILE: spider-flow-api/src/main/java/org/spiderflow/model/SpiderLog.java
  class SpiderLog (line 8) | public class SpiderLog {
    method SpiderLog (line 16) | public SpiderLog(String level,String message, List<Object> variables) {
    method getLevel (line 32) | public String getLevel() {
    method setLevel (line 36) | public void setLevel(String level) {
    method getMessage (line 40) | public String getMessage() {
    method setMessage (line 44) | public void setMessage(String message) {
    method getVariables (line 48) | public List<Object> getVariables() {
    method setVariables (line 52) | public void setVariables(List<Object> variables) {

FILE: spider-flow-api/src/main/java/org/spiderflow/model/SpiderNode.java
  class SpiderNode (line 17) | public class SpiderNode {
    method getNodeId (line 65) | public String getNodeId() {
    method setNodeId (line 69) | public void setNodeId(String nodeId) {
    method getNodeName (line 73) | public String getNodeName() {
    method setNodeName (line 77) | public void setNodeName(String nodeName) {
    method getStringJsonValue (line 81) | public String getStringJsonValue(String key){
    method getStringJsonValue (line 89) | public String getStringJsonValue(String key,String defaultValue){
    method getListJsonValue (line 94) | public List<Map<String,String>> getListJsonValue(String ... keys){
    method setJsonProperty (line 122) | public void setJsonProperty(Map<String, Object> jsonProperty) {
    method addNextNode (line 126) | public void addNextNode(SpiderNode nextNode){
    method getExceptionFlow (line 131) | public String getExceptionFlow(String fromNodeId) {
    method isTransmitVariable (line 135) | public boolean isTransmitVariable(String fromNodeId) {
    method setTransmitVariable (line 140) | public void setTransmitVariable(String fromNodeId,String value){
    method setExceptionFlow (line 144) | public void setExceptionFlow(String fromNodeId,String value){
    method getNextNodes (line 148) | public List<SpiderNode> getNextNodes() {
    method getCondition (line 152) | public String getCondition(String fromNodeId) {
    method setCondition (line 156) | public void setCondition(String fromNodeId,String condition) {
    method increment (line 160) | public void increment(){
    method decrement (line 164) | public void decrement(){
    method hasLeftNode (line 168) | public boolean hasLeftNode(String nodeId){
    method generateParents (line 177) | private void generateParents(Set<String> parents){
    method isDone (line 185) | public boolean isDone(){
    method isDone (line 188) | public boolean isDone(Set<String> visited){
    method toString (line 200) | @Override

FILE: spider-flow-api/src/main/java/org/spiderflow/model/SpiderOutput.java
  class SpiderOutput (line 6) | public class SpiderOutput {
    method getNodeName (line 28) | public String getNodeName() {
    method setNodeName (line 32) | public void setNodeName(String nodeName) {
    method getOutputNames (line 36) | public List<String> getOutputNames() {
    method setOutputNames (line 40) | public void setOutputNames(List<String> outputNames) {
    method getValues (line 44) | public List<Object> getValues() {
    method setValues (line 48) | public void setValues(List<Object> values) {
    method addOutput (line 52) | public void addOutput(String name,Object value){
    method getNodeId (line 57) | public String getNodeId() {
    method setNodeId (line 61) | public void setNodeId(String nodeId) {
    method toString (line 65) | @Override

FILE: spider-flow-api/src/main/java/org/spiderflow/utils/Maps.java
  class Maps (line 7) | public class Maps {
    method add (line 9) | public static <K,V> Map<K,V> add(Map<K,V> srcMap,K k,V v){
    method newMap (line 15) | public static <K,V> Map<K,V> newMap(K key,V value){
    method add (line 21) | public static <K,V> Map<K,V> add(Map<K,V> srcMap,List<K> ks,List<V> vs){

FILE: spider-flow-core/src/main/java/org/spiderflow/core/Spider.java
  class Spider (line 41) | @Component
    method init (line 65) | @PostConstruct
    method run (line 70) | public List<SpiderOutput> run(SpiderFlow spiderFlow, SpiderContext con...
    method run (line 83) | public List<SpiderOutput> run(SpiderFlow spiderFlow, SpiderContext con...
    method runWithTest (line 87) | public void runWithTest(SpiderNode root, SpiderContext context) {
    method executeRoot (line 109) | private void executeRoot(SpiderNode root, SpiderContext context, Map<S...
    method executeNextNodes (line 189) | private void executeNextNodes(SpiderNode node, SpiderContext context, ...
    method executeNode (line 201) | public void executeNode(SpiderNode fromNode, SpiderNode node, SpiderCo...
    method executeCondition (line 313) | private boolean executeCondition(SpiderNode fromNode, SpiderNode node,...
    class SpiderTask (line 341) | class SpiderTask{
      method SpiderTask (line 351) | public SpiderTask(Runnable runnable, SpiderNode node, Map<String, Ob...

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/Base64FunctionExecutor.java
  class Base64FunctionExecutor (line 14) | @Component
    method getFunctionPrefix (line 18) | @Override
    method encode (line 23) | @Comment("根据byte[]进行base64加密")
    method encode (line 29) | @Comment("根据String进行base64加密")
    method encode (line 35) | @Comment("根据String进行base64加密")
    method encodeBytes (line 41) | @Comment("根据byte[]进行base64加密")
    method encodeBytes (line 47) | @Comment("根据String进行base64加密")
    method encodeBytes (line 53) | @Comment("根据String进行base64加密")
    method decode (line 59) | @Comment("根据String进行base64解密")
    method decode (line 65) | @Comment("根据byte[]进行base64解密")
    method decodeString (line 71) | @Comment("根据String进行base64解密")
    method decodeString (line 77) | @Comment("根据byte[]进行base64解密")
    method decodeString (line 83) | @Comment("根据byte[]进行base64解密")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/DateFunctionExecutor.java
  class DateFunctionExecutor (line 18) | @Component
    method getFunctionPrefix (line 22) | @Override
    method format (line 29) | @Comment("格式化日期")
    method format (line 35) | @Comment("格式化日期")
    method format (line 41) | @Comment("格式化日期")
    method format (line 47) | @Comment("格式化日期")
    method parse (line 53) | @Comment("字符串转为日期类型")
    method parse (line 59) | @Comment("字符串转为日期类型")
    method parse (line 65) | @Comment("数字为日期类型")
    method now (line 71) | @Comment("获取当前时间")
    method addYears (line 77) | @Comment("获取指定日期n年后的日期")
    method addMonths (line 83) | @Comment("获取指定日期n月后的日期")
    method addDays (line 89) | @Comment("获取指定日期n天后的日期")
    method addHours (line 95) | @Comment("获取指定日期n小时后的日期")
    method addMinutes (line 101) | @Comment("获取指定日期n分钟后的日期")
    method addSeconds (line 107) | @Comment("获取指定日期n秒后的日期")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/ExtractFunctionExecutor.java
  class ExtractFunctionExecutor (line 13) | @Component
    method getFunctionPrefix (line 17) | @Override
    method jsonpath (line 22) | @Comment("根据jsonpath提取内容")
    method regx (line 28) | @Comment("根据正则表达式提取内容")
    method regx (line 34) | @Comment("根据正则表达式提取内容")
    method regx (line 40) | @Comment("根据正则表达式提取内容")
    method regxs (line 46) | @Comment("根据正则表达式提取内容")
    method regxs (line 52) | @Comment("根据正则表达式提取内容")
    method regxs (line 58) | @Comment("根据正则表达式提取内容")
    method xpath (line 64) | @Comment("根据xpath提取内容")
    method xpath (line 70) | @Comment("根据xpath提取内容")
    method xpaths (line 76) | @Comment("根据xpaths提取内容")
    method xpaths (line 82) | @Comment("根据xpaths提取内容")
    method selectors (line 88) | @Comment("根据css选择器提取内容")
    method selector (line 94) | @Comment("根据css选择器提取内容")
    method selector (line 107) | @Comment("根据css选择器提取内容")
    method selector (line 116) | @Comment("根据css选择器提取内容")
    method selectors (line 122) | @Comment("根据css选择器提取内容")
    method selectors (line 135) | @Comment("根据css选择器提取内容")
    method getElement (line 144) | private static Element getElement(Object object){

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/FileFunctionExecutor.java
  class FileFunctionExecutor (line 20) | @Component
    method getFunctionPrefix (line 24) | @Override
    method getFile (line 35) | private static File getFile(String path,boolean createDirectory){
    method write (line 43) | @Comment("写出文件")
    method write (line 49) | @Comment("写出文件")
    method write (line 55) | @Comment("写出文件")
    method write (line 61) | @Comment("写出文件")
    method write (line 69) | @Comment("写出文件")
    method write (line 75) | @Comment("写出文件")
    method write (line 81) | @Comment("写出文件")
    method write (line 87) | @Comment("写出文件")
    method download (line 93) | @Comment("下载Url资源")
    method download (line 103) | @Comment("下载Url资源")
    method bytes (line 111) | @Comment("读取文件")
    method string (line 119) | @Comment("读取文件")
    method string (line 125) | @Comment("读取文件")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/JsonFunctionExecutor.java
  class JsonFunctionExecutor (line 15) | @Component
    method getFunctionPrefix (line 19) | @Override
    method parse (line 24) | @Comment("将字符串转为json对象")
    method stringify (line 30) | @Comment("将对象转为json字符串")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/ListFunctionExecutor.java
  class ListFunctionExecutor (line 17) | @Component
    method getFunctionPrefix (line 21) | @Override
    method length (line 26) | @Comment("获取list的长度")
    method split (line 38) | @Comment("分割List")
    method sublist (line 54) | @Comment("截取List")
    method filterStr (line 60) | @Comment("过滤字符串list元素")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/MD5FunctionExecutor.java
  class MD5FunctionExecutor (line 12) | @Component
    method getFunctionPrefix (line 16) | @Override
    method string (line 21) | @Comment("md5加密")
    method string (line 27) | @Comment("md5加密")
    method string (line 33) | @Comment("md5加密")
    method bytes (line 39) | @Comment("md5加密")
    method bytes (line 45) | @Comment("md5加密")
    method bytes (line 51) | @Comment("md5加密")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/RandomFunctionExecutor.java
  class RandomFunctionExecutor (line 14) | @Component
    method getFunctionPrefix (line 17) | @Override
    method randomInt (line 22) | @Comment("随机获取int")
    method randomDouble (line 28) | @Comment("随机获取double")
    method randomLong (line 34) | @Comment("随机获取long")
    method string (line 46) | @Comment("随机获取字符串")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/StringFunctionExecutor.java
  class StringFunctionExecutor (line 19) | @Component
    method getFunctionPrefix (line 23) | @Override
    method substring (line 28) | @Comment("截取字符串方法")
    method substring (line 34) | @Comment("截取字符串方法")
    method lower (line 40) | @Comment("将字符串转为小写")
    method upper (line 46) | @Comment("将字符串转为大写")
    method indexOf (line 52) | @Comment("查找指定字符在字符串在中的位置")
    method lastIndexOf (line 58) | @Comment("查找指定字符在字符串中最后出现的位置")
    method indexOf (line 64) | @Comment("查找指定字符在字符串在中的位置")
    method toInt (line 70) | @Comment("将字符串转为int")
    method toInt (line 76) | @Comment("将字符串转为Integer")
    method replace (line 86) | @Comment("字符串替换")
    method replaceAll (line 92) | @Comment("正则替换字符串")
    method replaceFirst (line 98) | @Comment("正则替换字符串")
    method length (line 104) | @Comment("正则替换字符串")
    method trim (line 110) | @Comment("去除字符串两边的空格")
    method split (line 116) | @Comment("分割字符串")
    method bytes (line 122) | @Comment("获取字符串的byte[]")
    method bytes (line 128) | @Comment("获取字符串的byte[]")
    method newString (line 138) | @Comment("byte[]转String")
    method newString (line 144) | @Comment("byte[]转String")
    method equals (line 154) | @Comment("判断两个字符串是否相同")
    method uuid (line 160) | @Comment("生成UUID")
    method uuids (line 166) | @Comment("生成多个UUID")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/ThreadFunctionExecutor.java
  class ThreadFunctionExecutor (line 13) | @Component
    method getFunctionPrefix (line 16) | @Override
    method sleep (line 21) | @Comment("线程休眠")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/UrlFunctionExecutor.java
  class UrlFunctionExecutor (line 19) | @Component
    method getFunctionPrefix (line 22) | @Override
    method parameter (line 27) | @Comment("获取url参数")
    method parameterMap (line 33) | @Comment("获取url全部参数")
    method encode (line 64) | @Comment("url编码")
    method encode (line 70) | @Comment("url编码")
    method decode (line 80) | @Comment("url解码")
    method decode (line 86) | @Comment("url解码")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ArrayFunctionExtension.java
  class ArrayFunctionExtension (line 12) | @Component
    method support (line 15) | @Override
    method size (line 20) | @Comment("获取数组的长度")
    method join (line 26) | @Comment("将数组拼接起来")
    method join (line 32) | @Comment("将数组用separator拼接起来")
    method toList (line 38) | @Comment("将数组转为List")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/DateFunctionExtension.java
  class DateFunctionExtension (line 11) | @Component
    method support (line 14) | @Override
    method format (line 19) | @Comment("格式化日期")
    method format (line 25) | @Comment("格式化日期")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ElementFunctionExtension.java
  class ElementFunctionExtension (line 14) | @Component
    method support (line 17) | @Override
    method xpath (line 22) | @Comment("根据xpath提取内容")
    method xpaths (line 30) | @Comment("根据xpath提取内容")
    method regx (line 37) | @Comment("根据正则表达式提取内容")
    method regx (line 43) | @Comment("根据正则表达式提取内容")
    method regx (line 49) | @Comment("根据正则表达式提取内容")
    method regxs (line 55) | @Comment("根据正则表达式提取内容")
    method regxs (line 61) | @Comment("根据正则表达式提取内容")
    method regxs (line 67) | @Comment("根据正则表达式提取内容")
    method selector (line 73) | @Comment("根据css选择器提取内容")
    method selectors (line 79) | @Comment("根据css选择器提取内容")
    method subling (line 85) | @Comment("获取同级节点")
    method parent (line 91) | @Comment("获取上级节点")
    method parents (line 97) | @Comment("获取上级节点")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ElementsFunctionExtension.java
  class ElementsFunctionExtension (line 14) | @Component
    method support (line 17) | @Override
    method xpath (line 22) | @Comment("根据xpath提取内容")
    method xpaths (line 28) | @Comment("根据xpath提取内容")
    method regx (line 34) | @Comment("根据正则表达式提取内容")
    method regx (line 40) | @Comment("根据正则表达式提取内容")
    method regx (line 46) | @Comment("根据正则表达式提取内容")
    method regxs (line 52) | @Comment("根据正则表达式提取内容")
    method regxs (line 58) | @Comment("根据正则表达式提取内容")
    method regxs (line 64) | @Comment("根据正则表达式提取内容")
    method selector (line 70) | @Comment("根据css选择器提取内容")
    method attrs (line 80) | @Comment("返回所有attr")
    method vals (line 90) | @Comment("返回所有value")
    method texts (line 100) | @Comment("返回所有text")
    method htmls (line 110) | @Comment("返回所有html")
    method outerHtmls (line 120) | @Comment("返回所有outerHtml")
    method ownTexts (line 130) | @Comment("返回所有ownTexts")
    method wholeTexts (line 140) | @Comment("返回所有wholeText")
    method selectors (line 150) | @Comment("根据css选择器提取内容")
    method parents (line 156) | @Comment("获取上级节点")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ListFunctionExtension.java
  class ListFunctionExtension (line 13) | @Component
    method support (line 16) | @Override
    method length (line 21) | @Comment("获取list的长度")
    method join (line 27) | @Comment("将list拼接起来")
    method join (line 33) | @Comment("将list用separator拼接起来")
    method sort (line 43) | @Comment("将list<String>排序")
    method shuffle (line 50) | @Comment("将list打乱顺序")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/MapFunctionExtension.java
  class MapFunctionExtension (line 12) | @Component
    method support (line 15) | @Override
    method toList (line 20) | @Comment("将map转换为List")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ObjectFunctionExtension.java
  class ObjectFunctionExtension (line 13) | @Component
    method support (line 16) | @Override
    method string (line 21) | @Comment("将对象转为string类型")
    method jsonpath (line 30) | @Comment("根据jsonpath提取内容")
    method sleep (line 39) | @Comment("睡眠等待一段时间")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ResponseFunctionExtension.java
  class ResponseFunctionExtension (line 19) | @Component
    method support (line 22) | @Override
    method element (line 27) | @Comment("将请求结果转为Element对象")
    method xpath (line 33) | @Comment("根据xpath在请求结果中查找")
    method xpaths (line 40) | @Comment("根据xpath在请求结果中查找")
    method regx (line 46) | @Comment("根据正则表达式提取请求结果中的内容")
    method regx (line 52) | @Comment("根据正则表达式提取请求结果中的内容")
    method regx (line 58) | @Comment("根据正则表达式提取请求结果中的内容")
    method regxs (line 64) | @Comment("根据正则表达式提取请求结果中的内容")
    method regxs (line 70) | @Comment("根据正则表达式提取请求结果中的内容")
    method regxs (line 76) | @Comment("根据正则表达式提取请求结果中的内容")
    method selector (line 82) | @Comment("根据css选择器提取请求结果")
    method selectors (line 88) | @Comment("根据css选择器提取请求结果")
    method jsonpath (line 94) | @Comment("根据jsonpath提取请求结果")
    method links (line 100) | @Comment("获取页面上的链接")
    method links (line 109) | @Comment("获取页面上的链接")
    method images (line 119) | @Comment("获取当前页面所有图片链接")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/SqlRowSetExtension.java
  class SqlRowSetExtension (line 12) | @Component
    method support (line 16) | @Override
    method nextToMap (line 21) | @Example("${rs.nextToMap()}")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/StringFunctionExtension.java
  class StringFunctionExtension (line 21) | @Component
    method support (line 24) | @Override
    method regx (line 29) | @Comment("根据正则表达式提取String中的内容")
    method regx (line 35) | @Comment("根据正则表达式提取String中的内容")
    method regx (line 41) | @Comment("根据正则表达式提取String中的内容")
    method regxs (line 47) | @Comment("根据正则表达式提取String中的内容")
    method regxs (line 53) | @Comment("根据正则表达式提取String中的内容")
    method regxs (line 59) | @Comment("根据正则表达式提取String中的内容")
    method xpath (line 65) | @Comment("根据xpath在String变量中查找")
    method xpaths (line 72) | @Comment("根据xpath在String变量中查找")
    method element (line 78) | @Comment("将String变量转为Element对象")
    method selector (line 84) | @Comment("根据css选择器提取")
    method selectors (line 90) | @Comment("根据css选择器提取")
    method json (line 96) | @Comment("将string转为json对象")
    method jsonpath (line 102) | @Comment("根据jsonpath提取内容")
    method toInt (line 108) | @Comment("将字符串转为int类型")
    method toInt (line 114) | @Comment("将字符串转为int类型")
    method toDouble (line 120) | @Comment("将字符串转为double类型")
    method toLong (line 126) | @Comment("将字符串转为long类型")
    method toDate (line 132) | @Comment("将字符串转为date类型")
    method unescape (line 138) | @Comment("反转义字符串")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/CommentExecutor.java
  class CommentExecutor (line 10) | @Component
    method execute (line 13) | @Override
    method supportShape (line 18) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/ExecuteSQLExecutor.java
  class ExecuteSQLExecutor (line 31) | @Component
    method execute (line 56) | @Override
    method convertParameters (line 175) | private List<Object[]> convertParameters(Object[] params, int length) {
    method getValue (line 188) | private Object getValue(Object object, int index) {
    method supportShape (line 208) | @Override
    method grammers (line 213) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/ForkJoinExecutor.java
  class ForkJoinExecutor (line 15) | @Component
    method execute (line 23) | @Override
    method supportShape (line 27) | @Override
    method allowExecuteNext (line 32) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/FunctionExecutor.java
  class FunctionExecutor (line 21) | @Component
    method execute (line 28) | @Override
    method supportShape (line 45) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/LoopExecutor.java
  class LoopExecutor (line 15) | @Component
    method execute (line 24) | @Override
    method supportShape (line 28) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/OutputExecutor.java
  class OutputExecutor (line 31) | @Component
    method execute (line 59) | @Override
    method outputAll (line 114) | private void outputAll(SpiderOutput output,Map<String,Object> variables){
    method outputDB (line 138) | private void outputDB(String databaseId, String tableName, Map<String,...
    method outputCSV (line 164) | private void outputCSV(SpiderNode node, SpiderContext context, String ...
    method supportShape (line 204) | @Override
    method beforeStart (line 209) | @Override
    method afterEnd (line 214) | @Override
    method releasePrinters (line 219) | private void releasePrinters() {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/ProcessExecutor.java
  class ProcessExecutor (line 22) | @Component
    method execute (line 35) | @Override
    method supportShape (line 48) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/RequestExecutor.java
  class RequestExecutor (line 37) | @Component
    method supportShape (line 105) | @Override
    method init (line 110) | @PostConstruct
    method execute (line 116) | @Override
    method setRequestFormParameter (line 319) | private List<InputStream> setRequestFormParameter(SpiderNode node, Htt...
    method getRequestCookie (line 364) | private Map<String,String> getRequestCookie(SpiderNode node, List<Map<...
    method setRequestParameter (line 388) | private void setRequestParameter(SpiderNode node, HttpRequest request,...
    method setRequestHeader (line 408) | private void setRequestHeader(SpiderNode node,HttpRequest request, Lis...
    method grammers (line 428) | @Override
    method beforeStart (line 439) | @Override
    method createBloomFilter (line 444) | private BloomFilter<String> createBloomFilter(SpiderContext context){
    method afterEnd (line 465) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/StartExecutor.java
  class StartExecutor (line 15) | @Component
    method execute (line 18) | @Override
    method supportShape (line 23) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/VariableExecutor.java
  class VariableExecutor (line 20) | @Component
    method execute (line 29) | @Override
    method supportShape (line 48) | @Override
    method isThread (line 53) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/DefaultExpressionEngine.java
  class DefaultExpressionEngine (line 16) | @Component
    method init (line 25) | @PostConstruct
    method execute (line 32) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/ExpressionError.java
  class ExpressionError (line 9) | public class ExpressionError {
    method error (line 22) | public static void error (String message, TokenStream stream) {
    method error (line 36) | public static void error (String message, Span location, Throwable cau...
    method error (line 58) | public static void error (String message, Span location) {
    class TemplateException (line 64) | public static class TemplateException extends RuntimeException {
      method TemplateException (line 69) | private TemplateException (String message, Span location) {
      method TemplateException (line 75) | public TemplateException (String message, Span location, Throwable c...
      method getLocation (line 82) | public Span getLocation () {
      method getMessage (line 86) | @Override
    class StringLiteralException (line 113) | public static class StringLiteralException extends RuntimeException {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/ExpressionGlobalVariables.java
  class ExpressionGlobalVariables (line 8) | public class ExpressionGlobalVariables {
    method reset (line 14) | public static void reset(Map<String, String> map){
    method getVariables (line 25) | public static Map<String, String> getVariables(){

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/ExpressionTemplate.java
  class ExpressionTemplate (line 17) | public class ExpressionTemplate {
    method ExpressionTemplate (line 21) | private ExpressionTemplate (List<Node> nodes) {
    method create (line 25) | public static ExpressionTemplate create(String source){
    method getNodes (line 30) | public List<Node> getNodes () {
    method render (line 35) | public Object render (ExpressionTemplateContext context) {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/ExpressionTemplateContext.java
  class ExpressionTemplateContext (line 25) | public class ExpressionTemplateContext {
    method get (line 34) | public static ExpressionTemplateContext get(){
    method remove (line 38) | public static void remove(){
    method set (line 42) | public static void set(ExpressionTemplateContext context){
    method ExpressionTemplateContext (line 46) | public ExpressionTemplateContext () {
    method ExpressionTemplateContext (line 50) | public ExpressionTemplateContext(Map<String,Object> variables) {
    method set (line 59) | public ExpressionTemplateContext set (String name, Object value) {
    method setOnCurrentScope (line 74) | public ExpressionTemplateContext setOnCurrentScope (String name, Objec...
    method get (line 81) | public Object get (String name) {
    method getVariables (line 92) | public Set<String> getVariables () {
    method push (line 101) | public void push () {
    method pop (line 107) | public void pop () {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/interpreter/AstInterpreter.java
  class AstInterpreter (line 28) | public class AstInterpreter {
    method interpret (line 29) | public static Object interpret (ExpressionTemplate template, Expressio...
    method interpretNodeList (line 42) | public static Object interpretNodeList (List<Node> nodes, ExpressionTe...

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/interpreter/JavaReflection.java
  class JavaReflection (line 14) | public class JavaReflection extends Reflection {
    method getField (line 19) | @SuppressWarnings("rawtypes")
    method getFieldValue (line 57) | @Override
    method registerExtensionClass (line 67) | @Override
    method getExtensionMethod (line 89) | @Override
    method getExtensionMethod (line 98) | private Object getExtensionMethod(Class<?> cls, String name, Object......
    method getMethod (line 129) | @Override
    method findApply (line 185) | private static Method findApply (Class<?> cls) {
    method findMethod (line 192) | private static Method findMethod (List<Method> methods, Class<?>[] par...
    method findMethod (line 239) | private static Method findMethod (Class<?> cls, String name, Class<?>[...
    method isPrimitiveAssignableFrom (line 256) | private static boolean isPrimitiveAssignableFrom (Class<?> from, Class...
    method getStringTypes (line 268) | public static String[] getStringTypes(Object[] objects){
    method isCoercible (line 281) | private static boolean isCoercible (Class<?> from, Class<?> to) {
    method callMethod (line 320) | @Override
    class MethodSignature (line 331) | private static class MethodSignature {
      method MethodSignature (line 336) | @SuppressWarnings("rawtypes")
      method hashCode (line 347) | @Override
      method equals (line 352) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/interpreter/Reflection.java
  class Reflection (line 7) | public abstract class Reflection {
    method setInstance (line 11) | public synchronized static void setInstance (Reflection reflection) {
    method getInstance (line 16) | public synchronized static Reflection getInstance () {
    method getField (line 21) | public abstract Object getField (Object obj, String name);
    method getMethod (line 26) | public abstract Object getMethod (Object obj, String name, Object... a...
    method getExtensionMethod (line 28) | public abstract Object getExtensionMethod (Object obj, String name,Obj...
    method registerExtensionClass (line 30) | public abstract void registerExtensionClass(Class<?> target,Class<?> c...
    method getFieldValue (line 34) | public abstract Object getFieldValue (Object obj, Object field);
    method callMethod (line 38) | public abstract Object callMethod (Object obj, Object method, Object.....

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/Ast.java
  class Ast (line 24) | public abstract class Ast {
    class Node (line 28) | public abstract static class Node {
      method Node (line 31) | public Node (Span span) {
      method getSpan (line 36) | public Span getSpan () {
      method toString (line 40) | @Override
      method evaluate (line 45) | public abstract Object evaluate (ExpressionTemplate template, Expres...
    class Text (line 49) | public static class Text extends Node {
      method Text (line 52) | public Text (Span text) {
      method getContent (line 71) | public String getContent () {
      method evaluate (line 75) | @Override
    class Expression (line 83) | public abstract static class Expression extends Node {
      method Expression (line 84) | public Expression (Span span) {
    class UnaryOperation (line 90) | public static class UnaryOperation extends Expression {
      type UnaryOperator (line 92) | public static enum UnaryOperator {
        method getOperator (line 95) | public static UnaryOperator getOperator (Token op) {
      method UnaryOperation (line 113) | public UnaryOperation (Token operator, Expression operand) {
      method getOperator (line 119) | public UnaryOperator getOperator () {
      method getOperand (line 123) | public Expression getOperand () {
      method evaluate (line 127) | @Override
    class BinaryOperation (line 161) | public static class BinaryOperation extends Expression {
      type BinaryOperator (line 163) | public static enum BinaryOperator {
        method getOperator (line 166) | public static BinaryOperator getOperator (Token op) {
      method BinaryOperation (line 221) | public BinaryOperation (Expression leftOperand, Token operator, Expr...
      method getLeftOperand (line 228) | public Expression getLeftOperand () {
      method getOperator (line 232) | public BinaryOperator getOperator () {
      method getRightOperand (line 236) | public Expression getRightOperand () {
      method evaluateAddition (line 240) | private Object evaluateAddition (Object left, Object right) {
      method evaluateSubtraction (line 267) | private Object evaluateSubtraction (Object left, Object right) {
      method evaluateMultiplication (line 286) | private Object evaluateMultiplication (Object left, Object right) {
      method evaluateDivision (line 305) | private Object evaluateDivision (Object left, Object right) {
      method evaluateModulo (line 324) | private Object evaluateModulo (Object left, Object right) {
      method evaluateLess (line 343) | private boolean evaluateLess (Object left, Object right) {
      method evaluateLessEqual (line 362) | private Object evaluateLessEqual (Object left, Object right) {
      method evaluateGreater (line 381) | private Object evaluateGreater (Object left, Object right) {
      method evaluateGreaterEqual (line 400) | private Object evaluateGreaterEqual (Object left, Object right) {
      method evaluateAnd (line 419) | private Object evaluateAnd (Object left, ExpressionTemplate template...
      method evaluateOr (line 433) | private Object evaluateOr (Object left, ExpressionTemplate template,...
      method evaluateXor (line 447) | private Object evaluateXor (Object left, Object right) {
      method evaluateEqual (line 457) | private Object evaluateEqual (Object left, Object right) {
      method evaluateNotEqual (line 467) | private Object evaluateNotEqual (Object left, Object right) {
      method evaluate (line 471) | @Override
    class TernaryOperation (line 522) | public static class TernaryOperation extends Expression {
      method TernaryOperation (line 527) | public TernaryOperation (Expression condition, Expression trueExpres...
      method getCondition (line 534) | public Expression getCondition () {
      method getTrueExpression (line 538) | public Expression getTrueExpression () {
      method getFalseExpression (line 542) | public Expression getFalseExpression () {
      method evaluate (line 546) | @Override
    class NullLiteral (line 557) | public static class NullLiteral extends Expression {
      method NullLiteral (line 558) | public NullLiteral (Span span) {
      method evaluate (line 562) | @Override
    class BooleanLiteral (line 569) | public static class BooleanLiteral extends Expression {
      method BooleanLiteral (line 572) | public BooleanLiteral (Span literal) {
      method getValue (line 577) | public Boolean getValue () {
      method evaluate (line 581) | @Override
    class DoubleLiteral (line 588) | public static class DoubleLiteral extends Expression {
      method DoubleLiteral (line 591) | public DoubleLiteral (Span literal) {
      method getValue (line 596) | public Double getValue () {
      method evaluate (line 600) | @Override
    class FloatLiteral (line 607) | public static class FloatLiteral extends Expression {
      method FloatLiteral (line 610) | public FloatLiteral (Span literal) {
      method getValue (line 619) | public Float getValue () {
      method evaluate (line 623) | @Override
    class ByteLiteral (line 630) | public static class ByteLiteral extends Expression {
      method ByteLiteral (line 633) | public ByteLiteral (Span literal) {
      method getValue (line 638) | public Byte getValue () {
      method evaluate (line 642) | @Override
    class ShortLiteral (line 649) | public static class ShortLiteral extends Expression {
      method ShortLiteral (line 652) | public ShortLiteral (Span literal) {
      method getValue (line 657) | public Short getValue () {
      method evaluate (line 661) | @Override
    class IntegerLiteral (line 668) | public static class IntegerLiteral extends Expression {
      method IntegerLiteral (line 671) | public IntegerLiteral (Span literal) {
      method getValue (line 676) | public Integer getValue () {
      method evaluate (line 680) | @Override
    class LongLiteral (line 687) | public static class LongLiteral extends Expression {
      method LongLiteral (line 690) | public LongLiteral (Span literal) {
      method getValue (line 695) | public Long getValue () {
      method evaluate (line 699) | @Override
    class CharacterLiteral (line 706) | public static class CharacterLiteral extends Expression {
      method CharacterLiteral (line 709) | public CharacterLiteral (Span literal) {
      method getValue (line 733) | public Character getValue () {
      method evaluate (line 737) | @Override
    class StringLiteral (line 744) | public static class StringLiteral extends Expression {
      method StringLiteral (line 747) | public StringLiteral (Span literal) {
      method getValue (line 773) | public String getValue () {
      method evaluate (line 777) | @Override
    class VariableAccess (line 786) | public static class VariableAccess extends Expression {
      method VariableAccess (line 787) | public VariableAccess (Span name) {
      method getVariableName (line 791) | public Span getVariableName () {
      method evaluate (line 795) | @Override
    class MapOrArrayAccess (line 805) | public static class MapOrArrayAccess extends Expression {
      method MapOrArrayAccess (line 809) | public MapOrArrayAccess (Span span, Expression mapOrArray, Expressio...
      method getMapOrArray (line 816) | public Expression getMapOrArray () {
      method getKeyOrIndex (line 821) | public Expression getKeyOrIndex () {
      method evaluate (line 825) | @SuppressWarnings("rawtypes")
    class MemberAccess (line 877) | public static class MemberAccess extends Expression {
      method MemberAccess (line 882) | public MemberAccess (Expression object, Span name) {
      method getObject (line 889) | public Expression getObject () {
      method getName (line 894) | public Span getName () {
      method getCachedMember (line 900) | public Object getCachedMember () {
      method setCachedMember (line 908) | public void setCachedMember (Object cachedMember) {
      method evaluate (line 912) | @SuppressWarnings("rawtypes")
    class FunctionCall (line 985) | public static class FunctionCall extends Expression {
      method FunctionCall (line 991) | public FunctionCall (Span span, Expression function, List<Expression...
      method getFunction (line 999) | public Expression getFunction () {
      method getArguments (line 1004) | public List<Expression> getArguments () {
      method getCachedFunction (line 1010) | public Object getCachedFunction () {
      method setCachedFunction (line 1017) | public void setCachedFunction (Object cachedFunction) {
      method getCachedArguments (line 1023) | public Object[] getCachedArguments () {
      method clearCachedArguments (line 1033) | public void clearCachedArguments () {
      method evaluate (line 1040) | @Override
    class MethodCall (line 1101) | public static class MethodCall extends Expression {
      method MethodCall (line 1107) | public MethodCall (Span span, MemberAccess method, List<Expression> ...
      method getObject (line 1115) | public Expression getObject () {
      method getMethod (line 1120) | public MemberAccess getMethod () {
      method getArguments (line 1125) | public List<Expression> getArguments () {
      method getCachedMethod (line 1131) | public Object getCachedMethod () {
      method setCachedMethod (line 1138) | public void setCachedMethod (Object cachedMethod) {
      method getCachedArguments (line 1144) | public Object[] getCachedArguments () {
      method clearCachedArguments (line 1154) | public void clearCachedArguments () {
      method evaluate (line 1161) | @Override
    class MapLiteral (line 1258) | public static class MapLiteral extends Expression {
      method MapLiteral (line 1262) | public MapLiteral (Span span, List<Token> keys, List<Expression> val...
      method getKeys (line 1268) | public List<Token> getKeys () {
      method getValues (line 1272) | public List<Expression> getValues () {
      method evaluate (line 1276) | @Override
    class ListLiteral (line 1300) | public static class ListLiteral extends Expression {
      method ListLiteral (line 1303) | public ListLiteral (Span span, List<Expression> values) {
      method getValues (line 1308) | public List<Expression> getValues () {
      method evaluate (line 1312) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/CharacterStream.java
  class CharacterStream (line 8) | public class CharacterStream {
    method CharacterStream (line 15) | public CharacterStream (String source) {
    method CharacterStream (line 19) | public CharacterStream (String source, int start, int end) {
    method hasMore (line 31) | public boolean hasMore () {
    method peek (line 36) | public char peek () {
    method consume (line 42) | public char consume () {
    method match (line 49) | public boolean match (String needle, boolean consume) {
    method matchDigit (line 63) | public boolean matchDigit (boolean consume) {
    method matchIdentifierStart (line 75) | public boolean matchIdentifierStart (boolean consume) {
    method matchIdentifierPart (line 87) | public boolean matchIdentifierPart (boolean consume) {
    method skipWhiteSpace (line 98) | public void skipWhiteSpace () {
    method startSpan (line 112) | public void startSpan () {
    method endSpan (line 117) | public Span endSpan () {
    method isSpanEmpty (line 121) | public boolean isSpanEmpty () {
    method getPosition (line 126) | public int getPosition () {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/Parser.java
  class Parser (line 39) | public class Parser {
    method parse (line 42) | public static List<Node> parse (String source) {
    method parseStatement (line 53) | private static Node parseStatement (TokenStream tokens) {
    method parseExpression (line 69) | private static Expression parseExpression (TokenStream stream) {
    method parseTernaryOperator (line 73) | private static Expression parseTernaryOperator (TokenStream stream) {
    method parseBinaryOperator (line 90) | private static Expression parseBinaryOperator (TokenStream stream, int...
    method parseUnaryOperator (line 106) | private static Expression parseUnaryOperator (TokenStream stream) {
    method parseAccessOrCallOrLiteral (line 120) | private static Expression parseAccessOrCallOrLiteral (TokenStream stre...
    method parseMapLiteral (line 161) | private static Expression parseMapLiteral (TokenStream stream) {
    method parseListLiteral (line 181) | private static Expression parseListLiteral (TokenStream stream) {
    method parseAccessOrCall (line 194) | private static Expression parseAccessOrCall (TokenStream stream,TokenT...
    method parseArguments (line 233) | private static List<Expression> parseArguments (TokenStream stream) {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/Span.java
  class Span (line 5) | public class Span {
    method Span (line 18) | public Span (String source) {
    method Span (line 22) | public Span (String source, int start, int end) {
    method Span (line 35) | public Span (Span start, Span end) {
    method getText (line 49) | public String getText () {
    method getStart (line 54) | public int getStart () {
    method getEnd (line 59) | public int getEnd () {
    method getSource (line 64) | public String getSource () {
    method toString (line 68) | @Override
    method getLine (line 74) | public Line getLine () {
    class Line (line 112) | public static class Line {
      method Line (line 118) | public Line (String source, int start, int end, int lineNumber) {
      method getSource (line 125) | public String getSource () {
      method getStart (line 129) | public int getStart () {
      method getEnd (line 133) | public int getEnd () {
      method getLineNumber (line 137) | public int getLineNumber () {
      method getText (line 141) | public String getText () {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/Token.java
  class Token (line 5) | public class Token {
    method Token (line 9) | public Token (TokenType type, Span span) {
    method getType (line 14) | public TokenType getType () {
    method getSpan (line 18) | public Span getSpan () {
    method getText (line 22) | public String getText () {
    method toString (line 26) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/TokenStream.java
  class TokenStream (line 13) | public class TokenStream {
    method TokenStream (line 18) | public TokenStream (List<Token> tokens) {
    method hasMore (line 25) | public boolean hasMore () {
    method hasNext (line 29) | public boolean hasNext(){
    method hasPrev (line 33) | public boolean hasPrev(){
    method consume (line 38) | public Token consume () {
    method next (line 43) | public Token next(){
    method prev (line 48) | public Token prev(){
    method expect (line 57) | public Token expect (TokenType type) {
    method expect (line 74) | public Token expect (String text) {
    method match (line 90) | public boolean match (TokenType type, boolean consume) {
    method match (line 100) | public boolean match (String text, boolean consume) {
    method match (line 111) | public boolean match (boolean consume, TokenType... types) {
    method match (line 120) | public boolean match (boolean consume, String... tokenTexts) {
    method getSource (line 128) | public String getSource () {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/TokenType.java
  type TokenType (line 10) | public enum TokenType {
    method compare (line 63) | @Override
    method TokenType (line 76) | TokenType (String error) {
    method TokenType (line 81) | TokenType (String literal, String error) {
    method getLiteral (line 87) | public String getLiteral () {
    method getError (line 92) | public String getError () {
    method getSortedValues (line 98) | public static TokenType[] getSortedValues () {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/Tokenizer.java
  class Tokenizer (line 12) | public class Tokenizer {
    method tokenize (line 17) | public List<Token> tokenize (String source) {
    method tokenizeCodeSpan (line 58) | private static List<Token> tokenizeCodeSpan (Span span) {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/io/HttpRequest.java
  class HttpRequest (line 17) | public class HttpRequest {
    method create (line 21) | public static HttpRequest create(){
    method url (line 25) | public HttpRequest url(String url){
    method headers (line 32) | public HttpRequest headers(Map<String,String> headers){
    method header (line 37) | public HttpRequest header(String key,String value){
    method header (line 42) | public HttpRequest header(String key,Object value){
    method cookies (line 49) | public HttpRequest cookies(Map<String,String> cookies){
    method cookie (line 54) | public HttpRequest cookie(String name, String value) {
    method contentType (line 61) | public HttpRequest contentType(String contentType){
    method data (line 66) | public HttpRequest data(String key,String value){
    method data (line 71) | public HttpRequest data(String key,Object value){
    method data (line 78) | public HttpRequest data(String key,String filename,InputStream is){
    method data (line 83) | public HttpRequest data(Object body){
    method data (line 90) | public HttpRequest data(Map<String,String> data){
    method method (line 95) | public HttpRequest method(String method){
    method followRedirect (line 100) | public HttpRequest followRedirect(boolean followRedirects){
    method timeout (line 105) | public HttpRequest timeout(int timeout){
    method proxy (line 110) | public HttpRequest proxy(String host,int port){
    method validateTLSCertificates (line 115) | @SuppressWarnings("deprecation")
    method execute (line 121) | public HttpResponse execute() throws IOException{

FILE: spider-flow-core/src/main/java/org/spiderflow/core/io/HttpResponse.java
  class HttpResponse (line 16) | public class HttpResponse implements SpiderResponse{
    method HttpResponse (line 30) | public HttpResponse(Response response){
    method getStatusCode (line 37) | @Override
    method getTitle (line 42) | @Override
    method getHtml (line 52) | @Override
    method getJson (line 62) | @Override
    method getCookies (line 70) | @Override
    method getHeaders (line 75) | @Override
    method getBytes (line 80) | @Override
    method getContentType (line 85) | @Override
    method setCharset (line 90) | @Override
    method getUrl (line 95) | @Override
    method getStream (line 100) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/job/SpiderJob.java
  class SpiderJob (line 29) | @Component
    method executeInternal (line 51) | @Override
    method run (line 63) | public void run(String id) {
    method run (line 67) | public void run(SpiderFlow spiderFlow, Date nextExecuteTime) {
    method run (line 75) | public void run(SpiderFlow spiderFlow, Task task,Date nextExecuteTime) {
    method getSpiderContext (line 99) | public static SpiderContext getSpiderContext(Integer taskId) {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/job/SpiderJobContext.java
  class SpiderJobContext (line 14) | public class SpiderJobContext extends SpiderContext{
    method SpiderJobContext (line 26) | public SpiderJobContext(OutputStream outputstream,boolean output) {
    method close (line 32) | public void close(){
    method addOutput (line 39) | @Override
    method getOutputs (line 48) | @Override
    method getOutputstream (line 53) | public OutputStream getOutputstream(){
    method create (line 57) | public static SpiderJobContext create(String directory,String id,Integ...

FILE: spider-flow-core/src/main/java/org/spiderflow/core/job/SpiderJobManager.java
  class SpiderJobManager (line 26) | @Component
    method getJobKey (line 44) | private JobKey getJobKey(String id){
    method getTriggerKey (line 48) | private TriggerKey getTriggerKey(String id){
    method addJob (line 57) | public Date addJob(SpiderFlow spiderFlow){
    method run (line 73) | public void run(String id){
    method remove (line 79) | public boolean remove(String id){

FILE: spider-flow-core/src/main/java/org/spiderflow/core/mapper/DataSourceMapper.java
  type DataSourceMapper (line 10) | public interface DataSourceMapper extends BaseMapper<DataSource>{
    method selectAll (line 12) | @Select("select id,name from sp_datasource")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/mapper/FlowNoticeMapper.java
  type FlowNoticeMapper (line 7) | @Mapper

FILE: spider-flow-core/src/main/java/org/spiderflow/core/mapper/FunctionMapper.java
  type FunctionMapper (line 7) | @Mapper

FILE: spider-flow-core/src/main/java/org/spiderflow/core/mapper/SpiderFlowMapper.java
  type SpiderFlowMapper (line 21) | public interface SpiderFlowMapper extends BaseMapper<SpiderFlow>{
    method selectSpiderPage (line 23) | @Select({
    method insertSpiderFlow (line 37) | @Insert("insert into sp_flow(id,name,xml,enabled) values(#{id},#{name}...
    method updateSpiderFlow (line 40) | @Update("update sp_flow set name = #{name},xml = #{xml} where id = #{i...
    method resetExecuteCount (line 43) | @Update("update sp_flow set execute_count = 0 where id = #{id}")
    method executeCountIncrementAndExecuteTime (line 46) | @Update("update sp_flow set execute_count = ifnull(execute_count,0) + ...
    method executeCountIncrement (line 49) | @Update("update sp_flow set execute_count = ifnull(execute_count,0) + ...
    method resetCornExpression (line 52) | @Update("update sp_flow set cron = #{cron},next_execute_time = #{nextE...
    method resetSpiderStatus (line 55) | @Update("update sp_flow set enabled = #{enabled} where id = #{id}")
    method resetNextExecuteTime (line 58) | @Update("update sp_flow set next_execute_time = null where id = #{id}")
    method resetNextExecuteTime (line 61) | @Update("update sp_flow set next_execute_time = null")
    method selectFlows (line 64) | @Select("select id,name from sp_flow")
    method selectOtherFlows (line 67) | @Select("select id,name from sp_flow where id != #{id}")
    method getFlowMaxTaskId (line 70) | @Select("select max(a.id) from `sp_task` a left join sp_flow b on a.fl...
    method getCountById (line 73) | @Select("select COUNT(id) from sp_flow where id = #{id}")

FILE: spider-flow-core/src/main/java/org/spiderflow/core/mapper/TaskMapper.java
  type TaskMapper (line 7) | @Mapper

FILE: spider-flow-core/src/main/java/org/spiderflow/core/mapper/VariableMapper.java
  type VariableMapper (line 6) | public interface VariableMapper extends BaseMapper<Variable> {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/model/DataSource.java
  class DataSource (line 9) | @TableName("sp_datasource")
    method DataSource (line 27) | public DataSource() {
    method DataSource (line 30) | public DataSource(String id, String name) {
    method getId (line 35) | public String getId() {
    method setId (line 39) | public void setId(String id) {
    method getName (line 43) | public String getName() {
    method setName (line 47) | public void setName(String name) {
    method getDriverClassName (line 51) | public String getDriverClassName() {
    method setDriverClassName (line 55) | public void setDriverClassName(String driverClassName) {
    method getJdbcUrl (line 59) | public String getJdbcUrl() {
    method setJdbcUrl (line 63) | public void setJdbcUrl(String jdbcUrl) {
    method getUsername (line 67) | public String getUsername() {
    method setUsername (line 71) | public void setUsername(String username) {
    method getPassword (line 75) | public String getPassword() {
    method setPassword (line 79) | public void setPassword(String password) {
    method getCreateDate (line 83) | public Date getCreateDate() {
    method setCreateDate (line 87) | public void setCreateDate(Date createDate) {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/model/FlowNotice.java
  class FlowNotice (line 16) | @TableName("sp_flow_notice")
    method getId (line 49) | public String getId() {
    method setId (line 53) | public void setId(String id) {
    method getRecipients (line 57) | public String getRecipients() {
    method setRecipients (line 61) | public void setRecipients(String recipients) {
    method getNoticeWay (line 65) | public String getNoticeWay() {
    method setNoticeWay (line 69) | public void setNoticeWay(String noticeWay) {
    method getStartNotice (line 73) | public String getStartNotice() {
    method setStartNotice (line 77) | public void setStartNotice(String startNotice) {
    method getExceptionNotice (line 81) | public String getExceptionNotice() {
    method setExceptionNotice (line 85) | public void setExceptionNotice(String exceptionNotice) {
    method getEndNotice (line 89) | public String getEndNotice() {
    method setEndNotice (line 93) | public void setEndNotice(String endNotice) {
    method judgeStartNotice (line 97) | public boolean judgeStartNotice() {
    method judgeExceptionNotice (line 104) | public boolean judgeExceptionNotice() {
    method judgeEndNotice (line 111) | public boolean judgeEndNotice() {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/model/Function.java
  class Function (line 9) | @TableName("sp_function")
    method getId (line 23) | public String getId() {
    method setId (line 27) | public void setId(String id) {
    method getName (line 31) | public String getName() {
    method setName (line 35) | public void setName(String name) {
    method getParameter (line 39) | public String getParameter() {
    method setParameter (line 43) | public void setParameter(String parameter) {
    method getScript (line 47) | public String getScript() {
    method setScript (line 51) | public void setScript(String script) {
    method getCreateDate (line 55) | public Date getCreateDate() {
    method setCreateDate (line 59) | public void setCreateDate(Date createDate) {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/model/SpiderFlow.java
  class SpiderFlow (line 13) | @TableName("sp_flow")
    method SpiderFlow (line 48) | public SpiderFlow() {
    method SpiderFlow (line 51) | public SpiderFlow(String id, String name) {
    method getId (line 56) | public String getId() {
    method setId (line 60) | public void setId(String id) {
    method getName (line 64) | public String getName() {
    method setName (line 68) | public void setName(String name) {
    method getXml (line 72) | public String getXml() {
    method setXml (line 76) | public void setXml(String xml) {
    method getCron (line 80) | public String getCron() {
    method setCron (line 84) | public void setCron(String cron) {
    method getEnabled (line 88) | public String getEnabled() {
    method setEnabled (line 92) | public void setEnabled(String enabled) {
    method getCreateDate (line 96) | public Date getCreateDate() {
    method setCreateDate (line 100) | public void setCreateDate(Date createDate) {
    method getLastExecuteTime (line 104) | public Date getLastExecuteTime() {
    method setLastExecuteTime (line 108) | public void setLastExecuteTime(Date lastExecuteTime) {
    method getNextExecuteTime (line 112) | public Date getNextExecuteTime() {
    method setNextExecuteTime (line 116) | public void setNextExecuteTime(Date nextExecuteTime) {
    method getExecuteCount (line 120) | public Integer getExecuteCount() {
    method setExecuteCount (line 124) | public void setExecuteCount(Integer executeCount) {
    method getRunning (line 128) | public Integer getRunning() {
    method setRunning (line 132) | public void setRunning(Integer running) {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/model/Task.java
  class Task (line 9) | @TableName("sp_task")
    method getId (line 21) | public Integer getId() {
    method setId (line 25) | public void setId(Integer id) {
    method getFlowId (line 29) | public String getFlowId() {
    method setFlowId (line 33) | public void setFlowId(String flowId) {
    method getBeginTime (line 37) | public Date getBeginTime() {
    method setBeginTime (line 41) | public void setBeginTime(Date beginTime) {
    method getEndTime (line 45) | public Date getEndTime() {
    method setEndTime (line 49) | public void setEndTime(Date endTime) {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/model/Variable.java
  class Variable (line 9) | @TableName("sp_variable")
    method getId (line 23) | public Integer getId() {
    method setId (line 27) | public void setId(Integer id) {
    method getName (line 31) | public String getName() {
    method setName (line 35) | public void setName(String name) {
    method getValue (line 39) | public String getValue() {
    method setValue (line 43) | public void setValue(String value) {
    method getDescription (line 47) | public String getDescription() {
    method setDescription (line 51) | public void setDescription(String description) {
    method getCreateDate (line 55) | public Date getCreateDate() {
    method setCreateDate (line 59) | public void setCreateDate(Date createDate) {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/script/ScriptManager.java
  class ScriptManager (line 19) | public class ScriptManager {
    method setScriptEngine (line 29) | public static void setScriptEngine(ScriptEngine engine){
    method clearFunctions (line 48) | public static void clearFunctions(){
    method createEngine (line 52) | public static ScriptEngine createEngine(){
    method lock (line 56) | public static void lock(){
    method unlock (line 60) | public static void unlock(){
    method registerFunction (line 64) | public static void registerFunction(ScriptEngine engine,String functio...
    method concatScript (line 74) | private static String concatScript(String functionName,String paramete...
    method containsFunction (line 86) | public static boolean containsFunction(String functionName){
    method validScript (line 95) | public static void validScript(String functionName,String parameters,S...
    method eval (line 99) | public static Object eval(ExpressionTemplateContext context, String fu...
    method convertObject (line 118) | private static Object convertObject(Object object){

FILE: spider-flow-core/src/main/java/org/spiderflow/core/serializer/FastJsonSerializer.java
  class FastJsonSerializer (line 16) | public class FastJsonSerializer implements ObjectSerializer {
    method write (line 29) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/service/DataSourceService.java
  class DataSourceService (line 10) | @Service

FILE: spider-flow-core/src/main/java/org/spiderflow/core/service/FlowNoticeService.java
  class FlowNoticeService (line 29) | @Service
    method saveOrUpdate (line 46) | @Override
    method sendFlowNotice (line 62) | public void sendFlowNotice(SpiderFlow spiderFlow, FlowNoticeType type) {
    method getCurrentDate (line 127) | private String getCurrentDate() {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/service/FunctionService.java
  class FunctionService (line 16) | @Service
    method init (line 24) | @PostConstruct
    method saveFunction (line 39) | public String saveFunction(Function entity) {
    method removeById (line 51) | @Override

FILE: spider-flow-core/src/main/java/org/spiderflow/core/service/SpiderFlowService.java
  class SpiderFlowService (line 36) | @Service
    method initJobs (line 54) | @PostConstruct
    method selectSpiderPage (line 73) | public IPage<SpiderFlow> selectSpiderPage(Page<SpiderFlow> page, Strin...
    method executeCountIncrement (line 77) | public int executeCountIncrement(String id, Date lastExecuteTime, Date...
    method resetCornExpression (line 90) | public void resetCornExpression(String id, String cron){
    method save (line 103) | @Override
    method stop (line 134) | public void stop(String id){
    method copy (line 140) | public void copy(String id){
    method start (line 147) | public void start(String id){
    method run (line 158) | public void run(String id){
    method resetExecuteCount (line 162) | public void resetExecuteCount(String id){
    method remove (line 165) | public void remove(String id){
    method selectOtherFlows (line 171) | public List<SpiderFlow> selectOtherFlows(String id){
    method selectFlows (line 175) | public List<SpiderFlow> selectFlows(){
    method getRecentTriggerTime (line 185) | public List<String> getRecentTriggerTime(String cron,int numTimes) {
    method historyList (line 204) | public List<Long> historyList(String id){
    method readHistory (line 215) | public String readHistory(String id,String timestamp){
    method getFlowMaxTaskId (line 227) | public Integer getFlowMaxTaskId(String flowId){

FILE: spider-flow-core/src/main/java/org/spiderflow/core/service/TaskService.java
  class TaskService (line 8) | @Service

FILE: spider-flow-core/src/main/java/org/spiderflow/core/service/VariableService.java
  class VariableService (line 14) | @Service
    method removeById (line 17) | @Override
    method saveOrUpdate (line 24) | @Override
    method resetGlobalVariables (line 31) | @PostConstruct

FILE: spider-flow-core/src/main/java/org/spiderflow/core/utils/DataSourceUtils.java
  class DataSourceUtils (line 19) | @Component
    method createDataSource (line 26) | public static DataSource createDataSource(String className,String url,...
    method remove (line 38) | public static void remove(String dataSourceId){
    method getDataSource (line 47) | public synchronized static DataSource getDataSource(String dataSourceId){
    method setDataSourceService (line 59) | @Autowired

FILE: spider-flow-core/src/main/java/org/spiderflow/core/utils/EmailUtils.java
  class EmailUtils (line 15) | @Component
    method sendSimpleMail (line 34) | public void sendSimpleMail(String subject, String content, String... t...

FILE: spider-flow-core/src/main/java/org/spiderflow/core/utils/ExecutorsUtils.java
  class ExecutorsUtils (line 21) | @Component
    method ExecutorsUtils (line 33) | @Autowired
    method init (line 38) | @PostConstruct
    method setApplicationContext (line 43) | @Override
    method shapes (line 48) | public static List<Shape> shapes(){
    method get (line 52) | public static ShapeExecutor get(String shape){

FILE: spider-flow-core/src/main/java/org/spiderflow/core/utils/ExpressionUtils.java
  class ExpressionUtils (line 19) | @Component
    method ExpressionUtils (line 29) | @Autowired
    method executeCondition (line 34) | public static boolean executeCondition(SpiderNode fromNode, SpiderNode...
    method execute (line 55) | public static Object execute(String expression, Map<String, Object> va...

FILE: spider-flow-core/src/main/java/org/spiderflow/core/utils/ExtractUtils.java
  class ExtractUtils (line 21) | public class ExtractUtils {
    method compile (line 25) | private static Pattern compile(String regx){
    method getMatchers (line 34) | public static List<String> getMatchers(String content,String regx,bool...
    method getMatchers (line 38) | public static List<String> getMatchers(String content,String regx,int ...
    method getMatchers (line 47) | public static List<List<String>> getMatchers(String content,String reg...
    method getFirstMatcher (line 60) | public static String getFirstMatcher(String content,String regx,boolea...
    method getFirstMatcher (line 65) | public static String getFirstMatcher(String content,String regx,int gr...
    method getFirstMatcher (line 73) | public static List<String> getFirstMatcher(String content,String regx,...
    method getHostFromURL (line 84) | public static String getHostFromURL(String url){
    method getFirstHTMLBySelector (line 88) | public static String getFirstHTMLBySelector(Element element,String sel...
    method getFirstOuterHTMLBySelector (line 93) | public static String getFirstOuterHTMLBySelector(Element element,Strin...
    method getFirstTextBySelector (line 98) | public static String getFirstTextBySelector(Element element,String sel...
    method getFirstAttrBySelector (line 103) | public static String getFirstAttrBySelector(Element element,String sel...
    method getFirstElement (line 108) | public static Element getFirstElement(Element element,String selector){
    method getElements (line 112) | public static List<Element> getElements(Element element,String selector){
    method getHTMLBySelector (line 116) | public static List<String> getHTMLBySelector(Element element,String se...
    method getOuterHTMLBySelector (line 125) | public static List<String> getOuterHTMLBySelector(Element element,Stri...
    method getTextBySelector (line 134) | public static List<String> getTextBySelector(Element element,String se...
    method getAttrBySelector (line 143) | public static List<String> getAttrBySelector(Element element,String se...
    method getValueByJsonPath (line 152) | public static Object getValueByJsonPath(Object root,String jsonPath){
    method getValuesByXPath (line 156) | public static List<String> getValuesByXPath(Element element,String xpa...
    method getValuesByXPath (line 160) | public static List<String> getValuesByXPath(Elements elements,String x...
    method getValueByXPath (line 164) | public static String getValueByXPath(Element element,String xpath){
    method getValueByXPath (line 168) | public static String getValueByXPath(Elements elements,String xpath){
    method getElementByXPath (line 172) | public static String getElementByXPath(Element element,String xpath){
    method isNumber (line 176) | public static boolean isNumber(String str) {

FILE: spider-flow-core/src/main/java/org/spiderflow/core/utils/FileUtils.java
  class FileUtils (line 15) | public class FileUtils
    method writeBytes (line 28) | public static void writeBytes(String filePath, OutputStream os) throws...
    method deleteFile (line 83) | public static boolean deleteFile(String filePath)
    method isValidFilename (line 102) | public static boolean isValidFilename(String filename)
    method setFileDownloadHeader (line 114) | public static String setFileDownloadHeader(HttpServletRequest request,...
    type DownloadStatus (line 146) | public enum DownloadStatus {
      method DownloadStatus (line 157) | DownloadStatus(int code, String name){
      method getCode (line 162) | public int getCode() {
      method setCode (line 166) | public void setCode(int code) {
      method getName (line 170) | public String getName() {
      method setName (line 174) | public void setName(String name) {
    method downloadFile (line 179) | public static DownloadStatus downloadFile(String savePath, String file...

FILE: spider-flow-core/src/main/java/org/spiderflow/core/utils/SpiderFlowUtils.java
  class SpiderFlowUtils (line 23) | public class SpiderFlowUtils {
    method loadXMLFromString (line 30) | public static SpiderNode loadXMLFromString(String xmlString){
    method getSpiderFlowJsonProperty (line 79) | @SuppressWarnings("unchecked")

FILE: spider-flow-web/src/main/java/org/spiderflow/SpiderApplication.java
  class SpiderApplication (line 17) | @SpringBootApplication
    method main (line 22) | public static void main(String[] args) throws IOException {
    method onStartup (line 27) | @Override
    method paginationInterceptor (line 33) | @Bean

FILE: spider-flow-web/src/main/java/org/spiderflow/configuration/ResourcesConfiguration.java
  class ResourcesConfiguration (line 13) | @Configuration
    method addResourceHandlers (line 16) | @Override
    method addCorsMappings (line 21) | @Override

FILE: spider-flow-web/src/main/java/org/spiderflow/configuration/WebSocketConfiguration.java
  class WebSocketConfiguration (line 15) | @Configuration
    method endpointExporter (line 18) | @Bean
    method setSpider (line 23) | @Autowired

FILE: spider-flow-web/src/main/java/org/spiderflow/controller/DataSourceController.java
  class DataSourceController (line 21) | @RestController
    method list (line 28) | @RequestMapping("/list")
    method all (line 33) | @RequestMapping("/all")
    method save (line 38) | @RequestMapping("/save")
    method get (line 47) | @RequestMapping("/get")
    method remove (line 54) | @RequestMapping("/remove")
    method test (line 60) | @RequestMapping("/test")

FILE: spider-flow-web/src/main/java/org/spiderflow/controller/FlowNoticeController.java
  class FlowNoticeController (line 16) | @RestController
    method save (line 24) | @RequestMapping("/save")
    method find (line 38) | @RequestMapping("/find")
    method getNoticeWay (line 48) | @RequestMapping("/getNoticeWay")

FILE: spider-flow-web/src/main/java/org/spiderflow/controller/FunctionController.java
  class FunctionController (line 15) | @RestController
    method list (line 22) | @RequestMapping("/list")
    method save (line 32) | @RequestMapping("/save")
    method get (line 37) | @RequestMapping("/get")
    method remove (line 42) | @RequestMapping("/remove")

FILE: spider-flow-web/src/main/java/org/spiderflow/controller/SpiderFlowController.java
  class SpiderFlowController (line 48) | @RestController
    method init (line 74) | @PostConstruct
    method list (line 103) | @RequestMapping("/list")
    method save (line 108) | @RequestMapping("/save")
    method history (line 114) | @RequestMapping("/history")
    method get (line 123) | @RequestMapping("/get")
    method other (line 128) | @RequestMapping("/other")
    method remove (line 136) | @RequestMapping("/remove")
    method start (line 141) | @RequestMapping("/start")
    method stop (line 146) | @RequestMapping("/stop")
    method copy (line 151) | @RequestMapping("/copy")
    method run (line 156) | @RequestMapping("/run")
    method cron (line 161) | @RequestMapping("/cron")
    method xml (line 166) | @RequestMapping("/xml")
    method download (line 171) | @RequestMapping("/log/download")
    method log (line 184) | @RequestMapping("/log")
    method shapes (line 201) | @RequestMapping("/shapes")
    method pluginConfigs (line 206) | @RequestMapping("/pluginConfigs")
    method grammers (line 211) | @RequestMapping("/grammers")
    method getRecent5TriggerTime (line 216) | @GetMapping("/recent5TriggerTime")

FILE: spider-flow-web/src/main/java/org/spiderflow/controller/SpiderRestController.java
  class SpiderRestController (line 26) | @RestController
    method runAsync (line 52) | @RequestMapping("/runAsync/{id}")
    method stop (line 72) | @RequestMapping("/stop/{taskId}")
    method status (line 87) | @RequestMapping("/status/{taskId}")
    method run (line 102) | @RequestMapping("/run/{id}")

FILE: spider-flow-web/src/main/java/org/spiderflow/controller/TaskController.java
  class TaskController (line 16) | @RestController
    method list (line 23) | @RequestMapping("/list")
    method stop (line 33) | @RequestMapping("/stop")
    method remove (line 42) | @RequestMapping("/remove")

FILE: spider-flow-web/src/main/java/org/spiderflow/controller/VariableController.java
  class VariableController (line 10) | @RestController

FILE: spider-flow-web/src/main/java/org/spiderflow/logback/SpiderFlowFileAppender.java
  class SpiderFlowFileAppender (line 14) | public class SpiderFlowFileAppender extends FileAppender<ILoggingEvent> {
    method subAppend (line 16) | @Override
    method writeBytes (line 37) | private void writeBytes(OutputStream os, byte[] byteArray) throws IOEx...

FILE: spider-flow-web/src/main/java/org/spiderflow/logback/SpiderFlowWebSocketAppender.java
  class SpiderFlowWebSocketAppender (line 16) | public class SpiderFlowWebSocketAppender extends UnsynchronizedAppenderB...
    method append (line 18) | @Override

FILE: spider-flow-web/src/main/java/org/spiderflow/model/SpiderWebSocketContext.java
  class SpiderWebSocketContext (line 16) | public class SpiderWebSocketContext extends SpiderContext {
    method SpiderWebSocketContext (line 26) | public SpiderWebSocketContext(Session session) {
    method isDebug (line 30) | public boolean isDebug() {
    method setDebug (line 34) | public void setDebug(boolean debug) {
    method addOutput (line 38) | @Override
    method log (line 43) | public void log(SpiderLog log) {
    method write (line 47) | public <T> void write(WebSocketEvent<T> event) {
    method pause (line 59) | @Override
    method resume (line 76) | @Override
    method stop (line 85) | @Override
    class DebugInfo (line 94) | class DebugInfo{
      method DebugInfo (line 104) | public DebugInfo(String nodeId, String event, String key, Object val...
      method getNodeId (line 111) | public String getNodeId() {
      method setNodeId (line 115) | public void setNodeId(String nodeId) {
      method getEvent (line 119) | public String getEvent() {
      method setEvent (line 123) | public void setEvent(String event) {
      method getKey (line 127) | public String getKey() {
      method setKey (line 131) | public void setKey(String key) {
      method getValue (line 135) | public Object getValue() {
      method setValue (line 139) | public void setValue(Object value) {

FILE: spider-flow-web/src/main/java/org/spiderflow/model/WebSocketEvent.java
  class WebSocketEvent (line 9) | public class WebSocketEvent<T> {
    method getTimestamp (line 17) | public String getTimestamp() {
    method setTimestamp (line 21) | public void setTimestamp(String timestamp) {
    method WebSocketEvent (line 25) | public WebSocketEvent(String eventType, T message) {
    method WebSocketEvent (line 30) | public WebSocketEvent(String eventType, String timestamp, T message) {
    method getEventType (line 36) | public String getEventType() {
    method setEventType (line 40) | public void setEventType(String eventType) {
    method getMessage (line 44) | public T getMessage() {
    method setMessage (line 48) | public void setMessage(T message) {

FILE: spider-flow-web/src/main/java/org/spiderflow/websocket/WebSocketEditorServer.java
  class WebSocketEditorServer (line 21) | @ServerEndpoint("/ws")
    method onMessage (line 29) | @OnMessage
    method onClose (line 56) | @OnClose

FILE: spider-flow-web/src/main/resources/static/js/canvas-viewer.js
  function CanvasText (line 8) | function CanvasText(options){
  function CanvasViewer (line 17) | function CanvasViewer(options){

FILE: spider-flow-web/src/main/resources/static/js/codemirror/codemirror.js
  function classTest (line 50) | function classTest(cls) { return new RegExp("(^|\\s)" + cls + "(?:$|\\s)...
  function removeChildren (line 61) | function removeChildren(e) {
  function removeChildrenAndAdd (line 67) | function removeChildrenAndAdd(parent, e) {
  function elt (line 71) | function elt(tag, content, className, style) {
  function eltP (line 80) | function eltP(tag, content, className, style) {
  function contains (line 103) | function contains(parent, child) {
  function activeElt (line 114) | function activeElt() {
  function addClass (line 129) | function addClass(node, cls) {
  function joinClasses (line 133) | function joinClasses(a, b) {
  function bind (line 146) | function bind(f) {
  function copyObj (line 151) | function copyObj(obj, target, overwrite) {
  function countColumn (line 161) | function countColumn(string, end, tabSize, startIndex, startValue) {
  function indexOf (line 182) | function indexOf(array, elt) {
  function findColumn (line 200) | function findColumn(string, goal, tabSize) {
  function spaceStr (line 215) | function spaceStr(n) {
  function lst (line 221) | function lst(arr) { return arr[arr.length-1] }
  function map (line 223) | function map(array, f) {
  function insertSorted (line 229) | function insertSorted(array, value, score) {
  function nothing (line 235) | function nothing() {}
  function createObj (line 237) | function createObj(base, props) {
  function isWordCharBasic (line 250) | function isWordCharBasic(ch) {
  function isWordChar (line 254) | function isWordChar(ch, helper) {
  function isEmpty (line 260) | function isEmpty(obj) {
  function isExtendingChar (line 271) | function isExtendingChar(ch) { return ch.charCodeAt(0) >= 768 && extendi...
  function skipExtendingChars (line 274) | function skipExtendingChars(str, pos, dir) {
  function findFirst (line 282) | function findFirst(pred, from, to) {
  function iterateBidiSections (line 297) | function iterateBidiSections(order, from, to, f) {
  function getBidiPartAt (line 311) | function getBidiPartAt(order, ch, sticky) {
  function charType (line 357) | function charType(code) {
  function BidiSpan (line 370) | function BidiSpan(level, from, to) {
  function getOrder (line 502) | function getOrder(line, direction) {
  function getHandlers (line 526) | function getHandlers(emitter, type) {
  function off (line 530) | function off(emitter, type, f) {
  function signal (line 545) | function signal(emitter, type /*, values...*/) {
  function signalDOMEvent (line 555) | function signalDOMEvent(cm, e, override) {
  function signalCursorActivity (line 562) | function signalCursorActivity(cm) {
  function hasHandler (line 570) | function hasHandler(emitter, type) {
  function eventMixin (line 576) | function eventMixin(ctor) {
  function e_preventDefault (line 584) | function e_preventDefault(e) {
  function e_stopPropagation (line 588) | function e_stopPropagation(e) {
  function e_defaultPrevented (line 592) | function e_defaultPrevented(e) {
  function e_stop (line 595) | function e_stop(e) {e_preventDefault(e); e_stopPropagation(e);}
  function e_target (line 597) | function e_target(e) {return e.target || e.srcElement}
  function e_button (line 598) | function e_button(e) {
  function zeroWidthElement (line 619) | function zeroWidthElement(measure) {
  function hasBadBidiRects (line 634) | function hasBadBidiRects(measure) {
  function hasBadZoomedRects (line 683) | function hasBadZoomedRects(measure) {
  function defineMode (line 697) | function defineMode(name, mode) {
  function defineMIME (line 703) | function defineMIME(mime, spec) {
  function resolveMode (line 709) | function resolveMode(spec) {
  function getMode (line 728) | function getMode(options, spec) {
  function extendMode (line 752) | function extendMode(mode, properties) {
  function copyState (line 757) | function copyState(mode, state) {
  function innerMode (line 771) | function innerMode(mode, state) {
  function startState (line 782) | function startState(mode, a1, a2) {
  function getLine (line 874) | function getLine(doc, n) {
  function getBetween (line 890) | function getBetween(doc, start, end) {
  function getLines (line 902) | function getLines(doc, from, to) {
  function updateLineHeight (line 910) | function updateLineHeight(line, height) {
  function lineNo (line 917) | function lineNo(line) {
  function lineAtHeight (line 931) | function lineAtHeight(chunk, h) {
  function isLine (line 951) | function isLine(doc, l) {return l >= doc.first && l < doc.first + doc.size}
  function lineNumberFor (line 953) | function lineNumberFor(options, i) {
  function Pos (line 958) | function Pos(line, ch, sticky) {
  function cmp (line 969) | function cmp(a, b) { return a.line - b.line || a.ch - b.ch }
  function equalCursorPos (line 971) | function equalCursorPos(a, b) { return a.sticky == b.sticky && cmp(a, b)...
  function copyPos (line 973) | function copyPos(x) {return Pos(x.line, x.ch)}
  function maxPos (line 974) | function maxPos(a, b) { return cmp(a, b) < 0 ? b : a }
  function minPos (line 975) | function minPos(a, b) { return cmp(a, b) < 0 ? a : b }
  function clipLine (line 979) | function clipLine(doc, n) {return Math.max(doc.first, Math.min(n, doc.fi...
  function clipPos (line 980) | function clipPos(doc, pos) {
  function clipToLen (line 986) | function clipToLen(pos, linelen) {
  function clipPosArray (line 992) | function clipPosArray(doc, array) {
  function highlightLine (line 1051) | function highlightLine(cm, line, context, forceToEnd) {
  function getLineStyles (line 1096) | function getLineStyles(cm, line, updateFrontier) {
  function getContextBefore (line 1112) | function getContextBefore(cm, n, precise) {
  function processLine (line 1132) | function processLine(cm, text, context, startAt) {
  function callBlankLine (line 1143) | function callBlankLine(mode, state) {
  function readToken (line 1150) | function readToken(mode, stream, state, inner) {
  function takeToken (line 1167) | function takeToken(cm, pos, precise, asArray) {
  function extractLineClasses (line 1181) | function extractLineClasses(type, output) {
  function runMode (line 1196) | function runMode(cm, text, mode, context, f, lineClasses, forceToEnd) {
  function findStartLine (line 1240) | function findStartLine(cm, n, precise) {
  function retreatFrontier (line 1257) | function retreatFrontier(doc, n) {
  function seeReadOnlySpans (line 1277) | function seeReadOnlySpans() {
  function seeCollapsedSpans (line 1281) | function seeCollapsedSpans() {
  function MarkedSpan (line 1287) | function MarkedSpan(marker, from, to) {
  function getMarkedSpanFor (line 1293) | function getMarkedSpanFor(spans, marker) {
  function removeMarkedSpan (line 1301) | function removeMarkedSpan(spans, span) {
  function addMarkedSpan (line 1308) | function addMarkedSpan(line, span) {
  function markedSpansBefore (line 1317) | function markedSpansBefore(old, startCh, isInsert) {
  function markedSpansAfter (line 1329) | function markedSpansAfter(old, endCh, isInsert) {
  function stretchSpansOverChange (line 1349) | function stretchSpansOverChange(doc, change) {
  function clearEmptySpans (line 1411) | function clearEmptySpans(spans) {
  function removeReadOnlyRanges (line 1422) | function removeReadOnlyRanges(doc, from, to) {
  function detachMarkedSpans (line 1451) | function detachMarkedSpans(line) {
  function attachMarkedSpans (line 1458) | function attachMarkedSpans(line, spans) {
  function extraLeft (line 1467) | function extraLeft(marker) { return marker.inclusiveLeft ? -1 : 0 }
  function extraRight (line 1468) | function extraRight(marker) { return marker.inclusiveRight ? 1 : 0 }
  function compareCollapsedMarkers (line 1473) | function compareCollapsedMarkers(a, b) {
  function collapsedSpanAtSide (line 1486) | function collapsedSpanAtSide(line, start) {
  function collapsedSpanAtStart (line 1496) | function collapsedSpanAtStart(line) { return collapsedSpanAtSide(line, t...
  function collapsedSpanAtEnd (line 1497) | function collapsedSpanAtEnd(line) { return collapsedSpanAtSide(line, fal...
  function collapsedSpanAround (line 1499) | function collapsedSpanAround(line, ch) {
  function conflictingCollapsedRange (line 1512) | function conflictingCollapsedRange(doc, lineNo$$1, from, to, marker) {
  function visualLine (line 1532) | function visualLine(line) {
  function visualLineEnd (line 1539) | function visualLineEnd(line) {
  function visualLineContinued (line 1548) | function visualLineContinued(line) {
  function visualLineNo (line 1559) | function visualLineNo(doc, lineN) {
  function visualLineEndNo (line 1567) | function visualLineEndNo(doc, lineN) {
  function lineIsHidden (line 1579) | function lineIsHidden(doc, line) {
  function lineIsHiddenInner (line 1590) | function lineIsHiddenInner(doc, line, span) {
  function heightAtLine (line 1607) | function heightAtLine(lineObj) {
  function lineLength (line 1629) | function lineLength(line) {
  function findMaxLine (line 1648) | function findMaxLine(cm) {
  function updateLine (line 1678) | function updateLine(line, text, markedSpans, estimateHeight) {
  function cleanUpLine (line 1690) | function cleanUpLine(line) {
  function interpretTokenStyle (line 1699) | function interpretTokenStyle(style, options) {
  function buildLineContent (line 1711) | function buildLineContent(cm, lineView) {
  function defaultSpecialCharPlaceholder (line 1769) | function defaultSpecialCharPlaceholder(ch) {
  function buildToken (line 1778) | function buildToken(builder, text, style, startStyle, endStyle, css, att...
  function splitSpaces (line 1845) | function splitSpaces(text, trailingBefore) {
  function buildTokenBadBidi (line 1860) | function buildTokenBadBidi(inner, order) {
  function buildCollapsedSpan (line 1880) | function buildCollapsedSpan(builder, size, marker, ignoreWidget) {
  function insertLineContent (line 1898) | function insertLineContent(line, builder, styles) {
  function LineView (line 1977) | function LineView(doc, line, lineN) {
  function buildViewArray (line 1989) | function buildViewArray(cm, from, to) {
  function pushOperation (line 2001) | function pushOperation(op) {
  function fireCallbacksForOps (line 2012) | function fireCallbacksForOps(group) {
  function finishOperation (line 2028) | function finishOperation(op, endCb) {
  function signalLater (line 2048) | function signalLater(emitter, type /*, values...*/) {
  function fireOrphanDelayed (line 2068) | function fireOrphanDelayed() {
  function updateLineForChanges (line 2077) | function updateLineForChanges(cm, lineView, lineN, dims) {
  function ensureLineWrapped (line 2090) | function ensureLineWrapped(lineView) {
  function updateLineBackground (line 2101) | function updateLineBackground(cm, lineView) {
  function getLineContent (line 2116) | function getLineContent(cm, lineView) {
  function updateLineText (line 2129) | function updateLineText(cm, lineView) {
  function updateLineClasses (line 2144) | function updateLineClasses(cm, lineView) {
  function updateLineGutter (line 2154) | function updateLineGutter(cm, lineView, lineN, dims) {
  function updateLineWidgets (line 2192) | function updateLineWidgets(cm, lineView, dims) {
  function buildLineElement (line 2203) | function buildLineElement(cm, lineView, lineN, dims) {
  function insertLineWidgets (line 2217) | function insertLineWidgets(cm, lineView, dims) {
  function insertLineWidgetsFor (line 2223) | function insertLineWidgetsFor(cm, line, lineView, dims, allowAbove) {
  function positionLineWidget (line 2239) | function positionLineWidget(widget, node, lineView, dims) {
  function widgetHeight (line 2257) | function widgetHeight(widget) {
  function eventInWidget (line 2273) | function eventInWidget(display, e) {
  function paddingTop (line 2283) | function paddingTop(display) {return display.lineSpace.offsetTop}
  function paddingVert (line 2284) | function paddingVert(display) {return display.mover.offsetHeight - displ...
  function paddingH (line 2285) | function paddingH(display) {
  function scrollGap (line 2294) | function scrollGap(cm) { return scrollerGap - cm.display.nativeBarWidth }
  function displayWidth (line 2295) | function displayWidth(cm) {
  function displayHeight (line 2298) | function displayHeight(cm) {
  function ensureLineHeights (line 2306) | function ensureLineHeights(cm, lineView, rect) {
  function mapFromLineView (line 2327) | function mapFromLineView(lineView, line, lineN) {
  function updateExternalMeasurement (line 2340) | function updateExternalMeasurement(cm, line) {
  function measureChar (line 2353) | function measureChar(cm, line, ch, bias) {
  function findViewForLine (line 2358) | function findViewForLine(cm, lineN) {
  function prepareMeasureForLine (line 2371) | function prepareMeasureForLine(cm, line) {
  function measureCharPrepared (line 2393) | function measureCharPrepared(cm, prepared, ch, bias, varHeight) {
  function nodeAndOffsetInLineMap (line 2415) | function nodeAndOffsetInLineMap(map$$1, ch, bias) {
  function getUsefulRect (line 2453) | function getUsefulRect(rects, bias) {
  function measureCharInner (line 2463) | function measureCharInner(cm, prepared, ch, bias) {
  function maybeUpdateRectForZooming (line 2516) | function maybeUpdateRectForZooming(measure, rect) {
  function clearLineMeasurementCacheFor (line 2526) | function clearLineMeasurementCacheFor(lineView) {
  function clearLineMeasurementCache (line 2535) | function clearLineMeasurementCache(cm) {
  function clearCaches (line 2542) | function clearCaches(cm) {
  function pageScrollX (line 2549) | function pageScrollX() {
  function pageScrollY (line 2556) | function pageScrollY() {
  function widgetTopHeight (line 2561) | function widgetTopHeight(lineObj) {
  function intoCoordSystem (line 2572) | function intoCoordSystem(cm, lineObj, rect, context, includeWidgets) {
  function fromCoordSystem (line 2594) | function fromCoordSystem(cm, coords, context) {
  function charCoords (line 2611) | function charCoords(cm, pos, context, lineObj, bias) {
  function cursorCoords (line 2632) | function cursorCoords(cm, pos, context, lineObj, preparedMeasure, varHei...
  function estimateCoords (line 2663) | function estimateCoords(cm, pos) {
  function PosWithInfo (line 2678) | function PosWithInfo(line, ch, sticky, outside, xRel) {
  function coordsChar (line 2687) | function coordsChar(cm, x, y) {
  function wrappedLineExtent (line 2707) | function wrappedLineExtent(cm, lineObj, preparedMeasure, y) {
  function wrappedLineExtentChar (line 2715) | function wrappedLineExtentChar(cm, lineObj, preparedMeasure, target) {
  function boxIsAfter (line 2723) | function boxIsAfter(box, x, y, left) {
  function coordsCharInner (line 2727) | function coordsCharInner(cm, lineObj, lineNo$$1, x, y) {
  function coordsBidiPart (line 2794) | function coordsBidiPart(cm, lineObj, lineNo$$1, preparedMeasure, order, ...
  function coordsBidiPartWrapped (line 2818) | function coordsBidiPartWrapped(cm, lineObj, _lineNo, preparedMeasure, or...
  function textHeight (line 2853) | function textHeight(display) {
  function charWidth (line 2873) | function charWidth(display) {
  function getDimensions (line 2885) | function getDimensions(cm) {
  function compensateForHScroll (line 2903) | function compensateForHScroll(display) {
  function estimateHeight (line 2910) | function estimateHeight(cm) {
  function estimateLineHeights (line 2928) | function estimateLineHeights(cm) {
  function posFromMouse (line 2941) | function posFromMouse(cm, e, liberal, forRect) {
  function findViewIndex (line 2959) | function findViewIndex(cm, n) {
  function regChange (line 2976) | function regChange(cm, from, to, lendiff) {
  function regLineChange (line 3041) | function regLineChange(cm, line, type) {
  function resetView (line 3055) | function resetView(cm) {
  function viewCuttingPoint (line 3061) | function viewCuttingPoint(cm, oldN, newN, dir) {
  function adjustView (line 3088) | function adjustView(cm, from, to) {
  function countDirtyView (line 3109) | function countDirtyView(cm) {
  function updateSelection (line 3118) | function updateSelection(cm) {
  function prepareSelection (line 3122) | function prepareSelection(cm, primary) {
  function drawSelectionCursor (line 3143) | function drawSelectionCursor(cm, head, output) {
  function cmpCoords (line 3161) | function cmpCoords(a, b) { return a.top - b.top || a.left - b.left }
  function drawSelectionRange (line 3164) | function drawSelectionRange(cm, range$$1, output) {
  function restartBlink (line 3257) | function restartBlink(cm) {
  function ensureFocus (line 3270) | function ensureFocus(cm) {
  function delayBlurEvent (line 3274) | function delayBlurEvent(cm) {
  function onFocus (line 3282) | function onFocus(cm, e) {
  function onBlur (line 3301) | function onBlur(cm, e) {
  function updateHeightsInViewport (line 3315) | function updateHeightsInViewport(cm) {
  function updateWidgetHeight (line 3354) | function updateWidgetHeight(line) {
  function visibleLines (line 3364) | function visibleLines(display, doc, viewport) {
  function maybeScrollWindow (line 3389) | function maybeScrollWindow(cm, rect) {
  function scrollPosIntoView (line 3406) | function scrollPosIntoView(cm, pos, end, margin) {
  function scrollIntoView (line 3440) | function scrollIntoView(cm, rect) {
  function calculateScrollPos (line 3450) | function calculateScrollPos(cm, rect) {
  function addToScrollTop (line 3480) | function addToScrollTop(cm, top) {
  function ensureCursorVisible (line 3488) | function ensureCursorVisible(cm) {
  function scrollToCoords (line 3494) | function scrollToCoords(cm, x, y) {
  function scrollToRange (line 3500) | function scrollToRange(cm, range$$1) {
  function resolveScrollToPos (line 3509) | function resolveScrollToPos(cm) {
  function scrollToCoordsRange (line 3518) | function scrollToCoordsRange(cm, from, to, margin) {
  function updateScrollTop (line 3530) | function updateScrollTop(cm, val) {
  function setScrollTop (line 3538) | function setScrollTop(cm, val, forceScroll) {
  function setScrollLeft (line 3548) | function setScrollLeft(cm, val, isScroller, forceScroll) {
  function measureForScrollbars (line 3561) | function measureForScrollbars(cm) {
  function maybeDisable (line 3653) | function maybeDisable() {
  function updateScrollbars (line 3682) | function updateScrollbars(cm, measure) {
  function updateScrollbarsInner (line 3696) | function updateScrollbarsInner(cm, measure) {
  function initScrollbars (line 3718) | function initScrollbars(cm) {
  function startOperation (line 3748) | function startOperation(cm) {
  function endOperation (line 3770) | function endOperation(cm) {
  function endOperations (line 3781) | function endOperations(group) {
  function endOperation_R1 (line 3795) | function endOperation_R1(op) {
  function endOperation_W1 (line 3808) | function endOperation_W1(op) {
  function endOperation_R2 (line 3812) | function endOperation_R2(op) {
  function endOperation_W2 (line 3833) | function endOperation_W2(op) {
  function endOperation_finish (line 3858) | function endOperation_finish(op) {
  function runInOp (line 3897) | function runInOp(cm, f) {
  function operation (line 3904) | function operation(cm, f) {
  function methodOp (line 3914) | function methodOp(f) {
  function docMethodOp (line 3922) | function docMethodOp(f) {
  function startWorker (line 3934) | function startWorker(cm, time) {
  function highlightWorker (line 3939) | function highlightWorker(cm) {
  function maybeClipScrollbars (line 4009) | function maybeClipScrollbars(cm) {
  function selectionSnapshot (line 4020) | function selectionSnapshot(cm) {
  function restoreSelection (line 4037) | function restoreSelection(snapshot) {
  function updateDisplayIfNeeded (line 4053) | function updateDisplayIfNeeded(cm, update) {
  function postUpdateDisplay (line 4125) | function postUpdateDisplay(cm, update) {
  function updateDisplaySimple (line 4155) | function updateDisplaySimple(cm, viewport) {
  function patchDisplay (line 4172) | function patchDisplay(cm, updateNumbersFrom, dims) {
  function updateGutterSpace (line 4213) | function updateGutterSpace(display) {
  function setDocumentHeight (line 4218) | function setDocumentHeight(cm, measure) {
  function alignHorizontally (line 4226) | function alignHorizontally(cm) {
  function maybeUpdateLineNumberWidth (line 4249) | function maybeUpdateLineNumberWidth(cm) {
  function getGutters (line 4267) | function getGutters(gutters, lineNumbers) {
  function renderGutters (line 4284) | function renderGutters(display) {
  function updateGutters (line 4303) | function updateGutters(cm) {
  function Display (line 4313) | function Display(place, doc, input, options) {
  function wheelEventDelta (line 4436) | function wheelEventDelta(e) {
  function wheelEventPixels (line 4443) | function wheelEventPixels(e) {
  function onScrollWheel (line 4450) | function onScrollWheel(cm, e) {
  function normalizeSelection (line 4589) | function normalizeSelection(cm, ranges, primIndex) {
  function simpleSelection (line 4607) | function simpleSelection(anchor, head) {
  function changeEnd (line 4613) | function changeEnd(change) {
  function adjustForChange (line 4621) | function adjustForChange(pos, change) {
  function computeSelAfterChange (line 4630) | function computeSelAfterChange(doc, change) {
  function offsetPos (line 4640) | function offsetPos(pos, old, nw) {
  function computeReplacedSel (line 4649) | function computeReplacedSel(doc, changes, hint) {
  function loadMode (line 4670) | function loadMode(cm) {
  function resetModeState (line 4675) | function resetModeState(cm) {
  function isWholeLineUpdate (line 4691) | function isWholeLineUpdate(doc, change) {
  function updateDoc (line 4697) | function updateDoc(doc, change, markedSpans, estimateHeight$$1) {
  function linkedDocs (line 4749) | function linkedDocs(doc, f, sharedHistOnly) {
  function attachDoc (line 4764) | function attachDoc(cm, doc) {
  function setDirectionClass (line 4776) | function setDirectionClass(cm) {
  function directionChanged (line 4780) | function directionChanged(cm) {
  function History (line 4787) | function History(startGen) {
  function historyChangeFromChange (line 4804) | function historyChangeFromChange(doc, change) {
  function clearSelectionEvents (line 4813) | function clearSelectionEvents(array) {
  function lastChangeEvent (line 4823) | function lastChangeEvent(hist, force) {
  function addChangeToHistory (line 4838) | function addChangeToHistory(doc, change, selAfter, opId) {
  function selectionEventCanBeMerged (line 4881) | function selectionEventCanBeMerged(doc, origin, prev, sel) {
  function addSelectionToHistory (line 4894) | function addSelectionToHistory(doc, sel, opId, options) {
  function pushSelectionToHistory (line 4916) | function pushSelectionToHistory(sel, dest) {
  function attachLocalSpans (line 4923) | function attachLocalSpans(doc, change, from, to) {
  function removeClearedSpans (line 4934) | function removeClearedSpans(spans) {
  function getOldSpans (line 4945) | function getOldSpans(doc, change) {
  function mergeOldSpans (line 4958) | function mergeOldSpans(doc, change) {
  function copyHistoryArray (line 4982) | function copyHistoryArray(events, newGroup, instantiateSel) {
  function extendRange (line 5014) | function extendRange(range, head, other, extend) {
  function extendSelection (line 5033) | function extendSelection(doc, head, other, options, extend) {
  function extendSelections (line 5040) | function extendSelections(doc, heads, options) {
  function replaceOneSelection (line 5050) | function replaceOneSelection(doc, i, range, options) {
  function setSimpleSelection (line 5057) | function setSimpleSelection(doc, anchor, head, options) {
  function filterSelectionChange (line 5063) | function filterSelectionChange(doc, sel, options) {
  function setSelectionReplaceHistory (line 5082) | function setSelectionReplaceHistory(doc, sel, options) {
  function setSelection (line 5093) | function setSelection(doc, sel, options) {
  function setSelectionNoUndo (line 5098) | function setSelectionNoUndo(doc, sel, options) {
  function setSelectionInner (line 5110) | function setSelectionInner(doc, sel) {
  function reCheckSelection (line 5125) | function reCheckSelection(doc) {
  function skipAtomicInSelection (line 5131) | function skipAtomicInSelection(doc, sel, bias, mayClear) {
  function skipAtomicInner (line 5146) | function skipAtomicInner(doc, pos, oldPos, dir, mayClear) {
  function skipAtomic (line 5186) | function skipAtomic(doc, pos, oldPos, bias, mayClear) {
  function movePos (line 5199) | function movePos(doc, pos, dir, line) {
  function selectAll (line 5211) | function selectAll(cm) {
  function filterChange (line 5218) | function filterChange(doc, change, update) {
  function makeChange (line 5245) | function makeChange(doc, change, ignoreReadOnly) {
  function makeChangeInner (line 5267) | function makeChangeInner(doc, change) {
  function makeChangeFromHistory (line 5285) | function makeChangeFromHistory(doc, type, allowSelectionOnly) {
  function shiftDoc (line 5361) | function shiftDoc(doc, distance) {
  function makeChangeSingleDoc (line 5377) | function makeChangeSingleDoc(doc, change, selAfter, spans) {
  function makeChangeSingleDocInEditor (line 5413) | function makeChangeSingleDocInEditor(cm, change, spans) {
  function replaceRange (line 5471) | function replaceRange(doc, code, from, to, origin) {
  function rebaseHistSelSingle (line 5482) | function rebaseHistSelSingle(pos, from, to, diff) {
  function rebaseHistArray (line 5498) | function rebaseHistArray(array, from, to, diff) {
  function rebaseHist (line 5526) | function rebaseHist(hist, change) {
  function changeLine (line 5535) | function changeLine(doc, handle, changeType, op) {
  function LeafChunk (line 5557) | function LeafChunk(lines) {
  function BranchChunk (line 5610) | function BranchChunk(children) {
  function adjustScrollWhenAboveVisible (line 5773) | function adjustScrollWhenAboveVisible(cm, line, diff) {
  function addLineWidget (line 5778) | function addLineWidget(doc, handle, node, options) {
  function markText (line 5938) | function markText(doc, from, to, options, type) {
  function markTextShared (line 6037) | function markTextShared(doc, from, to, options, type) {
  function findSharedMarkers (line 6052) | function findSharedMarkers(doc) {
  function copySharedMarkers (line 6056) | function copySharedMarkers(doc, markers) {
  function detachSharedMarkers (line 6068) | function detachSharedMarkers(markers) {
  function onDrop (line 6510) | function onDrop(e) {
  function onDragStart (line 6570) | function onDragStart(cm, e) {
  function onDragOver (line 6593) | function onDragOver(cm, e) {
  function clearDragCursor (line 6605) | function clearDragCursor(cm) {
  function forEachCodeMirror (line 6616) | function forEachCodeMirror(f) {
  function ensureGlobalHandlers (line 6629) | function ensureGlobalHandlers() {
  function registerGlobalHandlers (line 6634) | function registerGlobalHandlers() {
  function onResize (line 6647) | function onResize(cm) {
  function normalizeKeyName (line 6718) | function normalizeKeyName(name) {
  function normalizeKeyMap (line 6742) | function normalizeKeyMap(keymap) {
  function lookupKey (line 6769) | function lookupKey(key, map$$1, handle, context) {
  function isModifierKey (line 6788) | function isModifierKey(value) {
  function addModifierNames (line 6793) | function addModifierNames(name, event, noShift) {
  function keyName (line 6803) | function keyName(event, noShift) {
  function getKeyMap (line 6813) | function getKeyMap(val) {
  function deleteNearSelection (line 6819) | function deleteNearSelection(cm, compute) {
  function moveCharLogically (line 6842) | function moveCharLogically(line, ch, dir) {
  function moveLogically (line 6847) | function moveLogically(line, start, dir) {
  function endOfLine (line 6852) | function endOfLine(visually, cm, lineObj, lineNo, dir) {
  function moveVisually (line 6879) | function moveVisually(cm, line, start, dir) {
  function lineStart (line 7091) | function lineStart(cm, lineN) {
  function lineEnd (line 7097) | function lineEnd(cm, lineN) {
  function lineStartSmart (line 7103) | function lineStartSmart(cm, pos) {
  function doHandleBinding (line 7116) | function doHandleBinding(cm, bound, dropShift) {
  function lookupKeyForEditor (line 7136) | function lookupKeyForEditor(cm, name, handle) {
  function dispatchKey (line 7150) | function dispatchKey(cm, name, e, handle) {
  function dispatchKeyInner (line 7168) | function dispatchKeyInner(cm, name, e, handle) {
  function handleKeyBinding (line 7185) | function handleKeyBinding(cm, e) {
  function handleCharBinding (line 7204) | function handleCharBinding(cm, e, ch) {
  function onKeyDown (line 7209) | function onKeyDown(e) {
  function showCrossHair (line 7230) | function showCrossHair(cm) {
  function onKeyUp (line 7245) | function onKeyUp(e) {
  function onKeyPress (line 7250) | function onKeyPress(e) {
  function clickRepeat (line 7277) | function clickRepeat(pos, button) {
  function onMouseDown (line 7298) | function onMouseDown(e) {
  function handleMappedButton (line 7335) | function handleMappedButton(cm, button, pos, repeat, event) {
  function configureMouse (line 7355) | function configureMouse(cm, repeat, event) {
  function leftButtonDown (line 7368) | function leftButtonDown(cm, pos, repeat, event) {
  function leftButtonStartDrag (line 7386) | function leftButtonStartDrag(cm, event, pos, behavior) {
  function rangeForUnit (line 7425) | function rangeForUnit(cm, pos, unit) {
  function leftButtonSelect (line 7434) | function leftButtonSelect(cm, event, start, behavior) {
  function bidiSimplify (line 7571) | function bidiSimplify(cm, range$$1) {
  function gutterEvent (line 7606) | function gutterEvent(cm, e, type, prevent) {
  function clickInGutter (line 7635) | function clickInGutter(cm, e) {
  function onContextMenu (line 7644) | function onContextMenu(cm, e) {
  function contextMenuInGutter (line 7650) | function contextMenuInGutter(cm, e) {
  function themeChanged (line 7655) | function themeChanged(cm) {
  function defineOptions (line 7666) | function defineOptions(CodeMirror) {
  function dragDropChanged (line 7804) | function dragDropChanged(cm, value, old) {
  function wrappingChanged (line 7817) | function wrappingChanged(cm) {
  function CodeMirror (line 7835) | function CodeMirror(place, options) {
  function registerEventHandlers (line 7910) | function registerEventHandlers(cm) {
  function indentLine (line 8025) | function indentLine(cm, n, how, aggressive) {
  function setLastCopied (line 8089) | function setLastCopied(newLastCopied) {
  function applyTextInput (line 8093) | function applyTextInput(cm, inserted, deleted, sel, origin) {
  function handlePaste (line 8141) | function handlePaste(e, cm) {
  function triggerElectric (line 8151) | function triggerElectric(cm, inserted) {
  function copyableRanges (line 8175) | function copyableRanges(cm) {
  function disableBrowserMagic (line 8186) | function disableBrowserMagic(field, spellcheck, autocorrect, autocapital...
  function hiddenTextarea (line 8192) | function hiddenTextarea() {
  function addEditorMethods (line 8215) | function addEditorMethods(CodeMirror) {
  function findPosH (line 8665) | function findPosH(doc, pos, dir, unit, visually) {
  function findPosV (line 8725) | function findPosV(cm, pos, dir, unit) {
  function onCopyCut (line 8788) | function onCopyCut(e) {
  function poll (line 8946) | function poll() {
  function posToDOM (line 9112) | function posToDOM(cm, pos) {
  function isInGutter (line 9128) | function isInGutter(node) {
  function badPos (line 9134) | function badPos(pos, bad) { if (bad) { pos.bad = true; } return pos }
  function domTextBetween (line 9136) | function domTextBetween(cm, from, to, fromLine, toLine) {
  function domToPos (line 9189) | function domToPos(cm, node, offset) {
  function locateNodeInLineView (line 9208) | function locateNodeInLineView(lineView, node, offset) {
  function prepareCopyCut (line 9305) | function prepareCopyCut(e) {
  function p (line 9453) | function p() {
  function prepareSelectAllHack (line 9557) | function prepareSelectAllHack() {
  function rehide (line 9570) | function rehide() {
  function fromTextArea (line 9617) | function fromTextArea(textarea, options) {
  function addLegacyProps (line 9674) | function addLegacyProps(CodeMirror) {

FILE: spider-flow-web/src/main/resources/static/js/codemirror/javascript.js
  function kw (line 25) | function kw(type) {return {type: type, style: "keyword"};}
  function readRegexp (line 46) | function readRegexp(stream) {
  function ret (line 61) | function ret(tp, style, cont) {
  function tokenBase (line 65) | function tokenBase(stream, state) {
  function tokenString (line 132) | function tokenString(quote) {
  function tokenComment (line 148) | function tokenComment(stream, state) {
  function tokenQuasi (line 160) | function tokenQuasi(stream, state) {
  function findFatArrow (line 180) | function findFatArrow(stream, state) {
  function JSLexical (line 219) | function JSLexical(indented, column, type, align, prev, info) {
  function inScope (line 228) | function inScope(state, varname) {
  function parseJS (line 237) | function parseJS(state, style, type, content, stream) {
  function pass (line 261) | function pass() {
  function cont (line 264) | function cont() {
  function inList (line 268) | function inList(name, list) {
  function register (line 272) | function register(varname) {
  function registerVarScoped (line 292) | function registerVarScoped(varname, context) {
  function isModifier (line 307) | function isModifier(name) {
  function Context (line 313) | function Context(prev, vars, block) { this.prev = prev; this.vars = vars...
  function Var (line 314) | function Var(name, next) { this.name = name; this.next = next }
  function pushcontext (line 317) | function pushcontext() {
  function pushblockcontext (line 321) | function pushblockcontext() {
  function popcontext (line 325) | function popcontext() {
  function pushlex (line 330) | function pushlex(type, info) {
  function poplex (line 341) | function poplex() {
  function expect (line 351) | function expect(wanted) {
  function statement (line 360) | function statement(type, value) {
  function maybeCatchBinding (line 409) | function maybeCatchBinding(type) {
  function expression (line 412) | function expression(type, value) {
  function expressionNoComma (line 415) | function expressionNoComma(type, value) {
  function parenExpr (line 418) | function parenExpr(type) {
  function expressionInner (line 422) | function expressionInner(type, value, noComma) {
  function maybeexpression (line 443) | function maybeexpression(type) {
  function maybeoperatorComma (line 448) | function maybeoperatorComma(type, value) {
  function maybeoperatorNoComma (line 452) | function maybeoperatorNoComma(type, value, noComma) {
  function quasi (line 475) | function quasi(type, value) {
  function continueQuasi (line 480) | function continueQuasi(type) {
  function arrowBody (line 487) | function arrowBody(type) {
  function arrowBodyNoComma (line 491) | function arrowBodyNoComma(type) {
  function maybeTarget (line 495) | function maybeTarget(noComma) {
  function target (line 502) | function target(_, value) {
  function targetNoComma (line 505) | function targetNoComma(_, value) {
  function maybelabel (line 508) | function maybelabel(type) {
  function property (line 512) | function property(type) {
  function objprop (line 515) | function objprop(type, value) {
  function getterSetter (line 545) | function getterSetter(type) {
  function afterprop (line 550) | function afterprop(type) {
  function commasep (line 554) | function commasep(what, end, sep) {
  function contCommasep (line 573) | function contCommasep(what, end, info) {
  function block (line 578) | function block(type) {
  function maybetype (line 582) | function maybetype(type, value) {
  function maybetypeOrIn (line 588) | function maybetypeOrIn(type, value) {
  function mayberettype (line 591) | function mayberettype(type) {
  function isKW (line 597) | function isKW(_, value) {
  function typeexpr (line 603) | function typeexpr(type, value) {
  function maybeReturnType (line 619) | function maybeReturnType(type) {
  function typeprop (line 622) | function typeprop(type, value) {
  function typearg (line 636) | function typearg(type, value) {
  function afterType (line 642) | function afterType(type, value) {
  function maybeTypeArgs (line 649) | function maybeTypeArgs(_, value) {
  function typeparam (line 652) | function typeparam() {
  function maybeTypeDefault (line 655) | function maybeTypeDefault(_, value) {
  function vardef (line 658) | function vardef(_, value) {
  function pattern (line 662) | function pattern(type, value) {
  function proppattern (line 669) | function proppattern(type, value) {
  function eltpattern (line 680) | function eltpattern() {
  function maybeAssign (line 683) | function maybeAssign(_type, value) {
  function vardefCont (line 686) | function vardefCont(type) {
  function maybeelse (line 689) | function maybeelse(type, value) {
  function forspec (line 692) | function forspec(type, value) {
  function forspec1 (line 696) | function forspec1(type) {
  function forspec2 (line 701) | function forspec2(type, value) {
  function functiondef (line 707) | function functiondef(type, value) {
  function functiondecl (line 713) | function functiondecl(type, value) {
  function typename (line 719) | function typename(type, value) {
  function funarg (line 727) | function funarg(type, value) {
  function classExpression (line 734) | function classExpression(type, value) {
  function className (line 739) | function className(type, value) {
  function classNameAfter (line 742) | function classNameAfter(type, value) {
  function classBody (line 750) | function classBody(type, value) {
  function classfield (line 774) | function classfield(type, value) {
  function afterExport (line 781) | function afterExport(type, value) {
  function exportField (line 787) | function exportField(type, value) {
  function afterImport (line 791) | function afterImport(type) {
  function importSpec (line 796) | function importSpec(type, value) {
  function maybeMoreImports (line 802) | function maybeMoreImports(type) {
  function maybeAs (line 805) | function maybeAs(_type, value) {
  function maybeFrom (line 808) | function maybeFrom(_type, value) {
  function arrayLiteral (line 811) | function arrayLiteral(type) {
  function enumdef (line 815) | function enumdef() {
  function enummember (line 818) | function enummember() {
  function isContinuedStatement (line 822) | function isContinuedStatement(state, textAfter) {
  function expressionAllowed (line 828) | function expressionAllowed(stream, state, backUp) {

FILE: spider-flow-web/src/main/resources/static/js/codemirror/placeholder.js
  function clearPlaceholder (line 31) | function clearPlaceholder(cm) {
  function setPlaceholder (line 37) | function setPlaceholder(cm) {
  function onBlur (line 49) | function onBlur(cm) {
  function onChange (line 52) | function onChange(cm) {
  function isEmpty (line 60) | function isEmpty(cm) {

FILE: spider-flow-web/src/main/resources/static/js/codemirror/show-hint.js
  function Completion (line 68) | function Completion(cm, options) {
  function isNewCompletion (line 219) | function isNewCompletion(old, nw) {
  function parseOptions (line 224) | function parseOptions(cm, pos, options) {
  function getText (line 236) | function getText(completion) {
  function buildKeyMap (line 241) | function buildKeyMap(completion, handle) {
  function getHintElement (line 276) | function getHintElement(hintsElement, el) {
  function Widget1 (line 282) | function Widget1(completion, data,comments) {
  function Widget (line 431) | function Widget(completion, data) {
  function applicableHelpers (line 654) | function applicableHelpers(cm, helpers) {
  function resolveAutoHints (line 662) | function resolveAutoHints(cm, pos) {

FILE: spider-flow-web/src/main/resources/static/js/codemirror/spiderflow-hint.js
  function searchGrammer (line 21) | function searchGrammer(keyword,isClass){
  function initHint (line 70) | function initHint(cm){

FILE: spider-flow-web/src/main/resources/static/js/codemirror/sql.js
  function tokenBase (line 27) | function tokenBase(stream, state) {
  function tokenLiteral (line 125) | function tokenLiteral(quote) {
  function tokenComment (line 138) | function tokenComment(depth) {
  function pushContext (line 149) | function pushContext(stream, state, type) {
  function popContext (line 158) | function popContext(state) {
  function hookIdentifier (line 207) | function hookIdentifier(stream) {
  function hookIdentifierDoublequote (line 219) | function hookIdentifierDoublequote(stream) {
  function hookVar (line 232) | function hookVar(stream) {
  function hookClient (line 259) | function hookClient(stream) {
  function set (line 274) | function set(str) {

FILE: spider-flow-web/src/main/resources/static/js/common.js
  function getQueryString (line 1) | function getQueryString(name) {

FILE: spider-flow-web/src/main/resources/static/js/cron/cron.js
  function btnFan (line 1) | function btnFan() {
  function initObj (line 27) | function initObj(strVal, strid) {
  function initDay (line 54) | function initDay(strVal) {
  function initMonth (line 87) | function initMonth(strVal) {
  function initWeek (line 116) | function initWeek(strVal) {
  function initYear (line 147) | function initYear(strVal) {
  function everyTime (line 162) | function everyTime(dom) {
  function unAppoint (line 171) | function unAppoint(dom) {
  function appoint (line 181) | function appoint(dom) {
  function cycle (line 188) | function cycle(dom) {
  function startOn (line 201) | function startOn(dom) {
  function lastDay (line 211) | function lastDay(dom) {
  function weekOfDay (line 217) | function weekOfDay(dom) {
  function lastWeek (line 227) | function lastWeek(dom) {
  function workDay (line 235) | function workDay(dom) {

FILE: spider-flow-web/src/main/resources/static/js/editor.js
  function renderCodeMirror (line 7) | function renderCodeMirror(){
  function getCellData (line 35) | function getCellData(cellId,keys){
  function serializeForm (line 51) | function serializeForm(){
  function resizeSlideBar (line 120) | function resizeSlideBar(){
  function validXML (line 130) | function validXML(callback){
  function loadTemplate (line 182) | function loadTemplate(cell,model,callback){
  function loadShapes (line 547) | function loadShapes(editor,container){
  function bindToolbarClickAction (line 649) | function bindToolbarClickAction(editor){
  function runSpider (line 765) | function runSpider(debug){
  function bindTooltip (line 1074) | function bindTooltip(content,selector){
  function onCanvasViewerClick (line 1094) | function onCanvasViewerClick(e,source){
  function createWebSocket (line 1129) | function createWebSocket(options){
  function Save (line 1146) | function Save(){
  function allowDrop (line 1168) | function allowDrop(ev){
  function drag (line 1172) | function drag(ev){
  function drop (line 1176) | function drop(ev){

FILE: spider-flow-web/src/main/resources/static/js/index.js
  function setCookie (line 2) | function setCookie(name,value){
  function getCookie (line 8) | function getCookie(name){
  function setTheSkin (line 15) | function setTheSkin(value){
  function openTab (line 22) | function openTab(title,id,href){
  function initMenu (line 49) | function initMenu() {

FILE: spider-flow-web/src/main/resources/static/js/jsontree/jsontree.js
  function Node (line 173) | function Node(label, val, isLast) {
  function _NodeSimple (line 215) | function _NodeSimple(label, val, isLast) {
  function NodeBoolean (line 340) | function NodeBoolean(label, val, isLast) {
  function NodeNumber (line 360) | function NodeNumber(label, val, isLast) {
  function NodeString (line 380) | function NodeString(label, val, isLast) {
  function NodeNull (line 399) | function NodeNull(label, val, isLast) {
  function _NodeComplex (line 436) | function _NodeComplex(label, val, isLast) {
  function NodeObject (line 649) | function NodeObject(label, val, isLast) {
  function NodeArray (line 670) | function NodeArray(label, val, isLast) {
  function Tree (line 696) | function Tree(jsonObj, domEl) {

FILE: spider-flow-web/src/main/resources/static/js/layui/ext/treeselect/treeselect.js
  function c (line 44) | function c(b){d.push(b);(b=e.nodeChildren(a,b))&&(d=d.concat(e.transform...
  function b (line 71) | function b(a,c){return{start:a,length:c,end:a+c}}
  function c (line 71) | function c(b,d){return!1===a||d.start<b.end&&b.start<d.end?Math.max(b.en...
  function d (line 71) | function d(a,b){return{x:a,y:b,translate:function(c,i){return d(a+c,b+i)}}}
  function e (line 71) | function e(a,b){for(;a;)b=b.translate(a.offsetLeft,a.offsetTop),
  function g (line 81) | function g(){j.addNodes(d,a,b,n,f==!0)}
  function n (line 82) | function n(){var b=l(a,d).get(0);b&&f!==!1&&j.scrollIntoView(d,b)}
  function e (line 87) | function e(){if(!c){var b=l(a,d).get(0);j.scrollIntoView(d,b)}}
  function c (line 138) | function c(a){if(m.dragFlag==0&&Math.abs(N-a.clientX)<f.edit.drag.minMov...
  function l (line 149) | function l(a){if(window.zTreeMoveTimer)clearTimeout(window.zTreeMoveTime...
  function g (line 153) | function g(){return!1}
  function j (line 174) | function j(){e.moveNode(m.setting,b,l,g,!1,o)}
  function ztreeFilter (line 386) | function ztreeFilter(zTreeObj, _keywords, callBackFunc) {
  function processShowNodes (line 420) | function processShowNodes(nodesShow, _keywords) {
  function searchNodeLazy (line 451) | function searchNodeLazy(_keywords) {

FILE: spider-flow-web/src/main/resources/static/js/layui/extends/formSelects-v4.js
  function _defineProperty (line 5) | function _defineProperty(obj, key, value) { if (key in obj) { Object.def...

FILE: spider-flow-web/src/main/resources/static/js/layui/extends/treeGrid.js
  function isReady (line 290) | function isReady() {
  function f (line 1306) | function f(o) {

FILE: spider-flow-web/src/main/resources/static/js/layui/layui.all.js
  function s (line 2) | function s(e,t){var n="PLaySTATION 3"===navigator.platform?/^complete$/:...
  function c (line 2) | function c(){l.push(layui[f]),e.length>1?y.use(e.slice(1),n,l):"function...
  function n (line 2) | function n(e){var t=!!e&&"length"in e&&e.length,n=pe.type(e);return"func...
  function r (line 2) | function r(e,t,n){if(pe.isFunction(t))return pe.grep(e,function(e,r){ret...
  function i (line 2) | function i(e,t){do e=e[t];while(e&&1!==e.nodeType);return e}
  function o (line 2) | function o(e){var t={};return pe.each(e.match(De)||[],function(e,n){t[n]...
  function a (line 2) | function a(){re.addEventListener?(re.removeEventListener("DOMContentLoad...
  function s (line 2) | function s(){(re.addEventListener||"load"===e.event.type||"complete"===r...
  function u (line 2) | function u(e,t,n){if(void 0===n&&1===e.nodeType){var r="data-"+t.replace...
  function l (line 2) | function l(e){var t;for(t in e)if(("data"!==t||!pe.isEmptyObject(e[t]))&...
  function c (line 2) | function c(e,t,n,r){if(He(e)){var i,o,a=pe.expando,s=e.nodeType,u=s?pe.c...
  function f (line 2) | function f(e,t,n){if(He(e)){var r,i,o=e.nodeType,a=o?pe.cache:e,s=o?e[pe...
  function d (line 2) | function d(e,t,n,r){var i,o=1,a=20,s=r?function(){return r.cur()}:functi...
  function p (line 2) | function p(e){var t=ze.split("|"),n=e.createDocumentFragment();if(n.crea...
  function h (line 2) | function h(e,t){var n,r,i=0,o="undefined"!=typeof e.getElementsByTagName...
  function g (line 2) | function g(e,t){for(var n,r=0;null!=(n=e[r]);r++)pe._data(n,"globalEval"...
  function m (line 2) | function m(e){Be.test(e.type)&&(e.defaultChecked=e.checked)}
  function y (line 2) | function y(e,t,n,r,i){for(var o,a,s,u,l,c,f,d=e.length,y=p(t),v=[],x=0;x...
  function v (line 2) | function v(){return!0}
  function x (line 2) | function x(){return!1}
  function b (line 2) | function b(){try{return re.activeElement}catch(e){}}
  function w (line 2) | function w(e,t,n,r,i,o){var a,s;if("object"==typeof t){"string"!=typeof ...
  function T (line 2) | function T(e,t){return pe.nodeName(e,"table")&&pe.nodeName(11!==t.nodeTy...
  function C (line 2) | function C(e){return e.type=(null!==pe.find.attr(e,"type"))+"/"+e.type,e}
  function E (line 2) | function E(e){var t=it.exec(e.type);return t?e.type=t[1]:e.removeAttribu...
  function N (line 2) | function N(e,t){if(1===t.nodeType&&pe.hasData(e)){var n,r,i,o=pe._data(e...
  function k (line 2) | function k(e,t){var n,r,i;if(1===t.nodeType){if(n=t.nodeName.toLowerCase...
  function S (line 2) | function S(e,t,n,r){t=oe.apply([],t);var i,o,a,s,u,l,c=0,f=e.length,d=f-...
  function A (line 2) | function A(e,t,n){for(var r,i=t?pe.filter(t,e):e,o=0;null!=(r=i[o]);o++)...
  function D (line 2) | function D(e,t){var n=pe(t.createElement(e)).appendTo(t.body),r=pe.css(n...
  function j (line 2) | function j(e){var t=re,n=lt[e];return n||(n=D(e,t),"none"!==n&&n||(ut=(u...
  function L (line 2) | function L(e,t){return{get:function(){return e()?void delete this.get:(t...
  function H (line 2) | function H(e){if(e in Et)return e;for(var t=e.charAt(0).toUpperCase()+e....
  function q (line 2) | function q(e,t){for(var n,r,i,o=[],a=0,s=e.length;a<s;a++)r=e[a],r.style...
  function _ (line 2) | function _(e,t,n){var r=bt.exec(t);return r?Math.max(0,r[1]-(n||0))+(r[2...
  function F (line 2) | function F(e,t,n,r,i){for(var o=n===(r?"border":"content")?4:"width"===t...
  function M (line 2) | function M(t,n,r){var i=!0,o="width"===n?t.offsetWidth:t.offsetHeight,a=...
  function O (line 2) | function O(e,t,n,r,i){return new O.prototype.init(e,t,n,r,i)}
  function R (line 2) | function R(){return e.setTimeout(function(){Nt=void 0}),Nt=pe.now()}
  function P (line 2) | function P(e,t){var n,r={height:e},i=0;for(t=t?1:0;i<4;i+=2-t)n=Oe[i],r[...
  function B (line 2) | function B(e,t,n){for(var r,i=($.tweeners[t]||[]).concat($.tweeners["*"]...
  function W (line 2) | function W(e,t,n){var r,i,o,a,s,u,l,c,f=this,d={},p=e.style,h=e.nodeType...
  function I (line 2) | function I(e,t){var n,r,i,o,a;for(n in e)if(r=pe.camelCase(n),i=t[r],o=e...
  function $ (line 2) | function $(e,t,n){var r,i,o=0,a=$.prefilters.length,s=pe.Deferred().alwa...
  function z (line 2) | function z(e){return pe.attr(e,"class")||""}
  function X (line 2) | function X(e){return function(t,n){"string"!=typeof t&&(n=t,t="*");var r...
  function U (line 2) | function U(e,t,n,r){function i(s){var u;return o[s]=!0,pe.each(e[s]||[],...
  function V (line 2) | function V(e,t){var n,r,i=pe.ajaxSettings.flatOptions||{};for(r in t)voi...
  function Y (line 2) | function Y(e,t,n){for(var r,i,o,a,s=e.contents,u=e.dataTypes;"*"===u[0];...
  function J (line 2) | function J(e,t,n,r){var i,o,a,s,u,l={},c=e.dataTypes.slice();if(c[1])for...
  function G (line 2) | function G(e){return e.style&&e.style.display||pe.css(e,"display")}
  function K (line 2) | function K(e){for(;e&&1===e.nodeType;){if("none"===G(e)||"hidden"===e.ty...
  function Q (line 2) | function Q(e,t,n,r){var i;if(pe.isArray(t))pe.each(t,function(t,i){n||rn...
  function Z (line 2) | function Z(){try{return new e.XMLHttpRequest}catch(t){}}
  function ee (line 2) | function ee(){try{return new e.ActiveXObject("Microsoft.XMLHTTP")}catch(...
  function te (line 2) | function te(e){return pe.isWindow(e)?e:9===e.nodeType&&(e.defaultView||e...
  function t (line 2) | function t(e,t,n,r){var i,o,a,s,u,l,f,p,h=t&&t.ownerDocument,g=t?t.nodeT...
  function n (line 2) | function n(){function e(n,r){return t.push(n+" ")>T.cacheLength&&delete ...
  function r (line 2) | function r(e){return e[P]=!0,e}
  function i (line 2) | function i(e){var t=H.createElement("div");try{return!!e(t)}catch(n){ret...
  function o (line 2) | function o(e,t){for(var n=e.split("|"),r=n.length;r--;)T.attrHandle[n[r]...
  function a (line 2) | function a(e,t){var n=t&&e,r=n&&1===e.nodeType&&1===t.nodeType&&(~t.sour...
  function s (line 2) | function s(e){return function(t){var n=t.nodeName.toLowerCase();return"i...
  function u (line 2) | function u(e){return function(t){var n=t.nodeName.toLowerCase();return("...
  function l (line 2) | function l(e){return r(function(t){return t=+t,r(function(n,r){for(var i...
  function c (line 2) | function c(e){return e&&"undefined"!=typeof e.getElementsByTagName&&e}
  function f (line 2) | function f(){}
  function d (line 2) | function d(e){for(var t=0,n=e.length,r="";t<n;t++)r+=e[t].value;return r}
  function p (line 2) | function p(e,t,n){var r=t.dir,i=n&&"parentNode"===r,o=I++;return t.first...
  function h (line 2) | function h(e){return e.length>1?function(t,n,r){for(var i=e.length;i--;)...
  function g (line 2) | function g(e,n,r){for(var i=0,o=n.length;i<o;i++)t(e,n[i],r);return r}
  function m (line 2) | function m(e,t,n,r,i){for(var o,a=[],s=0,u=e.length,l=null!=t;s<u;s++)(o...
  function y (line 2) | function y(e,t,n,i,o,a){return i&&!i[P]&&(i=y(i)),o&&!o[P]&&(o=y(o,a)),r...
  function v (line 2) | function v(e){for(var t,n,r,i=e.length,o=T.relative[e[0].type],a=o||T.re...
  function x (line 2) | function x(e,n){var i=n.length>0,o=e.length>0,a=function(r,a,s,u,l){var ...
  function t (line 4) | function t(){var t,c,f=re.documentElement;f.appendChild(u),l.style.cssTe...
  function r (line 4) | function r(t,n,r,i){var o,f,v,x,w,C=n;2!==b&&(b=2,u&&e.clearTimeout(u),c...
  function e (line 5) | function e(){var e=a.cancel&&a.cancel(t.index,n);e===!1||r.close(t.index)}
  function o (line 5) | function o(e,t,i){var n=new Image;return n.src=e,n.complete?t(n):(n.onlo...
  function a (line 5) | function a(){n=e[l](function(){r.each(function(){var e=t(this),i=e.width...
  function a (line 5) | function a(e,a,r){var o=t(this),l=t.data(this,g)||{};l.w=a!==i?a:o.width...

FILE: spider-flow-web/src/main/resources/static/js/log-viewer.js
  function LogViewer (line 1) | function LogViewer(options){
  function eventFunc (line 78) | function eventFunc(e){

FILE: spider-flow-web/src/main/resources/static/js/mxgraph/mxgraph.js
  function callback (line 320) | function callback()
  function mxDictionary (line 1131) | function mxDictionary()
  function mxPoint (line 1711) | function mxPoint(x, y)
  function mxRectangle (line 1766) | function mxRectangle(x, y, width, height)
  function doExtract (line 3018) | function doExtract(elts)
  function mxEventObject (line 8885) | function mxEventObject(name)
  function mxMouseEvent (line 9007) | function mxMouseEvent(evt, state)
  function mxEventSource (line 9239) | function mxEventSource(eventSource)
  function mxXmlRequest (line 10880) | function mxXmlRequest(url, params, method, async, username, password)
  function mxWindow (line 11666) | function mxWindow(title, content, x, y, width, height, minimizable, mova...
  function mxForm (line 12623) | function mxForm(className)
  function mxImage (line 12825) | function mxImage(src, width, height)
  function mxDivResizer (line 12891) | function mxDivResizer(div, container)
  function mxDragSource (line 13020) | function mxDragSource(element, dropHandler)
  function mxToolbar (line 13744) | function mxToolbar(container)
  function mxUndoableEdit (line 14296) | function mxUndoableEdit(source, significant)
  function mxUndoManager (line 14533) | function mxUndoManager(size)
  function mxPanningManager (line 14852) | function mxPanningManager(graph)
  function mxPopupMenu (line 15138) | function mxPopupMenu(factoryMethod)
  function mxAutoSaveManager (line 15750) | function mxAutoSaveManager(graph)
  function mxAnimation (line 15952) | function mxAnimation(delay)
  function mxMorphing (line 16067) | function mxMorphing(graph, steps, ease, delay)
  function mxImageBundle (line 16324) | function mxImageBundle(alt)
  function mxImageExport (line 16412) | function mxImageExport() { }
  function mxAbstractCanvas2D (line 16567) | function mxAbstractCanvas2D()
  function mxXmlCanvas2D (line 17225) | function mxXmlCanvas2D(root)
  function mxSvgCanvas2D (line 18468) | function mxSvgCanvas2D(root, styleEnabled)
  function mxGuide (line 21731) | function mxGuide(graph, states)
  function snapX (line 21873) | function snapX(x, state)
  function snapY (line 21920) | function snapY(y, state)
  function mxShape (line 22209) | function mxShape(stencil)
  function mxStencil (line 24001) | function mxStencil(desc)
  function createArrow (line 24750) | function createArrow(widthFactor)
  function createOpenArrow (line 24804) | function createOpenArrow(widthFactor)
  function diamond (line 24863) | function diamond(canvas, shape, type, pe, unitX, unitY, size, source, sw...
  function mxActor (line 24954) | function mxActor(bounds, fill, stroke, strokewidth)
  function mxCloud (line 25021) | function mxCloud(bounds, fill, stroke, strokewidth)
  function mxRectangleShape (line 25075) | function mxRectangleShape(bounds, fill, stroke, strokewidth)
  function mxEllipse (line 25202) | function mxEllipse(bounds, fill, stroke, strokewidth)
  function mxDoubleEllipse (line 25271) | function mxDoubleEllipse(bounds, fill, stroke, strokewidth)
  function mxRhombus (line 25364) | function mxRhombus(bounds, fill, stroke, strokewidth)
  function mxPolyline (line 25428) | function mxPolyline(points, stroke, strokewidth)
  function mxArrow (line 25568) | function mxArrow(points, fill, stroke, strokewidth, arrowWidth, spacing,...
  function mxArrowConnector (line 25683) | function mxArrowConnector(points, fill, stroke, strokewidth, arrowWidth,...
  function mxText (line 26194) | function mxText(value, bounds, align, valign, color,
  function mxTriangle (line 27413) | function mxTriangle()
  function mxHexagon (line 27456) | function mxHexagon()
  function mxLine (line 27501) | function mxLine(bounds, stroke, strokewidth)
  function mxImageShape (line 27553) | function mxImageShape(bounds, image, fill, stroke, strokewidth)
  function mxLabel (line 27795) | function mxLabel(bounds, fill, stroke, strokewidth)
  function mxCylinder (line 28072) | function mxCylinder(bounds, fill, stroke, strokewidth)
  function mxConnector (line 28190) | function mxConnector(points, stroke, strokewidth)
  function mxSwimlane (line 28342) | function mxSwimlane(bounds, fill, stroke, strokewidth)
  function mxGraphLayout (line 28807) | function mxGraphLayout(graph)
  function WeightedCellSorter (line 29286) | function WeightedCellSorter(cell, weightedValue)
  function mxStackLayout (line 29384) | function mxStackLayout(graph, horizontal, spacing, x0, y0, border)
  function mxPartitionLayout (line 29983) | function mxPartitionLayout(graph, horizontal, spacing, border)
  function mxCompactTreeLayout (line 30220) | function mxCompactTreeLayout(graph, horizontal, invert)
  function mxRadialTreeLayout (line 31335) | function mxRadialTreeLayout(graph)
  function mxFastOrganicLayout (line 31653) | function mxFastOrganicLayout(graph)
  function mxCircleLayout (line 32249) | function mxCircleLayout(graph, radius)
  function mxParallelEdgeLayout (line 32472) | function mxParallelEdgeLayout(graph)
  function mxCompositeLayout (line 32682) | function mxCompositeLayout(graph, layouts, master)
  function mxEdgeLabelLayout (line 32778) | function mxEdgeLabelLayout(graph, radius)
  function mxGraphAbstractHierarchyCell (line 32935) | function mxGraphAbstractHierarchyCell()
  function mxGraphHierarchyNode (line 33139) | function mxGraphHierarchyNode(cell)
  function mxGraphHierarchyEdge (line 33359) | function mxGraphHierarchyEdge(edges)
  function mxGraphHierarchyModel (line 33557) | function mxGraphHierarchyModel(layout, vertices, roots, parent, tightenT...
  function mxSwimlaneModel (line 34238) | function mxSwimlaneModel(layout, vertices, roots, parent, tightenToSource)
  function mxHierarchicalLayoutStage (line 35026) | function mxHierarchicalLayoutStage() { }
  function mxMedianHybridCrossingReduction (line 35058) | function mxMedianHybridCrossingReduction(layout)
  function MedianCellSorter (line 35665) | function MedianCellSorter()
  function mxMinimumCycleRemover (line 35725) | function mxMinimumCycleRemover(layout)
  function mxCoordinateAssignment (line 35841) | function mxCoordinateAssignment(layout, intraCellSpacing, interRankCellS...
  function mxSwimlaneOrdering (line 37576) | function mxSwimlaneOrdering(layout)
  function mxHierarchicalLayout (line 37679) | function mxHierarchicalLayout(graph, orientation, deterministic)
  function mxSwimlaneLayout (line 38528) | function mxSwimlaneLayout(graph, orientation, deterministic)
  function mxGraphModel (line 39639) | function mxGraphModel(root)
  function mxRootChange (line 41754) | function mxRootChange(model, root)
  function mxChildChange (line 41783) | function mxChildChange(model, parent, child, index)
  function mxTerminalChange (line 41887) | function mxTerminalChange(model, cell, terminal, source)
  function mxValueChange (line 41922) | function mxValueChange(model, cell, value)
  function mxStyleChange (line 41956) | function mxStyleChange(model, cell, style)
  function mxGeometryChange (line 41990) | function mxGeometryChange(model, cell, geometry)
  function mxCollapseChange (line 42024) | function mxCollapseChange(model, cell, collapsed)
  function mxVisibleChange (line 42058) | function mxVisibleChange(model, cell, visible)
  function mxCellAttributeChange (line 42114) | function mxCellAttributeChange(cell, attribute, value)
  function mxCell (line 42212) | function mxCell(value, geometry, style)
  function mxGeometry (line 43043) | function mxGeometry(x, y, width, height)
  function mxPrintPreview (line 44644) | function mxPrintPreview(graph, scale, pageFormat, border, x0, y0, border...
  function mxStylesheet (line 45741) | function mxStylesheet()
  function mxCellState (line 45983) | function mxCellState(view, cell, style)
  function mxGraphSelectionModel (line 46435) | function mxGraphSelectionModel(graph)
  function mxSelectionChange (line 46782) | function mxSelectionChange(selectionModel, added, removed)
  function mxCellEditor (line 46938) | function mxCellEditor(graph)
  function mxCellRenderer (line 48027) | function mxCellRenderer() { }
  function check (line 48971) | function check(property, stylename, defaultValue)
  function pushPoint (line 50200) | function pushPoint(pt)
  function mxGraphView (line 51301) | function mxGraphView(graph)
  function mxCurrentRootChange (line 54205) | function mxCurrentRootChange(view, root)
  function mxGraph (line 54898) | function mxGraph(container, model, renderHint, stylesheet)
  function mxCellOverlay (line 67207) | function mxCellOverlay(image, tooltip, align, verticalAlign, offset, cur...
  function mxOutline (line 67438) | function mxOutline(source, container)
  function mxMultiplicity (line 68192) | function mxMultiplicity(source, type, attr, value, min, max,
  function mxLayoutManager (line 68434) | function mxLayoutManager(graph)
  function mxSwimlaneManager (line 68830) | function mxSwimlaneManager(graph, horizontal, addEnabled, resizeEnabled)
  function mxTemporaryCellStates (line 69269) | function mxTemporaryCellStates(view, scale, cells, isCellVisibleFn, getL...
  function mxCellStatePreview (line 69411) | function mxCellStatePreview(graph)
  function mxConnectionConstraint (line 69618) | function mxConnectionConstraint(point, perimeter, name, dx, dy)
  function mxGraphHandler (line 69687) | function mxGraphHandler(graph)
  function mxPanningHandler (line 70799) | function mxPanningHandler(graph)
  function mxPopupMenuHandler (line 71264) | function mxPopupMenuHandler(graph, factoryMethod)
  function mxCellMarker (line 71513) | function mxCellMarker(graph, validColor, invalidColor, hotspot)
  function mxSelectionCellsHandler (line 71925) | function mxSelectionCellsHandler(graph)
  function mxConnectionHandler (line 72356) | function mxConnectionHandler(graph, factoryMethod)
  function mxConstraintHandler (line 74459) | function mxConstraintHandler(graph)
  function mxRubberband (line 74975) | function mxRubberband(graph)
  function createMouseEvent (line 75154) | function createMouseEvent(evt)
  function mxHandle (line 75400) | function mxHandle(state, cursor, image)
  function mxVertexHandler (line 75753) | function mxVertexHandler(state)
  function checkShape (line 76248) | function checkShape(shape)
  function mxEdgeHandler (line 77729) | function mxEdgeHandler(state)
  function checkShape (line 78483) | function checkShape(shape)
  function snapToPoint (line 78731) | function snapToPoint(pt)
  function snapToTerminal (line 78754) | function snapToTerminal(terminal)
  function checkRemove (line 78956) | function checkRemove(idx, tmp)
  function mxElbowEdgeHandler (line 80170) | function mxElbowEdgeHandler(state)
  function mxEdgeSegmentHandler (line 80383) | function mxEdgeSegmentHandler(state)
  function mxKeyHandler (line 80857) | function mxKeyHandler(graph, target)
  function mxTooltipHandler (line 81248) | function mxTooltipHandler(graph, delay)
  function mxCellTracker (line 81642) | function mxCellTracker(graph, color, funct)
  function mxCellHighlight (line 81731) | function mxCellHighlight(graph, highlightColor, strokeWidth, dashed)
  function mxDefaultKeyHandler (line 82072) | function mxDefaultKeyHandler(editor)
  function mxDefaultPopupMenu (line 82185) | function mxDefaultPopupMenu(config)
  function mxDefaultToolbar (line 82500) | function mxDefaultToolbar(container, editor)
  function mxEditor (line 83370) | function mxEditor(config)
  function mxCodec (line 86391) | function mxCodec(document)
  function mxObjectCodec (line 87098) | function mxObjectCodec(template, exclude, idrefs, mapping)

FILE: spider-flow-web/src/main/resources/static/js/spider-editor.js
  function JsonProperty (line 1) | function JsonProperty(object){
  function SpiderEditor (line 13) | function SpiderEditor(options){
Condensed preview — 213 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (4,664K chars).
[
  {
    "path": ".gitattributes",
    "chars": 86,
    "preview": "*.js linguist-language=java\n*.css linguist-language=java\n*.html linguist-language=java"
  },
  {
    "path": ".gitignore",
    "chars": 77,
    "preview": "target\r\n*.iml\r\nout/\r\n.idea\r\n.classpath\r\n.project\r\n.settings\r\nbin/\r\n.myeclipse"
  },
  {
    "path": "Dockerfile",
    "chars": 222,
    "preview": "FROM java:8\n\nMAINTAINER octopus\n\nRUN mkdir -p /spider-flow\n\nWORKDIR /spider-flow\n\nEXPOSE 8088\n\nADD ./spider-flow-web/tar"
  },
  {
    "path": "LICENSE",
    "chars": 1080,
    "preview": "MIT License\r\n\r\nCopyright (c) 2019 小东\r\n\r\nPermission is hereby granted, free of charge, to any person obtaining a copy\r\nof"
  },
  {
    "path": "README.md",
    "chars": 3271,
    "preview": "<p align=\"center\">\r\n    <img src=\"https://www.spiderflow.org/images/logo.svg\" width=\"600\">\r\n</p>\r\n<p align=\"center\">\r\n  "
  },
  {
    "path": "db/spiderflow.sql",
    "chars": 22521,
    "preview": "SET FOREIGN_KEY_CHECKS=0;\n\nCREATE DATABASE spiderflow;\nUSE spiderflow;\n\nDROP TABLE IF EXISTS `sp_flow`;\nCREATE TABLE `sp"
  },
  {
    "path": "pom.xml",
    "chars": 5613,
    "preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<project xmlns=\"http://maven.apache.org/POM/4.0.0\" xmlns:xsi=\"http://www.w3.org/2"
  },
  {
    "path": "spider-flow-api/pom.xml",
    "chars": 642,
    "preview": "<?xml version=\"1.0\"?>\n<project xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/ExpressionEngine.java",
    "chars": 254,
    "preview": "package org.spiderflow;\n\nimport java.util.Map;\n\n/**\n * 表达式引擎\n */\npublic interface ExpressionEngine {\n\n\t/**\n\t * 执行表达式\n\t *"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/Grammerable.java",
    "chars": 149,
    "preview": "package org.spiderflow;\n\nimport java.util.List;\nimport org.spiderflow.model.Grammer;\n\npublic interface Grammerable {\n\n\tL"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/annotation/Comment.java",
    "chars": 433,
    "preview": "package org.spiderflow.annotation;\r\n\r\nimport java.lang.annotation.Documented;\r\nimport java.lang.annotation.ElementType;\r"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/annotation/Example.java",
    "chars": 414,
    "preview": "package org.spiderflow.annotation;\r\n\r\nimport java.lang.annotation.Documented;\r\nimport java.lang.annotation.ElementType;\r"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/annotation/Return.java",
    "chars": 420,
    "preview": "package org.spiderflow.annotation;\r\n\r\nimport java.lang.annotation.Documented;\r\nimport java.lang.annotation.ElementType;\r"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/common/CURDController.java",
    "chars": 1386,
    "preview": "package org.spiderflow.common;\r\n\r\nimport org.spiderflow.model.JsonBean;\r\nimport org.springframework.beans.factory.annota"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/concurrent/ChildPriorThreadSubmitStrategy.java",
    "chars": 1081,
    "preview": "package org.spiderflow.concurrent;\n\nimport org.spiderflow.model.SpiderNode;\n\nimport java.util.Comparator;\nimport java.ut"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/concurrent/LinkedThreadSubmitStrategy.java",
    "chars": 715,
    "preview": "package org.spiderflow.concurrent;\n\nimport org.spiderflow.model.SpiderNode;\n\nimport java.util.Comparator;\nimport java.ut"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/concurrent/ParentPriorThreadSubmitStrategy.java",
    "chars": 1090,
    "preview": "package org.spiderflow.concurrent;\n\nimport org.spiderflow.model.SpiderNode;\n\nimport java.util.Comparator;\nimport java.ut"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/concurrent/RandomThreadSubmitStrategy.java",
    "chars": 824,
    "preview": "package org.spiderflow.concurrent;\n\nimport org.apache.commons.lang3.RandomUtils;\nimport org.spiderflow.model.SpiderNode;"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/concurrent/SpiderFlowThreadPoolExecutor.java",
    "chars": 4279,
    "preview": "package org.spiderflow.concurrent;\r\n\r\nimport org.spiderflow.model.SpiderNode;\r\n\r\nimport java.util.concurrent.*;\r\nimport "
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/concurrent/SpiderFutureTask.java",
    "chars": 682,
    "preview": "package org.spiderflow.concurrent;\n\nimport java.util.concurrent.FutureTask;\nimport org.spiderflow.concurrent.SpiderFlowT"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/concurrent/ThreadSubmitStrategy.java",
    "chars": 288,
    "preview": "package org.spiderflow.concurrent;\n\nimport org.spiderflow.model.SpiderNode;\n\nimport java.util.Comparator;\n\npublic interf"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/context/CookieContext.java",
    "chars": 144,
    "preview": "package org.spiderflow.context;\n\nimport java.util.HashMap;\n\n/**\n * Cookie上下文\n */\npublic class CookieContext extends Hash"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/context/SpiderContext.java",
    "chars": 2202,
    "preview": "package org.spiderflow.context;\n\nimport org.spiderflow.concurrent.SpiderFlowThreadPoolExecutor.SubThreadPoolExecutor;\nim"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/context/SpiderContextHolder.java",
    "chars": 432,
    "preview": "package org.spiderflow.context;\n\nimport com.alibaba.ttl.TransmittableThreadLocal;\n\npublic class SpiderContextHolder {\n\n\t"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/enums/FlowNoticeType.java",
    "chars": 242,
    "preview": "package org.spiderflow.enums;\n\n/**\n * 流程通知类型\n * \n * @author BillDowney\n * @date 2020年4月4日 上午1:32:53\n */\npublic enum Flow"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/enums/FlowNoticeWay.java",
    "chars": 609,
    "preview": "package org.spiderflow.enums;\n\nimport java.util.LinkedHashMap;\nimport java.util.Map;\n\n/**\n * 流程通知方式\n * \n * @author BillD"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/executor/FunctionExecutor.java",
    "chars": 104,
    "preview": "package org.spiderflow.executor;\n\npublic interface FunctionExecutor {\n\t\n\tString getFunctionPrefix();\n\n}\n"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/executor/FunctionExtension.java",
    "chars": 96,
    "preview": "package org.spiderflow.executor;\n\npublic interface FunctionExtension {\n\t\n\tClass<?> support();\n}\n"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/executor/PluginConfig.java",
    "chars": 134,
    "preview": "package org.spiderflow.executor;\r\n\r\nimport org.spiderflow.model.Plugin;\r\n\r\npublic interface PluginConfig {\r\n\t\r\n\tPlugin p"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/executor/ShapeExecutor.java",
    "chars": 856,
    "preview": "package org.spiderflow.executor;\n\nimport java.util.Map;\n\nimport org.spiderflow.context.SpiderContext;\nimport org.spiderf"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/expression/DynamicMethod.java",
    "chars": 159,
    "preview": "package org.spiderflow.expression;\n\nimport java.util.List;\n\npublic interface DynamicMethod {\n\t\n\tObject execute(String me"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/io/Line.java",
    "chars": 661,
    "preview": "package org.spiderflow.io;\n\npublic class Line {\n\n\tprivate long from;\n\n\tprivate String text;\n\n\tprivate long to;\n\n\tpublic "
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/io/RandomAccessFileReader.java",
    "chars": 4029,
    "preview": "package org.spiderflow.io;\n\nimport java.io.Closeable;\nimport java.io.IOException;\nimport java.io.RandomAccessFile;\nimpor"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/io/SpiderResponse.java",
    "chars": 1119,
    "preview": "package org.spiderflow.io;\n\nimport java.io.InputStream;\nimport java.util.Map;\n\nimport org.spiderflow.annotation.Comment;"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/listener/SpiderListener.java",
    "chars": 244,
    "preview": "package org.spiderflow.listener;\n\nimport org.spiderflow.context.SpiderContext;\n\npublic interface SpiderListener {\n\n\t/**\n"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/model/Grammer.java",
    "chars": 2588,
    "preview": "package org.spiderflow.model;\r\n\r\nimport java.lang.reflect.Method;\r\nimport java.lang.reflect.Modifier;\r\nimport java.util."
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/model/JsonBean.java",
    "chars": 748,
    "preview": "package org.spiderflow.model;\n\npublic class JsonBean<T> {\n\n\tprivate Integer code = 1;\n\t\n\tprivate String message = \"执行成功\""
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/model/Plugin.java",
    "chars": 394,
    "preview": "package org.spiderflow.model;\n\npublic class Plugin {\n\n\tprivate String name;\n\t\n\tprivate String url;\n\t\n\tpublic Plugin(Stri"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/model/Shape.java",
    "chars": 778,
    "preview": "package org.spiderflow.model;\r\n\r\npublic class Shape {\r\n\t\r\n\tprivate String name;\r\n\t\r\n\tprivate String label;\r\n\t\r\n\tprivate "
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/model/SpiderLog.java",
    "chars": 1165,
    "preview": "package org.spiderflow.model;\r\n\r\nimport java.util.ArrayList;\r\nimport java.util.List;\r\n\r\nimport org.apache.commons.lang3."
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/model/SpiderNode.java",
    "chars": 4695,
    "preview": "package org.spiderflow.model;\r\n\r\nimport java.util.*;\r\nimport java.util.concurrent.atomic.AtomicInteger;\r\n\r\nimport org.ap"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/model/SpiderOutput.java",
    "chars": 1252,
    "preview": "package org.spiderflow.model;\r\n\r\nimport java.util.ArrayList;\r\nimport java.util.List;\r\n\r\npublic class SpiderOutput {\r\n\t\r\n"
  },
  {
    "path": "spider-flow-api/src/main/java/org/spiderflow/utils/Maps.java",
    "chars": 755,
    "preview": "package org.spiderflow.utils;\r\n\r\nimport java.util.HashMap;\r\nimport java.util.List;\r\nimport java.util.Map;\r\n\r\npublic clas"
  },
  {
    "path": "spider-flow-core/pom.xml",
    "chars": 790,
    "preview": "<?xml version=\"1.0\"?>\n<project xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/Spider.java",
    "chars": 11849,
    "preview": "package org.spiderflow.core;\n\nimport com.alibaba.ttl.TtlRunnable;\nimport org.apache.commons.lang3.StringUtils;\nimport or"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/Base64FunctionExecutor.java",
    "chars": 2853,
    "preview": "package org.spiderflow.core.executor.function;\r\n\r\nimport org.apache.commons.codec.binary.Base64;\r\nimport org.spiderflow."
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/DateFunctionExecutor.java",
    "chars": 3249,
    "preview": "package org.spiderflow.core.executor.function;\r\n\r\nimport java.text.ParseException;\r\nimport java.util.Date;\r\n\r\nimport org"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/ExtractFunctionExecutor.java",
    "chars": 5413,
    "preview": "package org.spiderflow.core.executor.function;\r\n\r\nimport java.util.List;\r\n\r\nimport org.jsoup.Jsoup;\r\nimport org.jsoup.no"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/FileFunctionExecutor.java",
    "chars": 3926,
    "preview": "package org.spiderflow.core.executor.function;\n\nimport java.io.*;\nimport java.nio.charset.Charset;\nimport java.util.List"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/JsonFunctionExecutor.java",
    "chars": 892,
    "preview": "package org.spiderflow.core.executor.function;\r\n\r\nimport org.spiderflow.annotation.Comment;\r\nimport org.spiderflow.annot"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/ListFunctionExecutor.java",
    "chars": 1962,
    "preview": "package org.spiderflow.core.executor.function;\r\n\r\nimport java.util.ArrayList;\r\nimport java.util.List;\r\nimport java.util."
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/MD5FunctionExecutor.java",
    "chars": 1510,
    "preview": "package org.spiderflow.core.executor.function;\n\nimport org.apache.commons.codec.digest.DigestUtils;\nimport org.spiderflo"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/RandomFunctionExecutor.java",
    "chars": 1468,
    "preview": "package org.spiderflow.core.executor.function;\r\n\r\nimport org.apache.commons.lang3.RandomUtils;\r\nimport org.spiderflow.an"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/StringFunctionExecutor.java",
    "chars": 5231,
    "preview": "package org.spiderflow.core.executor.function;\r\n\r\nimport java.io.UnsupportedEncodingException;\r\nimport java.util.ArrayLi"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/ThreadFunctionExecutor.java",
    "chars": 723,
    "preview": "package org.spiderflow.core.executor.function;\n\nimport org.spiderflow.annotation.Comment;\nimport org.spiderflow.annotati"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/UrlFunctionExecutor.java",
    "chars": 2869,
    "preview": "package org.spiderflow.core.executor.function;\r\n\r\nimport java.io.UnsupportedEncodingException;\r\nimport java.net.URLDecod"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ArrayFunctionExtension.java",
    "chars": 1096,
    "preview": "package org.spiderflow.core.executor.function.extension;\r\n\r\nimport java.util.Arrays;\r\nimport java.util.List;\r\n\r\nimport o"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/DateFunctionExtension.java",
    "chars": 818,
    "preview": "package org.spiderflow.core.executor.function.extension;\r\n\r\nimport java.util.Date;\r\n\r\nimport org.apache.commons.lang3.ti"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ElementFunctionExtension.java",
    "chars": 3323,
    "preview": "package org.spiderflow.core.executor.function.extension;\r\n\r\nimport java.util.List;\r\n\r\nimport org.jsoup.nodes.Element;\r\ni"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ElementsFunctionExtension.java",
    "chars": 5022,
    "preview": "package org.spiderflow.core.executor.function.extension;\r\n\r\nimport org.jsoup.nodes.Element;\r\nimport org.jsoup.select.Ele"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ListFunctionExtension.java",
    "chars": 1377,
    "preview": "package org.spiderflow.core.executor.function.extension;\r\n\r\nimport org.apache.commons.lang3.StringUtils;\r\nimport org.spi"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/MapFunctionExtension.java",
    "chars": 720,
    "preview": "package org.spiderflow.core.executor.function.extension;\n\nimport org.spiderflow.annotation.Comment;\nimport org.spiderflo"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ObjectFunctionExtension.java",
    "chars": 1244,
    "preview": "package org.spiderflow.core.executor.function.extension;\r\n\r\nimport java.util.Objects;\r\n\r\nimport org.spiderflow.annotatio"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/ResponseFunctionExtension.java",
    "chars": 5013,
    "preview": "package org.spiderflow.core.executor.function.extension;\r\n\r\nimport java.util.List;\r\nimport java.util.regex.Pattern;\r\nimp"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/SqlRowSetExtension.java",
    "chars": 1214,
    "preview": "package org.spiderflow.core.executor.function.extension;\n\nimport org.apache.commons.lang3.exception.ExceptionUtils;\nimpo"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/function/extension/StringFunctionExtension.java",
    "chars": 4701,
    "preview": "package org.spiderflow.core.executor.function.extension;\r\n\r\nimport com.alibaba.fastjson.JSON;\r\nimport org.apache.commons"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/CommentExecutor.java",
    "chars": 496,
    "preview": "package org.spiderflow.core.executor.shape;\n\nimport org.spiderflow.context.SpiderContext;\nimport org.spiderflow.executor"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/ExecuteSQLExecutor.java",
    "chars": 7193,
    "preview": "package org.spiderflow.core.executor.shape;\n\nimport org.apache.commons.lang3.StringUtils;\nimport org.apache.commons.lang"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/ForkJoinExecutor.java",
    "chars": 1238,
    "preview": "package org.spiderflow.core.executor.shape;\n\nimport org.spiderflow.context.SpiderContext;\nimport org.spiderflow.executor"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/FunctionExecutor.java",
    "chars": 1396,
    "preview": "package org.spiderflow.core.executor.shape;\r\n\r\nimport java.util.List;\r\nimport java.util.Map;\r\n\r\nimport org.apache.common"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/LoopExecutor.java",
    "chars": 725,
    "preview": "package org.spiderflow.core.executor.shape;\r\n\r\nimport java.util.Map;\r\n\r\nimport org.spiderflow.context.SpiderContext;\r\nim"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/OutputExecutor.java",
    "chars": 7309,
    "preview": "package org.spiderflow.core.executor.shape;\r\n\r\nimport com.alibaba.fastjson.JSON;\r\nimport org.apache.commons.csv.CSVForma"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/ProcessExecutor.java",
    "chars": 1504,
    "preview": "package org.spiderflow.core.executor.shape;\r\n\r\nimport java.util.Map;\r\n\r\nimport org.slf4j.Logger;\r\nimport org.slf4j.Logge"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/RequestExecutor.java",
    "chars": 17137,
    "preview": "package org.spiderflow.core.executor.shape;\n\nimport com.google.common.hash.BloomFilter;\nimport com.google.common.hash.Fu"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/StartExecutor.java",
    "chars": 565,
    "preview": "package org.spiderflow.core.executor.shape;\r\n\r\nimport java.util.Map;\r\n\r\nimport org.spiderflow.context.SpiderContext;\r\nim"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/executor/shape/VariableExecutor.java",
    "chars": 1706,
    "preview": "package org.spiderflow.core.executor.shape;\r\n\r\nimport java.util.List;\r\nimport java.util.Map;\r\n\r\nimport org.apache.common"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/DefaultExpressionEngine.java",
    "chars": 1646,
    "preview": "package org.spiderflow.core.expression;\r\n\r\nimport java.util.List;\r\nimport java.util.Map;\r\n\r\nimport javax.annotation.Post"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/ExpressionError.java",
    "chars": 3940,
    "preview": "\npackage org.spiderflow.core.expression;\n\nimport org.spiderflow.core.expression.parsing.Span;\nimport org.spiderflow.core"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/ExpressionGlobalVariables.java",
    "chars": 762,
    "preview": "package org.spiderflow.core.expression;\n\nimport java.util.HashMap;\nimport java.util.Map;\nimport java.util.concurrent.loc"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/ExpressionTemplate.java",
    "chars": 1426,
    "preview": "\npackage org.spiderflow.core.expression;\n\nimport java.io.OutputStream;\nimport java.util.List;\n\nimport org.spiderflow.cor"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/ExpressionTemplateContext.java",
    "chars": 3562,
    "preview": "\npackage org.spiderflow.core.expression;\n\nimport java.util.ArrayList;\nimport java.util.HashMap;\nimport java.util.HashSet"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/interpreter/AstInterpreter.java",
    "chars": 2366,
    "preview": "\npackage org.spiderflow.core.expression.interpreter;\n\nimport java.io.IOException;\nimport java.util.List;\n\nimport org.spi"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/interpreter/JavaReflection.java",
    "chars": 12565,
    "preview": "\npackage org.spiderflow.core.expression.interpreter;\n\nimport java.lang.reflect.Field;\nimport java.lang.reflect.Method;\ni"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/interpreter/Reflection.java",
    "chars": 2017,
    "preview": "\npackage org.spiderflow.core.expression.interpreter;\n\n/** Used by {@link AstInterpreter} to access fields and methods of"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/Ast.java",
    "chars": 47799,
    "preview": "\npackage org.spiderflow.core.expression.parsing;\n\nimport org.apache.commons.lang3.StringUtils;\nimport org.apache.commons"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/CharacterStream.java",
    "chars": 3987,
    "preview": "\npackage org.spiderflow.core.expression.parsing;\n\nimport javax.xml.transform.Source;\n\n/** Wraps a the content of a {@lin"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/Parser.java",
    "chars": 10732,
    "preview": "package org.spiderflow.core.expression.parsing;\r\n\r\n\r\n\r\nimport java.util.ArrayList;\r\nimport java.util.List;\r\n\r\nimport jav"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/Span.java",
    "chars": 3754,
    "preview": "\npackage org.spiderflow.core.expression.parsing;\n\n/** A span within a source string denoted by start and end index, with"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/Token.java",
    "chars": 521,
    "preview": "\npackage org.spiderflow.core.expression.parsing;\n\n/** A token produced by the {@link Tokenizer}. */\npublic class Token {"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/TokenStream.java",
    "chars": 3980,
    "preview": "\npackage org.spiderflow.core.expression.parsing;\n\nimport java.util.List;\n\nimport javax.xml.transform.Source;\n\nimport org"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/TokenType.java",
    "chars": 2984,
    "preview": "\npackage org.spiderflow.core.expression.parsing;\n\nimport java.util.Arrays;\nimport java.util.Comparator;\n\n/** Enumeration"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/expression/parsing/Tokenizer.java",
    "chars": 7007,
    "preview": "\npackage org.spiderflow.core.expression.parsing;\n\nimport java.util.ArrayList;\nimport java.util.List;\n\nimport org.spiderf"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/io/HttpRequest.java",
    "chars": 2966,
    "preview": "package org.spiderflow.core.io;\r\n\r\nimport java.io.IOException;\r\nimport java.io.InputStream;\r\nimport java.util.Map;\r\n\r\nim"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/io/HttpResponse.java",
    "chars": 1745,
    "preview": "package org.spiderflow.core.io;\n\nimport com.alibaba.fastjson.JSON;\nimport org.jsoup.Connection.Response;\nimport org.jsou"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/job/SpiderJob.java",
    "chars": 3171,
    "preview": "package org.spiderflow.core.job;\r\n\r\nimport org.apache.commons.lang3.time.DateFormatUtils;\r\nimport org.quartz.JobDataMap;"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/job/SpiderJobContext.java",
    "chars": 1802,
    "preview": "package org.spiderflow.core.job;\r\n\r\nimport java.io.File;\r\nimport java.io.FileOutputStream;\r\nimport java.io.OutputStream;"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/job/SpiderJobManager.java",
    "chars": 2288,
    "preview": "package org.spiderflow.core.job;\r\n\r\nimport org.quartz.CronScheduleBuilder;\r\nimport org.quartz.CronTrigger;\r\nimport org.q"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/mapper/DataSourceMapper.java",
    "chars": 357,
    "preview": "package org.spiderflow.core.mapper;\n\nimport java.util.List;\n\nimport org.apache.ibatis.annotations.Select;\nimport org.spi"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/mapper/FlowNoticeMapper.java",
    "chars": 261,
    "preview": "package org.spiderflow.core.mapper;\n\nimport com.baomidou.mybatisplus.core.mapper.BaseMapper;\nimport org.apache.ibatis.an"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/mapper/FunctionMapper.java",
    "chars": 256,
    "preview": "package org.spiderflow.core.mapper;\n\nimport com.baomidou.mybatisplus.core.mapper.BaseMapper;\nimport org.apache.ibatis.an"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/mapper/SpiderFlowMapper.java",
    "chars": 3204,
    "preview": "package org.spiderflow.core.mapper;\n\nimport java.util.Date;\nimport java.util.List;\n\nimport com.baomidou.mybatisplus.core"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/mapper/TaskMapper.java",
    "chars": 243,
    "preview": "package org.spiderflow.core.mapper;\n\nimport com.baomidou.mybatisplus.core.mapper.BaseMapper;\nimport org.apache.ibatis.an"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/mapper/VariableMapper.java",
    "chars": 202,
    "preview": "package org.spiderflow.core.mapper;\n\nimport com.baomidou.mybatisplus.core.mapper.BaseMapper;\nimport org.spiderflow.core."
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/model/DataSource.java",
    "chars": 1712,
    "preview": "package org.spiderflow.core.model;\n\nimport com.baomidou.mybatisplus.annotation.IdType;\nimport com.baomidou.mybatisplus.a"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/model/FlowNotice.java",
    "chars": 2198,
    "preview": "package org.spiderflow.core.model;\n\nimport org.spiderflow.enums.FlowNoticeWay;\n\nimport com.baomidou.mybatisplus.annotati"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/model/Function.java",
    "chars": 1154,
    "preview": "package org.spiderflow.core.model;\n\nimport com.baomidou.mybatisplus.annotation.IdType;\nimport com.baomidou.mybatisplus.a"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/model/SpiderFlow.java",
    "chars": 2459,
    "preview": "package org.spiderflow.core.model;\n\nimport com.baomidou.mybatisplus.annotation.IdType;\nimport com.baomidou.mybatisplus.a"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/model/Task.java",
    "chars": 855,
    "preview": "package org.spiderflow.core.model;\n\nimport com.baomidou.mybatisplus.annotation.IdType;\nimport com.baomidou.mybatisplus.a"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/model/Variable.java",
    "chars": 1026,
    "preview": "package org.spiderflow.core.model;\n\nimport com.baomidou.mybatisplus.annotation.IdType;\nimport com.baomidou.mybatisplus.a"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/script/ScriptManager.java",
    "chars": 4908,
    "preview": "package org.spiderflow.core.script;\n\nimport jdk.nashorn.api.scripting.ScriptObjectMirror;\nimport org.slf4j.Logger;\nimpor"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/serializer/FastJsonSerializer.java",
    "chars": 1359,
    "preview": "package org.spiderflow.core.serializer;\n\nimport com.alibaba.fastjson.serializer.JSONSerializer;\nimport com.alibaba.fastj"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/service/DataSourceService.java",
    "chars": 362,
    "preview": "package org.spiderflow.core.service;\r\n\r\nimport org.spiderflow.core.mapper.DataSourceMapper;\r\nimport org.spiderflow.core."
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/service/FlowNoticeService.java",
    "chars": 4003,
    "preview": "package org.spiderflow.core.service;\n\nimport com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;\n\nimport java.t"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/service/FunctionService.java",
    "chars": 1768,
    "preview": "package org.spiderflow.core.service;\n\nimport com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;\nimport org.apa"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/service/SpiderFlowService.java",
    "chars": 7179,
    "preview": "package org.spiderflow.core.service;\n\nimport com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;\nimport com.bao"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/service/TaskService.java",
    "chars": 316,
    "preview": "package org.spiderflow.core.service;\n\nimport com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;\nimport org.spi"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/service/VariableService.java",
    "chars": 1050,
    "preview": "package org.spiderflow.core.service;\n\nimport com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;\nimport org.spi"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/utils/DataSourceUtils.java",
    "chars": 1890,
    "preview": "package org.spiderflow.core.utils;\r\n\r\nimport java.util.HashMap;\r\nimport java.util.Map;\r\n\r\nimport javax.sql.DataSource;\r\n"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/utils/EmailUtils.java",
    "chars": 983,
    "preview": "package org.spiderflow.core.utils;\n\nimport org.springframework.beans.factory.annotation.Autowired;\nimport org.springfram"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/utils/ExecutorsUtils.java",
    "chars": 1592,
    "preview": "package org.spiderflow.core.utils;\n\nimport org.spiderflow.executor.ShapeExecutor;\nimport org.spiderflow.model.Shape;\nimp"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/utils/ExpressionUtils.java",
    "chars": 1772,
    "preview": "package org.spiderflow.core.utils;\n\nimport org.apache.commons.lang3.StringUtils;\nimport org.slf4j.Logger;\nimport org.slf"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/utils/ExtractUtils.java",
    "chars": 5426,
    "preview": "package org.spiderflow.core.utils;\r\n\r\nimport java.util.ArrayList;\r\nimport java.util.HashMap;\r\nimport java.util.List;\r\nim"
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/utils/FileUtils.java",
    "chars": 6551,
    "preview": "package org.spiderflow.core.utils;\n\nimport org.slf4j.Logger;\nimport org.slf4j.LoggerFactory;\n\nimport javax.servlet.http."
  },
  {
    "path": "spider-flow-core/src/main/java/org/spiderflow/core/utils/SpiderFlowUtils.java",
    "chars": 2821,
    "preview": "package org.spiderflow.core.utils;\r\n\r\nimport java.util.Collections;\r\nimport java.util.HashMap;\r\nimport java.util.Map;\r\ni"
  },
  {
    "path": "spider-flow-web/pom.xml",
    "chars": 1101,
    "preview": "<?xml version=\"1.0\"?>\n<project xsi:schemaLocation=\"http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/SpiderApplication.java",
    "chars": 1221,
    "preview": "package org.spiderflow;\r\n\r\nimport java.io.IOException;\r\n\r\nimport javax.servlet.ServletContext;\r\nimport javax.servlet.Ser"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/configuration/ResourcesConfiguration.java",
    "chars": 829,
    "preview": "package org.spiderflow.configuration;\n\nimport org.springframework.context.annotation.Configuration;\nimport org.springfra"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/configuration/WebSocketConfiguration.java",
    "chars": 719,
    "preview": "package org.spiderflow.configuration;\r\n\r\nimport org.spiderflow.core.Spider;\r\nimport org.spiderflow.websocket.WebSocketEd"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/controller/DataSourceController.java",
    "chars": 3107,
    "preview": "package org.spiderflow.controller;\r\n\r\nimport java.sql.Connection;\r\nimport java.sql.DriverManager;\r\nimport java.util.List"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/controller/FlowNoticeController.java",
    "chars": 1620,
    "preview": "package org.spiderflow.controller;\n\nimport java.util.Map;\n\nimport org.slf4j.Logger;\nimport org.slf4j.LoggerFactory;\nimpo"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/controller/FunctionController.java",
    "chars": 1698,
    "preview": "package org.spiderflow.controller;\n\nimport com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;\nimport com.baomi"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/controller/SpiderFlowController.java",
    "chars": 7109,
    "preview": "package org.spiderflow.controller;\n\nimport com.baomidou.mybatisplus.core.metadata.IPage;\nimport com.baomidou.mybatisplus"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/controller/SpiderRestController.java",
    "chars": 3382,
    "preview": "package org.spiderflow.controller;\r\n\r\nimport org.slf4j.Logger;\r\nimport org.slf4j.LoggerFactory;\r\nimport org.spiderflow.c"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/controller/TaskController.java",
    "chars": 1693,
    "preview": "package org.spiderflow.controller;\n\nimport com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;\nimport com.baomi"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/controller/VariableController.java",
    "chars": 500,
    "preview": "package org.spiderflow.controller;\n\nimport org.spiderflow.common.CURDController;\nimport org.spiderflow.core.mapper.Varia"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/logback/SpiderFlowFileAppender.java",
    "chars": 1661,
    "preview": "package org.spiderflow.logback;\n\nimport ch.qos.logback.classic.spi.ILoggingEvent;\nimport ch.qos.logback.core.FileAppende"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/logback/SpiderFlowWebSocketAppender.java",
    "chars": 1272,
    "preview": "package org.spiderflow.logback;\n\nimport ch.qos.logback.classic.spi.ILoggingEvent;\nimport ch.qos.logback.classic.spi.Thro"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/model/SpiderWebSocketContext.java",
    "chars": 3403,
    "preview": "package org.spiderflow.model;\n\nimport com.alibaba.fastjson.JSON;\nimport org.apache.commons.lang3.time.DateFormatUtils;\ni"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/model/WebSocketEvent.java",
    "chars": 875,
    "preview": "package org.spiderflow.model;\n\n/**\n * WebSocket事件\n * @author Administrator\n *\n * @param <T>\n */\npublic class WebSocketEv"
  },
  {
    "path": "spider-flow-web/src/main/java/org/spiderflow/websocket/WebSocketEditorServer.java",
    "chars": 2050,
    "preview": "package org.spiderflow.websocket;\r\n\r\nimport com.alibaba.fastjson.JSON;\r\nimport com.alibaba.fastjson.JSONObject;\r\nimport "
  },
  {
    "path": "spider-flow-web/src/main/resources/application.properties",
    "chars": 2095,
    "preview": "server.port=8088\n\nlogging.level.root=INFO\n#logging.level.org.spiderflow=DEBUG\n#平台最大线程数\nspider.thread.max=64\n#单任务默认最大线程数\n"
  },
  {
    "path": "spider-flow-web/src/main/resources/logback-spring.xml",
    "chars": 1448,
    "preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<configuration>\n    <springProperty scope=\"context\" name=\"LOG_LEVEL\" source=\"logg"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/css/easyui.css",
    "chars": 40366,
    "preview": ".panel{overflow:hidden;text-align:left}.panel-header,.panel-body{border-width:1px;border-style:solid}.panel-header{paddi"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/css/editor.css",
    "chars": 31884,
    "preview": "*{\r\n\tmargin : 0;\r\n\tpadding : 0;\r\n}\r\nhtml,body{\r\n\twidth : 100%;\r\n\theight : 100%;\r\n\toverflow: hidden;\r\n\tfont-family: Micro"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/css/index.css",
    "chars": 798,
    "preview": ".layui-body .layui-tab{margin:0px;height:100%;}\r\n.layui-body .layui-tab .layui-tab-content{position: absolute;top: 40px;"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/css/layui-black-gray.css",
    "chars": 900,
    "preview": ".menu-list,\n.layui-nav.layui-nav-tree,\n.layui-layout-admin .layui-logo{\n    background: #304156;\n}\n.layui-nav-tree .layu"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/css/layui-blue.css",
    "chars": 5143,
    "preview": "/* start */\r\niframe{\r\n\tbackground:#fff;\r\n}\r\n.layui-layout-admin .layui-header,.layui-laypage .layui-laypage-curr .layui-"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/datasource-edit.html",
    "chars": 3797,
    "preview": "<!DOCTYPE html>\n<html>\n<head>\n\t<meta charset=\"UTF-8\">\n\t<title>DataSource</title>\n\t<link rel=\"stylesheet\" href=\"js/layui/"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/datasources.html",
    "chars": 1902,
    "preview": "<!DOCTYPE html>\n<html>\n<head>\n\t<meta charset=\"UTF-8\">\n\t<title>DataSource</title>\n\t<link rel=\"stylesheet\" href=\"js/layui/"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/editCron.html",
    "chars": 21112,
    "preview": "<!DOCTYPE html>\n<html>\n<head>\n<title>Cron表达式生成器</title>\n<link href=\"js/cron/easyui.min.css\" rel=\"stylesheet\" type=\"text"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/editor.html",
    "chars": 8042,
    "preview": "<!DOCTYPE html>\r\n<html>\r\n\t<head>\r\n\t\t<meta charset=\"utf-8\" />\r\n\t\t<title>SpiderFlow-Editor</title>\r\n\t\t<link rel=\"styleshee"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/function-edit.html",
    "chars": 3423,
    "preview": "<!DOCTYPE html>\n<html>\n<head>\n\t<meta charset=\"UTF-8\">\n\t<title>DataSource</title>\n\t<link rel=\"stylesheet\" href=\"js/layui/"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/functions.html",
    "chars": 2378,
    "preview": "<!DOCTYPE html>\n<html>\n<head>\n\t<meta charset=\"UTF-8\">\n\t<title>DataSource</title>\n\t<link rel=\"stylesheet\" href=\"js/layui/"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/index.html",
    "chars": 3059,
    "preview": "<!DOCTYPE html>\r\n<html>\r\n\t<head>\r\n\t\t<meta charset=\"UTF-8\">\r\n\t\t<title>SpiderFlow</title>\r\n\t\t<link rel=\"stylesheet\" href=\""
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/canvas-viewer.js",
    "chars": 11535,
    "preview": "window.requestAnimFrame = window.requestAnimationFrame || window.webkitRequestAnimationFrame || window.mozRequestAnimati"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/codemirror/codemirror.css",
    "chars": 8948,
    "preview": "/* BASICS */\n\n.CodeMirror {\n  /* Set height, width, borders, and global font properties here */\n  font-family: monospace"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/codemirror/codemirror.js",
    "chars": 395658,
    "preview": "// CodeMirror, copyright (c) by Marijn Haverbeke and others\n// Distributed under an MIT license: https://codemirror.net/"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/codemirror/dracula.css",
    "chars": 2042,
    "preview": "/*\n\n    Name:       dracula\n    Author:     Michael Kaminsky (http://github.com/mkaminsky11)\n\n    Original dracula color"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/codemirror/idea.css",
    "chars": 1672,
    "preview": "/**\n    Name:       IDEA default theme\n    From IntelliJ IDEA by JetBrains\n */\n\n.cm-s-idea span.cm-meta { color: #808000"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/codemirror/javascript.js",
    "chars": 37616,
    "preview": "// CodeMirror, copyright (c) by Marijn Haverbeke and others\n// Distributed under an MIT license: https://codemirror.net/"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/codemirror/placeholder.js",
    "chars": 2275,
    "preview": "// CodeMirror, copyright (c) by Marijn Haverbeke and others\r\n// Distributed under an MIT license: https://codemirror.net"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/codemirror/show-hint.css",
    "chars": 638,
    "preview": ".CodeMirror-hints {\n  position: absolute;\n  z-index: 214483647;\n  /*overflow: hidden;*/\n  list-style: none;\n\n  margin: 0"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/codemirror/show-hint.js",
    "chars": 27200,
    "preview": "// CodeMirror, copyright (c) by Marijn Haverbeke and others\n// Distributed under an MIT license: http://codemirror.net/L"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/codemirror/spiderflow-hint.js",
    "chars": 4334,
    "preview": "var grammers = [];\r\n$.ajax({\r\n\turl : 'spider/grammers',\r\n\ttype : 'post',\r\n\tdataType : 'json',\r\n\tsuccess : function(json)"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/codemirror/spiderflow.js",
    "chars": 10760,
    "preview": "/**\r\n * freemarker\r\n */\r\n\r\n(function(mod) {\r\n\tif (typeof exports == \"object\" && typeof module == \"object\" ) // CommonJS\r"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/codemirror/sql.js",
    "chars": 47451,
    "preview": "// CodeMirror, copyright (c) by Marijn Haverbeke and others\n// Distributed under an MIT license: https://codemirror.net/"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/common.js",
    "chars": 1634,
    "preview": "function getQueryString(name) {\r\n    var reg = new RegExp('(^|&)' + name + '=([^&]*)(&|$)', 'i');\r\n    var r = window.lo"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/cron/cron.js",
    "chars": 12913,
    "preview": "function btnFan() {\r\n\t// 获取参数中表达式的值\r\n\tvar txt = $(\"#cron\").val();\r\n\tif (txt) {\r\n\t\tvar regs = txt.split(' ');\r\n\t\t$(\"inpu"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/editor.js",
    "chars": 46772,
    "preview": "var $ = layui.$;\nvar editor;\nvar flows;\nvar codeMirrorInstances = {};\nvar socket;\nvar version = 'lastest';\nfunction rend"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/index.js",
    "chars": 2205,
    "preview": "var $ = layui.$;\r\nfunction setCookie(name,value){\r\n\tvar Days = 30;\r\n\tvar exp = new Date();\r\n\texp.setTime(exp.getTime() +"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/jsontree/jsontree.css",
    "chars": 2609,
    "preview": "/*\r\n * JSON Tree Viewer\r\n * http://github.com/summerstyle/jsonTreeViewer\r\n *\r\n * Copyright 2017 Vera Lobacheva (http://i"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/jsontree/jsontree.js",
    "chars": 25491,
    "preview": "/**\r\n * JSON Tree library (a part of jsonTreeViewer)\r\n * http://github.com/summerstyle/jsonTreeViewer\r\n *\r\n * Copyright "
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/layui/css/layui.css",
    "chars": 69524,
    "preview": "/** layui-v2.4.5 MIT License By https://www.layui.com */\n .layui-inline,img{display:inline-block;vertical-align:middle}h"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/layui/css/layui.mobile.css",
    "chars": 9885,
    "preview": "/** layui-v2.4.5 MIT License By https://www.layui.com */\n blockquote,body,button,dd,div,dl,dt,form,h1,h2,h3,h4,h5,h6,inp"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/layui/css/modules/code.css",
    "chars": 1063,
    "preview": "/** layui-v2.4.5 MIT License By https://www.layui.com */\n html #layuicss-skincodecss{display:none;position:absolute;widt"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/layui/css/modules/laydate/default/laydate.css",
    "chars": 7537,
    "preview": "/** layui-v2.4.5 MIT License By https://www.layui.com */\n .laydate-set-ym,.layui-laydate,.layui-laydate *,.layui-laydate"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/layui/css/modules/layer/default/layer.css",
    "chars": 14425,
    "preview": "/** layui-v2.4.5 MIT License By https://www.layui.com */\n .layui-layer-imgbar,.layui-layer-imgtit a,.layui-layer-tab .la"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/layui/ext/eleTree/eleTree.css",
    "chars": 3244,
    "preview": "/* #region tree */\r\n.eleTree{\r\n    position: relative;\r\n}\r\n.eleTree-hide{\r\n    display: none;\r\n}\r\n.eleTree-loadData{\r\n  "
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/layui/ext/eleTree/eleTree.js",
    "chars": 58956,
    "preview": "/**\r\n * 基于layui的tree重写\r\n * author: hsianglee\r\n * 最近修改时间: 2019/01/07\r\n */\r\n\r\nlayui.define([\"jquery\",\"laytpl\"], function ("
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/layui/ext/treeselect/treeselect.js",
    "chars": 92242,
    "preview": "\r\nlayui.define(['form', 'jquery'], function (exports) { //提示:模块也可以依赖其它模块,如:layui.define('layer', callback);\r\n  var jQuer"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/layui/extends/formSelects-v4.css",
    "chars": 18727,
    "preview": "/* formSelects多选css */\r\nselect[xm-select]{display: none !important;}\r\n.xm-select-parent * {margin: 0;padding: 0;font-fam"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/layui/extends/formSelects-v4.js",
    "chars": 57386,
    "preview": "'use strict';\n\nvar _typeof = typeof Symbol === \"function\" && typeof Symbol.iterator === \"symbol\" ? function (obj) { retu"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/layui/extends/treeGrid.js",
    "chars": 102923,
    "preview": "/**\n @Name:treeGrid树状表格\n @Author:beijiyi\n @version: 0.1\n 码云地址:https://gitee.com/beijiyi/tree_table_treegrid_based_on_lay"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/layui/layui.all.js",
    "chars": 260159,
    "preview": "/** layui-v2.4.5 MIT License By https://www.layui.com */\n ;!function(e){\"use strict\";var t=document,o={modules:{},status"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/log-viewer.js",
    "chars": 5344,
    "preview": "function LogViewer(options){\n    options = options || {};\n    this.element = options.element;\n    this.maxLines = option"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/mxgraph/css/common.css",
    "chars": 3382,
    "preview": "div.mxRubberband {\n\tposition: absolute;\n\toverflow: hidden;\n\tborder-style: solid;\n\tborder-width: 1px;\n\tborder-color: #000"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/mxgraph/css/explorer.css",
    "chars": 486,
    "preview": "div.mxTooltip {\n\tfilter:progid:DXImageTransform.Microsoft.DropShadow(OffX=4, OffY=4, \n        Color='#A2A2A2', Positive="
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/mxgraph/mxgraph.js",
    "chars": 2287545,
    "preview": "/**\r\n * Copyright (c) 2006-2017, JGraph Ltd\r\n * Copyright (c) 2006-2017, Gaudenz Alder\r\n */\r\nvar mxClient =\r\n{\r\n\t/**\r\n\t "
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/mxgraph/resources/editor.txt",
    "chars": 83,
    "preview": "askZoom=Enter zoom (%)\nproperties=Properties\noutline=Outline\ntasks=Tasks\nhelp=Help\n"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/mxgraph/resources/editor_de.txt",
    "chars": 96,
    "preview": "askZoom=Zoom eingeben (%)\nproperties=Eigenschaften\noutline=Uebersicht\ntasks=Aufgaben\nhelp=Hilfe\n"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/mxgraph/resources/editor_zh.txt",
    "chars": 59,
    "preview": "askZoom=进入缩放(%25)\nproperties=属性\noutline=轮廓\ntasks=任务\nhelp=帮助"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/mxgraph/resources/graph.txt",
    "chars": 340,
    "preview": "alreadyConnected=Nodes already connected\ncontainsValidationErrors=Contains validation errors\nupdatingDocument=Updating D"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/mxgraph/resources/graph_de.txt",
    "chars": 373,
    "preview": "alreadyConnected=Knoten schon verbunden\ncontainsValidationErrors=Enthält Validierungsfehler\nupdatingDocument=Aktualisier"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/mxgraph/resources/graph_zh.txt",
    "chars": 216,
    "preview": "alreadyConnected=节点已经连接\ncontainsValidationErrors=包含效验错误\nupdatingDocument=更新文档。请等候......\nupdatingSelection=更新所选项。请等候....."
  },
  {
    "path": "spider-flow-web/src/main/resources/static/js/spider-editor.js",
    "chars": 8522,
    "preview": "function JsonProperty(object){\r\n\tthis.object = object || {};\r\n}\r\nJsonProperty.prototype.reset = function(object){\r\n\tretu"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/log.html",
    "chars": 11582,
    "preview": "<!DOCTYPE html>\r\n<html>\r\n\t<head>\r\n\t\t<meta charset=\"UTF-8\">\r\n\t\t<title>SpiderFlow</title>\r\n\t\t<script type=\"text/javascript"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/resources/templates/comment.html",
    "chars": 519,
    "preview": "<div class=\"layui-tab layui-tab-fixed layui-tab-brief\">\n  <ul class=\"layui-tab-title\">\n    <li class=\"layui-this\">配置</li"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/resources/templates/edge.html",
    "chars": 3869,
    "preview": "<div class=\"layui-tab layui-tab-fixed layui-tab-brief\">\r\n  <ul class=\"layui-tab-title\">\r\n    <li class=\"layui-this\">配置</"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/resources/templates/executeSql.html",
    "chars": 3381,
    "preview": "<div class=\"layui-tab layui-tab-fixed layui-tab-brief\">\n  <ul class=\"layui-tab-title\">\n    <li class=\"layui-this\">配置</li"
  },
  {
    "path": "spider-flow-web/src/main/resources/static/resources/templates/forkJoin.html",
    "chars": 572,
    "preview": "<div class=\"layui-tab layui-tab-fixed layui-tab-brief\">\n  <ul class=\"layui-tab-title\">\n    <li class=\"layui-this\">配置</li"
  }
]

// ... and 13 more files (download for full content)

About this extraction

This page contains the full source code of the ssssssss-team/spider-flow GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 213 files (4.0 MB), approximately 1.1M tokens, and a symbol index with 1942 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.

Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.

Copied to clipboard!