★ ★ ★ ★ ★ 100分
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | country | NULL | index | PRIMARY, country_id |
country | 152 | NULL | 0 | 0.00% | ☠️ O(n) | Using index |
| 1 | SIMPLE | city | NULL | ref | idx_fk_country_id, idx_country_id_city, idx_all, idx_other |
idx_fk_country_id | 2 | sakila.country.country_id | 0 | 0.00% | O(log n) | Using index |
index: 全表扫描, 只是扫描表的时候按照索引次序进行而不是行. 主要优点就是避免了排序, 但是开销仍然非常大.
ref: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
tag
func format(query string) string {
// This variable will be populated with formatted html
result := ""
// Use an actual tab while formatting and then switch out with self::$tab at the end
tab := " "
indentLevel := 0
var newline bool
var inlineParentheses bool
var increaseSpecialIndent bool
var increaseBlockIndent bool
var addedNewline bool
var inlineCount int
var inlineIndented bool
var clauseLimit bool
indentTypes := list.New()
// Tokenize String
originalTokens := Tokenize(query)
// Remove existing whitespace//
var tokens []Token
for i, token := range originalTokens {
if token.Type != TokenTypeWhitespace {
token.i = i
tokens = append(tokens, token)
}
}
for i, token := range tokens {
highlighted := token.Val
// If we are increasing the special indent level now
if increaseSpecialIndent {
indentLevel++
increaseSpecialIndent = false
indentTypes.PushFront("special")
}
// If we are increasing the block indent level now
if increaseBlockIndent {
indentLevel++
increaseBlockIndent = false
indentTypes.PushFront("block")
}
// If we need a new line before the token
if newline {
result += "\n" + strings.Repeat(tab, indentLevel)
newline = false
addedNewline = true
} else {
addedNewline = false
}
// Display comments directly where they appear in the source
if token.Type == TokenTypeComment || token.Type == TokenTypeBlockComment {
if token.Type == TokenTypeBlockComment {
indent := strings.Repeat(tab, indentLevel)
result += "\n" + indent
highlighted = strings.Replace(highlighted, "\n", "\n"+indent, -1)
}
result += highlighted
newline = true
continue
}
if inlineParentheses {
// End of inline parentheses
if token.Val == ")" {
result = strings.TrimRight(result, " ")
if inlineIndented {
indentTypes.Remove(indentTypes.Front())
if indentLevel > 0 {
indentLevel--
}
result += strings.Repeat(tab, indentLevel)
}
inlineParentheses = false
result += highlighted + " "
continue
}
if token.Val == "," {
if inlineCount >= 30 {
inlineCount = 0
newline = true
}
}
inlineCount += len(token.Val)
}
// Opening parentheses increase the block indent level and start a new line
if token.Val == "(" {
// First check if this should be an inline parentheses block
// Examples are "NOW()", "COUNT(*)", "int(10)", key(`somecolumn`), DECIMAL(7,2)
// Allow up to 3 non-whitespace tokens inside inline parentheses
length := 0
for j := 1; j <= 250; j++ {
// Reached end of string
if i+j >= len(tokens) {
break
}
next := tokens[i+j]
// Reached closing parentheses, able to inline it
if next.Val == ")" {
inlineParentheses = true
inlineCount = 0
inlineIndented = false
break
}
// Reached an invalid token for inline parentheses
if next.Val == ";" || next.Val == "(" {
break
}
// Reached an invalid token type for inline parentheses
if next.Type == TokenTypeReservedToplevel ||
next.Type == TokenTypeReservedNewline ||
next.Type == TokenTypeComment ||
next.Type == TokenTypeBlockComment {
break
}
length += len(next.Val)
}
if inlineParentheses && length > 30 {
increaseBlockIndent = true
inlineIndented = true
newline = true
}
// Take out the preceding space unless there was whitespace there in the original query
if token.i != 0 && (token.i-1) > len(originalTokens)-1 &&
originalTokens[token.i-1].Type != TokenTypeWhitespace {
result = strings.TrimRight(result, " ")
}
if inlineParentheses {
increaseBlockIndent = true
// Add a newline after the parentheses
newline = true
}
} else if token.Val == ")" {
// Closing parentheses decrease the block indent level
// Remove whitespace before the closing parentheses
result = strings.TrimRight(result, " ")
if indentLevel > 0 {
indentLevel--
}
// Reset indent level
for j := indentTypes.Front(); indentTypes.Len() > 0; indentTypes.Remove(j) {
if j.Value.(string) == "special" {
if indentLevel > 0 {
indentLevel--
} else {
break
}
} else {
break
}
}
if indentLevel < 0 {
// This is an error
indentLevel = 0
}
// Add a newline before the closing parentheses (if not already added)
if !addedNewline {
result += "\n" + strings.Repeat(tab, indentLevel)
}
} else if token.Type == TokenTypeReservedToplevel {
// Top level reserved words start a new line and increase the special indent level
increaseSpecialIndent = true
// If the last indent type was 'special', decrease the special indent for this round
if indentTypes.Len() > 0 && indentTypes.Front().Value.(string) == "special" {
if indentLevel > 0 {
indentLevel--
}
indentTypes.Remove(indentTypes.Front())
}
// Add a newline after the top level reserved word
newline = true
// Add a newline before the top level reserved word (if not already added)
if !addedNewline {
result += "\n" + strings.Repeat(tab, indentLevel)
} else {
// If we already added a newline, redo the indentation since it may be different now
result = strings.TrimSuffix(result, tab) + strings.Repeat(tab, indentLevel)
}
// If the token may have extra whitespace
if strings.Index(token.Val, " ") != 0 ||
strings.Index(token.Val, "\n") != 0 ||
strings.Index(token.Val, "\t") != 0 {
re, _ := regexp.Compile(`\s+`)
highlighted = re.ReplaceAllString(highlighted, " ")
}
//if SQL 'LIMIT' clause, start variable to reset newline
if token.Val == "LIMIT" && inlineParentheses {
clauseLimit = true
}
} else if clauseLimit && token.Val != "," &&
token.Type != TokenTypeNumber &&
token.Type != TokenTypeWhitespace {
// Checks if we are out of the limit clause
clauseLimit = false
} else if token.Val == "," && !inlineParentheses {
// Commas start a new line (unless within inline parentheses or SQL 'LIMIT' clause)
if clauseLimit {
newline = false
clauseLimit = false
} else {
// All other cases of commas
newline = true
}
} else if token.Type == TokenTypeReservedNewline {
// Newline reserved words start a new line
// Add a newline before the reserved word (if not already added)
if !addedNewline {
result += "\n" + strings.Repeat(tab, indentLevel)
}
// If the token may have extra whitespace
if strings.Index(token.Val, " ") != 0 ||
strings.Index(token.Val, "\n") != 0 ||
strings.Index(token.Val, "\t") != 0 {
re, _ := regexp.Compile(`\s+`)
highlighted = re.ReplaceAllString(highlighted, " ")
}
} else if token.Type == TokenTypeBoundary {
// Multiple boundary characters in a row should not have spaces between them (not including parentheses)
if i != 0 && i < len(tokens) &&
tokens[i-1].Type == TokenTypeBoundary {
if token.i != 0 && token.i < len(originalTokens) &&
originalTokens[token.i-1].Type != TokenTypeWhitespace {
result = strings.TrimRight(result, " ")
}
}
}
// If the token shouldn't have a space before it
if token.Val == "." || token.Val == "," || token.Val == ";" {
result = strings.TrimRight(result, " ")
}
result += highlighted + " "
// If the token shouldn't have a space after it
if token.Val == "(" || token.Val == "." {
result = strings.TrimRight(result, " ")
}
// If this is the "-" of a negative number, it shouldn't have a space after it
if token.Val == "-" && i+1 < len(tokens) && tokens[i+1].Type == TokenTypeNumber && i != 0 {
prev := tokens[i-1].Type
if prev != TokenTypeQuote &&
prev != TokenTypeBacktickQuote &&
prev != TokenTypeWord &&
prev != TokenTypeNumber {
result = strings.TrimRight(result, " ")
}
}
}
// Replace tab characters with the configuration tab character
result = strings.TrimRight(strings.Replace(result, "\t", tab, -1), " ")
return result
}
================================================
FILE: ast/pretty_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ast
import (
"fmt"
"testing"
"github.com/XiaoMi/soar/common"
"vitess.io/vitess/go/vt/sqlparser"
)
var TestSqlsPretty = []string{
"select sourcetable, if(f.lastcontent = ?, f.lastupdate, f.lastcontent) as lastactivity, f.totalcount as activity, type.class as type, (f.nodeoptions & ?) as nounsubscribe from node as f inner join contenttype as type on type.contenttypeid = f.contenttypeid inner join subscribed as sd on sd.did = f.nodeid and sd.userid = ? union all select f.name as title, f.userid as keyval, ? as sourcetable, ifnull(f.lastpost, f.joindate) as lastactivity, f.posts as activity, ? as type, ? as nounsubscribe from user as f inner join userlist as ul on ul.relationid = f.userid and ul.userid = ? where ul.type = ? and ul.aq = ? order by title limit ?",
"administrator command: Init DB",
"CALL foo(1, 2, 3)",
"### Channels ###\n\u0009\u0009\u0009\u0009\u0009SELECT sourcetable, IF(f.lastcontent = 0, f.lastupdate, f.lastcontent) AS lastactivity,\n\u0009\u0009\u0009\u0009\u0009f.totalcount AS activity, type.class AS type,\n\u0009\u0009\u0009\u0009\u0009(f.nodeoptions \u0026 512) AS noUnsubscribe\n\u0009\u0009\u0009\u0009\u0009FROM node AS f\n\u0009\u0009\u0009\u0009\u0009INNER JOIN contenttype AS type ON type.contenttypeid = f.contenttypeid \n\n\u0009\u0009\u0009\u0009\u0009INNER JOIN subscribed AS sd ON sd.did = f.nodeid AND sd.userid = 15965\n UNION ALL \n\n\u0009\u0009\u0009\u0009\u0009### Users ###\n\u0009\u0009\u0009\u0009\u0009SELECT f.name AS title, f.userid AS keyval, 'user' AS sourcetable, IFNULL(f.lastpost, f.joindate) AS lastactivity,\n\u0009\u0009\u0009\u0009\u0009f.posts as activity, 'Member' AS type,\n\u0009\u0009\u0009\u0009\u00090 AS noUnsubscribe\n\u0009\u0009\u0009\u0009\u0009FROM user AS f\n\u0009\u0009\u0009\u0009\u0009INNER JOIN userlist AS ul ON ul.relationid = f.userid AND ul.userid = 15965\n\u0009\u0009\u0009\u0009\u0009WHERE ul.type = 'f' AND ul.aq = 'yes'\n ORDER BY title ASC LIMIT 100",
"CREATE DATABASE org235_percona345 COLLATE 'utf8_general_ci'",
"insert into abtemp.coxed select foo.bar from foo",
"insert into foo(a, b, c) value(2, 4, 5)",
"insert into foo(a, b, c) values(2, 4, 5)",
"insert into foo(a, b, c) values(2, 4, 5) , (2,4,5)",
"insert into foo values (1, '(2)', 'This is a trick: ). More values.', 4)",
"insert into tb values (1)",
"INSERT INTO t (ts) VALUES ('()', '\\(', '\\)')",
"INSERT INTO t (ts) VALUES (NOW())",
"INSERT INTO t () VALUES ()",
"insert into t values (1), (2), (3)\n\n\ton duplicate key update query_count=1",
"insert into t values (1) on duplicate key update query_count=COALESCE(query_count, 0) + VALUES(query_count)",
"LOAD DATA INFILE '/tmp/foo.txt' INTO db.tbl",
"select 0e0, +6e-30, -6.00 from foo where a = 5.5 or b=0.5 or c=.5",
"select 0x0, x'123', 0b1010, b'10101' from foo",
"select 123_foo from 123_foo",
"select 123foo from 123foo",
`SELECT 1 AS one FROM calls USE INDEX(index_name)`,
"SELECT /*!40001 SQL_NO_CACHE */ * FROM `film`",
"SELECT 'a' 'b' 'c' 'd' FROM kamil",
"SELECT BENCHMARK(100000000, pow(rand(), rand())), 1 FROM `-hj-7d6-shdj5-7jd-kf-g988h-`.`-aaahj-7d6-shdj5-7&^%$jd-kf-g988h-9+4-5*6ab-`",
"SELECT c FROM org235.t WHERE id=0xdeadbeaf",
"select c from t where i=1 order by c asc",
"SELECT c FROM t WHERE id=0xdeadbeaf",
"SELECT c FROM t WHERE id=1",
"select `col` from `table-1` where `id` = 5",
"SELECT `db`.*, (CASE WHEN (`date_start` <= '2014-09-10 09:17:59' AND `date_end` >= '2014-09-10 09:17:59') THEN 'open' WHEN (`date_start` > '2014-09-10 09:17:59' AND `date_end` > '2014-09-10 09:17:59') THEN 'tbd' ELSE 'none' END) AS `status` FROM `foo` AS `db` WHERE (a_b in ('1', '10101'))",
"select field from `-master-db-1`.`-table-1-` order by id, ?;",
"select foo",
"select foo_1 from foo_2_3",
"select foo -- bar\n",
"select foo-- bar\n,foo",
"select '\\\\' from foo",
"select * from foo limit 5",
"select * from foo limit 5, 10",
"select * from foo limit 5 offset 10",
"SELECT * from foo where a = 5",
"select * from foo where a in (5) and b in (5, 8,9 ,9 , 10)",
"SELECT '' '' '' FROM kamil",
" select * from\nfoo where a = 5",
"SELECT * FROM prices.rt_5min where id=1",
"SELECT * FROM table WHERE field = 'value' /*arbitrary/31*/ ",
"SELECT * FROM table WHERE field = 'value' /*arbitrary31*/ ",
"SELECT * FROM t WHERE 1=1 AND id=1",
"select * from t where (base.nid IN ('1412', '1410', '1411'))",
`select * from t where i=1 order by
a, b ASC, d DESC,
e asc`,
"select * from t where i=1 order by a, b ASC, d DESC, e asc",
"select 'hello'\n",
"select 'hello', '\nhello\n', \"hello\", '\\'' from foo",
"SELECT ID, name, parent, type FROM posts WHERE _name IN ('perf','caching') AND (type = 'page' OR type = 'attachment')",
"SELECT name, value FROM variable",
"select \n-- bar\n foo",
"select null, 5.001, 5001. from foo",
"select sleep(2) from test.n",
"SELECT t FROM field WHERE (entity_type = 'node') AND (entity_id IN ('609')) AND (language IN ('und')) AND (deleted = '0') ORDER BY delta ASC",
"select t.table_schema,t.table_name,engine from information_schema.tables t inner join information_schema.columns c on t.table_schema=c.table_schema and t.table_name=c.table_name group by t.table_schema,t.table_name having sum(if(column_key in ('PRI','UNI'),1,0))=0",
"/* -- S++ SU ABORTABLE -- spd_user: rspadim */SELECT SQL_SMALL_RESULT SQL_CACHE DISTINCT centro_atividade FROM est_dia WHERE unidade_id=1001 AND item_id=67 AND item_id_red=573",
`UPDATE groups_search SET charter = ' -------3\'\' XXXXXXXXX.\n \n -----------------------------------------------------', show_in_list = 'Y' WHERE group_id='aaaaaaaa'`,
"use `foo`",
"select sourcetable, if(f.lastcontent = ?, f.lastupdate, f.lastcontent) as lastactivity, f.totalcount as activity, type.class as type, (f.nodeoptions & ?) as nounsubscribe from node as f inner join contenttype as type on type.contenttypeid = f.contenttypeid inner join subscribed as sd on sd.did = f.nodeid and sd.userid = ? union all select f.name as title, f.userid as keyval, ? as sourcetable, ifnull(f.lastpost, f.joindate) as lastactivity, f.posts as activity, ? as type, ? as nounsubscribe from user as f inner join userlist as ul on ul.relationid = f.userid and ul.userid = ? where ul.type = ? and ul.aq = ? order by title limit ?",
"CREATE INDEX part_of_name ON customer (name(10));",
"alter table `sakila`.`t1` add index `idx_col`(`col`)",
"alter table `sakila`.`t1` add UNIQUE index `idx_col`(`col`)",
"alter table `sakila`.`t1` add index `idx_ID`(`ID`)",
// ADD|DROP COLUMN
"ALTER TABLE t2 DROP COLUMN c, DROP COLUMN d;",
"ALTER TABLE T2 ADD COLUMN C int;",
"ALTER TABLE T2 ADD COLUMN D int FIRST;",
"ALTER TABLE T2 ADD COLUMN E int AFTER D;",
// RENAME COLUMN
"ALTER TABLE t1 RENAME COLUMN a TO b",
// RENAME INDEX
"ALTER TABLE t1 RENAME INDEX idx_a TO idx_b",
"ALTER TABLE t1 RENAME KEY idx_a TO idx_b",
// RENAME TABLE
"ALTER TABLE db.old_table RENAME new_table;",
"ALTER TABLE old_table RENAME TO new_table;",
"ALTER TABLE old_table RENAME AS new_table;",
// MODIFY & CHANGE
"ALTER TABLE t1 MODIFY col1 BIGINT UNSIGNED DEFAULT 1 COMMENT 'my column';",
"ALTER TABLE t1 CHANGE b a INT NOT NULL;",
// COMMENT
"/*!40000 select 1*/;",
}
func TestPretty(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
err := common.GoldenDiff(func() {
for _, sql := range append(TestSqlsPretty, common.TestSQLs...) {
fmt.Println(sql)
fmt.Println(Pretty(sql, "builtin"))
}
orgMaxPrettySQLLength := common.Config.MaxPrettySQLLength
common.Config.MaxPrettySQLLength = 1
fmt.Println(Pretty("select 1", "builtin"))
common.Config.MaxPrettySQLLength = orgMaxPrettySQLLength
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestIsKeyword(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
tks := map[string]bool{
"AGAINST": true,
"AUTO_INCREMENT": true,
"ADD": true,
"BETWEEN": true,
".": false,
"actions": false,
`"`: false,
":": false,
}
for tk, v := range tks {
if IsMysqlKeyword(tk) != v {
t.Error("isKeyword:", tk)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRemoveComments(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
for _, sql := range TestSqlsPretty {
stmt, _ := sqlparser.Parse(sql)
newSQL := sqlparser.String(stmt)
if newSQL != sql {
fmt.Print(newSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
================================================
FILE: ast/rewrite.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ast
import (
"bytes"
"encoding/json"
"fmt"
"github.com/pingcap/parser/ast"
"reflect"
"regexp"
"strings"
"github.com/XiaoMi/soar/common"
"github.com/kr/pretty"
"vitess.io/vitess/go/vt/sqlparser"
)
// Rule SQL重写规则
type Rule struct {
Name string `json:"Name"`
Description string `json:"Description"`
Original string `json:"Original"` // 错误示范。为空或"暂不支持"不会出现在list-rewrite-rules中
Suggest string `json:"Suggest"` // 正确示范。
Func func(*Rewrite) *Rewrite `json:"-"` // 如果不定义 Func 需要多条 SQL 联动改写
}
// RewriteRules SQL重写规则，注意这个规则是有序的，先后顺序不能乱
var RewriteRules []Rule
func init() {
RewriteRules = []Rule{
{
Name: "dml2select",
Description: "将数据库更新请求转换为只读查询请求，便于执行EXPLAIN",
Original: "DELETE FROM film WHERE length > 100",
Suggest: "select * from film where length > 100",
Func: (*Rewrite).RewriteDML2Select,
},
{
Name: "reg2select",
Description: "使用正则的方式将数据库更新请求转换为只读查询请求，便于执行EXPLAIN",
Original: "DELETE FROM film WHERE length > 100",
Suggest: "select * from film where length > 100",
Func: (*Rewrite).RewriteReg2Select,
},
{
Name: "star2columns",
Description: "为SELECT *补全表的列信息",
Original: "SELECT * FROM film",
Suggest: "select film.film_id, film.title from film",
Func: (*Rewrite).RewriteStar2Columns,
},
{
Name: "insertcolumns",
Description: "为INSERT补全表的列信息",
Original: "insert into film values(1,2,3,4,5)",
Suggest: "insert into film(film_id, title, description, release_year, language_id) values (1, 2, 3, 4, 5)",
Func: (*Rewrite).RewriteInsertColumns,
},
{
Name: "having",
Description: "将查询的 HAVING 子句改写为 WHERE 中的查询条件",
Original: "SELECT state, COUNT(*) FROM Drivers GROUP BY state HAVING state IN ('GA', 'TX') ORDER BY state",
Suggest: "select state, COUNT(*) from Drivers where state in ('GA', 'TX') group by state order by state asc",
Func: (*Rewrite).RewriteHaving,
},
{
Name: "orderbynull",
Description: "如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 ORDER BY NULL",
Original: "SELECT sum(col1) FROM tbl GROUP BY col",
Suggest: "select sum(col1) from tbl group by col order by null",
Func: (*Rewrite).RewriteAddOrderByNull,
},
{
Name: "unionall",
Description: "可以接受重复的时间，使用 UNION ALL 替代 UNION 以提高查询效率",
Original: "select country_id from city union select country_id from country",
Suggest: "select country_id from city union all select country_id from country",
Func: (*Rewrite).RewriteUnionAll,
},
{
Name: "or2in",
Description: "将同一列不同条件的 OR 查询转写为 IN 查询",
Original: "select country_id from city where col1 = 1 or (col2 = 1 or col2 = 2 ) or col1 = 3;",
Suggest: "select country_id from city where (col2 in (1, 2)) or col1 in (1, 3);",
Func: (*Rewrite).RewriteOr2In,
},
{
Name: "innull",
Description: "如果 IN 条件中可能有 NULL 值而又想匹配 NULL 值时，建议添加OR col IS NULL",
Original: "暂不支持",
Suggest: "暂不支持",
Func: (*Rewrite).RewriteInNull,
},
// 把所有跟 or 相关的重写完之后才进行 or 转 union 的重写
{
Name: "or2union",
Description: "将不同列的 OR 查询转为 UNION 查询，建议结合 unionall 重写策略一起使用",
Original: "暂不支持",
Suggest: "暂不支持",
Func: (*Rewrite).RewriteOr2Union,
},
{
Name: "dmlorderby",
Description: "删除 DML 更新操作中无意义的 ORDER BY",
Original: "DELETE FROM tbl WHERE col1=1 ORDER BY col",
Suggest: "delete from tbl where col1 = 1",
Func: (*Rewrite).RewriteRemoveDMLOrderBy,
},
/*
{
Name: "groupbyconst",
Description: "删除无意义的GROUP BY常量",
Original: "SELECT sum(col1) FROM tbl GROUP BY 1;",
Suggest: "select sum(col1) from tbl",
Func: (*Rewrite).RewriteGroupByConst,
},
*/
{
Name: "sub2join",
Description: "将子查询转换为JOIN查询",
Original: "暂不支持",
Suggest: "暂不支持",
Func: (*Rewrite).RewriteSubQuery2Join,
},
{
Name: "join2sub",
Description: "将JOIN查询转换为子查询",
Original: "暂不支持",
Suggest: "暂不支持",
Func: (*Rewrite).RewriteJoin2SubQuery,
},
{
Name: "distinctstar",
Description: "DISTINCT *对有主键的表没有意义，可以将DISTINCT删掉",
Original: "SELECT DISTINCT * FROM film;",
Suggest: "SELECT * FROM film",
Func: (*Rewrite).RewriteDistinctStar,
},
{
Name: "standard",
Description: "SQL标准化，如：关键字转换为小写",
Original: "SELECT sum(col1) FROM tbl GROUP BY 1;",
Suggest: "select sum(col1) from tbl group by 1",
Func: (*Rewrite).RewriteStandard,
},
{
Name: "mergealter",
Description: "合并同一张表的多条ALTER语句",
Original: "ALTER TABLE t2 DROP COLUMN c;ALTER TABLE t2 DROP COLUMN d;",
Suggest: "ALTER TABLE t2 DROP COLUMN c, DROP COLUMN d;",
},
{
Name: "alwaystrue",
Description: "删除无用的恒真判断条件",
Original: "SELECT count(col) FROM tbl where 'a'= 'a' or ('b' = 'b' and a = 'b');",
Suggest: "select count(col) from tbl where (a = 'b');",
Func: (*Rewrite).RewriteAlwaysTrue,
},
{
Name: "countstar",
Description: "不建议使用COUNT(col)或COUNT(常量)，建议改写为COUNT(*)",
Original: "SELECT count(col) FROM tbl GROUP BY 1;",
Suggest: "SELECT count(*) FROM tbl GROUP BY 1;",
Func: (*Rewrite).RewriteCountStar,
},
{
Name: "innodb",
Description: "建表时建议使用InnoDB引擎，非 InnoDB 引擎表自动转 InnoDB",
Original: "CREATE TABLE t1(id bigint(20) NOT NULL AUTO_INCREMENT);",
Suggest: "create table t1 (\n\tid bigint(20) not null auto_increment\n) ENGINE=InnoDB;",
Func: (*Rewrite).RewriteInnoDB,
},
{
Name: "autoincrement",
Description: "将autoincrement初始化为1",
Original: "CREATE TABLE t1(id bigint(20) NOT NULL AUTO_INCREMENT) ENGINE=InnoDB AUTO_INCREMENT=123802;",
Suggest: "create table t1(id bigint(20) not null auto_increment) ENGINE=InnoDB auto_increment=1;",
Func: (*Rewrite).RewriteAutoIncrement,
},
{
Name: "intwidth",
Description: "整型数据类型修改默认显示宽度",
Original: "create table t1 (id int(20) not null auto_increment) ENGINE=InnoDB;",
Suggest: "create table t1 (id int(10) not null auto_increment) ENGINE=InnoDB;",
Func: (*Rewrite).RewriteIntWidth,
},
{
Name: "truncate",
Description: "不带 WHERE 条件的 DELETE 操作建议修改为 TRUNCATE",
Original: "DELETE FROM tbl",
Suggest: "truncate table tbl",
Func: (*Rewrite).RewriteTruncate,
},
{
Name: "rmparenthesis",
Description: "去除没有意义的括号",
Original: "select col from table where (col = 1);",
Suggest: "select col from table where col = 1;",
Func: (*Rewrite).RewriteRmParenthesis,
},
// delimiter要放在最后，不然补不上
{
Name: "delimiter",
Description: "补全DELIMITER",
Original: "use sakila",
Suggest: "use sakila;",
Func: (*Rewrite).RewriteDelimiter,
},
// TODO in to exists
// TODO exists to in
}
}
// ListRewriteRules 打印SQL重写规则
func ListRewriteRules(rules []Rule) {
switch common.Config.ReportType {
case "json":
js, err := json.MarshalIndent(rules, "", " ")
if err == nil {
fmt.Println(string(js))
}
default:
fmt.Print("# 重写规则\n\n[toc]\n\n")
for _, r := range rules {
if !common.Config.Verbose && (r.Original == "" || r.Original == "暂不支持") {
continue
}
fmt.Print("## ", common.MarkdownEscape(r.Name),
"\n* **Description**:", r.Description+"\n",
"\n* **Original**:\n\n```sql\n", r.Original, "\n```\n",
"\n* **Suggest**:\n\n```sql\n", r.Suggest, "\n```\n")
}
}
}
// Rewrite 用于重写SQL
type Rewrite struct {
SQL string
NewSQL string
Stmt sqlparser.Statement
Columns common.TableColumns
}
// NewRewrite 返回一个*Rewrite对象，如果SQL无法被正常解析，将错误输出到日志中，返回一个nil
func NewRewrite(sql string) *Rewrite {
stmt, err := sqlparser.Parse(sql)
if err != nil {
common.Log.Error(err.Error(), sql)
return nil
}
return &Rewrite{
SQL: sql,
Stmt: stmt,
}
}
// Rewrite 入口函数
func (rw *Rewrite) Rewrite() *Rewrite {
defer func() {
if err := recover(); err != nil {
common.Log.Error("Query rewrite Error: %s, maybe hit a bug.\nQuery: %s \nAST: %s",
err, rw.SQL, pretty.Sprint(rw.Stmt))
return
}
}()
for _, rule := range RewriteRules {
if RewriteRuleMatch(rule.Name) && rule.Func != nil {
rule.Func(rw)
common.Log.Debug("Rewrite Rule:%s Output NewSQL: %s", rule.Name, rw.NewSQL)
}
}
if rw.NewSQL == "" {
rw.NewSQL = rw.SQL
}
rw.Stmt, _ = sqlparser.Parse(rw.NewSQL)
// TODO: 重新前后返回结果一致性对比
// TODO: 前后SQL性能对比
return rw
}
// RewriteDelimiter delimiter: 补分号，可以指定不同的DELIMITER
func (rw *Rewrite) RewriteDelimiter() *Rewrite {
if rw.NewSQL != "" {
rw.NewSQL = strings.TrimSuffix(rw.NewSQL, common.Config.Delimiter) + common.Config.Delimiter
} else {
rw.NewSQL = strings.TrimSuffix(rw.SQL, common.Config.Delimiter) + common.Config.Delimiter
}
return rw
}
// RewriteStandard standard: 使用 vitess 提供的 String 功能将抽象语法树转写回 SQL，注意：这可能转写失败。
func (rw *Rewrite) RewriteStandard() *Rewrite {
if _, err := sqlparser.Parse(rw.SQL); err == nil {
rw.NewSQL = sqlparser.String(rw.Stmt)
}
return rw
}
// RewriteAlwaysTrue always true: 删除恒真条件
func (rw *Rewrite) RewriteAlwaysTrue() (reWriter *Rewrite) {
array := NewNodeList(rw.Stmt)
tNode := array.Head
for {
omitAlwaysTrue(tNode)
tNode = tNode.Next
if tNode == nil {
break
}
}
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// isAlwaysTrue 用于判断ComparisonExpr是否是恒真
func isAlwaysTrue(expr *sqlparser.ComparisonExpr) bool {
if expr == nil {
return true
}
var result bool
switch expr.Operator {
case "<>":
expr.Operator = "!="
case "<=>":
expr.Operator = "="
case ">=", "<=", "!=", "=", ">", "<":
default:
return false
}
var left []byte
var right []byte
// left
switch l := expr.Left.(type) {
case *sqlparser.SQLVal:
left = l.Val
default:
return false
}
// right
switch r := expr.Right.(type) {
case *sqlparser.SQLVal:
right = r.Val
default:
return false
}
switch expr.Operator {
case "=":
result = bytes.Equal(left, right)
case "!=":
result = !bytes.Equal(left, right)
case ">":
result = bytes.Compare(left, right) > 0
case ">=":
result = bytes.Compare(left, right) >= 0
case "<":
result = bytes.Compare(left, right) < 0
case "<=":
result = bytes.Compare(left, right) <= 0
default:
result = false
}
return result
}
// omitAlwaysTrue 移除AST中的恒真条件
func omitAlwaysTrue(node *NodeItem) {
if node == nil {
return
}
switch self := node.Self.(type) {
case *sqlparser.Where:
if self != nil {
switch cond := self.Expr.(type) {
case *sqlparser.ComparisonExpr:
if isAlwaysTrue(cond) {
self.Expr = nil
}
case *sqlparser.ParenExpr:
if cond.Expr == nil {
self.Expr = nil
}
}
}
case *sqlparser.ParenExpr:
if self != nil {
switch cond := self.Expr.(type) {
case *sqlparser.ComparisonExpr:
if isAlwaysTrue(cond) {
self.Expr = nil
}
}
}
case *sqlparser.AndExpr:
if self != nil {
var tmp sqlparser.Expr
isRightTrue := false
isLeftTrue := false
tmp = nil
// 查看左树的情况
switch l := self.Left.(type) {
case *sqlparser.ComparisonExpr:
if isAlwaysTrue(l) {
self.Left = nil
isLeftTrue = true
tmp = self.Right
}
case *sqlparser.ParenExpr:
if l.Expr == nil {
self.Left = nil
isLeftTrue = true
tmp = self.Right
}
default:
if l == nil {
isLeftTrue = true
tmp = self.Right
}
}
// 查看右树的情况
switch r := self.Right.(type) {
case *sqlparser.ComparisonExpr:
if isAlwaysTrue(r) {
self.Right = nil
isRightTrue = true
tmp = self.Left
}
case *sqlparser.ParenExpr:
if r.Expr == nil {
self.Right = nil
isRightTrue = true
tmp = self.Left
}
default:
if r == nil {
isRightTrue = true
tmp = self.Left
}
}
if isRightTrue && isLeftTrue {
tmp = nil
} else if !isLeftTrue && !isRightTrue {
return
}
// 根据类型开始替换节点
switch l := node.Prev.Self.(type) {
case *sqlparser.Where:
l.Expr = tmp
case *sqlparser.ParenExpr:
l.Expr = tmp
case *sqlparser.AndExpr:
if l.Left == self {
l.Left = tmp
} else if l.Right == self {
l.Right = tmp
}
case *sqlparser.OrExpr:
if l.Left == self {
l.Left = tmp
} else if l.Right == self {
l.Right = tmp
}
default:
// 未匹配到对应数据类型则从链表中移除该节点
err := node.Array.Remove(node.Prev)
common.LogIfError(err, "")
}
}
case *sqlparser.OrExpr:
// 与AndExpr相同
if self != nil {
var tmp sqlparser.Expr
isRightTrue := false
isLeftTrue := false
tmp = nil
switch l := self.Left.(type) {
case *sqlparser.ComparisonExpr:
if isAlwaysTrue(l) {
self.Left = nil
isLeftTrue = true
tmp = self.Right
}
case *sqlparser.ParenExpr:
if l.Expr == nil {
self.Left = nil
isLeftTrue = true
tmp = self.Right
}
default:
if l == nil {
isLeftTrue = true
tmp = self.Right
}
}
switch r := self.Right.(type) {
case *sqlparser.ComparisonExpr:
if isAlwaysTrue(r) {
self.Right = nil
isRightTrue = true
tmp = self.Left
}
case *sqlparser.ParenExpr:
if r.Expr == nil {
self.Right = nil
isRightTrue = true
tmp = self.Left
}
default:
if r == nil {
isRightTrue = true
tmp = self.Left
}
}
if isRightTrue && isLeftTrue {
tmp = nil
} else if !isLeftTrue && !isRightTrue {
return
}
switch l := node.Prev.Self.(type) {
case *sqlparser.Where:
l.Expr = tmp
case *sqlparser.ParenExpr:
l.Expr = tmp
case *sqlparser.AndExpr:
if l.Left == self {
l.Left = tmp
} else if l.Right == self {
l.Right = tmp
}
case *sqlparser.OrExpr:
if l.Left == self {
l.Left = tmp
} else if l.Right == self {
l.Right = tmp
}
default:
err := node.Array.Remove(node.Prev)
common.LogIfError(err, "")
}
}
}
omitAlwaysTrue(node.Prev)
}
// RewriteCountStar countstar: 将COUNT(col)改写为COUNT(*)
// COUNT(DISTINCT col)不能替换为COUNT(*)
func (rw *Rewrite) RewriteCountStar() *Rewrite {
err := sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch f := node.(type) {
case *sqlparser.FuncExpr:
if strings.ToLower(f.Name.String()) == "count" && len(f.Exprs) > 0 {
switch colExpr := f.Exprs[0].(type) {
case *sqlparser.AliasedExpr:
switch col := colExpr.Expr.(type) {
case *sqlparser.ColName:
f.Exprs[0] = &sqlparser.StarExpr{TableName: col.Qualifier}
}
}
}
}
return true, nil
}, rw.Stmt)
common.LogIfError(err, "")
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// RewriteInnoDB InnoDB: 为未指定 Engine 的表默认添加 InnoDB 引擎，将其他存储引擎转为 InnoDB
func (rw *Rewrite) RewriteInnoDB() *Rewrite {
switch create := rw.Stmt.(type) {
case *sqlparser.DDL:
if create.Action != "create" {
return rw
}
if strings.Contains(strings.ToLower(create.TableSpec.Options), "engine=") {
reg := regexp.MustCompile(`(?i)engine=[a-z]+`)
create.TableSpec.Options = reg.ReplaceAllString(create.TableSpec.Options, "ENGINE=InnoDB ")
} else {
create.TableSpec.Options = " ENGINE=InnoDB " + create.TableSpec.Options
}
}
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// RewriteAutoIncrement autoincrement: 将auto_increment设置为1
func (rw *Rewrite) RewriteAutoIncrement() *Rewrite {
switch create := rw.Stmt.(type) {
case *sqlparser.DDL:
if create.Action != "create" || create.TableSpec == nil {
return rw
}
if strings.Contains(strings.ToLower(create.TableSpec.Options), "auto_increment=") {
reg := regexp.MustCompile(`(?i)auto_increment=[0-9]+`)
create.TableSpec.Options = reg.ReplaceAllString(create.TableSpec.Options, "auto_increment=1 ")
}
}
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// RewriteIntWidth intwidth: int 类型转为 int(10)，bigint 类型转为 bigint(20)
func (rw *Rewrite) RewriteIntWidth() *Rewrite {
switch create := rw.Stmt.(type) {
case *sqlparser.DDL:
if create.Action != "create" || create.TableSpec == nil {
return rw
}
for _, col := range create.TableSpec.Columns {
switch col.Type.Type {
case "int", "integer":
if col.Type.Length != nil &&
(string(col.Type.Length.Val) != "10" && string(col.Type.Length.Val) != "11") {
col.Type.Length = sqlparser.NewIntVal([]byte("10"))
}
case "bigint":
if col.Type.Length != nil && string(col.Type.Length.Val) != "20" || col.Type.Length == nil {
col.Type.Length = sqlparser.NewIntVal([]byte("20"))
}
default:
}
}
}
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// RewriteStar2Columns star2columns: 对应COL.001，SELECT补全*指代的列名
func (rw *Rewrite) RewriteStar2Columns() *Rewrite {
// 如果未配置mysql环境或从环境中获取失败，*不进行替换
if common.Config.TestDSN.Disable || len(rw.Columns) == 0 {
common.Log.Debug("(rw *Rewrite) RewriteStar2Columns(): Rewrite failed. TestDSN.Disable: %v, len(rw.Columns):%d",
common.Config.TestDSN.Disable, len(rw.Columns))
return rw
}
// 单张表 select * 不补全表名，避免SQL过长，多张表的 select tb1.*, tb2.* 需要补全表名
var multiTable bool
if len(rw.Columns) > 1 {
multiTable = true
} else {
for db := range rw.Columns {
if len(rw.Columns[db]) > 1 {
multiTable = true
}
}
}
err := sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch n := node.(type) {
case *sqlparser.Select:
// select * 可能出现的情况：
// 1. select * from tb;
// 2. select * from tb1,tb2;
// 3. select tb1.* from tb1;
// 4. select tb1.*,tb2.col from tb1,tb2;
// 5. select db.tb1.* from tb1;
// 6. select db.tb1.*,db.tb2.col from db.tb1,db.tb2;
newSelectExprs := make(sqlparser.SelectExprs, 0)
for _, expr := range n.SelectExprs {
switch e := expr.(type) {
case *sqlparser.StarExpr:
// 一般情况下最外层循环不会超过两层
for _, tables := range rw.Columns {
for _, cols := range tables {
for _, col := range cols {
var table string
if multiTable {
table = col.Table
}
newExpr := &sqlparser.AliasedExpr{
Expr: &sqlparser.ColName{
Metadata: nil,
Name: sqlparser.NewColIdent(col.Name),
Qualifier: sqlparser.TableName{
Name: sqlparser.NewTableIdent(table),
// 因为不建议跨DB的查询，所以这里的db前缀将不进行补齐
Qualifier: sqlparser.TableIdent{},
},
},
As: sqlparser.ColIdent{},
}
if e.TableName.Name.IsEmpty() {
// 情况1，2
newSelectExprs = append(newSelectExprs, newExpr)
} else {
// 其他情况下只有在匹配表名的时候才会进行替换
if e.TableName.Name.String() == col.Table {
newSelectExprs = append(newSelectExprs, newExpr)
}
}
}
}
}
default:
newSelectExprs = append(newSelectExprs, e)
}
}
n.SelectExprs = newSelectExprs
}
return true, nil
}, rw.Stmt)
common.LogIfError(err, "")
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// RewriteInsertColumns insertcolumns: 对应COL.002，INSERT补全列名
func (rw *Rewrite) RewriteInsertColumns() *Rewrite {
switch insert := rw.Stmt.(type) {
case *sqlparser.Insert:
switch insert.Action {
case "insert", "replace":
if insert.Columns != nil {
return rw
}
newColumns := make(sqlparser.Columns, 0)
db := insert.Table.Qualifier.String()
table := insert.Table.Name.String()
// 支持INSERT/REPLACE INTO VALUES形式，支持INSERT/REPLACE INTO SELECT
colCount := 0
switch v := insert.Rows.(type) {
case sqlparser.Values:
if len(v) > 0 {
colCount = len(v[0])
}
case *sqlparser.Select:
if l := len(v.SelectExprs); l > 0 {
colCount = l
}
}
// 开始对ast进行替换，补全前N列
counter := 0
for dbName, tb := range rw.Columns {
for tbName, cols := range tb {
for _, col := range cols {
// 只有全部列补全完成的时候才会替换ast
if counter == colCount {
insert.Columns = newColumns
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
if db != "" {
// 指定了DB的时候，只能怼指定DB的列
if db == dbName && table == tbName {
newColumns = append(newColumns, sqlparser.NewColIdent(col.Name))
counter++
}
} else {
// 没有指定DB的时候，将column中的列按顺序往里怼
if table == tbName {
newColumns = append(newColumns, sqlparser.NewColIdent(col.Name))
counter++
}
}
}
}
}
}
}
return rw
}
// RewriteHaving having: 对应CLA.013，使用 WHERE 过滤条件替代 HAVING
func (rw *Rewrite) RewriteHaving() *Rewrite {
err := sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch n := node.(type) {
case *sqlparser.Select:
if n.Having != nil {
if n.Where == nil {
// WHERE 条件为空直接用 HAVING 替代 WHERE 即可
n.Where = n.Having
} else {
// WHERE 条件不为空，需要对已有的条件进行括号保护，然后再 AND+HAVING
n.Where = &sqlparser.Where{
Expr: &sqlparser.AndExpr{
Left: &sqlparser.ParenExpr{
Expr: n.Where.Expr,
},
Right: n.Having.Expr,
},
}
}
// 别忘了重置 HAVING 和 Where.Type
n.Where.Type = "where"
n.Having = nil
}
}
return true, nil
}, rw.Stmt)
common.LogIfError(err, "")
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// RewriteAddOrderByNull orderbynull: 对应 CLA.008，GROUP BY 无排序要求时添加 ORDER BY NULL
func (rw *Rewrite) RewriteAddOrderByNull() *Rewrite {
err := sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch n := node.(type) {
case *sqlparser.Select:
if n.GroupBy != nil && n.OrderBy == nil {
n.OrderBy = sqlparser.OrderBy{
&sqlparser.Order{
Expr: &sqlparser.NullVal{},
Direction: "asc",
},
}
}
}
return true, nil
}, rw.Stmt)
common.LogIfError(err, "")
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// RewriteOr2Union or2union: 将 OR 查询转写为 UNION ALL TODO: 暂无对应 HeuristicRules
// TODO: https://sqlperformance.com/2014/09/sql-plan/rewriting-queries-improve-performance
func (rw *Rewrite) RewriteOr2Union() *Rewrite {
return rw
}
// RewriteUnionAll unionall: 不介意重复数据的情况下使用 union all 替换 union
func (rw *Rewrite) RewriteUnionAll() *Rewrite {
err := sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch n := node.(type) {
case *sqlparser.Union:
n.Type = "union all"
}
return true, nil
}, rw.Stmt)
common.LogIfError(err, "")
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// RewriteOr2In or2in: 同一列的 OR 过滤条件使用 IN() 替代，如果值有相等的会进行合并
func (rw *Rewrite) RewriteOr2In() *Rewrite {
// 通过 AST 生成 node 的双向链表，链表顺序为书写顺序
nodeList := NewNodeList(rw.Stmt)
tNode := nodeList.First()
for {
tNode.or2in()
if tNode.Next == nil {
break
}
tNode = tNode.Next
}
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// or2in 用于将 or 转换成 in
func (node *NodeItem) or2in() {
if node == nil || node.Self == nil {
return
}
switch selfNode := node.Self.(type) {
case *sqlparser.OrExpr:
newExpr := mergeExprs(selfNode.Left, selfNode.Right)
if newExpr != nil {
// or 自身两个节点可以合并的情况下，将父节点中的 expr 替换成新的
switch pre := node.Prev.Self.(type) {
case *sqlparser.OrExpr:
if pre.Left == node.Self {
node.Self = newExpr
pre.Left = newExpr
} else if pre.Right == node.Self {
node.Self = newExpr
pre.Right = newExpr
}
case *sqlparser.AndExpr:
if pre.Left == node.Self {
node.Self = newExpr
pre.Left = newExpr
} else if pre.Right == node.Self {
node.Self = newExpr
pre.Right = newExpr
}
case *sqlparser.Where:
node.Self = newExpr
pre.Expr = newExpr
case *sqlparser.ParenExpr:
// 如果 SQL 书写中带了括号，暂不会进行跨括号的合并
// TODO: 无意义括号打平，加个 rewrite rule
node.Self = newExpr
pre.Expr = newExpr
}
} else {
// or 自身两个节点如不可以合并，则检测是否可以与父节点合并
// 与父节点的合并不能跨越and、括号等，可能会改变语义
// 检查自身左右节点是否能与上层节点中合并，or 只能与 or 合并
switch pre := node.Prev.Self.(type) {
case *sqlparser.OrExpr:
// AST 中如果出现复合条件，则一定在左树，所以只需要判断左边就可以
if pre.Left == selfNode {
switch n := pre.Right.(type) {
case *sqlparser.ComparisonExpr:
newLeftExpr := mergeExprs(selfNode.Left, n)
newRightExpr := mergeExprs(selfNode.Right, n)
// newLeftExpr 与 newRightExpr 一定有一个是 nil，
// 否则说明该 orExpr 下的两个节点可合并，可以通过最后的向前递归合并 pre 节点中的 expr
if newLeftExpr == nil || newRightExpr == nil {
if newLeftExpr != nil {
pre.Right = newLeftExpr
pre.Left = selfNode.Right
err := node.Array.Remove(node)
common.LogIfError(err, "")
}
if newRightExpr != nil {
pre.Right = newRightExpr
pre.Left = selfNode.Left
err := node.Array.Remove(node)
common.LogIfError(err, "")
}
}
}
}
}
}
}
// 逆向合并由更改 AST 后产生的新的可合并节点
node.Prev.or2in()
}
// mergeExprs 将两个属于同一个列的 ComparisonExpr 合并成一个，如果不能合并则返回 nil
func mergeExprs(left, right sqlparser.Expr) *sqlparser.ComparisonExpr {
// 用于对比两个列是否相同
colInLeft := ""
colInRight := ""
lOperator := ""
rOperator := ""
// 用于存放 expr 左右子树中的值
var values []sqlparser.SQLNode
// SQL 中使用到的列
var colName *sqlparser.ColName
// 左子树
switch l := left.(type) {
case *sqlparser.ComparisonExpr:
// 获取列名
colName, colInLeft = getColumnName(l.Left)
// 获取值
if colInLeft != "" {
switch v := l.Right.(type) {
case *sqlparser.SQLVal, sqlparser.ValTuple, *sqlparser.BoolVal, *sqlparser.NullVal:
values = append(values, v)
}
}
// 获取 operator
lOperator = l.Operator
default:
return nil
}
// 右子树
switch r := right.(type) {
case *sqlparser.ComparisonExpr:
// 获取列名
if colName.Name.String() != "" {
common.Log.Warn("colName shouldn't has value, but now it's %s", colName.Name.String())
}
colName, colInRight = getColumnName(r.Left)
// 获取值
if colInRight != "" {
switch v := r.Right.(type) {
case *sqlparser.SQLVal, sqlparser.ValTuple, *sqlparser.BoolVal, *sqlparser.NullVal:
values = append(values, v)
}
}
// 获取 operator
rOperator = r.Operator
default:
return nil
}
// operator 替换，用于在之后判断是否可以合并
switch lOperator {
case "in", "=":
lOperator = "="
default:
return nil
}
switch rOperator {
case "in", "=":
rOperator = "="
default:
return nil
}
// 不匹配则返回
if colInLeft == "" || colInLeft != colInRight ||
lOperator == "" || lOperator != rOperator {
return nil
}
// 合并左右子树的值
newValTuple := make(sqlparser.ValTuple, 0)
for _, v := range values {
switch v := v.(type) {
case *sqlparser.SQLVal:
newValTuple = append(newValTuple, v)
case *sqlparser.BoolVal:
newValTuple = append(newValTuple, v)
case *sqlparser.NullVal:
newValTuple = append(newValTuple, v)
case sqlparser.ValTuple:
newValTuple = append(newValTuple, v...)
}
}
// 去 expr 中除重复的 value,
newValTuple = removeDup(newValTuple...)
newExpr := &sqlparser.ComparisonExpr{
Operator: "in",
Left: colName,
Right: newValTuple,
}
// 如果只有一个值则是一个等式，没有必要转写成 in
if len(newValTuple) == 1 {
newExpr = &sqlparser.ComparisonExpr{
Operator: lOperator,
Left: colName,
Right: newValTuple[0],
}
}
return newExpr
}
// removeDup 清除 sqlparser.ValTuple 中重复的值
func removeDup(vt ...sqlparser.Expr) sqlparser.ValTuple {
uni := make(sqlparser.ValTuple, 0)
m := make(map[string]sqlparser.SQLNode)
for _, value := range vt {
switch v := value.(type) {
case *sqlparser.SQLVal:
// Type:Val, 冒号用于分隔 Type 和 Val，防止两种不同类型拼接后出现同一个值
if _, ok := m[fmt.Sprint(v.Type)+":"+sqlparser.String(v)]; !ok {
uni = append(uni, v)
m[fmt.Sprint(v.Type)+":"+sqlparser.String(v)] = v
}
case *sqlparser.BoolVal:
if _, ok := m[sqlparser.String(v)]; !ok {
uni = append(uni, v)
m[sqlparser.String(v)] = v
}
case *sqlparser.NullVal:
if _, ok := m[sqlparser.String(v)]; !ok {
uni = append(uni, v)
m[sqlparser.String(v)] = v
}
case sqlparser.ValTuple:
for _, val := range removeDup(v...) {
switch v := val.(type) {
case *sqlparser.SQLVal:
if _, ok := m[fmt.Sprint(v.Type)+":"+sqlparser.String(v)]; !ok {
uni = append(uni, v)
m[fmt.Sprint(v.Type)+":"+sqlparser.String(v)] = v
}
case *sqlparser.BoolVal:
if _, ok := m[sqlparser.String(v)]; !ok {
uni = append(uni, v)
m[sqlparser.String(v)] = v
}
case *sqlparser.NullVal:
if _, ok := m[sqlparser.String(v)]; !ok {
uni = append(uni, v)
m[sqlparser.String(v)] = v
}
}
}
}
}
return uni
}
// RewriteInNull innull: TODO: 对应ARG.004
func (rw *Rewrite) RewriteInNull() *Rewrite {
return rw
}
// RewriteRmParenthesis rmparenthesis: 去除无意义的括号
func (rw *Rewrite) RewriteRmParenthesis() *Rewrite {
rw.rmParenthesis()
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// rmParenthesis 用于语出无用的括号
func (rw *Rewrite) rmParenthesis() {
continueFlag := false
err := sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch node := node.(type) {
case *sqlparser.Where:
if node == nil {
return true, nil
}
switch paren := node.Expr.(type) {
case *sqlparser.ParenExpr:
switch paren.Expr.(type) {
case *sqlparser.ComparisonExpr:
node.Expr = paren.Expr
continueFlag = true
}
}
case *sqlparser.ParenExpr:
switch paren := node.Expr.(type) {
case *sqlparser.ParenExpr:
switch paren.Expr.(type) {
case *sqlparser.ComparisonExpr:
node.Expr = paren.Expr
continueFlag = true
}
}
case *sqlparser.AndExpr:
switch left := node.Left.(type) {
case *sqlparser.ParenExpr:
switch inner := left.Expr.(type) {
case *sqlparser.ComparisonExpr:
node.Left = inner
continueFlag = true
}
}
switch right := node.Right.(type) {
case *sqlparser.ParenExpr:
switch inner := right.Expr.(type) {
case *sqlparser.ComparisonExpr:
node.Right = inner
continueFlag = true
}
}
case *sqlparser.OrExpr:
switch left := node.Left.(type) {
case *sqlparser.ParenExpr:
switch inner := left.Expr.(type) {
case *sqlparser.ComparisonExpr:
node.Left = inner
continueFlag = true
}
}
switch right := node.Right.(type) {
case *sqlparser.ParenExpr:
switch inner := right.Expr.(type) {
case *sqlparser.ComparisonExpr:
node.Right = inner
continueFlag = true
}
}
}
return true, nil
}, rw.Stmt)
common.LogIfError(err, "")
// 本层的修改可能使得原本不符合条件的括号变为无意义括号
// 每次修改都需要再过滤一遍语法树
if continueFlag {
rw.rmParenthesis()
} else {
return
}
}
// RewriteRemoveDMLOrderBy dmlorderby: 对应 RES.004，删除无 LIMIT 条件时 UPDATE, DELETE 中包含的 ORDER BY
func (rw *Rewrite) RewriteRemoveDMLOrderBy() *Rewrite {
switch st := rw.Stmt.(type) {
case *sqlparser.Update:
err := sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch n := node.(type) {
case *sqlparser.Select:
if n.OrderBy != nil && n.Limit == nil {
n.OrderBy = nil
}
return false, nil
}
return true, nil
}, rw.Stmt)
common.LogIfError(err, "")
if st.OrderBy != nil && st.Limit == nil {
st.OrderBy = nil
}
case *sqlparser.Delete:
err := sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch n := node.(type) {
case *sqlparser.Select:
if n.OrderBy != nil && n.Limit == nil {
n.OrderBy = nil
}
return false, nil
}
return true, nil
}, rw.Stmt)
common.LogIfError(err, "")
if st.OrderBy != nil && st.Limit == nil {
st.OrderBy = nil
}
}
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// RewriteGroupByConst 对应CLA.004，将GROUP BY CONST替换为列名
// TODO:
func (rw *Rewrite) RewriteGroupByConst() *Rewrite {
err := sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch n := node.(type) {
case *sqlparser.Select:
groupByCol := false
if n.GroupBy != nil {
for _, group := range n.GroupBy {
switch group.(type) {
case *sqlparser.SQLVal:
default:
groupByCol = true
}
}
if !groupByCol {
// TODO: 这里只是去掉了GROUP BY并没解决问题
n.GroupBy = nil
}
}
}
return true, nil
}, rw.Stmt)
common.LogIfError(err, "")
rw.NewSQL = sqlparser.String(rw.Stmt)
return rw
}
// RewriteSubQuery2Join 将 subquery 转写成 join
func (rw *Rewrite) RewriteSubQuery2Join() *Rewrite {
var err error
// 如果未配置 mysql 环境或从环境中获取失败
if common.Config.TestDSN.Disable || len(rw.Columns) == 0 {
common.Log.Debug("(rw *Rewrite) RewriteSubQuery2Join(): Rewrite failed. TestDSN.Disable: %v, len(rw.Columns):%d",
common.Config.TestDSN.Disable, len(rw.Columns))
return rw
}
if rw.NewSQL == "" {
rw.NewSQL = sqlparser.String(rw.Stmt)
}
// query backup
backup := rw.NewSQL
var subQueryList []string
err = sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch sub := node.(type) {
case sqlparser.SelectStatement:
subStr := sqlparser.String(sub)
if strings.HasPrefix(subStr, "(") {
subStr = subStr[1 : len(subStr)-1]
}
subQueryList = append(subQueryList, subStr)
}
return true, nil
}, rw.Stmt)
common.LogIfError(err, "")
if length := len(subQueryList); length > 1 {
lastResult := ""
for i := length - 1; i > 0; i-- {
if lastResult == "" {
lastResult, err = rw.sub2Join(subQueryList[i-1], subQueryList[i])
} else {
// 将subquery的部分替换成上次合并的结果
subQueryList[i-1] = strings.Replace(subQueryList[i-1], subQueryList[i], lastResult, -1)
lastResult, err = rw.sub2Join(subQueryList[i-1], lastResult)
}
if err != nil {
common.Log.Error("RewriteSubQuery2Join Error: %v", err)
return rw
}
}
rw.NewSQL = lastResult
} else if length == 1 {
var newSQL string
newSQL, err = rw.sub2Join(rw.NewSQL, subQueryList[0])
if err == nil {
rw.NewSQL = newSQL
}
}
// 因为这个修改不会直接修改rw.stmt，所以需要将rw.stmt也更新一下
newStmt, err := sqlparser.Parse(rw.NewSQL)
if err != nil {
rw.NewSQL = backup
rw.Stmt, _ = sqlparser.Parse(backup)
} else {
rw.Stmt = newStmt
}
return rw
}
// sub2Join 将 subquery 转写成 join
func (rw *Rewrite) sub2Join(parent, sub string) (string, error) {
// 只处理SelectStatement
if sqlparser.Preview(parent) != sqlparser.StmtSelect || sqlparser.Preview(sub) != sqlparser.StmtSelect {
return "", nil
}
// 如果子查询不属于parent,则不处理
if !strings.Contains(parent, sub) {
return "", nil
}
// 解析外层SQL语法树
stmt, err := sqlparser.Parse(parent)
if err != nil {
common.Log.Warn("(rw *Rewrite) RewriteSubQuery2Join() sub2Join sql `%s` parsed error: %v", parent, err)
return "", err
}
switch stmt.(type) {
case sqlparser.SelectStatement:
default:
common.Log.Debug("Query `%s` not select statement.", parent)
return "", nil
}
// 解析子查询语法树
subStmt, err := sqlparser.Parse(sub)
if err != nil {
common.Log.Warn("(rw *Rewrite) RewriteSubQuery2Join() sub2Join sql `%s` parsed error: %v", sub, err)
return "", err
}
// 获取外部SQL用到的表
stmtMeta := GetTableFromExprs(stmt.(*sqlparser.Select).From)
// 获取内部SQL用到的表
subMeta := GetTableFromExprs(subStmt.(*sqlparser.Select).From)
// 处理关联条件
err = sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch p := node.(type) {
case *sqlparser.ComparisonExpr:
// a in (select * from tb)
switch subquery := p.Right.(type) {
case *sqlparser.Subquery:
// 获取左边的列
var leftColumn *sqlparser.ColName
switch l := p.Left.(type) {
case *sqlparser.ColName:
leftColumn = l
default:
return false, nil
}
// 用于存放获取的subquery中的列，有且只有一个
var rightColumn sqlparser.SQLNode
// 对subquery中的列进行替换
switch subSelectStmt := subquery.Select.(type) {
case *sqlparser.Select:
cachingOperator := p.Operator
rightColumn = subSelectStmt.SelectExprs[0]
rightCol, _ := getColumnName(rightColumn.(*sqlparser.AliasedExpr).Expr)
if rightCol != nil {
// 将subquery替换为等值条件
p.Operator = "="
// selectExpr 信息补齐
var newExprs []sqlparser.SelectExpr
err = sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch col := node.(type) {
case *sqlparser.StarExpr:
if col.TableName.Name.IsEmpty() {
for dbName, db := range stmtMeta {
for tbName := range db.Table {
col.TableName.Name = sqlparser.NewTableIdent(tbName)
if dbName != "" {
col.TableName.Qualifier = sqlparser.NewTableIdent(dbName)
}
newExprs = append(newExprs, col)
}
}
}
case *sqlparser.AliasedExpr:
switch n := col.Expr.(type) {
case *sqlparser.ColName:
col.Expr = columnFromWhere(n, stmtMeta, rw.Columns)
}
}
return true, nil
}, stmt.(*sqlparser.Select).SelectExprs)
common.LogIfError(err, "")
// 原节点列信息补齐
p.Left = columnFromWhere(leftColumn, stmtMeta, rw.Columns)
// 将子查询中的节点上提，补充前缀信息
p.Right = columnFromWhere(rightCol, subMeta, rw.Columns)
// subquery Where条件中的列信息补齐
subWhereExpr := subStmt.(*sqlparser.Select).Where
err = sqlparser.Walk(func(node sqlparser.SQLNode) (kontinue bool, err error) {
switch n := node.(type) {
case *sqlparser.ComparisonExpr:
switch left := n.Left.(type) {
case *sqlparser.ColName:
n.Left = columnFromWhere(left, subMeta, rw.Columns)
}
switch right := n.Right.(type) {
case *sqlparser.ColName:
n.Right = columnFromWhere(right, subMeta, rw.Columns)
}
}
return true, nil
}, subWhereExpr)
common.LogIfError(err, "")
// 如果 subquery 中存在 Where 条件，怼在 parent 的 where 中后面
if subWhereExpr != nil {
if stmt.(*sqlparser.Select).Where != nil {
stmt.(*sqlparser.Select).Where.Expr = &sqlparser.AndExpr{
Left: stmt.(*sqlparser.Select).Where.Expr,
Right: subWhereExpr.Expr,
}
} else {
stmt.(*sqlparser.Select).Where = subWhereExpr
}
}
switch cachingOperator {
case "in":
// 将表以 inner join 的形式追加到 parent 的 from 中
var newTables []sqlparser.TableExpr
for _, subExpr := range subStmt.(*sqlparser.Select).From {
has := false
for _, expr := range stmt.(*sqlparser.Select).From {
if reflect.DeepEqual(expr, subExpr) {
has = true
}
}
if !has {
newTables = append(newTables, subExpr)
}
}
stmt.(*sqlparser.Select).From = append(stmt.(*sqlparser.Select).From, newTables...)
case "not in":
// 将表以left join 的形式 追加到 parent 的 from 中
// TODO:
}
}
}
}
}
return true, nil
}, stmt)
common.LogIfError(err, "")
newSQL := sqlparser.String(stmt)
return newSQL, nil
}
// columnFromWhere 获取列是来自哪个表，并补充前缀
func columnFromWhere(col *sqlparser.ColName, meta common.Meta, columns common.TableColumns) *sqlparser.ColName {
for dbName, db := range meta {
for tbName := range db.Table {
for _, tables := range columns {
for _, columns := range tables {
for _, column := range columns {
if strings.EqualFold(col.Name.String(), column.Name) {
if col.Qualifier.Name.IsEmpty() && tbName == column.Table {
col.Qualifier.Name = sqlparser.NewTableIdent(column.Table)
return col
}
if (dbName == "" && tbName == column.Table) || (tbName == column.Table && dbName == column.DB) {
col.Qualifier.Name = sqlparser.NewTableIdent(column.Table)
if dbName != "" {
col.Qualifier.Qualifier = sqlparser.NewTableIdent(column.DB)
}
return col
}
}
}
}
}
}
}
return col
}
// RewriteJoin2SubQuery join2sub: TODO:
// https://mariadb.com/kb/en/library/subqueries-and-joins/
func (rw *Rewrite) RewriteJoin2SubQuery() *Rewrite {
return rw
}
// RewriteDistinctStar distinctstar: 对应DIS.003，将多余的`DISTINCT *`删除
func (rw *Rewrite) RewriteDistinctStar() *Rewrite {
// 注意：这里并未对表是否有主键做检查，按照我们的SQL编程规范，一张表必须有主键
switch rw.Stmt.(type) {
case *sqlparser.Select:
meta := GetMeta(rw.Stmt, nil)
for _, m := range meta {
if len(m.Table) == 1 {
// distinct tbl.*, distinct *, count(distinct *)
re := regexp.MustCompile(`(?i)((distinct\s*\*)|(distinct\s+[0-9a-z_` + "`" + `]*\.\*))`)
if re.MatchString(rw.SQL) {
rw.NewSQL = re.ReplaceAllString(rw.SQL, "*")
}
}
break
}
}
if rw.NewSQL == "" {
rw.NewSQL = rw.SQL
}
rw.Stmt, _ = sqlparser.Parse(rw.NewSQL)
return rw
}
// RewriteTruncate truncate: DELETE 全表修改为 TRUNCATE TABLE
func (rw *Rewrite) RewriteTruncate() *Rewrite {
switch n := rw.Stmt.(type) {
case *sqlparser.Delete:
meta := GetMeta(rw.Stmt, nil)
if len(meta) == 1 && n.Where == nil {
for _, db := range meta {
for _, tbl := range db.Table {
rw.NewSQL = "truncate table " + tbl.TableName
}
}
}
}
return rw
}
// RewriteDML2Select dml2select: DML 转成 SELECT，兼容低版本的 EXPLAIN
func (rw *Rewrite) RewriteDML2Select() *Rewrite {
if rw.Stmt == nil {
return rw
}
switch stmt := rw.Stmt.(type) {
case *sqlparser.Select:
rw.NewSQL = rw.SQL
case *sqlparser.Delete: // Multi DELETE not support yet.
rw.NewSQL = delete2Select(stmt)
case *sqlparser.Insert:
rw.NewSQL = insert2Select(stmt)
case *sqlparser.Update: // Multi UPDATE not support yet.
rw.NewSQL = update2Select(stmt)
}
rw.Stmt, _ = sqlparser.Parse(rw.NewSQL)
return rw
}
func (rw *Rewrite) RewriteReg2Select() *Rewrite {
var pre = 9
if len(rw.SQL) < pre {
// SQL to short no need convert
return rw
}
if strings.HasPrefix(strings.ToLower(rw.SQL[:pre]), "select") {
rw.NewSQL = rw.SQL
}
if strings.HasPrefix(strings.ToLower(rw.SQL[:pre]), "update") {
rw.NewSQL = regUpdate2Select(rw.SQL)
}
if strings.HasPrefix(strings.ToLower(rw.SQL[:pre]), "delete") {
rw.NewSQL = regDelete2Select(rw.SQL)
}
return rw
}
// delete2Select 将 Delete 语句改写成 Select
func delete2Select(stmt *sqlparser.Delete) string {
newSQL := &sqlparser.Select{
SelectExprs: []sqlparser.SelectExpr{
new(sqlparser.StarExpr),
},
From: stmt.TableExprs,
Where: stmt.Where,
OrderBy: stmt.OrderBy,
}
return sqlparser.String(newSQL)
}
// regDelete2Select convert delete to select by regexp
func regDelete2Select(sql string) string {
sql = strings.TrimSpace(sql)
sqlRegexp := regexp.MustCompile(`^(?i)delete\s+from\s+(.*)$`)
params := sqlRegexp.FindStringSubmatch(sql)
if len(params) > 1 {
return fmt.Sprintf(`select * from %s`, params[1])
}
return sql
}
// update2Select 将 Update 语句改写成 Select
func update2Select(stmt *sqlparser.Update) string {
newSQL := &sqlparser.Select{
SelectExprs: []sqlparser.SelectExpr{
new(sqlparser.StarExpr),
},
From: stmt.TableExprs,
Where: stmt.Where,
OrderBy: stmt.OrderBy,
Limit: stmt.Limit,
}
return sqlparser.String(newSQL)
}
// regUpdate2Select convert update to select by regexp
func regUpdate2Select(sql string) string {
sql = strings.TrimSpace(sql)
sqlRegexp := regexp.MustCompile(`^(?i)update\s+(.*)\s+set\s+(.*)\s+(where\s+.*)$`)
params := sqlRegexp.FindStringSubmatch(sql)
if len(params) > 2 {
return fmt.Sprintf(`select * from %s %s`, params[1], params[3])
}
return sql
}
// insert2Select 将 Insert 语句改写成 Select
func insert2Select(stmt *sqlparser.Insert) string {
switch row := stmt.Rows.(type) {
// 如果insert包含子查询，只需要explain该子树
case *sqlparser.Select, *sqlparser.Union, *sqlparser.ParenSelect:
return sqlparser.String(row)
}
return "select 1 from DUAL"
}
// AlterAffectTable 获取ALTER影响的库表名，返回：`db`.`table`
func AlterAffectTable(stmt sqlparser.Statement) string {
switch n := stmt.(type) {
case *sqlparser.DDL:
tableName := strings.ToLower(n.Table.Name.String())
dbName := strings.ToLower(n.Table.Qualifier.String())
if tableName != "" && tableName != "dual" {
if dbName == "" {
return "`" + tableName + "`"
}
return "`" + dbName + "`.`" + tableName + "`"
}
}
return ""
}
// MergeAlterTables mergealter: 将同一张表的多条 ALTER 语句合成一条 ALTER 语句
// @input: sql, alter string
// @output: [[db.]table]sql, 如果找不到 DB，key 为表名；如果找得到 DB，key 为 db.table
func MergeAlterTables(sqls ...string) map[string]string {
alterSQLs := make(map[string][]string)
mergedAlterStr := make(map[string]string)
// table/column/index name can be quoted in back ticks
backTicks := "(`[^\\s]*`)"
alterExp := regexp.MustCompile(`(?i)alter\s*table\s*(` + backTicks + `|([^\s]*))\s*`) // ALTER TABLE
renameExp := regexp.MustCompile(`(?i)rename\s*table\s*(` + backTicks + `|([^\s]*))\s*`) // RENAME TABLE
// CREATE [UNIQUE|FULLTEXT|SPATIAL|PRIMARY] [KEY|INDEX] idx_name ON tbl_name
createIndexExp := regexp.MustCompile(`(?i)create((unique)|(fulltext)|(spatial)|(primary)|(\s*)\s*)((index)|(key))\s*`)
indexNameExp := regexp.MustCompile(`(?i)(` + backTicks + `|([^\s]*))\s*`)
for _, sql := range sqls {
sql = strings.Trim(sql, common.Config.Delimiter)
stmts, err := TiParse(sql, "", "")
if err != nil {
common.Log.Warn(err.Error())
continue
}
// stmt, _ := sqlparser.Parse(sql)
alterSQL := ""
dbName := ""
tableName := ""
for _, stmt := range stmts {
switch n := stmt.(type) {
case *ast.AlterTableStmt:
// 注意: 表名和库名不区分大小写
tableName = n.Table.Name.L
dbName = n.Table.Schema.L
if alterExp.MatchString(sql) {
common.Log.Debug("rename alterExp: ALTER %v %v", tableName, alterExp.ReplaceAllString(sql, ""))
alterSQL = fmt.Sprint(alterExp.ReplaceAllString(sql, ""))
}
case *ast.CreateIndexStmt:
tableName = n.Table.Name.L
dbName = n.Table.Schema.L
buf := createIndexExp.ReplaceAllString(sql, "")
idxName := strings.TrimSpace(indexNameExp.FindString(buf))
buf = string([]byte(buf)[strings.Index(buf, "("):])
common.Log.Error(buf)
common.Log.Debug("alter createIndexExp: ALTER %v ADD INDEX %v %v", tableName, "ADD INDEX", idxName, buf)
alterSQL = fmt.Sprint("ADD INDEX", " "+idxName+" ", buf)
case *ast.RenameTableStmt:
// 注意: 表名和库名不区分大小写
tableName = n.TableToTables[0].OldTable.Name.L
dbName = n.TableToTables[0].OldTable.Schema.L
if alterExp.MatchString(sql) {
common.Log.Debug("rename alterExp: ALTER %v %v", tableName, alterExp.ReplaceAllString(sql, ""))
alterSQL = fmt.Sprint(alterExp.ReplaceAllString(sql, ""))
} else if renameExp.MatchString(sql) {
common.Log.Debug("rename renameExp: ALTER %v %v", tableName, alterExp.ReplaceAllString(sql, ""))
alterSQL = fmt.Sprint(alterExp.ReplaceAllString(sql, ""))
} else {
common.Log.Warn("rename not match: ALTER %v %v", tableName, sql)
}
default:
}
}
if alterSQL != "" && tableName != "" && tableName != "dual" {
if dbName == "" {
alterSQLs["`"+tableName+"`"] = append(alterSQLs["`"+tableName+"`"], alterSQL)
} else {
alterSQLs["`"+dbName+"`.`"+tableName+"`"] = append(alterSQLs["`"+dbName+"`.`"+tableName+"`"], alterSQL)
}
}
}
for k, v := range alterSQLs {
mergedAlterStr[k] = fmt.Sprintln("ALTER TABLE", k, strings.Join(v, ", "), common.Config.Delimiter)
}
return mergedAlterStr
}
// RewriteRuleMatch 检查重写规则是否生效
func RewriteRuleMatch(name string) bool {
for _, r := range common.Config.RewriteRules {
if r == name {
return true
}
}
return false
}
================================================
FILE: ast/rewrite_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ast
import (
"fmt"
"sort"
"testing"
"github.com/XiaoMi/soar/common"
)
func TestRewrite(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
orgTestDSNStatus := common.Config.TestDSN.Disable
common.Config.TestDSN.Disable = false
testSQL := []map[string]string{
{
"input": `SELECT * FROM film`,
"output": `select film_id, title, description, release_year, language_id, original_language_id, rental_duration from film;`,
},
{
"input": `SELECT film.*, actor.actor_id FROM film,actor`,
"output": `select film_id, title, description, release_year, language_id, original_language_id, rental_duration, actor.actor_id from film, actor;`,
},
{
"input": `insert into film values(1,2,3,4,5)`,
"output": `insert into film(film_id, title, description, release_year, language_id) values (1, 2, 3, 4, 5);`,
},
{
"input": `insert into sakila.film values(1,2)`,
"output": `insert into sakila.film(film_id, title) values (1, 2);`,
},
{
"input": `replace into sakila.film select id from tb`,
"output": `replace into sakila.film(film_id) select id from tb;`,
},
{
"input": `replace into sakila.film select id, title, description from tb`,
"output": `replace into sakila.film(film_id, title, description) select id, title, description from tb;`,
},
{
"input": `insert into film values(1,2,3,4,5)`,
"output": `insert into film(film_id, title, description, release_year, language_id) values (1, 2, 3, 4, 5);`,
},
{
"input": `insert into sakila.film values(1,2)`,
"output": `insert into sakila.film(film_id, title) values (1, 2);`,
},
{
"input": `replace into sakila.film select id from tb`,
"output": `replace into sakila.film(film_id) select id from tb;`,
},
{
"input": `replace into sakila.film select id, title, description from tb`,
"output": `replace into sakila.film(film_id, title, description) select id, title, description from tb;`,
},
{
"input": "DELETE FROM tbl WHERE col1=1 ORDER BY col",
"output": "delete from tbl where col1 = 1;",
},
{
"input": "UPDATE tbl SET col =1 WHERE col1=1 ORDER BY col",
"output": "update tbl set col = 1 where col1 = 1;",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"])
rw.Columns = map[string]map[string][]*common.Column{
"sakila": {
"film": {
{Name: "film_id", Table: "film"},
{Name: "title", Table: "film"},
{Name: "description", Table: "film"},
{Name: "release_year", Table: "film"},
{Name: "language_id", Table: "film"},
{Name: "original_language_id", Table: "film"},
{Name: "rental_duration", Table: "film"},
},
},
}
rw.Rewrite()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Config.TestDSN.Disable = orgTestDSNStatus
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteStar2Columns(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
orgTestDSNStatus := common.Config.TestDSN.Disable
common.Config.TestDSN.Disable = false
testSQL := []map[string]string{
{
"input": `SELECT * FROM film`,
"output": `select film_id, title from film`,
},
{
"input": `SELECT film.* FROM film`,
"output": `select film_id, title from film`,
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"])
rw.Columns = map[string]map[string][]*common.Column{
"sakila": {
"film": {
{Name: "film_id", Table: "film"},
{Name: "title", Table: "film"},
},
},
}
rw.RewriteStar2Columns()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
testSQL2 := []map[string]string{
{
"input": `SELECT film.* FROM film, actor`,
"output": `select film.film_id, film.title from film, actor`,
},
{
"input": `SELECT film.*, actor.actor_id FROM film, actor`,
"output": `select film.film_id, film.title, actor.actor_id from film, actor`,
},
}
for _, sql := range testSQL2 {
rw := NewRewrite(sql["input"])
rw.Columns = map[string]map[string][]*common.Column{
"sakila": {
"film": {
{Name: "film_id", Table: "film"},
{Name: "title", Table: "film"},
},
"actor": {
{Name: "actor_id", Table: "actor"},
},
},
}
rw.RewriteStar2Columns()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Config.TestDSN.Disable = orgTestDSNStatus
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteInsertColumns(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": `insert into film values(1,2,3,4,5)`,
"output": `insert into film(film_id, title, description, release_year, language_id) values (1, 2, 3, 4, 5)`,
},
{
"input": `insert into sakila.film values(1,2)`,
"output": `insert into sakila.film(film_id, title) values (1, 2)`,
},
{
"input": `replace into sakila.film select id from tb`,
"output": `replace into sakila.film(film_id) select id from tb`,
},
{
"input": `replace into sakila.film select id, title, description from tb`,
"output": `replace into sakila.film(film_id, title, description) select id, title, description from tb`,
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"])
rw.Columns = map[string]map[string][]*common.Column{
"sakila": {
"film": {
{Name: "film_id", Table: "film"},
{Name: "title", Table: "film"},
{Name: "description", Table: "film"},
{Name: "release_year", Table: "film"},
{Name: "language_id", Table: "film"},
{Name: "original_language_id", Table: "film"},
{Name: "rental_duration", Table: "film"},
},
},
}
rw.RewriteInsertColumns()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteHaving(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": `SELECT state, COUNT(*) FROM Drivers GROUP BY state HAVING state IN ('GA', 'TX') ORDER BY state`,
"output": "select state, COUNT(*) from Drivers where state in ('GA', 'TX') group by state order by state asc",
},
{
"input": `SELECT state, COUNT(*) FROM Drivers WHERE col =1 GROUP BY state HAVING state IN ('GA', 'TX') ORDER BY state`,
"output": "select state, COUNT(*) from Drivers where (col = 1) and state in ('GA', 'TX') group by state order by state asc",
},
{
"input": `SELECT state, COUNT(*) FROM Drivers WHERE col =1 or col1 =2 GROUP BY state HAVING state IN ('GA', 'TX') ORDER BY state`,
"output": "select state, COUNT(*) from Drivers where (col = 1 or col1 = 2) and state in ('GA', 'TX') group by state order by state asc",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteHaving()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteAddOrderByNull(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": "SELECT sum(col1) FROM tbl GROUP BY col",
"output": "select sum(col1) from tbl group by col order by null",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteAddOrderByNull()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteRemoveDMLOrderBy(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": "DELETE FROM tbl WHERE col1=1 ORDER BY col",
"output": "delete from tbl where col1 = 1",
},
{
"input": "UPDATE tbl SET col =1 WHERE col1=1 ORDER BY col",
"output": "update tbl set col = 1 where col1 = 1",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteRemoveDMLOrderBy()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteGroupByConst(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": "select 1;",
"output": "select 1 from dual",
},
/*
{
"input": "SELECT col1 FROM tbl GROUP BY 1;",
"output": "select col1 from tbl GROUP BY col1",
},
{
"input": "SELECT col1, col2 FROM tbl GROUP BY 1, 2;",
"output": "select col1, col2 from tbl GROUP BY col1, col2",
},
{
"input": "SELECT col1, col2, col3 FROM tbl GROUP BY 1, 3;",
"output": "select col1, col2, col3 from tbl GROUP BY col1, col3",
},
*/
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteGroupByConst()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteStandard(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": "SELECT sum(col1) FROM tbl GROUP BY 1;",
"output": "select sum(col1) from tbl group by 1",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteStandard()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteCountStar(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": "SELECT count(col) FROM tbl GROUP BY 1;",
"output": "select count(*) from tbl group by 1",
},
{
"input": "SELECT COUNT(tb.col) FROM tbl GROUP BY 1;",
"output": "select COUNT(tb.*) from tbl group by 1",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteCountStar()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteInnoDB(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": "CREATE TABLE t1(id bigint(20) NOT NULL AUTO_INCREMENT);",
"output": "create table t1 (\n\tid bigint(20) not null auto_increment\n) ENGINE=InnoDB ",
},
{
"input": "create table t1 (\n\tid bigint(20) not null auto_increment\n) ENGINE=memory ",
"output": "create table t1 (\n\tid bigint(20) not null auto_increment\n) ENGINE=InnoDB ",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteInnoDB()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteAutoIncrement(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": "CREATE TABLE t1(id bigint(20) NOT NULL AUTO_INCREMENT) ENGINE=InnoDB AUTO_INCREMENT=123802;",
"output": "create table t1 (\n\tid bigint(20) not null auto_increment\n) ENGINE=InnoDB auto_increment=1 ",
},
{
"input": "create table t1 (\n\tid bigint(20) not null auto_increment\n) ENGINE=InnoDB",
"output": "create table t1 (\n\tid bigint(20) not null auto_increment\n) ENGINE=InnoDB",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteAutoIncrement()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteIntWidth(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": "CREATE TABLE t1(id bigint(10) NOT NULL AUTO_INCREMENT) ENGINE=InnoDB AUTO_INCREMENT=123802;",
"output": "create table t1 (\n\tid bigint(20) not null auto_increment\n) ENGINE=InnoDB AUTO_INCREMENT=123802",
},
{
"input": "CREATE TABLE t1(id bigint NOT NULL AUTO_INCREMENT) ENGINE=InnoDB AUTO_INCREMENT=123802;",
"output": "create table t1 (\n\tid bigint(20) not null auto_increment\n) ENGINE=InnoDB AUTO_INCREMENT=123802",
},
{
"input": "create table t1(id int(20) not null auto_increment) ENGINE=InnoDB;",
"output": "create table t1 (\n\tid int(10) not null auto_increment\n) ENGINE=InnoDB",
},
{
"input": "create table t1(id int not null auto_increment) ENGINE=InnoDB;",
"output": "create table t1 (\n\tid int not null auto_increment\n) ENGINE=InnoDB",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteIntWidth()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteAlwaysTrue(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": "SELECT count(col) FROM tbl where 1=1;",
"output": "select count(col) from tbl",
},
{
"input": "SELECT count(col) FROM tbl where col=col;",
"output": "select count(col) from tbl where col = col",
},
{
"input": "SELECT count(col) FROM tbl where col=col2;",
"output": "select count(col) from tbl where col = col2",
},
{
"input": "SELECT count(col) FROM tbl where 1>=1;",
"output": "select count(col) from tbl",
},
{
"input": "SELECT count(col) FROM tbl where 2>1;",
"output": "select count(col) from tbl",
},
{
"input": "SELECT count(col) FROM tbl where 1<=1;",
"output": "select count(col) from tbl",
},
{
"input": "SELECT count(col) FROM tbl where 1<2;",
"output": "select count(col) from tbl",
},
{
"input": "SELECT count(col) FROM tbl where 1=1 and 2=2;",
"output": "select count(col) from tbl",
},
{
"input": "SELECT count(col) FROM tbl where 1=1 or 2=3;",
"output": "select count(col) from tbl where 2 = 3",
},
{
"input": "SELECT count(col) FROM tbl where 1=1 and 3=3 or 2=3;",
"output": "select count(col) from tbl where 2 = 3",
},
{
"input": "SELECT count(col) FROM tbl where 1=1 and 3=3 or 2!=3;",
"output": "select count(col) from tbl",
},
{
"input": "SELECT count(col) FROM tbl where 1=1 or 2=3 and 3=3 ;",
"output": "select count(col) from tbl where 2 = 3",
},
{
"input": "SELECT count(col) FROM tbl where (1=1);",
"output": "select count(col) from tbl",
},
{
"input": "SELECT count(col) FROM tbl where a=1;",
"output": "select count(col) from tbl where a = 1",
},
{
"input": "SELECT count(col) FROM tbl where ('a'= 'a' or 'b' = 'b') and a = 'b';",
"output": "select count(col) from tbl where a = 'b'",
},
{
"input": "SELECT count(col) FROM tbl where (('a'= 'a' or 'b' = 'b') and a = 'b');",
"output": "select count(col) from tbl where (a = 'b')",
},
{
"input": "SELECT count(col) FROM tbl where 'a'= 'a' or ('b' = 'b' and a = 'b');",
"output": "select count(col) from tbl where (a = 'b')",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteAlwaysTrue()
if rw == nil {
t.Errorf("NoRw")
} else if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
// TODO:
func TestRewriteSubQuery2Join(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
orgTestDSNStatus := common.Config.TestDSN.Disable
common.Config.TestDSN.Disable = true
testSQL := []map[string]string{
{
// 这个case是官方文档给的，但不一定正确，需要视表结构的定义来进行判断
"input": `SELECT * FROM t1 WHERE id IN (SELECT id FROM t2);`,
"output": "",
//"output": `SELECT DISTINCT t1.* FROM t1, t2 WHERE t1.id=t2.id;`,
},
{
"input": `SELECT * FROM t1 WHERE id NOT IN (SELECT id FROM t2);`,
"output": "",
//"output": `SELECT table1.* FROM t1 LEFT JOIN t2 ON t1.id=t2.id WHERE t2.id IS NULL;`,
},
{
"input": `SELECT * FROM t1 WHERE NOT EXISTS (SELECT id FROM t2 WHERE t1.id=t2.id);`,
"output": "",
//"output": `SELECT table1.* FROM table1 LEFT JOIN table2 ON table1.id=table2.id WHERE table2.id IS NULL;`,
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteSubQuery2Join()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Config.TestDSN.Disable = orgTestDSNStatus
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteDML2Select(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": "DELETE city, country FROM city INNER JOIN country using (country_id) WHERE city.city_id = 1;",
"output": "select * from city join country using (country_id) where city.city_id = 1",
}, {
"input": "DELETE city FROM city LEFT JOIN country ON city.country_id = country.country_id WHERE country.country IS NULL;",
"output": "select * from city left join country on city.country_id = country.country_id where country.country is null",
}, {
"input": "DELETE a1, a2 FROM city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id",
"output": "select * from city as a1 join country as a2 where a1.country_id = a2.country_id",
}, {
"input": "DELETE FROM a1, a2 USING city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id",
"output": "select * from city as a1 join country as a2 where a1.country_id = a2.country_id",
}, {
"input": "DELETE FROM film WHERE length > 100;",
"output": "select * from film where length > 100",
}, {
"input": "UPDATE city INNER JOIN country USING(country_id) SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;",
"output": "select * from city join country using (country_id) where city.city_id = 10",
}, {
"input": "UPDATE city INNER JOIN country ON city.country_id = country.country_id INNER JOIN address ON city.city_id = address.city_id SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;",
"output": "select * from city join country on city.country_id = country.country_id join address on city.city_id = address.city_id where city.city_id = 10",
}, {
"input": "UPDATE city, country SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.country_id = country.country_id AND city.city_id=10;",
"output": "select * from city, country where city.country_id = country.country_id and city.city_id = 10",
}, {
"input": "UPDATE film SET length = 10 WHERE language_id = 20;",
"output": "select * from film where language_id = 20",
}, {
"input": "INSERT INTO city (country_id) SELECT country_id FROM country;",
"output": "select country_id from country",
}, {
"input": "INSERT INTO city (country_id) VALUES (1),(2),(3);",
"output": "select 1 from DUAL",
}, {
"input": "INSERT INTO city (country_id) VALUES (10);",
"output": "select 1 from DUAL",
}, {
"input": "INSERT INTO city (country_id) SELECT 10 FROM DUAL;",
"output": "select 10 from dual",
}, {
"input": "replace INTO city (country_id) SELECT 10 FROM DUAL;",
"output": "select 10 from dual",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteDML2Select()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteReg2Select(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": "select 1 from dual",
"output": "select 1 from dual",
},
{
"input": "delete from dual",
"output": "select * from dual",
},
{
"input": "update tb set col = 1 where col = 2",
"output": "select * from tb where col = 2",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteReg2Select()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteDistinctStar(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": `SELECT DISTINCT * FROM film;`,
"output": "SELECT * FROM film;",
},
{
"input": `SELECT COUNT(DISTINCT *) FROM film;`,
"output": "SELECT COUNT(*) FROM film;",
},
{
"input": `SELECT DISTINCT film.* FROM film;`,
"output": "SELECT * FROM film;",
},
{
"input": "SELECT DISTINCT col FROM film;",
"output": "SELECT DISTINCT col FROM film;",
},
{
"input": "SELECT DISTINCT film.* FROM film, tbl;",
"output": "SELECT DISTINCT film.* FROM film, tbl;",
},
{
"input": "SELECT DISTINCT * FROM film, tbl;",
"output": "SELECT DISTINCT * FROM film, tbl;",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteDistinctStar()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestMergeAlterTables(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
sqls := []string{
// ADD|DROP INDEX
// TODO: PRIMARY KEY, [UNIQUE|FULLTEXT|SPATIAL] INDEX
"CREATE INDEX part_of_name ON customer (name(10));",
"create index idx_test_cca on test_bb(test_cc);", // https://github.com/XiaoMi/soar/issues/205
"alter table `sakila`.`t1` add index `idx_col`(`col`)",
"alter table `sakila`.`t1` add UNIQUE index `idx_col`(`col`)",
"alter table `sakila`.`t1` add index `idx_ID`(`ID`)",
// ADD|DROP COLUMN
"ALTER TABLE t2 DROP COLUMN c, DROP COLUMN d;",
"ALTER TABLE T2 ADD COLUMN C int;",
"ALTER TABLE T2 ADD COLUMN D int FIRST;",
"ALTER TABLE T2 ADD COLUMN E int AFTER D;",
// RENAME COLUMN
"ALTER TABLE t1 RENAME COLUMN a TO b",
// RENAME INDEX
"ALTER TABLE t1 RENAME INDEX idx_a TO idx_b",
"ALTER TABLE t1 RENAME KEY idx_a TO idx_b",
// RENAME TABLE
"ALTER TABLE db.old_table RENAME new_table;",
"ALTER TABLE old_table RENAME TO new_table;",
"ALTER TABLE old_table RENAME AS new_table;",
// MODIFY & CHANGE
"ALTER TABLE t1 MODIFY col1 BIGINT UNSIGNED DEFAULT 1 COMMENT 'my column';",
"ALTER TABLE t1 CHANGE b a INT NOT NULL;",
// table name quote in back ticks
"alter table `t3`add index `idx_a`(a)",
"alter table`t3`drop index`idx_b`",
}
alterSQLs := MergeAlterTables(sqls...)
var sortedAlterSQLs []string
for item := range alterSQLs {
sortedAlterSQLs = append(sortedAlterSQLs, item)
}
sort.Strings(sortedAlterSQLs)
err := common.GoldenDiff(func() {
for _, tb := range sortedAlterSQLs {
fmt.Println(tb, ":", alterSQLs[tb])
}
}, t.Name(), update)
if err != nil {
t.Error(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteUnionAll(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": `select country_id from city union select country_id from country;`,
"output": "select country_id from city union all select country_id from country",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteUnionAll()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteTruncate(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": `delete from tbl;`,
"output": "truncate table tbl",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteTruncate()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRewriteOr2In(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": `select country_id from city where country_id = 1 or country_id = 2 or country_id = 3;`,
"output": "select country_id from city where country_id in (1, 2, 3)",
},
// TODO or中的恒真条件
{
"input": `select country_id from city where country_id != 1 or country_id != 2 or country_id = 3;`,
"output": "select country_id from city where country_id != 1 or country_id != 2 or country_id = 3",
},
// col = 1 or col is null不可转为IN
{
"input": `select country_id from city where col = 1 or col is null;`,
"output": "select country_id from city where col = 1 or col is null",
},
{
"input": `select country_id from city where col1 = 1 or col2 = 1 or col2 = 2;`,
"output": "select country_id from city where col1 = 1 or col2 in (1, 2)",
},
{
"input": `select country_id from city where col1 = 1 or col2 = 1 or col2 = 2 or col1 = 3;`,
"output": "select country_id from city where col2 in (1, 2) or col1 in (1, 3)",
},
{
"input": `select country_id from city where (col1 = 1 or col2 = 1 or col2 = 2 ) or col1 = 3;`,
"output": "select country_id from city where (col1 = 1 or col2 in (1, 2)) or col1 = 3",
},
{
"input": `select country_id from city where col1 = 1 or (col2 = 1 or col2 = 2 ) or col1 = 3;`,
"output": "select country_id from city where (col2 in (1, 2)) or col1 in (1, 3)",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteOr2In()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRmParenthesis(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
testSQL := []map[string]string{
{
"input": `select country_id from city where (country_id = 1);`,
"output": "select country_id from city where country_id = 1",
},
{
"input": `select * from city where a = 1 and (country_id = 1);`,
"output": "select * from city where a = 1 and country_id = 1",
},
{
"input": `select country_id from city where (country_id = 1) or country_id = 1 ;`,
"output": "select country_id from city where country_id = 1 or country_id = 1",
},
{
"input": `select country_id from city where col = 1 or (country_id = 1) or country_id = 1 ;`,
"output": "select country_id from city where col = 1 or country_id = 1 or country_id = 1",
},
}
for _, sql := range testSQL {
rw := NewRewrite(sql["input"]).RewriteRmParenthesis()
if rw.NewSQL != sql["output"] {
t.Errorf("want: %s\ngot: %s", sql["output"], rw.NewSQL)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestListRewriteRules(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
err := common.GoldenDiff(func() {
ListRewriteRules(RewriteRules)
orgReportType := common.Config.ReportType
common.Config.ReportType = "json"
ListRewriteRules(RewriteRules)
common.Config.ReportType = orgReportType
}, t.Name(), update)
if err != nil {
t.Error(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
================================================
FILE: ast/testdata/TestCompress.golden
================================================
SELECT * FROM film WHERE length = 86;
SELECT * FROM film WHERE length = 86;
SELECT * FROM film WHERE length IS NULL;
SELECT * FROM film WHERE length IS NULL;
SELECT * FROM film HAVING title = 'abc';
SELECT * FROM film HAVING title = 'abc';
SELECT * FROM sakila.film WHERE length >= 60;
SELECT * FROM sakila.film WHERE length >= 60;
SELECT * FROM sakila.film WHERE length >= '60';
SELECT * FROM sakila.film WHERE length >= '60';
SELECT * FROM film WHERE length BETWEEN 60 AND 84;
SELECT * FROM film WHERE length BETWEEN 60 AND 84;
SELECT * FROM film WHERE title LIKE 'AIR%';
SELECT * FROM film WHERE title LIKE 'AIR%';
SELECT * FROM film WHERE title IS NOT NULL;
SELECT * FROM film WHERE title IS NOT NULL;
SELECT * FROM film WHERE length = 114 and title = 'ALABAMA DEVIL';
SELECT * FROM film WHERE length = 114 and title = 'ALABAMA DEVIL';
SELECT * FROM film WHERE length > 100 and title = 'ALABAMA DEVIL';
SELECT * FROM film WHERE length > 100 and title = 'ALABAMA DEVIL';
SELECT * FROM film WHERE length > 100 and language_id < 10 and title = 'xyz';
SELECT * FROM film WHERE length > 100 and language_id < 10 and title = 'xyz';
SELECT * FROM film WHERE length > 100 and language_id < 10;
SELECT * FROM film WHERE length > 100 and language_id < 10;
SELECT release_year, sum(length) FROM film WHERE length = 123 AND language_id = 1 GROUP BY release_year;
SELECT release_year, sum(length) FROM film WHERE length = 123 AND language_id = 1 GROUP BY release_year;
SELECT release_year, sum(length) FROM film WHERE length >= 123 GROUP BY release_year;
SELECT release_year, sum(length) FROM film WHERE length >= 123 GROUP BY release_year;
SELECT release_year, language_id, sum(length) FROM film GROUP BY release_year, language_id;
SELECT release_year, language_id, sum(length) FROM film GROUP BY release_year, language_id;
SELECT release_year, sum(length) FROM film WHERE length = 123 GROUP BY release_year,(length+language_id);
SELECT release_year, sum(length) FROM film WHERE length = 123 GROUP BY release_year,(length+language_id);
SELECT release_year, sum(film_id) FROM film GROUP BY release_year;
SELECT release_year, sum(film_id) FROM film GROUP BY release_year;
SELECT * FROM address GROUP BY address,district;
SELECT * FROM address GROUP BY address,district;
SELECT title FROM film WHERE ABS(language_id) = 3 GROUP BY title;
SELECT title FROM film WHERE ABS(language_id) = 3 GROUP BY title;
SELECT language_id FROM film WHERE length = 123 GROUP BY release_year ORDER BY language_id;
SELECT language_id FROM film WHERE length = 123 GROUP BY release_year ORDER BY language_id;
SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year;
SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year;
SELECT * FROM film WHERE length = 123 ORDER BY release_year ASC, language_id DESC;
SELECT * FROM film WHERE length = 123 ORDER BY release_year ASC, language_id DESC;
SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year LIMIT 10;
SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year LIMIT 10;
SELECT * FROM film WHERE length = 123 ORDER BY release_year LIMIT 10;
SELECT * FROM film WHERE length = 123 ORDER BY release_year LIMIT 10;
SELECT * FROM film ORDER BY release_year LIMIT 10;
SELECT * FROM film ORDER BY release_year LIMIT 10;
SELECT film_id FROM film ORDER BY release_year LIMIT 10;
SELECT film_id FROM film ORDER BY release_year LIMIT 10;
SELECT * FROM film WHERE length > 100 ORDER BY length LIMIT 10;
SELECT * FROM film WHERE length > 100 ORDER BY length LIMIT 10;
SELECT * FROM film WHERE length < 100 ORDER BY length LIMIT 10;
SELECT * FROM film WHERE length < 100 ORDER BY length LIMIT 10;
SELECT * FROM customer WHERE address_id in (224,510) ORDER BY last_name;
SELECT * FROM customer WHERE address_id in (224,510) ORDER BY last_name;
SELECT * FROM film WHERE release_year = 2016 AND length != 1 ORDER BY title;
SELECT * FROM film WHERE release_year = 2016 AND length != 1 ORDER BY title;
SELECT title FROM film WHERE release_year = 1995;
SELECT title FROM film WHERE release_year = 1995;
SELECT title, replacement_cost FROM film WHERE language_id = 5 AND length = 70;
SELECT title, replacement_cost FROM film WHERE language_id = 5 AND length = 70;
SELECT title FROM film WHERE language_id > 5 AND length > 70;
SELECT title FROM film WHERE language_id > 5 AND length > 70;
SELECT * FROM film WHERE length = 100 and title = 'xyz' ORDER BY release_year;
SELECT * FROM film WHERE length = 100 and title = 'xyz' ORDER BY release_year;
SELECT * FROM film WHERE length > 100 and title = 'xyz' ORDER BY release_year;
SELECT * FROM film WHERE length > 100 and title = 'xyz' ORDER BY release_year;
SELECT * FROM film WHERE length > 100 ORDER BY release_year;
SELECT * FROM film WHERE length > 100 ORDER BY release_year;
SELECT * FROM city a INNER JOIN country b ON a.country_id=b.country_id;
SELECT * FROM city a INNER JOIN country b ON a.country_id=b.country_id;
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id;
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id;
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL;
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL;
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id UNION SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id UNION SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL UNION SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL UNION SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;
SELECT country_id, last_update FROM city NATURAL JOIN country;
SELECT country_id, last_update FROM city NATURAL JOIN country;
SELECT country_id, last_update FROM city NATURAL LEFT JOIN country;
SELECT country_id, last_update FROM city NATURAL LEFT JOIN country;
SELECT country_id, last_update FROM city NATURAL RIGHT JOIN country;
SELECT country_id, last_update FROM city NATURAL RIGHT JOIN country;
SELECT a.country_id, a.last_update FROM city a STRAIGHT_JOIN country b ON a.country_id=b.country_id;
SELECT a.country_id, a.last_update FROM city a STRAIGHT_JOIN country b ON a.country_id=b.country_id;
SELECT a.address, a.postal_code FROM sakila.address a WHERE a.city_id IN (SELECT c.city_id FROM sakila.city c);
SELECT a.address, a.postal_code FROM sakila.address a WHERE a.city_id IN (SELECT c.city_id FROM sakila.city c);
SELECT city FROM( SELECT city_id FROM city WHERE city = "A Corua (La Corua)" ORDER BY last_update DESC LIMIT 50, 10) I JOIN city ON (I.city_id = city.city_id) JOIN country ON (country.country_id = city.country_id) ORDER BY city DESC;
SELECT city FROM( SELECT city_id FROM city WHERE city = "A Corua (La Corua)" ORDER BY last_update DESC LIMIT 50, 10) I JOIN city ON (I.city_id = city.city_id) JOIN country ON (country.country_id = city.country_id) ORDER BY city DESC;
DELETE city, country FROM city INNER JOIN country using (country_id) WHERE city.city_id = 1;
DELETE city, country FROM city INNER JOIN country using (country_id) WHERE city.city_id = 1;
DELETE city FROM city LEFT JOIN country ON city.country_id = country.country_id WHERE country.country IS NULL;
DELETE city FROM city LEFT JOIN country ON city.country_id = country.country_id WHERE country.country IS NULL;
DELETE a1, a2 FROM city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;
DELETE a1, a2 FROM city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;
DELETE FROM a1, a2 USING city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;
DELETE FROM a1, a2 USING city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;
DELETE FROM film WHERE length > 100;
DELETE FROM film WHERE length > 100;
UPDATE city INNER JOIN country USING(country_id) SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;
UPDATE city INNER JOIN country USING(country_id) SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;
UPDATE city INNER JOIN country ON city.country_id = country.country_id INNER JOIN address ON city.city_id = address.city_id SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;
UPDATE city INNER JOIN country ON city.country_id = country.country_id INNER JOIN address ON city.city_id = address.city_id SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;
UPDATE city, country SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.country_id = country.country_id AND city.city_id=10;
UPDATE city, country SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.country_id = country.country_id AND city.city_id=10;
UPDATE film SET length = 10 WHERE language_id = 20;
UPDATE film SET length = 10 WHERE language_id = 20;
INSERT INTO city (country_id) SELECT country_id FROM country;
INSERT INTO city (country_id) SELECT country_id FROM country;
INSERT INTO city (country_id) VALUES (1),(2),(3);
INSERT INTO city (country_id) VALUES (1),(2),(3);
INSERT INTO city (country_id) VALUES (10);
INSERT INTO city (country_id) VALUES (10);
INSERT INTO city (country_id) SELECT 10 FROM DUAL;
INSERT INTO city (country_id) SELECT 10 FROM DUAL;
REPLACE INTO city (country_id) SELECT country_id FROM country;
REPLACE INTO city (country_id) SELECT country_id FROM country;
REPLACE INTO city (country_id) VALUES (1),(2),(3);
REPLACE INTO city (country_id) VALUES (1),(2),(3);
REPLACE INTO city (country_id) VALUES (10);
REPLACE INTO city (country_id) VALUES (10);
REPLACE INTO city (country_id) SELECT 10 FROM DUAL;
REPLACE INTO city (country_id) SELECT 10 FROM DUAL;
SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film;
SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film;
SELECT * FROM film WHERE language_id = (SELECT language_id FROM language LIMIT 1);
SELECT * FROM film WHERE language_id = (SELECT language_id FROM language LIMIT 1);
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;
SELECT * FROM (SELECT * FROM actor WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE') t WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE' GROUP BY first_name;
SELECT * FROM (SELECT * FROM actor WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE') t WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE' GROUP BY first_name;
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id WHERE o.country_id is null union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id WHERE i.city_id is null;
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id WHERE o.country_id is null union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id WHERE i.city_id is null;
SELECT first_name,last_name,email FROM customer STRAIGHT_JOIN address ON customer.address_id=address.address_id;
SELECT first_name,last_name,email FROM customer STRAIGHT_JOIN address ON customer.address_id=address.address_id;
SELECT ID,name FROM (SELECT address FROM customer_list WHERE SID=1 order by phone limit 50,10) a JOIN customer_list l ON (a.address=l.address) JOIN city c ON (c.city=l.city) order by phone desc;
SELECT ID,name FROM (SELECT address FROM customer_list WHERE SID=1 order by phone limit 50,10) a JOIN customer_list l ON (a.address=l.address) JOIN city c ON (c.city=l.city) order by phone desc;
SELECT * FROM film WHERE date(last_update)='2006-02-15';
SELECT * FROM film WHERE date(last_update)='2006-02-15';
SELECT last_update FROM film GROUP BY date(last_update);
SELECT last_update FROM film GROUP BY date(last_update);
SELECT last_update FROM film order by date(last_update);
SELECT last_update FROM film order by date(last_update);
SELECT description FROM film WHERE description IN('NEWS','asd') GROUP BY description;
SELECT description FROM film WHERE description IN('NEWS','asd') GROUP BY description;
alter table address add index idx_city_id(city_id);
alter table address add index idx_city_id(city_id);
alter table inventory add index `idx_store_film` (`store_id`,`film_id`);
alter table inventory add index `idx_store_film` (`store_id`,`film_id`);
alter table inventory add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`);
alter table inventory add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`);
SELECT DATE_FORMAT(t.last_update, '%Y-%m-%d'), COUNT(DISTINCT (t.city)) FROM city t WHERE t.last_update > '2018-10-22 00:00:00' AND t.city LIKE '%Chrome%' AND t.city = 'eip' GROUP BY DATE_FORMAT(t.last_update, '%Y-%m-%d') ORDER BY DATE_FORMAT(t.last_update, '%Y-%m-%d');
SELECT DATE_FORMAT(t.last_update, '%Y-%m-%d'), COUNT(DISTINCT (t.city)) FROM city t WHERE t.last_update > '2018-10-22 00:00:00' AND t.city LIKE '%Chrome%' AND t.city = 'eip' GROUP BY DATE_FORMAT(t.last_update, '%Y-%m-%d') ORDER BY DATE_FORMAT(t.last_update, '%Y-%m-%d');
create table hello.t (id int unsigned);
create table hello.t (id int unsigned);
select * from tb where data >= '';
select * from tb where data >= '';
alter table tb alter column id drop default;
alter table tb alter column id drop default;
select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film where last_update > '2016-03-27 02:01:01') as d;
select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film where last_update > '2016-03-27 02:01:01') as d;
select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film) as d;
select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film) as d;
================================================
FILE: ast/testdata/TestFormat.golden
================================================
SELECT * FROM film WHERE length = 86;
SELECT
*
FROM
film
WHERE
LENGTH = 86;
SELECT * FROM film WHERE length IS NULL;
SELECT
*
FROM
film
WHERE
LENGTH IS NULL;
SELECT * FROM film HAVING title = 'abc';
SELECT
*
FROM
film
HAVING
title = 'abc';
SELECT * FROM sakila.film WHERE length >= 60;
SELECT
*
FROM
sakila. film
WHERE
LENGTH >= 60;
SELECT * FROM sakila.film WHERE length >= '60';
SELECT
*
FROM
sakila. film
WHERE
LENGTH >= '60';
SELECT * FROM film WHERE length BETWEEN 60 AND 84;
SELECT
*
FROM
film
WHERE
LENGTH BETWEEN 60
AND 84;
SELECT * FROM film WHERE title LIKE 'AIR%';
SELECT
*
FROM
film
WHERE
title LIKE 'AIR%';
SELECT * FROM film WHERE title IS NOT NULL;
SELECT
*
FROM
film
WHERE
title IS NOT NULL;
SELECT * FROM film WHERE length = 114 and title = 'ALABAMA DEVIL';
SELECT
*
FROM
film
WHERE
LENGTH = 114
AND title = 'ALABAMA DEVIL';
SELECT * FROM film WHERE length > 100 and title = 'ALABAMA DEVIL';
SELECT
*
FROM
film
WHERE
LENGTH > 100
AND title = 'ALABAMA DEVIL';
SELECT * FROM film WHERE length > 100 and language_id < 10 and title = 'xyz';
SELECT
*
FROM
film
WHERE
LENGTH > 100
AND language_id < 10
AND title = 'xyz';
SELECT * FROM film WHERE length > 100 and language_id < 10;
SELECT
*
FROM
film
WHERE
LENGTH > 100
AND language_id < 10;
SELECT release_year, sum(length) FROM film WHERE length = 123 AND language_id = 1 GROUP BY release_year;
SELECT
release_year, SUM( LENGTH)
FROM
film
WHERE
LENGTH = 123
AND language_id = 1
GROUP BY
release_year;
SELECT release_year, sum(length) FROM film WHERE length >= 123 GROUP BY release_year;
SELECT
release_year, SUM( LENGTH)
FROM
film
WHERE
LENGTH >= 123
GROUP BY
release_year;
SELECT release_year, language_id, sum(length) FROM film GROUP BY release_year, language_id;
SELECT
release_year, language_id, SUM( LENGTH)
FROM
film
GROUP BY
release_year, language_id;
SELECT release_year, sum(length) FROM film WHERE length = 123 GROUP BY release_year,(length+language_id);
SELECT
release_year, SUM( LENGTH)
FROM
film
WHERE
LENGTH = 123
GROUP BY
release_year, (LENGTH+ language_id);
SELECT release_year, sum(film_id) FROM film GROUP BY release_year;
SELECT
release_year, SUM( film_id)
FROM
film
GROUP BY
release_year;
SELECT * FROM address GROUP BY address,district;
SELECT
*
FROM
address
GROUP BY
address, district;
SELECT title FROM film WHERE ABS(language_id) = 3 GROUP BY title;
SELECT
title
FROM
film
WHERE
ABS( language_id) = 3
GROUP BY
title;
SELECT language_id FROM film WHERE length = 123 GROUP BY release_year ORDER BY language_id;
SELECT
language_id
FROM
film
WHERE
LENGTH = 123
GROUP BY
release_year
ORDER BY
language_id;
SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year;
SELECT
release_year
FROM
film
WHERE
LENGTH = 123
GROUP BY
release_year
ORDER BY
release_year;
SELECT * FROM film WHERE length = 123 ORDER BY release_year ASC, language_id DESC;
SELECT
*
FROM
film
WHERE
LENGTH = 123
ORDER BY
release_year ASC, language_id DESC;
SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year LIMIT 10;
SELECT
release_year
FROM
film
WHERE
LENGTH = 123
GROUP BY
release_year
ORDER BY
release_year
LIMIT
10;
SELECT * FROM film WHERE length = 123 ORDER BY release_year LIMIT 10;
SELECT
*
FROM
film
WHERE
LENGTH = 123
ORDER BY
release_year
LIMIT
10;
SELECT * FROM film ORDER BY release_year LIMIT 10;
SELECT
*
FROM
film
ORDER BY
release_year
LIMIT
10;
SELECT film_id FROM film ORDER BY release_year LIMIT 10;
SELECT
film_id
FROM
film
ORDER BY
release_year
LIMIT
10;
SELECT * FROM film WHERE length > 100 ORDER BY length LIMIT 10;
SELECT
*
FROM
film
WHERE
LENGTH > 100
ORDER BY
LENGTH
LIMIT
10;
SELECT * FROM film WHERE length < 100 ORDER BY length LIMIT 10;
SELECT
*
FROM
film
WHERE
LENGTH < 100
ORDER BY
LENGTH
LIMIT
10;
SELECT * FROM customer WHERE address_id in (224,510) ORDER BY last_name;
SELECT
*
FROM
customer
WHERE
address_id in (224, 510)
ORDER BY
last_name;
SELECT * FROM film WHERE release_year = 2016 AND length != 1 ORDER BY title;
SELECT
*
FROM
film
WHERE
release_year = 2016
AND LENGTH != 1
ORDER BY
title;
SELECT title FROM film WHERE release_year = 1995;
SELECT
title
FROM
film
WHERE
release_year = 1995;
SELECT title, replacement_cost FROM film WHERE language_id = 5 AND length = 70;
SELECT
title, replacement_cost
FROM
film
WHERE
language_id = 5
AND LENGTH = 70;
SELECT title FROM film WHERE language_id > 5 AND length > 70;
SELECT
title
FROM
film
WHERE
language_id > 5
AND LENGTH > 70;
SELECT * FROM film WHERE length = 100 and title = 'xyz' ORDER BY release_year;
SELECT
*
FROM
film
WHERE
LENGTH = 100
AND title = 'xyz'
ORDER BY
release_year;
SELECT * FROM film WHERE length > 100 and title = 'xyz' ORDER BY release_year;
SELECT
*
FROM
film
WHERE
LENGTH > 100
AND title = 'xyz'
ORDER BY
release_year;
SELECT * FROM film WHERE length > 100 ORDER BY release_year;
SELECT
*
FROM
film
WHERE
LENGTH > 100
ORDER BY
release_year;
SELECT * FROM city a INNER JOIN country b ON a.country_id=b.country_id;
SELECT
*
FROM
city a
INNER JOIN country b ON a. country_id= b. country_id;
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id;
SELECT
*
FROM
city a
LEFT JOIN country b ON a. country_id= b. country_id;
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;
SELECT
*
FROM
city a
RIGHT JOIN country b ON a. country_id= b. country_id;
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;
SELECT
*
FROM
city a
LEFT JOIN country b ON a. country_id= b. country_id
WHERE
b. last_update IS NULL;
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL;
SELECT
*
FROM
city a
RIGHT JOIN country b ON a. country_id= b. country_id
WHERE
a. last_update IS NULL;
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id UNION SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;
SELECT
*
FROM
city a
LEFT JOIN country b ON a. country_id= b. country_id
UNION
SELECT
*
FROM
city a
RIGHT JOIN country b ON a. country_id= b. country_id;
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL UNION SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;
SELECT
*
FROM
city a
RIGHT JOIN country b ON a. country_id= b. country_id
WHERE
a. last_update IS NULL
UNION
SELECT
*
FROM
city a
LEFT JOIN country b ON a. country_id= b. country_id
WHERE
b. last_update IS NULL;
SELECT country_id, last_update FROM city NATURAL JOIN country;
SELECT
country_id, last_update
FROM
city NATURAL
JOIN country;
SELECT country_id, last_update FROM city NATURAL LEFT JOIN country;
SELECT
country_id, last_update
FROM
city NATURAL
LEFT JOIN country;
SELECT country_id, last_update FROM city NATURAL RIGHT JOIN country;
SELECT
country_id, last_update
FROM
city NATURAL
RIGHT JOIN country;
SELECT a.country_id, a.last_update FROM city a STRAIGHT_JOIN country b ON a.country_id=b.country_id;
SELECT
a. country_id, a. last_update
FROM
city a STRAIGHT_JOIN country b ON a. country_id= b. country_id;
SELECT a.address, a.postal_code FROM sakila.address a WHERE a.city_id IN (SELECT c.city_id FROM sakila.city c);
SELECT
a. address, a. postal_code
FROM
sakila. address a
WHERE
a. city_id IN (
SELECT
c. city_id
FROM
sakila. city c);
SELECT city FROM( SELECT city_id FROM city WHERE city = "A Corua (La Corua)" ORDER BY last_update DESC LIMIT 50, 10) I JOIN city ON (I.city_id = city.city_id) JOIN country ON (country.country_id = city.country_id) ORDER BY city DESC;
SELECT
city
FROM(
SELECT
city_id
FROM
city
WHERE
city = "A Corua (La Corua)"
ORDER BY
last_update DESC
LIMIT
50, 10) I
JOIN city ON (I. city_id = city. city_id)
JOIN country ON (country. country_id = city. country_id)
ORDER BY
city DESC;
DELETE city, country FROM city INNER JOIN country using (country_id) WHERE city.city_id = 1;
DELETE city, country
FROM
city
INNER JOIN country using (country_id)
WHERE
city. city_id = 1;
DELETE city FROM city LEFT JOIN country ON city.country_id = country.country_id WHERE country.country IS NULL;
DELETE city
FROM
city
LEFT JOIN country ON city. country_id = country. country_id
WHERE
country. country IS NULL;
DELETE a1, a2 FROM city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;
DELETE a1, a2
FROM
city AS a1
INNER JOIN country AS a2
WHERE
a1. country_id= a2. country_id;
DELETE FROM a1, a2 USING city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;
DELETE FROM
a1, a2 USING city AS a1
INNER JOIN country AS a2
WHERE
a1. country_id= a2. country_id;
DELETE FROM film WHERE length > 100;
DELETE FROM
film
WHERE
LENGTH > 100;
UPDATE city INNER JOIN country USING(country_id) SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;
UPDATE
city
INNER JOIN country USING( country_id)
SET
city. city = 'Abha',
city. last_update = '2006-02-15 04:45:25',
country. country = 'Afghanistan'
WHERE
city. city_id= 10;
UPDATE city INNER JOIN country ON city.country_id = country.country_id INNER JOIN address ON city.city_id = address.city_id SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;
UPDATE
city
INNER JOIN country ON city. country_id = country. country_id
INNER JOIN address ON city. city_id = address. city_id
SET
city. city = 'Abha',
city. last_update = '2006-02-15 04:45:25',
country. country = 'Afghanistan'
WHERE
city. city_id= 10;
UPDATE city, country SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.country_id = country.country_id AND city.city_id=10;
UPDATE
city, country
SET
city. city = 'Abha',
city. last_update = '2006-02-15 04:45:25',
country. country = 'Afghanistan'
WHERE
city. country_id = country. country_id
AND city. city_id= 10;
UPDATE film SET length = 10 WHERE language_id = 20;
UPDATE
film
SET
LENGTH = 10
WHERE
language_id = 20;
INSERT INTO city (country_id) SELECT country_id FROM country;
INSERT INTO city (country_id)
SELECT
country_id
FROM
country;
INSERT INTO city (country_id) VALUES (1),(2),(3);
INSERT INTO city (country_id)
VALUES
(1),
(2),
(3);
INSERT INTO city (country_id) VALUES (10);
INSERT INTO city (country_id)
VALUES
(10);
INSERT INTO city (country_id) SELECT 10 FROM DUAL;
INSERT INTO city (country_id)
SELECT
10
FROM
DUAL;
REPLACE INTO city (country_id) SELECT country_id FROM country;
REPLACE INTO city (country_id)
SELECT
country_id
FROM
country;
REPLACE INTO city (country_id) VALUES (1),(2),(3);
REPLACE INTO city (country_id)
VALUES
(1),
(2),
(3);
REPLACE INTO city (country_id) VALUES (10);
REPLACE INTO city (country_id)
VALUES
(10);
REPLACE INTO city (country_id) SELECT 10 FROM DUAL;
REPLACE INTO city (country_id)
SELECT
10
FROM
DUAL;
SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film;
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film;
SELECT * FROM film WHERE language_id = (SELECT language_id FROM language LIMIT 1);
SELECT
*
FROM
film
WHERE
language_id = (
SELECT
language_id
FROM
language
LIMIT
1);
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;
SELECT
*
FROM
city i
LEFT JOIN country o ON i. city_id= o. country_id
UNION
SELECT
*
FROM
city i
RIGHT JOIN country o ON i. city_id= o. country_id;
SELECT * FROM (SELECT * FROM actor WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE') t WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE' GROUP BY first_name;
SELECT
*
FROM
(
SELECT
*
FROM
actor
WHERE
last_update= '2006-02-15 04:34:33'
AND last_name= 'CHASE'
) t
WHERE
last_update= '2006-02-15 04:34:33'
AND last_name= 'CHASE'
GROUP BY
first_name;
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;
SELECT
*
FROM
city i
LEFT JOIN country o ON i. city_id= o. country_id
UNION
SELECT
*
FROM
city i
RIGHT JOIN country o ON i. city_id= o. country_id;
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id WHERE o.country_id is null union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id WHERE i.city_id is null;
SELECT
*
FROM
city i
LEFT JOIN country o ON i. city_id= o. country_id
WHERE
o. country_id is null
UNION
SELECT
*
FROM
city i
RIGHT JOIN country o ON i. city_id= o. country_id
WHERE
i. city_id is null;
SELECT first_name,last_name,email FROM customer STRAIGHT_JOIN address ON customer.address_id=address.address_id;
SELECT
first_name, last_name, email
FROM
customer STRAIGHT_JOIN address ON customer. address_id= address. address_id;
SELECT ID,name FROM (SELECT address FROM customer_list WHERE SID=1 order by phone limit 50,10) a JOIN customer_list l ON (a.address=l.address) JOIN city c ON (c.city=l.city) order by phone desc;
SELECT
ID, name
FROM
(
SELECT
address
FROM
customer_list
WHERE
SID= 1
ORDER BY
phone
LIMIT
50, 10) a
JOIN customer_list l ON (a. address= l. address)
JOIN city c ON (c. city= l. city)
ORDER BY
phone desc;
SELECT * FROM film WHERE date(last_update)='2006-02-15';
SELECT
*
FROM
film
WHERE
DATE( last_update) = '2006-02-15';
SELECT last_update FROM film GROUP BY date(last_update);
SELECT
last_update
FROM
film
GROUP BY
DATE( last_update);
SELECT last_update FROM film order by date(last_update);
SELECT
last_update
FROM
film
ORDER BY
DATE( last_update);
SELECT description FROM film WHERE description IN('NEWS','asd') GROUP BY description;
SELECT
description
FROM
film
WHERE
description IN( 'NEWS',
'asd'
)
GROUP BY
description;
alter table address add index idx_city_id(city_id);
ALTER TABLE
address
ADD
index idx_city_id( city_id);
alter table inventory add index `idx_store_film` (`store_id`,`film_id`);
ALTER TABLE
inventory
ADD
index `idx_store_film` (
`store_id`, `film_id`);
alter table inventory add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`);
ALTER TABLE
inventory
ADD
index `idx_store_film` (
`store_id`, `film_id`),
ADD
index `idx_store_film` (
`store_id`, `film_id`),
ADD
index `idx_store_film` (
`store_id`, `film_id`);
SELECT DATE_FORMAT(t.last_update, '%Y-%m-%d'), COUNT(DISTINCT (t.city)) FROM city t WHERE t.last_update > '2018-10-22 00:00:00' AND t.city LIKE '%Chrome%' AND t.city = 'eip' GROUP BY DATE_FORMAT(t.last_update, '%Y-%m-%d') ORDER BY DATE_FORMAT(t.last_update, '%Y-%m-%d');
SELECT
DATE_FORMAT( t. last_update, '%Y-%m-%d'
),
COUNT( DISTINCT (
t. city))
FROM
city t
WHERE
t. last_update > '2018-10-22 00:00:00'
AND t. city LIKE '%Chrome%'
AND t. city = 'eip'
GROUP BY
DATE_FORMAT( t. last_update, '%Y-%m-%d'
)
ORDER BY
DATE_FORMAT( t. last_update, '%Y-%m-%d'
);
create table hello.t (id int unsigned);
create table hello. t (id int unsigned);
select * from tb where data >= '';
SELECT
*
FROM
tb
WHERE
data >= '';
alter table tb alter column id drop default;
ALTER TABLE
tb alter column id
DROP
DEFAULT;
select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film where last_update > '2016-03-27 02:01:01') as d;
SELECT
maxId, minId
FROM
(
SELECT
MAX( film_id) maxId, MIN( film_id) minId
FROM
film
WHERE
last_update > '2016-03-27 02:01:01'
) as d;
select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film) as d;
SELECT
maxId, minId
FROM
(
SELECT
MAX( film_id) maxId, MIN( film_id) minId
FROM
film) as d;
================================================
FILE: ast/testdata/TestGetQuotedString.golden
================================================
orignal: "hello world"
quoted: "hello world"
orignal: `hello world`
quoted: `hello world`
orignal: 'hello world'
quoted: 'hello world'
orignal: hello world
quoted:
orignal: 'hello \'world'
quoted: 'hello \'world'
orignal: "hello \"wor\"ld"
quoted: "hello \"wor\"ld"
orignal: "hello \"world"
quoted: "hello \"world"
orignal: ""
quoted: ""
orignal: ''
quoted: ''
orignal: ``
quoted: ``
orignal: 'hello 'world'
quoted: 'hello '
orignal: "hello "world"
quoted: "hello "
orignal: ' LOGGER.error(""); }'
quoted: ' LOGGER.error(""); }'
================================================
FILE: ast/testdata/TestLeftNewLines.golden
================================================
1
0
0
================================================
FILE: ast/testdata/TestListRewriteRules.golden
================================================
# 重写规则
[toc]
## dml2select
* **Description**:将数据库更新请求转换为只读查询请求，便于执行EXPLAIN
* **Original**:
```sql
DELETE FROM film WHERE length > 100
```
* **Suggest**:
```sql
select * from film where length > 100
```
## reg2select
* **Description**:使用正则的方式将数据库更新请求转换为只读查询请求，便于执行EXPLAIN
* **Original**:
```sql
DELETE FROM film WHERE length > 100
```
* **Suggest**:
```sql
select * from film where length > 100
```
## star2columns
* **Description**:为SELECT *补全表的列信息
* **Original**:
```sql
SELECT * FROM film
```
* **Suggest**:
```sql
select film.film_id, film.title from film
```
## insertcolumns
* **Description**:为INSERT补全表的列信息
* **Original**:
```sql
insert into film values(1,2,3,4,5)
```
* **Suggest**:
```sql
insert into film(film_id, title, description, release_year, language_id) values (1, 2, 3, 4, 5)
```
## having
* **Description**:将查询的 HAVING 子句改写为 WHERE 中的查询条件
* **Original**:
```sql
SELECT state, COUNT(*) FROM Drivers GROUP BY state HAVING state IN ('GA', 'TX') ORDER BY state
```
* **Suggest**:
```sql
select state, COUNT(*) from Drivers where state in ('GA', 'TX') group by state order by state asc
```
## orderbynull
* **Description**:如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 ORDER BY NULL
* **Original**:
```sql
SELECT sum(col1) FROM tbl GROUP BY col
```
* **Suggest**:
```sql
select sum(col1) from tbl group by col order by null
```
## unionall
* **Description**:可以接受重复的时间，使用 UNION ALL 替代 UNION 以提高查询效率
* **Original**:
```sql
select country_id from city union select country_id from country
```
* **Suggest**:
```sql
select country_id from city union all select country_id from country
```
## or2in
* **Description**:将同一列不同条件的 OR 查询转写为 IN 查询
* **Original**:
```sql
select country_id from city where col1 = 1 or (col2 = 1 or col2 = 2 ) or col1 = 3;
```
* **Suggest**:
```sql
select country_id from city where (col2 in (1, 2)) or col1 in (1, 3);
```
## dmlorderby
* **Description**:删除 DML 更新操作中无意义的 ORDER BY
* **Original**:
```sql
DELETE FROM tbl WHERE col1=1 ORDER BY col
```
* **Suggest**:
```sql
delete from tbl where col1 = 1
```
## distinctstar
* **Description**:DISTINCT *对有主键的表没有意义，可以将DISTINCT删掉
* **Original**:
```sql
SELECT DISTINCT * FROM film;
```
* **Suggest**:
```sql
SELECT * FROM film
```
## standard
* **Description**:SQL标准化，如：关键字转换为小写
* **Original**:
```sql
SELECT sum(col1) FROM tbl GROUP BY 1;
```
* **Suggest**:
```sql
select sum(col1) from tbl group by 1
```
## mergealter
* **Description**:合并同一张表的多条ALTER语句
* **Original**:
```sql
ALTER TABLE t2 DROP COLUMN c;ALTER TABLE t2 DROP COLUMN d;
```
* **Suggest**:
```sql
ALTER TABLE t2 DROP COLUMN c, DROP COLUMN d;
```
## alwaystrue
* **Description**:删除无用的恒真判断条件
* **Original**:
```sql
SELECT count(col) FROM tbl where 'a'= 'a' or ('b' = 'b' and a = 'b');
```
* **Suggest**:
```sql
select count(col) from tbl where (a = 'b');
```
## countstar
* **Description**:不建议使用COUNT(col)或COUNT(常量)，建议改写为COUNT(*)
* **Original**:
```sql
SELECT count(col) FROM tbl GROUP BY 1;
```
* **Suggest**:
```sql
SELECT count(*) FROM tbl GROUP BY 1;
```
## innodb
* **Description**:建表时建议使用InnoDB引擎，非 InnoDB 引擎表自动转 InnoDB
* **Original**:
```sql
CREATE TABLE t1(id bigint(20) NOT NULL AUTO_INCREMENT);
```
* **Suggest**:
```sql
create table t1 (
id bigint(20) not null auto_increment
) ENGINE=InnoDB;
```
## autoincrement
* **Description**:将autoincrement初始化为1
* **Original**:
```sql
CREATE TABLE t1(id bigint(20) NOT NULL AUTO_INCREMENT) ENGINE=InnoDB AUTO_INCREMENT=123802;
```
* **Suggest**:
```sql
create table t1(id bigint(20) not null auto_increment) ENGINE=InnoDB auto_increment=1;
```
## intwidth
* **Description**:整型数据类型修改默认显示宽度
* **Original**:
```sql
create table t1 (id int(20) not null auto_increment) ENGINE=InnoDB;
```
* **Suggest**:
```sql
create table t1 (id int(10) not null auto_increment) ENGINE=InnoDB;
```
## truncate
* **Description**:不带 WHERE 条件的 DELETE 操作建议修改为 TRUNCATE
* **Original**:
```sql
DELETE FROM tbl
```
* **Suggest**:
```sql
truncate table tbl
```
## rmparenthesis
* **Description**:去除没有意义的括号
* **Original**:
```sql
select col from table where (col = 1);
```
* **Suggest**:
```sql
select col from table where col = 1;
```
## delimiter
* **Description**:补全DELIMITER
* **Original**:
```sql
use sakila
```
* **Suggest**:
```sql
use sakila;
```
[
{
"Name": "dml2select",
"Description": "将数据库更新请求转换为只读查询请求，便于执行EXPLAIN",
"Original": "DELETE FROM film WHERE length \u003e 100",
"Suggest": "select * from film where length \u003e 100"
},
{
"Name": "reg2select",
"Description": "使用正则的方式将数据库更新请求转换为只读查询请求，便于执行EXPLAIN",
"Original": "DELETE FROM film WHERE length \u003e 100",
"Suggest": "select * from film where length \u003e 100"
},
{
"Name": "star2columns",
"Description": "为SELECT *补全表的列信息",
"Original": "SELECT * FROM film",
"Suggest": "select film.film_id, film.title from film"
},
{
"Name": "insertcolumns",
"Description": "为INSERT补全表的列信息",
"Original": "insert into film values(1,2,3,4,5)",
"Suggest": "insert into film(film_id, title, description, release_year, language_id) values (1, 2, 3, 4, 5)"
},
{
"Name": "having",
"Description": "将查询的 HAVING 子句改写为 WHERE 中的查询条件",
"Original": "SELECT state, COUNT(*) FROM Drivers GROUP BY state HAVING state IN ('GA', 'TX') ORDER BY state",
"Suggest": "select state, COUNT(*) from Drivers where state in ('GA', 'TX') group by state order by state asc"
},
{
"Name": "orderbynull",
"Description": "如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 ORDER BY NULL",
"Original": "SELECT sum(col1) FROM tbl GROUP BY col",
"Suggest": "select sum(col1) from tbl group by col order by null"
},
{
"Name": "unionall",
"Description": "可以接受重复的时间，使用 UNION ALL 替代 UNION 以提高查询效率",
"Original": "select country_id from city union select country_id from country",
"Suggest": "select country_id from city union all select country_id from country"
},
{
"Name": "or2in",
"Description": "将同一列不同条件的 OR 查询转写为 IN 查询",
"Original": "select country_id from city where col1 = 1 or (col2 = 1 or col2 = 2 ) or col1 = 3;",
"Suggest": "select country_id from city where (col2 in (1, 2)) or col1 in (1, 3);"
},
{
"Name": "innull",
"Description": "如果 IN 条件中可能有 NULL 值而又想匹配 NULL 值时，建议添加OR col IS NULL",
"Original": "暂不支持",
"Suggest": "暂不支持"
},
{
"Name": "or2union",
"Description": "将不同列的 OR 查询转为 UNION 查询，建议结合 unionall 重写策略一起使用",
"Original": "暂不支持",
"Suggest": "暂不支持"
},
{
"Name": "dmlorderby",
"Description": "删除 DML 更新操作中无意义的 ORDER BY",
"Original": "DELETE FROM tbl WHERE col1=1 ORDER BY col",
"Suggest": "delete from tbl where col1 = 1"
},
{
"Name": "sub2join",
"Description": "将子查询转换为JOIN查询",
"Original": "暂不支持",
"Suggest": "暂不支持"
},
{
"Name": "join2sub",
"Description": "将JOIN查询转换为子查询",
"Original": "暂不支持",
"Suggest": "暂不支持"
},
{
"Name": "distinctstar",
"Description": "DISTINCT *对有主键的表没有意义，可以将DISTINCT删掉",
"Original": "SELECT DISTINCT * FROM film;",
"Suggest": "SELECT * FROM film"
},
{
"Name": "standard",
"Description": "SQL标准化，如：关键字转换为小写",
"Original": "SELECT sum(col1) FROM tbl GROUP BY 1;",
"Suggest": "select sum(col1) from tbl group by 1"
},
{
"Name": "mergealter",
"Description": "合并同一张表的多条ALTER语句",
"Original": "ALTER TABLE t2 DROP COLUMN c;ALTER TABLE t2 DROP COLUMN d;",
"Suggest": "ALTER TABLE t2 DROP COLUMN c, DROP COLUMN d;"
},
{
"Name": "alwaystrue",
"Description": "删除无用的恒真判断条件",
"Original": "SELECT count(col) FROM tbl where 'a'= 'a' or ('b' = 'b' and a = 'b');",
"Suggest": "select count(col) from tbl where (a = 'b');"
},
{
"Name": "countstar",
"Description": "不建议使用COUNT(col)或COUNT(常量)，建议改写为COUNT(*)",
"Original": "SELECT count(col) FROM tbl GROUP BY 1;",
"Suggest": "SELECT count(*) FROM tbl GROUP BY 1;"
},
{
"Name": "innodb",
"Description": "建表时建议使用InnoDB引擎，非 InnoDB 引擎表自动转 InnoDB",
"Original": "CREATE TABLE t1(id bigint(20) NOT NULL AUTO_INCREMENT);",
"Suggest": "create table t1 (\n\tid bigint(20) not null auto_increment\n) ENGINE=InnoDB;"
},
{
"Name": "autoincrement",
"Description": "将autoincrement初始化为1",
"Original": "CREATE TABLE t1(id bigint(20) NOT NULL AUTO_INCREMENT) ENGINE=InnoDB AUTO_INCREMENT=123802;",
"Suggest": "create table t1(id bigint(20) not null auto_increment) ENGINE=InnoDB auto_increment=1;"
},
{
"Name": "intwidth",
"Description": "整型数据类型修改默认显示宽度",
"Original": "create table t1 (id int(20) not null auto_increment) ENGINE=InnoDB;",
"Suggest": "create table t1 (id int(10) not null auto_increment) ENGINE=InnoDB;"
},
{
"Name": "truncate",
"Description": "不带 WHERE 条件的 DELETE 操作建议修改为 TRUNCATE",
"Original": "DELETE FROM tbl",
"Suggest": "truncate table tbl"
},
{
"Name": "rmparenthesis",
"Description": "去除没有意义的括号",
"Original": "select col from table where (col = 1);",
"Suggest": "select col from table where col = 1;"
},
{
"Name": "delimiter",
"Description": "补全DELIMITER",
"Original": "use sakila",
"Suggest": "use sakila;"
}
]
================================================
FILE: ast/testdata/TestMergeAlterTables.golden
================================================
`customer` : ALTER TABLE `customer` ADD INDEX part_of_name (name(10)) ;
`db`.`old_table` : ALTER TABLE `db`.`old_table` RENAME new_table ;
`old_table` : ALTER TABLE `old_table` RENAME TO new_table, RENAME AS new_table ;
`sakila`.`t1` : ALTER TABLE `sakila`.`t1` add index `idx_col`(`col`), add UNIQUE index `idx_col`(`col`), add index `idx_ID`(`ID`) ;
`t1` : ALTER TABLE `t1` RENAME COLUMN a TO b, RENAME INDEX idx_a TO idx_b, RENAME KEY idx_a TO idx_b, MODIFY col1 BIGINT UNSIGNED DEFAULT 1 COMMENT 'my column', CHANGE b a INT NOT NULL ;
`t2` : ALTER TABLE `t2` DROP COLUMN c, DROP COLUMN d, ADD COLUMN C int, ADD COLUMN D int FIRST, ADD COLUMN E int AFTER D ;
`t3` : ALTER TABLE `t3` add index `idx_a`(a), drop index`idx_b` ;
`test_bb` : ALTER TABLE `test_bb` ADD INDEX idx_test_cca (test_cc) ;
================================================
FILE: ast/testdata/TestNewLines.golden
================================================
1
2
0
================================================
FILE: ast/testdata/TestPretty.golden
================================================
select sourcetable, if(f.lastcontent = ?, f.lastupdate, f.lastcontent) as lastactivity, f.totalcount as activity, type.class as type, (f.nodeoptions & ?) as nounsubscribe from node as f inner join contenttype as type on type.contenttypeid = f.contenttypeid inner join subscribed as sd on sd.did = f.nodeid and sd.userid = ? union all select f.name as title, f.userid as keyval, ? as sourcetable, ifnull(f.lastpost, f.joindate) as lastactivity, f.posts as activity, ? as type, ? as nounsubscribe from user as f inner join userlist as ul on ul.relationid = f.userid and ul.userid = ? where ul.type = ? and ul.aq = ? order by title limit ?
SELECT
sourcetable, IF( f. lastcontent = ?, f. lastupdate, f. lastcontent) as lastactivity, f. totalcount as activity, type. class as type, (f. nodeoptions & ?) as nounsubscribe
FROM
node as f
INNER JOIN contenttype as type on type. contenttypeid = f. contenttypeid
INNER JOIN subscribed as sd on sd. did = f. nodeid
AND sd. userid = ?
UNION ALL
SELECT
f. name as title, f. userid as keyval, ? as sourcetable, IFNULL( f. lastpost, f. joindate) as lastactivity, f. posts as activity, ? as type, ? as nounsubscribe
FROM
USER as f
INNER JOIN userlist as ul on ul. relationid = f. userid
AND ul. userid = ?
WHERE
ul. type = ?
AND ul. aq = ?
ORDER BY
title
LIMIT
?
administrator command: Init DB
administrator command: Init DB
CALL foo(1, 2, 3)
CALL foo( 1, 2, 3)
### Channels ###
SELECT sourcetable, IF(f.lastcontent = 0, f.lastupdate, f.lastcontent) AS lastactivity,
f.totalcount AS activity, type.class AS type,
(f.nodeoptions & 512) AS noUnsubscribe
FROM node AS f
INNER JOIN contenttype AS type ON type.contenttypeid = f.contenttypeid
INNER JOIN subscribed AS sd ON sd.did = f.nodeid AND sd.userid = 15965
UNION ALL
### Users ###
SELECT f.name AS title, f.userid AS keyval, 'user' AS sourcetable, IFNULL(f.lastpost, f.joindate) AS lastactivity,
f.posts as activity, 'Member' AS type,
0 AS noUnsubscribe
FROM user AS f
INNER JOIN userlist AS ul ON ul.relationid = f.userid AND ul.userid = 15965
WHERE ul.type = 'f' AND ul.aq = 'yes'
ORDER BY title ASC LIMIT 100
### Channels ###
SELECT
sourcetable, IF( f. lastcontent = 0, f. lastupdate, f. lastcontent) AS lastactivity, f. totalcount AS activity, type. class AS type, (f. nodeoptions & 512) AS noUnsubscribe
FROM
node AS f
INNER JOIN contenttype AS type ON type. contenttypeid = f. contenttypeid
INNER JOIN subscribed AS sd ON sd. did = f. nodeid
AND sd. userid = 15965
UNION
ALL ### Users ###
SELECT
f. name AS title, f. userid AS keyval, 'user' AS sourcetable, IFNULL( f. lastpost, f. joindate) AS lastactivity, f. posts as activity, 'Member' AS type, 0 AS noUnsubscribe
FROM
USER AS f
INNER JOIN userlist AS ul ON ul. relationid = f. userid
AND ul. userid = 15965
WHERE
ul. type = 'f'
AND ul. aq = 'yes'
ORDER BY
title ASC
LIMIT
100
CREATE DATABASE org235_percona345 COLLATE 'utf8_general_ci'
CREATE DATABASE org235_percona345 COLLATE 'utf8_general_ci'
insert into abtemp.coxed select foo.bar from foo
INSERT into abtemp. coxed
SELECT
foo. bar
FROM
foo
insert into foo(a, b, c) value(2, 4, 5)
INSERT into foo( a, b, c) value( 2, 4, 5)
insert into foo(a, b, c) values(2, 4, 5)
INSERT into foo( a, b, c)
VALUES(
2, 4, 5)
insert into foo(a, b, c) values(2, 4, 5) , (2,4,5)
INSERT into foo( a, b, c)
VALUES(
2, 4, 5),
(2, 4, 5)
insert into foo values (1, '(2)', 'This is a trick: ). More values.', 4)
INSERT into foo
VALUES
(1, '(2)',
'This is a trick: ). More values.',
4)
insert into tb values (1)
INSERT into tb
VALUES
(1)
INSERT INTO t (ts) VALUES ('()', '\(', '\)')
INSERT INTO t (ts)
VALUES
(
'()', '\(', '\)')
INSERT INTO t (ts) VALUES (NOW())
INSERT INTO t (ts)
VALUES
(
NOW()
)
INSERT INTO t () VALUES ()
INSERT INTO t (
)
VALUES
(
)
insert into t values (1), (2), (3)
on duplicate key update query_count=1
INSERT into t
VALUES
(1),
(2),
(3) on duplicate key
UPDATE
query_count= 1
insert into t values (1) on duplicate key update query_count=COALESCE(query_count, 0) + VALUES(query_count)
INSERT into t
VALUES
(1) on duplicate key
UPDATE
query_count= COALESCE( query_count, 0) +
VALUES(
query_count)
LOAD DATA INFILE '/tmp/foo.txt' INTO db.tbl
LOAD DATA INFILE '/tmp/foo.txt' INTO db. tbl
select 0e0, +6e-30, -6.00 from foo where a = 5.5 or b=0.5 or c=.5
SELECT
0e0, + 6e- 30, - 6.00
FROM
foo
WHERE
a = 5.5
OR b= 0.5
OR c=.5
select 0x0, x'123', 0b1010, b'10101' from foo
SELECT
0x0, x' 123',
0b1010, b' 10101'
FROM
foo
select 123_foo from 123_foo
SELECT
123_foo
FROM
123_foo
select 123foo from 123foo
SELECT
123foo
FROM
123foo
SELECT 1 AS one FROM calls USE INDEX(index_name)
SELECT
1 AS one
FROM
calls USE INDEX( index_name)
SELECT /*!40001 SQL_NO_CACHE */ * FROM `film`
SELECT
/*!40001 SQL_NO_CACHE
*/ *
FROM
`film`
SELECT 'a' 'b' 'c' 'd' FROM kamil
SELECT
'a' 'b' 'c' 'd'
FROM
kamil
SELECT BENCHMARK(100000000, pow(rand(), rand())), 1 FROM `-hj-7d6-shdj5-7jd-kf-g988h-`.`-aaahj-7d6-shdj5-7&^%$jd-kf-g988h-9+4-5*6ab-`
SELECT
BENCHMARK( 100000000, POW( RAND(
),
RAND(
)
)
),
1
FROM
`-hj-7d6-shdj5-7jd-kf-g988h-`.`-aaahj-7d6-shdj5-7&^%$jd-kf-g988h-9+4-5*6ab-`
SELECT c FROM org235.t WHERE id=0xdeadbeaf
SELECT
c
FROM
org235. t
WHERE
id= 0xdeadbeaf
select c from t where i=1 order by c asc
SELECT
c
FROM
t
WHERE
i= 1
ORDER BY
c asc
SELECT c FROM t WHERE id=0xdeadbeaf
SELECT
c
FROM
t
WHERE
id= 0xdeadbeaf
SELECT c FROM t WHERE id=1
SELECT
c
FROM
t
WHERE
id= 1
select `col` from `table-1` where `id` = 5
SELECT
`col`
FROM
`table-1`
WHERE
`id` = 5
SELECT `db`.*, (CASE WHEN (`date_start` <= '2014-09-10 09:17:59' AND `date_end` >= '2014-09-10 09:17:59') THEN 'open' WHEN (`date_start` > '2014-09-10 09:17:59' AND `date_end` > '2014-09-10 09:17:59') THEN 'tbd' ELSE 'none' END) AS `status` FROM `foo` AS `db` WHERE (a_b in ('1', '10101'))
SELECT
`db`.*,
(CASE WHEN (`date_start` <= '2014-09-10 09:17:59'
AND `date_end` >= '2014-09-10 09:17:59'
) THEN 'open' WHEN (`date_start` > '2014-09-10 09:17:59'
AND `date_end` > '2014-09-10 09:17:59'
) THEN 'tbd' ELSE 'none' END) AS `status`
FROM
`foo` AS `db`
WHERE
(a_b in (
'1', '10101')
)
select field from `-master-db-1`.`-table-1-` order by id, ?;
SELECT
FIELD
FROM
`-master-db-1`.`-table-1-`
ORDER BY
id, ?;
select foo
SELECT
foo
select foo_1 from foo_2_3
SELECT
foo_1
FROM
foo_2_3
select foo -- bar
SELECT
foo -- bar
select foo-- bar
,foo
SELECT
foo- - bar
,
foo
select '\\' from foo
SELECT
'\\'
FROM
foo
select * from foo limit 5
SELECT
*
FROM
foo
LIMIT
5
select * from foo limit 5, 10
SELECT
*
FROM
foo
LIMIT
5, 10
select * from foo limit 5 offset 10
SELECT
*
FROM
foo
LIMIT
5 offset 10
SELECT * from foo where a = 5
SELECT
*
FROM
foo
WHERE
a = 5
select * from foo where a in (5) and b in (5, 8,9 ,9 , 10)
SELECT
*
FROM
foo
WHERE
a in (5)
AND b in (5, 8, 9,
9,
10)
SELECT '' '' '' FROM kamil
SELECT
'' '' ''
FROM
kamil
select * from
foo where a = 5
SELECT
*
FROM
foo
WHERE
a = 5
SELECT * FROM prices.rt_5min where id=1
SELECT
*
FROM
prices. rt_5min
WHERE
id= 1
SELECT * FROM table WHERE field = 'value' /*arbitrary/31*/
SELECT
*
FROM
table
WHERE
FIELD = 'value'
/*arbitrary/31
*/
SELECT * FROM table WHERE field = 'value' /*arbitrary31*/
SELECT
*
FROM
table
WHERE
FIELD = 'value'
/*arbitrary31
*/
SELECT * FROM t WHERE 1=1 AND id=1
SELECT
*
FROM
t
WHERE
1= 1
AND id= 1
select * from t where (base.nid IN ('1412', '1410', '1411'))
SELECT
*
FROM
t
WHERE
(base. nid IN (
'1412', '1410', '1411')
)
select * from t where i=1 order by
a, b ASC, d DESC,
e asc
SELECT
*
FROM
t
WHERE
i= 1 order by
a, b ASC, d DESC, e asc
select * from t where i=1 order by a, b ASC, d DESC, e asc
SELECT
*
FROM
t
WHERE
i= 1
ORDER BY
a, b ASC, d DESC, e asc
select 'hello'
SELECT
'hello'
select 'hello', '
hello
', "hello", '\'' from foo
SELECT
'hello',
'
hello
',
"hello",
'\''
FROM
foo
SELECT ID, name, parent, type FROM posts WHERE _name IN ('perf','caching') AND (type = 'page' OR type = 'attachment')
SELECT
ID, name, parent, type
FROM
posts
WHERE
_name IN (
'perf', 'caching')
AND (type = 'page'
OR type = 'attachment'
)
SELECT name, value FROM variable
SELECT
name, value
FROM
variable
select
-- bar
foo
SELECT
-- bar
foo
select null, 5.001, 5001. from foo
SELECT
null, 5.001, 5001.
FROM
foo
select sleep(2) from test.n
SELECT
SLEEP( 2)
FROM
test. n
SELECT t FROM field WHERE (entity_type = 'node') AND (entity_id IN ('609')) AND (language IN ('und')) AND (deleted = '0') ORDER BY delta ASC
SELECT
t
FROM
FIELD
WHERE
(
entity_type = 'node')
AND (entity_id IN (
'609')
)
AND (language IN (
'und')
)
AND (
deleted = '0')
ORDER BY
delta ASC
select t.table_schema,t.table_name,engine from information_schema.tables t inner join information_schema.columns c on t.table_schema=c.table_schema and t.table_name=c.table_name group by t.table_schema,t.table_name having sum(if(column_key in ('PRI','UNI'),1,0))=0
SELECT
t. table_schema, t. table_name, engine
FROM
information_schema. tables t
INNER JOIN information_schema. columns c on t. table_schema= c. table_schema
AND t. table_name= c. table_name
GROUP BY
t. table_schema, t. table_name
HAVING
SUM( IF( column_key in (
'PRI', 'UNI'),
1, 0)
)= 0
/* -- S++ SU ABORTABLE -- spd_user: rspadim */SELECT SQL_SMALL_RESULT SQL_CACHE DISTINCT centro_atividade FROM est_dia WHERE unidade_id=1001 AND item_id=67 AND item_id_red=573
/* -- S++ SU ABORTABLE -- spd_user: rspadim
*/
SELECT
SQL_SMALL_RESULT SQL_CACHE DISTINCT centro_atividade
FROM
est_dia
WHERE
unidade_id= 1001
AND item_id= 67
AND item_id_red= 573
UPDATE groups_search SET charter = ' -------3\'\' XXXXXXXXX.\n \n -----------------------------------------------------', show_in_list = 'Y' WHERE group_id='aaaaaaaa'
UPDATE
groups_search
SET
charter = ' -------3\'\' XXXXXXXXX.\n \n -----------------------------------------------------',
show_in_list = 'Y'
WHERE
group_id= 'aaaaaaaa'
use `foo`
use `foo`
select sourcetable, if(f.lastcontent = ?, f.lastupdate, f.lastcontent) as lastactivity, f.totalcount as activity, type.class as type, (f.nodeoptions & ?) as nounsubscribe from node as f inner join contenttype as type on type.contenttypeid = f.contenttypeid inner join subscribed as sd on sd.did = f.nodeid and sd.userid = ? union all select f.name as title, f.userid as keyval, ? as sourcetable, ifnull(f.lastpost, f.joindate) as lastactivity, f.posts as activity, ? as type, ? as nounsubscribe from user as f inner join userlist as ul on ul.relationid = f.userid and ul.userid = ? where ul.type = ? and ul.aq = ? order by title limit ?
SELECT
sourcetable, IF( f. lastcontent = ?, f. lastupdate, f. lastcontent) as lastactivity, f. totalcount as activity, type. class as type, (f. nodeoptions & ?) as nounsubscribe
FROM
node as f
INNER JOIN contenttype as type on type. contenttypeid = f. contenttypeid
INNER JOIN subscribed as sd on sd. did = f. nodeid
AND sd. userid = ?
UNION ALL
SELECT
f. name as title, f. userid as keyval, ? as sourcetable, IFNULL( f. lastpost, f. joindate) as lastactivity, f. posts as activity, ? as type, ? as nounsubscribe
FROM
USER as f
INNER JOIN userlist as ul on ul. relationid = f. userid
AND ul. userid = ?
WHERE
ul. type = ?
AND ul. aq = ?
ORDER BY
title
LIMIT
?
CREATE INDEX part_of_name ON customer (name(10));
CREATE INDEX part_of_name ON customer (
name( 10));
alter table `sakila`.`t1` add index `idx_col`(`col`)
ALTER TABLE
`sakila`.`t1`
ADD
index `idx_col` (
`col`)
alter table `sakila`.`t1` add UNIQUE index `idx_col`(`col`)
ALTER TABLE
`sakila`.`t1`
ADD
UNIQUE index `idx_col` (
`col`)
alter table `sakila`.`t1` add index `idx_ID`(`ID`)
ALTER TABLE
`sakila`.`t1`
ADD
index `idx_ID` (
`ID`)
ALTER TABLE t2 DROP COLUMN c, DROP COLUMN d;
ALTER TABLE
t2
DROP
COLUMN c,
DROP
COLUMN d;
ALTER TABLE T2 ADD COLUMN C int;
ALTER TABLE
T2
ADD
COLUMN C int;
ALTER TABLE T2 ADD COLUMN D int FIRST;
ALTER TABLE
T2
ADD
COLUMN D int FIRST;
ALTER TABLE T2 ADD COLUMN E int AFTER D;
ALTER TABLE
T2
ADD
COLUMN E int
AFTER
D;
ALTER TABLE t1 RENAME COLUMN a TO b
ALTER TABLE
t1 RENAME COLUMN a TO b
ALTER TABLE t1 RENAME INDEX idx_a TO idx_b
ALTER TABLE
t1 RENAME INDEX idx_a TO idx_b
ALTER TABLE t1 RENAME KEY idx_a TO idx_b
ALTER TABLE
t1 RENAME KEY idx_a TO idx_b
ALTER TABLE db.old_table RENAME new_table;
ALTER TABLE
db. old_table RENAME new_table;
ALTER TABLE old_table RENAME TO new_table;
ALTER TABLE
old_table RENAME TO new_table;
ALTER TABLE old_table RENAME AS new_table;
ALTER TABLE
old_table RENAME AS new_table;
ALTER TABLE t1 MODIFY col1 BIGINT UNSIGNED DEFAULT 1 COMMENT 'my column';
ALTER TABLE
t1 MODIFY col1 BIGINT UNSIGNED DEFAULT 1 COMMENT 'my column';
ALTER TABLE t1 CHANGE b a INT NOT NULL;
ALTER TABLE
t1 CHANGE b a INT NOT NULL;
/*!40000 select 1*/;
/*!40000 select 1
*/;
SELECT * FROM film WHERE length = 86;
SELECT
*
FROM
film
WHERE
LENGTH = 86;
SELECT * FROM film WHERE length IS NULL;
SELECT
*
FROM
film
WHERE
LENGTH IS NULL;
SELECT * FROM film HAVING title = 'abc';
SELECT
*
FROM
film
HAVING
title = 'abc';
SELECT * FROM sakila.film WHERE length >= 60;
SELECT
*
FROM
sakila. film
WHERE
LENGTH >= 60;
SELECT * FROM sakila.film WHERE length >= '60';
SELECT
*
FROM
sakila. film
WHERE
LENGTH >= '60';
SELECT * FROM film WHERE length BETWEEN 60 AND 84;
SELECT
*
FROM
film
WHERE
LENGTH BETWEEN 60
AND 84;
SELECT * FROM film WHERE title LIKE 'AIR%';
SELECT
*
FROM
film
WHERE
title LIKE 'AIR%';
SELECT * FROM film WHERE title IS NOT NULL;
SELECT
*
FROM
film
WHERE
title IS NOT NULL;
SELECT * FROM film WHERE length = 114 and title = 'ALABAMA DEVIL';
SELECT
*
FROM
film
WHERE
LENGTH = 114
AND title = 'ALABAMA DEVIL';
SELECT * FROM film WHERE length > 100 and title = 'ALABAMA DEVIL';
SELECT
*
FROM
film
WHERE
LENGTH > 100
AND title = 'ALABAMA DEVIL';
SELECT * FROM film WHERE length > 100 and language_id < 10 and title = 'xyz';
SELECT
*
FROM
film
WHERE
LENGTH > 100
AND language_id < 10
AND title = 'xyz';
SELECT * FROM film WHERE length > 100 and language_id < 10;
SELECT
*
FROM
film
WHERE
LENGTH > 100
AND language_id < 10;
SELECT release_year, sum(length) FROM film WHERE length = 123 AND language_id = 1 GROUP BY release_year;
SELECT
release_year, SUM( LENGTH)
FROM
film
WHERE
LENGTH = 123
AND language_id = 1
GROUP BY
release_year;
SELECT release_year, sum(length) FROM film WHERE length >= 123 GROUP BY release_year;
SELECT
release_year, SUM( LENGTH)
FROM
film
WHERE
LENGTH >= 123
GROUP BY
release_year;
SELECT release_year, language_id, sum(length) FROM film GROUP BY release_year, language_id;
SELECT
release_year, language_id, SUM( LENGTH)
FROM
film
GROUP BY
release_year, language_id;
SELECT release_year, sum(length) FROM film WHERE length = 123 GROUP BY release_year,(length+language_id);
SELECT
release_year, SUM( LENGTH)
FROM
film
WHERE
LENGTH = 123
GROUP BY
release_year, (LENGTH+ language_id);
SELECT release_year, sum(film_id) FROM film GROUP BY release_year;
SELECT
release_year, SUM( film_id)
FROM
film
GROUP BY
release_year;
SELECT * FROM address GROUP BY address,district;
SELECT
*
FROM
address
GROUP BY
address, district;
SELECT title FROM film WHERE ABS(language_id) = 3 GROUP BY title;
SELECT
title
FROM
film
WHERE
ABS( language_id) = 3
GROUP BY
title;
SELECT language_id FROM film WHERE length = 123 GROUP BY release_year ORDER BY language_id;
SELECT
language_id
FROM
film
WHERE
LENGTH = 123
GROUP BY
release_year
ORDER BY
language_id;
SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year;
SELECT
release_year
FROM
film
WHERE
LENGTH = 123
GROUP BY
release_year
ORDER BY
release_year;
SELECT * FROM film WHERE length = 123 ORDER BY release_year ASC, language_id DESC;
SELECT
*
FROM
film
WHERE
LENGTH = 123
ORDER BY
release_year ASC, language_id DESC;
SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year LIMIT 10;
SELECT
release_year
FROM
film
WHERE
LENGTH = 123
GROUP BY
release_year
ORDER BY
release_year
LIMIT
10;
SELECT * FROM film WHERE length = 123 ORDER BY release_year LIMIT 10;
SELECT
*
FROM
film
WHERE
LENGTH = 123
ORDER BY
release_year
LIMIT
10;
SELECT * FROM film ORDER BY release_year LIMIT 10;
SELECT
*
FROM
film
ORDER BY
release_year
LIMIT
10;
SELECT film_id FROM film ORDER BY release_year LIMIT 10;
SELECT
film_id
FROM
film
ORDER BY
release_year
LIMIT
10;
SELECT * FROM film WHERE length > 100 ORDER BY length LIMIT 10;
SELECT
*
FROM
film
WHERE
LENGTH > 100
ORDER BY
LENGTH
LIMIT
10;
SELECT * FROM film WHERE length < 100 ORDER BY length LIMIT 10;
SELECT
*
FROM
film
WHERE
LENGTH < 100
ORDER BY
LENGTH
LIMIT
10;
SELECT * FROM customer WHERE address_id in (224,510) ORDER BY last_name;
SELECT
*
FROM
customer
WHERE
address_id in (224, 510)
ORDER BY
last_name;
SELECT * FROM film WHERE release_year = 2016 AND length != 1 ORDER BY title;
SELECT
*
FROM
film
WHERE
release_year = 2016
AND LENGTH != 1
ORDER BY
title;
SELECT title FROM film WHERE release_year = 1995;
SELECT
title
FROM
film
WHERE
release_year = 1995;
SELECT title, replacement_cost FROM film WHERE language_id = 5 AND length = 70;
SELECT
title, replacement_cost
FROM
film
WHERE
language_id = 5
AND LENGTH = 70;
SELECT title FROM film WHERE language_id > 5 AND length > 70;
SELECT
title
FROM
film
WHERE
language_id > 5
AND LENGTH > 70;
SELECT * FROM film WHERE length = 100 and title = 'xyz' ORDER BY release_year;
SELECT
*
FROM
film
WHERE
LENGTH = 100
AND title = 'xyz'
ORDER BY
release_year;
SELECT * FROM film WHERE length > 100 and title = 'xyz' ORDER BY release_year;
SELECT
*
FROM
film
WHERE
LENGTH > 100
AND title = 'xyz'
ORDER BY
release_year;
SELECT * FROM film WHERE length > 100 ORDER BY release_year;
SELECT
*
FROM
film
WHERE
LENGTH > 100
ORDER BY
release_year;
SELECT * FROM city a INNER JOIN country b ON a.country_id=b.country_id;
SELECT
*
FROM
city a
INNER JOIN country b ON a. country_id= b. country_id;
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id;
SELECT
*
FROM
city a
LEFT JOIN country b ON a. country_id= b. country_id;
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;
SELECT
*
FROM
city a
RIGHT JOIN country b ON a. country_id= b. country_id;
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;
SELECT
*
FROM
city a
LEFT JOIN country b ON a. country_id= b. country_id
WHERE
b. last_update IS NULL;
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL;
SELECT
*
FROM
city a
RIGHT JOIN country b ON a. country_id= b. country_id
WHERE
a. last_update IS NULL;
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id UNION SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;
SELECT
*
FROM
city a
LEFT JOIN country b ON a. country_id= b. country_id
UNION
SELECT
*
FROM
city a
RIGHT JOIN country b ON a. country_id= b. country_id;
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL UNION SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;
SELECT
*
FROM
city a
RIGHT JOIN country b ON a. country_id= b. country_id
WHERE
a. last_update IS NULL
UNION
SELECT
*
FROM
city a
LEFT JOIN country b ON a. country_id= b. country_id
WHERE
b. last_update IS NULL;
SELECT country_id, last_update FROM city NATURAL JOIN country;
SELECT
country_id, last_update
FROM
city NATURAL
JOIN country;
SELECT country_id, last_update FROM city NATURAL LEFT JOIN country;
SELECT
country_id, last_update
FROM
city NATURAL
LEFT JOIN country;
SELECT country_id, last_update FROM city NATURAL RIGHT JOIN country;
SELECT
country_id, last_update
FROM
city NATURAL
RIGHT JOIN country;
SELECT a.country_id, a.last_update FROM city a STRAIGHT_JOIN country b ON a.country_id=b.country_id;
SELECT
a. country_id, a. last_update
FROM
city a STRAIGHT_JOIN country b ON a. country_id= b. country_id;
SELECT a.address, a.postal_code FROM sakila.address a WHERE a.city_id IN (SELECT c.city_id FROM sakila.city c);
SELECT
a. address, a. postal_code
FROM
sakila. address a
WHERE
a. city_id IN (
SELECT
c. city_id
FROM
sakila. city c);
SELECT city FROM( SELECT city_id FROM city WHERE city = "A Corua (La Corua)" ORDER BY last_update DESC LIMIT 50, 10) I JOIN city ON (I.city_id = city.city_id) JOIN country ON (country.country_id = city.country_id) ORDER BY city DESC;
SELECT
city
FROM(
SELECT
city_id
FROM
city
WHERE
city = "A Corua (La Corua)"
ORDER BY
last_update DESC
LIMIT
50, 10) I
JOIN city ON (I. city_id = city. city_id)
JOIN country ON (country. country_id = city. country_id)
ORDER BY
city DESC;
DELETE city, country FROM city INNER JOIN country using (country_id) WHERE city.city_id = 1;
DELETE city, country
FROM
city
INNER JOIN country using (country_id)
WHERE
city. city_id = 1;
DELETE city FROM city LEFT JOIN country ON city.country_id = country.country_id WHERE country.country IS NULL;
DELETE city
FROM
city
LEFT JOIN country ON city. country_id = country. country_id
WHERE
country. country IS NULL;
DELETE a1, a2 FROM city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;
DELETE a1, a2
FROM
city AS a1
INNER JOIN country AS a2
WHERE
a1. country_id= a2. country_id;
DELETE FROM a1, a2 USING city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;
DELETE FROM
a1, a2 USING city AS a1
INNER JOIN country AS a2
WHERE
a1. country_id= a2. country_id;
DELETE FROM film WHERE length > 100;
DELETE FROM
film
WHERE
LENGTH > 100;
UPDATE city INNER JOIN country USING(country_id) SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;
UPDATE
city
INNER JOIN country USING( country_id)
SET
city. city = 'Abha',
city. last_update = '2006-02-15 04:45:25',
country. country = 'Afghanistan'
WHERE
city. city_id= 10;
UPDATE city INNER JOIN country ON city.country_id = country.country_id INNER JOIN address ON city.city_id = address.city_id SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;
UPDATE
city
INNER JOIN country ON city. country_id = country. country_id
INNER JOIN address ON city. city_id = address. city_id
SET
city. city = 'Abha',
city. last_update = '2006-02-15 04:45:25',
country. country = 'Afghanistan'
WHERE
city. city_id= 10;
UPDATE city, country SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.country_id = country.country_id AND city.city_id=10;
UPDATE
city, country
SET
city. city = 'Abha',
city. last_update = '2006-02-15 04:45:25',
country. country = 'Afghanistan'
WHERE
city. country_id = country. country_id
AND city. city_id= 10;
UPDATE film SET length = 10 WHERE language_id = 20;
UPDATE
film
SET
LENGTH = 10
WHERE
language_id = 20;
INSERT INTO city (country_id) SELECT country_id FROM country;
INSERT INTO city (country_id)
SELECT
country_id
FROM
country;
INSERT INTO city (country_id) VALUES (1),(2),(3);
INSERT INTO city (country_id)
VALUES
(1),
(2),
(3);
INSERT INTO city (country_id) VALUES (10);
INSERT INTO city (country_id)
VALUES
(10);
INSERT INTO city (country_id) SELECT 10 FROM DUAL;
INSERT INTO city (country_id)
SELECT
10
FROM
DUAL;
REPLACE INTO city (country_id) SELECT country_id FROM country;
REPLACE INTO city (country_id)
SELECT
country_id
FROM
country;
REPLACE INTO city (country_id) VALUES (1),(2),(3);
REPLACE INTO city (country_id)
VALUES
(1),
(2),
(3);
REPLACE INTO city (country_id) VALUES (10);
REPLACE INTO city (country_id)
VALUES
(10);
REPLACE INTO city (country_id) SELECT 10 FROM DUAL;
REPLACE INTO city (country_id)
SELECT
10
FROM
DUAL;
SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film;
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film;
SELECT * FROM film WHERE language_id = (SELECT language_id FROM language LIMIT 1);
SELECT
*
FROM
film
WHERE
language_id = (
SELECT
language_id
FROM
language
LIMIT
1);
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;
SELECT
*
FROM
city i
LEFT JOIN country o ON i. city_id= o. country_id
UNION
SELECT
*
FROM
city i
RIGHT JOIN country o ON i. city_id= o. country_id;
SELECT * FROM (SELECT * FROM actor WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE') t WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE' GROUP BY first_name;
SELECT
*
FROM
(
SELECT
*
FROM
actor
WHERE
last_update= '2006-02-15 04:34:33'
AND last_name= 'CHASE'
) t
WHERE
last_update= '2006-02-15 04:34:33'
AND last_name= 'CHASE'
GROUP BY
first_name;
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;
SELECT
*
FROM
city i
LEFT JOIN country o ON i. city_id= o. country_id
UNION
SELECT
*
FROM
city i
RIGHT JOIN country o ON i. city_id= o. country_id;
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id WHERE o.country_id is null union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id WHERE i.city_id is null;
SELECT
*
FROM
city i
LEFT JOIN country o ON i. city_id= o. country_id
WHERE
o. country_id is null
UNION
SELECT
*
FROM
city i
RIGHT JOIN country o ON i. city_id= o. country_id
WHERE
i. city_id is null;
SELECT first_name,last_name,email FROM customer STRAIGHT_JOIN address ON customer.address_id=address.address_id;
SELECT
first_name, last_name, email
FROM
customer STRAIGHT_JOIN address ON customer. address_id= address. address_id;
SELECT ID,name FROM (SELECT address FROM customer_list WHERE SID=1 order by phone limit 50,10) a JOIN customer_list l ON (a.address=l.address) JOIN city c ON (c.city=l.city) order by phone desc;
SELECT
ID, name
FROM
(
SELECT
address
FROM
customer_list
WHERE
SID= 1
ORDER BY
phone
LIMIT
50, 10) a
JOIN customer_list l ON (a. address= l. address)
JOIN city c ON (c. city= l. city)
ORDER BY
phone desc;
SELECT * FROM film WHERE date(last_update)='2006-02-15';
SELECT
*
FROM
film
WHERE
DATE( last_update) = '2006-02-15';
SELECT last_update FROM film GROUP BY date(last_update);
SELECT
last_update
FROM
film
GROUP BY
DATE( last_update);
SELECT last_update FROM film order by date(last_update);
SELECT
last_update
FROM
film
ORDER BY
DATE( last_update);
SELECT description FROM film WHERE description IN('NEWS','asd') GROUP BY description;
SELECT
description
FROM
film
WHERE
description IN( 'NEWS',
'asd'
)
GROUP BY
description;
alter table address add index idx_city_id(city_id);
ALTER TABLE
address
ADD
index idx_city_id( city_id);
alter table inventory add index `idx_store_film` (`store_id`,`film_id`);
ALTER TABLE
inventory
ADD
index `idx_store_film` (
`store_id`, `film_id`);
alter table inventory add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`);
ALTER TABLE
inventory
ADD
index `idx_store_film` (
`store_id`, `film_id`),
ADD
index `idx_store_film` (
`store_id`, `film_id`),
ADD
index `idx_store_film` (
`store_id`, `film_id`);
SELECT DATE_FORMAT(t.last_update, '%Y-%m-%d'), COUNT(DISTINCT (t.city)) FROM city t WHERE t.last_update > '2018-10-22 00:00:00' AND t.city LIKE '%Chrome%' AND t.city = 'eip' GROUP BY DATE_FORMAT(t.last_update, '%Y-%m-%d') ORDER BY DATE_FORMAT(t.last_update, '%Y-%m-%d');
SELECT
DATE_FORMAT( t. last_update, '%Y-%m-%d'
),
COUNT( DISTINCT (
t. city))
FROM
city t
WHERE
t. last_update > '2018-10-22 00:00:00'
AND t. city LIKE '%Chrome%'
AND t. city = 'eip'
GROUP BY
DATE_FORMAT( t. last_update, '%Y-%m-%d'
)
ORDER BY
DATE_FORMAT( t. last_update, '%Y-%m-%d'
);
create table hello.t (id int unsigned);
create table hello. t (id int unsigned);
select * from tb where data >= '';
SELECT
*
FROM
tb
WHERE
data >= '';
alter table tb alter column id drop default;
ALTER TABLE
tb alter column id
DROP
DEFAULT;
select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film where last_update > '2016-03-27 02:01:01') as d;
SELECT
maxId, minId
FROM
(
SELECT
MAX( film_id) maxId, MIN( film_id) minId
FROM
film
WHERE
last_update > '2016-03-27 02:01:01'
) as d;
select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film) as d;
SELECT
maxId, minId
FROM
(
SELECT
MAX( film_id) maxId, MIN( film_id) minId
FROM
film) as d;
select 1
================================================
FILE: ast/testdata/TestPrintPrettyStmtNode.golden
================================================
[]ast.StmtNode{
&ast.SelectStmt{
dmlNode: ast.dmlNode{
stmtNode: ast.stmtNode{
node: ast.node{text:"select 1", offset:0},
},
},
SelectStmtOpts: &ast.SelectStmtOpts{
Distinct: false,
SQLBigResult: false,
SQLBufferResult: false,
SQLCache: true,
SQLSmallResult: false,
CalcFoundRows: false,
StraightJoin: false,
Priority: 0,
TableHints: nil,
ExplicitAll: false,
},
Distinct: false,
From: (*ast.TableRefsClause)(nil),
Where: nil,
Fields: &ast.FieldList{
node: ast.node{},
Fields: {
&ast.SelectField{
node: ast.node{text:"1", offset:0},
Offset: 7,
WildCard: (*ast.WildCardField)(nil),
Expr: &driver.ValueExpr{
TexprNode: ast.exprNode{
node: ast.node{text:"", offset:7},
Type: types.FieldType{
Tp: 0x8,
Flag: 0x80,
Flen: 1,
Decimal: 0,
Charset: "binary",
Collate: "binary",
Elems: nil,
},
flag: 0x0,
},
Datum: types.Datum{
k: 0x1,
decimal: 0x0,
length: 0x0,
i: 1,
collation: "",
b: nil,
x: nil,
},
projectionOffset: -1,
},
AsName: model.CIStr{},
Auxiliary: false,
},
},
},
GroupBy: (*ast.GroupByClause)(nil),
Having: (*ast.HavingClause)(nil),
WindowSpecs: nil,
OrderBy: (*ast.OrderByClause)(nil),
Limit: (*ast.Limit)(nil),
LockInfo: (*ast.SelectLockInfo)(nil),
TableHints: nil,
IsInBraces: false,
WithBeforeBraces: false,
QueryBlockOffset: 0,
SelectIntoOpt: (*ast.SelectIntoOption)(nil),
AfterSetOperator: (*ast.SetOprType)(nil),
Kind: 0x0,
Lists: nil,
With: (*ast.WithClause)(nil),
},
}
================================================
FILE: ast/testdata/TestPrintPrettyVitessStmtNode.golden
================================================
&sqlparser.Select{
Cache: "",
Comments: nil,
Distinct: "",
Hints: "",
SelectExprs: {
&sqlparser.AliasedExpr{
Expr: &sqlparser.SQLVal{
Type: 1,
Val: {0x31},
},
As: sqlparser.ColIdent{},
},
},
From: {
&sqlparser.AliasedTableExpr{
Expr: sqlparser.TableName{
Name: sqlparser.TableIdent{v:"dual"},
Qualifier: sqlparser.TableIdent{},
},
Partitions: nil,
As: sqlparser.TableIdent{},
Hints: (*sqlparser.IndexHints)(nil),
},
},
Where: (*sqlparser.Where)(nil),
GroupBy: nil,
Having: (*sqlparser.Where)(nil),
OrderBy: nil,
Limit: (*sqlparser.Limit)(nil),
Lock: "",
}
================================================
FILE: ast/testdata/TestQueryType.golden
================================================
SELECT
SELECT
SELECT
GRANT
REVOKE
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
DELETE
DELETE
DELETE
DELETE
DELETE
UPDATE
UPDATE
UPDATE
UPDATE
INSERT
INSERT
INSERT
INSERT
REPLACE
REPLACE
REPLACE
REPLACE
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
SELECT
ALTER
ALTER
ALTER
SELECT
CREATE
SELECT
ALTER
SELECT
SELECT
================================================
FILE: ast/testdata/TestSchemaMetaInfo.golden
================================================
use world_x;
[`world_x`.`dual`]
select 1;
[]
syntax error case
[]
select * from ta join tb using (id)
[`sakila`.`ta` `sakila`.`tb`]
select * from ta, tb limit 1
[`sakila`.`ta` `sakila`.`tb`]
drop table tb
[`sakila`.`tb`]
drop table db.tb
[`db`.`tb`]
drop database db
[`db`.`dual`]
create database db
[`db`.`dual`]
create index idx_col on tbl (col)
[`sakila`.`tbl`]
DROP INDEX idx_col on tbl
[`sakila`.`tbl`]
SELECT * FROM film WHERE length = 86;
[`sakila`.`film`]
SELECT * FROM film WHERE length IS NULL;
[`sakila`.`film`]
SELECT * FROM film HAVING title = 'abc';
[`sakila`.`film`]
SELECT * FROM sakila.film WHERE length >= 60;
[`sakila`.`film`]
SELECT * FROM sakila.film WHERE length >= '60';
[`sakila`.`film`]
SELECT * FROM film WHERE length BETWEEN 60 AND 84;
[`sakila`.`film`]
SELECT * FROM film WHERE title LIKE 'AIR%';
[`sakila`.`film`]
SELECT * FROM film WHERE title IS NOT NULL;
[`sakila`.`film`]
SELECT * FROM film WHERE length = 114 and title = 'ALABAMA DEVIL';
[`sakila`.`film`]
SELECT * FROM film WHERE length > 100 and title = 'ALABAMA DEVIL';
[`sakila`.`film`]
SELECT * FROM film WHERE length > 100 and language_id < 10 and title = 'xyz';
[`sakila`.`film`]
SELECT * FROM film WHERE length > 100 and language_id < 10;
[`sakila`.`film`]
SELECT release_year, sum(length) FROM film WHERE length = 123 AND language_id = 1 GROUP BY release_year;
[`sakila`.`film`]
SELECT release_year, sum(length) FROM film WHERE length >= 123 GROUP BY release_year;
[`sakila`.`film`]
SELECT release_year, language_id, sum(length) FROM film GROUP BY release_year, language_id;
[`sakila`.`film`]
SELECT release_year, sum(length) FROM film WHERE length = 123 GROUP BY release_year,(length+language_id);
[`sakila`.`film`]
SELECT release_year, sum(film_id) FROM film GROUP BY release_year;
[`sakila`.`film`]
SELECT * FROM address GROUP BY address,district;
[`sakila`.`address`]
SELECT title FROM film WHERE ABS(language_id) = 3 GROUP BY title;
[`sakila`.`film`]
SELECT language_id FROM film WHERE length = 123 GROUP BY release_year ORDER BY language_id;
[`sakila`.`film`]
SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year;
[`sakila`.`film`]
SELECT * FROM film WHERE length = 123 ORDER BY release_year ASC, language_id DESC;
[`sakila`.`film`]
SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year LIMIT 10;
[`sakila`.`film`]
SELECT * FROM film WHERE length = 123 ORDER BY release_year LIMIT 10;
[`sakila`.`film`]
SELECT * FROM film ORDER BY release_year LIMIT 10;
[`sakila`.`film`]
SELECT film_id FROM film ORDER BY release_year LIMIT 10;
[`sakila`.`film`]
SELECT * FROM film WHERE length > 100 ORDER BY length LIMIT 10;
[`sakila`.`film`]
SELECT * FROM film WHERE length < 100 ORDER BY length LIMIT 10;
[`sakila`.`film`]
SELECT * FROM customer WHERE address_id in (224,510) ORDER BY last_name;
[`sakila`.`customer`]
SELECT * FROM film WHERE release_year = 2016 AND length != 1 ORDER BY title;
[`sakila`.`film`]
SELECT title FROM film WHERE release_year = 1995;
[`sakila`.`film`]
SELECT title, replacement_cost FROM film WHERE language_id = 5 AND length = 70;
[`sakila`.`film`]
SELECT title FROM film WHERE language_id > 5 AND length > 70;
[`sakila`.`film`]
SELECT * FROM film WHERE length = 100 and title = 'xyz' ORDER BY release_year;
[`sakila`.`film`]
SELECT * FROM film WHERE length > 100 and title = 'xyz' ORDER BY release_year;
[`sakila`.`film`]
SELECT * FROM film WHERE length > 100 ORDER BY release_year;
[`sakila`.`film`]
SELECT * FROM city a INNER JOIN country b ON a.country_id=b.country_id;
[`sakila`.`city` `sakila`.`country`]
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id;
[`sakila`.`city` `sakila`.`country`]
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;
[`sakila`.`city` `sakila`.`country`]
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;
[`sakila`.`city` `sakila`.`country`]
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL;
[`sakila`.`city` `sakila`.`country`]
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id UNION SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;
[`sakila`.`city` `sakila`.`country`]
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL UNION SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;
[`sakila`.`city` `sakila`.`country`]
SELECT country_id, last_update FROM city NATURAL JOIN country;
[`sakila`.`city` `sakila`.`country`]
SELECT country_id, last_update FROM city NATURAL LEFT JOIN country;
[`sakila`.`city` `sakila`.`country`]
SELECT country_id, last_update FROM city NATURAL RIGHT JOIN country;
[`sakila`.`city` `sakila`.`country`]
SELECT a.country_id, a.last_update FROM city a STRAIGHT_JOIN country b ON a.country_id=b.country_id;
[`sakila`.`city` `sakila`.`country`]
SELECT a.address, a.postal_code FROM sakila.address a WHERE a.city_id IN (SELECT c.city_id FROM sakila.city c);
[`sakila`.`address` `sakila`.`city`]
SELECT city FROM( SELECT city_id FROM city WHERE city = "A Corua (La Corua)" ORDER BY last_update DESC LIMIT 50, 10) I JOIN city ON (I.city_id = city.city_id) JOIN country ON (country.country_id = city.country_id) ORDER BY city DESC;
[`sakila`.`city` `sakila`.`country`]
DELETE city, country FROM city INNER JOIN country using (country_id) WHERE city.city_id = 1;
[`sakila`.`city` `sakila`.`country`]
DELETE city FROM city LEFT JOIN country ON city.country_id = country.country_id WHERE country.country IS NULL;
[`sakila`.`city` `sakila`.`country`]
DELETE a1, a2 FROM city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;
[`sakila`.`city` `sakila`.`country`]
DELETE FROM a1, a2 USING city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;
[`sakila`.`city` `sakila`.`country`]
DELETE FROM film WHERE length > 100;
[`sakila`.`film`]
UPDATE city INNER JOIN country USING(country_id) SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;
[`sakila`.`city` `sakila`.`country`]
UPDATE city INNER JOIN country ON city.country_id = country.country_id INNER JOIN address ON city.city_id = address.city_id SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;
[`sakila`.`address` `sakila`.`city` `sakila`.`country`]
UPDATE city, country SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.country_id = country.country_id AND city.city_id=10;
[`sakila`.`city` `sakila`.`country`]
UPDATE film SET length = 10 WHERE language_id = 20;
[`sakila`.`film`]
INSERT INTO city (country_id) SELECT country_id FROM country;
[`sakila`.`city` `sakila`.`country`]
INSERT INTO city (country_id) VALUES (1),(2),(3);
[`sakila`.`city`]
INSERT INTO city (country_id) VALUES (10);
[`sakila`.`city`]
INSERT INTO city (country_id) SELECT 10 FROM DUAL;
[`sakila`.`city`]
REPLACE INTO city (country_id) SELECT country_id FROM country;
[`sakila`.`city` `sakila`.`country`]
REPLACE INTO city (country_id) VALUES (1),(2),(3);
[`sakila`.`city`]
REPLACE INTO city (country_id) VALUES (10);
[`sakila`.`city`]
REPLACE INTO city (country_id) SELECT 10 FROM DUAL;
[`sakila`.`city`]
SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film;
[`sakila`.`film`]
SELECT * FROM film WHERE language_id = (SELECT language_id FROM language LIMIT 1);
[`sakila`.`film` `sakila`.`language`]
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;
[`sakila`.`city` `sakila`.`country`]
SELECT * FROM (SELECT * FROM actor WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE') t WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE' GROUP BY first_name;
[`sakila`.`actor`]
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;
[`sakila`.`city` `sakila`.`country`]
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id WHERE o.country_id is null union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id WHERE i.city_id is null;
[`sakila`.`city` `sakila`.`country`]
SELECT first_name,last_name,email FROM customer STRAIGHT_JOIN address ON customer.address_id=address.address_id;
[`sakila`.`address` `sakila`.`customer`]
SELECT ID,name FROM (SELECT address FROM customer_list WHERE SID=1 order by phone limit 50,10) a JOIN customer_list l ON (a.address=l.address) JOIN city c ON (c.city=l.city) order by phone desc;
[`sakila`.`city` `sakila`.`customer_list`]
SELECT * FROM film WHERE date(last_update)='2006-02-15';
[`sakila`.`film`]
SELECT last_update FROM film GROUP BY date(last_update);
[`sakila`.`film`]
SELECT last_update FROM film order by date(last_update);
[`sakila`.`film`]
SELECT description FROM film WHERE description IN('NEWS','asd') GROUP BY description;
[`sakila`.`film`]
alter table address add index idx_city_id(city_id);
[`sakila`.`address`]
alter table inventory add index `idx_store_film` (`store_id`,`film_id`);
[`sakila`.`inventory`]
alter table inventory add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`);
[`sakila`.`inventory`]
SELECT DATE_FORMAT(t.last_update, '%Y-%m-%d'), COUNT(DISTINCT (t.city)) FROM city t WHERE t.last_update > '2018-10-22 00:00:00' AND t.city LIKE '%Chrome%' AND t.city = 'eip' GROUP BY DATE_FORMAT(t.last_update, '%Y-%m-%d') ORDER BY DATE_FORMAT(t.last_update, '%Y-%m-%d');
[`sakila`.`city`]
create table hello.t (id int unsigned);
[`hello`.`t`]
select * from tb where data >= '';
[`sakila`.`tb`]
alter table tb alter column id drop default;
[`sakila`.`tb`]
select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film where last_update > '2016-03-27 02:01:01') as d;
[`sakila`.`film`]
select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film) as d;
[`sakila`.`film`]
================================================
FILE: ast/testdata/TestSplitStatement.golden
================================================
0 select * from test;
1 select 'asd;fas', col from test;
2 -- select * from test;hello
3 #select * from test;hello
4 select * /*comment*/from test;
5 select * /*comment;*/from test;
6 select * /*comment
;*/
from test;
7 select * from test
8 /*comment*/
9 /*comment*/;
10 --
11 -- comment
12 # comment
13 select
*
-- comment
from tb
where col = 1
14 select
* --
from tb
where col = 1
15 select
* #
from tb
where col = 1
16 select
*
--
from tb
where col = 1
17 select * from
-- comment
tb;
18
-- comment
19 INSERT /*+ SET_VAR(foreign_key_checks=OFF) */ INTO t2 VALUES(2);
20 select /*!50000 1,*/ 1;
21 UPDATE xxx SET c1=' LOGGER.error(""); }' WHERE id = 2 ;
22 UPDATE `xxx` SET aaa='a;' WHERE `id` = 15;
23 UPDATE `xxx` SET aaa='a -- b' WHERE `id` = 15;
0 select * from test\G
1 select 'hello\Gworld', col from test\G
2 -- select * from test\Ghello
3 #select * from test\Ghello
4 select * /*comment*/from test\G
5 select * /*comment;*/from test\G
6 select * /*comment
\\G*/
from test\\G
================================================
FILE: ast/testdata/TestStmtNode2JSON.golden
================================================
[
{
"text": "select 1",
"offset": 0,
"SQLBigResult": false,
"SQLBufferResult": false,
"SQLCache": true,
"SQLSmallResult": false,
"CalcFoundRows": false,
"StraightJoin": false,
"Priority": 0,
"ExplicitAll": false,
"Distinct": false,
"From": null,
"Where": null,
"Fields": {
"text": "",
"offset": 0,
"Fields": [
{
"text": "1",
"offset": 0,
"Offset": 7,
"WildCard": null,
"Expr": {
"text": "",
"offset": 7,
"Type": {
"Tp": 8,
"Flag": 128,
"Flen": 1,
"Decimal": 0,
"Charset": "binary",
"Collate": "binary",
"Elems": null
},
"flag": 0,
"k": 1,
"decimal": 0,
"length": 0,
"i": 1,
"collation": "",
"b": null,
"x": null,
"projectionOffset": -1
},
"AsName": {
"O": "",
"L": ""
},
"Auxiliary": false
}
]
},
"GroupBy": null,
"Having": null,
"WindowSpecs": null,
"OrderBy": null,
"Limit": null,
"LockInfo": null,
"TableHints": null,
"IsInBraces": false,
"WithBeforeBraces": false,
"QueryBlockOffset": 0,
"SelectIntoOpt": null,
"AfterSetOperator": null,
"Kind": 0,
"Lists": null,
"With": null
}
]
================================================
FILE: ast/testdata/TestTokenize.golden
================================================
SELECT * FROM film WHERE length = 86;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 = 0} {0 0} {10 86; 0}]
SELECT * FROM film WHERE length IS NULL;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {1 IS 0} {1 NULL; 0}]
SELECT * FROM film HAVING title = 'abc';
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 HAVING 0} {1 title 0} {7 = 0} {0 0} {2 'abc' 0} {7 ; 0}]
SELECT * FROM sakila.film WHERE length >= 60;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 sakila. 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 >= 0} {0 0} {10 60; 0}]
SELECT * FROM sakila.film WHERE length >= '60';
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 sakila. 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 >= 0} {0 0} {2 '60' 0} {7 ; 0}]
SELECT * FROM film WHERE length BETWEEN 60 AND 84;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {1 BETWEEN 0} {10 60 0} {6 AND 0} {10 84; 0}]
SELECT * FROM film WHERE title LIKE 'AIR%';
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {1 title 0} {1 LIKE 0} {2 'AIR%' 0} {7 ; 0}]
SELECT * FROM film WHERE title IS NOT NULL;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {1 title 0} {1 IS 0} {1 NOT 0} {1 NULL; 0}]
SELECT * FROM film WHERE length = 114 and title = 'ALABAMA DEVIL';
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 = 0} {0 0} {10 114 0} {6 AND 0} {1 title 0} {7 = 0} {0 0} {2 'ALABAMA DEVIL' 0} {7 ; 0}]
SELECT * FROM film WHERE length > 100 and title = 'ALABAMA DEVIL';
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 > 0} {0 0} {10 100 0} {6 AND 0} {1 title 0} {7 = 0} {0 0} {2 'ALABAMA DEVIL' 0} {7 ; 0}]
SELECT * FROM film WHERE length > 100 and language_id < 10 and title = 'xyz';
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 > 0} {0 0} {10 100 0} {6 AND 0} {1 language_id 0} {7 < 0} {0 0} {10 10 0} {6 AND 0} {1 title 0} {7 = 0} {0 0} {2 'xyz' 0} {7 ; 0}]
SELECT * FROM film WHERE length > 100 and language_id < 10;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 > 0} {0 0} {10 100 0} {6 AND 0} {1 language_id 0} {7 < 0} {0 0} {10 10; 0}]
SELECT release_year, sum(length) FROM film WHERE length = 123 AND language_id = 1 GROUP BY release_year;
[{5 SELECT 0} {1 release_year, 0} {0 0} {4 SUM( 0} {4 LENGTH) 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 = 0} {0 0} {10 123 0} {6 AND 0} {1 language_id 0} {7 = 0} {0 0} {10 1 0} {5 GROUP BY 0} {1 release_year; 0}]
SELECT release_year, sum(length) FROM film WHERE length >= 123 GROUP BY release_year;
[{5 SELECT 0} {1 release_year, 0} {0 0} {4 SUM( 0} {4 LENGTH) 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 >= 0} {0 0} {10 123 0} {5 GROUP BY 0} {1 release_year; 0}]
SELECT release_year, language_id, sum(length) FROM film GROUP BY release_year, language_id;
[{5 SELECT 0} {1 release_year, 0} {0 0} {1 language_id, 0} {0 0} {4 SUM( 0} {4 LENGTH) 0} {0 0} {5 FROM 0} {1 film 0} {5 GROUP BY 0} {1 release_year, 0} {0 0} {1 language_id; 0}]
SELECT release_year, sum(length) FROM film WHERE length = 123 GROUP BY release_year,(length+language_id);
[{5 SELECT 0} {1 release_year, 0} {0 0} {4 SUM( 0} {4 LENGTH) 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 = 0} {0 0} {10 123 0} {5 GROUP BY 0} {1 release_year, 0} {7 ( 0} {4 LENGTH+ 0} {1 language_id) 0} {7 ; 0}]
SELECT release_year, sum(film_id) FROM film GROUP BY release_year;
[{5 SELECT 0} {1 release_year, 0} {0 0} {4 SUM( 0} {1 film_id) 0} {0 0} {5 FROM 0} {1 film 0} {5 GROUP BY 0} {1 release_year; 0}]
SELECT * FROM address GROUP BY address,district;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 address 0} {5 GROUP BY 0} {1 address, 0} {1 district; 0}]
SELECT title FROM film WHERE ABS(language_id) = 3 GROUP BY title;
[{5 SELECT 0} {1 title 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 ABS( 0} {1 language_id) 0} {0 0} {7 = 0} {0 0} {10 3 0} {5 GROUP BY 0} {1 title; 0}]
SELECT language_id FROM film WHERE length = 123 GROUP BY release_year ORDER BY language_id;
[{5 SELECT 0} {1 language_id 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 = 0} {0 0} {10 123 0} {5 GROUP BY 0} {1 release_year 0} {5 ORDER BY 0} {1 language_id; 0}]
SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year;
[{5 SELECT 0} {1 release_year 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 = 0} {0 0} {10 123 0} {5 GROUP BY 0} {1 release_year 0} {5 ORDER BY 0} {1 release_year; 0}]
SELECT * FROM film WHERE length = 123 ORDER BY release_year ASC, language_id DESC;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 = 0} {0 0} {10 123 0} {5 ORDER BY 0} {1 release_year 0} {1 ASC, 0} {0 0} {1 language_id 0} {1 DESC; 0}]
SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year LIMIT 10;
[{5 SELECT 0} {1 release_year 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 = 0} {0 0} {10 123 0} {5 GROUP BY 0} {1 release_year 0} {5 ORDER BY 0} {1 release_year 0} {5 LIMIT 0} {10 10; 0}]
SELECT * FROM film WHERE length = 123 ORDER BY release_year LIMIT 10;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 = 0} {0 0} {10 123 0} {5 ORDER BY 0} {1 release_year 0} {5 LIMIT 0} {10 10; 0}]
SELECT * FROM film ORDER BY release_year LIMIT 10;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 ORDER BY 0} {1 release_year 0} {5 LIMIT 0} {10 10; 0}]
SELECT film_id FROM film ORDER BY release_year LIMIT 10;
[{5 SELECT 0} {1 film_id 0} {5 FROM 0} {1 film 0} {5 ORDER BY 0} {1 release_year 0} {5 LIMIT 0} {10 10; 0}]
SELECT * FROM film WHERE length > 100 ORDER BY length LIMIT 10;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 > 0} {0 0} {10 100 0} {5 ORDER BY 0} {4 LENGTH 0} {5 LIMIT 0} {10 10; 0}]
SELECT * FROM film WHERE length < 100 ORDER BY length LIMIT 10;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 < 0} {0 0} {10 100 0} {5 ORDER BY 0} {4 LENGTH 0} {5 LIMIT 0} {10 10; 0}]
SELECT * FROM customer WHERE address_id in (224,510) ORDER BY last_name;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 customer 0} {5 WHERE 0} {1 address_id 0} {1 in 0} {7 ( 0} {10 224, 0} {10 510) 0} {0 0} {5 ORDER BY 0} {1 last_name; 0}]
SELECT * FROM film WHERE release_year = 2016 AND length != 1 ORDER BY title;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {1 release_year 0} {7 = 0} {0 0} {10 2016 0} {6 AND 0} {4 LENGTH 0} {7 != 0} {0 0} {10 1 0} {5 ORDER BY 0} {1 title; 0}]
SELECT title FROM film WHERE release_year = 1995;
[{5 SELECT 0} {1 title 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {1 release_year 0} {7 = 0} {0 0} {10 1995; 0}]
SELECT title, replacement_cost FROM film WHERE language_id = 5 AND length = 70;
[{5 SELECT 0} {1 title, 0} {0 0} {1 replacement_cost 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {1 language_id 0} {7 = 0} {0 0} {10 5 0} {6 AND 0} {4 LENGTH 0} {7 = 0} {0 0} {10 70; 0}]
SELECT title FROM film WHERE language_id > 5 AND length > 70;
[{5 SELECT 0} {1 title 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {1 language_id 0} {7 > 0} {0 0} {10 5 0} {6 AND 0} {4 LENGTH 0} {7 > 0} {0 0} {10 70; 0}]
SELECT * FROM film WHERE length = 100 and title = 'xyz' ORDER BY release_year;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 = 0} {0 0} {10 100 0} {6 AND 0} {1 title 0} {7 = 0} {0 0} {2 'xyz' 0} {0 0} {5 ORDER BY 0} {1 release_year; 0}]
SELECT * FROM film WHERE length > 100 and title = 'xyz' ORDER BY release_year;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 > 0} {0 0} {10 100 0} {6 AND 0} {1 title 0} {7 = 0} {0 0} {2 'xyz' 0} {0 0} {5 ORDER BY 0} {1 release_year; 0}]
SELECT * FROM film WHERE length > 100 ORDER BY release_year;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 > 0} {0 0} {10 100 0} {5 ORDER BY 0} {1 release_year; 0}]
SELECT * FROM city a INNER JOIN country b ON a.country_id=b.country_id;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 a 0} {6 INNER JOIN 0} {1 country 0} {1 b 0} {1 ON 0} {1 a. 0} {1 country_id= 0} {1 b. 0} {1 country_id; 0}]
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 a 0} {6 LEFT JOIN 0} {1 country 0} {1 b 0} {1 ON 0} {1 a. 0} {1 country_id= 0} {1 b. 0} {1 country_id; 0}]
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 a 0} {6 RIGHT JOIN 0} {1 country 0} {1 b 0} {1 ON 0} {1 a. 0} {1 country_id= 0} {1 b. 0} {1 country_id; 0}]
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 a 0} {6 LEFT JOIN 0} {1 country 0} {1 b 0} {1 ON 0} {1 a. 0} {1 country_id= 0} {1 b. 0} {1 country_id 0} {5 WHERE 0} {1 b. 0} {1 last_update 0} {1 IS 0} {1 NULL; 0}]
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 a 0} {6 RIGHT JOIN 0} {1 country 0} {1 b 0} {1 ON 0} {1 a. 0} {1 country_id= 0} {1 b. 0} {1 country_id 0} {5 WHERE 0} {1 a. 0} {1 last_update 0} {1 IS 0} {1 NULL; 0}]
SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id UNION SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 a 0} {6 LEFT JOIN 0} {1 country 0} {1 b 0} {1 ON 0} {1 a. 0} {1 country_id= 0} {1 b. 0} {1 country_id 0} {5 UNION 0} {5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 a 0} {6 RIGHT JOIN 0} {1 country 0} {1 b 0} {1 ON 0} {1 a. 0} {1 country_id= 0} {1 b. 0} {1 country_id; 0}]
SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL UNION SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 a 0} {6 RIGHT JOIN 0} {1 country 0} {1 b 0} {1 ON 0} {1 a. 0} {1 country_id= 0} {1 b. 0} {1 country_id 0} {5 WHERE 0} {1 a. 0} {1 last_update 0} {1 IS 0} {1 NULL 0} {5 UNION 0} {5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 a 0} {6 LEFT JOIN 0} {1 country 0} {1 b 0} {1 ON 0} {1 a. 0} {1 country_id= 0} {1 b. 0} {1 country_id 0} {5 WHERE 0} {1 b. 0} {1 last_update 0} {1 IS 0} {1 NULL; 0}]
SELECT country_id, last_update FROM city NATURAL JOIN country;
[{5 SELECT 0} {1 country_id, 0} {0 0} {1 last_update 0} {5 FROM 0} {1 city 0} {1 NATURAL 0} {6 JOIN 0} {1 country; 0}]
SELECT country_id, last_update FROM city NATURAL LEFT JOIN country;
[{5 SELECT 0} {1 country_id, 0} {0 0} {1 last_update 0} {5 FROM 0} {1 city 0} {1 NATURAL 0} {6 LEFT JOIN 0} {1 country; 0}]
SELECT country_id, last_update FROM city NATURAL RIGHT JOIN country;
[{5 SELECT 0} {1 country_id, 0} {0 0} {1 last_update 0} {5 FROM 0} {1 city 0} {1 NATURAL 0} {6 RIGHT JOIN 0} {1 country; 0}]
SELECT a.country_id, a.last_update FROM city a STRAIGHT_JOIN country b ON a.country_id=b.country_id;
[{5 SELECT 0} {1 a. 0} {1 country_id, 0} {0 0} {1 a. 0} {1 last_update 0} {5 FROM 0} {1 city 0} {1 a 0} {1 STRAIGHT_JOIN 0} {1 country 0} {1 b 0} {1 ON 0} {1 a. 0} {1 country_id= 0} {1 b. 0} {1 country_id; 0}]
SELECT a.address, a.postal_code FROM sakila.address a WHERE a.city_id IN (SELECT c.city_id FROM sakila.city c);
[{5 SELECT 0} {1 a. 0} {1 address, 0} {0 0} {1 a. 0} {1 postal_code 0} {5 FROM 0} {1 sakila. 0} {1 address 0} {1 a 0} {5 WHERE 0} {1 a. 0} {1 city_id 0} {1 IN 0} {0 0} {7 ( 0} {5 SELECT 0} {1 c. 0} {1 city_id 0} {5 FROM 0} {1 sakila. 0} {1 city 0} {1 c) 0} {7 ; 0}]
SELECT city FROM( SELECT city_id FROM city WHERE city = "A Corua (La Corua)" ORDER BY last_update DESC LIMIT 50, 10) I JOIN city ON (I.city_id = city.city_id) JOIN country ON (country.country_id = city.country_id) ORDER BY city DESC;
[{5 SELECT 0} {1 city 0} {5 FROM( 0} {0 0} {5 SELECT 0} {1 city_id 0} {5 FROM 0} {1 city 0} {5 WHERE 0} {1 city 0} {7 = 0} {0 0} {2 "A Corua (La Corua)" 0} {0 0} {5 ORDER BY 0} {1 last_update 0} {1 DESC 0} {5 LIMIT 0} {10 50, 0} {0 0} {10 10) 0} {0 0} {1 I 0} {6 JOIN 0} {1 city 0} {1 ON 0} {7 ( 0} {1 I. 0} {1 city_id 0} {7 = 0} {0 0} {1 city. 0} {1 city_id) 0} {0 0} {6 JOIN 0} {1 country 0} {1 ON 0} {7 ( 0} {1 country. 0} {1 country_id 0} {7 = 0} {0 0} {1 city. 0} {1 country_id) 0} {0 0} {5 ORDER BY 0} {1 city 0} {1 DESC; 0}]
DELETE city, country FROM city INNER JOIN country using (country_id) WHERE city.city_id = 1;
[{1 DELETE 0} {1 city, 0} {0 0} {1 country 0} {5 FROM 0} {1 city 0} {6 INNER JOIN 0} {1 country 0} {1 using 0} {7 ( 0} {1 country_id) 0} {0 0} {5 WHERE 0} {1 city. 0} {1 city_id 0} {7 = 0} {0 0} {10 1; 0}]
DELETE city FROM city LEFT JOIN country ON city.country_id = country.country_id WHERE country.country IS NULL;
[{1 DELETE 0} {1 city 0} {5 FROM 0} {1 city 0} {6 LEFT JOIN 0} {1 country 0} {1 ON 0} {1 city. 0} {1 country_id 0} {7 = 0} {0 0} {1 country. 0} {1 country_id 0} {5 WHERE 0} {1 country. 0} {1 country 0} {1 IS 0} {1 NULL; 0}]
DELETE a1, a2 FROM city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;
[{1 DELETE 0} {1 a1, 0} {0 0} {1 a2 0} {5 FROM 0} {1 city 0} {1 AS 0} {1 a1 0} {6 INNER JOIN 0} {1 country 0} {1 AS 0} {1 a2 0} {5 WHERE 0} {1 a1. 0} {1 country_id= 0} {1 a2. 0} {1 country_id; 0}]
DELETE FROM a1, a2 USING city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;
[{5 DELETE FROM 0} {1 a1, 0} {0 0} {1 a2 0} {1 USING 0} {1 city 0} {1 AS 0} {1 a1 0} {6 INNER JOIN 0} {1 country 0} {1 AS 0} {1 a2 0} {5 WHERE 0} {1 a1. 0} {1 country_id= 0} {1 a2. 0} {1 country_id; 0}]
DELETE FROM film WHERE length > 100;
[{5 DELETE FROM 0} {1 film 0} {5 WHERE 0} {4 LENGTH 0} {7 > 0} {0 0} {10 100; 0}]
UPDATE city INNER JOIN country USING(country_id) SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;
[{5 UPDATE 0} {1 city 0} {6 INNER JOIN 0} {1 country 0} {1 USING( 0} {1 country_id) 0} {0 0} {5 SET 0} {1 city. 0} {1 city 0} {7 = 0} {0 0} {2 'Abha' 0} {7 , 0} {0 0} {1 city. 0} {1 last_update 0} {7 = 0} {0 0} {2 '2006-02-15 04:45:25' 0} {7 , 0} {0 0} {1 country. 0} {1 country 0} {7 = 0} {0 0} {2 'Afghanistan' 0} {0 0} {5 WHERE 0} {1 city. 0} {1 city_id= 0} {10 10; 0}]
UPDATE city INNER JOIN country ON city.country_id = country.country_id INNER JOIN address ON city.city_id = address.city_id SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;
[{5 UPDATE 0} {1 city 0} {6 INNER JOIN 0} {1 country 0} {1 ON 0} {1 city. 0} {1 country_id 0} {7 = 0} {0 0} {1 country. 0} {1 country_id 0} {6 INNER JOIN 0} {1 address 0} {1 ON 0} {1 city. 0} {1 city_id 0} {7 = 0} {0 0} {1 address. 0} {1 city_id 0} {5 SET 0} {1 city. 0} {1 city 0} {7 = 0} {0 0} {2 'Abha' 0} {7 , 0} {0 0} {1 city. 0} {1 last_update 0} {7 = 0} {0 0} {2 '2006-02-15 04:45:25' 0} {7 , 0} {0 0} {1 country. 0} {1 country 0} {7 = 0} {0 0} {2 'Afghanistan' 0} {0 0} {5 WHERE 0} {1 city. 0} {1 city_id= 0} {10 10; 0}]
UPDATE city, country SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.country_id = country.country_id AND city.city_id=10;
[{5 UPDATE 0} {1 city, 0} {0 0} {1 country 0} {5 SET 0} {1 city. 0} {1 city 0} {7 = 0} {0 0} {2 'Abha' 0} {7 , 0} {0 0} {1 city. 0} {1 last_update 0} {7 = 0} {0 0} {2 '2006-02-15 04:45:25' 0} {7 , 0} {0 0} {1 country. 0} {1 country 0} {7 = 0} {0 0} {2 'Afghanistan' 0} {0 0} {5 WHERE 0} {1 city. 0} {1 country_id 0} {7 = 0} {0 0} {1 country. 0} {1 country_id 0} {6 AND 0} {1 city. 0} {1 city_id= 0} {10 10; 0}]
UPDATE film SET length = 10 WHERE language_id = 20;
[{5 UPDATE 0} {1 film 0} {5 SET 0} {4 LENGTH 0} {7 = 0} {0 0} {10 10 0} {5 WHERE 0} {1 language_id 0} {7 = 0} {0 0} {10 20; 0}]
INSERT INTO city (country_id) SELECT country_id FROM country;
[{4 INSERT 0} {1 INTO 0} {1 city 0} {7 ( 0} {1 country_id) 0} {0 0} {5 SELECT 0} {1 country_id 0} {5 FROM 0} {1 country; 0}]
INSERT INTO city (country_id) VALUES (1),(2),(3);
[{4 INSERT 0} {1 INTO 0} {1 city 0} {7 ( 0} {1 country_id) 0} {0 0} {5 VALUES 0} {7 ( 0} {10 1) 0} {7 , 0} {7 ( 0} {10 2) 0} {7 , 0} {7 ( 0} {10 3) 0} {7 ; 0}]
INSERT INTO city (country_id) VALUES (10);
[{4 INSERT 0} {1 INTO 0} {1 city 0} {7 ( 0} {1 country_id) 0} {0 0} {5 VALUES 0} {7 ( 0} {10 10) 0} {7 ; 0}]
INSERT INTO city (country_id) SELECT 10 FROM DUAL;
[{4 INSERT 0} {1 INTO 0} {1 city 0} {7 ( 0} {1 country_id) 0} {0 0} {5 SELECT 0} {10 10 0} {5 FROM 0} {1 DUAL; 0}]
REPLACE INTO city (country_id) SELECT country_id FROM country;
[{4 REPLACE 0} {1 INTO 0} {1 city 0} {7 ( 0} {1 country_id) 0} {0 0} {5 SELECT 0} {1 country_id 0} {5 FROM 0} {1 country; 0}]
REPLACE INTO city (country_id) VALUES (1),(2),(3);
[{4 REPLACE 0} {1 INTO 0} {1 city 0} {7 ( 0} {1 country_id) 0} {0 0} {5 VALUES 0} {7 ( 0} {10 1) 0} {7 , 0} {7 ( 0} {10 2) 0} {7 , 0} {7 ( 0} {10 3) 0} {7 ; 0}]
REPLACE INTO city (country_id) VALUES (10);
[{4 REPLACE 0} {1 INTO 0} {1 city 0} {7 ( 0} {1 country_id) 0} {0 0} {5 VALUES 0} {7 ( 0} {10 10) 0} {7 ; 0}]
REPLACE INTO city (country_id) SELECT 10 FROM DUAL;
[{4 REPLACE 0} {1 INTO 0} {1 city 0} {7 ( 0} {1 country_id) 0} {0 0} {5 SELECT 0} {10 10 0} {5 FROM 0} {1 DUAL; 0}]
SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film;
[{5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {7 ( 0} {0 0} {5 SELECT 0} {1 film_id 0} {5 FROM 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film 0} {7 ) 0} {0 0} {1 film; 0}]
SELECT * FROM film WHERE language_id = (SELECT language_id FROM language LIMIT 1);
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {1 language_id 0} {7 = 0} {0 0} {7 ( 0} {5 SELECT 0} {1 language_id 0} {5 FROM 0} {1 language 0} {5 LIMIT 0} {10 1) 0} {7 ; 0}]
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 i 0} {6 LEFT JOIN 0} {1 country 0} {1 o 0} {1 ON 0} {1 i. 0} {1 city_id= 0} {1 o. 0} {1 country_id 0} {5 UNION 0} {5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 i 0} {6 RIGHT JOIN 0} {1 country 0} {1 o 0} {1 ON 0} {1 i. 0} {1 city_id= 0} {1 o. 0} {1 country_id; 0}]
SELECT * FROM (SELECT * FROM actor WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE') t WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE' GROUP BY first_name;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {7 ( 0} {5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 actor 0} {5 WHERE 0} {1 last_update= 0} {2 '2006-02-15 04:34:33' 0} {0 0} {6 AND 0} {1 last_name= 0} {2 'CHASE' 0} {7 ) 0} {0 0} {1 t 0} {5 WHERE 0} {1 last_update= 0} {2 '2006-02-15 04:34:33' 0} {0 0} {6 AND 0} {1 last_name= 0} {2 'CHASE' 0} {0 0} {5 GROUP BY 0} {1 first_name; 0}]
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 i 0} {6 LEFT JOIN 0} {1 country 0} {1 o 0} {1 ON 0} {1 i. 0} {1 city_id= 0} {1 o. 0} {1 country_id 0} {5 UNION 0} {5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 i 0} {6 RIGHT JOIN 0} {1 country 0} {1 o 0} {1 ON 0} {1 i. 0} {1 city_id= 0} {1 o. 0} {1 country_id; 0}]
SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id WHERE o.country_id is null union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id WHERE i.city_id is null;
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 i 0} {6 LEFT JOIN 0} {1 country 0} {1 o 0} {1 ON 0} {1 i. 0} {1 city_id= 0} {1 o. 0} {1 country_id 0} {5 WHERE 0} {1 o. 0} {1 country_id 0} {1 is 0} {1 null 0} {5 UNION 0} {5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 city 0} {1 i 0} {6 RIGHT JOIN 0} {1 country 0} {1 o 0} {1 ON 0} {1 i. 0} {1 city_id= 0} {1 o. 0} {1 country_id 0} {5 WHERE 0} {1 i. 0} {1 city_id 0} {1 is 0} {1 null; 0}]
SELECT first_name,last_name,email FROM customer STRAIGHT_JOIN address ON customer.address_id=address.address_id;
[{5 SELECT 0} {1 first_name, 0} {1 last_name, 0} {1 email 0} {5 FROM 0} {1 customer 0} {1 STRAIGHT_JOIN 0} {1 address 0} {1 ON 0} {1 customer. 0} {1 address_id= 0} {1 address. 0} {1 address_id; 0}]
SELECT ID,name FROM (SELECT address FROM customer_list WHERE SID=1 order by phone limit 50,10) a JOIN customer_list l ON (a.address=l.address) JOIN city c ON (c.city=l.city) order by phone desc;
[{5 SELECT 0} {1 ID, 0} {1 name 0} {5 FROM 0} {7 ( 0} {5 SELECT 0} {1 address 0} {5 FROM 0} {1 customer_list 0} {5 WHERE 0} {1 SID= 0} {10 1 0} {5 ORDER BY 0} {1 phone 0} {5 LIMIT 0} {10 50, 0} {10 10) 0} {0 0} {1 a 0} {6 JOIN 0} {1 customer_list 0} {1 l 0} {1 ON 0} {7 ( 0} {1 a. 0} {1 address= 0} {1 l. 0} {1 address) 0} {0 0} {6 JOIN 0} {1 city 0} {1 c 0} {1 ON 0} {7 ( 0} {1 c. 0} {1 city= 0} {1 l. 0} {1 city) 0} {0 0} {5 ORDER BY 0} {1 phone 0} {1 desc; 0}]
SELECT * FROM film WHERE date(last_update)='2006-02-15';
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {4 DATE( 0} {1 last_update) 0} {7 = 0} {2 '2006-02-15' 0} {7 ; 0}]
SELECT last_update FROM film GROUP BY date(last_update);
[{5 SELECT 0} {1 last_update 0} {5 FROM 0} {1 film 0} {5 GROUP BY 0} {4 DATE( 0} {1 last_update) 0} {7 ; 0}]
SELECT last_update FROM film order by date(last_update);
[{5 SELECT 0} {1 last_update 0} {5 FROM 0} {1 film 0} {5 ORDER BY 0} {4 DATE( 0} {1 last_update) 0} {7 ; 0}]
SELECT description FROM film WHERE description IN('NEWS','asd') GROUP BY description;
[{5 SELECT 0} {1 description 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {1 description 0} {1 IN( 0} {2 'NEWS' 0} {7 , 0} {2 'asd' 0} {7 ) 0} {0 0} {5 GROUP BY 0} {1 description; 0}]
alter table address add index idx_city_id(city_id);
[{5 ALTER TABLE 0} {1 address 0} {5 ADD 0} {1 index 0} {1 idx_city_id( 0} {1 city_id) 0} {7 ; 0}]
alter table inventory add index `idx_store_film` (`store_id`,`film_id`);
[{5 ALTER TABLE 0} {1 inventory 0} {5 ADD 0} {1 index 0} {3 `idx_store_film` 0} {0 0} {7 ( 0} {3 `store_id` 0} {7 , 0} {3 `film_id` 0} {7 ) 0} {7 ; 0}]
alter table inventory add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`);
[{5 ALTER TABLE 0} {1 inventory 0} {5 ADD 0} {1 index 0} {3 `idx_store_film` 0} {0 0} {7 ( 0} {3 `store_id` 0} {7 , 0} {3 `film_id` 0} {7 ) 0} {7 , 0} {5 ADD 0} {1 index 0} {3 `idx_store_film` 0} {0 0} {7 ( 0} {3 `store_id` 0} {7 , 0} {3 `film_id` 0} {7 ) 0} {7 , 0} {5 ADD 0} {1 index 0} {3 `idx_store_film` 0} {0 0} {7 ( 0} {3 `store_id` 0} {7 , 0} {3 `film_id` 0} {7 ) 0} {7 ; 0}]
SELECT DATE_FORMAT(t.last_update, '%Y-%m-%d'), COUNT(DISTINCT (t.city)) FROM city t WHERE t.last_update > '2018-10-22 00:00:00' AND t.city LIKE '%Chrome%' AND t.city = 'eip' GROUP BY DATE_FORMAT(t.last_update, '%Y-%m-%d') ORDER BY DATE_FORMAT(t.last_update, '%Y-%m-%d');
[{5 SELECT 0} {4 DATE_FORMAT( 0} {1 t. 0} {1 last_update, 0} {0 0} {2 '%Y-%m-%d' 0} {7 ) 0} {7 , 0} {0 0} {4 COUNT( 0} {1 DISTINCT 0} {7 ( 0} {1 t. 0} {1 city) 0} {7 ) 0} {0 0} {5 FROM 0} {1 city 0} {1 t 0} {5 WHERE 0} {1 t. 0} {1 last_update 0} {7 > 0} {0 0} {2 '2018-10-22 00:00:00' 0} {0 0} {6 AND 0} {1 t. 0} {1 city 0} {1 LIKE 0} {2 '%Chrome%' 0} {0 0} {6 AND 0} {1 t. 0} {1 city 0} {7 = 0} {0 0} {2 'eip' 0} {0 0} {5 GROUP BY 0} {4 DATE_FORMAT( 0} {1 t. 0} {1 last_update, 0} {0 0} {2 '%Y-%m-%d' 0} {7 ) 0} {0 0} {5 ORDER BY 0} {4 DATE_FORMAT( 0} {1 t. 0} {1 last_update, 0} {0 0} {2 '%Y-%m-%d' 0} {7 ) 0} {7 ; 0}]
create table hello.t (id int unsigned);
[{1 create 0} {1 table 0} {1 hello. 0} {1 t 0} {7 ( 0} {1 id 0} {1 int 0} {1 unsigned) 0} {7 ; 0}]
select * from tb where data >= '';
[{5 SELECT 0} {7 * 0} {0 0} {5 FROM 0} {1 tb 0} {5 WHERE 0} {1 data 0} {7 >= 0} {0 0} {2 '' 0} {7 ; 0}]
alter table tb alter column id drop default;
[{5 ALTER TABLE 0} {1 tb 0} {1 alter 0} {1 column 0} {1 id 0} {5 DROP 0} {4 DEFAULT; 0}]
select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film where last_update > '2016-03-27 02:01:01') as d;
[{5 SELECT 0} {1 maxId, 0} {0 0} {1 minId 0} {5 FROM 0} {7 ( 0} {5 SELECT 0} {4 MAX( 0} {1 film_id) 0} {0 0} {1 maxId, 0} {0 0} {4 MIN( 0} {1 film_id) 0} {0 0} {1 minId 0} {5 FROM 0} {1 film 0} {5 WHERE 0} {1 last_update 0} {7 > 0} {0 0} {2 '2016-03-27 02:01:01' 0} {7 ) 0} {0 0} {1 as 0} {1 d; 0}]
select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film) as d;
[{5 SELECT 0} {1 maxId, 0} {0 0} {1 minId 0} {5 FROM 0} {7 ( 0} {5 SELECT 0} {4 MAX( 0} {1 film_id) 0} {0 0} {1 maxId, 0} {0 0} {4 MIN( 0} {1 film_id) 0} {0 0} {1 minId 0} {5 FROM 0} {1 film) 0} {0 0} {1 as 0} {1 d; 0}]
================================================
FILE: ast/testdata/TestTokenizer.golden
================================================
[]ast.Token{
{Type:57348, Val:"select", i:0},
{Type:57407, Val:"-- comment 1", i:0},
}
[]ast.Token{
{Type:57348, Val:"select", i:0},
{Type:57397, Val:"c1", i:0},
{Type:44, Val:",", i:0},
{Type:57397, Val:"c2", i:0},
{Type:44, Val:",", i:0},
{Type:57397, Val:"c3", i:0},
{Type:57353, Val:"from", i:0},
{Type:57397, Val:"t1", i:0},
{Type:44, Val:",", i:0},
{Type:57397, Val:"t2", i:0},
{Type:57385, Val:"join", i:0},
{Type:57397, Val:"t3", i:0},
{Type:57395, Val:"on", i:0},
{Type:57397, Val:"t1", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"c1", i:0},
{Type:61, Val:"=", i:0},
{Type:57397, Val:"t2", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"c1", i:0},
{Type:57415, Val:"and", i:0},
{Type:57397, Val:"t1", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"c3", i:0},
{Type:61, Val:"=", i:0},
{Type:57397, Val:"t3", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"c1", i:0},
{Type:57354, Val:"where", i:0},
{Type:57397, Val:"id", i:0},
{Type:62, Val:">", i:0},
{Type:57402, Val:"1000", i:0},
}
[]ast.Token{
{Type:57348, Val:"select", i:0},
{Type:57397, Val:"sourcetable", i:0},
{Type:44, Val:",", i:0},
{Type:57456, Val:"if", i:0},
{Type:40, Val:"(", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"lastcontent", i:0},
{Type:61, Val:"=", i:0},
{Type:57405, Val:":v1", i:0},
{Type:44, Val:",", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"lastupdate", i:0},
{Type:44, Val:",", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"lastcontent", i:0},
{Type:41, Val:")", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"lastactivity", i:0},
{Type:44, Val:",", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"totalcount", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"activity", i:0},
{Type:44, Val:",", i:0},
{Type:57397, Val:"type", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"class", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"type", i:0},
{Type:44, Val:",", i:0},
{Type:40, Val:"(", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"nodeoptions", i:0},
{Type:38, Val:"&", i:0},
{Type:57405, Val:":v2", i:0},
{Type:41, Val:")", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"nounsubscribe", i:0},
{Type:57353, Val:"from", i:0},
{Type:57397, Val:"node", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"f", i:0},
{Type:57389, Val:"inner", i:0},
{Type:57385, Val:"join", i:0},
{Type:57397, Val:"contenttype", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"type", i:0},
{Type:57395, Val:"on", i:0},
{Type:57397, Val:"type", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"contenttypeid", i:0},
{Type:61, Val:"=", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"contenttypeid", i:0},
{Type:57389, Val:"inner", i:0},
{Type:57385, Val:"join", i:0},
{Type:57397, Val:"subscribed", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"sd", i:0},
{Type:57395, Val:"on", i:0},
{Type:57397, Val:"sd", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"did", i:0},
{Type:61, Val:"=", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"nodeid", i:0},
{Type:57415, Val:"and", i:0},
{Type:57397, Val:"sd", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"userid", i:0},
{Type:61, Val:"=", i:0},
{Type:57405, Val:":v3", i:0},
{Type:57347, Val:"union", i:0},
{Type:57362, Val:"all", i:0},
{Type:57348, Val:"select", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"name", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"title", i:0},
{Type:44, Val:",", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"userid", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"keyval", i:0},
{Type:44, Val:",", i:0},
{Type:57405, Val:":v4", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"sourcetable", i:0},
{Type:44, Val:",", i:0},
{Type:57397, Val:"ifnull", i:0},
{Type:40, Val:"(", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"lastpost", i:0},
{Type:44, Val:",", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"joindate", i:0},
{Type:41, Val:")", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"lastactivity", i:0},
{Type:44, Val:",", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"posts", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"activity", i:0},
{Type:44, Val:",", i:0},
{Type:57405, Val:":v5", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"type", i:0},
{Type:44, Val:",", i:0},
{Type:57405, Val:":v6", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"nounsubscribe", i:0},
{Type:57353, Val:"from", i:0},
{Type:57397, Val:"user", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"f", i:0},
{Type:57389, Val:"inner", i:0},
{Type:57385, Val:"join", i:0},
{Type:57397, Val:"userlist", i:0},
{Type:57364, Val:"as", i:0},
{Type:57397, Val:"ul", i:0},
{Type:57395, Val:"on", i:0},
{Type:57397, Val:"ul", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"relationid", i:0},
{Type:61, Val:"=", i:0},
{Type:57397, Val:"f", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"userid", i:0},
{Type:57415, Val:"and", i:0},
{Type:57397, Val:"ul", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"userid", i:0},
{Type:61, Val:"=", i:0},
{Type:57405, Val:":v7", i:0},
{Type:57354, Val:"where", i:0},
{Type:57397, Val:"ul", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"type", i:0},
{Type:61, Val:"=", i:0},
{Type:57405, Val:":v8", i:0},
{Type:57415, Val:"and", i:0},
{Type:57397, Val:"ul", i:0},
{Type:46, Val:".", i:0},
{Type:57397, Val:"aq", i:0},
{Type:61, Val:"=", i:0},
{Type:57405, Val:":v9", i:0},
{Type:57357, Val:"order", i:0},
{Type:57358, Val:"by", i:0},
{Type:57397, Val:"title", i:0},
{Type:57359, Val:"limit", i:0},
{Type:57405, Val:":v10", i:0},
}
[]ast.Token{
{Type:57348, Val:"select", i:0},
{Type:57397, Val:"c1", i:0},
{Type:57353, Val:"from", i:0},
{Type:57397, Val:"t1", i:0},
{Type:57354, Val:"where", i:0},
{Type:57397, Val:"id", i:0},
{Type:57424, Val:">=", i:0},
{Type:57402, Val:"1000", i:0},
}
[]ast.Token{
{Type:57348, Val:"select", i:0},
{Type:57593, Val:"SQL_CALC_FOUND_ROWS", i:0},
{Type:57397, Val:"col", i:0},
{Type:57353, Val:"from", i:0},
{Type:57397, Val:"tbl", i:0},
{Type:57354, Val:"where", i:0},
{Type:57397, Val:"id", i:0},
{Type:62, Val:">", i:0},
{Type:57402, Val:"1000", i:0},
}
[]ast.Token{
{Type:57348, Val:"SELECT", i:0},
{Type:42, Val:"*", i:0},
{Type:57353, Val:"FROM", i:0},
{Type:57397, Val:"tb", i:0},
{Type:57354, Val:"WHERE", i:0},
{Type:57397, Val:"id", i:0},
{Type:61, Val:"=", i:0},
{Type:57405, Val:":v1", i:0},
{Type:59, Val:";", i:0},
}
[]ast.Token{
{Type:57348, Val:"SELECT", i:0},
{Type:42, Val:"*", i:0},
{Type:57353, Val:"FROM", i:0},
{Type:57397, Val:"tb", i:0},
{Type:57354, Val:"WHERE", i:0},
{Type:57397, Val:"id", i:0},
{Type:57427, Val:"is", i:0},
{Type:57410, Val:"null", i:0},
{Type:59, Val:";", i:0},
}
[]ast.Token{
{Type:57348, Val:"SELECT", i:0},
{Type:42, Val:"*", i:0},
{Type:57353, Val:"FROM", i:0},
{Type:57397, Val:"tb", i:0},
{Type:57354, Val:"WHERE", i:0},
{Type:57397, Val:"id", i:0},
{Type:57427, Val:"is", i:0},
{Type:57416, Val:"not", i:0},
{Type:57410, Val:"null", i:0},
{Type:59, Val:";", i:0},
}
[]ast.Token{
{Type:57348, Val:"SELECT", i:0},
{Type:42, Val:"*", i:0},
{Type:57353, Val:"FROM", i:0},
{Type:57397, Val:"tb", i:0},
{Type:57354, Val:"WHERE", i:0},
{Type:57397, Val:"id", i:0},
{Type:57417, Val:"between", i:0},
{Type:57402, Val:"1", i:0},
{Type:57415, Val:"and", i:0},
{Type:57402, Val:"3", i:0},
{Type:59, Val:";", i:0},
}
[]ast.Token{
{Type:57444, Val:"alter", i:0},
{Type:57451, Val:"table", i:0},
{Type:57397, Val:"inventory", i:0},
{Type:57448, Val:"add", i:0},
{Type:57452, Val:"index", i:0},
{Type:57397, Val:"idx_store_film", i:0},
{Type:57397, Val:" (", i:0},
{Type:57397, Val:"store_id", i:0},
{Type:57397, Val:",", i:0},
{Type:57397, Val:"film_id", i:0},
{Type:57346, Val:");", i:0},
}
[]ast.Token{
{Type:57351, Val:"UPDATE", i:0},
{Type:57397, Val:"xxx", i:0},
{Type:57372, Val:"SET", i:0},
{Type:57397, Val:"c1", i:0},
{Type:61, Val:"=", i:0},
{Type:57401, Val:" LOGGER.error(\"\"); }", i:0},
{Type:57354, Val:"WHERE", i:0},
{Type:57397, Val:"id", i:0},
{Type:61, Val:"=", i:0},
{Type:57402, Val:"2", i:0},
{Type:59, Val:";", i:0},
}
[]ast.Token{
{Type:57351, Val:"update", i:0},
{Type:57346, Val:"\xc2", i:0},
{Type:57346, Val:"\xa0", i:0},
{Type:57397, Val:"tb", i:0},
{Type:57372, Val:"set", i:0},
{Type:57346, Val:"\xc2", i:0},
{Type:57346, Val:"\xa0", i:0},
{Type:57488, Val:"status", i:0},
{Type:57346, Val:"\xc2", i:0},
{Type:57346, Val:"\xa0", i:0},
{Type:61, Val:"=", i:0},
{Type:57346, Val:"\xc2", i:0},
{Type:57346, Val:"\xa0", i:0},
{Type:57402, Val:"1", i:0},
{Type:57354, Val:"where", i:0},
{Type:57346, Val:"\xc2", i:0},
{Type:57346, Val:"\xa0", i:0},
{Type:57397, Val:"id", i:0},
{Type:57346, Val:"\xc2", i:0},
{Type:57346, Val:"\xa0", i:0},
{Type:61, Val:"=", i:0},
{Type:57346, Val:"\xc2", i:0},
{Type:57346, Val:"\xa0", i:0},
{Type:57402, Val:"1", i:0},
{Type:59, Val:";", i:0},
}
================================================
FILE: ast/testdata/TestVitessStmtNode2JSON.golden
================================================
{
"Cache": "",
"Comments": null,
"Distinct": "",
"Hints": "",
"SelectExprs": [
{
"Expr": {
"Type": 1,
"Val": "MQ=="
},
"As": ""
}
],
"From": [
{
"Expr": {
"Name": "dual",
"Qualifier": ""
},
"Partitions": null,
"As": "",
"Hints": null
}
],
"Where": null,
"GroupBy": null,
"Having": null,
"OrderBy": null,
"Limit": null,
"Lock": ""
}
================================================
FILE: ast/tidb.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ast
import (
"fmt"
"regexp"
"strings"
"github.com/XiaoMi/soar/common"
json "github.com/CorgiMan/json2"
"github.com/kr/pretty"
"github.com/pingcap/parser"
"github.com/pingcap/parser/ast"
"github.com/tidwall/gjson"
// for pingcap parser
_ "github.com/pingcap/tidb/types/parser_driver"
)
// TiParse TiDB 语法解析
func TiParse(sql, charset, collation string) ([]ast.StmtNode, error) {
p := parser.New()
sql = removeIncompatibleWords(sql)
stmt, warn, err := p.Parse(sql, charset, collation)
if err != nil {
// issue: https://github.com/XiaoMi/soar/issues/235
// TODO: bypass charset error, pingcap/parser not support so much charsets
if strings.Contains(err.Error(), "Unknown character set") {
err = nil
}
}
// TODO: bypass warning info
for _, w := range warn {
common.Log.Warn(w.Error())
}
return stmt, err
}
// removeIncompatibleWords remove pingcap/parser not support words from schema
func removeIncompatibleWords(sql string) string {
fields := strings.Fields(strings.TrimSpace(sql))
if len(fields) == 0 {
return sql
}
switch strings.ToLower(fields[0]) {
case "create", "alter":
default:
return sql
}
// CONSTRAINT col_fk FOREIGN KEY (col) REFERENCES tb (id) ON UPDATE CASCADE
re := regexp.MustCompile(`(?i) ON UPDATE CASCADE`)
sql = re.ReplaceAllString(sql, "")
// FULLTEXT KEY col_fk (col) /*!50100 WITH PARSER `ngram` */
// /*!50100 PARTITION BY LIST (col)
re = regexp.MustCompile(`/\*!5`)
sql = re.ReplaceAllString(sql, "/* 5")
// col varchar(10) CHARACTER SET gbk DEFAULT NULL
re = regexp.MustCompile(`(?i)CHARACTER SET [a-z_0-9]* `)
sql = re.ReplaceAllString(sql, "")
// CREATE TEMPORARY TABLE IF NOT EXISTS t_film AS (SELECT * FROM film);
re = regexp.MustCompile(`(?i)CREATE TEMPORARY TABLE`)
sql = re.ReplaceAllString(sql, "CREATE TABLE")
return sql
}
// PrintPrettyStmtNode 打印TiParse语法树
func PrintPrettyStmtNode(sql, charset, collation string) {
tree, err := TiParse(sql, charset, collation)
if err != nil {
common.Log.Warning(err.Error())
} else {
_, err = pretty.Println(tree)
common.LogIfWarn(err, "")
}
}
// StmtNode2JSON TiParse AST tree into json format
func StmtNode2JSON(sql, charset, collation string) string {
var str string
tree, err := TiParse(sql, charset, collation)
if err != nil {
common.Log.Warning(err.Error())
} else {
b, err := json.MarshalIndent(tree, "", " ")
if err != nil {
common.Log.Error(err.Error())
} else {
str = string(b)
}
}
return str
}
// SchemaMetaInfo get used database, table name from SQL
func SchemaMetaInfo(sql string, defaultDatabase string) []string {
var tables []string
tree, err := TiParse(sql, "", "")
if err != nil {
return tables
}
jsonString := StmtNode2JSON(sql, "", "")
for _, node := range tree {
switch n := node.(type) {
case *ast.UseStmt:
tables = append(tables, fmt.Sprintf("`%s`.`dual`", n.DBName))
// SetOprStmt represents "union/except/intersect statement"
case *ast.InsertStmt, *ast.SelectStmt, *ast.SetOprStmt, *ast.UpdateStmt, *ast.DeleteStmt:
// DML/DQL: INSERT, SELECT, UPDATE, DELETE
for _, tableRef := range common.JSONFind(jsonString, "TableRefs") {
for _, source := range common.JSONFind(tableRef, "Source") {
database := gjson.Get(source, "Schema.O")
table := gjson.Get(source, "Name.O")
if database.String() == "" {
if table.String() != "" {
tables = append(tables, fmt.Sprintf("`%s`.`%s`", defaultDatabase, table.String()))
}
} else {
if table.String() != "" {
tables = append(tables, fmt.Sprintf("`%s`.`%s`", database.String(), table.String()))
} else {
tables = append(tables, fmt.Sprintf("`%s`.`dual`", database.String()))
}
}
}
}
case *ast.DropTableStmt:
// DDL: DROP TABLE|VIEW
schemas := common.JSONFind(jsonString, "Tables")
for _, tabs := range schemas {
for _, table := range gjson.Parse(tabs).Array() {
db := gjson.Get(table.String(), "Schema.O")
tb := gjson.Get(table.String(), "Name.O")
if db.String() == "" {
if tb.String() != "" {
tables = append(tables, fmt.Sprintf("`%s`.`%s`", defaultDatabase, tb.String()))
}
} else {
if tb.String() != "" {
tables = append(tables, fmt.Sprintf("`%s`.`%s`", db.String(), tb.String()))
}
}
}
}
case *ast.DropDatabaseStmt, *ast.CreateDatabaseStmt:
// DDL: DROP|CREATE DATABASE
schemas := common.JSONFind(jsonString, "Name")
for _, schema := range schemas {
tables = append(tables, fmt.Sprintf("`%s`.`dual`", schema))
}
default:
// DDL: CREATE TABLE|DATABASE|INDEX|VIEW, DROP INDEX
schemas := common.JSONFind(jsonString, "Table")
for _, table := range schemas {
db := gjson.Get(table, "Schema.O")
tb := gjson.Get(table, "Name.O")
if db.String() == "" {
if tb.String() != "" {
tables = append(tables, fmt.Sprintf("`%s`.`%s`", defaultDatabase, tb.String()))
}
} else {
if tb.String() != "" {
tables = append(tables, fmt.Sprintf("`%s`.`%s`", db.String(), tb.String()))
}
}
}
}
}
return common.RemoveDuplicatesItem(tables)
}
================================================
FILE: ast/tidb_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ast
import (
"fmt"
"testing"
"github.com/XiaoMi/soar/common"
)
func TestPrintPrettyStmtNode(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
sqls := []string{
`select 1`,
`select * f`, // syntax error case
}
err := common.GoldenDiff(func() {
for _, sql := range sqls {
PrintPrettyStmtNode(sql, "", "")
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestStmtNode2JSON(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
sqls := []string{
`select 1`,
`select * f`, // syntax error case
}
err := common.GoldenDiff(func() {
for _, sql := range sqls {
fmt.Println(StmtNode2JSON(sql, "", ""))
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestSchemaMetaInfo(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
sqls := []string{
"use world_x;",
"select 1;",
"syntax error case",
"select * from ta join tb using (id)",
"select * from ta, tb limit 1",
"drop table tb",
"drop table db.tb",
"drop database db",
"create database db",
"create index idx_col on tbl (col)",
"DROP INDEX idx_col on tbl",
}
// fmt.Println(sqls[len(sqls)-1])
// fmt.Println(SchemaMetaInfo(sqls[len(sqls)-1], "sakila"))
// return
err := common.GoldenDiff(func() {
for _, sql := range append(sqls, common.TestSQLs...) {
fmt.Println(sql)
fmt.Println(SchemaMetaInfo(sql, "sakila"))
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestRemoveIncompatibleWords(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
sqls := [][]string{
{
`CREATE TEMPORARY TABLE IF NOT EXISTS t_film AS (SELECT * FROM film)`,
`CREATE CONSTRAINT col_fk FOREIGN KEY (col) REFERENCES tb (id) ON UPDATE CASCADE`,
"CREATE FULLTEXT KEY col_fk (col) /*!50100 WITH PARSER `ngram` */",
`CREATE /*!50100 PARTITION BY LIST (col)`,
`CREATE col varchar(10) CHARACTER SET gbk DEFAULT NULL`,
},
{
`CREATE TABLE IF NOT EXISTS t_film AS (SELECT * FROM film)`,
`CREATE CONSTRAINT col_fk FOREIGN KEY (col) REFERENCES tb (id)`,
"CREATE FULLTEXT KEY col_fk (col) /* 50100 WITH PARSER `ngram` */",
`CREATE /* 50100 PARTITION BY LIST (col)`,
`CREATE col varchar(10) DEFAULT NULL`,
},
}
for k, sql := range sqls[0] {
sql = removeIncompatibleWords(sql)
if sqls[1][k] != sql {
fmt.Println(sql)
t.Fatal(sql)
}
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
================================================
FILE: ast/token.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ast
import (
"fmt"
"regexp"
"strings"
"unicode"
"vitess.io/vitess/go/vt/sqlparser"
)
// TokenType
const (
TokenTypeWhitespace = 0
TokenTypeWord = 1
TokenTypeQuote = 2
TokenTypeBacktickQuote = 3
TokenTypeReserved = 4
TokenTypeReservedToplevel = 5
TokenTypeReservedNewline = 6
TokenTypeBoundary = 7
TokenTypeComment = 8
TokenTypeBlockComment = 9
TokenTypeNumber = 10
TokenTypeError = 11
TokenTypeVariable = 12
)
var maxCachekeySize = 15
var cacheHits int
var cacheMisses int
var tokenBoundaries = []string{
// multi character
"(>=)", "(<=)", "(!=)", "(<>)",
// single characters
",", ";", ":", "\\)", "\\(", "\\.", "=", "<", ">", "\\+", "-", "\\*", "/", "!", "\\^", "%", "\\|", "&", "#",
}
var tokenReserved = []string{
"ACCESSIBLE", "ACTION", "AGAINST", "AGGREGATE", "ALGORITHM", "ALL", "ALTER", "ANALYSE", "ANALYZE", "AS", "ASC",
"AUTOCOMMIT", "AUTO_INCREMENT", "BACKUP", "BEGIN", "BETWEEN", "BINLOG", "BOTH", "CASCADE", "CASE", "CHANGE", "CHANGED", "CHARACTER SET",
"CHARSET", "CHECK", "CHECKSUM", "COLLATE", "COLLATION", "COLUMN", "COLUMNS", "COMMENT", "COMMIT", "COMMITTED", "COMPRESSED", "CONCURRENT",
"CONSTRAINT", "CONTAINS", "CONVERT", "CREATE", "CROSS", "CURRENT_TIMESTAMP", "DATABASE", "DATABASES", "DAY", "DAY_HOUR", "DAY_MINUTE",
"DAY_SECOND", "DEFAULT", "DEFINER", "DELAYED", "DELETE", "DESC", "DESCRIBE", "DETERMINISTIC", "DISTINCT", "DISTINCTROW", "DIV",
"DO", "DUMPFILE", "DUPLICATE", "DYNAMIC", "ELSE", "ENCLOSED", "END", "ENGINE", "ENGINE_TYPE", "ENGINES", "ESCAPE", "ESCAPED", "EVENTS", "EXEC",
"EXECUTE", "EXISTS", "EXPLAIN", "EXTENDED", "FAST", "FIELDS", "FILE", "FIRST", "FIXED", "FLUSH", "FOR", "FORCE", "FOREIGN", "FULL", "FULLTEXT",
"FUNCTION", "GLOBAL", "GRANT", "GRANTS", "GROUP_CONCAT", "HEAP", "HIGH_PRIORITY", "HOSTS", "HOUR", "HOUR_MINUTE",
"HOUR_SECOND", "IDENTIFIED", "IF", "IFNULL", "IGNORE", "IN", "INDEX", "INDEXES", "INFILE", "INSERT", "INSERT_ID", "INSERT_METHOD", "INTERVAL",
"INTO", "INVOKER", "IS", "ISOLATION", "KEY", "KEYS", "KILL", "LAST_INSERT_ID", "LEADING", "LEVEL", "LIKE", "LINEAR",
"LINES", "LOAD", "LOCAL", "LOCK", "LOCKS", "LOGS", "LOW_PRIORITY", "MARIA", "MASTER", "MASTER_CONNECT_RETRY", "MASTER_HOST", "MASTER_LOG_FILE",
"MATCH", "MAX_CONNECTIONS_PER_HOUR", "MAX_QUERIES_PER_HOUR", "MAX_ROWS", "MAX_UPDATES_PER_HOUR", "MAX_USER_CONNECTIONS",
"MEDIUM", "MERGE", "MINUTE", "MINUTE_SECOND", "MIN_ROWS", "MODE", "MODIFY",
"MONTH", "MRG_MYISAM", "MYISAM", "NAMES", "NATURAL", "NOT", "NOW()", "NULL", "OFFSET", "ON", "OPEN", "OPTIMIZE", "OPTION", "OPTIONALLY",
"ON UPDATE", "ON DELETE", "OUTFILE", "PACK_KEYS", "PAGE", "PARTIAL", "PARTITION", "PARTITIONS", "PASSWORD", "PRIMARY", "PRIVILEGES", "PROCEDURE",
"PROCESS", "PROCESSLIST", "PURGE", "QUICK", "RANGE", "RAID0", "RAID_CHUNKS", "RAID_CHUNKSIZE", "RAID_TYPE", "READ", "READ_ONLY",
"READ_WRITE", "REFERENCES", "REGEXP", "RELOAD", "RENAME", "REPAIR", "REPEATABLE", "REPLACE", "REPLICATION", "RESET", "RESTORE", "RESTRICT",
"RETURN", "RETURNS", "REVOKE", "RLIKE", "ROLLBACK", "ROW", "ROWS", "ROW_FORMAT", "SECOND", "SECURITY", "SEPARATOR",
"SERIALIZABLE", "SESSION", "SHARE", "SHOW", "SHUTDOWN", "SLAVE", "SONAME", "SOUNDS", "SQL", "SQL_AUTO_IS_NULL", "SQL_BIG_RESULT",
"SQL_BIG_SELECTS", "SQL_BIG_TABLES", "SQL_BUFFER_RESULT", "SQL_CALC_FOUND_ROWS", "SQL_LOG_BIN", "SQL_LOG_OFF", "SQL_LOG_UPDATE",
"SQL_LOW_PRIORITY_UPDATES", "SQL_MAX_JOIN_SIZE", "SQL_QUOTE_SHOW_CREATE", "SQL_SAFE_UPDATES", "SQL_SELECT_LIMIT", "SQL_SLAVE_SKIP_COUNTER",
"SQL_SMALL_RESULT", "SQL_WARNINGS", "SQL_CACHE", "SQL_NO_CACHE", "START", "STARTING", "STATUS", "STOP", "STORAGE",
"STRAIGHT_JOIN", "STRING", "STRIPED", "SUPER", "TABLE", "TABLES", "TEMPORARY", "TERMINATED", "THEN", "TO", "TRAILING", "TRANSACTIONAL", "TRUE",
"TRUNCATE", "TYPE", "TYPES", "UNCOMMITTED", "UNIQUE", "UNLOCK", "UNSIGNED", "USAGE", "USE", "USING", "VARIABLES",
"VIEW", "WHEN", "WITH", "WORK", "WRITE", "YEAR_MONTH",
}
var tokenReservedTopLevel = []string{
"SELECT", "FROM", "WHERE", "SET", "ORDER BY", "GROUP BY", "LIMIT", "DROP",
"VALUES", "UPDATE", "HAVING", "ADD", "AFTER", "ALTER TABLE", "DELETE FROM", "UNION ALL", "UNION", "EXCEPT", "INTERSECT",
}
var tokenFunction = []string{
"ABS", "ACOS", "ADDDATE", "ADDTIME", "AES_DECRYPT", "AES_ENCRYPT", "AREA", "ASBINARY", "ASCII", "ASIN", "ASTEXT", "ATAN", "ATAN2",
"AVG", "BDMPOLYFROMTEXT", "BDMPOLYFROMWKB", "BDPOLYFROMTEXT", "BDPOLYFROMWKB", "BENCHMARK", "BIN", "BIT_AND", "BIT_COUNT", "BIT_LENGTH",
"BIT_OR", "BIT_XOR", "BOUNDARY", "BUFFER", "CAST", "CEIL", "CEILING", "CENTROID", "CHAR", "CHARACTER_LENGTH", "CHARSET", "CHAR_LENGTH",
"COALESCE", "COERCIBILITY", "COLLATION", "COMPRESS", "CONCAT", "CONCAT_WS", "CONNECTION_ID", "CONTAINS", "CONV", "CONVERT", "CONVERT_TZ",
"CONVEXHULL", "COS", "COT", "COUNT", "CRC32", "CROSSES", "CURDATE", "CURRENT_DATE", "CURRENT_TIME", "CURRENT_TIMESTAMP", "CURRENT_USER",
"CURTIME", "DATABASE", "DATE", "DATEDIFF", "DATE_ADD", "DATE_DIFF", "DATE_FORMAT", "DATE_SUB", "DAY", "DAYNAME", "DAYOFMONTH", "DAYOFWEEK",
"DAYOFYEAR", "DECODE", "DEFAULT", "DEGREES", "DES_DECRYPT", "DES_ENCRYPT", "DIFFERENCE", "DIMENSION", "DISJOINT", "DISTANCE", "ELT", "ENCODE",
"ENCRYPT", "ENDPOINT", "ENVELOPE", "EQUALS", "EXP", "EXPORT_SET", "EXTERIORRING", "EXTRACT", "EXTRACTVALUE", "FIELD", "FIND_IN_SET", "FLOOR",
"FORMAT", "FOUND_ROWS", "FROM_DAYS", "FROM_UNIXTIME", "GEOMCOLLFROMTEXT", "GEOMCOLLFROMWKB", "GEOMETRYCOLLECTION", "GEOMETRYCOLLECTIONFROMTEXT",
"GEOMETRYCOLLECTIONFROMWKB", "GEOMETRYFROMTEXT", "GEOMETRYFROMWKB", "GEOMETRYN", "GEOMETRYTYPE", "GEOMFROMTEXT", "GEOMFROMWKB", "GET_FORMAT",
"GET_LOCK", "GLENGTH", "GREATEST", "GROUP_CONCAT", "GROUP_UNIQUE_USERS", "HEX", "HOUR", "IF", "IFNULL", "INET_ATON", "INET_NTOA", "INSERT", "INSTR",
"INTERIORRINGN", "INTERSECTION", "INTERSECTS", "INTERVAL", "ISCLOSED", "ISEMPTY", "ISNULL", "ISRING", "ISSIMPLE", "IS_FREE_LOCK", "IS_USED_LOCK",
"LAST_DAY", "LAST_INSERT_ID", "LCASE", "LEAST", "LEFT", "LENGTH", "LINEFROMTEXT", "LINEFROMWKB", "LINESTRING", "LINESTRINGFROMTEXT", "LINESTRINGFROMWKB",
"LN", "LOAD_FILE", "LOCALTIME", "LOCALTIMESTAMP", "LOCATE", "LOG", "LOG10", "LOG2", "LOWER", "LPAD", "LTRIM", "MAKEDATE", "MAKETIME", "MAKE_SET",
"MASTER_POS_WAIT", "MAX", "MBRCONTAINS", "MBRDISJOINT", "MBREQUAL", "MBRINTERSECTS", "MBROVERLAPS", "MBRTOUCHES", "MBRWITHIN", "MD5", "MICROSECOND",
"MID", "MIN", "MINUTE", "MLINEFROMTEXT", "MLINEFROMWKB", "MOD", "MONTH", "MONTHNAME", "MPOINTFROMTEXT", "MPOINTFROMWKB", "MPOLYFROMTEXT", "MPOLYFROMWKB",
"MULTILINESTRING", "MULTILINESTRINGFROMTEXT", "MULTILINESTRINGFROMWKB", "MULTIPOINT", "MULTIPOINTFROMTEXT", "MULTIPOINTFROMWKB", "MULTIPOLYGON",
"MULTIPOLYGONFROMTEXT", "MULTIPOLYGONFROMWKB", "NAME_CONST", "NULLIF", "NUMGEOMETRIES", "NUMINTERIORRINGS", "NUMPOINTS", "OCT", "OCTET_LENGTH",
"OLD_PASSWORD", "ORD", "OVERLAPS", "PASSWORD", "PERIOD_ADD", "PERIOD_DIFF", "PI", "POINT", "POINTFROMTEXT", "POINTFROMWKB", "POINTN", "POINTONSURFACE",
"POLYFROMTEXT", "POLYFROMWKB", "POLYGON", "POLYGONFROMTEXT", "POLYGONFROMWKB", "POSITION", "POW", "POWER", "QUARTER", "QUOTE", "RADIANS", "RAND",
"RELATED", "RELEASE_LOCK", "REPEAT", "REPLACE", "REVERSE", "RIGHT", "ROUND", "ROW_COUNT", "RPAD", "RTRIM", "SCHEMA", "SECOND", "SEC_TO_TIME",
"SESSION_USER", "SHA", "SHA1", "SIGN", "SIN", "SLEEP", "SOUNDEX", "SPACE", "SQRT", "SRID", "STARTPOINT", "STD", "STDDEV", "STDDEV_POP", "STDDEV_SAMP",
"STRCMP", "STR_TO_DATE", "SUBDATE", "SUBSTR", "SUBSTRING", "SUBSTRING_INDEX", "SUBTIME", "SUM", "SYMDIFFERENCE", "SYSDATE", "SYSTEM_USER", "TAN",
"TIME", "TIMEDIFF", "TIMESTAMP", "TIMESTAMPADD", "TIMESTAMPDIFF", "TIME_FORMAT", "TIME_TO_SEC", "TOUCHES", "TO_DAYS", "TRIM", "TRUNCATE", "UCASE",
"UNCOMPRESS", "UNCOMPRESSED_LENGTH", "UNHEX", "UNIQUE_USERS", "UNIX_TIMESTAMP", "UPDATEXML", "UPPER", "USER", "UTC_DATE", "UTC_TIME", "UTC_TIMESTAMP",
"UUID", "VARIANCE", "VAR_POP", "VAR_SAMP", "VERSION", "WEEK", "WEEKDAY", "WEEKOFYEAR", "WITHIN", "X", "Y", "YEAR", "YEARWEEK",
}
var tokenReservedNewLine = []string{
"LEFT OUTER JOIN", "RIGHT OUTER JOIN", "LEFT JOIN", "RIGHT JOIN", "OUTER JOIN", "INNER JOIN", "JOIN", "XOR", "OR", "AND",
}
var regBoundariesString string
var regReservedToplevelString string
var regReservedNewlineString string
var regReservedString string
var regFunctionString string
func init() {
var regs []string
regs = append(regs, tokenBoundaries...)
regBoundariesString = "(" + strings.Join(regs, "|") + ")"
regs = make([]string, 0)
for _, reg := range tokenReservedTopLevel {
regs = append(regs, regexp.QuoteMeta(reg))
}
regReservedToplevelString = "(" + strings.Join(regs, "|") + ")"
regs = make([]string, 0)
for _, reg := range tokenReservedNewLine {
regs = append(regs, regexp.QuoteMeta(reg))
}
regReservedNewlineString = "(" + strings.Join(regs, "|") + ")"
regs = make([]string, 0)
for _, reg := range tokenReserved {
regs = append(regs, regexp.QuoteMeta(reg))
}
regReservedString = "(" + strings.Join(regs, "|") + ")"
regs = make([]string, 0)
for _, reg := range tokenFunction {
regs = append(regs, regexp.QuoteMeta(reg))
}
regFunctionString = "(" + strings.Join(regs, "|") + ")"
}
// TokenString sqlparser tokens
var TokenString = map[int]string{
sqlparser.LEX_ERROR: "",
sqlparser.UNION: "union",
sqlparser.SELECT: "select",
sqlparser.STREAM: "stream",
sqlparser.INSERT: "insert",
sqlparser.UPDATE: "update",
sqlparser.DELETE: "delete",
sqlparser.FROM: "from",
sqlparser.WHERE: "where",
sqlparser.GROUP: "group",
sqlparser.HAVING: "having",
sqlparser.ORDER: "order",
sqlparser.BY: "by",
sqlparser.LIMIT: "limit",
sqlparser.OFFSET: "offset",
sqlparser.FOR: "for",
sqlparser.ALL: "all",
sqlparser.DISTINCT: "distinct",
sqlparser.AS: "as",
sqlparser.EXISTS: "exists",
sqlparser.ASC: "asc",
sqlparser.DESC: "desc",
sqlparser.INTO: "into",
sqlparser.DUPLICATE: "duplicate",
sqlparser.KEY: "key",
sqlparser.DEFAULT: "default",
sqlparser.SET: "set",
sqlparser.LOCK: "lock",
sqlparser.KEYS: "keys",
sqlparser.VALUES: "values",
sqlparser.LAST_INSERT_ID: "last_insert_id",
sqlparser.NEXT: "next",
sqlparser.VALUE: "value",
sqlparser.SHARE: "share",
sqlparser.MODE: "mode",
sqlparser.SQL_NO_CACHE: "sql_no_cache",
sqlparser.SQL_CACHE: "sql_cache",
sqlparser.JOIN: "join",
sqlparser.STRAIGHT_JOIN: "straight_join",
sqlparser.LEFT: "left",
sqlparser.RIGHT: "right",
sqlparser.INNER: "inner",
sqlparser.OUTER: "outer",
sqlparser.CROSS: "cross",
sqlparser.NATURAL: "natural",
sqlparser.USE: "use",
sqlparser.FORCE: "force",
sqlparser.ON: "on",
sqlparser.USING: "using",
sqlparser.ID: "id",
sqlparser.HEX: "hex",
sqlparser.STRING: "string",
sqlparser.INTEGRAL: "integral",
sqlparser.FLOAT: "float",
sqlparser.HEXNUM: "hexnum",
sqlparser.VALUE_ARG: "?",
sqlparser.LIST_ARG: ":",
sqlparser.COMMENT: "",
sqlparser.COMMENT_KEYWORD: "comment",
sqlparser.BIT_LITERAL: "bit_literal",
sqlparser.NULL: "null",
sqlparser.TRUE: "true",
sqlparser.FALSE: "false",
sqlparser.OR: "||",
sqlparser.AND: "&&",
sqlparser.NOT: "not",
sqlparser.BETWEEN: "between",
sqlparser.CASE: "case",
sqlparser.WHEN: "when",
sqlparser.THEN: "then",
sqlparser.ELSE: "else",
sqlparser.END: "end",
sqlparser.LE: "<",
sqlparser.GE: ">=",
sqlparser.NE: "<>",
sqlparser.NULL_SAFE_EQUAL: "<=>",
sqlparser.IS: "is",
sqlparser.LIKE: "like",
sqlparser.REGEXP: "regexp",
sqlparser.IN: "in",
sqlparser.SHIFT_LEFT: "<<",
sqlparser.SHIFT_RIGHT: ">>",
sqlparser.DIV: "div",
sqlparser.MOD: "mod",
sqlparser.UNARY: "unary",
sqlparser.COLLATE: "collate",
sqlparser.BINARY: "binary",
sqlparser.UNDERSCORE_BINARY: "_binary",
sqlparser.INTERVAL: "interval",
sqlparser.JSON_EXTRACT_OP: "->>",
sqlparser.JSON_UNQUOTE_EXTRACT_OP: "->",
sqlparser.CREATE: "create",
sqlparser.ALTER: "alter",
sqlparser.DROP: "drop",
sqlparser.RENAME: "rename",
sqlparser.ANALYZE: "analyze",
sqlparser.ADD: "add",
sqlparser.SCHEMA: "schema",
sqlparser.TABLE: "table",
sqlparser.INDEX: "index",
sqlparser.VIEW: "view",
sqlparser.TO: "to",
sqlparser.IGNORE: "ignore",
sqlparser.IF: "if",
sqlparser.UNIQUE: "unique",
sqlparser.PRIMARY: "primary",
sqlparser.COLUMN: "column",
sqlparser.CONSTRAINT: "constraint",
sqlparser.SPATIAL: "spatial",
sqlparser.FULLTEXT: "fulltext",
sqlparser.FOREIGN: "foreign",
sqlparser.SHOW: "show",
sqlparser.DESCRIBE: "describe",
sqlparser.EXPLAIN: "explain",
sqlparser.DATE: "date",
sqlparser.ESCAPE: "escape",
sqlparser.REPAIR: "repair",
sqlparser.OPTIMIZE: "optimize",
sqlparser.TRUNCATE: "truncate",
sqlparser.MAXVALUE: "maxvalue",
sqlparser.PARTITION: "partition",
sqlparser.REORGANIZE: "reorganize",
sqlparser.LESS: "less",
sqlparser.THAN: "than",
sqlparser.PROCEDURE: "procedure",
sqlparser.TRIGGER: "trigger",
sqlparser.VINDEX: "vindex",
sqlparser.VINDEXES: "vindexes",
sqlparser.STATUS: "status",
sqlparser.VARIABLES: "variables",
sqlparser.BEGIN: "begin",
sqlparser.START: "start",
sqlparser.TRANSACTION: "transaction",
sqlparser.COMMIT: "commit",
sqlparser.ROLLBACK: "rollback",
sqlparser.BIT: "bit",
sqlparser.TINYINT: "tinyint",
sqlparser.SMALLINT: "smallint",
sqlparser.MEDIUMINT: "mediumint",
sqlparser.INT: "int",
sqlparser.INTEGER: "integer",
sqlparser.BIGINT: "bigint",
sqlparser.INTNUM: "intnum",
sqlparser.REAL: "real",
sqlparser.DOUBLE: "double",
sqlparser.FLOAT_TYPE: "float_type",
sqlparser.DECIMAL: "decimal",
sqlparser.NUMERIC: "numeric",
sqlparser.TIME: "time",
sqlparser.TIMESTAMP: "timestamp",
sqlparser.DATETIME: "datetime",
sqlparser.YEAR: "year",
sqlparser.CHAR: "char",
sqlparser.VARCHAR: "varchar",
sqlparser.BOOL: "bool",
sqlparser.CHARACTER: "character",
sqlparser.VARBINARY: "varbinary",
sqlparser.NCHAR: "nchar",
sqlparser.TEXT: "text",
sqlparser.TINYTEXT: "tinytext",
sqlparser.MEDIUMTEXT: "mediumtext",
sqlparser.LONGTEXT: "longtext",
sqlparser.BLOB: "blob",
sqlparser.TINYBLOB: "tinyblob",
sqlparser.MEDIUMBLOB: "mediumblob",
sqlparser.LONGBLOB: "longblob",
sqlparser.JSON: "json",
sqlparser.ENUM: "enum",
sqlparser.GEOMETRY: "geometry",
sqlparser.POINT: "point",
sqlparser.LINESTRING: "linestring",
sqlparser.POLYGON: "polygon",
sqlparser.GEOMETRYCOLLECTION: "geometrycollection",
sqlparser.MULTIPOINT: "multipoint",
sqlparser.MULTILINESTRING: "multilinestring",
sqlparser.MULTIPOLYGON: "multipolygon",
sqlparser.NULLX: "nullx",
sqlparser.AUTO_INCREMENT: "auto_increment",
sqlparser.APPROXNUM: "approxnum",
sqlparser.SIGNED: "signed",
sqlparser.UNSIGNED: "unsigned",
sqlparser.ZEROFILL: "zerofill",
sqlparser.DATABASES: "databases",
sqlparser.TABLES: "tables",
sqlparser.NAMES: "names",
sqlparser.CHARSET: "charset",
sqlparser.GLOBAL: "global",
sqlparser.SESSION: "session",
sqlparser.CURRENT_TIMESTAMP: "current_timestamp",
sqlparser.DATABASE: "database",
sqlparser.CURRENT_DATE: "current_date",
sqlparser.CURRENT_TIME: "current_time",
sqlparser.LOCALTIME: "localtime",
sqlparser.LOCALTIMESTAMP: "localtimestamp",
sqlparser.UTC_DATE: "utc_date",
sqlparser.UTC_TIME: "utc_time",
sqlparser.UTC_TIMESTAMP: "utc_timestamp",
sqlparser.REPLACE: "replace",
sqlparser.CONVERT: "convert",
sqlparser.CAST: "cast",
sqlparser.SUBSTR: "substr",
sqlparser.SUBSTRING: "substring",
sqlparser.GROUP_CONCAT: "group_concat",
sqlparser.SEPARATOR: "separator",
sqlparser.VSCHEMA: "vschema",
sqlparser.SEQUENCE: "sequence",
sqlparser.MATCH: "match",
sqlparser.AGAINST: "against",
sqlparser.BOOLEAN: "boolean",
sqlparser.LANGUAGE: "language",
sqlparser.WITH: "with",
sqlparser.QUERY: "query",
sqlparser.EXPANSION: "expansion",
sqlparser.UNUSED: "",
}
// 这个变更从vitess更新过来，如果vitess新开了一个关键字这里也要同步开
var mySQLKeywords = map[string]string{
"add": "ADD",
"against": "AGAINST",
"all": "ALL",
"alter": "ALTER",
"analyze": "ANALYZE",
"and": "AND",
"as": "AS",
"asc": "ASC",
"auto_increment": "AUTO_INCREMENT",
"begin": "BEGIN",
"between": "BETWEEN",
"bigint": "BIGINT",
"binary": "BINARY",
"_binary": "UNDERSCORE_BINARY",
"bit": "BIT",
"blob": "BLOB",
"bool": "BOOL",
"boolean": "BOOLEAN",
"by": "BY",
"case": "CASE",
"cast": "CAST",
"char": "CHAR",
"character": "CHARACTER",
"charset": "CHARSET",
"collate": "COLLATE",
"column": "COLUMN",
"comment": "COMMENT_KEYWORD",
"commit": "COMMIT",
"constraint": "CONSTRAINT",
"convert": "CONVERT",
"substr": "SUBSTR",
"substring": "SUBSTRING",
"create": "CREATE",
"cross": "CROSS",
"current_date": "CURRENT_DATE",
"current_time": "CURRENT_TIME",
"current_timestamp": "CURRENT_TIMESTAMP",
"database": "DATABASE",
"databases": "DATABASES",
"date": "DATE",
"datetime": "DATETIME",
"decimal": "DECIMAL",
"default": "DEFAULT",
"delete": "DELETE",
"desc": "DESC",
"describe": "DESCRIBE",
"distinct": "DISTINCT",
"div": "DIV",
"double": "DOUBLE",
"drop": "DROP",
"duplicate": "DUPLICATE",
"else": "ELSE",
"end": "END",
"enum": "ENUM",
"escape": "ESCAPE",
"exists": "EXISTS",
"explain": "EXPLAIN",
"expansion": "EXPANSION",
"false": "FALSE",
"float": "FLOAT_TYPE",
"for": "FOR",
"force": "FORCE",
"foreign": "FOREIGN",
"from": "FROM",
"fulltext": "FULLTEXT",
"geometry": "GEOMETRY",
"geometrycollection": "GEOMETRYCOLLECTION",
"global": "GLOBAL",
"grant": "GRANT",
"group": "GROUP",
"group_concat": "GROUP_CONCAT",
"having": "HAVING",
"if": "IF",
"ignore": "IGNORE",
"in": "IN",
"index": "INDEX",
"inner": "INNER",
"insert": "INSERT",
"int": "INT",
"integer": "INTEGER",
"interval": "INTERVAL",
"into": "INTO",
"is": "IS",
"join": "JOIN",
"json": "JSON",
"key": "KEY",
"keys": "KEYS",
"key_block_size": "KEY_BLOCK_SIZE",
"language": "LANGUAGE",
"last_insert_id": "LAST_INSERT_ID",
"left": "LEFT",
"less": "LESS",
"like": "LIKE",
"limit": "LIMIT",
"linestring": "LINESTRING",
"localtime": "LOCALTIME",
"localtimestamp": "LOCALTIMESTAMP",
"lock": "LOCK",
"longblob": "LONGBLOB",
"longtext": "LONGTEXT",
"match": "MATCH",
"maxvalue": "MAXVALUE",
"mediumblob": "MEDIUMBLOB",
"mediumint": "MEDIUMINT",
"mediumtext": "MEDIUMTEXT",
"mod": "MOD",
"mode": "MODE",
"multilinestring": "MULTILINESTRING",
"multipoint": "MULTIPOINT",
"multipolygon": "MULTIPOLYGON",
"names": "NAMES",
"natural": "NATURAL",
"nchar": "NCHAR",
"next": "NEXT",
"not": "NOT",
"null": "NULL",
"numeric": "NUMERIC",
"offset": "OFFSET",
"on": "ON",
"optimize": "OPTIMIZE",
"or": "OR",
"order": "ORDER",
"outer": "OUTER",
"partition": "PARTITION",
"point": "POINT",
"polygon": "POLYGON",
"primary": "PRIMARY",
"procedure": "PROCEDURE",
"query": "QUERY",
"real": "REAL",
"regexp": "REGEXP",
"rename": "RENAME",
"reorganize": "REORGANIZE",
"repair": "REPAIR",
"replace": "REPLACE",
"revoke": "REVOKE",
"right": "RIGHT",
"rlike": "REGEXP",
"rollback": "ROLLBACK",
"schema": "SCHEMA",
"select": "SELECT",
"separator": "SEPARATOR",
"session": "SESSION",
"set": "SET",
"share": "SHARE",
"show": "SHOW",
"signed": "SIGNED",
"smallint": "SMALLINT",
"spatial": "SPATIAL",
"sql_cache": "SQL_CACHE",
"sql_no_cache": "SQL_NO_CACHE",
"start": "START",
"status": "STATUS",
"straight_join": "STRAIGHT_JOIN",
"stream": "STREAM",
"table": "TABLE",
"tables": "TABLES",
"text": "TEXT",
"than": "THAN",
"then": "THEN",
"time": "TIME",
"timestamp": "TIMESTAMP",
"tinyblob": "TINYBLOB",
"tinyint": "TINYINT",
"tinytext": "TINYTEXT",
"to": "TO",
"transaction": "TRANSACTION",
"trigger": "TRIGGER",
"true": "TRUE",
"truncate": "TRUNCATE",
"union": "UNION",
"unique": "UNIQUE",
"unsigned": "UNSIGNED",
"update": "UPDATE",
"use": "USE",
"using": "USING",
"utc_date": "UTC_DATE",
"utc_time": "UTC_TIME",
"utc_timestamp": "UTC_TIMESTAMP",
"values": "VALUES",
"variables": "VARIABLES",
"varbinary": "VARBINARY",
"varchar": "VARCHAR",
"vindex": "VINDEX",
"vindexes": "VINDEXES",
"view": "VIEW",
"vitess_keyspaces": "VITESS_KEYSPACES",
"vitess_shards": "VITESS_SHARDS",
"vitess_tablets": "VITESS_TABLETS",
"vschema_tables": "VSCHEMA_TABLES",
"when": "WHEN",
"where": "WHERE",
"with": "WITH",
"year": "YEAR",
"zerofill": "ZEROFILL",
}
// Token 基本定义
type Token struct {
Type int
Val string
i int
}
// Tokenizer 用于初始化 token，区别于 Tokenize 函数，这个函数使用 vitess 的切词方式
func Tokenizer(sql string) []Token {
var tokens []Token
tkn := sqlparser.NewStringTokenizer(sql)
typ, val := tkn.Scan()
for typ != 0 {
if val != nil {
tokens = append(tokens, Token{Type: typ, Val: string(val)})
} else {
if typ > 255 {
if v, ok := TokenString[typ]; ok {
tokens = append(tokens, Token{
Type: typ,
Val: v,
})
} else {
tokens = append(tokens, Token{
Type: typ,
Val: "",
})
}
} else {
tokens = append(tokens, Token{
Type: typ,
Val: fmt.Sprintf("%c", typ),
})
}
}
typ, val = tkn.Scan()
}
return tokens
}
// IsMysqlKeyword 判断是否是关键字
func IsMysqlKeyword(name string) bool {
_, ok := mySQLKeywords[strings.ToLower(strings.TrimSpace(name))]
return ok
}
// getNextToken 从 buf 中获取 token
func getNextToken(buf string, previous Token) Token {
var typ int // TOKEN_TYPE
// Whitespace
whiteSpaceReg := regexp.MustCompile(`^\s+`)
if whiteSpaceReg.MatchString(buf) {
return Token{
Type: TokenTypeWhitespace,
Val: " ",
}
}
// Comment (#, --, /**/)
if buf[0] == '#' || (len(buf) > 1 && (buf[:2] == "--" || buf[:2] == "/*")) {
var last int
if buf[0] == '-' || buf[0] == '#' {
// Comment until end of line
last = strings.Index(buf, "\n")
typ = TokenTypeComment
} else {
// Comment until closing comment tag
last = strings.Index(buf[2:], "*/") + 2
typ = TokenTypeBlockComment
}
if last <= 0 {
last = len(buf)
}
return Token{
Type: typ,
Val: buf[:last],
}
}
// Quoted String
if buf[0] == '"' || buf[0] == '\'' || buf[0] == '`' || buf[0] == '[' {
var typ int
switch buf[0] {
case '`', '[':
typ = TokenTypeBacktickQuote
default:
typ = TokenTypeQuote
}
return Token{
Type: typ,
Val: getQuotedString(buf),
}
}
// User-defined Variable
if (buf[0] == '@' || buf[0] == ':') && len(buf) > 1 {
ret := Token{
Type: TokenTypeVariable,
Val: "",
}
if buf[1] == '"' || buf[1] == '\'' || buf[1] == '`' {
// If the variable name is quoted
ret.Val = string(buf[0]) + getQuotedString(buf[1:])
} else {
// Non-quoted variable name
varReg := regexp.MustCompile(`^(` + string(buf[0]) + `[a-zA-Z0-9\._\$]+)`)
if varReg.MatchString(buf) {
ret.Val = varReg.FindString(buf)
}
}
if ret.Val != "" {
return ret
}
}
// Number(decimal, binary, hex...)
numReg := regexp.MustCompile(`^([0-9]+(\.[0-9]+)?|0x[0-9a-fA-F]+|0b[01]+)($|\s|"'` + "`|" + regBoundariesString + ")")
if numReg.MatchString(buf) {
return Token{
Type: TokenTypeNumber,
Val: numReg.FindString(buf),
}
}
// Boundary Character(punctuation and symbols)
boundaryReg := regexp.MustCompile(`^(` + regBoundariesString + `)`)
if boundaryReg.MatchString(buf) {
return Token{
Type: TokenTypeBoundary,
Val: boundaryReg.FindString(buf),
}
}
sqlUpper := strings.ToUpper(buf)
// A reserved word cannot be preceded by a '.'
// this makes it so in "mytable.from", "from" is not considered a reserved word
if previous.Val != "." {
// Top Level Reserved Word
reservedToplevelReg := regexp.MustCompile(`^(` + regReservedToplevelString + `)($|\s|` + regBoundariesString + `)`)
if reservedToplevelReg.MatchString(sqlUpper) {
return Token{
Type: TokenTypeReservedToplevel,
Val: reservedToplevelReg.FindString(sqlUpper),
}
}
// Newline Reserved Word
reservedNewlineReg := regexp.MustCompile(`^(` + regReservedNewlineString + `)($|\s|` + regBoundariesString + `)`)
if reservedNewlineReg.MatchString(sqlUpper) {
return Token{
Type: TokenTypeReservedNewline,
Val: reservedNewlineReg.FindString(sqlUpper),
}
}
// Other Reserved Word
reservedReg := regexp.MustCompile(`^(` + regReservedString + `)($|\s|` + regBoundariesString + `)`)
if reservedNewlineReg.MatchString(sqlUpper) {
return Token{
Type: TokenTypeReserved,
Val: reservedReg.FindString(sqlUpper),
}
}
}
// function
// A function must be succeeded by '('
// this makes it so "count(" is considered a function, but "count" alone is not
functionReg := regexp.MustCompile(`^(` + regFunctionString + `)($|\s|` + regBoundariesString + `)`)
if functionReg.MatchString(sqlUpper) {
return Token{
Type: TokenTypeReserved,
Val: functionReg.FindString(sqlUpper),
}
}
// Non reserved word
noReservedReg := regexp.MustCompile(`(.*?)($|\s|["'` + "`]|" + regBoundariesString + `)`)
if noReservedReg.MatchString(buf) {
return Token{
Type: TokenTypeWord,
Val: noReservedReg.FindString(buf),
}
}
return Token{}
}
// getQuotedString 获取quote
func getQuotedString(buf string) string {
// This checks for the following patterns:
// 1. backtick quoted string using `` to escape
// 2. double quoted string using "" or \" to escape
// 3. single quoted string using '' or \' to escape
start := string(buf[0])
switch start {
case "\"", "`", "'":
reg := fmt.Sprintf(`(^%s[^%s\\]*(?:\\.[^%s\\]*)*(%s|$))+`, start, start, start, start)
quotedReg := regexp.MustCompile(reg)
if quotedReg.MatchString(buf) {
buf = quotedReg.FindString(buf)
} else {
buf = ""
}
default:
buf = ""
}
return buf
}
// Tokenize 序列化 token，区别于 Tokenizer 函数，这个函数是 soar built-in 实现的切词
func Tokenize(sql string) []Token {
var token Token
var tokenLength int
var tokens []Token
tokenCache := make(map[string]Token)
// Used to make sure the string keeps shrinking on each iteration
oldStringLen := len(sql) + 1
currentLength := len(sql)
for currentLength > 0 {
// If the string stopped shrinking, there was a problem
if oldStringLen <= currentLength {
tokens = []Token{
{
Type: TokenTypeError,
Val: sql,
},
}
return tokens
}
oldStringLen = currentLength
cacheKey := ""
// Determine if we can use caching
if currentLength >= maxCachekeySize {
cacheKey = sql[:maxCachekeySize]
}
// See if the token is already cached
if _, ok := tokenCache[cacheKey]; ok {
// Retrieve from cache
token = tokenCache[cacheKey]
tokenLength = len(token.Val)
cacheHits = cacheHits + 1
} else {
// Get the next token and the token type
token = getNextToken(sql, token)
tokenLength = len(token.Val)
cacheMisses = cacheMisses + 1
// If the token is shorter than the max length, store it in cache
if cacheKey != "" && tokenLength < maxCachekeySize {
tokenCache[cacheKey] = token
}
}
tokens = append(tokens, token)
// Advance the string
sql = sql[tokenLength:]
currentLength = currentLength - tokenLength
}
return tokens
}
// Compress compress sql
// this method is inspired by eversql.com
func Compress(sql string) string {
regLineTab := regexp.MustCompile(`(?i)([\n\t])`)
regSpace := regexp.MustCompile(`\s\s+`)
sql = regSpace.ReplaceAllString(regLineTab.ReplaceAllString(sql, " "), " ")
return sql
}
// SplitStatement SQL切分
// return 1. original sql, 2. remove comment sql, 3. left over buf
func SplitStatement(buf []byte, delimiter []byte) (string, string, []byte) {
var singleLineComment bool
var multiLineComment bool
var quoted bool
var quoteRune byte
var sql string
for i := 0; i < len(buf); i++ {
b := buf[i]
// single line comment
if b == '-' {
if !quoted && i+2 < len(buf) && buf[i+1] == '-' && buf[i+2] == ' ' {
singleLineComment = true
i = i + 2
continue
}
if !quoted && i+2 < len(buf) && i == 0 && buf[i+1] == '-' && (buf[i+2] == '\n' || buf[i+2] == '\r') {
sql = "--\n"
break
}
}
if b == '#' {
if !multiLineComment && !quoted && !singleLineComment {
singleLineComment = true
continue
}
}
// new line end single line comment
if b == '\r' || b == '\n' {
if singleLineComment {
sql = string(buf[:i])
singleLineComment = false
if strings.HasPrefix(strings.TrimSpace(sql), "--") ||
strings.HasPrefix(strings.TrimSpace(sql), "#") {
// just comment, query start with '--', '#'
break
}
// comment in multi-line sql
continue
}
continue
}
// multi line comment
// https://dev.mysql.com/doc/refman/8.0/en/comments.html
// https://dev.mysql.com/doc/refman/8.0/en/optimizer-hints.html
if b == '/' && i+1 < len(buf) && buf[i+1] == '*' {
if !multiLineComment && !singleLineComment && !quoted &&
(buf[i+2] != '!' && buf[i+2] != '+') {
i = i + 2
multiLineComment = true
continue
}
}
if b == '*' && i+1 < len(buf) && buf[i+1] == '/' {
if multiLineComment && !quoted && !singleLineComment {
i = i + 2
multiLineComment = false
// '/*comment*/'
if i == len(buf) {
sql = string(buf[:i])
}
// '/*comment*/;', 'select 1/*comment*/;'
if string(buf[i:]) == string(delimiter) {
sql = string(buf)
}
continue
}
}
// quoted string
switch b {
case '`', '\'', '"':
if i > 1 && buf[i-1] != '\\' {
if quoted && b == quoteRune {
quoted = false
quoteRune = 0
} else {
// check if first time found quote
if quoteRune == 0 {
quoted = true
quoteRune = b
}
}
}
}
// delimiter
if !quoted && !singleLineComment && !multiLineComment {
eof := true
for k, c := range delimiter {
if len(buf) > i+k && buf[i+k] != c {
eof = false
}
}
if eof {
i = i + len(delimiter)
sql = string(buf[:i])
break
}
}
// ended of buf
if i == len(buf)-1 {
sql = string(buf)
}
}
orgSQL := string(buf[:len(sql)])
buf = buf[len(sql):]
return orgSQL, strings.TrimSuffix(sql, string(delimiter)), buf
}
// LeftNewLines cal left new lines in space
func LeftNewLines(buf []byte) int {
newLines := 0
for _, b := range buf {
if !unicode.IsSpace(rune(b)) {
break
}
if b == byte('\n') {
newLines++
}
}
return newLines
}
// NewLines cal all new lines
func NewLines(buf []byte) int {
newLines := 0
for _, b := range buf {
if b == byte('\n') {
newLines++
}
}
return newLines
}
// QueryType get query type such as SELECT/INSERT/DELETE/CREATE/ALTER
func QueryType(sql string) string {
tokens := Tokenize(sql)
for _, token := range tokens {
// use strings.Fields for 'ALTER TABLE' token split
for _, tk := range strings.Fields(strings.TrimSpace(token.Val)) {
if val, ok := mySQLKeywords[strings.ToLower(tk)]; ok {
return val
}
}
}
return "UNKNOWN"
}
================================================
FILE: ast/token_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ast
import (
"fmt"
"testing"
"github.com/XiaoMi/soar/common"
"github.com/kr/pretty"
)
func TestTokenize(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
err := common.GoldenDiff(func() {
for _, sql := range common.TestSQLs {
fmt.Println(sql)
fmt.Println(Tokenize(sql))
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestTokenizer(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
sqls := []string{
"select -- comment 1",
"select c1,c2,c3 from t1,t2 join t3 on t1.c1=t2.c1 and t1.c3=t3.c1 where id>1000",
"select sourcetable, if(f.lastcontent = ?, f.lastupdate, f.lastcontent) as lastactivity, f.totalcount as activity, type.class as type, (f.nodeoptions & ?) as nounsubscribe from node as f inner join contenttype as type on type.contenttypeid = f.contenttypeid inner join subscribed as sd on sd.did = f.nodeid and sd.userid = ? union all select f.name as title, f.userid as keyval, ? as sourcetable, ifnull(f.lastpost, f.joindate) as lastactivity, f.posts as activity, ? as type, ? as nounsubscribe from user as f inner join userlist as ul on ul.relationid = f.userid and ul.userid = ? where ul.type = ? and ul.aq = ? order by title limit ?",
"select c1 from t1 where id>=1000", // test ">="
"select SQL_CALC_FOUND_ROWS col from tbl where id>1000",
"SELECT * FROM tb WHERE id=?;",
"SELECT * FROM tb WHERE id is null;",
"SELECT * FROM tb WHERE id is not null;",
"SELECT * FROM tb WHERE id between 1 and 3;",
"alter table inventory add index idx_store_film` (`store_id`,`film_id`);",
`UPDATE xxx SET c1=' LOGGER.error(""); }' WHERE id = 2 ;`,
`update tb set status = 1 where id = 1;`, // SQL 中包含 non-broken-space
}
err := common.GoldenDiff(func() {
for _, sql := range sqls {
pretty.Println(Tokenizer(sql))
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestGetQuotedString(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
var str = []string{
`"hello world"`,
"`hello world`",
`'hello world'`,
"hello world",
`'hello \'world'`,
`"hello \"wor\"ld"`,
`"hello \"world"`,
`""`,
`''`,
"``",
`'hello 'world'`,
`"hello "world"`,
`' LOGGER.error(""); }'`,
}
err := common.GoldenDiff(func() {
for _, s := range str {
fmt.Printf("orignal: %s\nquoted: %s\n", s, getQuotedString(s))
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestCompress(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
err := common.GoldenDiff(func() {
for _, sql := range common.TestSQLs {
fmt.Println(sql)
fmt.Println(Compress(sql))
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestFormat(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
err := common.GoldenDiff(func() {
for _, sql := range common.TestSQLs {
fmt.Println(sql)
fmt.Println(format(sql))
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestSplitStatement(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
bufs := [][]byte{
[]byte("select * from test;hello"), // 0
[]byte("select 'asd;fas', col from test;hello"), // 1
[]byte("-- select * from test;hello"), // 2
[]byte("#select * from test;hello"), // 3
[]byte("select * /*comment*/from test;hello"), // 4
[]byte("select * /*comment;*/from test;hello"), // 5
[]byte(`select * /*comment
;*/
from test;hello`), // 6
[]byte(`select * from test`), // 7
// https://github.com/XiaoMi/soar/issues/66
[]byte(`/*comment*/`), // 8
[]byte(`/*comment*/;`), // 9
[]byte(`--`), // 10
[]byte(`-- comment`), // 11
[]byte(`# comment`), // 12
// https://github.com/XiaoMi/soar/issues/116
[]byte(`select
*
-- comment
from tb
where col = 1`), // 13
[]byte(`select
* --
from tb
where col = 1`), // 14
[]byte(`select
* #
from tb
where col = 1`), // 15
[]byte(`select
*
--
from tb
where col = 1`), // 16
[]byte(`select * from
-- comment
tb;
select col from tb where col = 1;`), // 17
// https://github.com/XiaoMi/soar/issues/120
[]byte(`
-- comment
select col from tb;
select col from tb;
`), // 18
[]byte(`INSERT /*+ SET_VAR(foreign_key_checks=OFF) */ INTO t2 VALUES(2);`), // 19
[]byte(`select /*!50000 1,*/ 1;`), // 20
[]byte(`UPDATE xxx SET c1=' LOGGER.error(""); }' WHERE id = 2 ;`), // 21
[]byte("UPDATE `xxx` SET aaa='a;' WHERE `id` = 15;"), // 22
[]byte("UPDATE `xxx` SET aaa='a -- b' WHERE `id` = 15; UPDATE `xxx` SET aaa='c -- d' WHERE `id` = 16;"), // 23
// []byte(`/* comment here */ SET MAX_JOIN_SIZE=#`), // 24
}
// \G 分隔符
buf2s := [][]byte{
[]byte("select * from test\\Ghello"), // 0
[]byte("select 'hello\\Gworld', col from test\\Ghello"), // 1
[]byte("-- select * from test\\Ghello"), // 2
[]byte("#select * from test\\Ghello"), // 3
[]byte("select * /*comment*/from test\\Ghello"), // 4
[]byte("select * /*comment;*/from test\\Ghello"), // 5
[]byte(`select * /*comment
\\G*/
from test\\Ghello`), // 6
}
//// single sql test
//b, t1, t2 := SplitStatement(bufs[23], []byte(";"))
//fmt.Println("buf: ", b, "sql: ", t1, "left: ", string(t2))
//return
err := common.GoldenDiff(func() {
for i, buf := range bufs {
sql, _, _ := SplitStatement(buf, []byte(";"))
fmt.Println(i, sql)
}
for i, buf := range buf2s {
sql, _, _ := SplitStatement(buf, []byte("\\G"))
fmt.Println(i, sql)
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestLeftNewLines(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
bufs := [][]byte{
[]byte(`
select * from test;hello`),
[]byte(`select * /*comment
;*/
from test;hello`),
[]byte(`select * from test`),
}
err := common.GoldenDiff(func() {
for _, buf := range bufs {
fmt.Println(LeftNewLines(buf))
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestNewLines(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
bufs := [][]byte{
[]byte(`
select * from test;hello`),
[]byte(`select * /*comment
;*/
from test;hello`),
[]byte(`select * from test`),
}
err := common.GoldenDiff(func() {
for _, buf := range bufs {
fmt.Println(NewLines(buf))
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestQueryType(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
var testSQLs = []string{
` select 1`,
`/*comment*/ select 1`,
`(select 1)`,
`grant select on *.* to user@'localhost'`,
`REVOKE INSERT ON *.* FROM 'jeffrey'@'localhost';`,
}
// fmt.Println(testSQLs[len(testSQLs)-1])
// fmt.Println(QueryType(testSQLs[len(testSQLs)-1]))
// return
err := common.GoldenDiff(func() {
for _, buf := range append(testSQLs, common.TestSQLs...) {
fmt.Println(QueryType(buf))
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
================================================
FILE: ast/vitess.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ast
import (
"encoding/json"
"github.com/XiaoMi/soar/common"
"github.com/kr/pretty"
"vitess.io/vitess/go/vt/sqlparser"
)
// PrintPrettyVitessStmtNode print vitess AST struct data
func PrintPrettyVitessStmtNode(sql string) {
tree, err := sqlparser.Parse(sql)
if err != nil {
common.Log.Warning(err.Error())
} else {
_, err = pretty.Println(tree)
common.LogIfWarn(err, "")
}
}
// VitessStmtNode2JSON vitess AST tree into json format
func VitessStmtNode2JSON(sql string) string {
var str string
tree, err := sqlparser.Parse(sql)
if err != nil {
common.Log.Warning(err.Error())
} else {
b, err := json.MarshalIndent(tree, "", " ")
if err != nil {
common.Log.Error(err.Error())
} else {
str = string(b)
}
}
return str
}
================================================
FILE: ast/vitess_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ast
import (
"fmt"
"testing"
"github.com/XiaoMi/soar/common"
)
func TestPrintPrettyVitessStmtNode(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
sqls := []string{
`select 1`,
`select * f`, // syntax error case
}
err := common.GoldenDiff(func() {
for _, sql := range sqls {
PrintPrettyVitessStmtNode(sql)
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func TestVitessStmtNode2JSON(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
sqls := []string{
`select 1`,
`select * f`, // syntax error case
}
err := common.GoldenDiff(func() {
for _, sql := range sqls {
fmt.Println(VitessStmtNode2JSON(sql))
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
================================================
FILE: cmd/soar/doc.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
// Package soar is the main program of SOAR
package main
================================================
FILE: cmd/soar/soar.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package main
import (
"encoding/json"
"fmt"
"os"
"strings"
"github.com/XiaoMi/soar/advisor"
"github.com/XiaoMi/soar/ast"
"github.com/XiaoMi/soar/common"
"github.com/XiaoMi/soar/database"
"github.com/XiaoMi/soar/env"
"github.com/go-sql-driver/mysql"
"github.com/kr/pretty"
"github.com/percona/go-mysql/query"
)
func main() {
// 全局变量
var err error
var sql string // 单条评审指定的 sql 或 explain
var currentDB string // 当前 SQL 使用的 database
sqlCounter := 1 // SQL 计数器
lineCounter := 1 // 行计数器
var alterSQLs []string // 待评审的 SQL 中所有 ALTER 请求
alterTableTimes := make(map[string]int) // 待评审的 SQL 中同一经表 ALTER 请求计数器
suggestMerged := make(map[string]map[string]advisor.Rule) // 优化建议去重, key 为 sql 的 fingerprint.ID
var suggestStr []string // string 形式格式化之后的优化建议，用于 -report-type json
tables := make(map[string][]string) // SQL 使用的库表名
// 配置文件&命令行参数解析
initConfig()
// 命令行帮助工具，如 -list-report-types, -check-config等。
if isContinue, exitCode := helpTools(); !isContinue {
os.Exit(exitCode)
}
// 环境初始化，连接检查线上环境+构建测试环境
vEnv, rEnv := env.BuildEnv()
// 使用 -cleanup-test-database 命令手动清理残余的 optimizer_xxx 数据库
if common.Config.CleanupTestDatabase {
vEnv.CleanupTestDatabase()
return
}
// 如果使用到测试环境，在这里环境清理
if common.Config.DropTestTemporary {
defer vEnv.CleanUp()
}
// 当程序卡死的时候，或者由于某些原因程序没有退出，可以通过捕获信号量的形式让程序优雅退出并且清理测试环境
common.HandleSignal(func() {
shutdown(vEnv, rEnv)
})
// 对指定的库表进行索引重复检查
if common.Config.ReportType == "duplicate-key-checker" {
dupKeySuggest := advisor.DuplicateKeyChecker(rEnv)
_, str := advisor.FormatSuggest("", currentDB, common.Config.ReportType, dupKeySuggest)
if str == "" {
fmt.Printf("%s/%s 未发现重复索引\n", common.Config.OnlineDSN.Addr, common.Config.OnlineDSN.Schema)
} else {
fmt.Println(str)
}
return
}
// 读入待优化 SQL ，当配置文件或命令行参数未指定 SQL 时从管道读取
buf := initQuery(common.Config.Query)
lineCounter += ast.LeftNewLines([]byte(buf))
buf = strings.TrimSpace(buf)
// remove bom from file header
var bom []byte
buf, bom = common.RemoveBOM([]byte(buf))
if isContinue, exitCode := reportTool(buf, bom); !isContinue {
os.Exit(exitCode)
}
// 逐条SQL给出优化建议
for ; ; sqlCounter++ {
var id string // fingerprint.ID
heuristicSuggest := make(map[string]advisor.Rule) // 启发式建议
expSuggest := make(map[string]advisor.Rule) // EXPLAIN 解读
idxSuggest := make(map[string]advisor.Rule) // 索引建议
proSuggest := make(map[string]advisor.Rule) // Profiling 信息
traceSuggest := make(map[string]advisor.Rule) // Trace 信息
mysqlSuggest := make(map[string]advisor.Rule) // MySQL 返回的 ERROR 信息
if buf == "" {
common.Log.Debug("Ending, buf: '%s', sql: '%s'", buf, sql)
break
}
// 查询请求切分
orgSQL, sql, bufBytes := ast.SplitStatement([]byte(buf), []byte(common.Config.Delimiter))
// lineCounter
lc := ast.NewLines([]byte(orgSQL))
// leftLineCounter
llc := ast.LeftNewLines([]byte(orgSQL))
lineCounter += llc
if len(buf) == len(bufBytes) {
// 防止切分死循环，当剩余的内容和原 SQL 相同时直接清空 buf
buf = ""
orgSQL = string(bufBytes)
sql = orgSQL
} else {
buf = string(bufBytes)
}
// 去除无用的备注和空格
sql = database.RemoveSQLComments(sql)
if sql == "" {
common.Log.Debug("empty query or comment, buf: %s", buf)
continue
}
common.Log.Debug("main loop SQL: %s", sql)
// +++++++++++++++++++++小工具集[开始]+++++++++++++++++++++++{
fingerprint := strings.TrimSpace(query.Fingerprint(sql))
// SQL 签名
id = query.Id(fingerprint)
currentDB = env.CurrentDB(sql, currentDB)
switch common.Config.ReportType {
case "fingerprint":
// SQL 指纹
if common.Config.Verbose {
fmt.Printf("-- ID: %s\n", id)
}
fmt.Println(fingerprint)
continue
case "pretty":
// SQL 美化
fmt.Println(ast.Pretty(sql, "builtin") + common.Config.Delimiter)
continue
case "compress":
// SQL 压缩
fmt.Println(ast.Compress(sql) + common.Config.Delimiter)
continue
case "ast":
// print vitess AST data struct
ast.PrintPrettyVitessStmtNode(sql)
continue
case "ast-json":
// print vitess SQL AST into json format
fmt.Println(ast.VitessStmtNode2JSON(sql))
continue
case "tiast":
// print TiDB AST data struct
ast.PrintPrettyStmtNode(sql, "", "")
continue
case "tiast-json":
// print TiDB SQL AST into json format
fmt.Println(ast.StmtNode2JSON(sql, "", ""))
continue
case "tokenize":
// SQL 切词
_, err = pretty.Println(ast.Tokenize(sql))
common.LogIfWarn(err, "")
continue
default:
// 建议去重，减少评审整个文件耗时
// TODO: 由于 a = 11 和 a = '11' 的 fingerprint 相同，这里一旦跳过即无法检查有些建议了，如： ARG.003
if _, ok := suggestMerged[id]; ok {
// `use ?` 不可以去重，去重后将导致无法切换数据库
if !strings.HasPrefix(fingerprint, "use") {
continue
}
}
// 黑名单中的SQL不给建议
if advisor.InBlackList(fingerprint) {
// `use ?` 不可以出现在黑名单中
if !strings.HasPrefix(fingerprint, "use") {
continue
}
}
}
tables[id] = ast.SchemaMetaInfo(sql, currentDB)
// +++++++++++++++++++++小工具集[结束]+++++++++++++++++++++++}
// +++++++++++++++++++++语法检查[开始]+++++++++++++++++++++++{
q, syntaxErr := advisor.NewQuery4Audit(sql)
stmt := q.Stmt
// 如果语法检查出错则不需要给优化建议
if syntaxErr != nil {
errContent := fmt.Sprintf("At SQL %d : %v", sqlCounter, syntaxErr)
common.Log.Warning(errContent)
if common.Config.OnlySyntaxCheck || common.Config.ReportType == "rewrite" ||
common.Config.ReportType == "query-type" {
fmt.Println(errContent)
os.Exit(1)
}
// tidb parser 语法检查给出的建议 ERR.000
mysqlSuggest["ERR.000"] = advisor.RuleMySQLError("ERR.000", syntaxErr)
}
// 如果只想检查语法直接跳过后面的步骤
if common.Config.OnlySyntaxCheck {
continue
}
// +++++++++++++++++++++语法检查[结束]+++++++++++++++++++++++}
switch common.Config.ReportType {
case "tables":
continue
case "query-type":
// query type by first key word
fmt.Println(ast.QueryType(sql))
continue
}
// +++++++++++++++++++++启发式规则建议[开始]+++++++++++++++++++++++{
common.Log.Debug("start of heuristic advisor Query: %s", q.Query)
for item, rule := range advisor.HeuristicRules {
// 去除忽略的建议检查
okFunc := (*advisor.Query4Audit).RuleOK
if !advisor.IsIgnoreRule(item) && &rule.Func != &okFunc {
r := rule.Func(q)
if r.Item == item {
heuristicSuggest[item] = r
}
}
}
common.Log.Debug("end of heuristic advisor Query: %s", q.Query)
// +++++++++++++++++++++启发式规则建议[结束]+++++++++++++++++++++++}
// +++++++++++++++++++++索引优化建议[开始]+++++++++++++++++++++++{
// 如果配置了索引建议过滤规则，不进行索引优化建议
// 在配置文件 ignore-rules 中添加 'IDX.*' 即可屏蔽索引优化建议
common.Log.Debug("start of index advisor Query: %s", q.Query)
if !advisor.IsIgnoreRule("IDX.") {
if vEnv.BuildVirtualEnv(rEnv, q.Query) {
idxAdvisor, err := advisor.NewAdvisor(vEnv, *rEnv, *q)
if err != nil || (idxAdvisor == nil && vEnv.Error == nil) {
if idxAdvisor == nil {
// 如果 SQL 是 DDL 语句，则返回的 idxAdvisor 为 nil，可以忽略不处理
// TODO alter table add index 语句检查索引是否已经存在
common.Log.Debug("idxAdvisor by pass Query: %s", q.Query)
} else {
common.Log.Warning("advisor.NewAdvisor Error: %v", err)
}
} else {
// 创建环境时没有出现错误，生成索引建议
if vEnv.Error == nil {
idxSuggest = idxAdvisor.IndexAdvise().Format()
// 依赖数据字典的启发式建议
for i, r := range idxAdvisor.HeuristicCheck(*q) {
heuristicSuggest[i] = r
}
} else {
// 根据错误号输出建议
switch vEnv.Error.(*mysql.MySQLError).Number {
case 1061:
idxSuggest["IDX.001"] = advisor.Rule{
Item: "IDX.001",
Severity: "L2",
Summary: "索引名称已存在",
Content: strings.Trim(strings.Split(vEnv.Error.Error(), ":")[1], " "),
Case: sql,
}
default:
// vEnv.VEnvBuild 阶段给出的 ERROR 是 ERR.001
delete(mysqlSuggest, "ERR.000")
mysqlSuggest["ERR.001"] = advisor.RuleMySQLError("ERR.001", vEnv.Error)
common.Log.Error("BuildVirtualEnv DDL Execute Error : %v", vEnv.Error)
}
}
}
} else {
common.Log.Error("vEnv.BuildVirtualEnv Error: prepare SQL '%s' in vEnv failed.", q.Query)
}
}
common.Log.Debug("end of index advisor Query: %s", q.Query)
// +++++++++++++++++++++索引优化建议[结束]+++++++++++++++++++++++}
// +++++++++++++++++++++EXPLAIN 建议[开始]+++++++++++++++++++++++{
// 如果未配置 Online 或 Test 无法给 Explain 建议
common.Log.Debug("start of explain Query: %s", q.Query)
if !common.Config.OnlineDSN.Disable && !common.Config.TestDSN.Disable {
// 因为 EXPLAIN 依赖数据库环境，所以把这段逻辑放在启发式建议和索引建议后面
if common.Config.Explain {
// 执行 EXPLAIN
explainInfo, err := rEnv.Explain(q.Query,
database.ExplainType[common.Config.ExplainType],
database.ExplainFormatType[common.Config.ExplainFormat])
if err != nil {
// 线上环境执行失败才到测试环境 EXPLAIN，比如在用户提供建表语句及查询语句的场景
common.Log.Warn("rEnv.Explain Warn: %v", err)
explainInfo, err = vEnv.Explain(q.Query,
database.ExplainType[common.Config.ExplainType],
database.ExplainFormatType[common.Config.ExplainFormat])
if err != nil {
// EXPLAIN 阶段给出的 ERROR 是 ERR.002
mysqlSuggest["ERR.002"] = advisor.RuleMySQLError("ERR.002", err)
common.Log.Error("vEnv.Explain Error: %v", err)
}
}
// 分析 EXPLAIN 结果
if explainInfo != nil {
expSuggest = advisor.ExplainAdvisor(explainInfo)
} else {
common.Log.Warn("rEnv&vEnv.Explain explainInfo nil, SQL: %s", q.Query)
}
}
}
common.Log.Debug("end of explain Query: %s", q.Query)
// +++++++++++++++++++++ EXPLAIN 建议[结束]+++++++++++++++++++++++}
// +++++++++++++++++++++ Profiling [开始]+++++++++++++++++++++++++{
common.Log.Debug("start of profiling Query: %s", q.Query)
if common.Config.Profiling {
res, err := vEnv.Profiling(q.Query)
if err == nil {
proSuggest["PRO.001"] = advisor.Rule{
Item: "PRO.001",
Severity: "L0",
Content: database.FormatProfiling(res),
}
} else {
common.Log.Error("Profiling Error: %v", err)
}
}
common.Log.Debug("end of profiling Query: %s", q.Query)
// +++++++++++++++++++++ Profiling [结束]++++++++++++++++++++++++++}
// +++++++++++++++++++++ Trace [开始]+++++++++++++++++++++++++{
common.Log.Debug("start of trace Query: %s", q.Query)
if common.Config.Trace {
res, err := vEnv.Trace(q.Query)
if err == nil {
traceSuggest["TRA.001"] = advisor.Rule{
Item: "TRA.001",
Severity: "L0",
Content: database.FormatTrace(res),
}
} else {
common.Log.Error("Trace Error: %v", err)
}
}
common.Log.Debug("end of trace Query: %s", q.Query)
// +++++++++++++++++++++Trace [结束]++++++++++++++++++++++++++}
// +++++++++++++++++++++SQL 重写[开始]+++++++++++++++++++++++++{
common.Log.Debug("start of rewrite Query: %s", q.Query)
if common.Config.ReportType == "rewrite" {
if strings.HasPrefix(strings.TrimSpace(strings.ToLower(sql)), "create") ||
strings.HasPrefix(strings.TrimSpace(strings.ToLower(sql)), "alter") ||
strings.HasPrefix(strings.TrimSpace(strings.ToLower(sql)), "rename") {
// 依赖上下文件的 SQL 重写，如：多条 ALTER SQL 合并
// vitess 对 DDL 语法的支持不好，大部分 DDL 会语法解析出错，但即使出错了还是会生成一个 stmt 而且里面的 db.table 还是准确的。
alterSQLs = append(alterSQLs, sql)
alterTbl := ast.AlterAffectTable(stmt)
if alterTbl != "" && alterTbl != "dual" {
if _, ok := alterTableTimes[alterTbl]; ok {
heuristicSuggest["ALT.002"] = advisor.HeuristicRules["ALT.002"]
alterTableTimes[alterTbl] = alterTableTimes[alterTbl] + 1
} else {
alterTableTimes[alterTbl] = 1
}
}
} else {
// 其他不依赖上下文件的 SQL 重写
rw := ast.NewRewrite(sql)
if rw == nil {
// 都到这一步了 sql 不会语法不正确，因此 rw 一般不会为 nil
common.Log.Critical("NewRewrite nil point error, SQL: %s", sql)
os.Exit(1)
}
// SQL 转写需要的源信息采集，如果没有配置环境则只做有限改写
meta := ast.GetMeta(rw.Stmt, nil)
rw.Columns = vEnv.GenTableColumns(meta)
// 执行定义好的 SQL 重写规则
rw.Rewrite()
fmt.Println(strings.TrimSpace(rw.NewSQL))
}
}
common.Log.Debug("end of rewrite Query: %s", q.Query)
// +++++++++++++++++++++ SQL 重写[结束]++++++++++++++++++++++++++}
// +++++++++++++++++++++打印单条 SQL 优化建议[开始]++++++++++++++++++++++++++{
common.Log.Debug("start of print suggestions, Query: %s", q.Query)
if strings.HasPrefix(fingerprint, "use") {
continue
}
sug, str := advisor.FormatSuggest(q.Query, currentDB, common.Config.ReportType, heuristicSuggest, idxSuggest, expSuggest, proSuggest, traceSuggest, mysqlSuggest)
suggestMerged[id] = sug
switch common.Config.ReportType {
case "json":
suggestStr = append(suggestStr, str)
case "tables":
case "duplicate-key-checker":
case "rewrite":
case "lint":
for _, s := range strings.Split(str, "\n") {
// ignore empty output
if strings.TrimSpace(s) == "" {
continue
}
if common.Config.Query != "" {
if _, err = os.Stat(common.Config.Query); err == nil {
fmt.Printf("%s:%d:%s\n", common.Config.Query, lineCounter, s)
} else {
fmt.Printf("null:%d:%s\n", lineCounter, s)
}
} else {
fmt.Printf("stdin:%d:%s\n", lineCounter, s)
}
}
lineCounter += lc - llc
case "html":
fmt.Println(common.Markdown2HTML(str))
default:
fmt.Println(str)
}
common.Log.Debug("end of print suggestions, Query: %s", q.Query)
// +++++++++++++++++++++打印单条 SQL 优化建议[结束]++++++++++++++++++++++++++}
}
// 同一张表的多条 ALTER 语句合并为一条
if ast.RewriteRuleMatch("mergealter") {
for _, v := range ast.MergeAlterTables(alterSQLs...) {
fmt.Println(strings.TrimSpace(v))
}
return
}
// 以 JSON 格式化输出
if common.Config.ReportType == "json" {
fmt.Println("[\n", strings.Join(suggestStr, ",\n"), "\n]")
}
// 以 JSON 格式输出 SQL 影响的库表名
if common.Config.ReportType == "tables" {
js, err := json.MarshalIndent(tables, "", " ")
if err == nil {
fmt.Println(string(js))
} else {
common.Log.Error("FormatSuggest json.Marshal Error: %v", err)
}
return
}
verboseInfo()
}
================================================
FILE: cmd/soar/soar_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package main
import (
"flag"
"fmt"
"os"
"path/filepath"
"runtime"
"testing"
"github.com/XiaoMi/soar/common"
)
var update = flag.Bool("update", false, "update .golden files")
func TestMain(m *testing.M) {
// 初始化 init
if common.DevPath == "" {
_, file, _, _ := runtime.Caller(0)
common.DevPath, _ = filepath.Abs(filepath.Dir(filepath.Join(file, ".."+string(filepath.Separator))))
}
common.BaseDir = common.DevPath
err := common.ParseConfig("")
common.LogIfError(err, "init ParseConfig")
common.Log.Debug("mysql_test init")
_ = update // check if var success init
// 分割线
flag.Parse()
m.Run()
// 环境清理
//
}
func Test_Main(_ *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
common.Config.OnlineDSN.Disable = true
common.Config.LogLevel = 0
common.Config.Query = "select * syntaxError"
main()
common.Config.Query = "select * from film;alter table city add index idx_country_id(country_id);"
main()
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func Test_Main_More(_ *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
common.Config.LogLevel = 0
common.Config.Profiling = true
common.Config.Explain = true
common.Config.Query = "select * from film where country_id = 1;use sakila;alter table city add index idx_country_id(country_id);"
orgRerportType := common.Config.ReportType
for _, typ := range []string{
"json", "html", "markdown", "fingerprint", "compress", "pretty", "rewrite",
"ast", "tiast", "ast-json", "tiast-json", "tokenize", "lint", "tables", "query-type",
} {
common.Config.ReportType = typ
main()
}
common.Config.ReportType = orgRerportType
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func Test_Main_initQuery(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
// direct query
query := initQuery("select 1")
if query != "select 1" {
t.Errorf("want 'select 1', got %s", query)
}
// read from file
initQuery(common.DevPath + "/README.md")
orgStdin := os.Stdin
tmpStdin, err := os.Open(common.DevPath + "/VERSION")
if err != nil {
t.Error(err)
}
os.Stdin = tmpStdin
fmt.Println(initQuery(""))
os.Stdin = orgStdin
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func Test_Main_reportTool(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
orgRerportType := common.Config.ReportType
types := []string{"html", "md2html", "explain-digest", "chardet", "remove-comment"}
for _, tp := range types {
common.Config.ReportType = tp
fmt.Println(reportTool(tp, []byte{}))
}
common.Config.ReportType = orgRerportType
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func Test_Main_helpTools(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
orgConfig := common.CheckConfig
common.CheckConfig = true
helpTools()
common.CheckConfig = orgConfig
orgConfig = common.PrintVersion
common.PrintVersion = true
helpTools()
common.PrintVersion = orgConfig
orgConfig = common.PrintConfig
common.PrintConfig = true
helpTools()
common.PrintConfig = orgConfig
orgConfig = common.Config.ListHeuristicRules
common.Config.ListHeuristicRules = true
helpTools()
common.Config.ListHeuristicRules = orgConfig
orgConfig = common.Config.ListRewriteRules
common.Config.ListRewriteRules = true
helpTools()
common.Config.ListRewriteRules = orgConfig
orgConfig = common.Config.ListTestSqls
common.Config.ListTestSqls = true
helpTools()
common.Config.ListTestSqls = orgConfig
orgConfig = common.Config.ListReportTypes
common.Config.ListReportTypes = true
helpTools()
common.Config.ListReportTypes = orgConfig
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
func Test_Main_verboseInfo(t *testing.T) {
common.Log.Debug("Entering function: %s", common.GetFunctionName())
orgVerbose := common.Config.Verbose
common.Config.Verbose = true
err := common.GoldenDiff(func() {
// Syntax check OK
orgSyntaxCheck := common.Config.OnlySyntaxCheck
common.Config.OnlySyntaxCheck = true
verboseInfo()
common.Config.OnlySyntaxCheck = orgSyntaxCheck
// MySQL environment verbose info
orgTestDSNDisable := common.Config.TestDSN.Disable
common.Config.TestDSN.Disable = true
verboseInfo()
common.Config.TestDSN.Disable = orgTestDSNDisable
orgOnlineDSNDisable := common.Config.OnlineDSN.Disable
common.Config.OnlineDSN.Disable = true
verboseInfo()
common.Config.OnlineDSN.Disable = orgOnlineDSNDisable
}, t.Name(), update)
if err != nil {
t.Error(err)
}
common.Config.Verbose = orgVerbose
common.Log.Debug("Exiting function: %s", common.GetFunctionName())
}
================================================
FILE: cmd/soar/testdata/Test_Main.golden
================================================
# Query: 16219655761820A2
★ ★ ★ ★ ★ 100分
```sql
SELECT
1
```
## ✔️
================================================
FILE: cmd/soar/testdata/Test_Main_verboseInfo.golden
================================================
Syntax check OK!
MySQL environment verbose info
* test-dsn: 127.0.0.1:3306 is disable, please check log.
* online-dsn: 127.0.0.1:3306 is disable, please check log.
MySQL environment verbose info
* test-dsn: 127.0.0.1:3306 is disable, please check log.
* online-dsn: 127.0.0.1:3306 is disable, please check log.
================================================
FILE: cmd/soar/tool.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package main
import (
"fmt"
"io/ioutil"
"os"
"path/filepath"
"strings"
"github.com/XiaoMi/soar/advisor"
"github.com/XiaoMi/soar/ast"
"github.com/XiaoMi/soar/common"
"github.com/XiaoMi/soar/database"
"github.com/XiaoMi/soar/env"
)
// initConfig load config from default->file->cmdFlag
func initConfig() {
// 更新 binary 文件所在路径为 BaseDir
ex, err := os.Executable()
if err != nil {
panic(err)
}
common.BaseDir = filepath.Dir(ex)
for i, c := range os.Args {
// 如果指定了 -config, 它必须是第一个参数
if strings.HasPrefix(c, "-config") && i != 1 {
fmt.Println("-config must be the first arg")
os.Exit(1)
}
// 等号两边请不要加空格
if c == "=" {
// -config = soar.yaml not support
fmt.Println("wrong format, no space between '=', eg: -config=soar.yaml")
os.Exit(1)
}
}
// 加载配置文件，处理命令行参数
err = common.ParseConfig(common.ArgConfig())
// 检查配置文件及命令行参数是否正确
if common.CheckConfig && err != nil {
fmt.Println(err.Error())
os.Exit(1)
}
// 更新 HeuristicRules 中与配置相关的文字
advisor.InitHeuristicRules()
common.LogIfWarn(err, "")
}
// checkConfig for `-check-config` flag
// if error found return non-zero, no error return zero
func checkConfig() int {
// TestDSN connection check
connTest, err := database.NewConnector(common.Config.TestDSN)
if err != nil {
fmt.Println("test-dsn:", common.Config.TestDSN.Addr, err.Error())
return 1
}
testVersion, err := connTest.Version()
if err != nil && !common.Config.TestDSN.Disable {
fmt.Println("test-dsn:", connTest, err.Error())
return 1
}
if common.Config.Verbose {
if err == nil {
fmt.Println("test-dsn", connTest, "Version:", testVersion)
} else {
fmt.Println("test-dsn", common.Config.TestDSN)
}
}
if !connTest.HasAllPrivilege() {
fmt.Printf("test-dsn: %s, need all privileges", common.FormatDSN(common.Config.TestDSN))
return 1
}
// OnlineDSN connection check
connOnline, err := database.NewConnector(common.Config.OnlineDSN)
if err != nil {
fmt.Println("test-dsn:", common.Config.OnlineDSN.Addr, err.Error())
return 1
}
onlineVersion, err := connOnline.Version()
if err != nil && !common.Config.OnlineDSN.Disable {
fmt.Println("online-dsn:", connOnline, err.Error())
return 1
}
if common.Config.Verbose {
if err == nil {
fmt.Println("online-dsn", connOnline, "Version:", onlineVersion)
} else {
fmt.Println("online-dsn", common.Config.OnlineDSN)
}
}
if !connOnline.HasSelectPrivilege() {
fmt.Printf("online-dsn: %s, need all privileges", common.FormatDSN(common.Config.OnlineDSN))
return 1
}
return 0
}
// helpTools help tools in cmd flags
func helpTools() (isContinue bool, exitCode int) {
// environment error check, eg. MySQL password error
if common.CheckConfig {
return false, checkConfig()
}
// 打印 SOAR 版本信息
if common.PrintVersion {
common.SoarVersion()
return false, 0
}
// 打印已加载配置的各配置项，检查配置是否生效
if common.PrintConfig {
common.PrintConfiguration()
return false, 0
}
// 打印支持启发式建议
if common.Config.ListHeuristicRules {
advisor.ListHeuristicRules(advisor.HeuristicRules)
return false, 0
}
// 打印支持的 SQL 重写规则
if common.Config.ListRewriteRules {
ast.ListRewriteRules(ast.RewriteRules)
return false, 0
}
// 打印所有的测试 SQL
if common.Config.ListTestSqls {
advisor.ListTestSQLs()
return false, 0
}
// 打印支持的 report-type
if common.Config.ListReportTypes {
common.ListReportTypes()
return false, 0
}
return true, 0
}
// reportTool tools in report type
func reportTool(sql string, bom []byte) (isContinue bool, exitCode int) {
switch common.Config.ReportType {
case "html":
// HTML 格式输入 CSS 加载
fmt.Println(common.MarkdownHTMLHeader())
return true, 0
case "md2html":
// markdown2html 转换小工具
fmt.Println(common.MarkdownHTMLHeader())
fmt.Println(common.Markdown2HTML(sql))
return false, 0
case "explain-digest":
// 当用户输入为 EXPLAIN 信息，只对 Explain 信息进行分析
// 注意： 这里只能处理一条 SQL 的 EXPLAIN 信息，用户一次反馈多条 SQL 的 EXPLAIN 信息无法处理
advisor.DigestExplainText(sql)
return false, 0
case "chardet":
// Get charset of input
charset := common.CheckCharsetByBOM(bom)
if charset == "" {
charset = common.Chardet([]byte(sql))
}
fmt.Println(charset)
return false, 0
case "remove-comment":
fmt.Println(database.RemoveSQLComments(sql))
return false, 0
default:
return true, 0
}
}
// initQuery
func initQuery(query string) string {
// 读入待优化 SQL ，当配置文件或命令行参数未指定 SQL 时从管道读取
if query == "" {
// check stdin is pipe or terminal
// https://stackoverflow.com/questions/22744443/check-if-there-is-something-to-read-on-stdin-in-golang
stat, err := os.Stdin.Stat()
if stat == nil {
common.Log.Critical("os.Stdin.Stat Error: %v", err)
os.Exit(1)
}
if (stat.Mode() & os.ModeCharDevice) != 0 {
// stdin is from a terminal
fmt.Println("Args format error, use --help see how to use it!")
os.Exit(1)
}
// read from pipe
var data []byte
data, err = ioutil.ReadAll(os.Stdin)
if err != nil {
common.Log.Critical("ioutil.ReadAll Error: %v", err)
}
common.Log.Debug("initQuery get query from os.Stdin")
return string(data)
}
if _, err := os.Stat(query); err == nil {
var data []byte
data, err = ioutil.ReadFile(query)
if err != nil {
common.Log.Critical("ioutil.ReadFile Error: %v", err)
}
common.Log.Debug("initQuery get query from file: %s", query)
return string(data)
}
return query
}
func shutdown(vEnv *env.VirtualEnv, rEnv *database.Connector) {
if common.Config.DropTestTemporary {
vEnv.CleanUp()
}
err := vEnv.Conn.Close()
common.LogIfWarn(err, "")
err = rEnv.Conn.Close()
common.LogIfWarn(err, "")
os.Exit(0)
}
func verboseInfo() {
if !common.Config.Verbose {
return
}
// syntax check verbose mode, add output for success!
if common.Config.OnlySyntaxCheck {
fmt.Println("Syntax check OK!")
return
}
switch common.Config.ReportType {
case "markdown":
if common.Config.TestDSN.Disable || common.Config.OnlineDSN.Disable {
fmt.Println("MySQL environment verbose info")
// TestDSN
if common.Config.TestDSN.Disable {
fmt.Println("* test-dsn:", common.Config.TestDSN.Addr, "is disable, please check log.")
}
// OnlineDSN
if common.Config.OnlineDSN.Disable {
fmt.Println("* online-dsn:", common.Config.OnlineDSN.Addr, "is disable, please check log.")
}
}
}
}
================================================
FILE: common/cases.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
// TestSQLs 测试SQL大集合
var TestSQLs []string
func init() {
// 所有的SQL都要以分号结尾，-list-test-sqls 参数会打印这个 list，以分号结尾可方便测试
// 如：./soar -list-test-sql | ./soar
TestSQLs = []string{
// single equality
"SELECT * FROM film WHERE length = 86;", // index(length)
"SELECT * FROM film WHERE length IS NULL;", // index(length)
"SELECT * FROM film HAVING title = 'abc';", // 无法使用索引
// single inequality
"SELECT * FROM sakila.film WHERE length >= 60;", // any of <, <=, >=, >; but not <>, !=, IS NOT NULL"
"SELECT * FROM sakila.film WHERE length >= '60';", // Implicit Conversion
"SELECT * FROM film WHERE length BETWEEN 60 AND 84;",
"SELECT * FROM film WHERE title LIKE 'AIR%';", // but not LIKE '%blah'",
"SELECT * FROM film WHERE title IS NOT NULL;",
// multiple equalities
"SELECT * FROM film WHERE length = 114 and title = 'ALABAMA DEVIL';", // index(title,length) or index(length,title)",
// equality and inequality
"SELECT * FROM film WHERE length > 100 and title = 'ALABAMA DEVIL';", // index(title, length)",
// multiple inequality
"SELECT * FROM film WHERE length > 100 and language_id < 10 and title = 'xyz';", // index(d, b) or index(d, c) 依赖数据",
"SELECT * FROM film WHERE length > 100 and language_id < 10;", // index(b) or index(c)",
// GROUP BY
"SELECT release_year, sum(length) FROM film WHERE length = 123 AND language_id = 1 GROUP BY release_year;", // INDEX(length, language_id, release_year) or INDEX(language_id, length, release_year)",
"SELECT release_year, sum(length) FROM film WHERE length >= 123 GROUP BY release_year;", // INDEX(length)",
"SELECT release_year, language_id, sum(length) FROM film GROUP BY release_year, language_id;", // INDEX(release_year, language_id) (no WHERE)",
"SELECT release_year, sum(length) FROM film WHERE length = 123 GROUP BY release_year,(length+language_id);", // INDEX(length) expression in GROUP BY, so no use including even release_year.",
"SELECT release_year, sum(film_id) FROM film GROUP BY release_year;", // INDEX(`release_year`)
"SELECT * FROM address GROUP BY address,district;", // INDEX(address, district)
"SELECT title FROM film WHERE ABS(language_id) = 3 GROUP BY title;", // 无法使用索引
// ORDER BY
"SELECT language_id FROM film WHERE length = 123 GROUP BY release_year ORDER BY language_id;", // INDEX(length, release_year) should have stopped with Step 2b",
"SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year;", // INDEX(length, release_year) the release_year will be used for both GROUP BY and ORDER BY",
"SELECT * FROM film WHERE length = 123 ORDER BY release_year ASC, language_id DESC;", // INDEX(length) mixture of ASC and DESC.",
"SELECT release_year FROM film WHERE length = 123 GROUP BY release_year ORDER BY release_year LIMIT 10;", // INDEX(length, release_year)",
"SELECT * FROM film WHERE length = 123 ORDER BY release_year LIMIT 10;", // INDEX(length, release_year)",
"SELECT * FROM film ORDER BY release_year LIMIT 10;", // 不能单独给release_year加索引
"SELECT film_id FROM film ORDER BY release_year LIMIT 10;", // TODO: INDEX(release_year)，film_id 是主键查询列满足索引覆盖的情况才会使用到 release_year 索引
"SELECT * FROM film WHERE length > 100 ORDER BY length LIMIT 10;", // INDEX(length) This "range" is compatible with ORDER BY
"SELECT * FROM film WHERE length < 100 ORDER BY length LIMIT 10;", // INDEX(length) also works
"SELECT * FROM customer WHERE address_id in (224,510) ORDER BY last_name;", // INDEX(address_id)
"SELECT * FROM film WHERE release_year = 2016 AND length != 1 ORDER BY title;", // INDEX(`release_year`, `length`, `title`)
// "Covering" Rows
"SELECT title FROM film WHERE release_year = 1995;", // INDEX(release_year, title)",
"SELECT title, replacement_cost FROM film WHERE language_id = 5 AND length = 70;", // INDEX(language_id, length, title, replacement_cos film ), title, replacement_cost顺序无关，language_id, length顺序视散粒度情况.
"SELECT title FROM film WHERE language_id > 5 AND length > 70;", // INDEX(language_id, length, title) language_id or length first (that's as far as the Algorithm goes), then the other two fields afterwards.
// equalities and sort
"SELECT * FROM film WHERE length = 100 and title = 'xyz' ORDER BY release_year;", // 依赖数据特征，index(length, title, release_year) or index(title, length, release_year)需要评估
// inequality and sort
"SELECT * FROM film WHERE length > 100 and title = 'xyz' ORDER BY release_year;", // 依赖数据特征， index(title, release_year)，index(title, length)需要评估
"SELECT * FROM film WHERE length > 100 ORDER BY release_year;", // 依赖数据特征， index(length)，index(release_year)需要评估
// Join
// 内连接 INNER JOIN
// 在mysql中，inner join...on , join...on , 逗号...WHERE ，cross join...on是一样的含义。
// 但是在标准SQL中，它们并不等价，标准 SQL 中 INNER JOIN 与 ON共同使用, CROSS JOIN用于其他情况。
// 逗号不支持 on 和 using 语法, 逗号的优先级要低于INNER JOIN, CROSS JOIN, LEFT JOIN
// ON子句的语法格式为：tb1.col1 = tb2.col2列名可以不同，筛选连接后的结果，两表的对应列值相同才在结果集中。
// 当模式设计对联接表的列采用了相同的命名样式时，就可以使用 USING 语法来简化 ON 语法
// join, inner join, cross join等价，优先选择小结果集条件表为驱动表
// left [outer] join左表为驱动表
// right [outer] join右表为驱动表
// 驱动表连接列如果没其他条件可以不考虑加索引，反正是需要foreach
// 被驱动表连接列需要加索引。即:left [outer] join的右表连接列需要加索引，right [outer] join的左表连接列需要加索引，inner join结果集较大表的连接列需要加索引
// 其他索引添加算法与单表索引优化算法相同
// 总结：被驱动表列需要添加索引
// 建议：将无索引的表通常作为驱动表
"SELECT * FROM city a INNER JOIN country b ON a.country_id=b.country_id;",
// 左外连接 LEFT [OUTER] JOIN
"SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id;",
// 右外连接 RIGHT [OUTER] JOIN
"SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;",
// 左连接
"SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;",
// 右连接
"SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL;",
// 全连接 FULL JOIN 因为在mysql中并不支持，所以我们用union实现
"SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id " +
"UNION " +
"SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id;",
// 两张表中不共同满足的数据集
"SELECT * FROM city a RIGHT JOIN country b ON a.country_id=b.country_id WHERE a.last_update IS NULL " +
"UNION " +
"SELECT * FROM city a LEFT JOIN country b ON a.country_id=b.country_id WHERE b.last_update IS NULL;",
// NATURAL JOIN 默认是同名字段完全匹配的INNER JOIN
"SELECT country_id, last_update FROM city NATURAL JOIN country;",
// NATURAL LEFT JOIN
"SELECT country_id, last_update FROM city NATURAL LEFT JOIN country;",
// NATURAL RIGHT JOIN
"SELECT country_id, last_update FROM city NATURAL RIGHT JOIN country;",
// STRAIGHT_JOIN 实际上与内连接 INNER JOIN 表现完全一致，
// 不同的是使用了 STRAIGHT_JOIN 后指定表载入的顺序，city 先于 country 载入
"SELECT a.country_id, a.last_update FROM city a STRAIGHT_JOIN country b ON a.country_id=b.country_id;",
// SEMI JOIN
// 半连接： 当一张表在另一张表找到匹配的记录之后，半连接（semi-join）返回第一张表中的记录。
// 与条件连接相反，即使在右节点中找到几条匹配的记录，左节点的表也只会返回一条记录。
// 另外，右节点的表一条记录也不会返回。半连接通常使用 IN 或 EXISTS 作为连接条件
"SELECT a.address, a.postal_code FROM sakila.address a WHERE a.city_id IN (SELECT c.city_id FROM sakila.city c);",
// Delayed Join
// https://www.percona.com/blog/2007/04/06/using-delayed-join-to-optimize-count-and-limit-queries/
`SELECT city FROM( SELECT city_id FROM city WHERE city = "A Corua (La Corua)" ORDER BY last_update DESC LIMIT 50, 10) I JOIN city ON (I.city_id = city.city_id) JOIN country ON (country.country_id = city.country_id) ORDER BY city DESC;`,
// DELETE
"DELETE city, country FROM city INNER JOIN country using (country_id) WHERE city.city_id = 1;",
"DELETE city FROM city LEFT JOIN country ON city.country_id = country.country_id WHERE country.country IS NULL;",
"DELETE a1, a2 FROM city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;",
"DELETE FROM a1, a2 USING city AS a1 INNER JOIN country AS a2 WHERE a1.country_id=a2.country_id;",
"DELETE FROM film WHERE length > 100;",
// UPDATE
"UPDATE city INNER JOIN country USING(country_id) SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;",
"UPDATE city INNER JOIN country ON city.country_id = country.country_id INNER JOIN address ON city.city_id = address.city_id SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.city_id=10;",
"UPDATE city, country SET city.city = 'Abha', city.last_update = '2006-02-15 04:45:25', country.country = 'Afghanistan' WHERE city.country_id = country.country_id AND city.city_id=10;",
"UPDATE film SET length = 10 WHERE language_id = 20;",
// INSERT
"INSERT INTO city (country_id) SELECT country_id FROM country;",
"INSERT INTO city (country_id) VALUES (1),(2),(3);",
"INSERT INTO city (country_id) VALUES (10);",
"INSERT INTO city (country_id) SELECT 10 FROM DUAL;",
// REPLACE
"REPLACE INTO city (country_id) SELECT country_id FROM country;",
"REPLACE INTO city (country_id) VALUES (1),(2),(3);",
"REPLACE INTO city (country_id) VALUES (10);",
"REPLACE INTO city (country_id) SELECT 10 FROM DUAL;",
// DEPTH
"SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM ( SELECT film_id FROM film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film ) film;",
// SUBQUERY
"SELECT * FROM film WHERE language_id = (SELECT language_id FROM language LIMIT 1);",
// "SELECT COUNT(*) /* no hint */ FROM t2 WHERE NOT EXISTS (SELECT * FROM t3 WHERE ROW(5 * t2.s1, 77) = (SELECT 50, 11 * s1 FROM t4 UNION SELECT 50, 77 FROM (SELECT * FROM t5) AS t5 ) ) ;",
"SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;",
"SELECT * FROM (SELECT * FROM actor WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE') t WHERE last_update='2006-02-15 04:34:33' and last_name='CHASE' GROUP BY first_name;",
"SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id;",
"SELECT * FROM city i left JOIN country o ON i.city_id=o.country_id WHERE o.country_id is null union SELECT * FROM city i right JOIN country o ON i.city_id=o.country_id WHERE i.city_id is null;",
"SELECT first_name,last_name,email FROM customer STRAIGHT_JOIN address ON customer.address_id=address.address_id;",
"SELECT ID,name FROM (SELECT address FROM customer_list WHERE SID=1 order by phone limit 50,10) a JOIN customer_list l ON (a.address=l.address) JOIN city c ON (c.city=l.city) order by phone desc;",
// function in conditions
"SELECT * FROM film WHERE date(last_update)='2006-02-15';",
"SELECT last_update FROM film GROUP BY date(last_update);",
"SELECT last_update FROM film order by date(last_update);",
// CLA.004
"SELECT description FROM film WHERE description IN('NEWS','asd') GROUP BY description;",
// ALTER TABLE ADD INDEX
// 已经存在索引的列应该提醒索引已存在
"alter table address add index idx_city_id(city_id);",
"alter table inventory add index `idx_store_film` (`store_id`,`film_id`);",
"alter table inventory add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`),add index `idx_store_film` (`store_id`,`film_id`);",
// https://github.com/XiaoMi/soar/issues/47
`SELECT DATE_FORMAT(t.last_update, '%Y-%m-%d'), COUNT(DISTINCT (t.city)) FROM city t WHERE t.last_update > '2018-10-22 00:00:00' AND t.city LIKE '%Chrome%' AND t.city = 'eip' GROUP BY DATE_FORMAT(t.last_update, '%Y-%m-%d') ORDER BY DATE_FORMAT(t.last_update, '%Y-%m-%d');`,
// https://github.com/XiaoMi/soar/issues/17
"create table hello.t (id int unsigned);",
// https://github.com/XiaoMi/soar/issues/146
"select * from tb where data >= '';",
// https://github.com/XiaoMi/soar/issues/163
"alter table tb alter column id drop default;",
// explain extra info "Select tables optimized away"
"select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film where last_update > '2016-03-27 02:01:01') as d;",
"select maxId, minId from (select max(film_id) maxId, min(film_id) minId from film) as d;",
}
}
================================================
FILE: common/chardet.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"github.com/kr/pretty"
"github.com/saintfish/chardet"
)
// Chardet get best match charset
func Chardet(buf []byte) string {
// check character set by file BOM
charset := CheckCharsetByBOM(buf)
if charset != "" {
return charset
}
// use chardet pkg check file charset
charset = "unknown"
var confidence int
detector := chardet.NewTextDetector()
// detector.DetectBest is unstable
// when the confidence value are equally, the best detect charset will be random
result, err := detector.DetectAll(buf)
if err != nil {
return charset
}
Log.Debug("Chardet DetectAll Result: %s", pretty.Sprint(result))
// SOAR's main user speak Chinese, GB-18030, UTF-8 are higher suggested
for _, r := range result {
if confidence > r.Confidence && r.Confidence != 0 {
return charset
}
confidence = r.Confidence
if r.Charset == "GB-18030" || r.Charset == "UTF-8" {
return r.Charset
}
charset = r.Charset
}
return charset
}
// CheckCharsetByBOM ref: https://en.wikipedia.org/wiki/Byte_order_mark
func CheckCharsetByBOM(buf []byte) string {
// TODO: There are many kind of BOM
// UTF-8 EF BB BF
if len(buf) >= 3 {
if buf[0] == 0xef && buf[1] == 0xbb && buf[2] == 0xbf {
return "UTF-8"
}
}
// GB-18030 84 31 95 33
if len(buf) >= 4 {
if buf[0] == 0x84 && buf[1] == 0x31 && buf[2] == 0x95 && buf[3] == 0x33 {
return "GB-18030"
}
}
return ""
}
// RemoveBOM remove bom from file
func RemoveBOM(buf []byte) (string, []byte) {
// ef bb bf, UTF-8 BOM
if len(buf) > 3 {
if buf[0] == 0xef && buf[1] == 0xbb && buf[2] == 0xbf {
return string(buf[3:]), buf[:3]
}
}
// ff fe, UTF-16 (LE) BOM
if len(buf) > 2 {
if buf[0] == 0xff && buf[1] == 0xfe {
return string(buf[2:]), buf[:2]
}
}
return string(buf), []byte{}
}
================================================
FILE: common/chardet_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"fmt"
"io/ioutil"
"testing"
)
func TestChardet(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
charsets := []string{
"GB-18030",
"UTF-8",
}
for _, c := range charsets {
fileName := DevPath + "/common/testdata/chardet_" + c + ".txt"
buf, err := ioutil.ReadFile(fileName)
if err != nil {
t.Errorf("ioutil.ReadFile %s, Error: %s", fileName, err.Error())
}
name := Chardet(buf)
if name != c {
t.Errorf("file: %s, Want: %s, Get: %s", fileName, c, name)
}
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestRemoveBOM(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
fileName := DevPath + "/common/testdata/UTF-8.bom.sql"
buf, err := ioutil.ReadFile(fileName)
if err != nil {
t.Errorf("ioutil.ReadFile %s, Error: %s", fileName, err.Error())
}
GoldenDiff(func() {
fmt.Println(RemoveBOM(buf))
}, t.Name(), update)
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestCheckCharsetByBOM(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
fileName := DevPath + "/common/testdata/UTF-8.bom.sql"
buf, err := ioutil.ReadFile(fileName)
if err != nil {
t.Errorf("ioutil.ReadFile %s, Error: %s", fileName, err.Error())
}
if CheckCharsetByBOM(buf) != "UTF-8" {
t.Errorf("checkCharsetByBOM Want: UTF-8, Get: %s", CheckCharsetByBOM(buf))
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
================================================
FILE: common/config.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"bufio"
"bytes"
"encoding/json"
"flag"
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
"regexp"
"runtime"
"strings"
"time"
"github.com/go-sql-driver/mysql"
yaml "gopkg.in/yaml.v2"
)
var (
// BlackList 黑名单中的SQL不会被评审
BlackList []string
// PrintConfig -print-config
PrintConfig bool
// PrintVersion -print-config
PrintVersion bool
// CheckConfig -check-config
CheckConfig bool
// 防止 readCmdFlags 函数重入
hasParsed bool
)
// Configuration 配置文件定义结构体
type Configuration struct {
// +++++++++++++++测试环境+++++++++++++++++
OnlineDSN *Dsn `yaml:"online-dsn"` // 线上环境数据库配置
TestDSN *Dsn `yaml:"test-dsn"` // 测试环境数据库配置
AllowOnlineAsTest bool `yaml:"allow-online-as-test"` // 允许 Online 环境也可以当作 Test 环境
DisableVersionCheck bool `yaml:"disable-version-check"` // 是否禁用环境检测，开启后表示允许测试环境版本低于线上环境 不建议开启，可能会导致语句执行异常
DropTestTemporary bool `yaml:"drop-test-temporary"` // 是否清理Test环境产生的临时库表
CleanupTestDatabase bool `yaml:"cleanup-test-database"` // 清理残余的测试数据库（程序异常退出或未开启drop-test-temporary） issue #48
OnlySyntaxCheck bool `yaml:"only-syntax-check"` // 只做语法检查不输出优化建议
SamplingStatisticTarget int `yaml:"sampling-statistic-target"` // 数据采样因子，对应 PostgreSQL 的 default_statistics_target
Sampling bool `yaml:"sampling"` // 数据采样开关
SamplingCondition string `yaml:"sampling-condition"` // 指定采样条件，如：WHERE xxx LIMIT xxx;
Profiling bool `yaml:"profiling"` // 在开启数据采样的情况下，在测试环境执行进行profile
Trace bool `yaml:"trace"` // 在开启数据采样的情况下，在测试环境执行进行Trace
Explain bool `yaml:"explain"` // Explain开关
Delimiter string `yaml:"delimiter"` // SQL分隔符
// +++++++++++++++日志相关+++++++++++++++++
// 日志级别，这里使用了 beego 的 log 包
// [0:Emergency, 1:Alert, 2:Critical, 3:Error, 4:Warning, 5:Notice, 6:Informational, 7:Debug]
LogLevel int `yaml:"log-level"`
// 日志输出位置，默认日志输出到控制台
// 目前只支持['console', 'file']两种形式，如非console形式这里需要指定文件的路径，可以是相对路径
LogOutput string `yaml:"log-output"`
// 优化建议输出格式，目前支持: json, text, markdown格式，如指定其他格式会给 pretty.Println 的输出
ReportType string `yaml:"report-type"`
// 当 ReportType 为 html 格式时使用的 css 风格，如不指定会提供一个默认风格。CSS可 以是本地文件，也可以是一个URL
ReportCSS string `yaml:"report-css"`
// 当 ReportType 为 html 格式时使用的 javascript 脚本，如不指定默认会加载SQL pretty 使用的 javascript。像CSS一样可以是本地文件，也可以是一个URL
ReportJavascript string `yaml:"report-javascript"`
// 当ReportType 为 html 格式时，HTML 的 title
ReportTitle string `yaml:"report-title"`
// blackfriday markdown2html config
MarkdownExtensions int `yaml:"markdown-extensions"` // markdown 转 html 支持的扩展包, 参考blackfriday
MarkdownHTMLFlags int `yaml:"markdown-html-flags"` // markdown 转 html 支持的 flag, 参考blackfriday, default 0
// ++++++++++++++优化建议相关++++++++++++++
IgnoreRules []string `yaml:"ignore-rules"` // 忽略的优化建议规则
RewriteRules []string `yaml:"rewrite-rules"` // 生效的重写规则
BlackList string `yaml:"blacklist"` // blacklist 中的 SQL 不会被评审，可以是指纹，也可以是正则
MaxJoinTableCount int `yaml:"max-join-table-count"` // 单条 SQL 中 JOIN 表的最大数量
MaxGroupByColsCount int `yaml:"max-group-by-cols-count"` // 单条 SQL 中 GroupBy 包含列的最大数量
MaxDistinctCount int `yaml:"max-distinct-count"` // 单条 SQL 中 Distinct 的最大数量
MaxIdxColsCount int `yaml:"max-index-cols-count"` // 复合索引中包含列的最大数量
MaxTextColsCount int `yaml:"max-text-cols-count"` // 表中含有的 text/blob 列的最大数量
MaxTotalRows uint64 `yaml:"max-total-rows"` // 计算散粒度时，当数据行数大于 MaxTotalRows 即开启数据库保护模式，散粒度返回结果可信度下降
MaxQueryCost int64 `yaml:"max-query-cost"` // last_query_cost 超过该值时将给予警告
SpaghettiQueryLength int `yaml:"spaghetti-query-length"` // SQL最大长度警告，超过该长度会给警告
AllowDropIndex bool `yaml:"allow-drop-index"` // 允许输出删除重复索引的建议
MaxInCount int `yaml:"max-in-count"` // IN()最大数量
MaxIdxBytesPerColumn int `yaml:"max-index-bytes-percolumn"` // 索引中单列最大字节数，默认767
MaxIdxBytes int `yaml:"max-index-bytes"` // 索引总长度限制，默认3072
AllowCharsets []string `yaml:"allow-charsets"` // 允许使用的 DEFAULT CHARSET
AllowCollates []string `yaml:"allow-collates"` // 允许使用的 COLLATE
AllowEngines []string `yaml:"allow-engines"` // 允许使用的存储引擎
MaxIdxCount int `yaml:"max-index-count"` // 单张表允许最多索引数
MaxColCount int `yaml:"max-column-count"` // 单张表允许最大列数
MaxValueCount int `yaml:"max-value-count"` // INSERT/REPLACE 单次允许批量写入的行数
IdxPrefix string `yaml:"index-prefix"` // 普通索引建议使用的前缀
UkPrefix string `yaml:"unique-key-prefix"` // 唯一键建议使用的前缀
MaxSubqueryDepth int `yaml:"max-subquery-depth"` // 子查询最大尝试
MaxVarcharLength int `yaml:"max-varchar-length"` // varchar最大长度
ColumnNotAllowType []string `yaml:"column-not-allow-type"` // 字段不允许使用的数据类型
MinCardinality float64 `yaml:"min-cardinality"` // 添加索引散粒度阈值，范围 0~100
// ++++++++++++++EXPLAIN检查项+++++++++++++
ExplainSQLReportType string `yaml:"explain-sql-report-type"` // EXPLAIN markdown 格式输出 SQL 样式，支持 sample, fingerprint, pretty 等
ExplainType string `yaml:"explain-type"` // EXPLAIN方式 [traditional, extended, partitions]
ExplainFormat string `yaml:"explain-format"` // FORMAT=[json, traditional]
ExplainWarnSelectType []string `yaml:"explain-warn-select-type"` // 哪些 select_type 不建议使用
ExplainWarnAccessType []string `yaml:"explain-warn-access-type"` // 哪些 access type 不建议使用
ExplainMaxKeyLength int `yaml:"explain-max-keys"` // 最大 key_len
ExplainMinPossibleKeys int `yaml:"explain-min-keys"` // 最小 possible_keys 警告
ExplainMaxRows int64 `yaml:"explain-max-rows"` // 最大扫描行数警告
ExplainWarnExtra []string `yaml:"explain-warn-extra"` // 哪些 extra 信息会给警告
ExplainMaxFiltered float64 `yaml:"explain-max-filtered"` // filtered 大于该配置给出警告
ExplainWarnScalability []string `yaml:"explain-warn-scalability"` // 复杂度警告名单
ShowWarnings bool `yaml:"show-warnings"` // explain extended with show warnings
ShowLastQueryCost bool `yaml:"show-last-query-cost"` // switch with show status like 'last_query_cost'
// ++++++++++++++其他配置项+++++++++++++++
Query string `yaml:"query"` // 需要进行调优的SQL
ListHeuristicRules bool `yaml:"list-heuristic-rules"` // 打印支持的评审规则列表
ListRewriteRules bool `yaml:"list-rewrite-rules"` // 打印重写规则
ListTestSqls bool `yaml:"list-test-sqls"` // 打印测试case用于测试
ListReportTypes bool `yaml:"list-report-types"` // 打印支持的报告输出类型
Verbose bool `yaml:"verbose"` // verbose模式，会多输出一些信息
DryRun bool `yaml:"dry-run"` // 是否在预演环境执行
MaxPrettySQLLength int `yaml:"max-pretty-sql-length"` // 超出该长度的SQL会转换成指纹输出
}
// Config 默认设置
var Config = &Configuration{
OnlineDSN: newDSN(nil),
TestDSN: newDSN(nil),
AllowOnlineAsTest: false,
DropTestTemporary: true,
CleanupTestDatabase: false,
DryRun: true,
OnlySyntaxCheck: false,
SamplingStatisticTarget: 100,
Sampling: false,
Profiling: false,
Trace: false,
Explain: true,
Delimiter: ";",
MinCardinality: 0,
MaxJoinTableCount: 5,
MaxGroupByColsCount: 5,
MaxDistinctCount: 5,
MaxIdxColsCount: 5,
MaxTextColsCount: 2,
MaxIdxBytesPerColumn: 767,
MaxIdxBytes: 3072,
MaxTotalRows: 9999999,
MaxQueryCost: 9999,
SpaghettiQueryLength: 2048,
AllowDropIndex: false,
LogLevel: 3,
LogOutput: "soar.log",
ReportType: "markdown",
ReportCSS: "",
ReportJavascript: "",
ReportTitle: "SQL优化分析报告",
BlackList: "",
AllowCharsets: []string{"utf8", "utf8mb4"},
AllowCollates: []string{},
AllowEngines: []string{"innodb"},
MaxIdxCount: 10,
MaxColCount: 40,
MaxValueCount: 100,
MaxInCount: 10,
IdxPrefix: "idx_",
UkPrefix: "uk_",
MaxSubqueryDepth: 5,
MaxVarcharLength: 1024,
ColumnNotAllowType: []string{"boolean"},
MarkdownExtensions: 94,
MarkdownHTMLFlags: 0,
ExplainSQLReportType: "pretty",
ExplainType: "extended",
ExplainFormat: "traditional",
ExplainWarnSelectType: []string{""},
ExplainWarnAccessType: []string{"ALL"},
ExplainMaxKeyLength: 3,
ExplainMinPossibleKeys: 0,
ExplainMaxRows: 10000,
ExplainWarnExtra: []string{"Using temporary", "Using filesort"},
ExplainMaxFiltered: 100.0,
ExplainWarnScalability: []string{"O(n)"},
ShowWarnings: false,
ShowLastQueryCost: false,
IgnoreRules: []string{
"COL.011",
},
RewriteRules: []string{
"delimiter",
"orderbynull",
"groupbyconst",
"dmlorderby",
"having",
"star2columns",
"insertcolumns",
"distinctstar",
},
ListHeuristicRules: false,
ListRewriteRules: false,
ListTestSqls: false,
ListReportTypes: false,
MaxPrettySQLLength: 1024,
}
// Dsn Data source name
type Dsn struct {
User string `yaml:"user"` // Usernames
Password string `yaml:"password"` // Password (requires User)
Net string `yaml:"net"` // Network type
Addr string `yaml:"addr"` // Network address (requires Net)
Schema string `yaml:"schema"` // Database name
Charset string `yaml:"charset"` // SET NAMES charset
Collation string `yaml:"collation"` // Connection collation
Loc string `yaml:"loc"` // Location for time.Time values
TLS string `yaml:"tls"` // TLS configuration name
ServerPubKey string `yaml:"server-public-key"` // Server public key name
MaxAllowedPacket int `yaml:"max-allowed-packet"` // Max packet size allowed
Params map[string]string `yaml:"params"` // Other Connection parameters, `SET param=val`, `SET NAMES charset`
Timeout string `yaml:"timeout"` // Dial timeout
ReadTimeout string `yaml:"read-timeout"` // I/O read timeout
WriteTimeout string `yaml:"write-timeout"` // I/O write timeout
AllowNativePasswords bool `yaml:"allow-native-passwords"` // Allows the native password authentication method
AllowOldPasswords bool `yaml:"allow-old-passwords"` // Allows the old insecure password method
Disable bool `yaml:"disable"`
Version int `yaml:"-"` // 版本自动检查，不可配置
}
// newDSN create default Dsn struct
func newDSN(cfg *mysql.Config) *Dsn {
dsn := &Dsn{
Net: "tcp",
Schema: "information_schema",
Charset: "utf8",
Timeout: "3s",
AllowNativePasswords: true,
Params: make(map[string]string),
MaxAllowedPacket: 4 << 20, // 4 MiB
// Disable: true,
Version: 99999,
}
if cfg == nil {
return dsn
}
dsn.User = cfg.User
dsn.Password = cfg.Passwd
dsn.Net = cfg.Net
dsn.Addr = cfg.Addr
dsn.Schema = cfg.DBName
dsn.Params = make(map[string]string)
for k, v := range cfg.Params {
dsn.Params[k] = v
}
if _, ok := cfg.Params["charset"]; ok {
dsn.Charset = cfg.Params["charset"]
}
dsn.Collation = cfg.Collation
dsn.Loc = cfg.Loc.String()
dsn.MaxAllowedPacket = cfg.MaxAllowedPacket
dsn.ServerPubKey = cfg.ServerPubKey
dsn.TLS = cfg.TLSConfig
dsn.Timeout = cfg.Timeout.String()
dsn.ReadTimeout = cfg.ReadTimeout.String()
dsn.WriteTimeout = cfg.WriteTimeout.String()
dsn.AllowNativePasswords = cfg.AllowNativePasswords
dsn.AllowOldPasswords = cfg.AllowOldPasswords
return dsn
}
// newMySQLConfig convert Dsn to go-sql-drive Config
func (env *Dsn) newMySQLConfig() (*mysql.Config, error) {
var err error
dsn := mysql.NewConfig()
dsn.User = env.User
dsn.Passwd = env.Password
dsn.Net = env.Net
dsn.Addr = env.Addr
dsn.DBName = env.Schema
dsn.Params = make(map[string]string)
for k, v := range env.Params {
dsn.Params[k] = v
}
dsn.Params["charset"] = env.Charset
dsn.Collation = env.Collation
dsn.Loc, err = time.LoadLocation(env.Loc)
if err != nil {
return nil, err
}
dsn.MaxAllowedPacket = env.MaxAllowedPacket
dsn.ServerPubKey = env.ServerPubKey
dsn.TLSConfig = env.TLS
if env.Timeout != "" {
dsn.Timeout, err = time.ParseDuration(env.Timeout)
LogIfError(err, "timeout: '%s'", env.Timeout)
}
if env.WriteTimeout != "" {
dsn.WriteTimeout, err = time.ParseDuration(env.WriteTimeout)
LogIfError(err, "writeTimeout: '%s'", env.WriteTimeout)
}
if env.ReadTimeout != "" {
dsn.ReadTimeout, err = time.ParseDuration(env.ReadTimeout)
LogIfError(err, "readTimeout: '%s'", env.ReadTimeout)
}
dsn.AllowNativePasswords = env.AllowNativePasswords
dsn.AllowOldPasswords = env.AllowOldPasswords
return dsn, err
}
// 解析命令行DSN输入
func parseDSN(odbc string, d *Dsn) *Dsn {
dsn := newDSN(nil)
var addr, user, password, schema, charset, timeout string
if odbc == FormatDSN(d) {
return d
}
if d != nil {
// 原来有个判断，后来判断条件被删除了就导致第一次addr无论如何都会被修改。所以这边先注释掉
// addr = d.Addr
user = d.User
password = d.Password
schema = d.Schema
charset = d.Charset
timeout = d.Timeout
}
// 设置为空表示禁用环境
odbc = strings.TrimSpace(odbc)
if odbc == "" {
return &Dsn{Disable: true}
}
var userInfo, hostInfo, query string
// DSN 格式匹配
if res := regexp.MustCompile(`^(.*)@(.*?)/(.*?)($|\?)(.*)`).FindStringSubmatch(odbc); len(res) > 5 {
// userInfo@hostInfo/database
userInfo = res[1]
hostInfo = res[2]
schema = res[3]
query = res[5]
} else if res := regexp.MustCompile(`^(.*)/(.*?)($|\?)(.*)`).FindStringSubmatch(odbc); len(res) > 4 {
// hostInfo/database
hostInfo = res[1]
schema = res[2]
query = res[4]
} else if res := regexp.MustCompile(`^(.*)@(.*?)($|\?)(.*)`).FindStringSubmatch(odbc); len(res) > 4 {
// userInfo@hostInfo
userInfo = res[1]
hostInfo = res[2]
query = res[4]
} else if res := regexp.MustCompile(`^(.*?)($|\?)(.*)`).FindStringSubmatch(odbc); len(res) > 3 {
// hostInfo
hostInfo = res[1]
query = res[3]
}
// 解析用户信息
if userInfo != "" {
user = strings.Split(userInfo, ":")[0]
// 防止密码中含有与用户名相同的字符, 所以用正则替换, 剩下的就是密码
password = strings.TrimLeft(regexp.MustCompile("^"+user).ReplaceAllString(userInfo, ""), ":")
}
// 解析主机信息
host := strings.Split(hostInfo, ":")[0]
port := strings.TrimLeft(strings.Replace(hostInfo, host, "", 1), ":")
if host == "" {
host = "127.0.0.1"
}
if port == "" {
port = "3306"
}
addr = host + ":" + port
// 解析查询字符串
if query != "" {
params := strings.Split(query, "&")
for _, f := range params {
attr := strings.Split(f, "=")
if len(attr) > 1 {
arg := strings.TrimSpace(attr[0])
val := strings.TrimSpace(attr[1])
switch arg {
case "charset":
charset = val
case "timeout":
timeout = val
default:
}
}
}
}
// 默认用information_schema库
if schema == "" {
schema = "information_schema"
}
// 默认 utf8 使用字符集
if charset == "" {
charset = "utf8"
}
// 默认连接数据库超时时间 3s
if timeout == "" {
timeout = "3s"
}
dsn.Addr = addr
dsn.User = user
dsn.Password = password
dsn.Schema = schema
dsn.Charset = charset
dsn.Timeout = timeout
return dsn
}
// ParseDSN compatible with old version soar < 0.11.0
func ParseDSN(odbc string, d *Dsn) *Dsn {
cfg, err := mysql.ParseDSN(odbc)
if err != nil {
// Log.Debug("go-sql-driver/mysql.ParseDSN Error: %s, DSN: %s, try to use old version parseDSN", err.Error(), odbc)
return parseDSN(odbc, d)
}
return newDSN(cfg)
}
// FormatDSN 格式化打印DSN
func FormatDSN(env *Dsn) string {
if env == nil || env.Disable {
return ""
}
dsn, err := env.newMySQLConfig()
if err != nil {
return ""
}
return dsn.FormatDSN()
}
// SoarVersion soar version information
func SoarVersion() {
fmt.Println("Version:", Version)
fmt.Println("Branch:", Branch)
fmt.Println("Compile:", Compile)
fmt.Println("GitDirty:", GitDirty)
}
// 因为vitess sqlparser 使用了 glog 中也会使用 flag，为了不让用户困扰我们单独写一个 usage
func usage() {
regPwd := regexp.MustCompile(`:.*@`)
vitessHelp := []string{
"-alsologtostderr",
"log to standard error as well as files",
"-log_backtrace_at value",
"when logging hits line file:N, emit a stack trace",
"-log_dir string",
"If non-empty, write log files in this directory",
"-logtostderr",
"log to standard error instead of files",
"-sql-max-length-errors int",
"truncate queries in error logs to the given length (default unlimited)",
"-sql-max-length-ui int",
"truncate queries in debug UIs to the given length (default 512) (default 512)",
"-stderrthreshold value",
"logs at or above this threshold go to stderr",
"-v value",
"log level for V logs",
"-vmodule value",
"comma-separated list of pattern=N settings for file-filtered logging",
}
// io redirect
restoreStdout := os.Stdout
restoreStderr := os.Stderr
stdin, stdout, _ := os.Pipe()
os.Stderr = stdout
os.Stdout = stdout
flag.PrintDefaults()
// copy the output in a separate goroutine so printing can't block indefinitely
outC := make(chan string)
go func() {
var buf bytes.Buffer
_, err := io.Copy(&buf, stdin)
if err != nil {
fmt.Println(err.Error())
}
outC <- buf.String()
}()
// back to normal state
stdout.Close()
os.Stdout = restoreStdout // restoring the real stderr
os.Stderr = restoreStderr
fmt.Printf("Usage of %s:\n", os.Args[0])
// reading our temp stdout
out := <-outC
for _, line := range strings.Split(out, "\n") {
found := false
for _, ignore := range vitessHelp {
if strings.TrimSpace(line) == strings.TrimSpace(ignore) {
found = true
}
if regPwd.MatchString(line) && !Config.Verbose {
line = regPwd.ReplaceAllString(line, ":********@")
}
}
if !found {
fmt.Println(line)
}
}
}
// PrintConfiguration for `-print-config` flag
func PrintConfiguration() {
// 打印配置的时候密码不显示
if !Config.Verbose {
Config.OnlineDSN.Password = "********"
Config.TestDSN.Password = "********"
}
data, _ := yaml.Marshal(Config)
fmt.Print(string(data))
}
// 加载配置文件
func (conf *Configuration) readConfigFile(path string) error {
configFile, err := os.Open(path)
if err != nil {
Log.Warning("readConfigFile(%s) os.Open failed: %v", path, err)
return err
}
defer configFile.Close()
content, err := ioutil.ReadAll(configFile)
if err != nil {
Log.Warning("readConfigFile(%s) ioutil.ReadAll failed: %v", path, err)
return err
}
err = yaml.Unmarshal(content, Config)
if err != nil {
Log.Warning("readConfigFile(%s) yaml.Unmarshal failed: %v", path, err)
return err
}
return nil
}
// 从命令行参数读配置
func readCmdFlags() error {
if hasParsed {
Log.Debug("Skip read cmd flags.")
return nil
}
_ = flag.String("config", "", "Config file path")
// +++++++++++++++测试环境+++++++++++++++++
onlineDSN := flag.String("online-dsn", FormatDSN(Config.OnlineDSN), "OnlineDSN, 线上环境数据库配置, username:password@tcp(ip:port)/schema")
testDSN := flag.String("test-dsn", FormatDSN(Config.TestDSN), "TestDSN, 测试环境数据库配置, username:password@tcp(ip:port)/schema")
allowOnlineAsTest := flag.Bool("allow-online-as-test", Config.AllowOnlineAsTest, "AllowOnlineAsTest, 允许线上环境也可以当作测试环境")
dropTestTemporary := flag.Bool("drop-test-temporary", Config.DropTestTemporary, "DropTestTemporary, 是否清理测试环境产生的临时库表")
cleanupTestDatabase := flag.Bool("cleanup-test-database", Config.CleanupTestDatabase, "单次运行清理历史1小时前残余的测试库。")
onlySyntaxCheck := flag.Bool("only-syntax-check", Config.OnlySyntaxCheck, "OnlySyntaxCheck, 只做语法检查不输出优化建议")
profiling := flag.Bool("profiling", Config.Profiling, "Profiling, 开启数据采样的情况下在测试环境执行Profile")
trace := flag.Bool("trace", Config.Trace, "Trace, 开启数据采样的情况下在测试环境执行Trace")
explain := flag.Bool("explain", Config.Explain, "Explain, 是否开启Explain执行计划分析")
sampling := flag.Bool("sampling", Config.Sampling, "Sampling, 数据采样开关")
samplingStatisticTarget := flag.Int("sampling-statistic-target", Config.SamplingStatisticTarget, "SamplingStatisticTarget, 数据采样因子，对应 PostgreSQL 的 default_statistics_target")
samplingCondition := flag.String("sampling-condition", Config.SamplingCondition, "SamplingCondition, 数据采样条件，如： WHERE xxx LIMIT xxx")
delimiter := flag.String("delimiter", Config.Delimiter, "Delimiter, SQL分隔符")
minCardinality := flag.Float64("min-cardinality", Config.MinCardinality, "MinCardinality，索引列散粒度最低阈值，散粒度低于该值的列不添加索引，建议范围0.0 ~ 100.0")
// +++++++++++++++日志相关+++++++++++++++++
logLevel := flag.Int("log-level", Config.LogLevel, "LogLevel, 日志级别, [0:Emergency, 1:Alert, 2:Critical, 3:Error, 4:Warning, 5:Notice, 6:Informational, 7:Debug]")
logOutput := flag.String("log-output", Config.LogOutput, "LogOutput, 日志输出位置")
reportType := flag.String("report-type", Config.ReportType, "ReportType, 优化建议输出格式，目前支持: json, text, markdown, html等")
reportCSS := flag.String("report-css", Config.ReportCSS, "ReportCSS, 当 ReportType 为 html 格式时使用的 css 风格，如不指定会提供一个默认风格。CSS可以是本地文件，也可以是一个URL")
reportJavascript := flag.String("report-javascript", Config.ReportJavascript, "ReportJavascript, 当 ReportType 为 html 格式时使用的javascript脚本，如不指定默认会加载SQL pretty 使用的 javascript。像CSS一样可以是本地文件，也可以是一个URL")
reportTitle := flag.String("report-title", Config.ReportTitle, "ReportTitle, 当 ReportType 为 html 格式时，HTML 的 title")
// +++++++++++++++markdown+++++++++++++++++
markdownExtensions := flag.Int("markdown-extensions", Config.MarkdownExtensions, "MarkdownExtensions, markdown 转 html支持的扩展包, 参考blackfriday")
markdownHTMLFlags := flag.Int("markdown-html-flags", Config.MarkdownHTMLFlags, "MarkdownHTMLFlags, markdown 转 html 支持的 flag, 参考blackfriday")
// ++++++++++++++优化建议相关++++++++++++++
ignoreRules := flag.String("ignore-rules", strings.Join(Config.IgnoreRules, ","), "IgnoreRules, 忽略的优化建议规则")
rewriteRules := flag.String("rewrite-rules", strings.Join(Config.RewriteRules, ","), "RewriteRules, 生效的重写规则")
blackList := flag.String("blacklist", Config.BlackList, "指定 blacklist 配置文件的位置，文件中的 SQL 不会被评审。一行一条SQL，可以是指纹，也可以是正则")
maxJoinTableCount := flag.Int("max-join-table-count", Config.MaxJoinTableCount, "MaxJoinTableCount, 单条 SQL 中 JOIN 表的最大数量")
maxGroupByColsCount := flag.Int("max-group-by-cols-count", Config.MaxGroupByColsCount, "MaxGroupByColsCount, 单条 SQL 中 GroupBy 包含列的最大数量")
maxDistinctCount := flag.Int("max-distinct-count", Config.MaxDistinctCount, "MaxDistinctCount, 单条 SQL 中 Distinct 的最大数量")
maxIdxColsCount := flag.Int("max-index-cols-count", Config.MaxIdxColsCount, "MaxIdxColsCount, 复合索引中包含列的最大数量")
maxTextColsCount := flag.Int("max-text-cols-count", Config.MaxTextColsCount, "MaxTextColsCount, 表中含有的 text/blob 列的最大数量")
maxTotalRows := flag.Uint64("max-total-rows", Config.MaxTotalRows, "MaxTotalRows, 计算散粒度时，当数据行数大于MaxTotalRows即开启数据库保护模式，不计算散粒度")
maxQueryCost := flag.Int64("max-query-cost", Config.MaxQueryCost, "MaxQueryCost, last_query_cost 超过该值时将给予警告")
spaghettiQueryLength := flag.Int("spaghetti-query-length", Config.SpaghettiQueryLength, "SpaghettiQueryLength, SQL最大长度警告，超过该长度会给警告")
allowDropIdx := flag.Bool("allow-drop-index", Config.AllowDropIndex, "AllowDropIndex, 允许输出删除重复索引的建议")
maxInCount := flag.Int("max-in-count", Config.MaxInCount, "MaxInCount, IN()最大数量")
maxIdxBytesPerColumn := flag.Int("max-index-bytes-percolumn", Config.MaxIdxBytesPerColumn, "MaxIdxBytesPerColumn, 索引中单列最大字节数")
maxIdxBytes := flag.Int("max-index-bytes", Config.MaxIdxBytes, "MaxIdxBytes, 索引总长度限制")
allowCharsets := flag.String("allow-charsets", strings.ToLower(strings.Join(Config.AllowCharsets, ",")), "AllowCharsets")
allowCollates := flag.String("allow-collates", strings.ToLower(strings.Join(Config.AllowCollates, ",")), "AllowCollates")
allowEngines := flag.String("allow-engines", strings.ToLower(strings.Join(Config.AllowEngines, ",")), "AllowEngines")
maxIdxCount := flag.Int("max-index-count", Config.MaxIdxCount, "MaxIdxCount, 单表最大索引个数")
maxColCount := flag.Int("max-column-count", Config.MaxColCount, "MaxColCount, 单表允许的最大列数")
maxValueCount := flag.Int("max-value-count", Config.MaxValueCount, "MaxValueCount, INSERT/REPLACE 单次批量写入允许的行数")
idxPrefix := flag.String("index-prefix", Config.IdxPrefix, "IdxPrefix")
ukPrefix := flag.String("unique-key-prefix", Config.UkPrefix, "UkPrefix")
maxSubqueryDepth := flag.Int("max-subquery-depth", Config.MaxSubqueryDepth, "MaxSubqueryDepth")
maxVarcharLength := flag.Int("max-varchar-length", Config.MaxVarcharLength, "MaxVarcharLength")
columnNotAllowType := flag.String("column-not-allow-type", strings.Join(Config.ColumnNotAllowType, ","), "ColumnNotAllowType")
// ++++++++++++++EXPLAIN检查项+++++++++++++
explainSQLReportType := flag.String("explain-sql-report-type", strings.ToLower(Config.ExplainSQLReportType), "ExplainSQLReportType [pretty, sample, fingerprint]")
explainType := flag.String("explain-type", strings.ToLower(Config.ExplainType), "ExplainType [extended, partitions, traditional]")
explainFormat := flag.String("explain-format", strings.ToLower(Config.ExplainFormat), "ExplainFormat [json, traditional]")
explainWarnSelectType := flag.String("explain-warn-select-type", strings.Join(Config.ExplainWarnSelectType, ","), "ExplainWarnSelectType, 哪些select_type不建议使用")
explainWarnAccessType := flag.String("explain-warn-access-type", strings.Join(Config.ExplainWarnAccessType, ","), "ExplainWarnAccessType, 哪些access type不建议使用")
explainMaxKeyLength := flag.Int("explain-max-keys", Config.ExplainMaxKeyLength, "ExplainMaxKeyLength, 最大key_len")
explainMinPossibleKeys := flag.Int("explain-min-keys", Config.ExplainMinPossibleKeys, "ExplainMinPossibleKeys, 最小possible_keys警告")
explainMaxRows := flag.Int64("explain-max-rows", Config.ExplainMaxRows, "ExplainMaxRows, 最大扫描行数警告")
explainWarnExtra := flag.String("explain-warn-extra", strings.Join(Config.ExplainWarnExtra, ","), "ExplainWarnExtra, 哪些extra信息会给警告")
explainMaxFiltered := flag.Float64("explain-max-filtered", Config.ExplainMaxFiltered, "ExplainMaxFiltered, filtered大于该配置给出警告")
explainWarnScalability := flag.String("explain-warn-scalability", strings.Join(Config.ExplainWarnScalability, ","), "ExplainWarnScalability, 复杂度警告名单, 支持O(n),O(log n),O(1),O(?)")
showWarnings := flag.Bool("show-warnings", Config.ShowWarnings, "ShowWarnings")
showLastQueryCost := flag.Bool("show-last-query-cost", Config.ShowLastQueryCost, "ShowLastQueryCost")
// +++++++++++++++++其他+++++++++++++++++++
printConfig := flag.Bool("print-config", false, "Print configs")
checkConfig := flag.Bool("check-config", false, "Check configs")
printVersion := flag.Bool("version", false, "Print version info")
query := flag.String("query", Config.Query, "待评审的 SQL 或 SQL 文件，如 SQL 中包含特殊字符建议使用文件名。")
listHeuristicRules := flag.Bool("list-heuristic-rules", Config.ListHeuristicRules, "ListHeuristicRules, 打印支持的评审规则列表")
listRewriteRules := flag.Bool("list-rewrite-rules", Config.ListRewriteRules, "ListRewriteRules, 打印支持的重写规则列表")
listTestSQLs := flag.Bool("list-test-sqls", Config.ListTestSqls, "ListTestSqls, 打印测试case用于测试")
listReportTypes := flag.Bool("list-report-types", Config.ListReportTypes, "ListReportTypes, 打印支持的报告输出类型")
verbose := flag.Bool("verbose", Config.Verbose, "Verbose")
dryrun := flag.Bool("dry-run", Config.DryRun, "是否在预演环境执行")
maxPrettySQLLength := flag.Int("max-pretty-sql-length", Config.MaxPrettySQLLength, "MaxPrettySQLLength, 超出该长度的SQL会转换成指纹输出")
// 一个不存在 log-level，用于更新 usage。
// 因为 vitess 里面也用了 flag，这些 vitess 的参数我们不需要关注
if !Config.Verbose && runtime.GOOS != "windows" {
flag.Usage = usage
}
flag.Parse()
Config.OnlineDSN = ParseDSN(*onlineDSN, Config.OnlineDSN)
Config.TestDSN = ParseDSN(*testDSN, Config.TestDSN)
Config.AllowOnlineAsTest = *allowOnlineAsTest
Config.DropTestTemporary = *dropTestTemporary
Config.CleanupTestDatabase = *cleanupTestDatabase
Config.OnlySyntaxCheck = *onlySyntaxCheck
Config.Profiling = *profiling
Config.Trace = *trace
Config.Explain = *explain
Config.Sampling = *sampling
Config.SamplingStatisticTarget = *samplingStatisticTarget
Config.SamplingCondition = *samplingCondition
Config.LogLevel = *logLevel
if filepath.IsAbs(*logOutput) || *logOutput == "" {
Config.LogOutput = *logOutput
} else {
Config.LogOutput = filepath.Join(BaseDir, *logOutput)
}
Config.ReportType = strings.ToLower(*reportType)
if Config.ReportType == "tables" && Config.TestDSN.Schema == "information_schema" {
Config.TestDSN.Schema = "unknown"
}
Config.ReportCSS = *reportCSS
Config.ReportJavascript = *reportJavascript
Config.ReportTitle = *reportTitle
Config.MarkdownExtensions = *markdownExtensions
Config.MarkdownHTMLFlags = *markdownHTMLFlags
Config.IgnoreRules = strings.Split(*ignoreRules, ",")
Config.RewriteRules = strings.Split(*rewriteRules, ",")
*blackList = strings.TrimSpace(*blackList)
Config.MinCardinality = *minCardinality
if filepath.IsAbs(*blackList) || *blackList == "" {
Config.BlackList = *blackList
} else {
Config.BlackList = filepath.Join(BaseDir, *blackList)
}
Config.MaxJoinTableCount = *maxJoinTableCount
Config.MaxGroupByColsCount = *maxGroupByColsCount
Config.MaxDistinctCount = *maxDistinctCount
if *maxIdxColsCount < 16 {
Config.MaxIdxColsCount = *maxIdxColsCount
} else {
Config.MaxIdxColsCount = 16
}
Config.MaxTextColsCount = *maxTextColsCount
Config.MaxIdxBytesPerColumn = *maxIdxBytesPerColumn
Config.MaxIdxBytes = *maxIdxBytes
if *allowCharsets != "" {
Config.AllowCharsets = strings.Split(strings.ToLower(*allowCharsets), ",")
}
if *allowCollates != "" {
Config.AllowCollates = strings.Split(strings.ToLower(*allowCollates), ",")
}
if *allowEngines != "" {
Config.AllowEngines = strings.Split(strings.ToLower(*allowEngines), ",")
}
Config.MaxIdxCount = *maxIdxCount
Config.MaxColCount = *maxColCount
Config.MaxValueCount = *maxValueCount
Config.IdxPrefix = *idxPrefix
Config.UkPrefix = *ukPrefix
Config.MaxSubqueryDepth = *maxSubqueryDepth
Config.MaxTotalRows = *maxTotalRows
Config.MaxQueryCost = *maxQueryCost
Config.AllowDropIndex = *allowDropIdx
Config.MaxInCount = *maxInCount
Config.SpaghettiQueryLength = *spaghettiQueryLength
Config.Query = *query
Config.Delimiter = *delimiter
Config.ExplainSQLReportType = strings.ToLower(*explainSQLReportType)
Config.ExplainType = strings.ToLower(*explainType)
Config.ExplainFormat = strings.ToLower(*explainFormat)
Config.ExplainWarnSelectType = strings.Split(*explainWarnSelectType, ",")
Config.ExplainWarnAccessType = strings.Split(*explainWarnAccessType, ",")
Config.ExplainMaxKeyLength = *explainMaxKeyLength
Config.ExplainMinPossibleKeys = *explainMinPossibleKeys
Config.ExplainMaxRows = *explainMaxRows
Config.ExplainWarnExtra = strings.Split(*explainWarnExtra, ",")
Config.ExplainMaxFiltered = *explainMaxFiltered
Config.ExplainWarnScalability = strings.Split(*explainWarnScalability, ",")
Config.ShowWarnings = *showWarnings
Config.ShowLastQueryCost = *showLastQueryCost
Config.ListHeuristicRules = *listHeuristicRules
Config.ListRewriteRules = *listRewriteRules
Config.ListTestSqls = *listTestSQLs
Config.ListReportTypes = *listReportTypes
Config.Verbose = *verbose
Config.DryRun = *dryrun
Config.MaxPrettySQLLength = *maxPrettySQLLength
Config.MaxVarcharLength = *maxVarcharLength
if *columnNotAllowType != "" {
Config.ColumnNotAllowType = strings.Split(strings.ToLower(*columnNotAllowType), ",")
}
PrintVersion = *printVersion
PrintConfig = *printConfig
CheckConfig = *checkConfig
hasParsed = true
return nil
}
// ParseConfig 加载配置文件和命令行参数
func ParseConfig(configFile string) error {
var err error
var configs []string
// 指定了配置文件优先读配置文件，未指定配置文件按如下顺序加载，先找到哪个加载哪个
if configFile == "" {
configs = []string{
"/etc/soar.yaml",
filepath.Join(BaseDir, "etc", "soar.yaml"),
filepath.Join(BaseDir, "soar.yaml"),
}
} else {
configs = []string{
configFile,
}
}
for _, config := range configs {
if _, err = os.Stat(config); err == nil {
err = Config.readConfigFile(config)
if err != nil {
Log.Error("ParseConfig Config.readConfigFile Error: %v", err)
}
// LogOutput now is "console", if add Log.Debug here will print into stdout anyway.
// Log.Debug("ParseConfig use config file: %s", config)
break
}
}
err = readCmdFlags()
if err != nil {
Log.Error("ParseConfig readCmdFlags Error: %v", err)
return err
}
// parse blacklist & ignore blacklist file parse error
if _, e := os.Stat(Config.BlackList); e == nil {
var blFd *os.File
blFd, err = os.Open(Config.BlackList)
if err == nil {
bl := bufio.NewReader(blFd)
for {
rule, e := bl.ReadString('\n')
if e != nil {
break
}
rule = strings.TrimSpace(rule)
if strings.HasPrefix(rule, "#") || rule == "" {
continue
}
BlackList = append(BlackList, rule)
}
}
defer blFd.Close()
}
LoggerInit()
return err
}
// ReportType 元数据结构定义
type ReportType struct {
Name string `json:"Name"`
Description string `json:"Description"`
Example string `json:"Example"`
}
// ReportTypes 命令行-report-type支持的形式
var ReportTypes = []ReportType{
{
Name: "lint",
Description: "参考sqlint格式，以插件形式集成到代码编辑器，显示输出更加友好",
Example: `soar -report-type lint -query test.sql`,
},
{
Name: "markdown",
Description: "该格式为默认输出格式，以markdown格式展现，可以用网页浏览器插件直接打开，也可以用markdown编辑器打开",
Example: `echo "select * from film" | soar`,
},
{
Name: "rewrite",
Description: "SQL重写功能，配合-rewrite-rules参数一起使用，可以通过-list-rewrite-rules 查看所有支持的 SQL 重写规则",
Example: `echo "select * from film" | soar -rewrite-rules star2columns,delimiter -report-type rewrite`,
},
{
Name: "ast",
Description: "输出 SQL 的抽象语法树，主要用于测试",
Example: `echo "select * from film" | soar -report-type ast`,
},
{
Name: "ast-json",
Description: "以 JSON 格式输出 SQL 的抽象语法树，主要用于测试",
Example: `echo "select * from film" | soar -report-type ast-json`,
},
{
Name: "tiast",
Description: "输出 SQL 的 TiDB抽象语法树，主要用于测试",
Example: `echo "select * from film" | soar -report-type tiast`,
},
{
Name: "tiast-json",
Description: "以 JSON 格式输出 SQL 的 TiDB抽象语法树，主要用于测试",
Example: `echo "select * from film" | soar -report-type tiast-json`,
},
{
Name: "tables",
Description: "以 JSON 格式输出 SQL 使用的库表名",
Example: `echo "select * from film" | soar -report-type tables`,
},
{
Name: "query-type",
Description: "SQL 语句的请求类型",
Example: `echo "select * from film" | soar -report-type query-type`,
},
{
Name: "fingerprint",
Description: "输出SQL的指纹",
Example: `echo "select * from film where language_id=1" | soar -report-type fingerprint`,
},
{
Name: "md2html",
Description: "markdown 格式转 html 格式小工具",
Example: `soar -list-heuristic-rules | soar -report-type md2html > heuristic_rules.html`,
},
{
Name: "explain-digest",
Description: "输入为EXPLAIN的表格，JSON 或 Vertical格式，对其进行分析，给出分析结果",
Example: `soar -report-type explain-digest << EOF
+----+-------------+-------+------+---------------+------+---------+------+------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+------+-------+
| 1 | SIMPLE | film | ALL | NULL | NULL | NULL | NULL | 1131 | |
+----+-------------+-------+------+---------------+------+---------+------+------+-------+
EOF`,
},
{
Name: "duplicate-key-checker",
Description: "对 OnlineDsn 中指定的 database 进行索引重复检查",
Example: `soar -report-type duplicate-key-checker -online-dsn user:password@127.0.0.1:3306/db`,
},
{
Name: "html",
Description: "以HTML格式输出报表",
Example: `echo "select * from film" | soar -report-type html`,
},
{
Name: "json",
Description: "输出JSON格式报表，方便应用程序处理",
Example: `echo "select * from film" | soar -report-type json`,
},
{
Name: "tokenize",
Description: "对SQL进行切词，主要用于测试",
Example: `echo "select * from film" | soar -report-type tokenize`,
},
{
Name: "compress",
Description: "SQL压缩小工具，使用内置SQL压缩逻辑，测试中的功能",
Example: `echo "select
*
from
film" | soar -report-type compress`,
},
{
Name: "pretty",
Description: "使用kr/pretty打印报告，主要用于测试",
Example: `echo "select * from film" | soar -report-type pretty`,
},
{
Name: "remove-comment",
Description: "去除SQL语句中的注释，支持单行多行注释的去除",
Example: `echo "select/*comment*/ * from film" | soar -report-type remove-comment`,
},
{
Name: "chardet",
Description: "猜测输入的 SQL 使用的字符集",
Example: "echo '中文' | soar -report-type chardet",
},
}
// ListReportTypes 查看所有支持的report-type
func ListReportTypes() {
switch Config.ReportType {
case "json":
js, err := json.MarshalIndent(ReportTypes, "", " ")
if err == nil {
fmt.Println(string(js))
}
default:
fmt.Print("# 支持的报告类型\n\n[toc]\n\n")
for _, r := range ReportTypes {
fmt.Print("## ", MarkdownEscape(r.Name),
"\n* **Description**:", r.Description+"\n",
"\n* **Example**:\n\n```bash\n", r.Example, "\n```\n")
}
}
}
// ArgConfig get -config arg value from cli
func ArgConfig() string {
var configFile string
if len(os.Args) > 1 && strings.HasPrefix(os.Args[1], "-config") {
if os.Args[1] == "-config" && len(os.Args) > 2 {
if os.Args[2] == "=" && len(os.Args) > 3 {
// -config = soar.yaml not support
fmt.Println("wrong format, no space between '=', eg: -config=soar.yaml")
} else {
// -config soar.yaml
configFile = os.Args[2]
}
if strings.HasPrefix(configFile, "=") {
// -config =soar.yaml
configFile = strings.Split(configFile, "=")[1]
}
}
if strings.Contains(os.Args[1], "=") {
// -config=soar.yaml
configFile = strings.Split(os.Args[1], "=")[1]
}
} else {
for i, c := range os.Args {
if strings.HasPrefix(c, "-config") && i != 1 {
fmt.Println("-config must be the first arg")
}
}
}
return configFile
}
================================================
FILE: common/config_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"flag"
"os"
"path/filepath"
"runtime"
"testing"
"github.com/kr/pretty"
)
var update = flag.Bool("update", false, "update .golden files")
func TestMain(m *testing.M) {
// 初始化 init
if DevPath == "" {
_, file, _, _ := runtime.Caller(0)
DevPath, _ = filepath.Abs(filepath.Dir(filepath.Join(file, ".."+string(filepath.Separator))))
}
BaseDir = DevPath
err := ParseConfig("")
LogIfError(err, "init ParseConfig")
Log.Debug("mysql_test init")
// 分割线
flag.Parse()
m.Run()
// 环境清理
//
}
func TestParseConfig(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
err := ParseConfig("")
if err != nil {
t.Error("sqlparser.Parse Error:", err)
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestReadConfigFile(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
if Config == nil {
Config = new(Configuration)
}
Config.readConfigFile(filepath.Join(DevPath, "etc/soar.yaml"))
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestParseDSN(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
var dsns = []string{
// version < 0.11.0
"",
"user:password@hostname:3307/database",
"user:password@hostname:3307/database?charset=utf8",
"user:password@hostname:3307",
"user:password@hostname:/database",
"user:password@:3307/database",
"user@hostname/database",
"user:pwd:pwd@pwd/pwd@hostname/database",
"user:password@",
"hostname:3307/database",
"@hostname:3307/database",
"@hostname",
"hostname",
"@/database",
"@hostname:3307",
"@:3307/database",
":3307/database",
"/database",
// go-sql-driver dsn
"user@unix(/path/to/socket)/dbname",
"root:pw@unix(/tmp/mysql.sock)/myDatabase?loc=Local",
"user:password@tcp(localhost:5555)/dbname?tls=skip-verify",
"user:password@tcp(localhost:5555)/dbname?autocommit=true",
"user:password@/dbname?sql_mode=TRADITIONAL",
"user:password@tcp([de:ad:be:ef::ca:fe]:80)/dbname?timeout=90s",
"user:password@tcp([de:ad:be:ef::ca:fe]:80)/dbname?collation=utf8mb4_unicode_ci",
"id:password@tcp(your-amazonaws-uri.com:3306)/dbname",
"user@cloudsql(project-id:instance-name)/dbname",
"user@cloudsql(project-id:regionname:instance-name)/dbname",
//"user:password@tcp/dbname?charset=utf8mb4,utf8&sys_var=esc%40ped", multi key in map, pretty print in random order, by pass
"user:password@tcp/dbname?sys_var=esc%40ped",
"user:password@/dbname",
"user:password@/",
"user:password@tcp(localhost:3307)/database?timeout=5s",
}
err := GoldenDiff(func() {
for _, dsn := range dsns {
pretty.Println(dsn)
pretty.Println(ParseDSN(dsn, nil))
}
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestListReportTypes(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
err := GoldenDiff(func() { ListReportTypes() }, t.Name(), update)
if nil != err {
t.Fatal(err)
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestArgConfig(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
testArgs1 := [][]string{
{"soar", "-config", "=", "soar.yaml"},
{"soar", "-print-config", "-config", "soar.yaml"},
}
testArgs2 := [][]string{
{"soar", "-config", "soar.yaml"},
{"soar", "-config", "=soar.yaml"},
{"soar", "-config=soar.yaml"},
}
for _, args := range testArgs1 {
os.Args = args
configFile := ArgConfig()
if configFile != "" {
t.Errorf("should return '', but got %s", configFile)
}
}
for _, args := range testArgs2 {
os.Args = args
configFile := ArgConfig()
if configFile != "soar.yaml" {
t.Errorf("should return soar.yaml, but got %s", configFile)
}
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestPrintConfiguration(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
Config.readConfigFile(filepath.Join(DevPath, "etc/soar.yaml"))
oldLogOutput := Config.LogOutput
Config.LogOutput = "soar.log"
err := GoldenDiff(func() {
PrintConfiguration()
}, t.Name(), update)
if err != nil {
t.Error(err)
}
Config.LogOutput = oldLogOutput
Log.Debug("Exiting function: %s", GetFunctionName())
}
================================================
FILE: common/doc.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
// Package common contain many useful functions for logging, formatting and so on.
package common
================================================
FILE: common/example_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import "fmt"
func ExampleFormatDSN() {
Log.Debug("Entering function: %s", GetFunctionName())
dsxExp := newDSN(nil)
dsxExp.Addr = "127.0.0.1:3306"
dsxExp.Schema = "mysql"
dsxExp.User = "root"
dsxExp.Password = "1t'sB1g3rt"
dsxExp.Charset = "utf8mb4"
dsxExp.Disable = false
// 根据 &dsn 生成 dsnStr
fmt.Println(FormatDSN(dsxExp))
// Output: root:1t'sB1g3rt@tcp(127.0.0.1:3306)/mysql?timeout=3s&charset=utf8mb4
Log.Debug("Exiting function: %s", GetFunctionName())
}
func ExampleIsColsPart() {
Log.Debug("Entering function: %s", GetFunctionName())
// IsColsPart() 会 按照顺序 检查两个Column队列是否是包含（或相等）关系。
a := []*Column{{Name: "1"}, {Name: "2"}, {Name: "3"}}
b := []*Column{{Name: "1"}, {Name: "2"}}
c := []*Column{{Name: "1"}, {Name: "3"}}
d := []*Column{{Name: "1"}, {Name: "2"}, {Name: "3"}, {Name: "4"}}
id := []*Column{{Name: "id"}}
iD := []*Column{{Name: "iD"}}
ab := IsColsPart(a, b)
ac := IsColsPart(a, c)
ad := IsColsPart(a, d)
idiD := IsColsPart(id, iD) // 大小写对比
fmt.Println(ab, ac, ad, idiD)
// Output: true false true true
Log.Debug("Exiting function: %s", GetFunctionName())
}
func ExampleSortedKey() {
Log.Debug("Entering function: %s", GetFunctionName())
ages := map[string]int{
"a": 1,
"c": 3,
"d": 4,
"b": 2,
}
for _, name := range SortedKey(ages) {
fmt.Print(ages[name])
}
// Output: 1234
Log.Debug("Exiting function: %s", GetFunctionName())
}
================================================
FILE: common/logger.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"encoding/json"
"fmt"
"regexp"
"runtime"
"strings"
"github.com/astaxie/beego/logs"
)
// Log 使用 beego 的 log 库
var Log *logs.BeeLogger
// BaseDir 日志打印在binary的根路径
var BaseDir string
func init() {
Log = logs.NewLogger(0)
Log.EnableFuncCallDepth(true)
}
// LoggerInit Log配置初始化
func LoggerInit() {
Log.SetLevel(Config.LogLevel)
func() { _ = Log.DelLogger(logs.AdapterFile) }()
logConfig := map[string]interface{}{
"filename": Config.LogOutput,
"level": 7,
"maxlines": 0,
"maxsize": 0,
"daily": false,
"maxdays": 0,
}
logConfigJSON, _ := json.Marshal(logConfig)
err := Log.SetLogger(logs.AdapterFile, string(logConfigJSON))
if err != nil {
fmt.Println(err.Error())
}
}
// Caller returns the caller of the function that called it :)
// https://stackoverflow.com/questions/35212985/is-it-possible-get-information-about-caller-function-in-golang
func Caller() string {
// we get the callers as uintptrs - but we just need 1
fpcs := make([]uintptr, 1)
// skip 3 levels to get to the caller of whoever called Caller()
n := runtime.Callers(3, fpcs)
if n == 0 {
return "n/a" // proper error her would be better
}
// get the info of the actual function that's in the pointer
fun := runtime.FuncForPC(fpcs[0] - 1)
if fun == nil {
return "n/a"
}
// return its name
return fun.Name()
}
// GetFunctionName 获取调当前函数名
func GetFunctionName() string {
// Skip this function, and fetch the PC and file for its parent
pc, _, _, _ := runtime.Caller(1)
// Retrieve a Function object this functions parent
functionObject := runtime.FuncForPC(pc)
// Regex to extract just the function name (and not the module path)
extractFnName := regexp.MustCompile(`^.*\.(.*)$`)
fnName := extractFnName.ReplaceAllString(functionObject.Name(), "$1")
return fnName
}
// fileName get filename from path
func fileName(original string) string {
i := strings.LastIndex(original, "/")
if i == -1 {
return original
}
return original[i+1:]
}
// LogIfError 简化if err != nil 打 Error 日志代码长度
func LogIfError(err error, format string, v ...interface{}) {
if err != nil {
_, fn, line, _ := runtime.Caller(1)
if format == "" {
format = "[%s:%d] %s"
Log.Error(format, fileName(fn), line, err.Error())
} else {
format = "[%s:%d] " + format + " Error: %s"
Log.Error(format, fileName(fn), line, v, err.Error())
}
}
}
// LogIfWarn 简化if err != nil 打 Warn 日志代码长度
func LogIfWarn(err error, format string, v ...interface{}) {
if err != nil {
_, fn, line, _ := runtime.Caller(1)
if format == "" {
format = "[%s:%d] %s"
Log.Warn(format, fileName(fn), line, err.Error())
} else {
format = "[%s:%d] " + format + " Error: %s"
Log.Warn(format, fileName(fn), line, v, err.Error())
}
}
}
================================================
FILE: common/logger_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"errors"
"testing"
)
func TestLogger(t *testing.T) {
Log.Info("TestLogger_Info")
Log.Debug("TestLogger_Debug")
Log.Warning("TestLogger_Warning")
Log.Error("Warning_Error")
}
func TestCaller(t *testing.T) {
caller := Caller()
if caller != "testing.tRunner" {
t.Error("get caller failed")
}
}
func TestGetFunctionName(t *testing.T) {
f := GetFunctionName()
if f != "TestGetFunctionName" {
t.Error("get functionname failed")
}
}
func TestIfError(t *testing.T) {
err := errors.New("TestIfError")
LogIfError(err, "")
LogIfError(err, "func %s", "func_test")
}
func TestIfWarn(t *testing.T) {
err := errors.New("test")
LogIfWarn(err, "")
LogIfWarn(err, "func %s", "func_test")
}
================================================
FILE: common/markdown.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"encoding/base64"
"fmt"
"io/ioutil"
"net/http"
"os"
"strings"
"github.com/russross/blackfriday"
)
// BuiltinCSS 内置HTML风格
var BuiltinCSS = `
a:link,a:visited{text-decoration:none}h3,h4{margin-top:2em}h5,h6{margin-top:20px}h3,h4,h5,h6{margin-bottom:.5em;color:#000}body,h1,h2,h3,h4,h5,h6{color:#000}ol,ul{margin:0 0 0 30px;padding:0 0 12px 6px}ol,ol ol{list-style-position:outside}table td p,table th p{margin-bottom:0}input,select{vertical-align:middle;padding:0}h5,h6,input,select{padding:0}hr,table,textarea{width:100%}body{margin:20px auto;width:800px;background-color:#fff;font:13px "Myriad Pro","Lucida Grande",Lucida,Verdana,sans-serif}h1,table th p{font-weight:700}a:link{color:#00f}a:visited{color:#00a}a:active,a:hover{color:#f60;text-decoration:underline}* html code,* html pre{font-size:101%}code,pre{font-size:11px;font-family:monaco,courier,consolas,monospace}pre{border:1px solid #c7cfd5;background:#f1f5f9;margin:20px 0;padding:8px;text-align:left}hr{color:#919699;size:1;noshade:"noshade"}h1,h2,h3,h4,h5,h6{font-family:"Myriad Pro","Lucida Grande",Lucida,Verdana,sans-serif;font-weight:700}h1{margin-top:1em;margin-bottom:25px;font-size:30px}h2{margin-top:2.5em;font-size:24px;padding-bottom:2px;border-bottom:1px solid #919699}h3{font-size:17px}h4{font-size:15px}h5{font-size:13px}h6{font-size:11px}table td,table th{font-size:12px;border-bottom:1px solid #919699;border-right:1px solid #919699}p{margin-top:0;margin-bottom:10px}ul{list-style:square}li{margin-top:7px}ol{list-style-type:decimal}ol ol{list-style-type:lower-alpha;margin:7px 0 0 30px;padding:0 0 0 10px}ul ul{margin-left:40px;padding:0 0 0 6px}li>p{display:inline}li>a+p,li>p+p{display:block}table{border-top:1px solid #919699;border-left:1px solid #919699;border-spacing:0}table th{padding:4px 8px;background:#E2E2E2}table td{padding:8px;vertical-align:top}table td p+p,table td p+p+p{margin-top:5px}form{margin:0}button{margin:3px 0 10px}input{margin:0 0 5px}select{margin:0 0 3px}textarea{margin:0 0 10px}
`
// BuiltinJavascript 内置 SQL 美化 Javascript 脚本
var BuiltinJavascript = `IWZ1bmN0aW9uKGUsRSl7Im9iamVjdCI9PXR5cGVvZiBleHBvcnRzJiYib2JqZWN0Ij09dHlwZW9mIG1vZHVsZT9tb2R1bGUuZXhwb3J0cz1FKCk6ImZ1bmN0aW9uIj09dHlwZW9mIGRlZmluZSYmZGVmaW5lLmFtZD9kZWZpbmUoW10sRSk6Im9iamVjdCI9PXR5cGVvZiBleHBvcnRzP2V4cG9ydHMuc3FsRm9ybWF0dGVyPUUoKTplLnNxbEZvcm1hdHRlcj1FKCl9KHRoaXMsZnVuY3Rpb24oKXtyZXR1cm4gZnVuY3Rpb24oZSl7ZnVuY3Rpb24gRShuKXtpZih0W25dKXJldHVybiB0W25dLmV4cG9ydHM7dmFyIHI9dFtuXT17ZXhwb3J0czp7fSxpZDpuLGxvYWRlZDohMX07cmV0dXJuIGVbbl0uY2FsbChyLmV4cG9ydHMscixyLmV4cG9ydHMsRSksci5sb2FkZWQ9ITAsci5leHBvcnRzfXZhciB0PXt9O3JldHVybiBFLm09ZSxFLmM9dCxFLnA9IiIsRSgwKX0oW2Z1bmN0aW9uKGUsRSx0KXsidXNlIHN0cmljdCI7ZnVuY3Rpb24gbihlKXtyZXR1cm4gZSYmZS5fX2VzTW9kdWxlP2U6eyJkZWZhdWx0IjplfX1FLl9fZXNNb2R1bGU9ITA7dmFyIHI9dCgxOCksVD1uKHIpLFI9dCgxOSksbz1uKFIpLE49dCgyMCksQT1uKE4pLEk9dCgyMSksTz1uKEkpO0VbImRlZmF1bHQiXT17Zm9ybWF0OmZ1bmN0aW9uKGUsRSl7c3dpdGNoKEU9RXx8e30sRS5sYW5ndWFnZSl7Y2FzZSJkYjIiOnJldHVybiBuZXcgVFsiZGVmYXVsdCJdKEUpLmZvcm1hdChlKTtjYXNlIm4xcWwiOnJldHVybiBuZXcgb1siZGVmYXVsdCJdKEUpLmZvcm1hdChlKTtjYXNlInBsL3NxbCI6cmV0dXJuIG5ldyBBWyJkZWZhdWx0Il0oRSkuZm9ybWF0KGUpO2Nhc2Uic3FsIjpjYXNlIHZvaWQgMDpyZXR1cm4gbmV3IE9bImRlZmF1bHQiXShFKS5mb3JtYXQoZSk7ZGVmYXVsdDp0aHJvdyBFcnJvcigiVW5zdXBwb3J0ZWQgU1FMIGRpYWxlY3Q6ICIrRS5sYW5ndWFnZSl9fX0sZS5leHBvcnRzPUVbImRlZmF1bHQiXX0sZnVuY3Rpb24oZSxFKXsidXNlIHN0cmljdCI7RS5fX2VzTW9kdWxlPSEwLEVbImRlZmF1bHQiXT1mdW5jdGlvbihlLEUpe2lmKCEoZSBpbnN0YW5jZW9mIEUpKXRocm93IG5ldyBUeXBlRXJyb3IoIkNhbm5vdCBjYWxsIGEgY2xhc3MgYXMgYSBmdW5jdGlvbiIpfX0sZnVuY3Rpb24oZSxFLHQpe3ZhciBuPXQoMzkpLHI9Im9iamVjdCI9PXR5cGVvZiBzZWxmJiZzZWxmJiZzZWxmLk9iamVjdD09PU9iamVjdCYmc2VsZixUPW58fHJ8fEZ1bmN0aW9uKCJyZXR1cm4gdGhpcyIpKCk7ZS5leHBvcnRzPVR9LGZ1bmN0aW9uKGUsRSx0KXtmdW5jdGlvbiBuKGUsRSl7dmFyIHQ9VChlLEUpO3JldHVybiByKHQpP3Q6dm9pZCAwfXZhciByPXQoMzMpLFQ9dCg0MSk7ZS5leHBvcnRzPW59LGZ1bmN0aW9uKGUsRSx0KXsidXNlIHN0cmljdCI7ZnVuY3Rpb24gbihlKXtyZXR1cm4gZSYmZS5fX2VzTW9kdWxlP2U6eyJkZWZhdWx0IjplfX1FLl9fZXNNb2R1bGU9ITA7dmFyIHI9dCgxKSxUPW4ociksUj10KDY2KSxvPW4oUiksTj10KDcpLEE9bihOKSxJPXQoMTUpLE89bihJKSxpPXQoMTYpLFM9bihpKSx1PXQoMTcpLEw9bih1KSxDPWZ1bmN0aW9uKCl7ZnVuY3Rpb24gZShFLHQpeygwLFRbImRlZmF1bHQiXSkodGhpcyxlKSx0aGlzLmNmZz1FfHx7fSx0aGlzLmluZGVudGF0aW9uPW5ldyBPWyJkZWZhdWx0Il0odGhpcy5jZmcuaW5kZW50KSx0aGlzLmlubGluZUJsb2NrPW5ldyBTWyJkZWZhdWx0Il0sdGhpcy5wYXJhbXM9bmV3IExbImRlZmF1bHQiXSh0aGlzLmNmZy5wYXJhbXMpLHRoaXMudG9rZW5pemVyPXQsdGhpcy5wcmV2aW91c1Jlc2VydmVkV29yZD17fX1yZXR1cm4gZS5wcm90b3R5cGUuZm9ybWF0PWZ1bmN0aW9uKGUpe3ZhciBFPXRoaXMudG9rZW5pemVyLnRva2VuaXplKGUpLHQ9dGhpcy5nZXRGb3JtYXR0ZWRRdWVyeUZyb21Ub2tlbnMoRSk7cmV0dXJuIHQudHJpbSgpfSxlLnByb3RvdHlwZS5nZXRGb3JtYXR0ZWRRdWVyeUZyb21Ub2tlbnM9ZnVuY3Rpb24oZSl7dmFyIEU9dGhpcyx0PSIiO3JldHVybiBlLmZvckVhY2goZnVuY3Rpb24obixyKXtuLnR5cGUhPT1BWyJkZWZhdWx0Il0uV0hJVEVTUEFDRSYmKG4udHlwZT09PUFbImRlZmF1bHQiXS5MSU5FX0NPTU1FTlQ/dD1FLmZvcm1hdExpbmVDb21tZW50KG4sdCk6bi50eXBlPT09QVsiZGVmYXVsdCJdLkJMT0NLX0NPTU1FTlQ/dD1FLmZvcm1hdEJsb2NrQ29tbWVudChuLHQpOm4udHlwZT09PUFbImRlZmF1bHQiXS5SRVNFUlZFRF9UT1BMRVZFTD8odD1FLmZvcm1hdFRvcGxldmVsUmVzZXJ2ZWRXb3JkKG4sdCksRS5wcmV2aW91c1Jlc2VydmVkV29yZD1uKTpuLnR5cGU9PT1BWyJkZWZhdWx0Il0uUkVTRVJWRURfTkVXTElORT8odD1FLmZvcm1hdE5ld2xpbmVSZXNlcnZlZFdvcmQobix0KSxFLnByZXZpb3VzUmVzZXJ2ZWRXb3JkPW4pOm4udHlwZT09PUFbImRlZmF1bHQiXS5SRVNFUlZFRD8odD1FLmZvcm1hdFdpdGhTcGFjZXMobix0KSxFLnByZXZpb3VzUmVzZXJ2ZWRXb3JkPW4pOnQ9bi50eXBlPT09QVsiZGVmYXVsdCJdLk9QRU5fUEFSRU4/RS5mb3JtYXRPcGVuaW5nUGFyZW50aGVzZXMoZSxyLHQpOm4udHlwZT09PUFbImRlZmF1bHQiXS5DTE9TRV9QQVJFTj9FLmZvcm1hdENsb3NpbmdQYXJlbnRoZXNlcyhuLHQpOm4udHlwZT09PUFbImRlZmF1bHQiXS5QTEFDRUhPTERFUj9FLmZvcm1hdFBsYWNlaG9sZGVyKG4sdCk6IiwiPT09bi52YWx1ZT9FLmZvcm1hdENvbW1hKG4sdCk6IjoiPT09bi52YWx1ZT9FLmZvcm1hdFdpdGhTcGFjZUFmdGVyKG4sdCk6Ii4iPT09bi52YWx1ZXx8IjsiPT09bi52YWx1ZT9FLmZvcm1hdFdpdGhvdXRTcGFjZXMobix0KTpFLmZvcm1hdFdpdGhTcGFjZXMobix0KSl9KSx0fSxlLnByb3RvdHlwZS5mb3JtYXRMaW5lQ29tbWVudD1mdW5jdGlvbihlLEUpe3JldHVybiB0aGlzLmFkZE5ld2xpbmUoRStlLnZhbHVlKX0sZS5wcm90b3R5cGUuZm9ybWF0QmxvY2tDb21tZW50PWZ1bmN0aW9uKGUsRSl7cmV0dXJuIHRoaXMuYWRkTmV3bGluZSh0aGlzLmFkZE5ld2xpbmUoRSkrdGhpcy5pbmRlbnRDb21tZW50KGUudmFsdWUpKX0sZS5wcm90b3R5cGUuaW5kZW50Q29tbWVudD1mdW5jdGlvbihlKXtyZXR1cm4gZS5yZXBsYWNlKC9cbi9nLCJcbiIrdGhpcy5pbmRlbnRhdGlvbi5nZXRJbmRlbnQoKSl9LGUucHJvdG90eXBlLmZvcm1hdFRvcGxldmVsUmVzZXJ2ZWRXb3JkPWZ1bmN0aW9uKGUsRSl7cmV0dXJuIHRoaXMuaW5kZW50YXRpb24uZGVjcmVhc2VUb3BMZXZlbCgpLEU9dGhpcy5hZGROZXdsaW5lKEUpLHRoaXMuaW5kZW50YXRpb24uaW5jcmVhc2VUb3BsZXZlbCgpLEUrPXRoaXMuZXF1YWxpemVXaGl0ZXNwYWNlKGUudmFsdWUpLHRoaXMuYWRkTmV3bGluZShFKX0sZS5wcm90b3R5cGUuZm9ybWF0TmV3bGluZVJlc2VydmVkV29yZD1mdW5jdGlvbihlLEUpe3JldHVybiB0aGlzLmFkZE5ld2xpbmUoRSkrdGhpcy5lcXVhbGl6ZVdoaXRlc3BhY2UoZS52YWx1ZSkrIiAifSxlLnByb3RvdHlwZS5lcXVhbGl6ZVdoaXRlc3BhY2U9ZnVuY3Rpb24oZSl7cmV0dXJuIGUucmVwbGFjZSgvXHMrL2csIiAiKX0sZS5wcm90b3R5cGUuZm9ybWF0T3BlbmluZ1BhcmVudGhlc2VzPWZ1bmN0aW9uKGUsRSx0KXt2YXIgbj1lW0UtMV07cmV0dXJuIG4mJm4udHlwZSE9PUFbImRlZmF1bHQiXS5XSElURVNQQUNFJiZuLnR5cGUhPT1BWyJkZWZhdWx0Il0uT1BFTl9QQVJFTiYmKHQ9KDAsb1siZGVmYXVsdCJdKSh0KSksdCs9ZVtFXS52YWx1ZSx0aGlzLmlubGluZUJsb2NrLmJlZ2luSWZQb3NzaWJsZShlLEUpLHRoaXMuaW5saW5lQmxvY2suaXNBY3RpdmUoKXx8KHRoaXMuaW5kZW50YXRpb24uaW5jcmVhc2VCbG9ja0xldmVsKCksdD10aGlzLmFkZE5ld2xpbmUodCkpLHR9LGUucHJvdG90eXBlLmZvcm1hdENsb3NpbmdQYXJlbnRoZXNlcz1mdW5jdGlvbihlLEUpe3JldHVybiB0aGlzLmlubGluZUJsb2NrLmlzQWN0aXZlKCk/KHRoaXMuaW5saW5lQmxvY2suZW5kKCksdGhpcy5mb3JtYXRXaXRoU3BhY2VBZnRlcihlLEUpKToodGhpcy5pbmRlbnRhdGlvbi5kZWNyZWFzZUJsb2NrTGV2ZWwoKSx0aGlzLmZvcm1hdFdpdGhTcGFjZXMoZSx0aGlzLmFkZE5ld2xpbmUoRSkpKX0sZS5wcm90b3R5cGUuZm9ybWF0UGxhY2Vob2xkZXI9ZnVuY3Rpb24oZSxFKXtyZXR1cm4gRSt0aGlzLnBhcmFtcy5nZXQoZSkrIiAifSxlLnByb3RvdHlwZS5mb3JtYXRDb21tYT1mdW5jdGlvbihlLEUpe3JldHVybiBFPSgwLG9bImRlZmF1bHQiXSkoRSkrZS52YWx1ZSsiICIsdGhpcy5pbmxpbmVCbG9jay5pc0FjdGl2ZSgpP0U6L15MSU1JVCQvaS50ZXN0KHRoaXMucHJldmlvdXNSZXNlcnZlZFdvcmQudmFsdWUpP0U6dGhpcy5hZGROZXdsaW5lKEUpfSxlLnByb3RvdHlwZS5mb3JtYXRXaXRoU3BhY2VBZnRlcj1mdW5jdGlvbihlLEUpe3JldHVybigwLG9bImRlZmF1bHQiXSkoRSkrZS52YWx1ZSsiICJ9LGUucHJvdG90eXBlLmZvcm1hdFdpdGhvdXRTcGFjZXM9ZnVuY3Rpb24oZSxFKXtyZXR1cm4oMCxvWyJkZWZhdWx0Il0pKEUpK2UudmFsdWV9LGUucHJvdG90eXBlLmZvcm1hdFdpdGhTcGFjZXM9ZnVuY3Rpb24oZSxFKXtyZXR1cm4gRStlLnZhbHVlKyIgIn0sZS5wcm90b3R5cGUuYWRkTmV3bGluZT1mdW5jdGlvbihlKXtyZXR1cm4oMCxvWyJkZWZhdWx0Il0pKGUpKyJcbiIrdGhpcy5pbmRlbnRhdGlvbi5nZXRJbmRlbnQoKX0sZX0oKTtFWyJkZWZhdWx0Il09QyxlLmV4cG9ydHM9RVsiZGVmYXVsdCJdfSxmdW5jdGlvbihlLEUsdCl7InVzZSBzdHJpY3QiO2Z1bmN0aW9uIG4oZSl7cmV0dXJuIGUmJmUuX19lc01vZHVsZT9lOnsiZGVmYXVsdCI6ZX19RS5fX2VzTW9kdWxlPSEwO3ZhciByPXQoMSksVD1uKHIpLFI9dCg1OCksbz1uKFIpLE49dCg1MyksQT1uKE4pLEk9dCg3KSxPPW4oSSksaT1mdW5jdGlvbigpe2Z1bmN0aW9uIGUoRSl7KDAsVFsiZGVmYXVsdCJdKSh0aGlzLGUpLHRoaXMuV0hJVEVTUEFDRV9SRUdFWD0vXihccyspLyx0aGlzLk5VTUJFUl9SRUdFWD0vXigoLVxzKik/WzAtOV0rKFwuWzAtOV0rKT98MHhbMC05YS1mQS1GXSt8MGJbMDFdKylcYi8sdGhpcy5PUEVSQVRPUl9SRUdFWD0vXighPXw8Pnw9PXw8PXw+PXwhPHwhPnxcfFx8fDo6fC0+PnwtPnx+flwqfH5+fCF+flwqfCF+fnx+XCp8IX5cKnwhfnwuKS8sdGhpcy5CTE9DS19DT01NRU5UX1JFR0VYPS9eKFwvXCpbXl0qPyg/OlwqXC98JCkpLyx0aGlzLkxJTkVfQ09NTUVOVF9SRUdFWD10aGlzLmNyZWF0ZUxpbmVDb21tZW50UmVnZXgoRS5saW5lQ29tbWVudFR5cGVzKSx0aGlzLlJFU0VSVkVEX1RPUExFVkVMX1JFR0VYPXRoaXMuY3JlYXRlUmVzZXJ2ZWRXb3JkUmVnZXgoRS5yZXNlcnZlZFRvcGxldmVsV29yZHMpLHRoaXMuUkVTRVJWRURfTkVXTElORV9SRUdFWD10aGlzLmNyZWF0ZVJlc2VydmVkV29yZFJlZ2V4KEUucmVzZXJ2ZWROZXdsaW5lV29yZHMpLHRoaXMuUkVTRVJWRURfUExBSU5fUkVHRVg9dGhpcy5jcmVhdGVSZXNlcnZlZFdvcmRSZWdleChFLnJlc2VydmVkV29yZHMpLHRoaXMuV09SRF9SRUdFWD10aGlzLmNyZWF0ZVdvcmRSZWdleChFLnNwZWNpYWxXb3JkQ2hhcnMpLHRoaXMuU1RSSU5HX1JFR0VYPXRoaXMuY3JlYXRlU3RyaW5nUmVnZXgoRS5zdHJpbmdUeXBlcyksdGhpcy5PUEVOX1BBUkVOX1JFR0VYPXRoaXMuY3JlYXRlUGFyZW5SZWdleChFLm9wZW5QYXJlbnMpLHRoaXMuQ0xPU0VfUEFSRU5fUkVHRVg9dGhpcy5jcmVhdGVQYXJlblJlZ2V4KEUuY2xvc2VQYXJlbnMpLHRoaXMuSU5ERVhFRF9QTEFDRUhPTERFUl9SRUdFWD10aGlzLmNyZWF0ZVBsYWNlaG9sZGVyUmVnZXgoRS5pbmRleGVkUGxhY2Vob2xkZXJUeXBlcywiWzAtOV0qIiksdGhpcy5JREVOVF9OQU1FRF9QTEFDRUhPTERFUl9SRUdFWD10aGlzLmNyZWF0ZVBsYWNlaG9sZGVyUmVnZXgoRS5uYW1lZFBsYWNlaG9sZGVyVHlwZXMsIlthLXpBLVowLTkuXyRdKyIpLHRoaXMuU1RSSU5HX05BTUVEX1BMQUNFSE9MREVSX1JFR0VYPXRoaXMuY3JlYXRlUGxhY2Vob2xkZXJSZWdleChFLm5hbWVkUGxhY2Vob2xkZXJUeXBlcyx0aGlzLmNyZWF0ZVN0cmluZ1BhdHRlcm4oRS5zdHJpbmdUeXBlcykpfXJldHVybiBlLnByb3RvdHlwZS5jcmVhdGVMaW5lQ29tbWVudFJlZ2V4PWZ1bmN0aW9uKGUpe3JldHVybiBSZWdFeHAoIl4oKD86IitlLm1hcChmdW5jdGlvbihlKXtyZXR1cm4oMCxBWyJkZWZhdWx0Il0pKGUpfSkuam9pbigifCIpKyIpLio/KD86XG58JCkpIil9LGUucHJvdG90eXBlLmNyZWF0ZVJlc2VydmVkV29yZFJlZ2V4PWZ1bmN0aW9uKGUpe3ZhciBFPWUuam9pbigifCIpLnJlcGxhY2UoLyAvZywiXFxzKyIpO3JldHVybiBSZWdFeHAoIl4oIitFKyIpXFxiIiwiaSIpfSxlLnByb3RvdHlwZS5jcmVhdGVXb3JkUmVnZXg9ZnVuY3Rpb24oKXt2YXIgZT1hcmd1bWVudHMubGVuZ3RoPjAmJnZvaWQgMCE9PWFyZ3VtZW50c1swXT9hcmd1bWVudHNbMF06W107cmV0dXJuIFJlZ0V4cCgiXihbXFx3IitlLmpvaW4oIiIpKyJdKykiKX0sZS5wcm90b3R5cGUuY3JlYXRlU3RyaW5nUmVnZXg9ZnVuY3Rpb24oZSl7cmV0dXJuIFJlZ0V4cCgiXigiK3RoaXMuY3JlYXRlU3RyaW5nUGF0dGVybihlKSsiKSIpfSxlLnByb3RvdHlwZS5jcmVhdGVTdHJpbmdQYXR0ZXJuPWZ1bmN0aW9uKGUpe3ZhciBFPXsiYGAiOiIoKGBbXmBdKigkfGApKSspIiwiW10iOiIoKFxcW1teXFxdXSooJHxcXF0pKShcXF1bXlxcXV0qKCR8XFxdKSkqKSIsJyIiJzonKCgiW14iXFxcXF0qKD86XFxcXC5bXiJcXFxcXSopKigifCQpKSspJywiJyciOiIoKCdbXidcXFxcXSooPzpcXFxcLlteJ1xcXFxdKikqKCd8JCkpKykiLCJOJyciOiIoKE4nW15OJ1xcXFxdKig/OlxcXFwuW15OJ1xcXFxdKikqKCd8JCkpKykifTtyZXR1cm4gZS5tYXAoZnVuY3Rpb24oZSl7cmV0dXJuIEVbZV19KS5qb2luKCJ8Iil9LGUucHJvdG90eXBlLmNyZWF0ZVBhcmVuUmVnZXg9ZnVuY3Rpb24oZSl7dmFyIEU9dGhpcztyZXR1cm4gUmVnRXhwKCJeKCIrZS5tYXAoZnVuY3Rpb24oZSl7cmV0dXJuIEUuZXNjYXBlUGFyZW4oZSl9KS5qb2luKCJ8IikrIikiLCJpIil9LGUucHJvdG90eXBlLmVzY2FwZVBhcmVuPWZ1bmN0aW9uKGUpe3JldHVybiAxPT09ZS5sZW5ndGg/KDAsQVsiZGVmYXVsdCJdKShlKToiXFxiIitlKyJcXGIifSxlLnByb3RvdHlwZS5jcmVhdGVQbGFjZWhvbGRlclJlZ2V4PWZ1bmN0aW9uKGUsRSl7aWYoKDAsb1siZGVmYXVsdCJdKShlKSlyZXR1cm4hMTt2YXIgdD1lLm1hcChBWyJkZWZhdWx0Il0pLmpvaW4oInwiKTtyZXR1cm4gUmVnRXhwKCJeKCg/OiIrdCsiKSg/OiIrRSsiKSkiKX0sZS5wcm90b3R5cGUudG9rZW5pemU9ZnVuY3Rpb24oZSl7Zm9yKHZhciBFPVtdLHQ9dm9pZCAwO2UubGVuZ3RoOyl0PXRoaXMuZ2V0TmV4dFRva2VuKGUsdCksZT1lLnN1YnN0cmluZyh0LnZhbHVlLmxlbmd0aCksRS5wdXNoKHQpO3JldHVybiBFfSxlLnByb3RvdHlwZS5nZXROZXh0VG9rZW49ZnVuY3Rpb24oZSxFKXtyZXR1cm4gdGhpcy5nZXRXaGl0ZXNwYWNlVG9rZW4oZSl8fHRoaXMuZ2V0Q29tbWVudFRva2VuKGUpfHx0aGlzLmdldFN0cmluZ1Rva2VuKGUpfHx0aGlzLmdldE9wZW5QYXJlblRva2VuKGUpfHx0aGlzLmdldENsb3NlUGFyZW5Ub2tlbihlKXx8dGhpcy5nZXRQbGFjZWhvbGRlclRva2VuKGUpfHx0aGlzLmdldE51bWJlclRva2VuKGUpfHx0aGlzLmdldFJlc2VydmVkV29yZFRva2VuKGUsRSl8fHRoaXMuZ2V0V29yZFRva2VuKGUpfHx0aGlzLmdldE9wZXJhdG9yVG9rZW4oZSl9LGUucHJvdG90eXBlLmdldFdoaXRlc3BhY2VUb2tlbj1mdW5jdGlvbihlKXtyZXR1cm4gdGhpcy5nZXRUb2tlbk9uRmlyc3RNYXRjaCh7aW5wdXQ6ZSx0eXBlOk9bImRlZmF1bHQiXS5XSElURVNQQUNFLHJlZ2V4OnRoaXMuV0hJVEVTUEFDRV9SRUdFWH0pfSxlLnByb3RvdHlwZS5nZXRDb21tZW50VG9rZW49ZnVuY3Rpb24oZSl7cmV0dXJuIHRoaXMuZ2V0TGluZUNvbW1lbnRUb2tlbihlKXx8dGhpcy5nZXRCbG9ja0NvbW1lbnRUb2tlbihlKX0sZS5wcm90b3R5cGUuZ2V0TGluZUNvbW1lbnRUb2tlbj1mdW5jdGlvbihlKXtyZXR1cm4gdGhpcy5nZXRUb2tlbk9uRmlyc3RNYXRjaCh7aW5wdXQ6ZSx0eXBlOk9bImRlZmF1bHQiXS5MSU5FX0NPTU1FTlQscmVnZXg6dGhpcy5MSU5FX0NPTU1FTlRfUkVHRVh9KX0sZS5wcm90b3R5cGUuZ2V0QmxvY2tDb21tZW50VG9rZW49ZnVuY3Rpb24oZSl7cmV0dXJuIHRoaXMuZ2V0VG9rZW5PbkZpcnN0TWF0Y2goe2lucHV0OmUsdHlwZTpPWyJkZWZhdWx0Il0uQkxPQ0tfQ09NTUVOVCxyZWdleDp0aGlzLkJMT0NLX0NPTU1FTlRfUkVHRVh9KX0sZS5wcm90b3R5cGUuZ2V0U3RyaW5nVG9rZW49ZnVuY3Rpb24oZSl7cmV0dXJuIHRoaXMuZ2V0VG9rZW5PbkZpcnN0TWF0Y2goe2lucHV0OmUsdHlwZTpPWyJkZWZhdWx0Il0uU1RSSU5HLHJlZ2V4OnRoaXMuU1RSSU5HX1JFR0VYfSl9LGUucHJvdG90eXBlLmdldE9wZW5QYXJlblRva2VuPWZ1bmN0aW9uKGUpe3JldHVybiB0aGlzLmdldFRva2VuT25GaXJzdE1hdGNoKHtpbnB1dDplLHR5cGU6T1siZGVmYXVsdCJdLk9QRU5fUEFSRU4scmVnZXg6dGhpcy5PUEVOX1BBUkVOX1JFR0VYfSl9LGUucHJvdG90eXBlLmdldENsb3NlUGFyZW5Ub2tlbj1mdW5jdGlvbihlKXtyZXR1cm4gdGhpcy5nZXRUb2tlbk9uRmlyc3RNYXRjaCh7aW5wdXQ6ZSx0eXBlOk9bImRlZmF1bHQiXS5DTE9TRV9QQVJFTixyZWdleDp0aGlzLkNMT1NFX1BBUkVOX1JFR0VYfSl9LGUucHJvdG90eXBlLmdldFBsYWNlaG9sZGVyVG9rZW49ZnVuY3Rpb24oZSl7cmV0dXJuIHRoaXMuZ2V0SWRlbnROYW1lZFBsYWNlaG9sZGVyVG9rZW4oZSl8fHRoaXMuZ2V0U3RyaW5nTmFtZWRQbGFjZWhvbGRlclRva2VuKGUpfHx0aGlzLmdldEluZGV4ZWRQbGFjZWhvbGRlclRva2VuKGUpfSxlLnByb3RvdHlwZS5nZXRJZGVudE5hbWVkUGxhY2Vob2xkZXJUb2tlbj1mdW5jdGlvbihlKXtyZXR1cm4gdGhpcy5nZXRQbGFjZWhvbGRlclRva2VuV2l0aEtleSh7aW5wdXQ6ZSxyZWdleDp0aGlzLklERU5UX05BTUVEX1BMQUNFSE9MREVSX1JFR0VYLHBhcnNlS2V5OmZ1bmN0aW9uKGUpe3JldHVybiBlLnNsaWNlKDEpfX0pfSxlLnByb3RvdHlwZS5nZXRTdHJpbmdOYW1lZFBsYWNlaG9sZGVyVG9rZW49ZnVuY3Rpb24oZSl7dmFyIEU9dGhpcztyZXR1cm4gdGhpcy5nZXRQbGFjZWhvbGRlclRva2VuV2l0aEtleSh7aW5wdXQ6ZSxyZWdleDp0aGlzLlNUUklOR19OQU1FRF9QTEFDRUhPTERFUl9SRUdFWCxwYXJzZUtleTpmdW5jdGlvbihlKXtyZXR1cm4gRS5nZXRFc2NhcGVkUGxhY2Vob2xkZXJLZXkoe2tleTplLnNsaWNlKDIsLTEpLHF1b3RlQ2hhcjplLnNsaWNlKC0xKX0pfX0pfSxlLnByb3RvdHlwZS5nZXRJbmRleGVkUGxhY2Vob2xkZXJUb2tlbj1mdW5jdGlvbihlKXtyZXR1cm4gdGhpcy5nZXRQbGFjZWhvbGRlclRva2VuV2l0aEtleSh7aW5wdXQ6ZSxyZWdleDp0aGlzLklOREVYRURfUExBQ0VIT0xERVJfUkVHRVgscGFyc2VLZXk6ZnVuY3Rpb24oZSl7cmV0dXJuIGUuc2xpY2UoMSl9fSl9LGUucHJvdG90eXBlLmdldFBsYWNlaG9sZGVyVG9rZW5XaXRoS2V5PWZ1bmN0aW9uKGUpe3ZhciBFPWUuaW5wdXQsdD1lLnJlZ2V4LG49ZS5wYXJzZUtleSxyPXRoaXMuZ2V0VG9rZW5PbkZpcnN0TWF0Y2goe2lucHV0OkUscmVnZXg6dCx0eXBlOk9bImRlZmF1bHQiXS5QTEFDRUhPTERFUn0pO3JldHVybiByJiYoci5rZXk9bihyLnZhbHVlKSkscn0sZS5wcm90b3R5cGUuZ2V0RXNjYXBlZFBsYWNlaG9sZGVyS2V5PWZ1bmN0aW9uKGUpe3ZhciBFPWUua2V5LHQ9ZS5xdW90ZUNoYXI7cmV0dXJuIEUucmVwbGFjZShSZWdFeHAoKDAsQVsiZGVmYXVsdCJdKSgiXFwiKSt0LCJnIiksdCl9LGUucHJvdG90eXBlLmdldE51bWJlclRva2VuPWZ1bmN0aW9uKGUpe3JldHVybiB0aGlzLmdldFRva2VuT25GaXJzdE1hdGNoKHtpbnB1dDplLHR5cGU6T1siZGVmYXVsdCJdLk5VTUJFUixyZWdleDp0aGlzLk5VTUJFUl9SRUdFWH0pfSxlLnByb3RvdHlwZS5nZXRPcGVyYXRvclRva2VuPWZ1bmN0aW9uKGUpe3JldHVybiB0aGlzLmdldFRva2VuT25GaXJzdE1hdGNoKHtpbnB1dDplLHR5cGU6T1siZGVmYXVsdCJdLk9QRVJBVE9SLHJlZ2V4OnRoaXMuT1BFUkFUT1JfUkVHRVh9KX0sZS5wcm90b3R5cGUuZ2V0UmVzZXJ2ZWRXb3JkVG9rZW49ZnVuY3Rpb24oZSxFKXtpZighRXx8IUUudmFsdWV8fCIuIiE9PUUudmFsdWUpcmV0dXJuIHRoaXMuZ2V0VG9wbGV2ZWxSZXNlcnZlZFRva2VuKGUpfHx0aGlzLmdldE5ld2xpbmVSZXNlcnZlZFRva2VuKGUpfHx0aGlzLmdldFBsYWluUmVzZXJ2ZWRUb2tlbihlKX0sZS5wcm90b3R5cGUuZ2V0VG9wbGV2ZWxSZXNlcnZlZFRva2VuPWZ1bmN0aW9uKGUpe3JldHVybiB0aGlzLmdldFRva2VuT25GaXJzdE1hdGNoKHtpbnB1dDplLHR5cGU6T1siZGVmYXVsdCJdLlJFU0VSVkVEX1RPUExFVkVMLHJlZ2V4OnRoaXMuUkVTRVJWRURfVE9QTEVWRUxfUkVHRVh9KX0sZS5wcm90b3R5cGUuZ2V0TmV3bGluZVJlc2VydmVkVG9rZW49ZnVuY3Rpb24oZSl7cmV0dXJuIHRoaXMuZ2V0VG9rZW5PbkZpcnN0TWF0Y2goe2lucHV0OmUsdHlwZTpPWyJkZWZhdWx0Il0uUkVTRVJWRURfTkVXTElORSxyZWdleDp0aGlzLlJFU0VSVkVEX05FV0xJTkVfUkVHRVh9KX0sZS5wcm90b3R5cGUuZ2V0UGxhaW5SZXNlcnZlZFRva2VuPWZ1bmN0aW9uKGUpe3JldHVybiB0aGlzLmdldFRva2VuT25GaXJzdE1hdGNoKHtpbnB1dDplLHR5cGU6T1siZGVmYXVsdCJdLlJFU0VSVkVELHJlZ2V4OnRoaXMuUkVTRVJWRURfUExBSU5fUkVHRVh9KX0sZS5wcm90b3R5cGUuZ2V0V29yZFRva2VuPWZ1bmN0aW9uKGUpe3JldHVybiB0aGlzLmdldFRva2VuT25GaXJzdE1hdGNoKHtpbnB1dDplLHR5cGU6T1siZGVmYXVsdCJdLldPUkQscmVnZXg6dGhpcy5XT1JEX1JFR0VYfSl9LGUucHJvdG90eXBlLmdldFRva2VuT25GaXJzdE1hdGNoPWZ1bmN0aW9uKGUpe3ZhciBFPWUuaW5wdXQsdD1lLnR5cGUsbj1lLnJlZ2V4LHI9RS5tYXRjaChuKTtpZihyKXJldHVybnt0eXBlOnQsdmFsdWU6clsxXX19LGV9KCk7RVsiZGVmYXVsdCJdPWksZS5leHBvcnRzPUVbImRlZmF1bHQiXX0sZnVuY3Rpb24oZSxFKXtmdW5jdGlvbiB0KGUpe3ZhciBFPXR5cGVvZiBlO3JldHVybiBudWxsIT1lJiYoIm9iamVjdCI9PUV8fCJmdW5jdGlvbiI9PUUpfWUuZXhwb3J0cz10fSxmdW5jdGlvbihlLEUpeyJ1c2Ugc3RyaWN0IjtFLl9fZXNNb2R1bGU9ITAsRVsiZGVmYXVsdCJdPXtXSElURVNQQUNFOiJ3aGl0ZXNwYWNlIixXT1JEOiJ3b3JkIixTVFJJTkc6InN0cmluZyIsUkVTRVJWRUQ6InJlc2VydmVkIixSRVNFUlZFRF9UT1BMRVZFTDoicmVzZXJ2ZWQtdG9wbGV2ZWwiLFJFU0VSVkVEX05FV0xJTkU6InJlc2VydmVkLW5ld2xpbmUiLE9QRVJBVE9SOiJvcGVyYXRvciIsT1BFTl9QQVJFTjoib3Blbi1wYXJlbiIsQ0xPU0VfUEFSRU46ImNsb3NlLXBhcmVuIixMSU5FX0NPTU1FTlQ6ImxpbmUtY29tbWVudCIsQkxPQ0tfQ09NTUVOVDoiYmxvY2stY29tbWVudCIsTlVNQkVSOiJudW1iZXIiLFBMQUNFSE9MREVSOiJwbGFjZWhvbGRlciJ9LGUuZXhwb3J0cz1FWyJkZWZhdWx0Il19LGZ1bmN0aW9uKGUsRSx0KXtmdW5jdGlvbiBuKGUpe3JldHVybiBudWxsIT1lJiZUKGUubGVuZ3RoKSYmIXIoZSl9dmFyIHI9dCgxMiksVD10KDU5KTtlLmV4cG9ydHM9bn0sZnVuY3Rpb24oZSxFLHQpe2Z1bmN0aW9uIG4oZSl7cmV0dXJuIG51bGw9PWU/IiI6cihlKX12YXIgcj10KDEwKTtlLmV4cG9ydHM9bn0sZnVuY3Rpb24oZSxFLHQpe2Z1bmN0aW9uIG4oZSl7aWYoInN0cmluZyI9PXR5cGVvZiBlKXJldHVybiBlO2lmKFQoZSkpcmV0dXJuIE4/Ti5jYWxsKGUpOiIiO3ZhciBFPWUrIiI7cmV0dXJuIjAiPT1FJiYxL2U9PS1SPyItMCI6RX12YXIgcj10KDI2KSxUPXQoMTQpLFI9MS8wLG89cj9yLnByb3RvdHlwZTp2b2lkIDAsTj1vP28udG9TdHJpbmc6dm9pZCAwO2UuZXhwb3J0cz1ufSxmdW5jdGlvbihlLEUpe2Z1bmN0aW9uIHQoZSl7aWYobnVsbCE9ZSl7dHJ5e3JldHVybiByLmNhbGwoZSl9Y2F0Y2goRSl7fXRyeXtyZXR1cm4gZSsiIn1jYXRjaChFKXt9fXJldHVybiIifXZhciBuPUZ1bmN0aW9uLnByb3RvdHlwZSxyPW4udG9TdHJpbmc7ZS5leHBvcnRzPXR9LGZ1bmN0aW9uKGUsRSx0KXtmdW5jdGlvbiBuKGUpe3ZhciBFPXIoZSk/Ti5jYWxsKGUpOiIiO3JldHVybiBFPT1UfHxFPT1SfXZhciByPXQoNiksVD0iW29iamVjdCBGdW5jdGlvbl0iLFI9IltvYmplY3QgR2VuZXJhdG9yRnVuY3Rpb25dIixvPU9iamVjdC5wcm90b3R5cGUsTj1vLnRvU3RyaW5nO2UuZXhwb3J0cz1ufSxmdW5jdGlvbihlLEUpe2Z1bmN0aW9uIHQoZSl7cmV0dXJuIG51bGwhPWUmJiJvYmplY3QiPT10eXBlb2YgZX1lLmV4cG9ydHM9dH0sZnVuY3Rpb24oZSxFLHQpe2Z1bmN0aW9uIG4oZSl7cmV0dXJuInN5bWJvbCI9PXR5cGVvZiBlfHxyKGUpJiZvLmNhbGwoZSk9PVR9dmFyIHI9dCgxMyksVD0iW29iamVjdCBTeW1ib2xdIixSPU9iamVjdC5wcm90b3R5cGUsbz1SLnRvU3RyaW5nO2UuZXhwb3J0cz1ufSxmdW5jdGlvbihlLEUsdCl7InVzZSBzdHJpY3QiO2Z1bmN0aW9uIG4oZSl7cmV0dXJuIGUmJmUuX19lc01vZHVsZT9lOnsiZGVmYXVsdCI6ZX19RS5fX2VzTW9kdWxlPSEwO3ZhciByPXQoMSksVD1uKHIpLFI9dCg2MSksbz1uKFIpLE49dCg2MCksQT1uKE4pLEk9InRvcC1sZXZlbCIsTz0iYmxvY2stbGV2ZWwiLGk9ZnVuY3Rpb24oKXtmdW5jdGlvbiBlKEUpeygwLFRbImRlZmF1bHQiXSkodGhpcyxlKSx0aGlzLmluZGVudD1FfHwiICAiLHRoaXMuaW5kZW50VHlwZXM9W119cmV0dXJuIGUucHJvdG90eXBlLmdldEluZGVudD1mdW5jdGlvbigpe3JldHVybigwLG9bImRlZmF1bHQiXSkodGhpcy5pbmRlbnQsdGhpcy5pbmRlbnRUeXBlcy5sZW5ndGgpfSxlLnByb3RvdHlwZS5pbmNyZWFzZVRvcGxldmVsPWZ1bmN0aW9uKCl7dGhpcy5pbmRlbnRUeXBlcy5wdXNoKEkpfSxlLnByb3RvdHlwZS5pbmNyZWFzZUJsb2NrTGV2ZWw9ZnVuY3Rpb24oKXt0aGlzLmluZGVudFR5cGVzLnB1c2goTyl9LGUucHJvdG90eXBlLmRlY3JlYXNlVG9wTGV2ZWw9ZnVuY3Rpb24oKXsoMCxBWyJkZWZhdWx0Il0pKHRoaXMuaW5kZW50VHlwZXMpPT09SSYmdGhpcy5pbmRlbnRUeXBlcy5wb3AoKX0sZS5wcm90b3R5cGUuZGVjcmVhc2VCbG9ja0xldmVsPWZ1bmN0aW9uKCl7Zm9yKDt0aGlzLmluZGVudFR5cGVzLmxlbmd0aD4wOyl7dmFyIGU9dGhpcy5pbmRlbnRUeXBlcy5wb3AoKTtpZihlIT09SSlicmVha319LGV9KCk7RVsiZGVmYXVsdCJdPWksZS5leHBvcnRzPUVbImRlZmF1bHQiXX0sZnVuY3Rpb24oZSxFLHQpeyJ1c2Ugc3RyaWN0IjtmdW5jdGlvbiBuKGUpe3JldHVybiBlJiZlLl9fZXNNb2R1bGU/ZTp7ImRlZmF1bHQiOmV9fUUuX19lc01vZHVsZT0hMDt2YXIgcj10KDEpLFQ9bihyKSxSPXQoNyksbz1uKFIpLE49NTAsQT1mdW5jdGlvbigpe2Z1bmN0aW9uIGUoKXsoMCxUWyJkZWZhdWx0Il0pKHRoaXMsZSksdGhpcy5sZXZlbD0wfXJldHVybiBlLnByb3RvdHlwZS5iZWdpbklmUG9zc2libGU9ZnVuY3Rpb24oZSxFKXswPT09dGhpcy5sZXZlbCYmdGhpcy5pc0lubGluZUJsb2NrKGUsRSk/dGhpcy5sZXZlbD0xOnRoaXMubGV2ZWw+MD90aGlzLmxldmVsKys6dGhpcy5sZXZlbD0wfSxlLnByb3RvdHlwZS5lbmQ9ZnVuY3Rpb24oKXt0aGlzLmxldmVsLS19LGUucHJvdG90eXBlLmlzQWN0aXZlPWZ1bmN0aW9uKCl7cmV0dXJuIHRoaXMubGV2ZWw+MH0sZS5wcm90b3R5cGUuaXNJbmxpbmVCbG9jaz1mdW5jdGlvbihlLEUpe2Zvcih2YXIgdD0wLG49MCxyPUU7ZS5sZW5ndGg+cjtyKyspe3ZhciBUPWVbcl07aWYodCs9VC52YWx1ZS5sZW5ndGgsdD5OKXJldHVybiExO2lmKFQudHlwZT09PW9bImRlZmF1bHQiXS5PUEVOX1BBUkVOKW4rKztlbHNlIGlmKFQudHlwZT09PW9bImRlZmF1bHQiXS5DTE9TRV9QQVJFTiYmKG4tLSwwPT09bikpcmV0dXJuITA7aWYodGhpcy5pc0ZvcmJpZGRlblRva2VuKFQpKXJldHVybiExfXJldHVybiExfSxlLnByb3RvdHlwZS5pc0ZvcmJpZGRlblRva2VuPWZ1bmN0aW9uKGUpe3ZhciBFPWUudHlwZSx0PWUudmFsdWU7cmV0dXJuIEU9PT1vWyJkZWZhdWx0Il0uUkVTRVJWRURfVE9QTEVWRUx8fEU9PT1vWyJkZWZhdWx0Il0uUkVTRVJWRURfTkVXTElORXx8RT09PW9bImRlZmF1bHQiXS5DT01NRU5UfHxFPT09b1siZGVmYXVsdCJdLkJMT0NLX0NPTU1FTlR8fCI7Ij09PXR9LGV9KCk7RVsiZGVmYXVsdCJdPUEsZS5leHBvcnRzPUVbImRlZmF1bHQiXX0sZnVuY3Rpb24oZSxFLHQpeyJ1c2Ugc3RyaWN0IjtmdW5jdGlvbiBuKGUpe3JldHVybiBlJiZlLl9fZXNNb2R1bGU/ZTp7ImRlZmF1bHQiOmV9fUUuX19lc01vZHVsZT0hMDt2YXIgcj10KDEpLFQ9bihyKSxSPWZ1bmN0aW9uKCl7ZnVuY3Rpb24gZShFKXsoMCxUWyJkZWZhdWx0Il0pKHRoaXMsZSksdGhpcy5wYXJhbXM9RSx0aGlzLmluZGV4PTB9cmV0dXJuIGUucHJvdG90eXBlLmdldD1mdW5jdGlvbihlKXt2YXIgRT1lLmtleSx0PWUudmFsdWU7cmV0dXJuIHRoaXMucGFyYW1zP0U/dGhpcy5wYXJhbXNbRV06dGhpcy5wYXJhbXNbdGhpcy5pbmRleCsrXTp0fSxlfSgpO0VbImRlZmF1bHQiXT1SLGUuZXhwb3J0cz1FWyJkZWZhdWx0Il19LGZ1bmN0aW9uKGUsRSx0KXsidXNlIHN0cmljdCI7ZnVuY3Rpb24gbihlKXtyZXR1cm4gZSYmZS5fX2VzTW9kdWxlP2U6eyJkZWZhdWx0IjplfX1FLl9fZXNNb2R1bGU9ITA7dmFyIHI9dCgxKSxUPW4ociksUj10KDQpLG89bihSKSxOPXQoNSksQT1uKE4pLEk9WyJBQlMiLCJBQ1RJVkFURSIsIkFMSUFTIiwiQUxMIiwiQUxMT0NBVEUiLCJBTExPVyIsIkFMVEVSIiwiQU5ZIiwiQVJFIiwiQVJSQVkiLCJBUyIsIkFTQyIsIkFTRU5TSVRJVkUiLCJBU1NPQ0lBVEUiLCJBU1VUSU1FIiwiQVNZTU1FVFJJQyIsIkFUIiwiQVRPTUlDIiwiQVRUUklCVVRFUyIsIkFVRElUIiwiQVVUSE9SSVpBVElPTiIsIkFVWCIsIkFVWElMSUFSWSIsIkFWRyIsIkJFRk9SRSIsIkJFR0lOIiwiQkVUV0VFTiIsIkJJR0lOVCIsIkJJTkFSWSIsIkJMT0IiLCJCT09MRUFOIiwiQk9USCIsIkJVRkZFUlBPT0wiLCJCWSIsIkNBQ0hFIiwiQ0FMTCIsIkNBTExFRCIsIkNBUFRVUkUiLCJDQVJESU5BTElUWSIsIkNBU0NBREVEIiwiQ0FTRSIsIkNBU1QiLCJDQ1NJRCIsIkNFSUwiLCJDRUlMSU5HIiwiQ0hBUiIsIkNIQVJBQ1RFUiIsIkNIQVJBQ1RFUl9MRU5HVEgiLCJDSEFSX0xFTkdUSCIsIkNIRUNLIiwiQ0xPQiIsIkNMT05FIiwiQ0xPU0UiLCJDTFVTVEVSIiwiQ09BTEVTQ0UiLCJDT0xMQVRFIiwiQ09MTEVDVCIsIkNPTExFQ1RJT04iLCJDT0xMSUQiLCJDT0xVTU4iLCJDT01NRU5UIiwiQ09NTUlUIiwiQ09OQ0FUIiwiQ09ORElUSU9OIiwiQ09OTkVDVCIsIkNPTk5FQ1RJT04iLCJDT05TVFJBSU5UIiwiQ09OVEFJTlMiLCJDT05USU5VRSIsIkNPTlZFUlQiLCJDT1JSIiwiQ09SUkVTUE9ORElORyIsIkNPVU5UIiwiQ09VTlRfQklHIiwiQ09WQVJfUE9QIiwiQ09WQVJfU0FNUCIsIkNSRUFURSIsIkNST1NTIiwiQ1VCRSIsIkNVTUVfRElTVCIsIkNVUlJFTlQiLCJDVVJSRU5UX0RBVEUiLCJDVVJSRU5UX0RFRkFVTFRfVFJBTlNGT1JNX0dST1VQIiwiQ1VSUkVOVF9MQ19DVFlQRSIsIkNVUlJFTlRfUEFUSCIsIkNVUlJFTlRfUk9MRSIsIkNVUlJFTlRfU0NIRU1BIiwiQ1VSUkVOVF9TRVJWRVIiLCJDVVJSRU5UX1RJTUUiLCJDVVJSRU5UX1RJTUVTVEFNUCIsIkNVUlJFTlRfVElNRVpPTkUiLCJDVVJSRU5UX1RSQU5TRk9STV9HUk9VUF9GT1JfVFlQRSIsIkNVUlJFTlRfVVNFUiIsIkNVUlNPUiIsIkNZQ0xFIiwiREFUQSIsIkRBVEFCQVNFIiwiREFUQVBBUlRJVElPTk5BTUUiLCJEQVRBUEFSVElUSU9OTlVNIiwiREFURSIsIkRBWSIsIkRBWVMiLCJEQjJHRU5FUkFMIiwiREIyR0VOUkwiLCJEQjJTUUwiLCJEQklORk8iLCJEQlBBUlRJVElPTk5BTUUiLCJEQlBBUlRJVElPTk5VTSIsIkRFQUxMT0NBVEUiLCJERUMiLCJERUNJTUFMIiwiREVDTEFSRSIsIkRFRkFVTFQiLCJERUZBVUxUUyIsIkRFRklOSVRJT04iLCJERUxFVEUiLCJERU5TRVJBTksiLCJERU5TRV9SQU5LIiwiREVSRUYiLCJERVNDUklCRSIsIkRFU0NSSVBUT1IiLCJERVRFUk1JTklTVElDIiwiRElBR05PU1RJQ1MiLCJESVNBQkxFIiwiRElTQUxMT1ciLCJESVNDT05ORUNUIiwiRElTVElOQ1QiLCJETyIsIkRPQ1VNRU5UIiwiRE9VQkxFIiwiRFJPUCIsIkRTU0laRSIsIkRZTkFNSUMiLCJFQUNIIiwiRURJVFBST0MiLCJFTEVNRU5UIiwiRUxTRSIsIkVMU0VJRiIsIkVOQUJMRSIsIkVOQ09ESU5HIiwiRU5DUllQVElPTiIsIkVORCIsIkVORC1FWEVDIiwiRU5ESU5HIiwiRVJBU0UiLCJFU0NBUEUiLCJFVkVSWSIsIkVYQ0VQVElPTiIsIkVYQ0xVRElORyIsIkVYQ0xVU0lWRSIsIkVYRUMiLCJFWEVDVVRFIiwiRVhJU1RTIiwiRVhJVCIsIkVYUCIsIkVYUExBSU4iLCJFWFRFTkRFRCIsIkVYVEVSTkFMIiwiRVhUUkFDVCIsIkZBTFNFIiwiRkVOQ0VEIiwiRkVUQ0giLCJGSUVMRFBST0MiLCJGSUxFIiwiRklMVEVSIiwiRklOQUwiLCJGSVJTVCIsIkZMT0FUIiwiRkxPT1IiLCJGT1IiLCJGT1JFSUdOIiwiRlJFRSIsIkZVTEwiLCJGVU5DVElPTiIsIkZVU0lPTiIsIkdFTkVSQUwiLCJHRU5FUkFURUQiLCJHRVQiLCJHTE9CQUwiLCJHT1RPIiwiR1JBTlQiLCJHUkFQSElDIiwiR1JPVVAiLCJHUk9VUElORyIsIkhBTkRMRVIiLCJIQVNIIiwiSEFTSEVEX1ZBTFVFIiwiSElOVCIsIkhPTEQiLCJIT1VSIiwiSE9VUlMiLCJJREVOVElUWSIsIklGIiwiSU1NRURJQVRFIiwiSU4iLCJJTkNMVURJTkciLCJJTkNMVVNJVkUiLCJJTkNSRU1FTlQiLCJJTkRFWCIsIklORElDQVRPUiIsIklORElDQVRPUlMiLCJJTkYiLCJJTkZJTklUWSIsIklOSEVSSVQiLCJJTk5FUiIsIklOT1VUIiwiSU5TRU5TSVRJVkUiLCJJTlNFUlQiLCJJTlQiLCJJTlRFR0VSIiwiSU5URUdSSVRZIiwiSU5URVJTRUNUSU9OIiwiSU5URVJWQUwiLCJJTlRPIiwiSVMiLCJJU09CSUQiLCJJU09MQVRJT04iLCJJVEVSQVRFIiwiSkFSIiwiSkFWQSIsIktFRVAiLCJLRVkiLCJMQUJFTCIsIkxBTkdVQUdFIiwiTEFSR0UiLCJMQVRFUkFMIiwiTENfQ1RZUEUiLCJMRUFESU5HIiwiTEVBVkUiLCJMRUZUIiwiTElLRSIsIkxJTktUWVBFIiwiTE4iLCJMT0NBTCIsIkxPQ0FMREFURSIsIkxPQ0FMRSIsIkxPQ0FMVElNRSIsIkxPQ0FMVElNRVNUQU1QIiwiTE9DQVRPUiIsIkxPQ0FUT1JTIiwiTE9DSyIsIkxPQ0tNQVgiLCJMT0NLU0laRSIsIkxPTkciLCJMT09QIiwiTE9XRVIiLCJNQUlOVEFJTkVEIiwiTUFUQ0giLCJNQVRFUklBTElaRUQiLCJNQVgiLCJNQVhWQUxVRSIsIk1FTUJFUiIsIk1FUkdFIiwiTUVUSE9EIiwiTUlDUk9TRUNPTkQiLCJNSUNST1NFQ09ORFMiLCJNSU4iLCJNSU5VVEUiLCJNSU5VVEVTIiwiTUlOVkFMVUUiLCJNT0QiLCJNT0RFIiwiTU9ESUZJRVMiLCJNT0RVTEUiLCJNT05USCIsIk1PTlRIUyIsIk1VTFRJU0VUIiwiTkFOIiwiTkFUSU9OQUwiLCJOQVRVUkFMIiwiTkNIQVIiLCJOQ0xPQiIsIk5FVyIsIk5FV19UQUJMRSIsIk5FWFRWQUwiLCJOTyIsIk5PQ0FDSEUiLCJOT0NZQ0xFIiwiTk9ERU5BTUUiLCJOT0RFTlVNQkVSIiwiTk9NQVhWQUxVRSIsIk5PTUlOVkFMVUUiLCJOT05FIiwiTk9PUkRFUiIsIk5PUk1BTElaRSIsIk5PUk1BTElaRUQiLCJOT1QiLCJOVUxMIiwiTlVMTElGIiwiTlVMTFMiLCJOVU1FUklDIiwiTlVNUEFSVFMiLCJPQklEIiwiT0NURVRfTEVOR1RIIiwiT0YiLCJPRkZTRVQiLCJPTEQiLCJPTERfVEFCTEUiLCJPTiIsIk9OTFkiLCJPUEVOIiwiT1BUSU1JWkFUSU9OIiwiT1BUSU1JWkUiLCJPUFRJT04iLCJPUkRFUiIsIk9VVCIsIk9VVEVSIiwiT1ZFUiIsIk9WRVJMQVBTIiwiT1ZFUkxBWSIsIk9WRVJSSURJTkciLCJQQUNLQUdFIiwiUEFEREVEIiwiUEFHRVNJWkUiLCJQQVJBTUVURVIiLCJQQVJUIiwiUEFSVElUSU9OIiwiUEFSVElUSU9ORUQiLCJQQVJUSVRJT05JTkciLCJQQVJUSVRJT05TIiwiUEFTU1dPUkQiLCJQQVRIIiwiUEVSQ0VOVElMRV9DT05UIiwiUEVSQ0VOVElMRV9ESVNDIiwiUEVSQ0VOVF9SQU5LIiwiUElFQ0VTSVpFIiwiUExBTiIsIlBPU0lUSU9OIiwiUE9XRVIiLCJQUkVDSVNJT04iLCJQUkVQQVJFIiwiUFJFVlZBTCIsIlBSSU1BUlkiLCJQUklRVFkiLCJQUklWSUxFR0VTIiwiUFJPQ0VEVVJFIiwiUFJPR1JBTSIsIlBTSUQiLCJQVUJMSUMiLCJRVUVSWSIsIlFVRVJZTk8iLCJSQU5HRSIsIlJBTksiLCJSRUFEIiwiUkVBRFMiLCJSRUFMIiwiUkVDT1ZFUlkiLCJSRUNVUlNJVkUiLCJSRUYiLCJSRUZFUkVOQ0VTIiwiUkVGRVJFTkNJTkciLCJSRUZSRVNIIiwiUkVHUl9BVkdYIiwiUkVHUl9BVkdZIiwiUkVHUl9DT1VOVCIsIlJFR1JfSU5URVJDRVBUIiwiUkVHUl9SMiIsIlJFR1JfU0xPUEUiLCJSRUdSX1NYWCIsIlJFR1JfU1hZIiwiUkVHUl9TWVkiLCJSRUxFQVNFIiwiUkVOQU1FIiwiUkVQRUFUIiwiUkVTRVQiLCJSRVNJR05BTCIsIlJFU1RBUlQiLCJSRVNUUklDVCIsIlJFU1VMVCIsIlJFU1VMVF9TRVRfTE9DQVRPUiIsIlJFVFVSTiIsIlJFVFVSTlMiLCJSRVZPS0UiLCJSSUdIVCIsIlJPTEUiLCJST0xMQkFDSyIsIlJPTExVUCIsIlJPVU5EX0NFSUxJTkciLCJST1VORF9ET1dOIiwiUk9VTkRfRkxPT1IiLCJST1VORF9IQUxGX0RPV04iLCJST1VORF9IQUxGX0VWRU4iLCJST1VORF9IQUxGX1VQIiwiUk9VTkRfVVAiLCJST1VUSU5FIiwiUk9XIiwiUk9XTlVNQkVSIiwiUk9XUyIsIlJPV1NFVCIsIlJPV19OVU1CRVIiLCJSUk4iLCJSVU4iLCJTQVZFUE9JTlQiLCJTQ0hFTUEiLCJTQ09QRSIsIlNDUkFUQ0hQQUQiLCJTQ1JPTEwiLCJTRUFSQ0giLCJTRUNPTkQiLCJTRUNPTkRTIiwiU0VDUVRZIiwiU0VDVVJJVFkiLCJTRU5TSVRJVkUiLCJTRVFVRU5DRSIsIlNFU1NJT04iLCJTRVNTSU9OX1VTRVIiLCJTSUdOQUwiLCJTSU1JTEFSIiwiU0lNUExFIiwiU01BTExJTlQiLCJTTkFOIiwiU09NRSIsIlNPVVJDRSIsIlNQRUNJRklDIiwiU1BFQ0lGSUNUWVBFIiwiU1FMIiwiU1FMRVhDRVBUSU9OIiwiU1FMSUQiLCJTUUxTVEFURSIsIlNRTFdBUk5JTkciLCJTUVJUIiwiU1RBQ0tFRCIsIlNUQU5EQVJEIiwiU1RBUlQiLCJTVEFSVElORyIsIlNUQVRFTUVOVCIsIlNUQVRJQyIsIlNUQVRNRU5UIiwiU1RBWSIsIlNURERFVl9QT1AiLCJTVERERVZfU0FNUCIsIlNUT0dST1VQIiwiU1RPUkVTIiwiU1RZTEUiLCJTVUJNVUxUSVNFVCIsIlNVQlNUUklORyIsIlNVTSIsIlNVTU1BUlkiLCJTWU1NRVRSSUMiLCJTWU5PTllNIiwiU1lTRlVOIiwiU1lTSUJNIiwiU1lTUFJPQyIsIlNZU1RFTSIsIlNZU1RFTV9VU0VSIiwiVEFCTEUiLCJUQUJMRVNBTVBMRSIsIlRBQkxFU1BBQ0UiLCJUSEVOIiwiVElNRSIsIlRJTUVTVEFNUCIsIlRJTUVaT05FX0hPVVIiLCJUSU1FWk9ORV9NSU5VVEUiLCJUTyIsIlRSQUlMSU5HIiwiVFJBTlNBQ1RJT04iLCJUUkFOU0xBVEUiLCJUUkFOU0xBVElPTiIsIlRSRUFUIiwiVFJJR0dFUiIsIlRSSU0iLCJUUlVFIiwiVFJVTkNBVEUiLCJUWVBFIiwiVUVTQ0FQRSIsIlVORE8iLCJVTklRVUUiLCJVTktOT1dOIiwiVU5ORVNUIiwiVU5USUwiLCJVUFBFUiIsIlVTQUdFIiwiVVNFUiIsIlVTSU5HIiwiVkFMSURQUk9DIiwiVkFMVUUiLCJWQVJDSEFSIiwiVkFSSUFCTEUiLCJWQVJJQU5UIiwiVkFSWUlORyIsIlZBUl9QT1AiLCJWQVJfU0FNUCIsIlZDQVQiLCJWRVJTSU9OIiwiVklFVyIsIlZPTEFUSUxFIiwiVk9MVU1FUyIsIldIRU4iLCJXSEVORVZFUiIsIldISUxFIiwiV0lEVEhfQlVDS0VUIiwiV0lORE9XIiwiV0lUSCIsIldJVEhJTiIsIldJVEhPVVQiLCJXTE0iLCJXUklURSIsIlhNTEVMRU1FTlQiLCJYTUxFWElTVFMiLCJYTUxOQU1FU1BBQ0VTIiwiWUVBUiIsIllFQVJTIl0sTz1bIkFERCIsIkFGVEVSIiwiQUxURVIgQ09MVU1OIiwiQUxURVIgVEFCTEUiLCJERUxFVEUgRlJPTSIsIkVYQ0VQVCIsIkZFVENIIEZJUlNUIiwiRlJPTSIsIkdST1VQIEJZIiwiR08iLCJIQVZJTkciLCJJTlNFUlQgSU5UTyIsIklOVEVSU0VDVCIsIkxJTUlUIiwiT1JERVIgQlkiLCJTRUxFQ1QiLCJTRVQgQ1VSUkVOVCBTQ0hFTUEiLCJTRVQgU0NIRU1BIiwiU0VUIiwiVU5JT04gQUxMIiwiVVBEQVRFIiwiVkFMVUVTIiwiV0hFUkUiXSxpPVsiQU5EIiwiQ1JPU1MgSk9JTiIsIklOTkVSIEpPSU4iLCJKT0lOIiwiTEVGVCBKT0lOIiwiTEVGVCBPVVRFUiBKT0lOIiwiT1IiLCJPVVRFUiBKT0lOIiwiUklHSFQgSk9JTiIsIlJJR0hUIE9VVEVSIEpPSU4iXSxTPXZvaWQgMCx1PWZ1bmN0aW9uKCl7ZnVuY3Rpb24gZShFKXsoMCxUWyJkZWZhdWx0Il0pKHRoaXMsZSksdGhpcy5jZmc9RX1yZXR1cm4gZS5wcm90b3R5cGUuZm9ybWF0PWZ1bmN0aW9uKGUpe3JldHVybiBTfHwoUz1uZXcgQVsiZGVmYXVsdCJdKHtyZXNlcnZlZFdvcmRzOkkscmVzZXJ2ZWRUb3BsZXZlbFdvcmRzOk8scmVzZXJ2ZWROZXdsaW5lV29yZHM6aSxzdHJpbmdUeXBlczpbJyIiJywiJyciLCJgYCIsIltdIl0sb3BlblBhcmVuczpbIigiXSxjbG9zZVBhcmVuczpbIikiXSxpbmRleGVkUGxhY2Vob2xkZXJUeXBlczpbIj8iXSxuYW1lZFBsYWNlaG9sZGVyVHlwZXM6WyI6Il0sbGluZUNvbW1lbnRUeXBlczpbIi0tIl0sc3BlY2lhbFdvcmRDaGFyczpbIiMiLCJAIl19KSksbmV3IG9bImRlZmF1bHQiXSh0aGlzLmNmZyxTKS5mb3JtYXQoZSl9LGV9KCk7RVsiZGVmYXVsdCJdPXUsZS5leHBvcnRzPUVbImRlZmF1bHQiXX0sZnVuY3Rpb24oZSxFLHQpeyJ1c2Ugc3RyaWN0IjtmdW5jdGlvbiBuKGUpe3JldHVybiBlJiZlLl9fZXNNb2R1bGU/ZTp7ImRlZmF1bHQiOmV9fUUuX19lc01vZHVsZT0hMDt2YXIgcj10KDEpLFQ9bihyKSxSPXQoNCksbz1uKFIpLE49dCg1KSxBPW4oTiksST1bIkFMTCIsIkFMVEVSIiwiQU5BTFlaRSIsIkFORCIsIkFOWSIsIkFSUkFZIiwiQVMiLCJBU0MiLCJCRUdJTiIsIkJFVFdFRU4iLCJCSU5BUlkiLCJCT09MRUFOIiwiQlJFQUsiLCJCVUNLRVQiLCJCVUlMRCIsIkJZIiwiQ0FMTCIsIkNBU0UiLCJDQVNUIiwiQ0xVU1RFUiIsIkNPTExBVEUiLCJDT0xMRUNUSU9OIiwiQ09NTUlUIiwiQ09OTkVDVCIsIkNPTlRJTlVFIiwiQ09SUkVMQVRFIiwiQ09WRVIiLCJDUkVBVEUiLCJEQVRBQkFTRSIsIkRBVEFTRVQiLCJEQVRBU1RPUkUiLCJERUNMQVJFIiwiREVDUkVNRU5UIiwiREVMRVRFIiwiREVSSVZFRCIsIkRFU0MiLCJERVNDUklCRSIsIkRJU1RJTkNUIiwiRE8iLCJEUk9QIiwiRUFDSCIsIkVMRU1FTlQiLCJFTFNFIiwiRU5EIiwiRVZFUlkiLCJFWENFUFQiLCJFWENMVURFIiwiRVhFQ1VURSIsIkVYSVNUUyIsIkVYUExBSU4iLCJGQUxTRSIsIkZFVENIIiwiRklSU1QiLCJGTEFUVEVOIiwiRk9SIiwiRk9SQ0UiLCJGUk9NIiwiRlVOQ1RJT04iLCJHUkFOVCIsIkdST1VQIiwiR1NJIiwiSEFWSU5HIiwiSUYiLCJJR05PUkUiLCJJTElLRSIsIklOIiwiSU5DTFVERSIsIklOQ1JFTUVOVCIsIklOREVYIiwiSU5GRVIiLCJJTkxJTkUiLCJJTk5FUiIsIklOU0VSVCIsIklOVEVSU0VDVCIsIklOVE8iLCJJUyIsIkpPSU4iLCJLRVkiLCJLRVlTIiwiS0VZU1BBQ0UiLCJLTk9XTiIsIkxBU1QiLCJMRUZUIiwiTEVUIiwiTEVUVElORyIsIkxJS0UiLCJMSU1JVCIsIkxTTSIsIk1BUCIsIk1BUFBJTkciLCJNQVRDSEVEIiwiTUFURVJJQUxJWkVEIiwiTUVSR0UiLCJNSU5VUyIsIk1JU1NJTkciLCJOQU1FU1BBQ0UiLCJORVNUIiwiTk9UIiwiTlVMTCIsIk5VTUJFUiIsIk9CSkVDVCIsIk9GRlNFVCIsIk9OIiwiT1BUSU9OIiwiT1IiLCJPUkRFUiIsIk9VVEVSIiwiT1ZFUiIsIlBBUlNFIiwiUEFSVElUSU9OIiwiUEFTU1dPUkQiLCJQQVRIIiwiUE9PTCIsIlBSRVBBUkUiLCJQUklNQVJZIiwiUFJJVkFURSIsIlBSSVZJTEVHRSIsIlBST0NFRFVSRSIsIlBVQkxJQyIsIlJBVyIsIlJFQUxNIiwiUkVEVUNFIiwiUkVOQU1FIiwiUkVUVVJOIiwiUkVUVVJOSU5HIiwiUkVWT0tFIiwiUklHSFQiLCJST0xFIiwiUk9MTEJBQ0siLCJTQVRJU0ZJRVMiLCJTQ0hFTUEiLCJTRUxFQ1QiLCJTRUxGIiwiU0VNSSIsIlNFVCIsIlNIT1ciLCJTT01FIiwiU1RBUlQiLCJTVEFUSVNUSUNTIiwiU1RSSU5HIiwiU1lTVEVNIiwiVEhFTiIsIlRPIiwiVFJBTlNBQ1RJT04iLCJUUklHR0VSIiwiVFJVRSIsIlRSVU5DQVRFIiwiVU5ERVIiLCJVTklPTiIsIlVOSVFVRSIsIlVOS05PV04iLCJVTk5FU1QiLCJVTlNFVCIsIlVQREFURSIsIlVQU0VSVCIsIlVTRSIsIlVTRVIiLCJVU0lORyIsIlZBTElEQVRFIiwiVkFMVUUiLCJWQUxVRUQiLCJWQUxVRVMiLCJWSUEiLCJWSUVXIiwiV0hFTiIsIldIRVJFIiwiV0hJTEUiLCJXSVRIIiwiV0lUSElOIiwiV09SSyIsIlhPUiJdLE89WyJERUxFVEUgRlJPTSIsIkVYQ0VQVCBBTEwiLCJFWENFUFQiLCJFWFBMQUlOIERFTEVURSBGUk9NIiwiRVhQTEFJTiBVUERBVEUiLCJFWFBMQUlOIFVQU0VSVCIsIkZST00iLCJHUk9VUCBCWSIsIkhBVklORyIsIklORkVSIiwiSU5TRVJUIElOVE8iLCJJTlRFUlNFQ1QgQUxMIiwiSU5URVJTRUNUIiwiTEVUIiwiTElNSVQiLCJNRVJHRSIsIk5FU1QiLCJPUkRFUiBCWSIsIlBSRVBBUkUiLCJTRUxFQ1QiLCJTRVQgQ1VSUkVOVCBTQ0hFTUEiLCJTRVQgU0NIRU1BIiwiU0VUIiwiVU5JT04gQUxMIiwiVU5JT04iLCJVTk5FU1QiLCJVUERBVEUiLCJVUFNFUlQiLCJVU0UgS0VZUyIsIlZBTFVFUyIsIldIRVJFIl0saT1bIkFORCIsIklOTkVSIEpPSU4iLCJKT0lOIiwiTEVGVCBKT0lOIiwiTEVGVCBPVVRFUiBKT0lOIiwiT1IiLCJPVVRFUiBKT0lOIiwiUklHSFQgSk9JTiIsIlJJR0hUIE9VVEVSIEpPSU4iLCJYT1IiXSxTPXZvaWQgMCx1PWZ1bmN0aW9uKCl7ZnVuY3Rpb24gZShFKXsoMCxUWyJkZWZhdWx0Il0pKHRoaXMsZSksdGhpcy5jZmc9RX1yZXR1cm4gZS5wcm90b3R5cGUuZm9ybWF0PWZ1bmN0aW9uKGUpe3JldHVybiBTfHwoUz1uZXcgQVsiZGVmYXVsdCJdKHtyZXNlcnZlZFdvcmRzOkkscmVzZXJ2ZWRUb3BsZXZlbFdvcmRzOk8scmVzZXJ2ZWROZXdsaW5lV29yZHM6aSxzdHJpbmdUeXBlczpbJyIiJywiJyciLCJgYCJdLG9wZW5QYXJlbnM6WyIoIiwiWyIsInsiXSxjbG9zZVBhcmVuczpbIikiLCJdIiwifSJdLG5hbWVkUGxhY2Vob2xkZXJUeXBlczpbIiQiXSxsaW5lQ29tbWVudFR5cGVzOlsiIyIsIi0tIl19KSksbmV3IG9bImRlZmF1bHQiXSh0aGlzLmNmZyxTKS5mb3JtYXQoZSl9LGV9KCk7RVsiZGVmYXVsdCJdPXUsZS5leHBvcnRzPUVbImRlZmF1bHQiXX0sZnVuY3Rpb24oZSxFLHQpeyJ1c2Ugc3RyaWN0IjtmdW5jdGlvbiBuKGUpe3JldHVybiBlJiZlLl9fZXNNb2R1bGU/ZTp7ImRlZmF1bHQiOmV9fUUuX19lc01vZHVsZT0hMDt2YXIgcj10KDEpLFQ9bihyKSxSPXQoNCksbz1uKFIpLE49dCg1KSxBPW4oTiksST1bIkEiLCJBQ0NFU1NJQkxFIiwiQUdFTlQiLCJBR0dSRUdBVEUiLCJBTEwiLCJBTFRFUiIsIkFOWSIsIkFSUkFZIiwiQVMiLCJBU0MiLCJBVCIsIkFUVFJJQlVURSIsIkFVVEhJRCIsIkFWRyIsIkJFVFdFRU4iLCJCRklMRV9CQVNFIiwiQklOQVJZX0lOVEVHRVIiLCJCSU5BUlkiLCJCTE9CX0JBU0UiLCJCTE9DSyIsIkJPRFkiLCJCT09MRUFOIiwiQk9USCIsIkJPVU5EIiwiQlVMSyIsIkJZIiwiQllURSIsIkMiLCJDQUxMIiwiQ0FMTElORyIsIkNBU0NBREUiLCJDQVNFIiwiQ0hBUl9CQVNFIiwiQ0hBUiIsIkNIQVJBQ1RFUiIsIkNIQVJTRVQiLCJDSEFSU0VURk9STSIsIkNIQVJTRVRJRCIsIkNIRUNLIiwiQ0xPQl9CQVNFIiwiQ0xPTkUiLCJDTE9TRSIsIkNMVVNURVIiLCJDTFVTVEVSUyIsIkNPQUxFU0NFIiwiQ09MQVVUSCIsIkNPTExFQ1QiLCJDT0xVTU5TIiwiQ09NTUVOVCIsIkNPTU1JVCIsIkNPTU1JVFRFRCIsIkNPTVBJTEVEIiwiQ09NUFJFU1MiLCJDT05ORUNUIiwiQ09OU1RBTlQiLCJDT05TVFJVQ1RPUiIsIkNPTlRFWFQiLCJDT05USU5VRSIsIkNPTlZFUlQiLCJDT1VOVCIsIkNSQVNIIiwiQ1JFQVRFIiwiQ1JFREVOVElBTCIsIkNVUlJFTlQiLCJDVVJSVkFMIiwiQ1VSU09SIiwiQ1VTVE9NREFUVU0iLCJEQU5HTElORyIsIkRBVEEiLCJEQVRFX0JBU0UiLCJEQVRFIiwiREFZIiwiREVDSU1BTCIsIkRFRkFVTFQiLCJERUZJTkUiLCJERUxFVEUiLCJERVNDIiwiREVURVJNSU5JU1RJQyIsIkRJUkVDVE9SWSIsIkRJU1RJTkNUIiwiRE8iLCJET1VCTEUiLCJEUk9QIiwiRFVSQVRJT04iLCJFTEVNRU5UIiwiRUxTSUYiLCJFTVBUWSIsIkVTQ0FQRSIsIkVYQ0VQVElPTlMiLCJFWENMVVNJVkUiLCJFWEVDVVRFIiwiRVhJU1RTIiwiRVhJVCIsIkVYVEVORFMiLCJFWFRFUk5BTCIsIkVYVFJBQ1QiLCJGQUxTRSIsIkZFVENIIiwiRklOQUwiLCJGSVJTVCIsIkZJWEVEIiwiRkxPQVQiLCJGT1IiLCJGT1JBTEwiLCJGT1JDRSIsIkZST00iLCJGVU5DVElPTiIsIkdFTkVSQUwiLCJHT1RPIiwiR1JBTlQiLCJHUk9VUCIsIkhBU0giLCJIRUFQIiwiSElEREVOIiwiSE9VUiIsIklERU5USUZJRUQiLCJJRiIsIklNTUVESUFURSIsIklOIiwiSU5DTFVESU5HIiwiSU5ERVgiLCJJTkRFWEVTIiwiSU5ESUNBVE9SIiwiSU5ESUNFUyIsIklORklOSVRFIiwiSU5TVEFOVElBQkxFIiwiSU5UIiwiSU5URUdFUiIsIklOVEVSRkFDRSIsIklOVEVSVkFMIiwiSU5UTyIsIklOVkFMSURBVEUiLCJJUyIsIklTT0xBVElPTiIsIkpBVkEiLCJMQU5HVUFHRSIsIkxBUkdFIiwiTEVBRElORyIsIkxFTkdUSCIsIkxFVkVMIiwiTElCUkFSWSIsIkxJS0UiLCJMSUtFMiIsIkxJS0U0IiwiTElLRUMiLCJMSU1JVEVEIiwiTE9DQUwiLCJMT0NLIiwiTE9ORyIsIk1BUCIsIk1BWCIsIk1BWExFTiIsIk1FTUJFUiIsIk1FUkdFIiwiTUlOIiwiTUlOVVMiLCJNSU5VVEUiLCJNTFNMQUJFTCIsIk1PRCIsIk1PREUiLCJNT05USCIsIk1VTFRJU0VUIiwiTkFNRSIsIk5BTiIsIk5BVElPTkFMIiwiTkFUSVZFIiwiTkFUVVJBTCIsIk5BVFVSQUxOIiwiTkNIQVIiLCJORVciLCJORVhUVkFMIiwiTk9DT01QUkVTUyIsIk5PQ09QWSIsIk5PVCIsIk5PV0FJVCIsIk5VTEwiLCJOVUxMSUYiLCJOVU1CRVJfQkFTRSIsIk5VTUJFUiIsIk9CSkVDVCIsIk9DSUNPTEwiLCJPQ0lEQVRFIiwiT0NJREFURVRJTUUiLCJPQ0lEVVJBVElPTiIsIk9DSUlOVEVSVkFMIiwiT0NJTE9CTE9DQVRPUiIsIk9DSU5VTUJFUiIsIk9DSVJBVyIsIk9DSVJFRiIsIk9DSVJFRkNVUlNPUiIsIk9DSVJPV0lEIiwiT0NJU1RSSU5HIiwiT0NJVFlQRSIsIk9GIiwiT0xEIiwiT04iLCJPTkxZIiwiT1BBUVVFIiwiT1BFTiIsIk9QRVJBVE9SIiwiT1BUSU9OIiwiT1JBQ0xFIiwiT1JBREFUQSIsIk9SREVSIiwiT1JHQU5JWkFUSU9OIiwiT1JMQU5ZIiwiT1JMVkFSWSIsIk9USEVSUyIsIk9VVCIsIk9WRVJMQVBTIiwiT1ZFUlJJRElORyIsIlBBQ0tBR0UiLCJQQVJBTExFTF9FTkFCTEUiLCJQQVJBTUVURVIiLCJQQVJBTUVURVJTIiwiUEFSRU5UIiwiUEFSVElUSU9OIiwiUEFTQ0FMIiwiUENURlJFRSIsIlBJUEUiLCJQSVBFTElORUQiLCJQTFNfSU5URUdFUiIsIlBMVUdHQUJMRSIsIlBPU0lUSVZFIiwiUE9TSVRJVkVOIiwiUFJBR01BIiwiUFJFQ0lTSU9OIiwiUFJJT1IiLCJQUklWQVRFIiwiUFJPQ0VEVVJFIiwiUFVCTElDIiwiUkFJU0UiLCJSQU5HRSIsIlJBVyIsIlJFQUQiLCJSRUFMIiwiUkVDT1JEIiwiUkVGIiwiUkVGRVJFTkNFIiwiUkVMRUFTRSIsIlJFTElFU19PTiIsIlJFTSIsIlJFTUFJTkRFUiIsIlJFTkFNRSIsIlJFU09VUkNFIiwiUkVTVUxUX0NBQ0hFIiwiUkVTVUxUIiwiUkVUVVJOIiwiUkVUVVJOSU5HIiwiUkVWRVJTRSIsIlJFVk9LRSIsIlJPTExCQUNLIiwiUk9XIiwiUk9XSUQiLCJST1dOVU0iLCJST1dUWVBFIiwiU0FNUExFIiwiU0FWRSIsIlNBVkVQT0lOVCIsIlNCMSIsIlNCMiIsIlNCNCIsIlNFQ09ORCIsIlNFR01FTlQiLCJTRUxGIiwiU0VQQVJBVEUiLCJTRVFVRU5DRSIsIlNFUklBTElaQUJMRSIsIlNIQVJFIiwiU0hPUlQiLCJTSVpFX1QiLCJTSVpFIiwiU01BTExJTlQiLCJTT01FIiwiU1BBQ0UiLCJTUEFSU0UiLCJTUUwiLCJTUUxDT0RFIiwiU1FMREFUQSIsIlNRTEVSUk0iLCJTUUxOQU1FIiwiU1FMU1RBVEUiLCJTVEFOREFSRCIsIlNUQVJUIiwiU1RBVElDIiwiU1REREVWIiwiU1RPUkVEIiwiU1RSSU5HIiwiU1RSVUNUIiwiU1RZTEUiLCJTVUJNVUxUSVNFVCIsIlNVQlBBUlRJVElPTiIsIlNVQlNUSVRVVEFCTEUiLCJTVUJUWVBFIiwiU1VDQ0VTU0ZVTCIsIlNVTSIsIlNZTk9OWU0iLCJTWVNEQVRFIiwiVEFCQVVUSCIsIlRBQkxFIiwiVERPIiwiVEhFIiwiVEhFTiIsIlRJTUUiLCJUSU1FU1RBTVAiLCJUSU1FWk9ORV9BQkJSIiwiVElNRVpPTkVfSE9VUiIsIlRJTUVaT05FX01JTlVURSIsIlRJTUVaT05FX1JFR0lPTiIsIlRPIiwiVFJBSUxJTkciLCJUUkFOU0FDVElPTiIsIlRSQU5TQUNUSU9OQUwiLCJUUklHR0VSIiwiVFJVRSIsIlRSVVNURUQiLCJUWVBFIiwiVUIxIiwiVUIyIiwiVUI0IiwiVUlEIiwiVU5ERVIiLCJVTklRVUUiLCJVTlBMVUciLCJVTlNJR05FRCIsIlVOVFJVU1RFRCIsIlVTRSIsIlVTRVIiLCJVU0lORyIsIlZBTElEQVRFIiwiVkFMSVNUIiwiVkFMVUUiLCJWQVJDSEFSIiwiVkFSQ0hBUjIiLCJWQVJJQUJMRSIsIlZBUklBTkNFIiwiVkFSUkFZIiwiVkFSWUlORyIsIlZJRVciLCJWSUVXUyIsIlZPSUQiLCJXSEVORVZFUiIsIldISUxFIiwiV0lUSCIsIldPUksiLCJXUkFQUEVEIiwiV1JJVEUiLCJZRUFSIiwiWk9ORSJdLE89WyJBREQiLCJBTFRFUiBDT0xVTU4iLCJBTFRFUiBUQUJMRSIsIkJFR0lOIiwiQ09OTkVDVCBCWSIsIkRFQ0xBUkUiLCJERUxFVEUgRlJPTSIsIkRFTEVURSIsIkVORCIsIkVYQ0VQVCIsIkVYQ0VQVElPTiIsIkZFVENIIEZJUlNUIiwiRlJPTSIsIkdST1VQIEJZIiwiSEFWSU5HIiwiSU5TRVJUIElOVE8iLCJJTlNFUlQiLCJJTlRFUlNFQ1QiLCJMSU1JVCIsIkxPT1AiLCJNT0RJRlkiLCJPUkRFUiBCWSIsIlNFTEVDVCIsIlNFVCBDVVJSRU5UIFNDSEVNQSIsIlNFVCBTQ0hFTUEiLCJTRVQiLCJTVEFSVCBXSVRIIiwiVU5JT04gQUxMIiwiVU5JT04iLCJVUERBVEUiLCJWQUxVRVMiLCJXSEVSRSJdLGk9WyJBTkQiLCJDUk9TUyBBUFBMWSIsIkNST1NTIEpPSU4iLCJFTFNFIiwiRU5EIiwiSU5ORVIgSk9JTiIsIkpPSU4iLCJMRUZUIEpPSU4iLCJMRUZUIE9VVEVSIEpPSU4iLCJPUiIsIk9VVEVSIEFQUExZIiwiT1VURVIgSk9JTiIsIlJJR0hUIEpPSU4iLCJSSUdIVCBPVVRFUiBKT0lOIiwiV0hFTiIsIlhPUiJdLFM9dm9pZCAwLHU9ZnVuY3Rpb24oKXtmdW5jdGlvbiBlKEUpeygwLFRbImRlZmF1bHQiXSkodGhpcyxlKSx0aGlzLmNmZz1FfXJldHVybiBlLnByb3RvdHlwZS5mb3JtYXQ9ZnVuY3Rpb24oZSl7cmV0dXJuIFN8fChTPW5ldyBBWyJkZWZhdWx0Il0oe3Jlc2VydmVkV29yZHM6SSxyZXNlcnZlZFRvcGxldmVsV29yZHM6TyxyZXNlcnZlZE5ld2xpbmVXb3JkczppLHN0cmluZ1R5cGVzOlsnIiInLCJOJyciLCInJyIsImBgIl0sb3BlblBhcmVuczpbIigiLCJDQVNFIl0sY2xvc2VQYXJlbnM6WyIpIiwiRU5EIl0saW5kZXhlZFBsYWNlaG9sZGVyVHlwZXM6WyI/Il0sbmFtZWRQbGFjZWhvbGRlclR5cGVzOlsiOiJdLGxpbmVDb21tZW50VHlwZXM6WyItLSJdLHNwZWNpYWxXb3JkQ2hhcnM6WyJfIiwiJCIsIiMiLCIuIiwiQCJdfSkpLG5ldyBvWyJkZWZhdWx0Il0odGhpcy5jZmcsUykuZm9ybWF0KGUpfSxlfSgpO0VbImRlZmF1bHQiXT11LGUuZXhwb3J0cz1FWyJkZWZhdWx0Il19LGZ1bmN0aW9uKGUsRSx0KXsidXNlIHN0cmljdCI7ZnVuY3Rpb24gbihlKXtyZXR1cm4gZSYmZS5fX2VzTW9kdWxlP2U6eyJkZWZhdWx0IjplfX1FLl9fZXNNb2R1bGU9ITA7dmFyIHI9dCgxKSxUPW4ociksUj10KDQpLG89bihSKSxOPXQoNSksQT1uKE4pLEk9WyJBQ0NFU1NJQkxFIiwiQUNUSU9OIiwiQUdBSU5TVCIsIkFHR1JFR0FURSIsIkFMR09SSVRITSIsIkFMTCIsIkFMVEVSIiwiQU5BTFlTRSIsIkFOQUxZWkUiLCJBUyIsIkFTQyIsIkFVVE9DT01NSVQiLCJBVVRPX0lOQ1JFTUVOVCIsIkJBQ0tVUCIsIkJFR0lOIiwiQkVUV0VFTiIsIkJJTkxPRyIsIkJPVEgiLCJDQVNDQURFIiwiQ0FTRSIsIkNIQU5HRSIsIkNIQU5HRUQiLCJDSEFSQUNURVIgU0VUIiwiQ0hBUlNFVCIsIkNIRUNLIiwiQ0hFQ0tTVU0iLCJDT0xMQVRFIiwiQ09MTEFUSU9OIiwiQ09MVU1OIiwiQ09MVU1OUyIsIkNPTU1FTlQiLCJDT01NSVQiLCJDT01NSVRURUQiLCJDT01QUkVTU0VEIiwiQ09OQ1VSUkVOVCIsIkNPTlNUUkFJTlQiLCJDT05UQUlOUyIsIkNPTlZFUlQiLCJDUkVBVEUiLCJDUk9TUyIsIkNVUlJFTlRfVElNRVNUQU1QIiwiREFUQUJBU0UiLCJEQVRBQkFTRVMiLCJEQVkiLCJEQVlfSE9VUiIsIkRBWV9NSU5VVEUiLCJEQVlfU0VDT05EIiwiREVGQVVMVCIsIkRFRklORVIiLCJERUxBWUVEIiwiREVMRVRFIiwiREVTQyIsIkRFU0NSSUJFIiwiREVURVJNSU5JU1RJQyIsIkRJU1RJTkNUIiwiRElTVElOQ1RST1ciLCJESVYiLCJETyIsIkRST1AiLCJEVU1QRklMRSIsIkRVUExJQ0FURSIsIkRZTkFNSUMiLCJFTFNFIiwiRU5DTE9TRUQiLCJFTkQiLCJFTkdJTkUiLCJFTkdJTkVTIiwiRU5HSU5FX1RZUEUiLCJFU0NBUEUiLCJFU0NBUEVEIiwiRVZFTlRTIiwiRVhFQyIsIkVYRUNVVEUiLCJFWElTVFMiLCJFWFBMQUlOIiwiRVhURU5ERUQiLCJGQVNUIiwiRkVUQ0giLCJGSUVMRFMiLCJGSUxFIiwiRklSU1QiLCJGSVhFRCIsIkZMVVNIIiwiRk9SIiwiRk9SQ0UiLCJGT1JFSUdOIiwiRlVMTCIsIkZVTExURVhUIiwiRlVOQ1RJT04iLCJHTE9CQUwiLCJHUkFOVCIsIkdSQU5UUyIsIkdST1VQX0NPTkNBVCIsIkhFQVAiLCJISUdIX1BSSU9SSVRZIiwiSE9TVFMiLCJIT1VSIiwiSE9VUl9NSU5VVEUiLCJIT1VSX1NFQ09ORCIsIklERU5USUZJRUQiLCJJRiIsIklGTlVMTCIsIklHTk9SRSIsIklOIiwiSU5ERVgiLCJJTkRFWEVTIiwiSU5GSUxFIiwiSU5TRVJUIiwiSU5TRVJUX0lEIiwiSU5TRVJUX01FVEhPRCIsIklOVEVSVkFMIiwiSU5UTyIsIklOVk9LRVIiLCJJUyIsIklTT0xBVElPTiIsIktFWSIsIktFWVMiLCJLSUxMIiwiTEFTVF9JTlNFUlRfSUQiLCJMRUFESU5HIiwiTEVWRUwiLCJMSUtFIiwiTElORUFSIiwiTElORVMiLCJMT0FEIiwiTE9DQUwiLCJMT0NLIiwiTE9DS1MiLCJMT0dTIiwiTE9XX1BSSU9SSVRZIiwiTUFSSUEiLCJNQVNURVIiLCJNQVNURVJfQ09OTkVDVF9SRVRSWSIsIk1BU1RFUl9IT1NUIiwiTUFTVEVSX0xPR19GSUxFIiwiTUFUQ0giLCJNQVhfQ09OTkVDVElPTlNfUEVSX0hPVVIiLCJNQVhfUVVFUklFU19QRVJfSE9VUiIsIk1BWF9ST1dTIiwiTUFYX1VQREFURVNfUEVSX0hPVVIiLCJNQVhfVVNFUl9DT05ORUNUSU9OUyIsIk1FRElVTSIsIk1FUkdFIiwiTUlOVVRFIiwiTUlOVVRFX1NFQ09ORCIsIk1JTl9ST1dTIiwiTU9ERSIsIk1PRElGWSIsIk1PTlRIIiwiTVJHX01ZSVNBTSIsIk1ZSVNBTSIsIk5BTUVTIiwiTkFUVVJBTCIsIk5PVCIsIk5PVygpIiwiTlVMTCIsIk9GRlNFVCIsIk9OIERFTEVURSIsIk9OIFVQREFURSIsIk9OIiwiT05MWSIsIk9QRU4iLCJPUFRJTUlaRSIsIk9QVElPTiIsIk9QVElPTkFMTFkiLCJPVVRGSUxFIiwiUEFDS19LRVlTIiwiUEFHRSIsIlBBUlRJQUwiLCJQQVJUSVRJT04iLCJQQVJUSVRJT05TIiwiUEFTU1dPUkQiLCJQUklNQVJZIiwiUFJJVklMRUdFUyIsIlBST0NFRFVSRSIsIlBST0NFU1MiLCJQUk9DRVNTTElTVCIsIlBVUkdFIiwiUVVJQ0siLCJSQUlEMCIsIlJBSURfQ0hVTktTIiwiUkFJRF9DSFVOS1NJWkUiLCJSQUlEX1RZUEUiLCJSQU5HRSIsIlJFQUQiLCJSRUFEX09OTFkiLCJSRUFEX1dSSVRFIiwiUkVGRVJFTkNFUyIsIlJFR0VYUCIsIlJFTE9BRCIsIlJFTkFNRSIsIlJFUEFJUiIsIlJFUEVBVEFCTEUiLCJSRVBMQUNFIiwiUkVQTElDQVRJT04iLCJSRVNFVCIsIlJFU1RPUkUiLCJSRVNUUklDVCIsIlJFVFVSTiIsIlJFVFVSTlMiLCJSRVZPS0UiLCJSTElLRSIsIlJPTExCQUNLIiwiUk9XIiwiUk9XUyIsIlJPV19GT1JNQVQiLCJTRUNPTkQiLCJTRUNVUklUWSIsIlNFUEFSQVRPUiIsIlNFUklBTElaQUJMRSIsIlNFU1NJT04iLCJTSEFSRSIsIlNIT1ciLCJTSFVURE9XTiIsIlNMQVZFIiwiU09OQU1FIiwiU09VTkRTIiwiU1FMIiwiU1FMX0FVVE9fSVNfTlVMTCIsIlNRTF9CSUdfUkVTVUxUIiwiU1FMX0JJR19TRUxFQ1RTIiwiU1FMX0JJR19UQUJMRVMiLCJTUUxfQlVGRkVSX1JFU1VMVCIsIlNRTF9DQUNIRSIsIlNRTF9DQUxDX0ZPVU5EX1JPV1MiLCJTUUxfTE9HX0JJTiIsIlNRTF9MT0dfT0ZGIiwiU1FMX0xPR19VUERBVEUiLCJTUUxfTE9XX1BSSU9SSVRZX1VQREFURVMiLCJTUUxfTUFYX0pPSU5fU0laRSIsIlNRTF9OT19DQUNIRSIsIlNRTF9RVU9URV9TSE9XX0NSRUFURSIsIlNRTF9TQUZFX1VQREFURVMiLCJTUUxfU0VMRUNUX0xJTUlUIiwiU1FMX1NMQVZFX1NLSVBfQ09VTlRFUiIsIlNRTF9TTUFMTF9SRVNVTFQiLCJTUUxfV0FSTklOR1MiLCJTVEFSVCIsIlNUQVJUSU5HIiwiU1RBVFVTIiwiU1RPUCIsIlNUT1JBR0UiLCJTVFJBSUdIVF9KT0lOIiwiU1RSSU5HIiwiU1RSSVBFRCIsIlNVUEVSIiwiVEFCTEUiLCJUQUJMRVMiLCJURU1QT1JBUlkiLCJURVJNSU5BVEVEIiwiVEhFTiIsIlRPIiwiVFJBSUxJTkciLCJUUkFOU0FDVElPTkFMIiwiVFJVRSIsIlRSVU5DQVRFIiwiVFlQRSIsIlRZUEVTIiwiVU5DT01NSVRURUQiLCJVTklRVUUiLCJVTkxPQ0siLCJVTlNJR05FRCIsIlVTQUdFIiwiVVNFIiwiVVNJTkciLCJWQVJJQUJMRVMiLCJWSUVXIiwiV0hFTiIsIldJVEgiLCJXT1JLIiwiV1JJVEUiLCJZRUFSX01PTlRIIl0sTz1bIkFERCIsIkFGVEVSIiwiQUxURVIgQ09MVU1OIiwiQUxURVIgVEFCTEUiLCJERUxFVEUgRlJPTSIsIkVYQ0VQVCIsIkZFVENIIEZJUlNUIiwiRlJPTSIsIkdST1VQIEJZIiwiR08iLCJIQVZJTkciLCJJTlNFUlQgSU5UTyIsIklOU0VSVCIsIklOVEVSU0VDVCIsIkxJTUlUIiwiTU9ESUZZIiwiT1JERVIgQlkiLCJTRUxFQ1QiLCJTRVQgQ1VSUkVOVCBTQ0hFTUEiLCJTRVQgU0NIRU1BIiwiU0VUIiwiVU5JT04gQUxMIiwiVU5JT04iLCJVUERBVEUiLCJWQUxVRVMiLCJXSEVSRSJdLGk9WyJBTkQiLCJDUk9TUyBBUFBMWSIsIkNST1NTIEpPSU4iLCJFTFNFIiwiSU5ORVIgSk9JTiIsIkpPSU4iLCJMRUZUIEpPSU4iLCJMRUZUIE9VVEVSIEpPSU4iLCJPUiIsIk9VVEVSIEFQUExZIiwiT1VURVIgSk9JTiIsIlJJR0hUIEpPSU4iLCJSSUdIVCBPVVRFUiBKT0lOIiwiV0hFTiIsIlhPUiJdLFM9dm9pZCAwLHU9ZnVuY3Rpb24oKXtmdW5jdGlvbiBlKEUpeygwLFRbImRlZmF1bHQiXSkodGhpcyxlKSx0aGlzLmNmZz1FfXJldHVybiBlLnByb3RvdHlwZS5mb3JtYXQ9ZnVuY3Rpb24oZSl7cmV0dXJuIFN8fChTPW5ldyBBWyJkZWZhdWx0Il0oe3Jlc2VydmVkV29yZHM6SSxyZXNlcnZlZFRvcGxldmVsV29yZHM6TyxyZXNlcnZlZE5ld2xpbmVXb3JkczppLHN0cmluZ1R5cGVzOlsnIiInLCJOJyciLCInJyIsImBgIiwiW10iXSxvcGVuUGFyZW5zOlsiKCIsIkNBU0UiXSxjbG9zZVBhcmVuczpbIikiLCJFTkQiXSxpbmRleGVkUGxhY2Vob2xkZXJUeXBlczpbIj8iXSxuYW1lZFBsYWNlaG9sZGVyVHlwZXM6WyJAIiwiOiJdLGxpbmVDb21tZW50VHlwZXM6WyIjIiwiLS0iXX0pKSxuZXcgb1siZGVmYXVsdCJdKHRoaXMuY2ZnLFMpLmZvcm1hdChlKX0sZX0oKTtFWyJkZWZhdWx0Il09dSxlLmV4cG9ydHM9RVsiZGVmYXVsdCJdfSxmdW5jdGlvbihlLEUsdCl7dmFyIG49dCgzKSxyPXQoMiksVD1uKHIsIkRhdGFWaWV3Iik7ZS5leHBvcnRzPVR9LGZ1bmN0aW9uKGUsRSx0KXt2YXIgbj10KDMpLHI9dCgyKSxUPW4ociwiTWFwIik7ZS5leHBvcnRzPVR9LGZ1bmN0aW9uKGUsRSx0KXt2YXIgbj10KDMpLHI9dCgyKSxUPW4ociwiUHJvbWlzZSIpO2UuZXhwb3J0cz1UfSxmdW5jdGlvbihlLEUsdCl7dmFyIG49dCgzKSxyPXQoMiksVD1uKHIsIlNldCIpO2UuZXhwb3J0cz1UfSxmdW5jdGlvbihlLEUsdCl7dmFyIG49dCgyKSxyPW4uU3ltYm9sO2UuZXhwb3J0cz1yfSxmdW5jdGlvbihlLEUsdCl7dmFyIG49dCgzKSxyPXQoMiksVD1uKHIsIldlYWtNYXAiKTtlLmV4cG9ydHM9VH0sZnVuY3Rpb24oZSxFKXtmdW5jdGlvbiB0KGUpe3JldHVybiBlLnNwbGl0KCIiKX1lLmV4cG9ydHM9dH0sZnVuY3Rpb24oZSxFKXtmdW5jdGlvbiB0KGUsRSx0LG4pe2Zvcih2YXIgcj1lLmxlbmd0aCxUPXQrKG4/MTotMSk7bj9ULS06KytUPHI7KWlmKEUoZVtUXSxULGUpKXJldHVybiBUOwpyZXR1cm4tMX1lLmV4cG9ydHM9dH0sZnVuY3Rpb24oZSxFKXtmdW5jdGlvbiB0KGUpe3JldHVybiByLmNhbGwoZSl9dmFyIG49T2JqZWN0LnByb3RvdHlwZSxyPW4udG9TdHJpbmc7ZS5leHBvcnRzPXR9LGZ1bmN0aW9uKGUsRSx0KXtmdW5jdGlvbiBuKGUsRSx0KXtyZXR1cm4gRT09PUU/UihlLEUsdCk6cihlLFQsdCl9dmFyIHI9dCgyOSksVD10KDMyKSxSPXQoNDkpO2UuZXhwb3J0cz1ufSxmdW5jdGlvbihlLEUpe2Z1bmN0aW9uIHQoZSl7cmV0dXJuIGUhPT1lfWUuZXhwb3J0cz10fSxmdW5jdGlvbihlLEUsdCl7ZnVuY3Rpb24gbihlKXtpZighUihlKXx8VChlKSlyZXR1cm4hMTt2YXIgRT1yKGUpP3U6QTtyZXR1cm4gRS50ZXN0KG8oZSkpfXZhciByPXQoMTIpLFQ9dCg0NSksUj10KDYpLG89dCgxMSksTj0vW1xcXiQuKis/KClbXF17fXxdL2csQT0vXlxbb2JqZWN0IC4rP0NvbnN0cnVjdG9yXF0kLyxJPUZ1bmN0aW9uLnByb3RvdHlwZSxPPU9iamVjdC5wcm90b3R5cGUsaT1JLnRvU3RyaW5nLFM9Ty5oYXNPd25Qcm9wZXJ0eSx1PVJlZ0V4cCgiXiIraS5jYWxsKFMpLnJlcGxhY2UoTiwiXFwkJiIpLnJlcGxhY2UoL2hhc093blByb3BlcnR5fChmdW5jdGlvbikuKj8oPz1cXFwoKXwgZm9yIC4rPyg/PVxcXF0pL2csIiQxLio/IikrIiQiKTtlLmV4cG9ydHM9bn0sZnVuY3Rpb24oZSxFKXtmdW5jdGlvbiB0KGUsRSl7dmFyIHQ9IiI7aWYoIWV8fDE+RXx8RT5uKXJldHVybiB0O2RvIEUlMiYmKHQrPWUpLEU9cihFLzIpLEUmJihlKz1lKTt3aGlsZShFKTtyZXR1cm4gdH12YXIgbj05MDA3MTk5MjU0NzQwOTkxLHI9TWF0aC5mbG9vcjtlLmV4cG9ydHM9dH0sZnVuY3Rpb24oZSxFKXtmdW5jdGlvbiB0KGUsRSx0KXt2YXIgbj0tMSxyPWUubGVuZ3RoOzA+RSYmKEU9LUU+cj8wOnIrRSksdD10PnI/cjp0LDA+dCYmKHQrPXIpLHI9RT50PzA6dC1FPj4+MCxFPj4+PTA7Zm9yKHZhciBUPUFycmF5KHIpOysrbjxyOylUW25dPWVbbitFXTtyZXR1cm4gVH1lLmV4cG9ydHM9dH0sZnVuY3Rpb24oZSxFLHQpe2Z1bmN0aW9uIG4oZSxFLHQpe3ZhciBuPWUubGVuZ3RoO3JldHVybiB0PXZvaWQgMD09PXQ/bjp0LEV8fG4+dD9yKGUsRSx0KTplfXZhciByPXQoMzUpO2UuZXhwb3J0cz1ufSxmdW5jdGlvbihlLEUsdCl7ZnVuY3Rpb24gbihlLEUpe2Zvcih2YXIgdD1lLmxlbmd0aDt0LS0mJnIoRSxlW3RdLDApPi0xOyk7cmV0dXJuIHR9dmFyIHI9dCgzMSk7ZS5leHBvcnRzPW59LGZ1bmN0aW9uKGUsRSx0KXt2YXIgbj10KDIpLHI9blsiX19jb3JlLWpzX3NoYXJlZF9fIl07ZS5leHBvcnRzPXJ9LGZ1bmN0aW9uKGUsRSl7KGZ1bmN0aW9uKEUpe3ZhciB0PSJvYmplY3QiPT10eXBlb2YgRSYmRSYmRS5PYmplY3Q9PT1PYmplY3QmJkU7ZS5leHBvcnRzPXR9KS5jYWxsKEUsZnVuY3Rpb24oKXtyZXR1cm4gdGhpc30oKSl9LGZ1bmN0aW9uKGUsRSx0KXt2YXIgbj10KDIyKSxyPXQoMjMpLFQ9dCgyNCksUj10KDI1KSxvPXQoMjcpLE49dCgzMCksQT10KDExKSxJPSJbb2JqZWN0IE1hcF0iLE89IltvYmplY3QgT2JqZWN0XSIsaT0iW29iamVjdCBQcm9taXNlXSIsUz0iW29iamVjdCBTZXRdIix1PSJbb2JqZWN0IFdlYWtNYXBdIixMPSJbb2JqZWN0IERhdGFWaWV3XSIsQz1PYmplY3QucHJvdG90eXBlLHM9Qy50b1N0cmluZyxhPUEobiksZj1BKHIpLGM9QShUKSxwPUEoUiksbD1BKG8pLEQ9TjsobiYmRChuZXcgbihuZXcgQXJyYXlCdWZmZXIoMSkpKSE9THx8ciYmRChuZXcgcikhPUl8fFQmJkQoVC5yZXNvbHZlKCkpIT1pfHxSJiZEKG5ldyBSKSE9U3x8byYmRChuZXcgbykhPXUpJiYoRD1mdW5jdGlvbihlKXt2YXIgRT1zLmNhbGwoZSksdD1FPT1PP2UuY29uc3RydWN0b3I6dm9pZCAwLG49dD9BKHQpOnZvaWQgMDtpZihuKXN3aXRjaChuKXtjYXNlIGE6cmV0dXJuIEw7Y2FzZSBmOnJldHVybiBJO2Nhc2UgYzpyZXR1cm4gaTtjYXNlIHA6cmV0dXJuIFM7Y2FzZSBsOnJldHVybiB1fXJldHVybiBFfSksZS5leHBvcnRzPUR9LGZ1bmN0aW9uKGUsRSl7ZnVuY3Rpb24gdChlLEUpe3JldHVybiBudWxsPT1lP3ZvaWQgMDplW0VdfWUuZXhwb3J0cz10fSxmdW5jdGlvbihlLEUpe2Z1bmN0aW9uIHQoZSl7cmV0dXJuIE4udGVzdChlKX12YXIgbj0iXFx1ZDgwMC1cXHVkZmZmIixyPSJcXHUwMzAwLVxcdTAzNmZcXHVmZTIwLVxcdWZlMjMiLFQ9IlxcdTIwZDAtXFx1MjBmMCIsUj0iXFx1ZmUwZVxcdWZlMGYiLG89IlxcdTIwMGQiLE49UmVnRXhwKCJbIitvK24rcitUK1IrIl0iKTtlLmV4cG9ydHM9dH0sZnVuY3Rpb24oZSxFKXtmdW5jdGlvbiB0KGUsRSl7cmV0dXJuIEU9bnVsbD09RT9uOkUsISFFJiYoIm51bWJlciI9PXR5cGVvZiBlfHxyLnRlc3QoZSkpJiZlPi0xJiZlJTE9PTAmJkU+ZX12YXIgbj05MDA3MTk5MjU0NzQwOTkxLHI9L14oPzowfFsxLTldXGQqKSQvO2UuZXhwb3J0cz10fSxmdW5jdGlvbihlLEUsdCl7ZnVuY3Rpb24gbihlLEUsdCl7aWYoIW8odCkpcmV0dXJuITE7dmFyIG49dHlwZW9mIEU7cmV0dXJuISEoIm51bWJlciI9PW4/VCh0KSYmUihFLHQubGVuZ3RoKToic3RyaW5nIj09biYmRSBpbiB0KSYmcih0W0VdLGUpfXZhciByPXQoNTIpLFQ9dCg4KSxSPXQoNDMpLG89dCg2KTtlLmV4cG9ydHM9bn0sZnVuY3Rpb24oZSxFLHQpe2Z1bmN0aW9uIG4oZSl7cmV0dXJuISFUJiZUIGluIGV9dmFyIHI9dCgzOCksVD1mdW5jdGlvbigpe3ZhciBlPS9bXi5dKyQvLmV4ZWMociYmci5rZXlzJiZyLmtleXMuSUVfUFJPVE98fCIiKTtyZXR1cm4gZT8iU3ltYm9sKHNyYylfMS4iK2U6IiJ9KCk7ZS5leHBvcnRzPW59LGZ1bmN0aW9uKGUsRSl7ZnVuY3Rpb24gdChlKXt2YXIgRT1lJiZlLmNvbnN0cnVjdG9yLHQ9ImZ1bmN0aW9uIj09dHlwZW9mIEUmJkUucHJvdG90eXBlfHxuO3JldHVybiBlPT09dH12YXIgbj1PYmplY3QucHJvdG90eXBlO2UuZXhwb3J0cz10fSxmdW5jdGlvbihlLEUsdCl7dmFyIG49dCg0OCkscj1uKE9iamVjdC5rZXlzLE9iamVjdCk7ZS5leHBvcnRzPXJ9LGZ1bmN0aW9uKGUsRSl7ZnVuY3Rpb24gdChlLEUpe3JldHVybiBmdW5jdGlvbih0KXtyZXR1cm4gZShFKHQpKX19ZS5leHBvcnRzPXR9LGZ1bmN0aW9uKGUsRSl7ZnVuY3Rpb24gdChlLEUsdCl7Zm9yKHZhciBuPXQtMSxyPWUubGVuZ3RoOysrbjxyOylpZihlW25dPT09RSlyZXR1cm4gbjtyZXR1cm4tMX1lLmV4cG9ydHM9dH0sZnVuY3Rpb24oZSxFLHQpe2Z1bmN0aW9uIG4oZSl7cmV0dXJuIFQoZSk/UihlKTpyKGUpfXZhciByPXQoMjgpLFQ9dCg0MiksUj10KDUxKTtlLmV4cG9ydHM9bn0sZnVuY3Rpb24oZSxFKXtmdW5jdGlvbiB0KGUpe3JldHVybiBlLm1hdGNoKGMpfHxbXX12YXIgbj0iXFx1ZDgwMC1cXHVkZmZmIixyPSJcXHUwMzAwLVxcdTAzNmZcXHVmZTIwLVxcdWZlMjMiLFQ9IlxcdTIwZDAtXFx1MjBmMCIsUj0iXFx1ZmUwZVxcdWZlMGYiLG89IlsiK24rIl0iLE49IlsiK3IrVCsiXSIsQT0iXFx1ZDgzY1tcXHVkZmZiLVxcdWRmZmZdIixJPSIoPzoiK04rInwiK0ErIikiLE89IlteIituKyJdIixpPSIoPzpcXHVkODNjW1xcdWRkZTYtXFx1ZGRmZl0pezJ9IixTPSJbXFx1ZDgwMC1cXHVkYmZmXVtcXHVkYzAwLVxcdWRmZmZdIix1PSJcXHUyMDBkIixMPUkrIj8iLEM9IlsiK1IrIl0/IixzPSIoPzoiK3UrIig/OiIrW08saSxTXS5qb2luKCJ8IikrIikiK0MrTCsiKSoiLGE9QytMK3MsZj0iKD86IitbTytOKyI/IixOLGksUyxvXS5qb2luKCJ8IikrIikiLGM9UmVnRXhwKEErIig/PSIrQSsiKXwiK2YrYSwiZyIpO2UuZXhwb3J0cz10fSxmdW5jdGlvbihlLEUpe2Z1bmN0aW9uIHQoZSxFKXtyZXR1cm4gZT09PUV8fGUhPT1lJiZFIT09RX1lLmV4cG9ydHM9dH0sZnVuY3Rpb24oZSxFLHQpe2Z1bmN0aW9uIG4oZSl7cmV0dXJuIGU9cihlKSxlJiZSLnRlc3QoZSk/ZS5yZXBsYWNlKFQsIlxcJCYiKTplfXZhciByPXQoOSksVD0vW1xcXiQuKis/KClbXF17fXxdL2csUj1SZWdFeHAoVC5zb3VyY2UpO2UuZXhwb3J0cz1ufSxmdW5jdGlvbihlLEUsdCl7ZnVuY3Rpb24gbihlKXtyZXR1cm4gcihlKSYmby5jYWxsKGUsImNhbGxlZSIpJiYoIUEuY2FsbChlLCJjYWxsZWUiKXx8Ti5jYWxsKGUpPT1UKX12YXIgcj10KDU2KSxUPSJbb2JqZWN0IEFyZ3VtZW50c10iLFI9T2JqZWN0LnByb3RvdHlwZSxvPVIuaGFzT3duUHJvcGVydHksTj1SLnRvU3RyaW5nLEE9Ui5wcm9wZXJ0eUlzRW51bWVyYWJsZTtlLmV4cG9ydHM9bn0sZnVuY3Rpb24oZSxFKXt2YXIgdD1BcnJheS5pc0FycmF5O2UuZXhwb3J0cz10fSxmdW5jdGlvbihlLEUsdCl7ZnVuY3Rpb24gbihlKXtyZXR1cm4gVChlKSYmcihlKX12YXIgcj10KDgpLFQ9dCgxMyk7ZS5leHBvcnRzPW59LGZ1bmN0aW9uKGUsRSx0KXsoZnVuY3Rpb24oZSl7dmFyIG49dCgyKSxyPXQoNjIpLFQ9Im9iamVjdCI9PXR5cGVvZiBFJiZFJiYhRS5ub2RlVHlwZSYmRSxSPVQmJiJvYmplY3QiPT10eXBlb2YgZSYmZSYmIWUubm9kZVR5cGUmJmUsbz1SJiZSLmV4cG9ydHM9PT1ULE49bz9uLkJ1ZmZlcjp2b2lkIDAsQT1OP04uaXNCdWZmZXI6dm9pZCAwLEk9QXx8cjtlLmV4cG9ydHM9SX0pLmNhbGwoRSx0KDY3KShlKSl9LGZ1bmN0aW9uKGUsRSx0KXtmdW5jdGlvbiBuKGUpe2lmKG8oZSkmJihSKGUpfHwic3RyaW5nIj09dHlwZW9mIGV8fCJmdW5jdGlvbiI9PXR5cGVvZiBlLnNwbGljZXx8TihlKXx8VChlKSkpcmV0dXJuIWUubGVuZ3RoO3ZhciBFPXIoZSk7aWYoRT09T3x8RT09aSlyZXR1cm4hZS5zaXplO2lmKEEoZSkpcmV0dXJuIUkoZSkubGVuZ3RoO2Zvcih2YXIgdCBpbiBlKWlmKHUuY2FsbChlLHQpKXJldHVybiExO3JldHVybiEwfXZhciByPXQoNDApLFQ9dCg1NCksUj10KDU1KSxvPXQoOCksTj10KDU3KSxBPXQoNDYpLEk9dCg0NyksTz0iW29iamVjdCBNYXBdIixpPSJbb2JqZWN0IFNldF0iLFM9T2JqZWN0LnByb3RvdHlwZSx1PVMuaGFzT3duUHJvcGVydHk7ZS5leHBvcnRzPW59LGZ1bmN0aW9uKGUsRSl7ZnVuY3Rpb24gdChlKXtyZXR1cm4ibnVtYmVyIj09dHlwZW9mIGUmJmU+LTEmJmUlMT09MCYmbj49ZX12YXIgbj05MDA3MTk5MjU0NzQwOTkxO2UuZXhwb3J0cz10fSxmdW5jdGlvbihlLEUpe2Z1bmN0aW9uIHQoZSl7dmFyIEU9ZT9lLmxlbmd0aDowO3JldHVybiBFP2VbRS0xXTp2b2lkIDB9ZS5leHBvcnRzPXR9LGZ1bmN0aW9uKGUsRSx0KXtmdW5jdGlvbiBuKGUsRSx0KXtyZXR1cm4gRT0odD9UKGUsRSx0KTp2b2lkIDA9PT1FKT8xOlIoRSkscihvKGUpLEUpfXZhciByPXQoMzQpLFQ9dCg0NCksUj10KDY0KSxvPXQoOSk7ZS5leHBvcnRzPW59LGZ1bmN0aW9uKGUsRSl7ZnVuY3Rpb24gdCgpe3JldHVybiExfWUuZXhwb3J0cz10fSxmdW5jdGlvbihlLEUsdCl7ZnVuY3Rpb24gbihlKXtpZighZSlyZXR1cm4gMD09PWU/ZTowO2lmKGU9cihlKSxlPT09VHx8ZT09PS1UKXt2YXIgRT0wPmU/LTE6MTtyZXR1cm4gRSpSfXJldHVybiBlPT09ZT9lOjB9dmFyIHI9dCg2NSksVD0xLzAsUj0xLjc5NzY5MzEzNDg2MjMxNTdlMzA4O2UuZXhwb3J0cz1ufSxmdW5jdGlvbihlLEUsdCl7ZnVuY3Rpb24gbihlKXt2YXIgRT1yKGUpLHQ9RSUxO3JldHVybiBFPT09RT90P0UtdDpFOjB9dmFyIHI9dCg2Myk7ZS5leHBvcnRzPW59LGZ1bmN0aW9uKGUsRSx0KXtmdW5jdGlvbiBuKGUpe2lmKCJudW1iZXIiPT10eXBlb2YgZSlyZXR1cm4gZTtpZihUKGUpKXJldHVybiBSO2lmKHIoZSkpe3ZhciBFPSJmdW5jdGlvbiI9PXR5cGVvZiBlLnZhbHVlT2Y/ZS52YWx1ZU9mKCk6ZTtlPXIoRSk/RSsiIjpFfWlmKCJzdHJpbmciIT10eXBlb2YgZSlyZXR1cm4gMD09PWU/ZTorZTtlPWUucmVwbGFjZShvLCIiKTt2YXIgdD1BLnRlc3QoZSk7cmV0dXJuIHR8fEkudGVzdChlKT9PKGUuc2xpY2UoMiksdD8yOjgpOk4udGVzdChlKT9SOitlfXZhciByPXQoNiksVD10KDE0KSxSPU5hTixvPS9eXHMrfFxzKyQvZyxOPS9eWy0rXTB4WzAtOWEtZl0rJC9pLEE9L14wYlswMV0rJC9pLEk9L14wb1swLTddKyQvaSxPPXBhcnNlSW50O2UuZXhwb3J0cz1ufSxmdW5jdGlvbihlLEUsdCl7ZnVuY3Rpb24gbihlLEUsdCl7aWYoZT1OKGUpLGUmJih0fHx2b2lkIDA9PT1FKSlyZXR1cm4gZS5yZXBsYWNlKEEsIiIpO2lmKCFlfHwhKEU9cihFKSkpcmV0dXJuIGU7dmFyIG49byhlKSxJPVIobixvKEUpKSsxO3JldHVybiBUKG4sMCxJKS5qb2luKCIiKX12YXIgcj10KDEwKSxUPXQoMzYpLFI9dCgzNyksbz10KDUwKSxOPXQoOSksQT0vXHMrJC87ZS5leHBvcnRzPW59LGZ1bmN0aW9uKGUsRSl7ZS5leHBvcnRzPWZ1bmN0aW9uKGUpe3JldHVybiBlLndlYnBhY2tQb2x5ZmlsbHx8KGUuZGVwcmVjYXRlPWZ1bmN0aW9uKCl7fSxlLnBhdGhzPVtdLGUuY2hpbGRyZW49W10sZS53ZWJwYWNrUG9seWZpbGw9MSksZX19XSl9KTsKCgkJZnVuY3Rpb24gZXNjYXBlMkh0bWwoc3RyKSB7CiAgICAJICAgIHZhciBhcnJFbnRpdGllcyA9IHsnbHQnOiAnPCcsICdndCc6ICc+JywgJ25ic3AnOiAnJywgJ2FtcCc6ICcmJywgJ3F1b3QnOiAnIid9OwogICAgCSAgICByZXR1cm4gc3RyLnJlcGxhY2UoLyYobHR8Z3R8bmJzcHxhbXB8cXVvdCk7L2lnLCBmdW5jdGlvbiAoYWxsLCB0KSB7CiAgICAJICAgICAgICByZXR1cm4gYXJyRW50aXRpZXNbdF07CiAgICAJICAgIH0pOwogICAgCX0KCQogICAgCWZ1bmN0aW9uIGxvYWQoKSB7CiAgICAJICAgIGxldCBjb2RlTGlzdCA9IGRvY3VtZW50LmdldEVsZW1lbnRzQnlDbGFzc05hbWUoJ2xhbmd1YWdlLXNxbCcpOwoJCiAgICAJICAgIGZvciAobGV0IGkgPSAwIDtpPGNvZGVMaXN0Lmxlbmd0aDtpKyspIHsKICAgIAkgICAgICAgIGNvZGVMaXN0W2ldLmlubmVySFRNTCA9IHdpbmRvdy5zcWxGb3JtYXR0ZXIuZm9ybWF0KGVzY2FwZTJIdG1sKGNvZGVMaXN0W2ldLmlubmVySFRNTCkpCiAgICAJICAgIH0KICAgIAl9Owo=
`
// MarkdownEscape markdown格式转义，原样输出
func MarkdownEscape(str string) string {
for _, b := range "_`*" {
str = strings.Replace(str, string(b), "\\"+string(b), -1)
}
return str
}
// loadExternalResource load js/css resource from http[s] url
func loadExternalResource(resource string) string {
var content string
var body []byte
if strings.HasPrefix(resource, "http") {
resp, err := http.Get(resource)
if err != nil {
Log.Error("http.Get %s Error: %v", resource, err)
return content
}
defer resp.Body.Close()
body, err = ioutil.ReadAll(resp.Body)
if err != nil {
Log.Error("ioutil.ReadAll %s Error: %v", resource, err)
} else {
content = string(body)
}
} else {
fd, err := os.Open(resource)
if err != nil {
Log.Error("os.Open %s Error: %v", resource, err)
return content
}
defer fd.Close()
body, err = ioutil.ReadAll(fd)
if err != nil {
Log.Error("ioutil.ReadAll %s Error: %v", resource, err)
} else {
content = string(body)
}
}
return content
}
// MarkdownHTMLHeader markdown 转 HTML 输出时添加 HTML 头
func MarkdownHTMLHeader() string {
// load css
var css string
if Config.ReportCSS == "" {
css = BuiltinCSS
} else {
css = loadExternalResource(Config.ReportCSS)
}
// load javascript
var js string
if Config.ReportJavascript == "" {
decode, _ := base64.StdEncoding.DecodeString(BuiltinJavascript)
js = string(decode)
} else {
js = loadExternalResource(Config.ReportJavascript)
}
header := `
` + Config.ReportTitle + `
`
return header
}
// Markdown2HTML markdown 转 HTML 输出
func Markdown2HTML(buf string) string {
// extensions default: 94
// extensions |= blackfriday.EXTENSION_TABLES
// extensions |= blackfriday.EXTENSION_FENCED_CODE
// extensions |= blackfriday.EXTENSION_AUTOLINK
// extensions |= blackfriday.EXTENSION_STRIKETHROUGH
// extensions |= blackfriday.EXTENSION_SPACE_HEADERS
extensions := Config.MarkdownExtensions
// htmlFlags
htmlFlags := Config.MarkdownHTMLFlags
renderer := blackfriday.HtmlRenderer(htmlFlags, "", "")
buf = string(blackfriday.Markdown([]byte(buf), renderer, extensions))
return buf
}
// Score SQL评审打分
func Score(score int) string {
// 不需要打分的功能
switch Config.ReportType {
case "duplicate-key-checker", "explain-digest":
return ""
}
s1, s2 := "★ ", "☆ "
if score > 100 {
score = 100
Log.Debug("Score Error: score larger than 100, %d", score)
}
if score < 0 {
score = 0
Log.Debug("Score Warn: score less than 0, %d", score)
}
s1Count := score / 20
s2Count := 5 - s1Count
str := fmt.Sprintf("%s %d分", strings.TrimSpace(strings.Repeat(s1, s1Count)+strings.Repeat(s2, s2Count)), score)
return str
}
================================================
FILE: common/markdown_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
"testing"
)
func TestMarkdownEscape(_ *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
var strs = []string{
"a`bc",
"abc",
"a'bc",
"a\"bc",
}
for _, str := range strs {
fmt.Println(MarkdownEscape(str))
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestMarkdown2Html(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
md := filepath.Join("testdata", t.Name()+".md")
buf, err := ioutil.ReadFile(md)
if err != nil {
t.Error(err.Error())
}
err = GoldenDiff(func() {
fmt.Println(Markdown2HTML(string(buf)))
}, t.Name(), update)
if nil != err {
t.Fatal(err)
}
// golden 文件拷贝成 html 文件，这步是给人看的
gd, err := os.OpenFile("testdata/"+t.Name()+".golden", os.O_RDONLY, 0666)
if nil != err {
t.Fatal(err)
}
html, err := os.OpenFile("testdata/"+t.Name()+".html", os.O_CREATE|os.O_RDWR, 0666)
if nil != err {
t.Fatal(err)
}
io.Copy(html, gd)
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestScore(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
scores := map[int]string{
50: "★ ★ ☆ ☆ ☆ 50分",
100: "★ ★ ★ ★ ★ 100分",
-1: "☆ ☆ ☆ ☆ ☆ 0分",
101: "★ ★ ★ ★ ★ 100分",
}
for score, want := range scores {
get := Score(score)
if want != get {
t.Error(score, want, get)
}
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestLoadExternalResource(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
buf := loadExternalResource("../doc/themes/github.css")
if buf == "" {
t.Error("loadExternalResource local error")
}
buf = loadExternalResource("http://www.baidu.com")
if buf == "" {
t.Error("loadExternalResource http error")
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestMarkdownHTMLHeader(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
err := GoldenDiff(func() {
MarkdownHTMLHeader()
}, t.Name(), update)
if err != nil {
t.Error(err)
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
================================================
FILE: common/meta.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"strconv"
"strings"
)
// Meta 以 'database' 为 key, DB 的 map, 按 db->table->column 组织的元数据
type Meta map[string]*DB
// DB 数据库相关的结构体
type DB struct {
Name string
Table map[string]*Table // ['table_name']*TableName
}
// NewDB 用于初始化*DB
func NewDB(db string) *DB {
return &DB{
Name: db,
Table: make(map[string]*Table),
}
}
// Table 含有表的属性
type Table struct {
TableName string
TableAliases []string
Column map[string]*Column
}
// NewTable 初始化*TableName
func NewTable(tb string) *Table {
return &Table{
TableName: tb,
TableAliases: make([]string, 0),
Column: make(map[string]*Column),
}
}
// KeyType 用于标志每个Key的类别
type KeyType int
// Column 含有列的定义属性
type Column struct {
Name string `json:"col_name"` // 列名
Alias []string `json:"alias"` // 别名
Table string `json:"tb_name"` // 表名
DB string `json:"db_name"` // 数据库名称
DataType string `json:"data_type"` // 数据类型
Character string `json:"character"` // 字符集
Collation string `json:"collation"` // collation
Cardinality float64 `json:"cardinality"` // 散粒度
Null string `json:"null"` // 是否为空: YES/NO
Key string `json:"key"` // 键类型
Default string `json:"default"` // 默认值
Extra string `json:"extra"` // 其他
Comment string `json:"comment"` // 备注
Privileges string `json:"privileges"` // 权限
}
// TableColumns 这个结构体中的元素是有序的 map[db]map[table][]columns
type TableColumns map[string]map[string][]*Column
// Equal 判断两个column是否相等
func (col *Column) Equal(column *Column) bool {
return col.Name == column.Name &&
col.Table == column.Table &&
col.DB == column.DB
}
// IsColsPart 判断两个column队列是否是包含关系（包括相等）
func IsColsPart(a, b []*Column) bool {
times := len(a)
if len(b) < times {
times = len(b)
}
for i := 0; i < times; i++ {
if strings.ToLower(a[i].DB) != strings.ToLower(b[i].DB) ||
strings.ToLower(a[i].Table) != strings.ToLower(b[i].Table) ||
strings.ToLower(a[i].Name) != strings.ToLower(b[i].Name) {
return false
}
}
return true
}
// JoinColumnsName 将所有的列合并
func JoinColumnsName(cols []*Column, sep string) string {
name := ""
for _, col := range cols {
name += col.Name + sep
}
return strings.Trim(name, sep)
}
// Tables 获取 Meta 中指定 db 的所有表名
// Input:数据库名
// Output:表名组成的list
func (b Meta) Tables(db string) []string {
var result []string
if b[db] != nil {
for tb := range b[db].Table {
result = append(result, tb)
}
}
return result
}
// SetDefault 设置默认值
func (b Meta) SetDefault(defaultDB string) Meta {
if defaultDB == "" {
return b
}
for db := range b {
if db == "" {
// 当获取到的 join 中的 DB 为空的时候，说明 SQL 未显示的指定 DB，即使用的是 rEnv 默认 DB, 需要将表合并到原 DB 中
if _, ok := b[defaultDB]; ok {
for tbName, table := range b[""].Table {
if _, ok := b[defaultDB].Table[tbName]; ok {
b[defaultDB].Table[tbName].TableAliases = append(
b[defaultDB].Table[tbName].TableAliases,
table.TableAliases...,
)
continue
}
b[defaultDB].Table[tbName] = table
}
delete(b, "")
}
// 如果没有出现DB指定不一致的情况，直接进行合并
b[defaultDB] = b[""]
delete(b, "")
}
}
return b
}
// MergeColumn 将使用到的列按 db->table 组织去重
// 注意：Column 中的 db, table 信息可能为空，需要提前通过env环境补齐再调用该函数。
// @input: 目标列list, 源列list（可以将多个源合并到一个目标列list）
// @output: 合并后的列list
func MergeColumn(dst []*Column, src ...*Column) []*Column {
var tmp []*Column
for _, newCol := range src {
if len(dst) == 0 {
tmp = append(tmp, newCol)
continue
}
has := false
for _, oldCol := range dst {
if (newCol.Name == oldCol.Name) && (newCol.Table == oldCol.Table) && (newCol.DB == oldCol.DB) {
has = true
}
}
if !has {
tmp = append(tmp, newCol)
}
}
return append(dst, tmp...)
}
// ColumnSort 通过散粒度对 colList 进行排序， 散粒度排序由大到小
func ColumnSort(colList []*Column) []*Column {
// 使用冒泡排序保持相等情况下左右两边顺序不变
if len(colList) < 2 {
return colList
}
for i := 0; i < len(colList)-1; i++ {
for j := i + 1; j < len(colList); j++ {
if colList[i].Cardinality < colList[j].Cardinality {
colList[i], colList[j] = colList[j], colList[i]
}
}
}
return colList
}
// GetDataTypeBase 获取dataType中的数据类型，忽略长度
func GetDataTypeBase(dataType string) string {
dataType = strings.TrimSpace(dataType)
if dataType == "" {
return dataType
}
// smallint unsigned, remove unsigned
dataType = strings.Fields(dataType)[0]
// int(10), remote (10)
if i := strings.Index(dataType, "("); i > 0 {
return dataType[0:i]
}
return dataType
}
// GetDataTypeLength 获取dataType中的数据类型长度
func GetDataTypeLength(dataType string) []int {
var length []int
if si := strings.Index(dataType, "("); si > 0 {
dataLength := dataType[si+1:]
if ei := strings.Index(dataLength, ")"); ei > 0 {
dataLength = dataLength[:ei]
if strings.HasPrefix(dataType, "enum") ||
strings.HasPrefix(dataType, "set") {
// set('one', 'two'), enum('G','PG','PG-13','R','NC-17')
length = []int{len(strings.Split(dataLength, ","))}
} else {
// char(10), varchar(10)
for _, l := range strings.Split(dataLength, ",") {
v, err := strconv.Atoi(l)
if err != nil {
Log.Debug("GetDataTypeLength() Error: %v", err)
return []int{-1}
}
length = append(length, v)
}
}
}
}
if len(length) == 0 {
length = []int{-1}
}
return length
}
// GetDataBytes 计算数据类型字节数
// https://dev.mysql.com/doc/refman/8.0/en/storage-requirements.html
// return -1 表示该列无法计算数据大小
func (col *Column) GetDataBytes(dbVersion int) int {
if col.DataType == "" {
Log.Warning("Can't get %s.%s data type", col.Table, col.Name)
return -1
}
switch strings.ToLower(GetDataTypeBase(col.DataType)) {
case "tinyint", "smallint", "mediumint",
"int", "integer", "bigint",
"double", "real", "float", "decimal",
"numeric", "bit", "smallint unsigned":
// numeric
return numericStorageReq(col.DataType)
case "year", "date", "time", "datetime", "timestamp":
// date & time
return timeStorageReq(col.DataType, dbVersion)
case "char", "binary", "varchar", "varbinary", "enum", "set":
// string
return StringStorageReq(col.DataType, col.Character)
case "tinyblob", "tinytext", "blob", "text", "mediumblob", "mediumtext",
"longblob", "longtext":
// strings length depend on it's values
// 这些字段为不定长字段，添加索引时必须指定前缀，索引前缀与字符集相关
return Config.MaxIdxBytesPerColumn + 1
default:
Log.Warning("Type %s not support:", col.DataType)
return -1
}
}
// Numeric Type Storage Requirements
// return bytes count
func numericStorageReq(dataType string) int {
typeLength := GetDataTypeLength(dataType)
baseType := strings.ToLower(GetDataTypeBase(dataType))
switch baseType {
case "tinyint":
return 1
case "smallint":
return 2
case "mediumint":
return 3
case "int", "integer":
return 4
case "bigint", "double", "real":
return 8
case "float":
if typeLength[0] == -1 || typeLength[0] >= 0 && typeLength[0] <= 24 {
// 4 bytes if 0 <= p <= 24
return 4
}
// 8 bytes if no p || 25 <= p <= 53
return 8
case "decimal", "numeric":
// Values for DECIMAL (and NUMERIC) columns are represented using a binary format
// that packs nine decimal (base 10) digits into four bytes. Storage for the integer
// and fractional parts of each value are determined separately. Each multiple of nine
// digits requires four bytes, and the “leftover” digits require some fraction of four bytes.
if typeLength[0] == -1 {
return 4
}
leftover := func(leftover int) int {
if leftover > 0 && leftover <= 2 {
return 1
} else if leftover > 2 && leftover <= 4 {
return 2
} else if leftover > 4 && leftover <= 6 {
return 3
} else if leftover > 6 && leftover <= 8 {
return 4
} else {
return 4
}
}
integer := typeLength[0]/9*4 + leftover(typeLength[0]%9)
fractional := typeLength[1]/9*4 + leftover(typeLength[1]%9)
return integer + fractional
case "bit":
// approximately (M+7)/8 bytes
if typeLength[0] == -1 {
return 1
}
return (typeLength[0] + 7) / 8
default:
Log.Error("No such numeric type: %s", baseType)
return 8
}
}
// Date and Time Type Storage Requirements
// return bytes count
func timeStorageReq(dataType string, version int) int {
/*
https://dev.mysql.com/doc/refman/8.0/en/storage-requirements.html
* ============================================================================================
* | Data Type | Storage Required Before MySQL 5.6.4 | Storage Required as of MySQL 5.6.4 |
* | ---------------------------------------------------------------------------------------- |
* | YEAR | 1 byte | 1 byte |
* | DATE | 3 bytes | 3 bytes |
* | TIME | 3 bytes | 3 bytes + fractional seconds storage |
* | DATETIME | 8 bytes | 5 bytes + fractional seconds storage |
* | TIMESTAMP | 4 bytes | 4 bytes + fractional seconds storage |
* ============================================================================================
* | Fractional Seconds Precision |Storage Required |
* | ------------------------------------------------ |
* | 0 |0 bytes |
* | 1, 2 |1 byte |
* | 3, 4 |2 bytes |
* | 5, 6 |3 bytes |
* ====================================================
*/
typeLength := GetDataTypeLength(dataType)
extr := func(length int) int {
if length > 0 && length <= 2 {
return 1
} else if length > 2 && length <= 4 {
return 2
} else if length > 4 && length <= 6 || length > 6 {
return 3
}
return 0
}
switch strings.ToLower(GetDataTypeBase(dataType)) {
case "year":
return 1
case "date":
return 3
case "time":
if version < 50604 {
return 3
}
// 3 bytes + fractional seconds storage
return 3 + extr(typeLength[0])
case "datetime":
if version < 50604 {
return 8
}
// 5 bytes + fractional seconds storage
return 5 + extr(typeLength[0])
case "timestamp":
if version < 50604 {
return 4
}
// 4 bytes + fractional seconds storage
return 4 + extr(typeLength[0])
default:
return 8
}
}
// SHOW CHARACTER SET
// CharSets character bytes per charcharacter bytes per char
var CharSets = map[string]int{
"armscii8": 1,
"ascii": 1,
"big5": 2,
"binary": 1,
"cp1250": 1,
"cp1251": 1,
"cp1256": 1,
"cp1257": 1,
"cp850": 1,
"cp852": 1,
"cp866": 1,
"cp932": 2,
"dec8": 1,
"eucjpms": 3,
"euckr": 2,
"gb18030": 4,
"gb2312": 2,
"gbk": 2,
"geostd8": 1,
"greek": 1,
"hebrew": 1,
"hp8": 1,
"keybcs2": 1,
"koi8r": 1,
"koi8u": 1,
"latin1": 1,
"latin2": 1,
"latin5": 1,
"latin7": 1,
"macce": 1,
"macroman": 1,
"sjis": 2,
"swe7": 1,
"tis620": 1,
"ucs2": 2,
"ujis": 3,
"utf16": 4,
"utf16le": 4,
"utf32": 4,
"utf8": 3,
"utf8mb4": 4,
}
// StringStorageReq String Type Storage Requirements return bytes count
func StringStorageReq(dataType string, charset string) int {
// get bytes per character, default 1
bysPerChar := 1
if _, ok := CharSets[strings.ToLower(charset)]; ok {
bysPerChar = CharSets[strings.ToLower(charset)]
}
// get length
typeLength := GetDataTypeLength(dataType)
if typeLength[0] == -1 {
return 0
}
// get type
baseType := strings.ToLower(GetDataTypeBase(dataType))
switch baseType {
case "char":
// Otherwise, M × w bytes, <= M <= 255,
// where w is the number of bytes required for the maximum-length character in the character set.
if typeLength[0] > 255 {
typeLength[0] = 255
}
return typeLength[0] * bysPerChar
case "binary":
// M bytes, 0 <= M <= 255
if typeLength[0] > 255 {
typeLength[0] = 255
}
return typeLength[0]
case "varchar", "varbinary":
if typeLength[0] < 255 {
return typeLength[0]*bysPerChar + 1
}
return typeLength[0]*bysPerChar + 2
case "enum":
// 1 or 2 bytes, depending on the number of enumeration values (65,535 values maximum)
return typeLength[0]/(2^15) + 1
case "set":
// 1, 2, 3, 4, or 8 bytes, depending on the number of set members (64 members maximum)
return typeLength[0]/8 + 1
default:
return 0
}
}
================================================
FILE: common/meta_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"fmt"
"testing"
)
func TestGetDataTypeLength(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
typeList := map[string][]int{
"varchar(20)": {20},
"int(2)": {2},
"int(2000000)": {2000000},
"DECIMAL(1,2)": {1, 2},
"int": {-1},
}
for typ, want := range typeList {
got := GetDataTypeLength(typ)
for i := 0; i < len(want); i++ {
if want[i] != got[i] {
t.Errorf("Not match, want %v, got %v", want, got)
}
}
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestGetDataTypeBase(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
typeList := map[string]string{
"varchar(20)": "varchar",
"int(2)": "int",
"int(2000000)": "int",
}
for typ := range typeList {
if got := GetDataTypeBase(typ); got != typeList[typ] {
t.Errorf("Not match, want %s, got %s", typeList[typ], got)
}
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestGetDataBytes(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
cols50604 := map[*Column]int{
// numeric type
{Name: "col000", DataType: "tinyint", Character: "utf8"}: 1,
{Name: "col001", DataType: "SMALLINT", Character: "utf8"}: 2,
{Name: "col002", DataType: "MEDIUMINT", Character: "utf8"}: 3,
{Name: "col003", DataType: "int(32)", Character: "utf8"}: 4,
{Name: "col004", DataType: "integer(32)", Character: "utf8"}: 4,
{Name: "col005", DataType: "bigint(10)", Character: "utf8"}: 8,
{Name: "col006", DataType: "float(12)", Character: "utf8"}: 4,
{Name: "col007", DataType: "float(50)", Character: "utf8"}: 8,
{Name: "col008", DataType: "float(100)", Character: "utf8"}: 8,
{Name: "col009", DataType: "float", Character: "utf8"}: 4,
{Name: "col010", DataType: "double", Character: "utf8"}: 8,
{Name: "col011", DataType: "real", Character: "utf8"}: 8,
{Name: "col012", DataType: "BIT(32)", Character: "utf8"}: 4,
{Name: "col013", DataType: "numeric(32,32)", Character: "utf8"}: 30,
{Name: "col013", DataType: "decimal(2,32)", Character: "utf8"}: 16,
{Name: "col014", DataType: "BIT(32)", Character: "utf8"}: 4,
// date & time
{Name: "col015", DataType: "year(32)", Character: "utf8mb4"}: 1,
{Name: "col016", DataType: "date", Character: "utf8mb4"}: 3,
{Name: "col017", DataType: "time", Character: "utf8mb4"}: 3,
{Name: "col018", DataType: "time(0)", Character: "utf8mb4"}: 3,
{Name: "col019", DataType: "time(2)", Character: "utf8mb4"}: 4,
{Name: "col020", DataType: "time(4)", Character: "utf8mb4"}: 5,
{Name: "col021", DataType: "time(6)", Character: "utf8mb4"}: 6,
{Name: "col022", DataType: "datetime", Character: "utf8mb4"}: 5,
{Name: "col023", DataType: "timestamp(32)", Character: "utf8mb4"}: 7,
// string
{Name: "col024", DataType: "varchar(255)", Character: "utf8"}: 767,
{Name: "col025", DataType: "varchar(191)", Character: "utf8mb4"}: 765,
}
for col, bytes := range cols50604 {
if got := col.GetDataBytes(50604); got != bytes {
t.Errorf("Version 5.6.4, %s Not match, want %d, got %d", col.Name, bytes, got)
}
}
cols50500 := map[*Column]int{
// numeric type
{Name: "col000", DataType: "tinyint", Character: "utf8"}: 1,
{Name: "col001", DataType: "SMALLINT", Character: "utf8"}: 2,
{Name: "col002", DataType: "MEDIUMINT", Character: "utf8"}: 3,
{Name: "col003", DataType: "int(32)", Character: "utf8"}: 4,
{Name: "col004", DataType: "integer(32)", Character: "utf8"}: 4,
{Name: "col005", DataType: "bigint(10)", Character: "utf8"}: 8,
{Name: "col006", DataType: "float(12)", Character: "utf8"}: 4,
{Name: "col007", DataType: "float(50)", Character: "utf8"}: 8,
{Name: "col008", DataType: "float(100)", Character: "utf8"}: 8,
{Name: "col009", DataType: "float", Character: "utf8"}: 4,
{Name: "col010", DataType: "double", Character: "utf8"}: 8,
{Name: "col011", DataType: "real", Character: "utf8"}: 8,
{Name: "col012", DataType: "BIT(32)", Character: "utf8"}: 4,
{Name: "col013", DataType: "numeric(32,32)", Character: "utf8"}: 30,
{Name: "col013", DataType: "decimal(2,32)", Character: "utf8"}: 16,
{Name: "col014", DataType: "BIT(32)", Character: "utf8"}: 4,
// date & time
{Name: "col015", DataType: "year(32)", Character: "utf8mb4"}: 1,
{Name: "col016", DataType: "date", Character: "utf8mb4"}: 3,
{Name: "col017", DataType: "time", Character: "utf8mb4"}: 3,
{Name: "col018", DataType: "time(0)", Character: "utf8mb4"}: 3,
{Name: "col019", DataType: "time(2)", Character: "utf8mb4"}: 3,
{Name: "col020", DataType: "time(4)", Character: "utf8mb4"}: 3,
{Name: "col021", DataType: "time(6)", Character: "utf8mb4"}: 3,
{Name: "col022", DataType: "datetime", Character: "utf8mb4"}: 8,
{Name: "col023", DataType: "timestamp(32)", Character: "utf8mb4"}: 4,
// string
{Name: "col024", DataType: "varchar(255)", Character: "utf8"}: 767,
{Name: "col025", DataType: "varchar(191)", Character: "utf8mb4"}: 765,
}
for col, bytes := range cols50500 {
if got := col.GetDataBytes(50500); got != bytes {
t.Errorf("Version: 5.5.0, %s Not match, want %d, got %d", col.Name, bytes, got)
}
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestStringStorageReq(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
dataTypes := []string{
"char(10)",
"char(256)",
"binary(10)",
"binary(256)",
"varchar(10)",
"varbinary(10)",
"enum('G','PG','PG-13','R','NC-17')",
"set('one', 'two')",
// wrong case
"not_exist",
"char(-1)",
}
err := GoldenDiff(func() {
for _, name := range SortedKey(CharSets) {
for _, tp := range dataTypes {
fmt.Println(tp, name, StringStorageReq(tp, name))
}
}
}, t.Name(), update)
if err != nil {
t.Error(err)
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
================================================
FILE: common/signal.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"os"
"os/signal"
"syscall"
)
// HandleSignal 当程序卡死的时候，或者由于某些原因程序没有退出，可以通过捕获信号量的形式让程序优雅退出并且清理测试环境
func HandleSignal(f func()) {
sc := make(chan os.Signal, 1)
signal.Notify(sc,
syscall.SIGHUP,
syscall.SIGINT,
syscall.SIGTERM,
syscall.SIGQUIT)
go func() {
n := <-sc
Log.Info("receive signal %v, closing", n)
f()
}()
}
================================================
FILE: common/signal_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"fmt"
"testing"
)
func TestHandleSignal(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
HandleSignal(func() {
fmt.Println("done")
})
Log.Debug("Exiting function: %s", GetFunctionName())
}
================================================
FILE: common/testdata/TestJSONFind.golden
================================================
[{
"Collate":{
"Collate":{
"key":"value"
}
}
} {
"Collate":{
"key":"value"
}
} {
"key":"value"
}]
[McLaughlin Hunter Harold]
[{
"title": "Sample Konfabulator Widget",
"name": "main_window",
"width": 500,
"height": 500
}]
[ binary binary utf8mb4_bin ]
================================================
FILE: common/testdata/TestListReportTypes.golden
================================================
# 支持的报告类型
[toc]
## lint
* **Description**:参考sqlint格式，以插件形式集成到代码编辑器，显示输出更加友好
* **Example**:
```bash
soar -report-type lint -query test.sql
```
## markdown
* **Description**:该格式为默认输出格式，以markdown格式展现，可以用网页浏览器插件直接打开，也可以用markdown编辑器打开
* **Example**:
```bash
echo "select * from film" | soar
```
## rewrite
* **Description**:SQL重写功能，配合-rewrite-rules参数一起使用，可以通过-list-rewrite-rules 查看所有支持的 SQL 重写规则
* **Example**:
```bash
echo "select * from film" | soar -rewrite-rules star2columns,delimiter -report-type rewrite
```
## ast
* **Description**:输出 SQL 的抽象语法树，主要用于测试
* **Example**:
```bash
echo "select * from film" | soar -report-type ast
```
## ast-json
* **Description**:以 JSON 格式输出 SQL 的抽象语法树，主要用于测试
* **Example**:
```bash
echo "select * from film" | soar -report-type ast-json
```
## tiast
* **Description**:输出 SQL 的 TiDB抽象语法树，主要用于测试
* **Example**:
```bash
echo "select * from film" | soar -report-type tiast
```
## tiast-json
* **Description**:以 JSON 格式输出 SQL 的 TiDB抽象语法树，主要用于测试
* **Example**:
```bash
echo "select * from film" | soar -report-type tiast-json
```
## tables
* **Description**:以 JSON 格式输出 SQL 使用的库表名
* **Example**:
```bash
echo "select * from film" | soar -report-type tables
```
## query-type
* **Description**:SQL 语句的请求类型
* **Example**:
```bash
echo "select * from film" | soar -report-type query-type
```
## fingerprint
* **Description**:输出SQL的指纹
* **Example**:
```bash
echo "select * from film where language_id=1" | soar -report-type fingerprint
```
## md2html
* **Description**:markdown 格式转 html 格式小工具
* **Example**:
```bash
soar -list-heuristic-rules | soar -report-type md2html > heuristic_rules.html
```
## explain-digest
* **Description**:输入为EXPLAIN的表格，JSON 或 Vertical格式，对其进行分析，给出分析结果
* **Example**:
```bash
soar -report-type explain-digest << EOF
+----+-------------+-------+------+---------------+------+---------+------+------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+------+-------+
| 1 | SIMPLE | film | ALL | NULL | NULL | NULL | NULL | 1131 | |
+----+-------------+-------+------+---------------+------+---------+------+------+-------+
EOF
```
## duplicate-key-checker
* **Description**:对 OnlineDsn 中指定的 database 进行索引重复检查
* **Example**:
```bash
soar -report-type duplicate-key-checker -online-dsn user:password@127.0.0.1:3306/db
```
## html
* **Description**:以HTML格式输出报表
* **Example**:
```bash
echo "select * from film" | soar -report-type html
```
## json
* **Description**:输出JSON格式报表，方便应用程序处理
* **Example**:
```bash
echo "select * from film" | soar -report-type json
```
## tokenize
* **Description**:对SQL进行切词，主要用于测试
* **Example**:
```bash
echo "select * from film" | soar -report-type tokenize
```
## compress
* **Description**:SQL压缩小工具，使用内置SQL压缩逻辑，测试中的功能
* **Example**:
```bash
echo "select
*
from
film" | soar -report-type compress
```
## pretty
* **Description**:使用kr/pretty打印报告，主要用于测试
* **Example**:
```bash
echo "select * from film" | soar -report-type pretty
```
## remove-comment
* **Description**:去除SQL语句中的注释，支持单行多行注释的去除
* **Example**:
```bash
echo "select/*comment*/ * from film" | soar -report-type remove-comment
```
## chardet
* **Description**:猜测输入的 SQL 使用的字符集
* **Example**:
```bash
echo '中文' | soar -report-type chardet
```
================================================
FILE: common/testdata/TestMarkdown2Html.golden
================================================
Markdown For Typora
Overview
Markdown is created by Daring Fireball, the original guideline is here. Its syntax, however, varies between different parsers or editors. Typora is using GitHub Flavored Markdown.
Please note that HTML fragments in markdown source will be recognized but not parsed or rendered. Also, there may be small reformatting on the original markdown source code after saving.
Outline
[TOC]
Block Elements
Paragraph and line breaks
A paragraph is simply one or more consecutive lines of text. In markdown source code, paragraphs are separated by more than one blank lines. In Typora, you only need to press Return to create a new paragraph.
Press Shift + Return to create a single line break. However, most markdown parser will ignore single line break, to make other markdown parsers recognize your line break, you can leave two whitespace at the end of the line, or insert <br/>.
Headers
Headers use 1-6 hash characters at the start of the line, corresponding to header levels 1-6. For example:
# This is an H1
## This is an H2
###### This is an H6
In typora, input ‘#’s followed by title content, and press Return key will create a header.
Blockquotes
Markdown uses email-style > characters for block quoting. They are presented as:
> This is a blockquote with two paragraphs. This is first paragraph.
>
> This is second pragraph.Vestibulum enim wisi, viverra nec, fringilla in, laoreet vitae, risus.
> This is another blockquote with one paragraph. There is three empty line to seperate two blockquote.
In typora, just input ‘>’ followed by quote contents a block quote is generated. Typora will insert proper ‘>’ or line break for you. Block quote inside anther block quote is allowed by adding additional levels of ‘>’.
Lists
Input * list item 1 will create an un-ordered list, the * symbol can be replace with + or -.
Input 1. list item 1 will create an ordered list, their markdown source code is like:
## un-ordered list
* Red
* Green
* Blue
## ordered list
1. Red
2. Green
3. Blue
Task List
Task lists are lists with items marked as either [ ] or x. For example:
- [ ] a task list item
- [ ] list syntax required
- [ ] normal **formatting**, @mentions, #1234 refs
- [ ] incomplete
- [x] completed
You can change the complete/incomplete state by click the checkbox before the item.
(Fenced) Code Blocks
Typora only support fences in Github Flavored Markdown. Original code blocks in markdown is not supported.
Using fences is easy: Input ``` and press return. Add an optional language identifier after ``` and we'll run it through syntax highlighting:
Here's an example:
```
function test() {
console.log("notice the blank line before this function?");
}
```
syntax highlighting:
```ruby
require 'redcarpet'
markdown = Redcarpet.new("Hello World!")
puts markdown.to_html
```
Math Blocks
You can render LaTeX mathematical expressions using MathJax.
Input $$, then press 'Return' key will trigger an input field which accept Tex/LaTex source. Following is an example:
$$
\mathbf{V}1 \times \mathbf{V}2 = \begin{vmatrix}
\mathbf{i} & \mathbf{j} & \mathbf{k} \
\frac{\partial X}{\partial u} & \frac{\partial Y}{\partial u} & 0 \
\frac{\partial X}{\partial v} & \frac{\partial Y}{\partial v} & 0 \
\end{vmatrix}
$$
In markdown source file, math block is LaTeX expression wrapped by ‘$$’ mark:
$$
\mathbf{V}_1 \times \mathbf{V}_2 = \begin{vmatrix}
\mathbf{i} & \mathbf{j} & \mathbf{k} \\
\frac{\partial X}{\partial u} & \frac{\partial Y}{\partial u} & 0 \\
\frac{\partial X}{\partial v} & \frac{\partial Y}{\partial v} & 0 \\
\end{vmatrix}
$$
Tables
Input | First Header | Second Header | and press return key will create a table with two column.
After table is created, focus on that table will pop up a toolbar for table, where you can resize, align, or delete table. You can also use context menu to copy and add/delete column/row.
Following descriptions can be skipped, as markdown source code for tables are generated by typora automatically.
In markdown source code, they look like:
| First Header | Second Header |
| ------------- | ------------- |
| Content Cell | Content Cell |
| Content Cell | Content Cell |
You can also include inline Markdown such as links, bold, italics, or strikethrough.
Finally, by including colons : within the header row, you can define text to be left-aligned, right-aligned, or center-aligned:
| Left-Aligned | Center Aligned | Right Aligned |
| :------------ |:---------------:| -----:|
| col 3 is | some wordy text | $1600 |
| col 2 is | centered | $12 |
| zebra stripes | are neat | $1 |
A colon on the left-most side indicates a left-aligned column; a colon on the right-most side indicates a right-aligned column; a colon on both sides indicates a center-aligned column.
Footnotes
You can create footnotes like this[^footnote].
[^footnote]: Here is the *text* of the **footnote**.
will produce:
You can create footnotes like this[^footnote].
[^footnote]: Here is the text of the footnote.
Mouse on the ‘footnote’ superscript to see content of the footnote.
Horizontal Rules
Input *** or --- on a blank line and press return will draw a horizontal line.
YAML Front Matter
Typora support YAML Front Matter now. Input --- at the top of the article and then press Enter will introduce one. Or insert one metadata block from the menu.
TableName of Contents (TOC)
Input [toc] then press Return key will create a section for “TableName of Contents” extracting all headers from one’s writing, its contents will be updated automatically.
Diagrams (Sequence, Flowchart and Mermaid)
Typora supports, sequence, flowchart and mermaid, after this feature is enabled from preference panel.
See this document for detail.
Span Elements
Span elements will be parsed and rendered right after your typing. Moving cursor in middle of those span elements will expand those elements into markdown source. Following will explain the syntax of those span element.
Links
Markdown supports two style of links: inline and reference.
In both styles, the link text is delimited by [square brackets].
To create an inline link, use a set of regular parentheses immediately after the link text’s closing square bracket. Inside the parentheses, put the URL where you want the link to point, along with an optional title for the link, surrounded in quotes. For example:
This is [an example](http://example.com/ "Title") inline link.
[This link](http://example.net/) has no title attribute.
will produce:
This is an example inline link. (<p>This is <a href="http://example.com/" title="Title">)
This link has no title attribute. (<p><a href="http://example.net/">This link</a> has no)
Internal Links
You can set the href to headers, which will create a bookmark that allow you to jump to that section after clicking. For example:
Command(on Windows: Ctrl) + Click This link will jump to header Block Elements. To see how to write that, please move cursor or click that link with ⌘ key pressed to expand the element into markdown source.
Reference Links
Reference-style links use a second set of square brackets, inside which you place a label of your choosing to identify the link:
This is [an example][id] reference-style link.
Then, anywhere in the document, you define your link label like this, on a line by itself:
[id]: http://example.com/ "Optional Title Here"
In typora, they will be rendered like:
This is an example reference-style link.
The implicit link name shortcut allows you to omit the name of the link, in which case the link text itself is used as the name. Just use an empty set of square brackets — e.g., to link the word “Google” to the google.com web site, you could simply write:
[Google][]
And then define the link:
[Google]: http://google.com/
In typora click link will expand it for editing, command+click will open the hyperlink in web browser.
URLs
Typora allows you to insert urls as links, wrapped by <brackets>.
<i@typora.io> becomes i@typora.io.
Typora will aslo auto link standard URLs. e.g: www.google.com.
Images
Image looks similar with links, but it requires an additional ! char before the start of link. Image syntax looks like this:
You are able to use drag & drop to insert image from image file or we browser. And modify the markdown source code by clicking on the image. Relative path will be used if image is in same directory or sub-directory with current editing document when drag & drop.
For more tips on images, please read http://support.typora.io//Images/
Emphasis
Markdown treats asterisks (*) and underscores (_) as indicators of emphasis. Text wrapped with one * or _ will be wrapped with an HTML <em> tag. E.g:
*single asterisks*
_single underscores_
output:
single asterisks
single underscores
GFM will ignores underscores in words, which is commonly used in code and names, like this:
wowgreatstuff
dothisanddothatandanother_thing.
To produce a literal asterisk or underscore at a position where it would otherwise be used as an emphasis delimiter, you can backslash escape it:
\*this text is surrounded by literal asterisks\*
Typora recommends to use * symbol.
Strong
double *’s or _’s will be wrapped with an HTML <strong> tag, e.g:
**double asterisks**
__double underscores__
output:
double asterisks
double underscores
Typora recommends to use ** symbol.
Code
To indicate a span of code, wrap it with backtick quotes (`). Unlike a pre-formatted code block, a code span indicates code within a normal paragraph. For example:
Use the `printf()` function.
will produce:
Use the printf() function.
Strikethrough
GFM adds syntax to create strikethrough text, which is missing from standard Markdown.
~~Mistaken text.~~ becomes Mistaken text.
Underline
Underline is powered by raw HTML.
<u>Underline</u> becomes Underline.
Emoji :happy:
Input emoji with syntax :smile:.
User can trigger auto-complete suggestions for emoji by pressing ESC key, or trigger it automatically after enable it on preference panel. Also, input UTF8 emoji char directly from Edit -> Emoji & Symbols from menu bar is also supported.
HTML
Typora cannot render html fragments. But typora can parse and render very limited HTML fragments, as an extension of Markdown, including:
<u>underline</u><img src="http://www.w3.org/html/logo/img/mark-word-icon.png" width="200px" /> (And width, height attribute in HTML tag, and width, height, zoom style in style attribute will be applied.)<!-- This is some comments --><a href="http://typora.io" target="_blank">link</a>.Most of their attributes, styles, or classes will be ignored. For other tags, typora will render them as raw HTML snippets.
But those HTML will be exported on print or export.
To use this feature, first, please enable it in Preference Panel -> Markdown Tab. Then use $ to wrap TeX command, for example: $\lim_{x \to \infty} \exp(-x) = 0$ will be rendered as LaTeX command.
To trigger inline preview for inline math: input “$”, then press ESC key, then input TeX command, a preview tooltip will be visible like below:
To use this feature, first, please enable it in Preference Panel -> Markdown Tab. Then use ~ to wrap subscript content, for example: H~2~O, X~long\ text~/
To use this feature, first, please enable it in Preference Panel -> Markdown Tab. Then use ^ to wrap superscript content, for example: X^2^.
To use this feature, first, please enable it in Preference Panel -> Markdown Tab. Then use == to wrap superscript content, for example: ==highlight==.
This is `)
[This link](http://example.net/) has no title attribute. (` This link has no`)
#### Internal Links
**You can set the href to headers**, which will create a bookmark that allow you to jump to that section after clicking. For example:
Command(on Windows: Ctrl) + Click [This link](#block-elements) will jump to header `Block Elements`. To see how to write that, please move cursor or click that link with `⌘` key pressed to expand the element into markdown source.
#### Reference Links
Reference-style links use a second set of square brackets, inside which you place a label of your choosing to identify the link:
``` markdown
This is [an example][id] reference-style link.
Then, anywhere in the document, you define your link label like this, on a line by itself:
[id]: http://example.com/ "Optional Title Here"
```
In typora, they will be rendered like:
This is [an example][id] reference-style link.
[id]: http://example.com/ "Optional Title Here"
The implicit link name shortcut allows you to omit the name of the link, in which case the link text itself is used as the name. Just use an empty set of square brackets — e.g., to link the word “Google” to the google.com web site, you could simply write:
``` markdown
[Google][]
And then define the link:
[Google]: http://google.com/
```
In typora click link will expand it for editing, command+click will open the hyperlink in web browser.
### URLs
Typora allows you to insert urls as links, wrapped by `<`brackets`>`.
` ★ ★ ★ ☆ ☆ 75分 Item: CLA.001 Severity: L4 Content: SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。 Item: COL.001 Severity: L1 Content: 当表结构变更时，使用 * 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
` (And `width`, `height` attribute in HTML tag, and `width`, `height`, `zoom` style in `style` attribute will be applied.)
- Comments: ``
- Hyperlink: `link`.
Most of their attributes, styles, or classes will be ignored. For other tags, typora will render them as raw HTML snippets.
But those HTML will be exported on print or export.
### Inline Math
To use this feature, first, please enable it in `Preference` Panel -> `Markdown` Tab. Then use `$` to wrap TeX command, for example: `$\lim_{x \to \infty} \exp(-x) = 0$` will be rendered as LaTeX command.
To trigger inline preview for inline math: input “$”, then press `ESC` key, then input TeX command, a preview tooltip will be visible like below:
### Subscript
To use this feature, first, please enable it in `Preference` Panel -> `Markdown` Tab. Then use `~` to wrap subscript content, for example: `H~2~O`, `X~long\ text~`/
### Superscript
To use this feature, first, please enable it in `Preference` Panel -> `Markdown` Tab. Then use `^` to wrap superscript content, for example: `X^2^`.
### Highlight
To use this feature, first, please enable it in `Preference` Panel -> `Markdown` Tab. Then use `==` to wrap superscript content, for example: `==highlight==`.
[GFM]: https://help.github.com/articles/github-flavored-markdown/
================================================
FILE: common/testdata/TestMarkdownHTMLHeader.golden
================================================
================================================
FILE: common/testdata/TestParseDSN.golden
================================================
&common.Dsn{
User: "",
Password: "",
Net: "tcp",
Addr: "127.0.0.1:3306",
Schema: "",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@hostname:3307/database
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "hostname:3307",
Schema: "database",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@hostname:3307/database?charset=utf8
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "hostname:3307",
Schema: "database",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@hostname:3307
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "hostname:3307",
Schema: "information_schema",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@hostname:/database
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "hostname:3306",
Schema: "database",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@:3307/database
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "127.0.0.1:3307",
Schema: "database",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user@hostname/database
&common.Dsn{
User: "user",
Password: "",
Net: "tcp",
Addr: "hostname:3306",
Schema: "database",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:pwd:pwd@pwd/pwd@hostname/database
&common.Dsn{
User: "user",
Password: "pwd:pwd@pwd/pwd",
Net: "tcp",
Addr: "hostname:3306",
Schema: "database",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "127.0.0.1:3306",
Schema: "information_schema",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
hostname:3307/database
&common.Dsn{
User: "",
Password: "",
Net: "tcp",
Addr: "hostname:3307",
Schema: "database",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
@hostname:3307/database
&common.Dsn{
User: "",
Password: "",
Net: "tcp",
Addr: "hostname:3307",
Schema: "database",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
@hostname
&common.Dsn{
User: "",
Password: "",
Net: "tcp",
Addr: "hostname:3306",
Schema: "information_schema",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
hostname
&common.Dsn{
User: "",
Password: "",
Net: "tcp",
Addr: "hostname:3306",
Schema: "information_schema",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
@/database
&common.Dsn{
User: "",
Password: "",
Net: "tcp",
Addr: "127.0.0.1:3306",
Schema: "database",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
@hostname:3307
&common.Dsn{
User: "",
Password: "",
Net: "tcp",
Addr: "hostname:3307",
Schema: "information_schema",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
@:3307/database
&common.Dsn{
User: "",
Password: "",
Net: "tcp",
Addr: "127.0.0.1:3307",
Schema: "database",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
:3307/database
&common.Dsn{
User: "",
Password: "",
Net: "tcp",
Addr: "127.0.0.1:3307",
Schema: "database",
Charset: "utf8",
Collation: "",
Loc: "",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "3s",
ReadTimeout: "",
WriteTimeout: "",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
/database
&common.Dsn{
User: "",
Password: "",
Net: "tcp",
Addr: "127.0.0.1:3306",
Schema: "database",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user@unix(/path/to/socket)/dbname
&common.Dsn{
User: "user",
Password: "",
Net: "unix",
Addr: "/path/to/socket",
Schema: "dbname",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
root:pw@unix(/tmp/mysql.sock)/myDatabase?loc=Local
&common.Dsn{
User: "root",
Password: "pw",
Net: "unix",
Addr: "/tmp/mysql.sock",
Schema: "myDatabase",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "Local",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@tcp(localhost:5555)/dbname?tls=skip-verify
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "localhost:5555",
Schema: "dbname",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "skip-verify",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@tcp(localhost:5555)/dbname?autocommit=true
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "localhost:5555",
Schema: "dbname",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {"autocommit":"true"},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@/dbname?sql_mode=TRADITIONAL
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "127.0.0.1:3306",
Schema: "dbname",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {"sql_mode":"TRADITIONAL"},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@tcp([de:ad:be:ef::ca:fe]:80)/dbname?timeout=90s
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "[de:ad:be:ef::ca:fe]:80",
Schema: "dbname",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "1m30s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@tcp([de:ad:be:ef::ca:fe]:80)/dbname?collation=utf8mb4_unicode_ci
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "[de:ad:be:ef::ca:fe]:80",
Schema: "dbname",
Charset: "utf8",
Collation: "utf8mb4_unicode_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
id:password@tcp(your-amazonaws-uri.com:3306)/dbname
&common.Dsn{
User: "id",
Password: "password",
Net: "tcp",
Addr: "your-amazonaws-uri.com:3306",
Schema: "dbname",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user@cloudsql(project-id:instance-name)/dbname
&common.Dsn{
User: "user",
Password: "",
Net: "cloudsql",
Addr: "project-id:instance-name",
Schema: "dbname",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user@cloudsql(project-id:regionname:instance-name)/dbname
&common.Dsn{
User: "user",
Password: "",
Net: "cloudsql",
Addr: "project-id:regionname:instance-name",
Schema: "dbname",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@tcp/dbname?sys_var=esc%40ped
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "127.0.0.1:3306",
Schema: "dbname",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {"sys_var":"esc@ped"},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@/dbname
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "127.0.0.1:3306",
Schema: "dbname",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@/
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "127.0.0.1:3306",
Schema: "",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "0s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
user:password@tcp(localhost:3307)/database?timeout=5s
&common.Dsn{
User: "user",
Password: "password",
Net: "tcp",
Addr: "localhost:3307",
Schema: "database",
Charset: "utf8",
Collation: "utf8mb4_general_ci",
Loc: "UTC",
TLS: "",
ServerPubKey: "",
MaxAllowedPacket: 4194304,
Params: {},
Timeout: "5s",
ReadTimeout: "0s",
WriteTimeout: "0s",
AllowNativePasswords: true,
AllowOldPasswords: false,
Disable: false,
Version: 99999,
}
================================================
FILE: common/testdata/TestPrintConfiguration.golden
================================================
online-dsn:
user: root
password: '********'
net: tcp
addr: 127.0.0.1:3306
schema: sakila
charset: utf8
collation: utf8mb4_general_ci
loc: UTC
tls: ""
server-public-key: ""
max-allowed-packet: 4194304
params:
charset: utf8
timeout: 3s
read-timeout: 0s
write-timeout: 0s
allow-native-passwords: true
allow-old-passwords: false
disable: false
test-dsn:
user: root
password: '********'
net: tcp
addr: 127.0.0.1:3306
schema: sakila
charset: utf8
collation: utf8mb4_general_ci
loc: UTC
tls: ""
server-public-key: ""
max-allowed-packet: 4194304
params:
charset: utf8
timeout: 3s
read-timeout: 0s
write-timeout: 0s
allow-native-passwords: true
allow-old-passwords: false
disable: false
allow-online-as-test: true
disable-version-check: false
drop-test-temporary: true
cleanup-test-database: false
only-syntax-check: false
sampling-statistic-target: 100
sampling: true
sampling-condition: ""
profiling: false
trace: false
explain: true
delimiter: ;
log-level: 7
log-output: soar.log
report-type: markdown
report-css: ""
report-javascript: ""
report-title: SQL优化分析报告
markdown-extensions: 94
markdown-html-flags: 0
ignore-rules:
- COL.011
rewrite-rules:
- delimiter
- orderbynull
- groupbyconst
- dmlorderby
- having
- star2columns
- insertcolumns
- distinctstar
blacklist: ""
max-join-table-count: 5
max-group-by-cols-count: 5
max-distinct-count: 5
max-index-cols-count: 5
max-text-cols-count: 2
max-total-rows: 9999999
max-query-cost: 9999
spaghetti-query-length: 2048
allow-drop-index: false
max-in-count: 10
max-index-bytes-percolumn: 767
max-index-bytes: 3072
allow-charsets:
- utf8
- utf8mb4
allow-collates: []
allow-engines:
- innodb
max-index-count: 10
max-column-count: 40
max-value-count: 100
index-prefix: idx_
unique-key-prefix: uk_
max-subquery-depth: 5
max-varchar-length: 1024
column-not-allow-type:
- json
- text
- boolean
min-cardinality: 0
explain-sql-report-type: pretty
explain-type: extended
explain-format: traditional
explain-warn-select-type:
- ""
explain-warn-access-type:
- ALL
explain-max-keys: 3
explain-min-keys: 0
explain-max-rows: 10000
explain-warn-extra:
- Using temporary
- Using filesort
explain-max-filtered: 100
explain-warn-scalability:
- O(n)
show-warnings: false
show-last-query-cost: false
query: ""
list-heuristic-rules: false
list-rewrite-rules: false
list-test-sqls: false
list-report-types: false
verbose: false
dry-run: true
max-pretty-sql-length: 1024
================================================
FILE: common/testdata/TestRemoveBOM.golden
================================================
select col from tb c = 1;
[239 187 191]
================================================
FILE: common/testdata/TestStringStorageReq.golden
================================================
char(10) armscii8 10
char(256) armscii8 255
binary(10) armscii8 10
binary(256) armscii8 255
varchar(10) armscii8 11
varbinary(10) armscii8 11
enum('G','PG','PG-13','R','NC-17') armscii8 1
set('one', 'two') armscii8 1
not_exist armscii8 0
char(-1) armscii8 0
char(10) ascii 10
char(256) ascii 255
binary(10) ascii 10
binary(256) ascii 255
varchar(10) ascii 11
varbinary(10) ascii 11
enum('G','PG','PG-13','R','NC-17') ascii 1
set('one', 'two') ascii 1
not_exist ascii 0
char(-1) ascii 0
char(10) big5 20
char(256) big5 510
binary(10) big5 10
binary(256) big5 255
varchar(10) big5 21
varbinary(10) big5 21
enum('G','PG','PG-13','R','NC-17') big5 1
set('one', 'two') big5 1
not_exist big5 0
char(-1) big5 0
char(10) binary 10
char(256) binary 255
binary(10) binary 10
binary(256) binary 255
varchar(10) binary 11
varbinary(10) binary 11
enum('G','PG','PG-13','R','NC-17') binary 1
set('one', 'two') binary 1
not_exist binary 0
char(-1) binary 0
char(10) cp1250 10
char(256) cp1250 255
binary(10) cp1250 10
binary(256) cp1250 255
varchar(10) cp1250 11
varbinary(10) cp1250 11
enum('G','PG','PG-13','R','NC-17') cp1250 1
set('one', 'two') cp1250 1
not_exist cp1250 0
char(-1) cp1250 0
char(10) cp1251 10
char(256) cp1251 255
binary(10) cp1251 10
binary(256) cp1251 255
varchar(10) cp1251 11
varbinary(10) cp1251 11
enum('G','PG','PG-13','R','NC-17') cp1251 1
set('one', 'two') cp1251 1
not_exist cp1251 0
char(-1) cp1251 0
char(10) cp1256 10
char(256) cp1256 255
binary(10) cp1256 10
binary(256) cp1256 255
varchar(10) cp1256 11
varbinary(10) cp1256 11
enum('G','PG','PG-13','R','NC-17') cp1256 1
set('one', 'two') cp1256 1
not_exist cp1256 0
char(-1) cp1256 0
char(10) cp1257 10
char(256) cp1257 255
binary(10) cp1257 10
binary(256) cp1257 255
varchar(10) cp1257 11
varbinary(10) cp1257 11
enum('G','PG','PG-13','R','NC-17') cp1257 1
set('one', 'two') cp1257 1
not_exist cp1257 0
char(-1) cp1257 0
char(10) cp850 10
char(256) cp850 255
binary(10) cp850 10
binary(256) cp850 255
varchar(10) cp850 11
varbinary(10) cp850 11
enum('G','PG','PG-13','R','NC-17') cp850 1
set('one', 'two') cp850 1
not_exist cp850 0
char(-1) cp850 0
char(10) cp852 10
char(256) cp852 255
binary(10) cp852 10
binary(256) cp852 255
varchar(10) cp852 11
varbinary(10) cp852 11
enum('G','PG','PG-13','R','NC-17') cp852 1
set('one', 'two') cp852 1
not_exist cp852 0
char(-1) cp852 0
char(10) cp866 10
char(256) cp866 255
binary(10) cp866 10
binary(256) cp866 255
varchar(10) cp866 11
varbinary(10) cp866 11
enum('G','PG','PG-13','R','NC-17') cp866 1
set('one', 'two') cp866 1
not_exist cp866 0
char(-1) cp866 0
char(10) cp932 20
char(256) cp932 510
binary(10) cp932 10
binary(256) cp932 255
varchar(10) cp932 21
varbinary(10) cp932 21
enum('G','PG','PG-13','R','NC-17') cp932 1
set('one', 'two') cp932 1
not_exist cp932 0
char(-1) cp932 0
char(10) dec8 10
char(256) dec8 255
binary(10) dec8 10
binary(256) dec8 255
varchar(10) dec8 11
varbinary(10) dec8 11
enum('G','PG','PG-13','R','NC-17') dec8 1
set('one', 'two') dec8 1
not_exist dec8 0
char(-1) dec8 0
char(10) eucjpms 30
char(256) eucjpms 765
binary(10) eucjpms 10
binary(256) eucjpms 255
varchar(10) eucjpms 31
varbinary(10) eucjpms 31
enum('G','PG','PG-13','R','NC-17') eucjpms 1
set('one', 'two') eucjpms 1
not_exist eucjpms 0
char(-1) eucjpms 0
char(10) euckr 20
char(256) euckr 510
binary(10) euckr 10
binary(256) euckr 255
varchar(10) euckr 21
varbinary(10) euckr 21
enum('G','PG','PG-13','R','NC-17') euckr 1
set('one', 'two') euckr 1
not_exist euckr 0
char(-1) euckr 0
char(10) gb18030 40
char(256) gb18030 1020
binary(10) gb18030 10
binary(256) gb18030 255
varchar(10) gb18030 41
varbinary(10) gb18030 41
enum('G','PG','PG-13','R','NC-17') gb18030 1
set('one', 'two') gb18030 1
not_exist gb18030 0
char(-1) gb18030 0
char(10) gb2312 20
char(256) gb2312 510
binary(10) gb2312 10
binary(256) gb2312 255
varchar(10) gb2312 21
varbinary(10) gb2312 21
enum('G','PG','PG-13','R','NC-17') gb2312 1
set('one', 'two') gb2312 1
not_exist gb2312 0
char(-1) gb2312 0
char(10) gbk 20
char(256) gbk 510
binary(10) gbk 10
binary(256) gbk 255
varchar(10) gbk 21
varbinary(10) gbk 21
enum('G','PG','PG-13','R','NC-17') gbk 1
set('one', 'two') gbk 1
not_exist gbk 0
char(-1) gbk 0
char(10) geostd8 10
char(256) geostd8 255
binary(10) geostd8 10
binary(256) geostd8 255
varchar(10) geostd8 11
varbinary(10) geostd8 11
enum('G','PG','PG-13','R','NC-17') geostd8 1
set('one', 'two') geostd8 1
not_exist geostd8 0
char(-1) geostd8 0
char(10) greek 10
char(256) greek 255
binary(10) greek 10
binary(256) greek 255
varchar(10) greek 11
varbinary(10) greek 11
enum('G','PG','PG-13','R','NC-17') greek 1
set('one', 'two') greek 1
not_exist greek 0
char(-1) greek 0
char(10) hebrew 10
char(256) hebrew 255
binary(10) hebrew 10
binary(256) hebrew 255
varchar(10) hebrew 11
varbinary(10) hebrew 11
enum('G','PG','PG-13','R','NC-17') hebrew 1
set('one', 'two') hebrew 1
not_exist hebrew 0
char(-1) hebrew 0
char(10) hp8 10
char(256) hp8 255
binary(10) hp8 10
binary(256) hp8 255
varchar(10) hp8 11
varbinary(10) hp8 11
enum('G','PG','PG-13','R','NC-17') hp8 1
set('one', 'two') hp8 1
not_exist hp8 0
char(-1) hp8 0
char(10) keybcs2 10
char(256) keybcs2 255
binary(10) keybcs2 10
binary(256) keybcs2 255
varchar(10) keybcs2 11
varbinary(10) keybcs2 11
enum('G','PG','PG-13','R','NC-17') keybcs2 1
set('one', 'two') keybcs2 1
not_exist keybcs2 0
char(-1) keybcs2 0
char(10) koi8r 10
char(256) koi8r 255
binary(10) koi8r 10
binary(256) koi8r 255
varchar(10) koi8r 11
varbinary(10) koi8r 11
enum('G','PG','PG-13','R','NC-17') koi8r 1
set('one', 'two') koi8r 1
not_exist koi8r 0
char(-1) koi8r 0
char(10) koi8u 10
char(256) koi8u 255
binary(10) koi8u 10
binary(256) koi8u 255
varchar(10) koi8u 11
varbinary(10) koi8u 11
enum('G','PG','PG-13','R','NC-17') koi8u 1
set('one', 'two') koi8u 1
not_exist koi8u 0
char(-1) koi8u 0
char(10) latin1 10
char(256) latin1 255
binary(10) latin1 10
binary(256) latin1 255
varchar(10) latin1 11
varbinary(10) latin1 11
enum('G','PG','PG-13','R','NC-17') latin1 1
set('one', 'two') latin1 1
not_exist latin1 0
char(-1) latin1 0
char(10) latin2 10
char(256) latin2 255
binary(10) latin2 10
binary(256) latin2 255
varchar(10) latin2 11
varbinary(10) latin2 11
enum('G','PG','PG-13','R','NC-17') latin2 1
set('one', 'two') latin2 1
not_exist latin2 0
char(-1) latin2 0
char(10) latin5 10
char(256) latin5 255
binary(10) latin5 10
binary(256) latin5 255
varchar(10) latin5 11
varbinary(10) latin5 11
enum('G','PG','PG-13','R','NC-17') latin5 1
set('one', 'two') latin5 1
not_exist latin5 0
char(-1) latin5 0
char(10) latin7 10
char(256) latin7 255
binary(10) latin7 10
binary(256) latin7 255
varchar(10) latin7 11
varbinary(10) latin7 11
enum('G','PG','PG-13','R','NC-17') latin7 1
set('one', 'two') latin7 1
not_exist latin7 0
char(-1) latin7 0
char(10) macce 10
char(256) macce 255
binary(10) macce 10
binary(256) macce 255
varchar(10) macce 11
varbinary(10) macce 11
enum('G','PG','PG-13','R','NC-17') macce 1
set('one', 'two') macce 1
not_exist macce 0
char(-1) macce 0
char(10) macroman 10
char(256) macroman 255
binary(10) macroman 10
binary(256) macroman 255
varchar(10) macroman 11
varbinary(10) macroman 11
enum('G','PG','PG-13','R','NC-17') macroman 1
set('one', 'two') macroman 1
not_exist macroman 0
char(-1) macroman 0
char(10) sjis 20
char(256) sjis 510
binary(10) sjis 10
binary(256) sjis 255
varchar(10) sjis 21
varbinary(10) sjis 21
enum('G','PG','PG-13','R','NC-17') sjis 1
set('one', 'two') sjis 1
not_exist sjis 0
char(-1) sjis 0
char(10) swe7 10
char(256) swe7 255
binary(10) swe7 10
binary(256) swe7 255
varchar(10) swe7 11
varbinary(10) swe7 11
enum('G','PG','PG-13','R','NC-17') swe7 1
set('one', 'two') swe7 1
not_exist swe7 0
char(-1) swe7 0
char(10) tis620 10
char(256) tis620 255
binary(10) tis620 10
binary(256) tis620 255
varchar(10) tis620 11
varbinary(10) tis620 11
enum('G','PG','PG-13','R','NC-17') tis620 1
set('one', 'two') tis620 1
not_exist tis620 0
char(-1) tis620 0
char(10) ucs2 20
char(256) ucs2 510
binary(10) ucs2 10
binary(256) ucs2 255
varchar(10) ucs2 21
varbinary(10) ucs2 21
enum('G','PG','PG-13','R','NC-17') ucs2 1
set('one', 'two') ucs2 1
not_exist ucs2 0
char(-1) ucs2 0
char(10) ujis 30
char(256) ujis 765
binary(10) ujis 10
binary(256) ujis 255
varchar(10) ujis 31
varbinary(10) ujis 31
enum('G','PG','PG-13','R','NC-17') ujis 1
set('one', 'two') ujis 1
not_exist ujis 0
char(-1) ujis 0
char(10) utf16 40
char(256) utf16 1020
binary(10) utf16 10
binary(256) utf16 255
varchar(10) utf16 41
varbinary(10) utf16 41
enum('G','PG','PG-13','R','NC-17') utf16 1
set('one', 'two') utf16 1
not_exist utf16 0
char(-1) utf16 0
char(10) utf16le 40
char(256) utf16le 1020
binary(10) utf16le 10
binary(256) utf16le 255
varchar(10) utf16le 41
varbinary(10) utf16le 41
enum('G','PG','PG-13','R','NC-17') utf16le 1
set('one', 'two') utf16le 1
not_exist utf16le 0
char(-1) utf16le 0
char(10) utf32 40
char(256) utf32 1020
binary(10) utf32 10
binary(256) utf32 255
varchar(10) utf32 41
varbinary(10) utf32 41
enum('G','PG','PG-13','R','NC-17') utf32 1
set('one', 'two') utf32 1
not_exist utf32 0
char(-1) utf32 0
char(10) utf8 30
char(256) utf8 765
binary(10) utf8 10
binary(256) utf8 255
varchar(10) utf8 31
varbinary(10) utf8 31
enum('G','PG','PG-13','R','NC-17') utf8 1
set('one', 'two') utf8 1
not_exist utf8 0
char(-1) utf8 0
char(10) utf8mb4 40
char(256) utf8mb4 1020
binary(10) utf8mb4 10
binary(256) utf8mb4 255
varchar(10) utf8mb4 41
varbinary(10) utf8mb4 41
enum('G','PG','PG-13','R','NC-17') utf8mb4 1
set('one', 'two') utf8mb4 1
not_exist utf8mb4 0
char(-1) utf8mb4 0
================================================
FILE: common/testdata/UTF-8.bom.sql
================================================
select col from tb c = 1;
================================================
FILE: common/testdata/chardet_BIG5.txt
================================================
================================================
FILE: common/testdata/chardet_GB-18030.txt
================================================
================================================
FILE: common/testdata/chardet_UTF-8.txt
================================================
中文
================================================
FILE: common/tricks.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"bufio"
"bytes"
"fmt"
"io/ioutil"
"os"
"path/filepath"
"reflect"
"sort"
"github.com/tidwall/gjson"
)
// GoldenDiff 从 gofmt 学来的测试方法
// https://medium.com/soon-london/testing-with-golden-files-in-go-7fccc71c43d3
func GoldenDiff(f func(), name string, update *bool) error {
var b bytes.Buffer
w := bufio.NewWriter(&b)
str := captureOutput(f)
_, err := w.WriteString(str)
if err != nil {
Log.Warning(err.Error())
}
err = w.Flush()
if err != nil {
Log.Warning(err.Error())
}
gp := filepath.Join("testdata", name+".golden")
if *update {
if err = ioutil.WriteFile(gp, b.Bytes(), 0644); err != nil {
err = fmt.Errorf("%s failed to update golden file: %s", name, err)
return err
}
}
g, err := ioutil.ReadFile(gp)
if err != nil {
err = fmt.Errorf("%s failed reading .golden: %s", name, err)
}
if !bytes.Equal(b.Bytes(), g) {
err = fmt.Errorf("%s does not match .golden file", name)
}
return err
}
// captureOutput 获取函数标准输出
func captureOutput(f func()) string {
// keep backup of the real stdout
oldStdout := os.Stdout
r, w, _ := os.Pipe()
os.Stdout = w
// copy the output in a separate goroutine so printing can't block indefinitely
outC := make(chan string)
go func() {
buf, err := ioutil.ReadAll(r)
if err != nil {
panic(err)
}
outC <- string(buf)
}()
// execute function
f()
// back to normal state
if err := w.Close(); err != nil {
panic(err)
}
os.Stdout = oldStdout // restoring the real stdout
out := <-outC
os.Stdout = oldStdout
return out
}
// SortedKey sort map[string]interface{}, use in range clause
func SortedKey(m interface{}) []string {
var keys []string
switch reflect.TypeOf(m).Kind() {
case reflect.Map:
switch reflect.TypeOf(m).Key().Kind() {
case reflect.String:
for _, k := range reflect.ValueOf(m).MapKeys() {
keys = append(keys, k.String())
}
}
}
sort.Strings(keys)
return keys
}
// jsonFind internal function
func jsonFind(json string, name string, find *[]string) (next []string) {
res := gjson.Parse(json)
res.ForEach(func(key, value gjson.Result) bool {
if key.String() == name {
*find = append(*find, value.String())
}
switch value.Type {
case gjson.Number, gjson.True, gjson.False, gjson.Null:
default:
// String, JSON
next = append(next, value.String())
}
return true // keep iterating
})
return next
}
// JSONFind iterate find name in json
// TODO: for complicate SQL JSONFind will run a long time for json interactions.
func JSONFind(json string, name string) []string {
Log.Debug("Entering function: %s", GetFunctionName())
var find []string
next := []string{json}
for len(next) > 0 {
var tmpNext []string
for _, subJSON := range next {
tmpNext = append(tmpNext, jsonFind(subJSON, name, &find)...)
}
next = tmpNext
}
Log.Debug("Exiting function: %s", GetFunctionName())
return find
}
// RemoveDuplicatesItem remove duplicate item from list
func RemoveDuplicatesItem(duplicate []string) []string {
m := make(map[string]bool)
for _, item := range duplicate {
if _, ok := m[item]; !ok {
m[item] = true
}
}
var unique []string
for item := range m {
unique = append(unique, item)
}
sort.Strings(unique)
return unique
}
================================================
FILE: common/tricks_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package common
import (
"fmt"
"strings"
"testing"
"time"
)
func TestCaptureOutput(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
c1 := make(chan string, 1)
// test output buf large than 65535
length := 1<<16 + 1
go func() {
str := captureOutput(
func() {
var str []string
for i := 0; i < length; i++ {
str = append(str, "a")
}
fmt.Println(strings.Join(str, ""))
},
)
c1 <- str
}()
select {
case res := <-c1:
if len(res) <= length {
t.Errorf("want %d, got %d", length, len(res))
}
case <-time.After(1 * time.Second):
t.Error("capture timeout, pipe read hangup")
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestJSONFind(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
jsons := []string{
`
{
"Collate":{
"Collate":{
"Collate":{
"key":"value"
}
}
}
}
`,
`{
"programmers": [
{
"firstName": "Janet",
"Collate": "McLaughlin",
}, {
"firstName": "Elliotte",
"Collate": "Hunter",
}, {
"firstName": "Jason",
"Collate": "Harold",
}
]
}`,
`
{
"widget": {
"debug": "on",
"Collate": {
"title": "Sample Konfabulator Widget",
"name": "main_window",
"width": 500,
"height": 500
},
"image": {
"src": "Images/Sun.png",
"hOffset": 250,
"vOffset": 250,
"alignment": "center"
},
"text": {
"data": "Click Here",
"size": 36,
"style": "bold",
"vOffset": 100,
"alignment": "center",
"onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;"
}
}
}
`,
`
[
{
"SQLCache": true,
"CalcFoundRows": false,
"StraightJoin": false,
"Priority": 0,
"Distinct": false,
"From": {
"TableRefs": {
"Left": {
"Source": {
"Schema": {
"O": "",
"L": ""
},
"Name": {
"O": "tb",
"L": "tb"
},
"DBInfo": null,
"TableInfo": null,
"IndexHints": null
},
"AsName": {
"O": "",
"L": ""
}
},
"Right": null,
"Tp": 0,
"On": null,
"Using": null,
"NaturalJoin": false,
"StraightJoin": false
}
},
"Where": {
"Type": {
"Tp": 0,
"Flag": 0,
"Flen": 0,
"Decimal": 0,
"Charset": "",
"Collate": "",
"Elems": null
},
"Op": 4,
"L": {
"Type": {
"Tp": 0,
"Flag": 0,
"Flen": 0,
"Decimal": 0,
"Charset": "",
"Collate": "",
"Elems": null
},
"Op": 7,
"L": {
"Type": {
"Tp": 0,
"Flag": 0,
"Flen": 0,
"Decimal": 0,
"Charset": "",
"Collate": "",
"Elems": null
},
"Name": {
"Schema": {
"O": "",
"L": ""
},
"Table": {
"O": "",
"L": ""
},
"Name": {
"O": "col3",
"L": "col3"
}
},
"Refer": null
},
"R": {
"Type": {
"Tp": 8,
"Flag": 128,
"Flen": 1,
"Decimal": 0,
"Charset": "binary",
"Collate": "binary",
"Elems": null
}
}
},
"R": {
"Type": {
"Tp": 0,
"Flag": 0,
"Flen": 0,
"Decimal": 0,
"Charset": "",
"Collate": "",
"Elems": null
},
"Op": 1,
"L": {
"Type": {
"Tp": 0,
"Flag": 0,
"Flen": 0,
"Decimal": 0,
"Charset": "",
"Collate": "",
"Elems": null
},
"Op": 7,
"L": {
"Type": {
"Tp": 0,
"Flag": 0,
"Flen": 0,
"Decimal": 0,
"Charset": "",
"Collate": "",
"Elems": null
},
"Name": {
"Schema": {
"O": "",
"L": ""
},
"Table": {
"O": "",
"L": ""
},
"Name": {
"O": "col3",
"L": "col3"
}
},
"Refer": null
},
"R": {
"Type": {
"Tp": 8,
"Flag": 128,
"Flen": 1,
"Decimal": 0,
"Charset": "binary",
"Collate": "binary",
"Elems": null
}
}
},
"R": {
"Type": {
"Tp": 0,
"Flag": 0,
"Flen": 0,
"Decimal": 0,
"Charset": "",
"Collate": "",
"Elems": null
},
"Op": 7,
"L": {
"Type": {
"Tp": 0,
"Flag": 0,
"Flen": 0,
"Decimal": 0,
"Charset": "",
"Collate": "",
"Elems": null
},
"Op": 7,
"L": {
"Type": {
"Tp": 0,
"Flag": 0,
"Flen": 0,
"Decimal": 0,
"Charset": "",
"Collate": "",
"Elems": null
},
"Name": {
"Schema": {
"O": "",
"L": ""
},
"Table": {
"O": "",
"L": ""
},
"Name": {
"O": "col1",
"L": "col1"
}
},
"Refer": null
},
"R": {
"Type": {
"Tp": 0,
"Flag": 0,
"Flen": 0,
"Decimal": 0,
"Charset": "",
"Collate": "",
"Elems": null
},
"Name": {
"Schema": {
"O": "",
"L": ""
},
"Table": {
"O": "",
"L": ""
},
"Name": {
"O": "col2",
"L": "col2"
}
},
"Refer": null
}
},
"R": {
"Type": {
"Tp": 253,
"Flag": 0,
"Flen": 3,
"Decimal": -1,
"Charset": "utf8mb4",
"Collate": "utf8mb4_bin",
"Elems": null
}
}
}
}
},
"Fields": {
"Fields": [
{
"Offset": 7,
"WildCard": {
"Table": {
"O": "",
"L": ""
},
"Schema": {
"O": "",
"L": ""
}
},
"Expr": null,
"AsName": {
"O": "",
"L": ""
},
"Auxiliary": false
}
]
},
"GroupBy": null,
"Having": null,
"WindowSpecs": null,
"OrderBy": null,
"Limit": null,
"LockTp": 0,
"TableHints": null,
"IsAfterUnionDistinct": false,
"IsInBraces": false
}
]
`,
}
err := GoldenDiff(func() {
for _, json := range jsons {
result := JSONFind(json, "Collate")
fmt.Println(result)
}
}, t.Name(), update)
if err != nil {
t.Error(err)
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
func TestRemoveDuplicatesItem(t *testing.T) {
Log.Debug("Entering function: %s", GetFunctionName())
unique := RemoveDuplicatesItem([]string{"a", "a", "b", "c"})
if len(unique) != 3 {
t.Error("string list length should 3", unique)
}
Log.Debug("Exiting function: %s", GetFunctionName())
}
================================================
FILE: common/version.go
================================================
package common
// -version输出信息
var (
Version = "No Version Provided"
Compile = ""
Branch = ""
GitDirty = ""
DevPath = ""
)
================================================
FILE: database/doc.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
// Package database will take cover of communicate with mysql database.
package database
================================================
FILE: database/explain.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package database
import (
"encoding/json"
"errors"
"fmt"
"regexp"
"runtime"
"sort"
"strconv"
"strings"
"github.com/XiaoMi/soar/ast"
"github.com/XiaoMi/soar/common"
tidb "github.com/pingcap/parser/ast"
"github.com/tidwall/gjson"
"vitess.io/vitess/go/vt/sqlparser"
)
// format_type 支持的输出格式
// https://dev.mysql.com/doc/refman/5.7/en/explain-output.html
const (
TraditionalFormatExplain = iota // 默认输出
JSONFormatExplain // JSON格式输出
)
// ExplainFormatType EXPLAIN 支持的 FORMAT_TYPE
var ExplainFormatType = map[string]int{
"traditional": 0,
"json": 1,
}
// explain_type
const (
TraditionalExplainType = iota // 默认转出
ExtendedExplainType // EXTENDED输出
PartitionsExplainType // PARTITIONS输出
)
// ExplainType EXPLAIN命令支持的参数
var ExplainType = map[string]int{
"traditional": 0,
"extended": 1,
"partitions": 2,
}
// 为TraditionalFormatExplain准备的结构体 { start
// ExplainInfo 用于存放Explain信息
type ExplainInfo struct {
SQL string
ExplainFormat int
ExplainRows []ExplainRow
ExplainJSON *ExplainJSON
Warnings []ExplainWarning
QueryCost float64
}
// ExplainRow 单行Explain
type ExplainRow struct {
ID int
SelectType string
TableName string
Partitions string // explain partitions
AccessType string
PossibleKeys []string
Key string
KeyLen string // 索引长度，如果发生了index_merge， KeyLen 格式为 N,N，所以不能定义为整型
Ref []string
Rows int64
Filtered float64 // 5.6 JSON, 5.7+, 5.5 EXTENDED
Scalability string // O(1), O(n), O(log n), O(log n)+
Extra string
}
// ExplainWarning explain extended 后 SHOW WARNINGS 输出的结果
type ExplainWarning struct {
Level string
Code int
Message string
}
// 为TraditionalFormatExplain准备的结构体 end }
// 为JSONFormatExplain准备的结构体 { start
// ExplainJSONCostInfo JSON
type ExplainJSONCostInfo struct {
ReadCost string `json:"read_cost"`
EvalCost string `json:"eval_cost"`
PrefixCost string `json:"prefix_cost"`
DataReadPerJoin string `json:"data_read_per_join"`
QueryCost string `json:"query_cost"`
SortCost string `json:"sort_cost"`
}
// ExplainJSONMaterializedFromSubquery JSON
type ExplainJSONMaterializedFromSubquery struct {
UsingTemporaryTable bool `json:"using_temporary_table"`
Dependent bool `json:"dependent"`
Cacheable bool `json:"cacheable"`
QueryBlock *ExplainJSONQueryBlock `json:"query_block"`
}
// 该变量用于存放 JSON 到 Traditional 模式的所有 ExplainJSONTable
var explainJSONTables []*ExplainJSONTable
// ExplainJSONTable JSON
type ExplainJSONTable struct {
TableName string `json:"table_name"`
AccessType string `json:"access_type"`
PossibleKeys []string `json:"possible_keys"`
Key string `json:"key"`
UsedKeyParts []string `json:"used_key_parts"`
KeyLength string `json:"key_length"`
Ref []string `json:"ref"`
RowsExaminedPerScan int64 `json:"rows_examined_per_scan"`
RowsProducedPerJoin int `json:"rows_produced_per_join"`
Filtered string `json:"filtered"`
UsingIndex bool `json:"using_index"`
UsingIndexForGroupBy bool `json:"using_index_for_group_by"`
CostInfo ExplainJSONCostInfo `json:"cost_info"`
UsedColumns []string `json:"used_columns"`
AttachedCondition string `json:"attached_condition"`
AttachedSubqueries []ExplainJSONSubqueries `json:"attached_subqueries"`
MaterializedFromSubquery ExplainJSONMaterializedFromSubquery `json:"materialized_from_subquery"`
}
// ExplainJSONNestedLoop JSON
type ExplainJSONNestedLoop struct {
Table ExplainJSONTable `json:"table"`
}
// ExplainJSONBufferResult JSON
type ExplainJSONBufferResult struct {
UsingTemporaryTable bool `json:"using_temporary_table"`
NestedLoop []ExplainJSONNestedLoop `json:"nested_loop"`
Table ExplainJSONTable `json:"table"`
}
// ExplainJSONSubqueries JSON
type ExplainJSONSubqueries struct {
Dependent bool `json:"dependent"`
Cacheable bool `json:"cacheable"`
QueryBlock ExplainJSONQueryBlock `json:"query_block"`
}
// ExplainJSONGroupingOperation JSON
type ExplainJSONGroupingOperation struct {
UsingTemporaryTable bool `json:"using_temporary_table"`
UsingFilesort bool `json:"using_filesort"`
Table ExplainJSONTable `json:"table"`
CostInfo ExplainJSONCostInfo `json:"cost_info"`
NestedLoop []ExplainJSONNestedLoop `json:"nested_loop"`
GroupBySubqueries []ExplainJSONSubqueries `json:"group_by_subqueries"`
}
// ExplainJSONDuplicatesRemoval JSON
type ExplainJSONDuplicatesRemoval struct {
UsingTemporaryTable bool `json:"using_temporary_table"`
UsingFilesort bool `json:"using_filesort"`
BufferResult ExplainJSONBufferResult `json:"buffer_result"`
GroupingOperation ExplainJSONGroupingOperation `json:"grouping_operation"`
Table ExplainJSONTable `json:"table"`
}
// ExplainJSONOrderingOperation JSON
type ExplainJSONOrderingOperation struct {
UsingFilesort bool `json:"using_filesort"`
Table ExplainJSONTable `json:"table"`
DuplicatesRemoval ExplainJSONDuplicatesRemoval `json:"duplicates_removal"`
GroupingOperation ExplainJSONGroupingOperation `json:"grouping_operation"`
OrderbySubqueries []ExplainJSONSubqueries `json:"order_by_subqueries"`
OptimizedAwaySubqueries []ExplainJSONSubqueries `json:"optimized_away_subqueries"`
}
// ExplainJSONQueryBlock JSON
type ExplainJSONQueryBlock struct {
SelectID int `json:"select_id"`
CostInfo ExplainJSONCostInfo `json:"cost_info"`
Table ExplainJSONTable `json:"table"`
NestedLoop []ExplainJSONNestedLoop `json:"nested_loop"`
OrderingOperation ExplainJSONOrderingOperation `json:"ordering_operation"`
GroupingOperation ExplainJSONGroupingOperation `json:"grouping_operation"`
OptimizedAwaySubqueries []ExplainJSONSubqueries `json:"optimized_away_subqueries"`
HavingSubqueries []ExplainJSONSubqueries `json:"having_subqueries"`
SelectListSubqueries []ExplainJSONSubqueries `json:"select_list_subqueries"`
UpdateValueSubqueries []ExplainJSONSubqueries `json:"update_value_subqueries"`
QuerySpecifications []ExplainJSONSubqueries `json:"query_specifications"`
UnionResult ExplainJSONUnionResult `json:"union_result"`
Message string `json:"message"`
}
// ExplainJSONUnionResult JSON
type ExplainJSONUnionResult struct {
UsingTemporaryTable bool `json:"using_temporary_table"`
TableName string `json:"table_name"`
AccessType string `json:"access_type"`
QuerySpecifications []ExplainJSONSubqueries `json:"query_specifications"`
}
// ExplainJSON 根结点
type ExplainJSON struct {
QueryBlock ExplainJSONQueryBlock `json:"query_block"`
}
// 为JSONFormatExplain准备的结构体 end }
// ExplainKeyWords 需要解释的关键字
var ExplainKeyWords = []string{
"access_type",
"attached_condition",
"attached_subqueries",
"buffer_result",
"cacheable",
"cost_info",
"data_read_per_join",
"dependent",
"duplicates_removal",
"eval_cost",
"filtered",
"group_by_subqueries",
"grouping_operation",
"having_subqueries",
"key",
"key_length",
"materialized_from_subquery",
"message",
"nested_loop",
"optimized_away_subqueries",
"order_by_subqueries",
"ordering_operation",
"possible_keys",
"prefix_cost",
"query_block",
"query_cost",
"query_specifications",
"read_cost",
"ref",
"rows_examined_per_scan",
"rows_produced_per_join",
"select_id",
"select_list_subqueries",
"sort_cost",
"table",
"table_name",
"union_result",
"update_value_subqueries",
"used_columns",
"used_key_parts",
"using_filesort",
"using_index",
"using_index_for_group_by",
"using_temporary_table",
}
/*
// ExplainColumnIndent EXPLAIN表头
var ExplainColumnIndent = map[string]string{
"id": "id为SELECT的标识符. 它是在SELECT查询中的顺序编号. 如果这一行表示其他行的union结果, 这个值可以为空. 在这种情况下, table列会显示为形如
")), "|",
common.MarkdownEscape(row.Key), "|",
row.KeyLen, "|",
common.MarkdownEscape(strings.Join(row.Ref, ",
")),
"|", rows, "|",
filtered, "|", scalability, "|",
strings.Replace(common.MarkdownEscape(row.Extra), ",", ",
", -1),
"|"))
}
}
buf = append(buf, "\n")
return strings.Join(buf, "")
}
================================================
FILE: database/explain_test.go
================================================
/*
* Copyright 2018 Xiaomi, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package database
import (
"fmt"
"testing"
"github.com/XiaoMi/soar/common"
"github.com/kr/pretty"
)
var sqls = []string{
`use sakila`, // not explain able sql, will convert to empty!
`select * from city where country_id = 44;`,
`select * from address where address2 is not null;`,
`select * from address where address2 is null;`,
`select * from address where address2 >= 44;`,
`select * from city where country_id between 44 and 107;`,
`select * from city where city like 'Ad%';`,
`select * from city where city = 'Aden' and country_id = 107;`,
`select * from city where country_id > 31 and city = 'Aden';`,
`select * from address where address_id > 8 and city_id < 400 and district = 'Nantou';`,
`select * from address where address_id > 8 and city_id < 400;`,
`select first_name from actor where last_update='2006-02-15 04:34:33' and last_name='CHASE' group by first_name;`,
`select district from address where last_update >='2014-09-25 22:33:47' group by district;`,
`select address from address group by address,district;`,
`select district from address where last_update='2014-09-25 22:30:27' group by district,(address_id+city_id);`,
`select * from customer where active=1 order by last_name limit 10;`,
`select * from customer order by last_name limit 10;`,
`select * from customer where address_id > 224 order by address_id limit 10;`,
`select * from customer where address_id < 224 order by address_id limit 10;`,
`select * from customer where active=1 order by last_name;`,
`select * from customer where address_id > 224 order by address_id;`,
`select * from customer where address_id in (224,510) order by last_name;`,
`select city from city where country_id = 44;`,
`select city,city_id from city where country_id = 44 and last_update='2006-02-15 04:45:25';`,
`select city from city where country_id > 44 and last_update > '2006-02-15 04:45:25';`,
`select * from city where country_id=1 and city='Kabul' order by last_update;`,
`select * from city where country_id>1 and city='Kabul' order by last_update;`,
`select * from city where city_id>251 order by last_update;`,
`select * from city i inner join country o on i.country_id=o.country_id;`,
`select * from city i left join country o on i.city_id=o.country_id;`,
`select * from city i right join country o on i.city_id=o.country_id;`,
`select * from city i left join country o on i.city_id=o.country_id where o.country_id is null;`,
`select * from city i right join country o on i.city_id=o.country_id where i.city_id is null;`,
`select * from city i left join country o on i.city_id=o.country_id union select * from city i right join country o on i.city_id=o.country_id;`,
`select * from city i left join country o on i.city_id=o.country_id where o.country_id is null union select * from city i right join country o on i.city_id=o.country_id where i.city_id is null;`,
`select first_name,last_name,email from customer natural left join address;`,
`select first_name,last_name,email from customer natural left join address;`,
`select first_name,last_name,email from customer natural right join address;`,
`select first_name,last_name,email from customer STRAIGHT_JOIN address on customer.address_id=address.address_id;`,
`select ID,name from (select address from customer_list where SID=1 order by phone limit 50,10) a join customer_list l on (a.address=l.address) join city c on (c.city=l.city) order by phone desc;`,
`SELECT a.table_name 表名, a.table_comment 表说明, b.COLUMN_NAME 字段名, b.column_comment 字段说明, b.column_type 字段类型, b.column_key 约束 FROM information_schema.TABLES a LEFT JOIN information_schema. COLUMNS b ON a.table_name = b.TABLE_NAME WHERE a.table_schema IN ('a') AND b.column_comment LIKE '%一%' ORDER BY a.table_name`,
}
var exp = []string{
`+----+-------------+---------+-------+---------------------------------------------------------+-------------------+---------+---------------------------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------+-------+---------------------------------------------------------+-------------------+---------+---------------------------+------+-------------+
| 1 | SIMPLE | country | index | PRIMARY,country_id | country | 152 | NULL | 109 | Using index |
| 1 | SIMPLE | city | ref | idx_fk_country_id,idx_country_id_city,idx_all,idx_other | idx_fk_country_id | 2 | sakila.country.country_id | 2 | Using index |
+----+-------------+---------+-------+---------------------------------------------------------+-------------------+---------+---------------------------+------+-------------+`,
`+----+-------------+---------+------------+-------+-------------------+-------------------+---------+---------------------------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+---------+------------+-------+-------------------+-------------------+---------+---------------------------+------+----------+-------------+
| 1 | SIMPLE | country | NULL | index | PRIMARY | PRIMARY | 2 | NULL | 109 | 100.00 | Using index |
| 1 | SIMPLE | city | NULL | ref | idx_fk_country_id | idx_fk_country_id | 2 | sakila.country.country_id | 5 | 100.00 | Using index |
+----+-------------+---------+------------+-------+-------------------+-------------------+---------+---------------------------+------+----------+-------------+`,
`*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: country
type: index
possible_keys: PRIMARY,country_id
key: country
key_len: 152
ref: NULL
rows: 109
Extra: Using index
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: city
type: ref
possible_keys: idx_fk_country_id,idx_country_id_city,idx_all,idx_other
key: idx_fk_country_id
key_len: 2
ref: sakila.country.country_id
rows: 2
Extra: Using index`,
`+----+-------------+---------+------------+-------+---------------------------------------------------------+-------------------+---------+---------------------------+------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------+------------+-------+---------------------------------------------------------+-------------------+---------+---------------------------+------+-------------+
| 1 | SIMPLE | country | NULL | index | PRIMARY,country_id | country | 152 | NULL | 109 | Using index |
| 1 | SIMPLE | city | NULL | ref | idx_fk_country_id,idx_country_id_city,idx_all,idx_other | idx_fk_country_id | 2 | sakila.country.country_id | 2 | Using index |
+----+-------------+---------+------------+-------+---------------------------------------------------------+-------------------+---------+---------------------------+------+-------------+`,
`{
"query_block": {
"select_id": 1,
"message": "No tables used"
} /* query_block */
}`,
`{
"query_block": {
"select_id": 1,
"message": "no matching row in const table"
}
}`,
`{
"query_block": {
"select_id": 1,
"table": {
"insert": true,
"table_name": "t1",
"access_type": "ALL"
} /* table */
} /* query_block */
}`,
`{
"query_block": {
"select_id": 1,
"message": "no matching row in const table"
} /* query_block */
}`,
`{
"query_block": {
"select_id": 1,
"message": "no matching row in const table"
} /* query_block */
}`,
`{
"query_block": {
"select_id": 1,
"cost_info": {
"query_cost": "13.50"
} /* cost_info */,
"table": {
"table_name": "a4",
"access_type": "ALL",
"rows_examined_per_scan": 14,
"rows_produced_per_join": 14,
"filtered": "100.00",
"cost_info": {
"read_cost": "10.70",
"eval_cost": "2.80",
"prefix_cost": "13.50",
"data_read_per_join": "224"
} /* cost_info */,
"used_columns": [
"i"
] /* used_columns */,
"materialized_from_subquery": {
"using_temporary_table": true,
"dependent": false,
"cacheable": true,
"query_block": {
"select_id": 2,
"cost_info": {
"query_cost": "13.50"
} /* cost_info */,
"table": {
"table_name": "a3",
"access_type": "ALL",
"rows_examined_per_scan": 14,
"rows_produced_per_join": 14,
"filtered": "100.00",
"cost_info": {
"read_cost": "10.70",
"eval_cost": "2.80",
"prefix_cost": "13.50",
"data_read_per_join": "224"
} /* cost_info */,
"used_columns": [
"i"
] /* used_columns */,
"materialized_from_subquery": {
"using_temporary_table": true,
"dependent": false,
"cacheable": true,
"query_block": {
"select_id": 3,
"cost_info": {
"query_cost": "13.50"
} /* cost_info */,
"table": {
"table_name": "a2",
"access_type": "ALL",
"rows_examined_per_scan": 14,
"rows_produced_per_join": 14,
"filtered": "100.00",
"cost_info": {
"read_cost": "10.70",
"eval_cost": "2.80",
"prefix_cost": "13.50",
"data_read_per_join": "224"
} /* cost_info */,
"used_columns": [
"i"
] /* used_columns */,
"materialized_from_subquery": {
"using_temporary_table": true,
"dependent": false,
"cacheable": true,
"query_block": {
"select_id": 4,
"cost_info": {
"query_cost": "15.55"
} /* cost_info */,
"nested_loop": [
{
"table": {
"table_name": "t2",
"access_type": "ALL",
"rows_examined_per_scan": 2,
"rows_produced_per_join": 2,
"filtered": "100.00",
"cost_info": {
"read_cost": "2.00",
"eval_cost": "0.40",
"prefix_cost": "2.40",
"data_read_per_join": "16"
} /* cost_info */
} /* table */
},
{
"table": {
"table_name": "a1",
"access_type": "ALL",
"rows_examined_per_scan": 7,
"rows_produced_per_join": 14,
"filtered": "100.00",
"using_join_buffer": "Block Nested Loop",
"cost_info": {
"read_cost": "10.35",
"eval_cost": "2.80",
"prefix_cost": "15.55",
"data_read_per_join": "224"
} /* cost_info */,
"used_columns": [
"i"
] /* used_columns */,
"materialized_from_subquery": {
"using_temporary_table": true,
"dependent": false,
"cacheable": true,
"query_block": {
"select_id": 5,
"cost_info": {
"query_cost": "3.41"
} /* cost_info */,
"table": {
"table_name": "t1",
"access_type": "ALL",
"rows_examined_per_scan": 7,
"rows_produced_per_join": 7,
"filtered": "100.00",
"cost_info": {
"read_cost": "2.01",
"eval_cost": "1.40",
"prefix_cost": "3.41",
"data_read_per_join": "56"
} /* cost_info */,
"used_columns": [
"i"
] /* used_columns */
} /* table */
} /* query_block */
} /* materialized_from_subquery */
} /* table */
}
] /* nested_loop */
} /* query_block */
} /* materialized_from_subquery */
} /* table */
} /* query_block */
} /* materialized_from_subquery */
} /* table */
} /* query_block */
} /* materialized_from_subquery */
} /* table */
} /* query_block */
}`,
`{
"query_block": {
"select_id": 1,
"cost_info": {
"query_cost": "5.81"
} /* cost_info */,
"nested_loop": [
{
"table": {
"table_name": "t1",
"access_type": "ALL",
"rows_examined_per_scan": 7,
"rows_produced_per_join": 0,
"filtered": "14.29",
"cost_info": {
"read_cost": "3.21",
"eval_cost": "0.20",
"prefix_cost": "3.41",
"data_read_per_join": "7"
} /* cost_info */,
"used_columns": [
"i"
] /* used_columns */,
"attached_condition": "(test.t1.i = 10)"
} /* table */
},
{
"table": {
"table_name": "t2",
"access_type": "ALL",
"rows_examined_per_scan": 2,
"rows_produced_per_join": 0,
"filtered": "50.00",
"first_match": "t1",
"using_join_buffer": "Block Nested Loop",
"cost_info": {
"read_cost": "2.20",
"eval_cost": "0.20",
"prefix_cost": "5.82",
"data_read_per_join": "7"
} /* cost_info */,
"used_columns": [
"i"
] /* used_columns */,
"attached_condition": "(test.t2.i = 10)"
} /* table */
}
] /* nested_loop */
} /* query_block */
}`,
`{
"query_block": {
"select_id": 1,
"cost_info": {
"query_cost": "3.41"
} /* cost_info */,
"table": {
"table_name": "t1",
"access_type": "ALL",
"rows_examined_per_scan": 7,
"rows_produced_per_join": 7,
"filtered": "100.00",
"cost_info": {
"read_cost": "2.01",
"eval_cost": "1.40",
"prefix_cost": "3.41",
"data_read_per_join": "56"
} /* cost_info */,
"used_columns": [
"i"
] /* used_columns */,
"attached_condition": "(Query: 687D590364E29465
select * from film
最外层 SELECT 未指定 WHERE 条件
不建议使用 SELECT * 类型查询
================================================
FILE: test/fixture/test_Check_soar_report_for_json.golden
================================================
[
{
"ID": "687D590364E29465",
"Fingerprint": "select * from film",
"Score": 75,
"Sample": "select * from film",
"Explain": null,
"HeuristicRules": [
{
"Item": "CLA.001",
"Severity": "L4",
"Summary": "最外层 SELECT 未指定 WHERE 条件",
"Content": "SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。",
"Case": "select id from tbl",
"Position": 0
},
{
"Item": "COL.001",
"Severity": "L1",
"Summary": "不建议使用 SELECT * 类型查询",
"Content": "当表结构变更时，使用 * 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。",
"Case": "select * from tbl where id=1",
"Position": 0
}
],
"IndexRules": null,
"Tables": [
"`information_schema`.`film`"
]
}
]
================================================
FILE: test/fixture/test_Check_soar_report_for_markdown.golden
================================================
# Query: 687D590364E29465
★ ★ ★ ☆ ☆ 75分
```sql
SELECT
*
FROM
film
```
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
================================================
FILE: test/fixture/test_Check_the_default_config_of_the_changes.golden
================================================
online-dsn:
user: online-test
password: '********'
net: tcp
addr: 192.168.12.200:3307
schema: information_schema
charset: utf8mb4
collation: utf8mb4_general_ci
loc: GMT
tls: dsfsfdsf
server-public-key: AAAAB3NzaC1yc2EAAAADAQABAAABAQC0JFhoEgrl5/51UHlIIlfWwhiJLR/EEeF8enGnY0PnAldLQ8STdWD8Um2BUtVjgE9COl1X3mN4vMvnSm8A6aPn66enHn0hKzwh1GvcuZNTPgeaZyGKWG0kcvbziUjAPsxxvRvvoaUspSkRYAP/9vpq3SImJKuIHCPfjnGMurKV1n7W/QfpmNjUEwYOswDjL1Ik6Jy6Lrzf8T0hQEy+dYoV4zNM0HcROCXFuu1LyG+WTch3FW660BecNT8+c4sVTHuUMXgGot8OUtwgfXrt5ZL5as7cuyKiWsLVrtrtvL3T0ZHlV8qxQ3DT1gqjSw6jBOzyDOx3jwthAbdsWjyK4Oqp
max-allowed-packet: 419437
params:
charset: utf8mb4
timeout: 60s
read-timeout: 70s
write-timeout: 80s
allow-native-passwords: false
allow-old-passwords: true
disable: false
test-dsn:
user: test-user
password: '********'
net: tcp
addr: 192.168.12.34:3309
schema: information_schema
charset: utf8mb4
collation: utf8mb4_general_ci
loc: GMT
tls: aabbbaa
server-public-key: this is a tset serverpublic
max-allowed-packet: 4194309
params:
charset: utf8mb4
timeout: 50s
read-timeout: 40s
write-timeout: 30s
allow-native-passwords: false
allow-old-passwords: true
disable: false
allow-online-as-test: true
disable-version-check: false
drop-test-temporary: false
cleanup-test-database: true
only-syntax-check: true
sampling-statistic-target: 110
sampling: true
sampling-condition: aaa
profiling: true
trace: true
explain: false
delimiter: ;
log-level: 3
log-output: /dev/null
report-type: html
report-css: sdfs
report-javascript: sdfsd
report-title: SQL优化分析报告-test
markdown-extensions: 92
markdown-html-flags: 10
ignore-rules:
- COL.012
rewrite-rules:
- delimiter
- orderbynull
- groupbyconst
- dmlorderby
- having
- star2columns
- insertcolumns
- distinctstar
blacklist: /tmp/blacklist
max-join-table-count: 12
max-group-by-cols-count: 15
max-distinct-count: 7
max-index-cols-count: 2
max-text-cols-count: 3
max-total-rows: 9999991
max-query-cost: 9992
spaghetti-query-length: 2041
allow-drop-index: true
max-in-count: 101
max-index-bytes-percolumn: 762
max-index-bytes: 3073
allow-charsets:
- utf8
- utf8mb4
allow-collates: []
allow-engines:
- innodb
- tokudb
max-index-count: 12
max-column-count: 41
max-value-count: 102
index-prefix: idx_
unique-key-prefix: uk_
max-subquery-depth: 6
max-varchar-length: 1022
column-not-allow-type:
- boolean
min-cardinality: 2
explain-sql-report-type: pretty
explain-type: extended
explain-format: traditional
explain-warn-select-type:
- ""
explain-warn-access-type:
- ALL
explain-max-keys: 31
explain-min-keys: 10
explain-max-rows: 10002
explain-warn-extra:
- Using temporary
- Using filesort
explain-max-filtered: 120
explain-warn-scalability:
- O(log(n))
show-warnings: true
show-last-query-cost: true
query: ""
list-heuristic-rules: true
list-rewrite-rules: true
list-test-sqls: true
list-report-types: true
verbose: true
dry-run: false
max-pretty-sql-length: 1022
================================================
FILE: test/fixture/test_Run_all_test_cases.golden
================================================
# Query: C3FAEDA6AD6D762B
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
LENGTH = 86
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\` (\`length\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: E969B9297DA79BA6
★ ★ ★ ★ ☆ 80分
```sql
SELECT
*
FROM
film
WHERE
LENGTH IS NULL
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\` (\`length\`) ;
## 应尽量避免在 WHERE 子句中对字段进行 NULL 值判断
* **Item:** ARG.006
* **Severity:** L1
* **Content:** 使用 IS NULL 或 IS NOT NULL 将可能导致引擎放弃使用索引而进行全表扫描，如：select id from t where num is null;可以在num上设置默认值0，确保表中 num 列没有 NULL 值，然后这样查询： select id from t where num=0;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 8A106444D14B9880
★ ★ ★ ☆ ☆ 60分
```sql
SELECT
*
FROM
film
HAVING
title = 'abc'
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## 不建议使用 HAVING 子句
* **Item:** CLA.013
* **Severity:** L3
* **Content:** 将查询的 HAVING 子句改写为 WHERE 中的查询条件，可以在查询处理期间使用索引。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: A0C5E62C724A121A
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
sakila. film
WHERE
LENGTH >= 60
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\` (\`length\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 868317D1973FD1B0
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
LENGTH BETWEEN 60
AND 84
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\` (\`length\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 707FE669669FA075
★ ★ ★ ★ ☆ 95分
```sql
SELECT
*
FROM
film
WHERE
title LIKE 'AIR%'
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | range | idx\_title | idx\_title | 767 | NULL | 2 | ☠️ **100.00%** | O(log n)+ | Using index condition |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **range**: 只检索给定范围的行, 使用一个索引来选择行. key列显示使用了哪个索引. key_len包含所使用索引的最长关键元素.
#### Extra信息解读
* **Using index condition**: 在5.6版本后加入的新特性（Index Condition Pushdown）。Using index condition 会先条件过滤索引，过滤完索引后找到所有符合索引条件的数据行，随后用 WHERE 子句中的其他条件去过滤这些数据行。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: DF916439ABD07664
★ ★ ★ ★ ☆ 90分
```sql
SELECT
*
FROM
film
WHERE
title IS NOT NULL
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
## 应尽量避免在 WHERE 子句中对字段进行 NULL 值判断
* **Item:** ARG.006
* **Severity:** L1
* **Content:** 使用 IS NULL 或 IS NOT NULL 将可能导致引擎放弃使用索引而进行全表扫描，如：select id from t where num is null;可以在num上设置默认值0，确保表中 num 列没有 NULL 值，然后这样查询： select id from t where num=0;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: B9336971FF3D3792
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
LENGTH = 114
AND title = 'ALABAMA DEVIL'
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ref | idx\_title | idx\_title | 767 | const | 1 | n% | O(log n) | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列title添加索引，散粒度为: n%; 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_title\_length\` (\`title\`,\`length\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 68E48001ECD53152
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
LENGTH > 100
AND title = 'ALABAMA DEVIL'
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ref | idx\_title | idx\_title | 767 | const | 1 | n% | O(log n) | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列title添加索引，散粒度为: n%; 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_title\_length\` (\`title\`,\`length\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 12FF1DAA3D425FA9
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
LENGTH > 100
AND language_id < 10
AND title = 'xyz'
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ref | idx\_title,
idx\_fk\_language\_id | idx\_title | 767 | const | 1 | n% | O(log n) | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列title添加索引，散粒度为: n%; 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_title\_length\` (\`title\`,\`length\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: E84CBAAC2E12BDEA
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
LENGTH > 100
AND language_id < 10
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | idx\_fk\_language\_id | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\` (\`length\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 6A0F035BD4E01018
★ ★ ★ ☆ ☆ 75分
```sql
SELECT
release_year, SUM( LENGTH)
FROM
film
WHERE
LENGTH = 123
AND language_id = 1
GROUP BY
release_year
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | idx\_fk\_language\_id | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using temporary |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%; 为列language\_id添加索引，散粒度为: n%; 为列release\_year添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\_language\_id\_release\_year\` (\`length\`,\`language\_id\`,\`release\_year\`) ;
## 请为 GROUP BY 显示添加 ORDER BY 条件
* **Item:** CLA.008
* **Severity:** L2
* **Content:** 默认 MySQL 会对 'GROUP BY col1, col2, ...' 请求按如下顺序排序 'ORDER BY col1, col2, ...'。如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 'ORDER BY NULL'。
## 使用 SUM(COL) 时需注意 NPE 问题
* **Item:** FUN.006
* **Severity:** L1
* **Content:** 当某一列的值全是 NULL 时，COUNT(COL) 的返回结果为0,但 SUM(COL) 的返回结果为 NULL，因此使用 SUM() 时需注意 NPE 问题。可以使用如下方式来避免 SUM 的 NPE 问题: SELECT IF(ISNULL(SUM(COL)), 0, SUM(COL)) FROM tbl
# Query: 23D176AEA2947002
★ ★ ★ ☆ ☆ 75分
```sql
SELECT
release_year, SUM( LENGTH)
FROM
film
WHERE
LENGTH >= 123
GROUP BY
release_year
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using temporary |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%; 为列release\_year添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\_release\_year\` (\`length\`,\`release\_year\`) ;
## 请为 GROUP BY 显示添加 ORDER BY 条件
* **Item:** CLA.008
* **Severity:** L2
* **Content:** 默认 MySQL 会对 'GROUP BY col1, col2, ...' 请求按如下顺序排序 'ORDER BY col1, col2, ...'。如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 'ORDER BY NULL'。
## 使用 SUM(COL) 时需注意 NPE 问题
* **Item:** FUN.006
* **Severity:** L1
* **Content:** 当某一列的值全是 NULL 时，COUNT(COL) 的返回结果为0,但 SUM(COL) 的返回结果为 NULL，因此使用 SUM() 时需注意 NPE 问题。可以使用如下方式来避免 SUM 的 NPE 问题: SELECT IF(ISNULL(SUM(COL)), 0, SUM(COL)) FROM tbl
# Query: 73DDF6E6D9E40384
★ ★ ☆ ☆ ☆ 55分
```sql
SELECT
release_year, language_id, SUM( LENGTH)
FROM
film
GROUP BY
release_year, language_id
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | Using temporary |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列release\_year添加索引，散粒度为: n%; 为列language\_id添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_release\_year\_language\_id\` (\`release\_year\`,\`language\_id\`) ;
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## 请为 GROUP BY 显示添加 ORDER BY 条件
* **Item:** CLA.008
* **Severity:** L2
* **Content:** 默认 MySQL 会对 'GROUP BY col1, col2, ...' 请求按如下顺序排序 'ORDER BY col1, col2, ...'。如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 'ORDER BY NULL'。
## 使用 SUM(COL) 时需注意 NPE 问题
* **Item:** FUN.006
* **Severity:** L1
* **Content:** 当某一列的值全是 NULL 时，COUNT(COL) 的返回结果为0,但 SUM(COL) 的返回结果为 NULL，因此使用 SUM() 时需注意 NPE 问题。可以使用如下方式来避免 SUM 的 NPE 问题: SELECT IF(ISNULL(SUM(COL)), 0, SUM(COL)) FROM tbl
# Query: B3C502B4AA344196
★ ★ ★ ☆ ☆ 65分
```sql
SELECT
release_year, SUM( LENGTH)
FROM
film
WHERE
LENGTH = 123
GROUP BY
release_year, (LENGTH+ language_id)
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using temporary |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\` (\`length\`) ;
## 请为 GROUP BY 显示添加 ORDER BY 条件
* **Item:** CLA.008
* **Severity:** L2
* **Content:** 默认 MySQL 会对 'GROUP BY col1, col2, ...' 请求按如下顺序排序 'ORDER BY col1, col2, ...'。如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 'ORDER BY NULL'。
## GROUP BY 的条件为表达式
* **Item:** CLA.010
* **Severity:** L2
* **Content:** 当 GROUP BY 条件为表达式或函数时会使用到临时表，如果在未指定 WHERE 或 WHERE 条件返回的结果集较大时性能会很差。
## 使用 SUM(COL) 时需注意 NPE 问题
* **Item:** FUN.006
* **Severity:** L1
* **Content:** 当某一列的值全是 NULL 时，COUNT(COL) 的返回结果为0,但 SUM(COL) 的返回结果为 NULL，因此使用 SUM() 时需注意 NPE 问题。可以使用如下方式来避免 SUM 的 NPE 问题: SELECT IF(ISNULL(SUM(COL)), 0, SUM(COL)) FROM tbl
# Query: 47044E1FE1A965A5
★ ★ ★ ☆ ☆ 60分
```sql
SELECT
release_year, SUM( film_id)
FROM
film
GROUP BY
release_year
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | Using temporary |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列release\_year添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_release\_year\` (\`release\_year\`) ;
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## 请为 GROUP BY 显示添加 ORDER BY 条件
* **Item:** CLA.008
* **Severity:** L2
* **Content:** 默认 MySQL 会对 'GROUP BY col1, col2, ...' 请求按如下顺序排序 'ORDER BY col1, col2, ...'。如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 'ORDER BY NULL'。
# Query: 2BA1217F6C8CF0AB
★ ☆ ☆ ☆ ☆ 35分
```sql
SELECT
*
FROM
address
GROUP BY
address, district
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *address* | NULL | ALL | NULL | NULL | NULL | NULL | 603 | ☠️ **100.00%** | ☠️ **O(n)** | Using temporary |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的address表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列address添加索引，散粒度为: n%; 为列district添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`address\` add index \`idx\_address\_district\` (\`address\`,\`district\`) ;
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## 请为 GROUP BY 显示添加 ORDER BY 条件
* **Item:** CLA.008
* **Severity:** L2
* **Content:** 默认 MySQL 会对 'GROUP BY col1, col2, ...' 请求按如下顺序排序 'ORDER BY col1, col2, ...'。如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 'ORDER BY NULL'。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
## 非确定性的 GROUP BY
* **Item:** RES.001
* **Severity:** L4
* **Content:** SQL返回的列既不在聚合函数中也不是 GROUP BY 表达式的列中，因此这些值的结果将是非确定性的。如：select a, b, c from tbl where foo="bar" group by a，该 SQL 返回的结果就是不确定的。
# Query: 863A85207E4F410D
★ ★ ★ ★ ☆ 80分
```sql
SELECT
title
FROM
film
WHERE
ABS( language_id) = 3
GROUP BY
title
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | index | idx\_title | idx\_title | 767 | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **index**: 全表扫描, 只是扫描表的时候按照索引次序进行而不是行. 主要优点就是避免了排序, 但是开销仍然非常大.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 请为 GROUP BY 显示添加 ORDER BY 条件
* **Item:** CLA.008
* **Severity:** L2
* **Content:** 默认 MySQL 会对 'GROUP BY col1, col2, ...' 请求按如下顺序排序 'ORDER BY col1, col2, ...'。如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 'ORDER BY NULL'。
## 避免在 WHERE 条件中使用函数或其他运算符
* **Item:** FUN.001
* **Severity:** L2
* **Content:** 虽然在 SQL 中使用函数可以简化很多复杂的查询，但使用了函数的查询无法利用表中已经建立的索引，该查询将会是全表扫描，性能较差。通常建议将列名写在比较运算符左侧，将查询过滤条件放在比较运算符右侧。也不建议在查询比较条件两侧书写多余的括号，这会对阅读产生比较大的困扰。
# Query: DF59FD602E4AA368
★ ★ ★ ☆ ☆ 70分
```sql
SELECT
language_id
FROM
film
WHERE
LENGTH = 123
GROUP BY
release_year
ORDER BY
language_id
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using temporary; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%; 为列release\_year添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\_release\_year\` (\`length\`,\`release\_year\`) ;
## 非确定性的 GROUP BY
* **Item:** RES.001
* **Severity:** L4
* **Content:** SQL返回的列既不在聚合函数中也不是 GROUP BY 表达式的列中，因此这些值的结果将是非确定性的。如：select a, b, c from tbl where foo="bar" group by a，该 SQL 返回的结果就是不确定的。
# Query: F6DBEAA606D800FC
★ ★ ★ ★ ☆ 90分
```sql
SELECT
release_year
FROM
film
WHERE
LENGTH = 123
GROUP BY
release_year
ORDER BY
release_year
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using temporary; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%; 为列release\_year添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\_release\_year\` (\`length\`,\`release\_year\`) ;
# Query: 6E9B96CA3F0E6BDA
★ ★ ★ ☆ ☆ 65分
```sql
SELECT
*
FROM
film
WHERE
LENGTH = 123
ORDER BY
release_year ASC, language_id DESC
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\` (\`length\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
## ORDER BY 多个列但排序方向不同时可能无法使用索引
* **Item:** KEY.008
* **Severity:** L4
* **Content:** 在 MySQL 8.0 之前当 ORDER BY 多个列指定的排序方向不同时将无法使用已经建立的索引。
# Query: 2EAACFD7030EA528
★ ★ ★ ★ ☆ 90分
```sql
SELECT
release_year
FROM
film
WHERE
LENGTH = 123
GROUP BY
release_year
ORDER BY
release_year
LIMIT
10
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using temporary; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%; 为列release\_year添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\_release\_year\` (\`length\`,\`release\_year\`) ;
# Query: 5CE2F187DBF2A710
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
LENGTH = 123
ORDER BY
release_year
LIMIT
10
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%; 为列release\_year添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\_release\_year\` (\`length\`,\`release\_year\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: E75234155B5E2E14
★ ★ ★ ☆ ☆ 75分
```sql
SELECT
*
FROM
film
ORDER BY
release_year
LIMIT
10
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: AFEEBF10A8D74E32
★ ★ ★ ★ ☆ 80分
```sql
SELECT
film_id
FROM
film
ORDER BY
release_year
LIMIT
10
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
# Query: 965D5AC955824512
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
LENGTH > 100
ORDER BY
LENGTH
LIMIT
10
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\` (\`length\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 1E2CF4145EE706A5
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
LENGTH < 100
ORDER BY
LENGTH
LIMIT
10
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\` (\`length\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: A314542EEE8571EE
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
customer
WHERE
address_id in (224, 510)
ORDER BY
last_name
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *customer* | NULL | range | idx\_fk\_address\_id | idx\_fk\_address\_id | 2 | NULL | 2 | ☠️ **100.00%** | O(log n)+ | Using index condition; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **range**: 只检索给定范围的行, 使用一个索引来选择行. key列显示使用了哪个索引. key_len包含所使用索引的最长关键元素.
#### Extra信息解读
* **Using index condition**: 在5.6版本后加入的新特性（Index Condition Pushdown）。Using index condition 会先条件过滤索引，过滤完索引后找到所有符合索引条件的数据行，随后用 WHERE 子句中的其他条件去过滤这些数据行。
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 为sakila库的customer表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列address\_id添加索引，散粒度为: n%; 为列last\_name添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`customer\` add index \`idx\_address\_id\_last\_name\` (\`address\_id\`,\`last\_name\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 0BE2D79E2F1E7CB0
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
release_year = 2016
AND LENGTH != 1
ORDER BY
title
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列release\_year添加索引，散粒度为: n%; 为列length添加索引，散粒度为: n%; 为列title添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_release\_year\_length\_title\` (\`release\_year\`,\`length\`,\`title\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
## '!=' 运算符是非标准的
* **Item:** STA.001
* **Severity:** L0
* **Content:** "<>"才是标准SQL中的不等于运算符。
# Query: 4E73AA068370E6A8
★ ★ ★ ★ ☆ 90分
```sql
SELECT
title
FROM
film
WHERE
release_year = 1995
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列release\_year添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_release\_year\` (\`release\_year\`) ;
# Query: BA7111449E4F1122
★ ★ ★ ★ ☆ 90分
```sql
SELECT
title, replacement_cost
FROM
film
WHERE
language_id = 5
AND LENGTH = 70
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ref | idx\_fk\_language\_id | idx\_fk\_language\_id | 1 | const | 1 | n% | O(log n) | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%; 为列language\_id添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\_language\_id\` (\`length\`,\`language\_id\`) ;
# Query: B13E0ACEAF8F3119
★ ★ ★ ★ ☆ 90分
```sql
SELECT
title
FROM
film
WHERE
language_id > 5
AND LENGTH > 70
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | range | idx\_fk\_language\_id | idx\_fk\_language\_id | 1 | NULL | 1 | n% | O(log n)+ | Using index condition; Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **range**: 只检索给定范围的行, 使用一个索引来选择行. key列显示使用了哪个索引. key_len包含所使用索引的最长关键元素.
#### Extra信息解读
* **Using index condition**: 在5.6版本后加入的新特性（Index Condition Pushdown）。Using index condition 会先条件过滤索引，过滤完索引后找到所有符合索引条件的数据行，随后用 WHERE 子句中的其他条件去过滤这些数据行。
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\` (\`length\`) ;
# Query: A3FAB6027484B88B
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
LENGTH = 100
AND title = 'xyz'
ORDER BY
release_year
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ref | idx\_title | idx\_title | 767 | const | 1 | n% | O(log n) | Using where; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列title添加索引，散粒度为: n%; 为列length添加索引，散粒度为: n%; 为列release\_year添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_title\_length\_release\_year\` (\`title\`,\`length\`,\`release\_year\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: CB42080E9F35AB07
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
LENGTH > 100
AND title = 'xyz'
ORDER BY
release_year
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ref | idx\_title | idx\_title | 767 | const | 1 | n% | O(log n) | Using where; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列title添加索引，散粒度为: n%; 为列length添加索引，散粒度为: n%; 为列release\_year添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_title\_length\_release\_year\` (\`title\`,\`length\`,\`release\_year\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: C4A212A42400411D
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
LENGTH > 100
ORDER BY
release_year
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%; 为列release\_year添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\_release\_year\` (\`length\`,\`release\_year\`) ;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 4ECCA9568BE69E68
★ ★ ★ ★ ☆ 95分
```sql
SELECT
*
FROM
city a
INNER JOIN country b ON a. country_id= b. country_id
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *b* | NULL | ALL | PRIMARY | NULL | NULL | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *a* | NULL | ref | idx\_fk\_country\_id | idx\_fk\_country\_id | 2 | sakila.b.country\_id | 5 | ☠️ **100.00%** | O(log n) | NULL |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 485D56FC88BBBDB9
★ ★ ★ ★ ☆ 95分
```sql
SELECT
*
FROM
city a
LEFT JOIN country b ON a. country_id= b. country_id
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *a* | NULL | ALL | NULL | NULL | NULL | NULL | 600 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *b* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.a.country\_id | 1 | ☠️ **100.00%** | O(log n) | NULL |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **eq_ref**: 除const类型外最好的可能实现的连接类型. 它用在一个索引的所有部分被连接使用并且索引是UNIQUE或PRIMARY KEY, 对于每个索引键, 表中只有一条记录与之匹配. 例: 'SELECT * FROM RefTbl, tbl WHERE RefTbl.col=tbl.col;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 0D0DABACEDFF5765
★ ★ ★ ★ ☆ 95分
```sql
SELECT
*
FROM
city a
RIGHT JOIN country b ON a. country_id= b. country_id
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *b* | NULL | ALL | NULL | NULL | NULL | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *a* | NULL | ref | idx\_fk\_country\_id | idx\_fk\_country\_id | 2 | sakila.b.country\_id | 5 | ☠️ **100.00%** | O(log n) | NULL |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 1E56C6CCEA2131CC
★ ★ ★ ★ ☆ 80分
```sql
SELECT
*
FROM
city a
LEFT JOIN country b ON a. country_id= b. country_id
WHERE
b. last_update IS NULL
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *a* | NULL | ALL | NULL | NULL | NULL | NULL | 600 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *b* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.a.country\_id | 1 | n% | O(log n) | Using where; Not exists |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **eq_ref**: 除const类型外最好的可能实现的连接类型. 它用在一个索引的所有部分被连接使用并且索引是UNIQUE或PRIMARY KEY, 对于每个索引键, 表中只有一条记录与之匹配. 例: 'SELECT * FROM RefTbl, tbl WHERE RefTbl.col=tbl.col;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Not exists**: MySQL能够对LEFT JOIN查询进行优化, 并且在查找到符合LEFT JOIN条件的行后, 则不再查找更多的行.
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的country表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列last\_update添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`country\` add index \`idx\_last\_update\` (\`last\_update\`) ;
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
## 应尽量避免在 WHERE 子句中对字段进行 NULL 值判断
* **Item:** ARG.006
* **Severity:** L1
* **Content:** 使用 IS NULL 或 IS NOT NULL 将可能导致引擎放弃使用索引而进行全表扫描，如：select id from t where num is null;可以在num上设置默认值0，确保表中 num 列没有 NULL 值，然后这样查询： select id from t where num=0;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: F5D30BCAC1E206A1
★ ★ ★ ★ ☆ 80分
```sql
SELECT
*
FROM
city a
RIGHT JOIN country b ON a. country_id= b. country_id
WHERE
a. last_update IS NULL
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *b* | NULL | ALL | NULL | NULL | NULL | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *a* | NULL | ref | idx\_fk\_country\_id | idx\_fk\_country\_id | 2 | sakila.b.country\_id | 5 | n% | O(log n) | Using where; Not exists |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Not exists**: MySQL能够对LEFT JOIN查询进行优化, 并且在查找到符合LEFT JOIN条件的行后, 则不再查找更多的行.
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的city表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列last\_update添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`city\` add index \`idx\_last\_update\` (\`last\_update\`) ;
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
## 应尽量避免在 WHERE 子句中对字段进行 NULL 值判断
* **Item:** ARG.006
* **Severity:** L1
* **Content:** 使用 IS NULL 或 IS NOT NULL 将可能导致引擎放弃使用索引而进行全表扫描，如：select id from t where num is null;可以在num上设置默认值0，确保表中 num 列没有 NULL 值，然后这样查询： select id from t where num=0;
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 17D5BCF21DC2364C
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
city a
LEFT JOIN country b ON a. country_id= b. country_id
UNION
SELECT
*
FROM
city a
RIGHT JOIN country b ON a. country_id= b. country_id
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | PRIMARY | *a* | NULL | ALL | NULL | NULL | NULL | NULL | 600 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | PRIMARY | *b* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.a.country\_id | 1 | ☠️ **100.00%** | O(log n) | NULL |
| 2 | UNION | *b* | NULL | ALL | NULL | NULL | NULL | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 2 | UNION | *a* | NULL | ref | idx\_fk\_country\_id | idx\_fk\_country\_id | 2 | sakila.b.country\_id | 5 | ☠️ **100.00%** | O(log n) | NULL |
| 2 | UNION | *a* | NULL | ref | idx\_fk\_country\_id | idx\_fk\_country\_id | 2 | sakila.b.country\_id | 5 | ☠️ **100.00%** | O(log n) | NULL |
### Explain信息解读
#### SelectType信息解读
* **PRIMARY**: 最外层的select.
* **UNION**: UNION中的第二个或后面的SELECT查询, 不依赖于外部查询的结果集.
#### Type信息解读
* **eq_ref**: 除const类型外最好的可能实现的连接类型. 它用在一个索引的所有部分被连接使用并且索引是UNIQUE或PRIMARY KEY, 对于每个索引键, 表中只有一条记录与之匹配. 例: 'SELECT * FROM RefTbl, tbl WHERE RefTbl.col=tbl.col;'.
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
## 如果您不在乎重复的话，建议使用 UNION ALL 替代 UNION
* **Item:** SUB.002
* **Severity:** L2
* **Content:** 与去除重复的UNION不同，UNION ALL允许重复元组。如果您不关心重复元组，那么使用UNION ALL将是一个更快的选项。
# Query: A4911095C201896F
★ ★ ★ ☆ ☆ 65分
```sql
SELECT
*
FROM
city a
RIGHT JOIN country b ON a. country_id= b. country_id
WHERE
a. last_update IS NULL
UNION
SELECT
*
FROM
city a
LEFT JOIN country b ON a. country_id= b. country_id
WHERE
b. last_update IS NULL
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | PRIMARY | *b* | NULL | ALL | NULL | NULL | NULL | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | PRIMARY | *a* | NULL | ref | idx\_fk\_country\_id | idx\_fk\_country\_id | 2 | sakila.b.country\_id | 5 | n% | O(log n) | Using where; Not exists |
| 2 | UNION | *a* | NULL | ALL | NULL | NULL | NULL | NULL | 600 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 2 | UNION | *b* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.a.country\_id | 1 | n% | O(log n) | Using where; Not exists |
| 2 | UNION | *b* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.a.country\_id | 1 | n% | O(log n) | Using where; Not exists |
### Explain信息解读
#### SelectType信息解读
* **PRIMARY**: 最外层的select.
* **UNION**: UNION中的第二个或后面的SELECT查询, 不依赖于外部查询的结果集.
#### Type信息解读
* **eq_ref**: 除const类型外最好的可能实现的连接类型. 它用在一个索引的所有部分被连接使用并且索引是UNIQUE或PRIMARY KEY, 对于每个索引键, 表中只有一条记录与之匹配. 例: 'SELECT * FROM RefTbl, tbl WHERE RefTbl.col=tbl.col;'.
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Not exists**: MySQL能够对LEFT JOIN查询进行优化, 并且在查找到符合LEFT JOIN条件的行后, 则不再查找更多的行.
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的city表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列last\_update添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`city\` add index \`idx\_last\_update\` (\`last\_update\`) ;
## 为sakila库的country表添加索引
* **Item:** IDX.002
* **Severity:** L2
* **Content:** 为列last\_update添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`country\` add index \`idx\_last\_update\` (\`last\_update\`) ;
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
## 如果您不在乎重复的话，建议使用 UNION ALL 替代 UNION
* **Item:** SUB.002
* **Severity:** L2
* **Content:** 与去除重复的UNION不同，UNION ALL允许重复元组。如果您不关心重复元组，那么使用UNION ALL将是一个更快的选项。
# Query: 3FF20E28EC9CBEF9
★ ★ ★ ★ ★ 100分
```sql
SELECT
country_id, last_update
FROM
city NATURAL
JOIN country
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *country* | NULL | ALL | PRIMARY | NULL | NULL | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *city* | NULL | ref | idx\_fk\_country\_id | idx\_fk\_country\_id | 2 | sakila.country.country\_id | 5 | n% | O(log n) | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
# Query: 5C547F08EADBB131
★ ★ ★ ★ ★ 100分
```sql
SELECT
country_id, last_update
FROM
city NATURAL
LEFT JOIN country
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *city* | NULL | ALL | NULL | NULL | NULL | NULL | 600 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *country* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.city.country\_id | 1 | ☠️ **100.00%** | O(log n) | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **eq_ref**: 除const类型外最好的可能实现的连接类型. 它用在一个索引的所有部分被连接使用并且索引是UNIQUE或PRIMARY KEY, 对于每个索引键, 表中只有一条记录与之匹配. 例: 'SELECT * FROM RefTbl, tbl WHERE RefTbl.col=tbl.col;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
# Query: AF0C1EB58B23D2FA
★ ★ ★ ★ ★ 100分
```sql
SELECT
country_id, last_update
FROM
city NATURAL
RIGHT JOIN country
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *country* | NULL | ALL | NULL | NULL | NULL | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *city* | NULL | ref | idx\_fk\_country\_id | idx\_fk\_country\_id | 2 | sakila.country.country\_id | 5 | ☠️ **100.00%** | O(log n) | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
# Query: 626571EAE84E2C8A
★ ★ ★ ★ ★ 100分
```sql
SELECT
a. country_id, a. last_update
FROM
city a STRAIGHT_JOIN country b ON a. country_id= b. country_id
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *a* | NULL | ALL | idx\_fk\_country\_id | NULL | NULL | NULL | 600 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *b* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.a.country\_id | 1 | ☠️ **100.00%** | O(log n) | Using index |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **eq_ref**: 除const类型外最好的可能实现的连接类型. 它用在一个索引的所有部分被连接使用并且索引是UNIQUE或PRIMARY KEY, 对于每个索引键, 表中只有一条记录与之匹配. 例: 'SELECT * FROM RefTbl, tbl WHERE RefTbl.col=tbl.col;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using index**: 只需通过索引就可以从表中获取列的信息, 无需额外去读取真实的行数据. 如果查询使用的列值仅仅是一个简单索引的部分值, 则会使用这种策略来优化查询.
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
# Query: 50F2AB4243CE2071
★ ★ ★ ☆ ☆ 60分
```sql
SELECT
a. address, a. postal_code
FROM
sakila. address a
WHERE
a. city_id IN (
SELECT
c. city_id
FROM
sakila. city c)
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *c* | NULL | index | PRIMARY | idx\_fk\_country\_id | 2 | NULL | 600 | ☠️ **100.00%** | ☠️ **O(n)** | Using index |
| 1 | SIMPLE | *a* | NULL | ref | idx\_fk\_city\_id | idx\_fk\_city\_id | 2 | sakila.c.city\_id | 1 | ☠️ **100.00%** | O(log n) | NULL |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **index**: 全表扫描, 只是扫描表的时候按照索引次序进行而不是行. 主要优点就是避免了排序, 但是开销仍然非常大.
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
#### Extra信息解读
* **Using index**: 只需通过索引就可以从表中获取列的信息, 无需额外去读取真实的行数据. 如果查询使用的列值仅仅是一个简单索引的部分值, 则会使用这种策略来优化查询.
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## MySQL 对子查询的优化效果不佳
* **Item:** SUB.001
* **Severity:** L4
* **Content:** MySQL 将外部查询中的每一行作为依赖子查询执行子查询。 这是导致严重性能问题的常见原因。这可能会在 MySQL 5.6 版本中得到改善, 但对于5.1及更早版本, 建议将该类查询分别重写为 JOIN 或 LEFT OUTER JOIN。
# Query: 584CCEC8069B6947
★ ★ ☆ ☆ ☆ 50分
```sql
SELECT
city
FROM(
SELECT
city_id
FROM
city
WHERE
city = "A Corua (La Corua)"
ORDER BY
last_update DESC
LIMIT
50, 10) I
JOIN city ON (I. city_id = city. city_id)
JOIN country ON (country. country_id = city. country_id)
ORDER BY
city DESC
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | PRIMARY | *
idx\_fk\_country\_id | PRIMARY | 2 | I.city\_id | 1 | ☠️ **100.00%** | O(log n) | NULL |
| 1 | PRIMARY | *country* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.city.country\_id | 1 | ☠️ **100.00%** | O(log n) | Using index |
| 2 | DERIVED | *city* | NULL | ALL | NULL | NULL | NULL | NULL | 600 | n% | ☠️ **O(n)** | Using where; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **DERIVED**: 用于from子句里有子查询的情况. MySQL会递归执行这些子查询, 把结果放在临时表里.
* **PRIMARY**: 最外层的select.
#### Type信息解读
* **eq_ref**: 除const类型外最好的可能实现的连接类型. 它用在一个索引的所有部分被连接使用并且索引是UNIQUE或PRIMARY KEY, 对于每个索引键, 表中只有一条记录与之匹配. 例: 'SELECT * FROM RefTbl, tbl WHERE RefTbl.col=tbl.col;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using index**: 只需通过索引就可以从表中获取列的信息, 无需额外去读取真实的行数据. 如果查询使用的列值仅仅是一个简单索引的部分值, 则会使用这种策略来优化查询.
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的city表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列city添加索引，散粒度为: n%; 为列last\_update添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`city\` add index \`idx\_city\_last\_update\` (\`city\`,\`last\_update\`) ;
## 同一张表被连接两次
* **Item:** JOI.002
* **Severity:** L4
* **Content:** 相同的表在 FROM 子句中至少出现两次，可以简化为对该表的单次访问。
## MySQL 对子查询的优化效果不佳
* **Item:** SUB.001
* **Severity:** L4
* **Content:** MySQL 将外部查询中的每一行作为依赖子查询执行子查询。 这是导致严重性能问题的常见原因。这可能会在 MySQL 5.6 版本中得到改善, 但对于5.1及更早版本, 建议将该类查询分别重写为 JOIN 或 LEFT OUTER JOIN。
# Query: 7F02E23D44A38A6D
★ ★ ★ ★ ☆ 80分
```sql
DELETE city, country
FROM
city
INNER JOIN country using (country_id)
WHERE
city. city_id = 1
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | DELETE | *city* | NULL | const | PRIMARY,
idx\_fk\_country\_id | PRIMARY | 2 | const | 1 | ☠️ **100.00%** | O(1) | NULL |
| 1 | DELETE | *country* | NULL | const | PRIMARY | PRIMARY | 2 | const | 1 | ☠️ **100.00%** | O(1) | NULL |
### Explain信息解读
#### Type信息解读
* **const**: const用于使用常数值比较PRIMARY KEY时, 当查询的表仅有一行时, 使用system. 例:SELECT * FROM tbl WHERE col = 1.
## 不建议使用联表删除或更新
* **Item:** JOI.007
* **Severity:** L4
* **Content:** 当需要同时删除或更新多张表时建议使用简单语句，一条 SQL 只删除或更新一张表，尽量不要将多张表的操作在同一条语句。
## 使用DELETE/DROP/TRUNCATE等操作时注意备份
* **Item:** SEC.003
* **Severity:** L0
* **Content:** 在执行高危操作之前对数据进行备份是十分有必要的。
# Query: F8314ABD1CBF2FF1
★ ★ ★ ☆ ☆ 70分
```sql
DELETE city
FROM
city
LEFT JOIN country ON city. country_id = country. country_id
WHERE
country. country IS NULL
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | DELETE | *city* | NULL | ALL | NULL | NULL | NULL | NULL | 600 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *country* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.city.country\_id | 1 | n% | O(log n) | Using where; Not exists |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **eq_ref**: 除const类型外最好的可能实现的连接类型. 它用在一个索引的所有部分被连接使用并且索引是UNIQUE或PRIMARY KEY, 对于每个索引键, 表中只有一条记录与之匹配. 例: 'SELECT * FROM RefTbl, tbl WHERE RefTbl.col=tbl.col;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Not exists**: MySQL能够对LEFT JOIN查询进行优化, 并且在查找到符合LEFT JOIN条件的行后, 则不再查找更多的行.
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的country表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列country添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`country\` add index \`idx\_country\` (\`country\`) ;
## 不建议使用联表删除或更新
* **Item:** JOI.007
* **Severity:** L4
* **Content:** 当需要同时删除或更新多张表时建议使用简单语句，一条 SQL 只删除或更新一张表，尽量不要将多张表的操作在同一条语句。
## 使用DELETE/DROP/TRUNCATE等操作时注意备份
* **Item:** SEC.003
* **Severity:** L0
* **Content:** 在执行高危操作之前对数据进行备份是十分有必要的。
# Query: 1A53649C43122975
★ ★ ★ ★ ☆ 80分
```sql
DELETE a1, a2
FROM
city AS a1
INNER JOIN country AS a2
WHERE
a1. country_id= a2. country_id
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | DELETE | *a2* | NULL | ALL | PRIMARY | NULL | NULL | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | DELETE | *a1* | NULL | ref | idx\_fk\_country\_id | idx\_fk\_country\_id | 2 | sakila.a2.country\_id | 5 | ☠️ **100.00%** | O(log n) | NULL |
### Explain信息解读
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
## 不建议使用联表删除或更新
* **Item:** JOI.007
* **Severity:** L4
* **Content:** 当需要同时删除或更新多张表时建议使用简单语句，一条 SQL 只删除或更新一张表，尽量不要将多张表的操作在同一条语句。
## 使用DELETE/DROP/TRUNCATE等操作时注意备份
* **Item:** SEC.003
* **Severity:** L0
* **Content:** 在执行高危操作之前对数据进行备份是十分有必要的。
# Query: B862978586C6338B
★ ★ ★ ★ ☆ 80分
```sql
DELETE FROM
a1, a2 USING city AS a1
INNER JOIN country AS a2
WHERE
a1. country_id= a2. country_id
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | DELETE | *a2* | NULL | ALL | PRIMARY | NULL | NULL | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | DELETE | *a1* | NULL | ref | idx\_fk\_country\_id | idx\_fk\_country\_id | 2 | sakila.a2.country\_id | 5 | ☠️ **100.00%** | O(log n) | NULL |
### Explain信息解读
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
## 不建议使用联表删除或更新
* **Item:** JOI.007
* **Severity:** L4
* **Content:** 当需要同时删除或更新多张表时建议使用简单语句，一条 SQL 只删除或更新一张表，尽量不要将多张表的操作在同一条语句。
## 使用DELETE/DROP/TRUNCATE等操作时注意备份
* **Item:** SEC.003
* **Severity:** L0
* **Content:** 在执行高危操作之前对数据进行备份是十分有必要的。
# Query: F16FD63381EF8299
★ ★ ★ ★ ☆ 90分
```sql
DELETE FROM
film
WHERE
LENGTH > 100
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | DELETE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | Using where |
### Explain信息解读
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列length添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_length\` (\`length\`) ;
## 使用DELETE/DROP/TRUNCATE等操作时注意备份
* **Item:** SEC.003
* **Severity:** L0
* **Content:** 在执行高危操作之前对数据进行备份是十分有必要的。
# Query: 08CFE41C7D20AAC8
★ ★ ★ ★ ☆ 80分
```sql
UPDATE
city
INNER JOIN country USING( country_id)
SET
city. city = 'Abha',
city. last_update = '2006-02-15 04:45:25',
country. country = 'Afghanistan'
WHERE
city. city_id= 10
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | UPDATE | *city* | NULL | const | PRIMARY,
idx\_fk\_country\_id | PRIMARY | 2 | const | 1 | ☠️ **100.00%** | O(1) | NULL |
| 1 | UPDATE | *country* | NULL | const | PRIMARY | PRIMARY | 2 | const | 1 | ☠️ **100.00%** | O(1) | NULL |
### Explain信息解读
#### Type信息解读
* **const**: const用于使用常数值比较PRIMARY KEY时, 当查询的表仅有一行时, 使用system. 例:SELECT * FROM tbl WHERE col = 1.
## 不建议使用联表删除或更新
* **Item:** JOI.007
* **Severity:** L4
* **Content:** 当需要同时删除或更新多张表时建议使用简单语句，一条 SQL 只删除或更新一张表，尽量不要将多张表的操作在同一条语句。
# Query: C15BDF2C73B5B7ED
★ ★ ★ ★ ☆ 80分
```sql
UPDATE
city
INNER JOIN country ON city. country_id = country. country_id
INNER JOIN address ON city. city_id = address. city_id
SET
city. city = 'Abha',
city. last_update = '2006-02-15 04:45:25',
country. country = 'Afghanistan'
WHERE
city. city_id= 10
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | UPDATE | *city* | NULL | const | PRIMARY,
idx\_fk\_country\_id | PRIMARY | 2 | const | 1 | ☠️ **100.00%** | O(1) | NULL |
| 1 | UPDATE | *country* | NULL | const | PRIMARY | PRIMARY | 2 | const | 1 | ☠️ **100.00%** | O(1) | NULL |
| 1 | SIMPLE | *address* | NULL | ref | idx\_fk\_city\_id | idx\_fk\_city\_id | 2 | const | 1 | ☠️ **100.00%** | O(log n) | Using index |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **const**: const用于使用常数值比较PRIMARY KEY时, 当查询的表仅有一行时, 使用system. 例:SELECT * FROM tbl WHERE col = 1.
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
#### Extra信息解读
* **Using index**: 只需通过索引就可以从表中获取列的信息, 无需额外去读取真实的行数据. 如果查询使用的列值仅仅是一个简单索引的部分值, 则会使用这种策略来优化查询.
## 不建议使用联表删除或更新
* **Item:** JOI.007
* **Severity:** L4
* **Content:** 当需要同时删除或更新多张表时建议使用简单语句，一条 SQL 只删除或更新一张表，尽量不要将多张表的操作在同一条语句。
# Query: FCD1ABF36F8CDAD7
★ ★ ★ ★ ★ 100分
```sql
UPDATE
city, country
SET
city. city = 'Abha',
city. last_update = '2006-02-15 04:45:25',
country. country = 'Afghanistan'
WHERE
city. country_id = country. country_id
AND city. city_id= 10
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | UPDATE | *city* | NULL | const | PRIMARY,
idx\_fk\_country\_id | PRIMARY | 2 | const | 1 | ☠️ **100.00%** | O(1) | NULL |
| 1 | UPDATE | *country* | NULL | const | PRIMARY | PRIMARY | 2 | const | 1 | ☠️ **100.00%** | O(1) | NULL |
### Explain信息解读
#### Type信息解读
* **const**: const用于使用常数值比较PRIMARY KEY时, 当查询的表仅有一行时, 使用system. 例:SELECT * FROM tbl WHERE col = 1.
# Query: FE409EB794EE91CF
★ ★ ★ ★ ★ 100分
```sql
UPDATE
film
SET
LENGTH = 10
WHERE
language_id = 20
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | UPDATE | *film* | NULL | range | idx\_fk\_language\_id | idx\_fk\_language\_id | 1 | const | 1 | ☠️ **100.00%** | O(log n)+ | Using where |
### Explain信息解读
#### Type信息解读
* **range**: 只检索给定范围的行, 使用一个索引来选择行. key列显示使用了哪个索引. key_len包含所使用索引的最长关键元素.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
# Query: 3656B13CC4F888E2
★ ★ ★ ☆ ☆ 65分
```sql
INSERT INTO city (country_id)
SELECT
country_id
FROM
country
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | INSERT | *city* | NULL | ALL | NULL | NULL | NULL | NULL | 0 | n% | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *country* | NULL | index | NULL | PRIMARY | 2 | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | Using index |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **index**: 全表扫描, 只是扫描表的时候按照索引次序进行而不是行. 主要优点就是避免了排序, 但是开销仍然非常大.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using index**: 只需通过索引就可以从表中获取列的信息, 无需额外去读取真实的行数据. 如果查询使用的列值仅仅是一个简单索引的部分值, 则会使用这种策略来优化查询.
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## INSERT INTO xx SELECT 加锁粒度较大请谨慎
* **Item:** LCK.001
* **Severity:** L3
* **Content:** INSERT INTO xx SELECT 加锁粒度较大请谨慎
# Query: 2F7439623B712317
★ ★ ★ ★ ★ 100分
```sql
INSERT INTO city (country_id)
VALUES
(1),
(2),
(3)
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | INSERT | *city* | NULL | ALL | NULL | NULL | NULL | NULL | 0 | n% | ☠️ **O(n)** | NULL |
### Explain信息解读
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
# Query: 11EC7AAACC97DC0F
★ ★ ★ ★ ☆ 85分
```sql
INSERT INTO city (country_id)
SELECT
10
FROM
DUAL
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | INSERT | *city* | NULL | ALL | NULL | NULL | NULL | NULL | 0 | n% | ☠️ **O(n)** | NULL |
### Explain信息解读
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
## INSERT INTO xx SELECT 加锁粒度较大请谨慎
* **Item:** LCK.001
* **Severity:** L3
* **Content:** INSERT INTO xx SELECT 加锁粒度较大请谨慎
# Query: E3DDA1A929236E72
★ ★ ★ ☆ ☆ 65分
```sql
REPLACE INTO city (country_id)
SELECT
country_id
FROM
country
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | REPLACE | *city* | NULL | ALL | NULL | NULL | NULL | NULL | 0 | n% | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *country* | NULL | index | NULL | PRIMARY | 2 | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | Using index |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **index**: 全表扫描, 只是扫描表的时候按照索引次序进行而不是行. 主要优点就是避免了排序, 但是开销仍然非常大.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using index**: 只需通过索引就可以从表中获取列的信息, 无需额外去读取真实的行数据. 如果查询使用的列值仅仅是一个简单索引的部分值, 则会使用这种策略来优化查询.
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## INSERT INTO xx SELECT 加锁粒度较大请谨慎
* **Item:** LCK.001
* **Severity:** L3
* **Content:** INSERT INTO xx SELECT 加锁粒度较大请谨慎
# Query: 466F1AC2F5851149
★ ★ ★ ★ ★ 100分
```sql
REPLACE INTO city (country_id)
VALUES
(1),
(2),
(3)
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | REPLACE | *city* | NULL | ALL | NULL | NULL | NULL | NULL | 0 | n% | ☠️ **O(n)** | NULL |
### Explain信息解读
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
# Query: A7973BDD268F926E
★ ★ ★ ★ ☆ 85分
```sql
REPLACE INTO city (country_id)
SELECT
10
FROM
DUAL
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | REPLACE | *city* | NULL | ALL | NULL | NULL | NULL | NULL | 0 | n% | ☠️ **O(n)** | NULL |
### Explain信息解读
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
## INSERT INTO xx SELECT 加锁粒度较大请谨慎
* **Item:** LCK.001
* **Severity:** L3
* **Content:** INSERT INTO xx SELECT 加锁粒度较大请谨慎
# Query: 105C870D5DFB6710
★ ★ ★ ☆ ☆ 65分
```sql
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
(
SELECT
film_id
FROM
film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
) film
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | index | NULL | idx\_fk\_language\_id | 1 | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | Using index |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **index**: 全表扫描, 只是扫描表的时候按照索引次序进行而不是行. 主要优点就是避免了排序, 但是开销仍然非常大.
#### Extra信息解读
* **Using index**: 只需通过索引就可以从表中获取列的信息, 无需额外去读取真实的行数据. 如果查询使用的列值仅仅是一个简单索引的部分值, 则会使用这种策略来优化查询.
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## 执行计划中嵌套连接深度过深
* **Item:** SUB.004
* **Severity:** L3
* **Content:** MySQL对子查询的优化效果不佳,MySQL将外部查询中的每一行作为依赖子查询执行子查询。 这是导致严重性能问题的常见原因。
# Query: 16C2B14E7DAA9906
★ ☆ ☆ ☆ ☆ 35分
```sql
SELECT
*
FROM
film
WHERE
language_id = (
SELECT
language_id
FROM
language
LIMIT
1)
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | PRIMARY | *film* | NULL | ALL | idx\_fk\_language\_id | NULL | NULL | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | Using where |
| 2 | SUBQUERY | *language* | NULL | index | NULL | PRIMARY | 1 | NULL | 6 | ☠️ **100.00%** | ☠️ **O(n)** | Using index |
### Explain信息解读
#### SelectType信息解读
* **PRIMARY**: 最外层的select.
* **SUBQUERY**: 子查询中的第一个SELECT查询, 不依赖于外部查询的结果集.
#### Type信息解读
* **index**: 全表扫描, 只是扫描表的时候按照索引次序进行而不是行. 主要优点就是避免了排序, 但是开销仍然非常大.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using index**: 只需通过索引就可以从表中获取列的信息, 无需额外去读取真实的行数据. 如果查询使用的列值仅仅是一个简单索引的部分值, 则会使用这种策略来优化查询.
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
## 未使用 ORDER BY 的 LIMIT 查询
* **Item:** RES.002
* **Severity:** L4
* **Content:** 没有 ORDER BY 的 LIMIT 会导致非确定性的结果，这取决于查询执行计划。
## MySQL 对子查询的优化效果不佳
* **Item:** SUB.001
* **Severity:** L4
* **Content:** MySQL 将外部查询中的每一行作为依赖子查询执行子查询。 这是导致严重性能问题的常见原因。这可能会在 MySQL 5.6 版本中得到改善, 但对于5.1及更早版本, 建议将该类查询分别重写为 JOIN 或 LEFT OUTER JOIN。
# Query: 16CB4628D2597D40
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
city i
LEFT JOIN country o ON i. city_id= o. country_id
UNION
SELECT
*
FROM
city i
RIGHT JOIN country o ON i. city_id= o. country_id
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | PRIMARY | *i* | NULL | ALL | NULL | NULL | NULL | NULL | 600 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | PRIMARY | *o* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.i.city\_id | 1 | ☠️ **100.00%** | O(log n) | NULL |
| 2 | UNION | *o* | NULL | ALL | NULL | NULL | NULL | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 2 | UNION | *i* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.o.country\_id | 1 | ☠️ **100.00%** | O(log n) | NULL |
| 2 | UNION | *i* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.o.country\_id | 1 | ☠️ **100.00%** | O(log n) | NULL |
### Explain信息解读
#### SelectType信息解读
* **PRIMARY**: 最外层的select.
* **UNION**: UNION中的第二个或后面的SELECT查询, 不依赖于外部查询的结果集.
#### Type信息解读
* **eq_ref**: 除const类型外最好的可能实现的连接类型. 它用在一个索引的所有部分被连接使用并且索引是UNIQUE或PRIMARY KEY, 对于每个索引键, 表中只有一条记录与之匹配. 例: 'SELECT * FROM RefTbl, tbl WHERE RefTbl.col=tbl.col;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
## 如果您不在乎重复的话，建议使用 UNION ALL 替代 UNION
* **Item:** SUB.002
* **Severity:** L2
* **Content:** 与去除重复的UNION不同，UNION ALL允许重复元组。如果您不关心重复元组，那么使用UNION ALL将是一个更快的选项。
# Query: EA50643B01E139A8
★ ☆ ☆ ☆ ☆ 35分
```sql
SELECT
*
FROM
(
SELECT
*
FROM
actor
WHERE
last_update= '2006-02-15 04:34:33'
AND last_name= 'CHASE'
) t
WHERE
last_update= '2006-02-15 04:34:33'
AND last_name= 'CHASE'
GROUP BY
first_name
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *actor* | NULL | ref | idx\_actor\_last\_name | idx\_actor\_last\_name | 137 | const | 2 | n% | O(log n) | Using where; Using temporary |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的actor表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列last\_name添加索引，散粒度为: n%; 为列last\_update添加索引，散粒度为: n%; 为列first\_name添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`actor\` add index \`idx\_last\_name\_last\_update\_first\_name\` (\`last\_name\`,\`last\_update\`,\`first\_name\`) ;
## 请为 GROUP BY 显示添加 ORDER BY 条件
* **Item:** CLA.008
* **Severity:** L2
* **Content:** 默认 MySQL 会对 'GROUP BY col1, col2, ...' 请求按如下顺序排序 'ORDER BY col1, col2, ...'。如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 'ORDER BY NULL'。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
## 非确定性的 GROUP BY
* **Item:** RES.001
* **Severity:** L4
* **Content:** SQL返回的列既不在聚合函数中也不是 GROUP BY 表达式的列中，因此这些值的结果将是非确定性的。如：select a, b, c from tbl where foo="bar" group by a，该 SQL 返回的结果就是不确定的。
## MySQL 对子查询的优化效果不佳
* **Item:** SUB.001
* **Severity:** L4
* **Content:** MySQL 将外部查询中的每一行作为依赖子查询执行子查询。 这是导致严重性能问题的常见原因。这可能会在 MySQL 5.6 版本中得到改善, 但对于5.1及更早版本, 建议将该类查询分别重写为 JOIN 或 LEFT OUTER JOIN。
# Query: 7598A4EDE6CFA6BE
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
city i
LEFT JOIN country o ON i. city_id= o. country_id
WHERE
o. country_id is null
UNION
SELECT
*
FROM
city i
RIGHT JOIN country o ON i. city_id= o. country_id
WHERE
i. city_id is null
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | PRIMARY | *i* | NULL | ALL | NULL | NULL | NULL | NULL | 600 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | PRIMARY | *o* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.i.city\_id | 1 | ☠️ **100.00%** | O(log n) | Using where; Not exists |
| 2 | UNION | *o* | NULL | ALL | NULL | NULL | NULL | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 2 | UNION | *i* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.o.country\_id | 1 | ☠️ **100.00%** | O(log n) | Using where; Not exists |
| 2 | UNION | *i* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.o.country\_id | 1 | ☠️ **100.00%** | O(log n) | Using where; Not exists |
### Explain信息解读
#### SelectType信息解读
* **PRIMARY**: 最外层的select.
* **UNION**: UNION中的第二个或后面的SELECT查询, 不依赖于外部查询的结果集.
#### Type信息解读
* **eq_ref**: 除const类型外最好的可能实现的连接类型. 它用在一个索引的所有部分被连接使用并且索引是UNIQUE或PRIMARY KEY, 对于每个索引键, 表中只有一条记录与之匹配. 例: 'SELECT * FROM RefTbl, tbl WHERE RefTbl.col=tbl.col;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Not exists**: MySQL能够对LEFT JOIN查询进行优化, 并且在查找到符合LEFT JOIN条件的行后, 则不再查找更多的行.
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
## 如果您不在乎重复的话，建议使用 UNION ALL 替代 UNION
* **Item:** SUB.002
* **Severity:** L2
* **Content:** 与去除重复的UNION不同，UNION ALL允许重复元组。如果您不关心重复元组，那么使用UNION ALL将是一个更快的选项。
# Query: 1E8B70E30062FD13
★ ★ ★ ★ ★ 100分
```sql
SELECT
first_name, last_name, email
FROM
customer STRAIGHT_JOIN address ON customer. address_id= address. address_id
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *customer* | NULL | ALL | idx\_fk\_address\_id | NULL | NULL | NULL | 599 | ☠️ **100.00%** | ☠️ **O(n)** | NULL |
| 1 | SIMPLE | *address* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.customer.address\_id | 1 | ☠️ **100.00%** | O(log n) | Using index |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* **eq_ref**: 除const类型外最好的可能实现的连接类型. 它用在一个索引的所有部分被连接使用并且索引是UNIQUE或PRIMARY KEY, 对于每个索引键, 表中只有一条记录与之匹配. 例: 'SELECT * FROM RefTbl, tbl WHERE RefTbl.col=tbl.col;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using index**: 只需通过索引就可以从表中获取列的信息, 无需额外去读取真实的行数据. 如果查询使用的列值仅仅是一个简单索引的部分值, 则会使用这种策略来优化查询.
# Query: E48A20D0413512DA
★ ★ ☆ ☆ ☆ 50分
```sql
SELECT
ID, name
FROM
(
SELECT
address
FROM
customer_list
WHERE
SID= 1
ORDER BY
phone
LIMIT
50, 10) a
JOIN customer_list l ON (a. address= l. address)
JOIN city c ON (c. city= l. city)
ORDER BY
phone desc
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | PRIMARY | *country* | NULL | index | PRIMARY | PRIMARY | 2 | NULL | 109 | ☠️ **100.00%** | ☠️ **O(n)** | Using index; Using temporary; Using filesort |
| 1 | PRIMARY | *city* | NULL | ref | PRIMARY,
idx\_fk\_country\_id | idx\_fk\_country\_id | 2 | sakila.country.country\_id | 5 | ☠️ **100.00%** | O(log n) | NULL |
| 1 | PRIMARY | *c* | NULL | ALL | NULL | NULL | NULL | NULL | 600 | n% | ☠️ **O(n)** | Using where; Using join buffer (hash join) |
| 1 | PRIMARY | *a* | NULL | ref | PRIMARY,
idx\_fk\_city\_id | idx\_fk\_city\_id | 2 | sakila.city.city\_id | 1 | ☠️ **100.00%** | O(log n) | NULL |
| 1 | PRIMARY | *cu* | NULL | ref | idx\_fk\_address\_id | idx\_fk\_address\_id | 2 | sakila.a.address\_id | 1 | ☠️ **100.00%** | O(log n) | NULL |
| 1 | PRIMARY | *
idx\_fk\_city\_id | NULL | NULL | NULL | 603 | ☠️ **100.00%** | ☠️ **O(n)** | Using filesort |
| 2 | DERIVED | *cu* | NULL | ref | idx\_fk\_store\_id,
idx\_fk\_address\_id | idx\_fk\_address\_id | 2 | sakila.a.address\_id | 1 | n% | O(log n) | Using where |
| 2 | DERIVED | *city* | NULL | eq\_ref | PRIMARY,
idx\_fk\_country\_id | PRIMARY | 2 | sakila.a.city\_id | 1 | ☠️ **100.00%** | O(log n) | NULL |
| 2 | DERIVED | *country* | NULL | eq\_ref | PRIMARY | PRIMARY | 2 | sakila.city.country\_id | 1 | ☠️ **100.00%** | O(log n) | Using index |
### Explain信息解读
#### SelectType信息解读
* **DERIVED**: 用于from子句里有子查询的情况. MySQL会递归执行这些子查询, 把结果放在临时表里.
* **PRIMARY**: 最外层的select.
#### Type信息解读
* **eq_ref**: 除const类型外最好的可能实现的连接类型. 它用在一个索引的所有部分被连接使用并且索引是UNIQUE或PRIMARY KEY, 对于每个索引键, 表中只有一条记录与之匹配. 例: 'SELECT * FROM RefTbl, tbl WHERE RefTbl.col=tbl.col;'.
* **index**: 全表扫描, 只是扫描表的时候按照索引次序进行而不是行. 主要优点就是避免了排序, 但是开销仍然非常大.
* **ref**: 连接不能基于关键字选择单个行, 可能查找到多个符合条件的行. 叫做ref是因为索引要跟某个参考值相比较. 这个参考值或者是一个数, 或者是来自一个表里的多表查询的结果值. 例:'SELECT * FROM tbl WHERE idx_col=expr;'.
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using index**: 只需通过索引就可以从表中获取列的信息, 无需额外去读取真实的行数据. 如果查询使用的列值仅仅是一个简单索引的部分值, 则会使用这种策略来优化查询.
* **Using join buffer**: 从已有连接中找被读入缓存的数据, 并且通过缓存来完成与当前表的连接.
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的city表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列city添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`city\` add index \`idx\_city\` (\`city\`) ;
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
## 同一张表被连接两次
* **Item:** JOI.002
* **Severity:** L4
* **Content:** 相同的表在 FROM 子句中至少出现两次，可以简化为对该表的单次访问。
## MySQL 对子查询的优化效果不佳
* **Item:** SUB.001
* **Severity:** L4
* **Content:** MySQL 将外部查询中的每一行作为依赖子查询执行子查询。 这是导致严重性能问题的常见原因。这可能会在 MySQL 5.6 版本中得到改善, 但对于5.1及更早版本, 建议将该类查询分别重写为 JOIN 或 LEFT OUTER JOIN。
# Query: B0BA5A7079EA16B3
★ ★ ★ ★ ☆ 85分
```sql
SELECT
*
FROM
film
WHERE
DATE( last_update) = '2006-02-15'
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | Using where |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
## 避免在 WHERE 条件中使用函数或其他运算符
* **Item:** FUN.001
* **Severity:** L2
* **Content:** 虽然在 SQL 中使用函数可以简化很多复杂的查询，但使用了函数的查询无法利用表中已经建立的索引，该查询将会是全表扫描，性能较差。通常建议将列名写在比较运算符左侧，将查询过滤条件放在比较运算符右侧。也不建议在查询比较条件两侧书写多余的括号，这会对阅读产生比较大的困扰。
# Query: 18A2AD1395A58EAE
★ ★ ★ ☆ ☆ 60分
```sql
SELECT
last_update
FROM
film
GROUP BY
DATE( last_update)
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | Using temporary |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## 请为 GROUP BY 显示添加 ORDER BY 条件
* **Item:** CLA.008
* **Severity:** L2
* **Content:** 默认 MySQL 会对 'GROUP BY col1, col2, ...' 请求按如下顺序排序 'ORDER BY col1, col2, ...'。如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 'ORDER BY NULL'。
## GROUP BY 的条件为表达式
* **Item:** CLA.010
* **Severity:** L2
* **Content:** 当 GROUP BY 条件为表达式或函数时会使用到临时表，如果在未指定 WHERE 或 WHERE 条件返回的结果集较大时性能会很差。
# Query: 60F234BA33AAC132
★ ★ ★ ☆ ☆ 70分
```sql
SELECT
last_update
FROM
film
ORDER BY
DATE( last_update)
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | ☠️ **100.00%** | ☠️ **O(n)** | Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 最外层 SELECT 未指定 WHERE 条件
* **Item:** CLA.001
* **Severity:** L4
* **Content:** SELECT 语句没有 WHERE 子句，可能检查比预期更多的行(全表扫描)。对于 SELECT COUNT(\*) 类型的请求如果不要求精度，建议使用 SHOW TABLE STATUS 或 EXPLAIN 替代。
## ORDER BY 的条件为表达式
* **Item:** CLA.009
* **Severity:** L2
* **Content:** 当 ORDER BY 条件为表达式或函数时会使用到临时表，如果在未指定 WHERE 或 WHERE 条件返回的结果集较大时性能会很差。
# Query: 1ED2B7ECBA4215E1
★ ★ ★ ★ ☆ 80分
```sql
SELECT
description
FROM
film
WHERE
description IN( 'NEWS',
'asd'
)
GROUP BY
description
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *film* | NULL | ALL | NULL | NULL | NULL | NULL | 1000 | n% | ☠️ **O(n)** | Using where; Using temporary |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using temporary**: 表示MySQL在对查询结果排序时使用临时表. 常见于排序order by和分组查询group by.
## 为sakila库的film表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列description添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`film\` add index \`idx\_description\` (\`description\`(191)) ;
## 请为 GROUP BY 显示添加 ORDER BY 条件
* **Item:** CLA.008
* **Severity:** L2
* **Content:** 默认 MySQL 会对 'GROUP BY col1, col2, ...' 请求按如下顺序排序 'ORDER BY col1, col2, ...'。如果 GROUP BY 语句不指定 ORDER BY 条件会导致无谓的排序产生，如果不需要排序建议添加 'ORDER BY NULL'。
# Query: 255BAC03F56CDBC7
★ ★ ★ ★ ★ 100分
```sql
ALTER TABLE
address
ADD
index idx_city_id( city_id)
```
## OK
# Query: C315BC4EE0F4E523
★ ★ ★ ★ ★ 100分
```sql
ALTER TABLE
inventory
ADD
index `idx_store_film` (
`store_id`, `film_id`)
```
## 提醒：请将索引属性顺序与查询对齐
* **Item:** KEY.004
* **Severity:** L0
* **Content:** 如果为列创建复合索引，请确保查询属性与索引属性的顺序相同，以便DBMS在处理查询时使用索引。如果查询和索引属性订单没有对齐，那么DBMS可能无法在查询处理期间使用索引。
# Query: 9BB74D074BA0727C
★ ★ ★ ★ ☆ 90分
```sql
ALTER TABLE
inventory
ADD
index `idx_store_film` (
`store_id`, `film_id`),
ADD
index `idx_store_film` (
`store_id`, `film_id`),
ADD
index `idx_store_film` (
`store_id`, `film_id`)
```
## 索引名称已存在
* **Item:** IDX.001
* **Severity:** L2
* **Content:** Duplicate key name 'idx\_store\_film'
* **Case:** alter table inventory add index \`idx\_store\_film\` (\`store\_id\`,\`film\_id\`),add index \`idx\_store\_film\` (\`store\_id\`,\`film\_id\`),add index \`idx\_store\_film\` (\`store\_id\`,\`film\_id\`)
## 提醒：请将索引属性顺序与查询对齐
* **Item:** KEY.004
* **Severity:** L0
* **Content:** 如果为列创建复合索引，请确保查询属性与索引属性的顺序相同，以便DBMS在处理查询时使用索引。如果查询和索引属性订单没有对齐，那么DBMS可能无法在查询处理期间使用索引。
# Query: C95B5C028C8FFF95
★ ☆ ☆ ☆ ☆ 30分
```sql
SELECT
DATE_FORMAT( t. last_update, '%Y-%m-%d'
),
COUNT( DISTINCT (
t. city))
FROM
city t
WHERE
t. last_update > '2018-10-22 00:00:00'
AND t. city LIKE '%Chrome%'
AND t. city = 'eip'
GROUP BY
DATE_FORMAT( t. last_update, '%Y-%m-%d'
)
ORDER BY
DATE_FORMAT( t. last_update, '%Y-%m-%d'
)
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SIMPLE | *t* | NULL | ALL | NULL | NULL | NULL | NULL | 600 | n% | ☠️ **O(n)** | Using where; Using filesort |
### Explain信息解读
#### SelectType信息解读
* **SIMPLE**: 简单SELECT(不使用UNION或子查询等).
#### Type信息解读
* ☠️ **ALL**: 最坏的情况, 从头到尾全表扫描.
#### Extra信息解读
* **Using where**: WHERE条件用于筛选出与下一个表匹配的数据然后返回给客户端. 除非故意做的全表扫描, 否则连接类型是ALL或者是index, 且在Extra列的值中没有Using Where, 则该查询可能是有问题的.
* ☠️ **Using filesort**: MySQL会对结果使用一个外部索引排序,而不是从表里按照索引次序读到相关内容. 可能在内存或者磁盘上进行排序. MySQL中无法利用索引完成的排序操作称为'文件排序'.
## 为sakila库的city表添加索引
* **Item:** IDX.001
* **Severity:** L2
* **Content:** 为列city添加索引，散粒度为: n%; 为列last\_update添加索引，散粒度为: n%;
* **Case:** ALTER TABLE \`sakila\`.\`city\` add index \`idx\_city\_last\_update\` (\`city\`,\`last\_update\`) ;
## 建议使用 AS 关键字显示声明一个别名
* **Item:** ALI.001
* **Severity:** L0
* **Content:** 在列或表别名(如"tbl AS alias")中, 明确使用 AS 关键字比隐含别名(如"tbl alias")更易懂。
## 不建议使用前项通配符查找
* **Item:** ARG.001
* **Severity:** L4
* **Content:** 例如 "％foo"，查询参数有一个前项通配符的情况无法使用已有索引。
## ORDER BY 的条件为表达式
* **Item:** CLA.009
* **Severity:** L2
* **Content:** 当 ORDER BY 条件为表达式或函数时会使用到临时表，如果在未指定 WHERE 或 WHERE 条件返回的结果集较大时性能会很差。
## GROUP BY 的条件为表达式
* **Item:** CLA.010
* **Severity:** L2
* **Content:** 当 GROUP BY 条件为表达式或函数时会使用到临时表，如果在未指定 WHERE 或 WHERE 条件返回的结果集较大时性能会很差。
## ORDER BY 多个列但排序方向不同时可能无法使用索引
* **Item:** KEY.008
* **Severity:** L4
* **Content:** 在 MySQL 8.0 之前当 ORDER BY 多个列指定的排序方向不同时将无法使用已经建立的索引。
# Query: C11ECE7AE5F80CE5
★ ★ ☆ ☆ ☆ 45分
```sql
create table hello. t (id int unsigned)
```
## 建议为表添加注释
* **Item:** CLA.011
* **Severity:** L1
* **Content:** 为表添加注释能够使得表的意义更明确，从而为日后的维护带来极大的便利。
## 请为列添加默认值
* **Item:** COL.004
* **Severity:** L1
* **Content:** 请为列添加默认值，如果是 ALTER 操作，请不要忘记将原字段的默认值写上。字段无默认值，当表较大时无法在线变更表结构。
## 列未添加注释
* **Item:** COL.005
* **Severity:** L1
* **Content:** 建议对表中每个列添加注释，来明确每个列在表中的含义及作用。
## 未指定主键或主键非 int 或 bigint
* **Item:** KEY.007
* **Severity:** L4
* **Content:** 未指定主键或主键非 int 或 bigint，建议将主键设置为 int unsigned 或 bigint unsigned。
## 请为表选择合适的存储引擎
* **Item:** TBL.002
* **Severity:** L4
* **Content:** 建表或修改表的存储引擎时建议使用推荐的存储引擎，如：innodb
# Query: 291F95B7DCB74C21
★ ★ ★ ★ ☆ 95分
```sql
SELECT
*
FROM
tb
WHERE
data >= ''
```
## 不建议使用 SELECT * 类型查询
* **Item:** COL.001
* **Severity:** L1
* **Content:** 当表结构变更时，使用 \* 通配符选择所有列将导致查询的含义和行为会发生更改，可能导致查询返回更多的数据。
# Query: 084DA3E3EE38DD85
★ ★ ★ ★ ★ 100分
```sql
ALTER TABLE
tb alter column id
DROP
DEFAULT
```
# Query: B48292EDB9D0E010
★ ★ ☆ ☆ ☆ 40分
```sql
SELECT
maxId, minId
FROM
(
SELECT
MAX( film_id) maxId, MIN( film_id) minId
FROM
film
WHERE
last_update > '2016-03-27 02:01:01'
) as d
```
## Explain信息
| id | select\_type | table | partitions | type | possible_keys | key | key\_len | ref | rows | filtered | scalability | Extra |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | PRIMARY | *