[ { "path": ".gitignore", "content": "TAGS\ntags\n.*.swp\n\n/confl-fuzz.zip\n/fuzz\n" }, { "path": ".travis.yml", "content": "language: go\n\ngo:\n - 1.9.x\n - 1.10.x\n\nbefore_install:\n - go get -t -v ./...\n\nscript:\n - bash go.test.sh\n\nafter_success:\n - bash <(curl -s https://codecov.io/bash)" }, { "path": "LICENSE", "content": "The MIT License (MIT)\n\nCopyright (c) 2012-2013 Apcera Inc\nCopyright (c) 2014 Lytics Inc.\nCopyright (c) 2013-2014 https://github.com/BurntSushi\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of\nthis software and associated documentation files (the \"Software\"), to deal in\nthe Software without restriction, including without limitation the rights to\nuse, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of\nthe Software, and to permit persons to whom the Software is furnished to do so,\nsubject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS\nFOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR\nCOPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER\nIN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\nCONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n" }, { "path": "README.md", "content": "## Yet another Config Parser for go\n\nThis is a config parser most similar to Nginx, supports\nJson format, but with line-breaks, comments, etc. Also, like\nNginx is more lenient.\n\nUses same syntax as https://github.com/vstakhov/libucl\n\n[![Code Coverage](https://codecov.io/gh/lytics/confl/branch/master/graph/badge.svg)](https://codecov.io/gh/lytics/confl)\n[![GoDoc](https://godoc.org/github.com/lytics/confl?status.svg)](http://godoc.org/github.com/lytics/confl)\n[![Build Status](https://travis-ci.org/lytics/confl.svg?branch=master)](https://travis-ci.org/lytics/confl)\n[![Go ReportCard](https://goreportcard.com/badge/lytics/confl)](https://goreportcard.com/report/lytics/confl)\n\n\n\nUse [SublimeText Nginx Plugin](https://github.com/brandonwamboldt/sublime-nginx) for formatting.\n\nCredit to [BurntSushi/Toml](https://github.com/BurntSushi/toml) and [Apcera/Gnatsd](https://github.com/apcera/gnatsd/tree/master/conf) from which \nthis was derived.\n\n### Other Options\nThere are a variety of options for config with comments in Go, this project started before some of them which are now probably better options:\n* https://github.com/hjson/hjson-go\n * Similar to above? https://github.com/tailscale/hujson \n* https://github.com/lalamove/konfig \n* https://github.com/hashicorp/hcl\n\n### Example\n\n```\n# nice, a config with comments!\n\n# support the name = value format\ntitle = \"conf Example\"\n# support json semicolon\ntitle2 : \"conf example2\"\n# support omitting = or : because key starts a line\ntitle3 \"conf example\"\n# note, we do not have to have quotes\ntitle4 = Without Quotes\n\n# for Sections we can use brackets\nhand {\n name = \"Tyrion\"\n organization = \"Lannisters\"\n bio = \"Imp\" // comments on fields\n dob = 1979-05-27T07:32:00Z # dates, and more comments on fields\n}\n\n// Note, double-slash comment\n// section name/value that is quoted and json valid, including commas\naddress : {\n \"street\" : \"1 Sky Cell\",\n \"city\" : \"Eyre\",\n \"region\" : \"Vale of Arryn\",\n \"country\" : \"Westeros\"\n}\n\n# sections can omit the colon, equal before bracket \nseenwith {\n # nested section\n # can be spaces or tabs for nesting\n jaime : {\n season = season1\n episode = \"episode1\"\n }\n\n cersei = {\n season = season1\n episode = \"episode1\"\n }\n\n}\n\n\n# Line breaks are OK when inside arrays\nseasons = [\n \"season1\",\n \"season2\",\n \"season3\",\n \"season4\",\n \"???\"\n]\n\n\n# long strings can use parens to allow multi-line\ndescription (\n we possibly\n can have\n multi line text with a block paren\n block ends with end paren on new line\n)\n\n\n\n```\n\nAnd the corresponding Go types are:\n\n```go\ntype Config struct {\n\tTitle string\n\tHand HandOfKing\n\tLocation *Address `confl:\"address\"`\n\tSeenwith map[string]Character\n\tSeasons []string\n\tDescription string\n}\n\ntype HandOfKing struct {\n\tName string\n\tOrg string `json:\"organization\"` // Reads either confl, or json attributes\n\tBio string\n\tDOB time.Time\n\tDeceased bool\n}\n\ntype Address struct {\n\tStreet string\n\tCity string\n\tRegion string\n\tZipCode int\n}\n\ntype Character struct {\n\tEpisode string\n\tSeason string\n}\n```\n\nNote that a case insensitive match will be tried if an exact match can't be\nfound.\n\nA working example of the above can be found in `_examples/example.{go,conf}`.\n\n\n\n### Examples\n\nThis package works similarly to how the Go standard library handles `XML`\nand `JSON`. Namely, data is loaded into Go values via reflection.\n\nFor the simplest example, consider a file as just a list of keys\nand values:\n\n```\n// Comments in Config\nAge = 25\n# another comment\nCats = [ \"Cauchy\", \"Plato\" ]\n# now, using quotes on key\n\"Pi\" = 3.14\nPerfection = [ 6, 28, 496, 8128 ]\nDOB = 1987-07-05T05:45:00Z\n```\n\nWhich could be defined in Go as:\n\n```go\ntype Config struct {\n Age int\n Cats []string\n Pi float64\n Perfection []int\n DOB time.Time \n}\n```\n\nAnd then decoded with:\n\n```go\nvar conf Config\nif err := confl.Unmarshal(byteData, &conf); err != nil {\n // handle error\n}\n```\n\nYou can also use struct tags if your struct field name doesn't map to a confl\nkey value directly:\n\n```\nsome_key_NAME = \"wat\"\n```\n\n```go\ntype Config struct {\n ObscureKey string `confl:\"some_key_NAME\"`\n}\n```\n\n### Using the `encoding.TextUnmarshaler` interface\n\nHere's an example that automatically parses duration strings into \n`time.Duration` values:\n\n```\nsong [\n\t{\n\t\tname = \"Thunder Road\"\n\t\tduration = \"4m49s\"\n\t},\n\t{\n\t\tname = \"Stairway to Heaven\"\n\t\tduration = \"8m03s\"\n\t}\n]\n```\n\nWhich can be decoded with:\n\n```go\ntype song struct {\n Name string\n Duration duration\n}\ntype songs struct {\n Song []song\n}\nvar favorites songs\nif err := confl.Unmarshal(blob, &favorites); err != nil {\n log.Fatal(err)\n}\n\nfor _, s := range favorites.Song {\n fmt.Printf(\"%s (%s)\\n\", s.Name, s.Duration)\n}\n```\n\nAnd you'll also need a `duration` type that satisfies the \n`encoding.TextUnmarshaler` interface:\n\n```go\ntype duration struct {\n\ttime.Duration\n}\n\nfunc (d *duration) UnmarshalText(text []byte) error {\n\tvar err error\n\td.Duration, err = time.ParseDuration(string(text))\n\treturn err\n}\n```\n\n" }, { "path": "_examples/example.conf", "content": "# Nice, we have comments\n\ntitle = \"Confl Example\"\n\nhand {\n name = \"Tyrion\"\n organization = \"Lannisters\"\n bio = \"Imp\" // comments on fields\n dob = 1979-05-27T07:32:00Z # dates, and more comments on fields\n}\n\n// Now, something that is json esque (but with comments)\naddress : {\n \"street\": \"1 Sky Cell\", // the street presumably\n \"city\": \"Eyre\", // not always here but....\n \"region\": \"Vale of Arryn\",\n \"country\": \"Westeros\"\n}\n\nseenwith {\n # You can indent as you please. Tabs or spaces\n jaime {\n season = season1\n episode = \"episode1\"\n }\n\n cersei {\n season = season1\n episode = \"episode1\"\n }\n\n}\n\n\n# Line breaks are OK when inside arrays\nseasons = [\n \"season1\",\n \"season2\",\n \"season3\",\n \"season4\",\n \"???\"\n]\n\n\ndescription (\n we possibly\n can have\n multi line text with a block paren\n block ends with end paren on new line\n)" }, { "path": "_examples/example.go", "content": "package main\n\nimport (\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/araddon/gou\"\n\n\t\"github.com/lytics/confl\"\n)\n\ntype Config struct {\n\tTitle string\n\tHand handOfKing\n\tLocation address `confl:\"address\"`\n\tSeenwith map[string]character\n\tSeasons []string\n\tDescription string\n}\n\n/*\nhand {\n name = \"Tyrion\"\n organization = \"Lannisters\"\n bio = \"Imp\" // comments on fields\n dob = 1979-05-27T07:32:00Z # dates, and more comments on fields\n}\n*/\ntype handOfKing struct {\n\tName string\n\tOrg string `confl:\"organization\"`\n\tBio string\n\tDOB time.Time\n\tDeceased bool\n}\n\ntype address struct {\n\tStreet string\n\tCity string\n\tRegion string\n\tZipCode int\n}\n\ntype character struct {\n\tEpisode string\n\tSeason string\n}\n\nfunc main() {\n\tgou.SetupLogging(\"debug\")\n\tgou.SetColorOutput()\n\n\tvar config Config\n\tif _, err := confl.DecodeFile(\"example.conf\", &config); err != nil {\n\t\tfmt.Println(err)\n\t\treturn\n\t}\n\n\tfmt.Printf(\"Title: %s\\n\", config.Title)\n\tfmt.Printf(\"Hand: %s %s, %s. Born: %s, Deceased? %v\\n\",\n\t\tconfig.Hand.Name, config.Hand.Org, config.Hand.Bio, config.Hand.DOB, config.Hand.Deceased)\n\tfmt.Printf(\"Location: %#v\\n\", config.Location)\n\tfor name, person := range config.Seenwith {\n\t\tfmt.Printf(\"Seen With: %s (%s, %s)\\n\", name, person.Episode, person.Season)\n\t}\n\tfmt.Printf(\"Seasons: %v\\n\", config.Seasons)\n\tfmt.Printf(\"Description: %v\\n\", config.Description)\n}\n" }, { "path": "cmd/conflv/COPYING", "content": " DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE\n Version 2, December 2004\n\n Copyright (C) 2004 Sam Hocevar \n\n Everyone is permitted to copy and distribute verbatim or modified\n copies of this license document, and changing it is allowed as long\n as the name is changed.\n\n DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE\n TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION\n\n 0. You just DO WHAT THE FUCK YOU WANT TO.\n\n" }, { "path": "cmd/conflv/README.md", "content": "# CONFL Validator\n\nIf Go is installed, it's simple to try it out:\n\n```bash\ngo get github.com/lytics/confl/cmd/conflv\nconflv myconf.conf\n```\n\nYou can see the types of every key in a conf file with:\n\n```bash\nconflv -types the.conf\n```\n\n\n" }, { "path": "cmd/conflv/main.go", "content": "package main\n\nimport (\n\t\"flag\"\n\t\"fmt\"\n\t\"log\"\n\t\"os\"\n\t\"path\"\n\t\"strings\"\n\t\"text/tabwriter\"\n\n\t\"github.com/lytics/confl\"\n)\n\nvar (\n\tflagTypes = false\n)\n\nfunc init() {\n\tlog.SetFlags(0)\n\n\tflag.BoolVar(&flagTypes, \"types\", flagTypes,\n\t\t\"When set, the types of every defined key will be shown.\")\n\n\tflag.Usage = usage\n\tflag.Parse()\n}\n\nfunc usage() {\n\tlog.Printf(\"Usage: %s name.conf [ file2 ... ]\\n\",\n\t\tpath.Base(os.Args[0]))\n\tflag.PrintDefaults()\n\n\tos.Exit(1)\n}\n\nfunc main() {\n\tif flag.NArg() < 1 {\n\t\tflag.Usage()\n\t}\n\tfor _, f := range flag.Args() {\n\t\tvar tmp interface{}\n\t\tmd, err := confl.DecodeFile(f, &tmp)\n\t\tif err != nil {\n\t\t\tlog.Fatalf(\"Error in '%s': %s\", f, err)\n\t\t}\n\t\tif flagTypes {\n\t\t\tprintTypes(md)\n\t\t}\n\t}\n}\n\nfunc printTypes(md confl.MetaData) {\n\ttabw := tabwriter.NewWriter(os.Stdout, 0, 0, 2, ' ', 0)\n\tfor _, key := range md.Keys() {\n\t\tfmt.Fprintf(tabw, \"%s%s\\t%s\\n\",\n\t\t\tstrings.Repeat(\" \", len(key)-1), key, md.Type(key...))\n\t}\n\ttabw.Flush()\n}\n" }, { "path": "codecov.yml", "content": "ignore:\n - \"cmd/*\"\n - \"_examples/*\"\n" }, { "path": "decode.go", "content": "package confl\n\nimport (\n\t\"fmt\"\n\tu \"github.com/araddon/gou\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"math\"\n\t\"reflect\"\n\t\"strings\"\n\t\"time\"\n)\n\nvar _ = u.EMPTY\nvar e = fmt.Errorf\n\n// Primitive is a value that hasn't been decoded into a Go value.\n// When using the various `Decode*` functions, the type `Primitive` may\n// be given to any value, and its decoding will be delayed.\n//\n// A `Primitive` value can be decoded using the `PrimitiveDecode` function.\n//\n// The underlying representation of a `Primitive` value is subject to change.\n// Do not rely on it.\n//\n// N.B. Primitive values are still parsed, so using them will only avoid\n// the overhead of reflection. They can be useful when you don't know the\n// exact type of data until run time.\ntype Primitive struct {\n\tundecoded interface{}\n\tcontext Key\n}\n\n// PrimitiveDecode is just like the other `Decode*` functions, except it\n// decodes a confl value that has already been parsed. Valid primitive values\n// can *only* be obtained from values filled by the decoder functions,\n// including this method. (i.e., `v` may contain more `Primitive`\n// values.)\n//\n// Meta data for primitive values is included in the meta data returned by\n// the `Decode*` functions with one exception: keys returned by the Undecoded\n// method will only reflect keys that were decoded. Namely, any keys hidden\n// behind a Primitive will be considered undecoded. Executing this method will\n// update the undecoded keys in the meta data. (See the example.)\nfunc (md *MetaData) PrimitiveDecode(primValue Primitive, v interface{}) error {\n\tmd.context = primValue.context\n\tdefer func() { md.context = nil }()\n\treturn md.unify(primValue.undecoded, rvalue(v))\n}\n\ntype Decoder struct {\n\treader io.Reader\n}\n\nfunc NewDecoder(r io.Reader) *Decoder {\n\treturn &Decoder{r}\n}\n\nfunc (dec *Decoder) Decode(v interface{}) error {\n\tbs, err := ioutil.ReadAll(dec.reader)\n\tif err != nil {\n\t\treturn err\n\t}\n\t_, err = Decode(string(bs), v)\n\treturn err\n}\n\nfunc Unmarshal(bs []byte, v interface{}) error {\n\t_, err := Decode(string(bs), v)\n\treturn err\n}\n\n// Decode will decode the contents of `data` in confl format into a pointer\n// `v`.\n//\n// confl hashes correspond to Go structs or maps. (Dealer's choice. They can be\n// used interchangeably.)\n//\n// confl arrays of tables correspond to either a slice of structs or a slice\n// of maps.\n//\n// confl datetimes correspond to Go `time.Time` values.\n//\n// All other confl types (float, string, int, bool and array) correspond\n// to the obvious Go types.\n//\n// An exception to the above rules is if a type implements the\n// encoding.TextUnmarshaler interface. In this case, any primitive confl value\n// (floats, strings, integers, booleans and datetimes) will be converted to\n// a byte string and given to the value's UnmarshalText method. See the\n// Unmarshaler example for a demonstration with time duration strings.\n//\n// Key mapping\n//\n// confl keys can map to either keys in a Go map or field names in a Go\n// struct. The special `confl` struct tag may be used to map confl keys to\n// struct fields that don't match the key name exactly. (See the example.)\n// A case insensitive match to struct names will be tried if an exact match\n// can't be found.\n//\n// The mapping between confl values and Go values is loose. That is, there\n// may exist confl values that cannot be placed into your representation, and\n// there may be parts of your representation that do not correspond to\n// confl values. This loose mapping can be made stricter by using the IsDefined\n// and/or Undecoded methods on the MetaData returned.\n//\n// This decoder will not handle cyclic types. If a cyclic type is passed,\n// `Decode` will not terminate.\nfunc Decode(data string, v interface{}) (MetaData, error) {\n\tp, err := parse(data)\n\tif err != nil {\n\t\treturn MetaData{}, err\n\t}\n\tmd := MetaData{\n\t\tp.mapping, p.types, p.ordered,\n\t\tmake(map[string]bool, len(p.ordered)), nil,\n\t}\n\treturn md, md.unify(p.mapping, rvalue(v))\n}\n\n// DecodeFile is just like Decode, except it will automatically read the\n// contents of the file at `fpath` and decode it for you.\nfunc DecodeFile(fpath string, v interface{}) (MetaData, error) {\n\tbs, err := ioutil.ReadFile(fpath)\n\tif err != nil {\n\t\treturn MetaData{}, err\n\t}\n\treturn Decode(string(bs), v)\n}\n\n// DecodeReader is just like Decode, except it will consume all bytes\n// from the reader and decode it for you.\nfunc DecodeReader(r io.Reader, v interface{}) (MetaData, error) {\n\tbs, err := ioutil.ReadAll(r)\n\tif err != nil {\n\t\treturn MetaData{}, err\n\t}\n\treturn Decode(string(bs), v)\n}\n\n// unify performs a sort of type unification based on the structure of `rv`,\n// which is the client representation.\n//\n// Any type mismatch produces an error. Finding a type that we don't know\n// how to handle produces an unsupported type error.\nfunc (md *MetaData) unify(data interface{}, rv reflect.Value) error {\n\t// Special case. Look for a `Primitive` value.\n\tif rv.Type() == reflect.TypeOf((*Primitive)(nil)).Elem() {\n\t\t// Save the undecoded data and the key context into the primitive\n\t\t// value.\n\t\tcontext := make(Key, len(md.context))\n\t\tcopy(context, md.context)\n\t\trv.Set(reflect.ValueOf(Primitive{\n\t\t\tundecoded: data,\n\t\t\tcontext: context,\n\t\t}))\n\t\treturn nil\n\t}\n\n\t// Special case. Handle time.Time values specifically.\n\t// TODO: Remove this code when we decide to drop support for Go 1.1.\n\t// This isn't necessary in Go 1.2 because time.Time satisfies the encoding\n\t// interfaces.\n\tif rv.Type().AssignableTo(rvalue(time.Time{}).Type()) {\n\t\treturn md.unifyDatetime(data, rv)\n\t}\n\n\t// Special case. Look for a value satisfying the TextUnmarshaler interface.\n\tif v, ok := rv.Interface().(TextUnmarshaler); ok {\n\t\treturn md.unifyText(data, v)\n\t}\n\t// BUG(burntsushi)\n\t// The behavior here is incorrect whenever a Go type satisfies the\n\t// encoding.TextUnmarshaler interface but also corresponds to a conf\n\t// hash or array. In particular, the unmarshaler should only be applied\n\t// to primitive conf values. But at this point, it will be applied to\n\t// all kinds of values and produce an incorrect error whenever those values\n\t// are hashes or arrays (including arrays of tables).\n\n\tk := rv.Kind()\n\n\t// laziness\n\tif k >= reflect.Int && k <= reflect.Uint64 {\n\t\treturn md.unifyInt(data, rv)\n\t}\n\tswitch k {\n\tcase reflect.Ptr:\n\t\telem := reflect.New(rv.Type().Elem())\n\t\terr := md.unify(data, reflect.Indirect(elem))\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\trv.Set(elem)\n\t\treturn nil\n\tcase reflect.Struct:\n\t\treturn md.unifyStruct(data, rv)\n\tcase reflect.Map:\n\t\treturn md.unifyMap(data, rv)\n\tcase reflect.Array:\n\t\treturn md.unifyArray(data, rv)\n\tcase reflect.Slice:\n\t\treturn md.unifySlice(data, rv)\n\tcase reflect.String:\n\t\treturn md.unifyString(data, rv)\n\tcase reflect.Bool:\n\t\treturn md.unifyBool(data, rv)\n\tcase reflect.Interface:\n\t\t// we only support empty interfaces.\n\t\tif rv.NumMethod() > 0 {\n\t\t\treturn e(\"Unsupported type '%s'.\", rv.Kind())\n\t\t}\n\t\treturn md.unifyAnything(data, rv)\n\tcase reflect.Float32:\n\t\tfallthrough\n\tcase reflect.Float64:\n\t\treturn md.unifyFloat64(data, rv)\n\t}\n\treturn e(\"Unsupported type '%s'.\", rv.Kind())\n}\n\nfunc (md *MetaData) unifyStruct(mapping interface{}, rv reflect.Value) error {\n\ttmap, ok := mapping.(map[string]interface{})\n\tif !ok {\n\t\treturn mismatch(rv, \"map\", mapping)\n\t}\n\n\tfor key, datum := range tmap {\n\t\tvar f *field\n\t\tfields := cachedTypeFields(rv.Type())\n\t\tfor i := range fields {\n\t\t\tff := &fields[i]\n\t\t\tif ff.name == key {\n\t\t\t\tf = ff\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tif f == nil && strings.EqualFold(ff.name, key) {\n\t\t\t\tf = ff\n\t\t\t}\n\t\t}\n\t\tif f != nil {\n\t\t\tsubv := rv\n\t\t\tfor _, i := range f.index {\n\t\t\t\tsubv = indirect(subv.Field(i))\n\t\t\t}\n\t\t\tif isUnifiable(subv) {\n\t\t\t\tmd.decoded[md.context.add(key).String()] = true\n\t\t\t\tmd.context = append(md.context, key)\n\t\t\t\tif err := md.unify(datum, subv); err != nil {\n\t\t\t\t\treturn e(\"Type mismatch for '%s.%s': %s\",\n\t\t\t\t\t\trv.Type().String(), f.name, err)\n\t\t\t\t}\n\t\t\t\tmd.context = md.context[0 : len(md.context)-1]\n\t\t\t} else if f.name != \"\" {\n\t\t\t\t// Bad user! No soup for you!\n\t\t\t\treturn e(\"Field '%s.%s' is unexported, and therefore cannot \"+\n\t\t\t\t\t\"be loaded with reflection.\", rv.Type().String(), f.name)\n\t\t\t}\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (md *MetaData) unifyMap(mapping interface{}, rv reflect.Value) error {\n\ttmap, ok := mapping.(map[string]interface{})\n\tif !ok {\n\t\treturn badtype(\"map\", mapping)\n\t}\n\tif rv.IsNil() {\n\t\trv.Set(reflect.MakeMap(rv.Type()))\n\t}\n\tfor k, v := range tmap {\n\t\tmd.decoded[md.context.add(k).String()] = true\n\t\tmd.context = append(md.context, k)\n\n\t\trvkey := indirect(reflect.New(rv.Type().Key()))\n\t\trvval := reflect.Indirect(reflect.New(rv.Type().Elem()))\n\t\tif err := md.unify(v, rvval); err != nil {\n\t\t\treturn err\n\t\t}\n\t\tmd.context = md.context[0 : len(md.context)-1]\n\n\t\trvkey.SetString(k)\n\t\trv.SetMapIndex(rvkey, rvval)\n\t}\n\treturn nil\n}\n\nfunc (md *MetaData) unifyArray(data interface{}, rv reflect.Value) error {\n\tdatav := reflect.ValueOf(data)\n\tif datav.Kind() != reflect.Slice {\n\t\treturn badtype(\"slice\", data)\n\t}\n\tsliceLen := datav.Len()\n\tif sliceLen != rv.Len() {\n\t\treturn e(\"expected array length %d; got array of length %d\",\n\t\t\trv.Len(), sliceLen)\n\t}\n\treturn md.unifySliceArray(datav, rv)\n}\n\nfunc (md *MetaData) unifySlice(data interface{}, rv reflect.Value) error {\n\tdatav := reflect.ValueOf(data)\n\tif datav.Kind() != reflect.Slice {\n\t\treturn badtype(\"slice\", data)\n\t}\n\tsliceLen := datav.Len()\n\tif rv.IsNil() {\n\t\trv.Set(reflect.MakeSlice(rv.Type(), sliceLen, sliceLen))\n\t}\n\treturn md.unifySliceArray(datav, rv)\n}\n\nfunc (md *MetaData) unifySliceArray(data, rv reflect.Value) error {\n\tsliceLen := data.Len()\n\tfor i := 0; i < sliceLen; i++ {\n\t\tv := data.Index(i).Interface()\n\t\tsliceval := indirect(rv.Index(i))\n\t\tif err := md.unify(v, sliceval); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (md *MetaData) unifyDatetime(data interface{}, rv reflect.Value) error {\n\tif _, ok := data.(time.Time); ok {\n\t\trv.Set(reflect.ValueOf(data))\n\t\treturn nil\n\t}\n\treturn badtype(\"time.Time\", data)\n}\n\nfunc (md *MetaData) unifyString(data interface{}, rv reflect.Value) error {\n\tif s, ok := data.(string); ok {\n\t\trv.SetString(s)\n\t\treturn nil\n\t}\n\treturn badtype(\"string\", data)\n}\n\nfunc (md *MetaData) unifyFloat64(data interface{}, rv reflect.Value) error {\n\tif num, ok := data.(float64); ok {\n\t\tswitch rv.Kind() {\n\t\tcase reflect.Float32:\n\t\t\tfallthrough\n\t\tcase reflect.Float64:\n\t\t\trv.SetFloat(num)\n\t\tdefault:\n\t\t\tpanic(\"bug\")\n\t\t}\n\t\treturn nil\n\t}\n\treturn badtype(\"float\", data)\n}\n\nfunc (md *MetaData) unifyInt(data interface{}, rv reflect.Value) error {\n\tif num, ok := data.(int64); ok {\n\t\tif rv.Kind() >= reflect.Int && rv.Kind() <= reflect.Int64 {\n\t\t\tswitch rv.Kind() {\n\t\t\tcase reflect.Int, reflect.Int64:\n\t\t\t\t// No bounds checking necessary.\n\t\t\tcase reflect.Int8:\n\t\t\t\tif num < math.MinInt8 || num > math.MaxInt8 {\n\t\t\t\t\treturn e(\"Value '%d' is out of range for int8.\", num)\n\t\t\t\t}\n\t\t\tcase reflect.Int16:\n\t\t\t\tif num < math.MinInt16 || num > math.MaxInt16 {\n\t\t\t\t\treturn e(\"Value '%d' is out of range for int16.\", num)\n\t\t\t\t}\n\t\t\tcase reflect.Int32:\n\t\t\t\tif num < math.MinInt32 || num > math.MaxInt32 {\n\t\t\t\t\treturn e(\"Value '%d' is out of range for int32.\", num)\n\t\t\t\t}\n\t\t\t}\n\t\t\trv.SetInt(num)\n\t\t} else if rv.Kind() >= reflect.Uint && rv.Kind() <= reflect.Uint64 {\n\t\t\tunum := uint64(num)\n\t\t\tswitch rv.Kind() {\n\t\t\tcase reflect.Uint, reflect.Uint64:\n\t\t\t\t// No bounds checking necessary.\n\t\t\tcase reflect.Uint8:\n\t\t\t\tif num < 0 || unum > math.MaxUint8 {\n\t\t\t\t\treturn e(\"Value '%d' is out of range for uint8.\", num)\n\t\t\t\t}\n\t\t\tcase reflect.Uint16:\n\t\t\t\tif num < 0 || unum > math.MaxUint16 {\n\t\t\t\t\treturn e(\"Value '%d' is out of range for uint16.\", num)\n\t\t\t\t}\n\t\t\tcase reflect.Uint32:\n\t\t\t\tif num < 0 || unum > math.MaxUint32 {\n\t\t\t\t\treturn e(\"Value '%d' is out of range for uint32.\", num)\n\t\t\t\t}\n\t\t\t}\n\t\t\trv.SetUint(unum)\n\t\t} else {\n\t\t\tpanic(\"unreachable\")\n\t\t}\n\t\treturn nil\n\t}\n\treturn badtype(\"integer\", data)\n}\n\nfunc (md *MetaData) unifyBool(data interface{}, rv reflect.Value) error {\n\tif b, ok := data.(bool); ok {\n\t\trv.SetBool(b)\n\t\treturn nil\n\t}\n\treturn badtype(\"boolean\", data)\n}\n\nfunc (md *MetaData) unifyAnything(data interface{}, rv reflect.Value) error {\n\trv.Set(reflect.ValueOf(data))\n\treturn nil\n}\n\nfunc (md *MetaData) unifyText(data interface{}, v TextUnmarshaler) error {\n\tvar s string\n\tswitch sdata := data.(type) {\n\tcase TextMarshaler:\n\t\ttext, err := sdata.MarshalText()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\ts = string(text)\n\tcase fmt.Stringer:\n\t\ts = sdata.String()\n\tcase string:\n\t\ts = sdata\n\tcase bool:\n\t\ts = fmt.Sprintf(\"%v\", sdata)\n\tcase int64:\n\t\ts = fmt.Sprintf(\"%d\", sdata)\n\tcase float64:\n\t\ts = fmt.Sprintf(\"%f\", sdata)\n\tdefault:\n\t\treturn badtype(\"primitive (string-like)\", data)\n\t}\n\tif err := v.UnmarshalText([]byte(s)); err != nil {\n\t\treturn err\n\t}\n\treturn nil\n}\n\n// rvalue returns a reflect.Value of `v`. All pointers are resolved.\nfunc rvalue(v interface{}) reflect.Value {\n\treturn indirect(reflect.ValueOf(v))\n}\n\n// indirect returns the value pointed to by a pointer.\n// Pointers are followed until the value is not a pointer.\n// New values are allocated for each nil pointer.\n//\n// An exception to this rule is if the value satisfies an interface of\n// interest to us (like encoding.TextUnmarshaler).\nfunc indirect(v reflect.Value) reflect.Value {\n\tif v.Kind() != reflect.Ptr {\n\t\tif v.CanAddr() {\n\t\t\tpv := v.Addr()\n\t\t\tif _, ok := pv.Interface().(TextUnmarshaler); ok {\n\t\t\t\treturn pv\n\t\t\t}\n\t\t}\n\t\treturn v\n\t}\n\tif v.IsNil() {\n\t\tv.Set(reflect.New(v.Type().Elem()))\n\t}\n\treturn indirect(reflect.Indirect(v))\n}\n\nfunc isUnifiable(rv reflect.Value) bool {\n\tif rv.CanSet() {\n\t\treturn true\n\t}\n\tif _, ok := rv.Interface().(TextUnmarshaler); ok {\n\t\treturn true\n\t}\n\treturn false\n}\n\nfunc badtype(expected string, data interface{}) error {\n\treturn e(\"Expected %s but found '%T'.\", expected, data)\n}\n\nfunc mismatch(user reflect.Value, expected string, data interface{}) error {\n\treturn e(\"Type mismatch for %s. Expected %s but found '%T'.\",\n\t\tuser.Type().String(), expected, data)\n}\n" }, { "path": "decode_meta.go", "content": "package confl\n\nimport \"strings\"\n\n// MetaData allows access to meta information about data that may not\n// be inferrable via reflection. In particular, whether a key has been defined\n// and the type of a key.\ntype MetaData struct {\n\tmapping map[string]interface{}\n\ttypes map[string]confType\n\tkeys []Key\n\tdecoded map[string]bool\n\tcontext Key // Used only during decoding.\n}\n\n// IsDefined returns true if the key given exists in the data. The key\n// should be specified hierarchially. e.g.,\n//\n//\t// access the key 'a.b.c'\n//\tIsDefined(\"a\", \"b\", \"c\")\n//\n// IsDefined will return false if an empty key given. Keys are case sensitive.\nfunc (md *MetaData) IsDefined(key ...string) bool {\n\tif len(key) == 0 {\n\t\treturn false\n\t}\n\n\tvar hash map[string]interface{}\n\tvar ok bool\n\tvar hashOrVal interface{} = md.mapping\n\tfor _, k := range key {\n\t\tif hash, ok = hashOrVal.(map[string]interface{}); !ok {\n\t\t\treturn false\n\t\t}\n\t\tif hashOrVal, ok = hash[k]; !ok {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn true\n}\n\n// Type returns a string representation of the type of the key specified.\n//\n// Type will return the empty string if given an empty key or a key that\n// does not exist. Keys are case sensitive.\nfunc (md *MetaData) Type(key ...string) string {\n\tfullkey := strings.Join(key, \".\")\n\tif typ, ok := md.types[fullkey]; ok {\n\t\treturn typ.typeString()\n\t}\n\treturn \"\"\n}\n\n// Key is the type of any key, including key groups. Use (MetaData).Keys\n// to get values of this type.\ntype Key []string\n\nfunc (k Key) String() string {\n\treturn strings.Join(k, \".\")\n}\n\nfunc (k Key) add(piece string) Key {\n\tnewKey := make(Key, len(k)+1)\n\tcopy(newKey, k)\n\tnewKey[len(k)] = piece\n\treturn newKey\n}\n\nfunc (k Key) insert(piece string) Key {\n\tnewKey := make(Key, len(k), len(k)+1)\n\tcopy(newKey, k)\n\tinsertedKey := make(Key, 1)\n\tinsertedKey[0] = piece\n\tnewKey = append(insertedKey, newKey...)\n\treturn newKey\n}\n\n// Keys returns a slice of every key in the data, including key groups.\n// Each key is itself a slice, where the first element is the top of the\n// hierarchy and the last is the most specific.\n//\n// The list will have the same order as the keys appeared in the data.\n//\n// All keys returned are non-empty.\nfunc (md *MetaData) Keys() []Key {\n\treturn md.keys\n}\n\n// Undecoded returns all keys that have not been decoded in the order in which\n// they appear in the original document.\n//\n// This includes keys that haven't been decoded because of a Primitive value.\n// Once the Primitive value is decoded, the keys will be considered decoded.\n//\n// Also note that decoding into an empty interface will result in no decoding,\n// and so no keys will be considered decoded.\n//\n// In this sense, the Undecoded keys correspond to keys in the document\n// that do not have a concrete type in your representation.\nfunc (md *MetaData) Undecoded() []Key {\n\tundecoded := make([]Key, 0, len(md.keys))\n\tfor _, key := range md.keys {\n\t\tif !md.decoded[key.String()] {\n\t\t\tundecoded = append(undecoded, key)\n\t\t}\n\t}\n\treturn undecoded\n}\n" }, { "path": "decode_test.go", "content": "package confl\n\nimport (\n\t\"flag\"\n\t\"fmt\"\n\t\"log\"\n\t\"reflect\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\tu \"github.com/araddon/gou\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nfunc init() {\n\tlog.SetFlags(0)\n\tflag.Parse()\n\tif testing.Verbose() {\n\t\tu.SetupLogging(\"debug\")\n\t\tu.SetColorOutput()\n\t}\n}\n\nfunc TestDecodeSimple(t *testing.T) {\n\tvar simpleConfigString = `\nage = 250\nageptr2 = 200\nandrew = \"gallant\"\nkait = \"brady\"\nnow = 1987-07-05T05:45:00Z \nyesOrNo = true\npi = 3.14\ncolors = [\n\t[\"red\", \"green\", \"blue\"],\n\t[\"cyan\", \"magenta\", \"yellow\", \"black\"],\n\t[pink,brown],\n]\n\nmy {\n Cats {\n plato = \"cat 1\"\n cauchy = \"cat 2\"\n }\n}\n\ngames [\n {\n \t// more comments behind tab\n name \"game of thrones\" # we have a comment\n sku \"got\" // another comment, empty line next\n\n }\n {\n name \"settlers of catan\"\n // a comment\n }\n]\n\n`\n\n\ttype cats struct {\n\t\tPlatoAlias string `json:\"plato,omitempty\"`\n\t\tCauchyConflAlias string `confl:\"cauchy\"`\n\t}\n\ttype game struct {\n\t\tName string\n\t\tSku string\n\t}\n\ttype simpleType struct {\n\t\tAge int\n\t\tAgePtr *int\n\t\tAgePtr2 *int\n\t\tColors [][]string\n\t\tPi float64\n\t\tYesOrNo bool\n\t\tNow time.Time\n\t\tAndrew string\n\t\tKait string\n\t\tMy map[string]cats\n\t\tGames []*game\n\t}\n\n\tvar simple simpleType\n\t_, err := Decode(simpleConfigString, &simple)\n\tassert.True(t, err == nil, \"err nil?%v\", err)\n\n\tnow, err := time.Parse(\"2006-01-02T15:04:05\", \"1987-07-05T05:45:00\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tage200 := int(200)\n\tvar answer = simpleType{\n\t\tAge: 250,\n\t\tAgePtr2: &age200,\n\t\tAndrew: \"gallant\",\n\t\tKait: \"brady\",\n\t\tNow: now,\n\t\tYesOrNo: true,\n\t\tPi: 3.14,\n\t\tColors: [][]string{\n\t\t\t{\"red\", \"green\", \"blue\"},\n\t\t\t{\"cyan\", \"magenta\", \"yellow\", \"black\"},\n\t\t\t{\"pink\", \"brown\"},\n\t\t},\n\t\tMy: map[string]cats{\n\t\t\t\"Cats\": cats{PlatoAlias: \"cat 1\", CauchyConflAlias: \"cat 2\"},\n\t\t},\n\t\tGames: []*game{\n\t\t\t&game{\"game of thrones\", \"got\"},\n\t\t\t&game{Name: \"settlers of catan\"},\n\t\t},\n\t}\n\tassert.True(t, simple.AgePtr == nil, \"must have nil ptr\")\n\tif !reflect.DeepEqual(simple, answer) {\n\t\tt.Fatalf(\"Expected\\n-----\\n%#v\\n-----\\nbut got\\n-----\\n%#v\\n\",\n\t\t\tanswer, simple)\n\t}\n\n\t// Now Try decoding using Decoder\n\tvar simpleDec simpleType\n\tdecoder := NewDecoder(strings.NewReader(simpleConfigString))\n\terr = decoder.Decode(&simpleDec)\n\tassert.True(t, err == nil, \"err nil?%v\", err)\n\n\tassert.True(t, simpleDec.AgePtr == nil, \"must have nil ptr\")\n\tif !reflect.DeepEqual(simpleDec, answer) {\n\t\tt.Fatalf(\"Expected\\n-----\\n%#v\\n-----\\nbut got\\n-----\\n%#v\\n\",\n\t\t\tanswer, simpleDec)\n\t}\n\n\t// Now Try decoding using Unmarshal\n\tvar simple2 simpleType\n\terr = Unmarshal([]byte(simpleConfigString), &simple2)\n\tassert.True(t, err == nil, \"err nil?%v\", err)\n\n\tassert.True(t, simple2.AgePtr == nil, \"must have nil ptr\")\n\tif !reflect.DeepEqual(simple2, answer) {\n\t\tt.Fatalf(\"Expected\\n-----\\n%#v\\n-----\\nbut got\\n-----\\n%#v\\n\",\n\t\t\tanswer, simple2)\n\t}\n}\n\nfunc TestDecodeEmbedded(t *testing.T) {\n\ttype Dog struct{ Name string }\n\ttype Age int\n\n\ttests := map[string]struct {\n\t\tinput string\n\t\tdecodeInto interface{}\n\t\twantDecoded interface{}\n\t}{\n\t\t\"embedded struct\": {\n\t\t\tinput: `Name = \"milton\"`,\n\t\t\tdecodeInto: &struct{ Dog }{},\n\t\t\twantDecoded: &struct{ Dog }{Dog{\"milton\"}},\n\t\t},\n\t\t\"embedded non-nil pointer to struct\": {\n\t\t\tinput: `Name = \"milton\"`,\n\t\t\tdecodeInto: &struct{ *Dog }{},\n\t\t\twantDecoded: &struct{ *Dog }{&Dog{\"milton\"}},\n\t\t},\n\t\t\"embedded nil pointer to struct\": {\n\t\t\tinput: ``,\n\t\t\tdecodeInto: &struct{ *Dog }{},\n\t\t\twantDecoded: &struct{ *Dog }{nil},\n\t\t},\n\t\t\"embedded int\": {\n\t\t\tinput: `Age = -5`,\n\t\t\tdecodeInto: &struct{ Age }{},\n\t\t\twantDecoded: &struct{ Age }{-5},\n\t\t},\n\t}\n\n\tfor label, test := range tests {\n\t\t_, err := Decode(test.input, test.decodeInto)\n\t\tif err != nil {\n\t\t\tt.Fatal(err)\n\t\t}\n\t\tif !reflect.DeepEqual(test.wantDecoded, test.decodeInto) {\n\t\t\tt.Errorf(\"%s: want decoded == %+v, got %+v\",\n\t\t\t\tlabel, test.wantDecoded, test.decodeInto)\n\t\t}\n\t}\n}\n\nfunc TestDecodeTableArrays(t *testing.T) {\n\tvar tableArrays = `\nalbums [\n\t{\n\t\tname = \"Born to Run\"\n\t songs [\n\t { name = \"Jungleland\" },\n\t { name = \"Meeting Across the River\" }\n\t\t]\n\t}\n\t{\n\t\tname = \"Born in the USA\"\n \t songs [\n\t { name = \"Glory Days\" },\n\t { name = \"Dancing in the Dark\" }\n\t ]\n }\n]`\n\n\ttype Song struct {\n\t\tName string\n\t}\n\n\ttype Album struct {\n\t\tName string\n\t\tSongs []Song\n\t}\n\n\ttype Music struct {\n\t\tAlbums []Album\n\t}\n\n\texpected := Music{[]Album{\n\t\t{\"Born to Run\", []Song{{\"Jungleland\"}, {\"Meeting Across the River\"}}},\n\t\t{\"Born in the USA\", []Song{{\"Glory Days\"}, {\"Dancing in the Dark\"}}},\n\t}}\n\tvar got Music\n\tif _, err := Decode(tableArrays, &got); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif !reflect.DeepEqual(expected, got) {\n\t\tt.Fatalf(\"\\n%#v\\n!=\\n%#v\\n\", expected, got)\n\t}\n}\n\n// Case insensitive matching tests.\n// A bit more comprehensive than needed given the current implementation,\n// but implementations change.\n// Probably still missing demonstrations of some ugly corner cases regarding\n// case insensitive matching and multiple fields.\nfunc TestDecodeCase(t *testing.T) {\n\tvar caseData = `\ntOpString = \"string\"\ntOpInt = 1\ntOpFloat = 1.1\ntOpBool = true\ntOpdate = 2006-01-02T15:04:05Z\ntOparray = [ \"array\" ]\nMatch = \"i should be in Match only\"\nMatcH = \"i should be in MatcH only\"\nonce = \"just once\"\nnEst {\n\teD {\n\t\tnEstedString = \"another string\"\n\t}\n}\n`\n\n\ttype InsensitiveEd struct {\n\t\tNestedString string\n\t}\n\n\ttype InsensitiveNest struct {\n\t\tEd InsensitiveEd\n\t}\n\n\ttype Insensitive struct {\n\t\tTopString string\n\t\tTopInt int\n\t\tTopFloat float64\n\t\tTopBool bool\n\t\tTopDate time.Time\n\t\tTopArray []string\n\t\tMatch string\n\t\tMatcH string\n\t\tOnce string\n\t\tOncE string\n\t\tNest InsensitiveNest\n\t}\n\n\ttme, err := time.Parse(time.RFC3339, time.RFC3339[:len(time.RFC3339)-5])\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\texpected := Insensitive{\n\t\tTopString: \"string\",\n\t\tTopInt: 1,\n\t\tTopFloat: 1.1,\n\t\tTopBool: true,\n\t\tTopDate: tme,\n\t\tTopArray: []string{\"array\"},\n\t\tMatcH: \"i should be in MatcH only\",\n\t\tMatch: \"i should be in Match only\",\n\t\tOnce: \"just once\",\n\t\tOncE: \"\",\n\t\tNest: InsensitiveNest{\n\t\t\tEd: InsensitiveEd{NestedString: \"another string\"},\n\t\t},\n\t}\n\tvar got Insensitive\n\tif _, err := Decode(caseData, &got); err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif !reflect.DeepEqual(expected, got) {\n\t\tt.Fatalf(\"\\n%#v\\n!=\\n%#v\\n\", expected, got)\n\t}\n}\n\nfunc TestDecodePointers(t *testing.T) {\n\ttype Object struct {\n\t\tType string\n\t\tDescription string\n\t}\n\n\ttype Dict struct {\n\t\tNamedObject map[string]*Object\n\t\tBaseObject *Object\n\t\tStrptr *string\n\t\tStrptrs []*string\n\t}\n\ts1, s2, s3 := \"blah\", \"abc\", \"def\"\n\texpected := &Dict{\n\t\tStrptr: &s1,\n\t\tStrptrs: []*string{&s2, &s3},\n\t\tNamedObject: map[string]*Object{\n\t\t\t\"foo\": {\"FOO\", \"fooooo!!!\"},\n\t\t\t\"bar\": {\"BAR\", \"ba-ba-ba-ba-barrrr!!!\"},\n\t\t},\n\t\tBaseObject: &Object{\"BASE\", \"da base\"},\n\t}\n\n\tex1 := `\nStrptr = \"blah\"\nStrptrs = [\"abc\", \"def\"]\n\nNamedObject {\n\tfoo {\n Type = \"FOO\"\n Description = \"fooooo!!!\"\n\t}\n\n bar {\n\t\tType = \"BAR\"\n\t\tDescription = \"ba-ba-ba-ba-barrrr!!!\"\n }\n}\n\nBaseObject {\n Type = \"BASE\"\n Description = \"da base\"\n}\n`\n\tdict := new(Dict)\n\t_, err := Decode(ex1, dict)\n\tif err != nil {\n\t\tt.Errorf(\"Decode error: %v\", err)\n\t}\n\tif !reflect.DeepEqual(expected, dict) {\n\t\tt.Fatalf(\"\\n%#v\\n!=\\n%#v\\n\", expected, dict)\n\t}\n}\n\ntype sphere struct {\n\tCenter [3]float64\n\tRadius float64\n}\n\nfunc TestDecodeSimpleArray(t *testing.T) {\n\tvar s1 sphere\n\tif _, err := Decode(`center = [0.0, 1.5, 0.0]`, &s1); err != nil {\n\t\tt.Fatal(err)\n\t}\n}\n\nfunc TestDecodeArrayWrongSize(t *testing.T) {\n\tvar s1 sphere\n\tif _, err := Decode(`center = [0.1, 2.3]`, &s1); err == nil {\n\t\tt.Fatal(\"Expected array type mismatch error\")\n\t}\n}\n\nfunc TestDecodeLargeIntoSmallInt(t *testing.T) {\n\ttype table struct {\n\t\tValue int8\n\t}\n\tvar tab table\n\tif _, err := Decode(`value = 500`, &tab); err == nil {\n\t\tt.Fatal(\"Expected integer out-of-bounds error.\")\n\t}\n}\n\nfunc TestDecodeSizedInts(t *testing.T) {\n\ttype table struct {\n\t\tU8 uint8\n\t\tU16 uint16\n\t\tU32 uint32\n\t\tU64 uint64\n\t\tU uint\n\t\tI8 int8\n\t\tI16 int16\n\t\tI32 int32\n\t\tI64 int64\n\t\tI int\n\t}\n\tanswer := table{1, 1, 1, 1, 1, -1, -1, -1, -1, -1}\n\tconfigStr := `\n\tu8 = 1\n\tu16 = 1\n\tu32 = 1\n\tu64 = 1\n\tu = 1\n\ti8 = -1\n\ti16 = -1\n\ti32 = -1\n\ti64 = -1\n\ti = -1\n\t`\n\tvar tab table\n\tif _, err := Decode(configStr, &tab); err != nil {\n\t\tt.Fatal(err.Error())\n\t}\n\tif answer != tab {\n\t\tt.Fatalf(\"Expected %#v but got %#v\", answer, tab)\n\t}\n}\n\nfunc ExampleMetaData_PrimitiveDecode() {\n\tvar md MetaData\n\tvar err error\n\n\tvar rawData = `\n\nranking = [\"Springsteen\", \"JGeils\"]\n\nbands {\n\tSpringsteen { \n\t\tstarted = 1973\n\t\talbums = [\"Greetings\", \"WIESS\", \"Born to Run\", \"Darkness\"]\n\t}\n\n\tJGeils {\n\t\tstarted = 1970\n\t\talbums = [\"The J. Geils Band\", \"Full House\", \"Blow Your Face Out\"]\n\t}\n}\n`\n\n\ttype band struct {\n\t\tStarted int\n\t\tAlbums []string\n\t}\n\ttype classics struct {\n\t\tRanking []string\n\t\tBands map[string]Primitive\n\t}\n\n\t// Do the initial decode. Reflection is delayed on Primitive values.\n\tvar music classics\n\tif md, err = Decode(rawData, &music); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// MetaData still includes information on Primitive values.\n\tfmt.Printf(\"Is `bands.Springsteen` defined? %v\\n\",\n\t\tmd.IsDefined(\"bands\", \"Springsteen\"))\n\n\t// Decode primitive data into Go values.\n\tfor _, artist := range music.Ranking {\n\t\t// A band is a primitive value, so we need to decode it to get a\n\t\t// real `band` value.\n\t\tprimValue := music.Bands[artist]\n\n\t\tvar aBand band\n\t\tif err = md.PrimitiveDecode(primValue, &aBand); err != nil {\n\t\t\tu.Warnf(\"failed on: \")\n\t\t\tlog.Fatalf(\"Failed on %v %v\", primValue, err)\n\t\t}\n\t\tfmt.Printf(\"%s started in %d.\\n\", artist, aBand.Started)\n\t}\n\t// Check to see if there were any fields left undecoded.\n\t// Note that this won't be empty before decoding the Primitive value!\n\tfmt.Printf(\"Undecoded: %q\\n\", md.Undecoded())\n\n\t// Output:\n\t// Is `bands.Springsteen` defined? true\n\t// Springsteen started in 1973.\n\t// JGeils started in 1970.\n\t// Undecoded: []\n}\n\nfunc ExampleDecode() {\n\tvar rawData = `\n# Some comments.\nalpha {\n\tip = \"10.0.0.1\"\n\t// config section\n\tconfig {\n\t\tPorts = [ 8001, 8002 ]\n\t\tLocation = \"Toronto\"\n\t\tCreated = 1987-07-05T05:45:00Z\n\t}\n}\n\n\nbeta {\n\tip = \"10.0.0.2\"\n\n\tconfig {\n\t\tPorts = [ 9001, 9002 ]\n\t\tLocation = \"New Jersey\"\n\t\tCreated = 1887-01-05T05:55:00Z\n\t}\n}\n\n`\n\n\ttype serverConfig struct {\n\t\tPorts []int\n\t\tLocation string\n\t\tCreated time.Time\n\t}\n\n\ttype server struct {\n\t\tIP string `confl:\"ip\"`\n\t\tConfig serverConfig `confl:\"config\"`\n\t}\n\n\ttype servers map[string]server\n\n\tvar config servers\n\tif _, err := Decode(rawData, &config); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tfor _, name := range []string{\"alpha\", \"beta\"} {\n\t\ts := config[name]\n\t\tfmt.Printf(\"Server: %s (ip: %s) in %s created on %s\\n\",\n\t\t\tname, s.IP, s.Config.Location,\n\t\t\ts.Config.Created.Format(\"2006-01-02\"))\n\t\tfmt.Printf(\"Ports: %v\\n\", s.Config.Ports)\n\t}\n\n\t// Output:\n\t// Server: alpha (ip: 10.0.0.1) in Toronto created on 1987-07-05\n\t// Ports: [8001 8002]\n\t// Server: beta (ip: 10.0.0.2) in New Jersey created on 1887-01-05\n\t// Ports: [9001 9002]\n}\n\ntype duration struct {\n\ttime.Duration\n}\n\nfunc (d *duration) UnmarshalText(text []byte) error {\n\tvar err error\n\td.Duration, err = time.ParseDuration(string(text))\n\treturn err\n}\n\n// Example Unmarshaler shows how to decode strings into your own\n// custom data type.\nfunc Example_unmarshaler() {\n\trawData := `\nsong [\n\t{\n\t\tname = \"Thunder Road\"\n\t\tduration = \"4m49s\"\n\t},\n\t{\n\t\tname = \"Stairway to Heaven\"\n\t\tduration = \"8m03s\"\n\t}\n]\n\n`\n\ttype song struct {\n\t\tName string\n\t\tDuration duration\n\t}\n\ttype songs struct {\n\t\tSong []song\n\t}\n\tvar favorites songs\n\tif _, err := Decode(rawData, &favorites); err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// Code to implement the TextUnmarshaler interface for `duration`:\n\t//\n\t// type duration struct {\n\t// \ttime.Duration\n\t// }\n\t//\n\t// func (d *duration) UnmarshalText(text []byte) error {\n\t// \tvar err error\n\t// \td.Duration, err = time.ParseDuration(string(text))\n\t// \treturn err\n\t// }\n\n\tfor _, s := range favorites.Song {\n\t\tfmt.Printf(\"%s (%s)\\n\", s.Name, s.Duration)\n\t}\n\t// Output:\n\t// Thunder Road (4m49s)\n\t// Stairway to Heaven (8m3s)\n}\n\n// Example StrictDecoding shows how to detect whether there are keys in the\n// config document that weren't decoded into the value given. This is useful\n// for returning an error to the user if they've included extraneous fields\n// in their configuration.\n// func Example_strictDecoding() {\n// \tvar rawData = `\n// key1 = \"value1\"\n// key2 = \"value2\"\n// key3 = \"value3\"\n// `\n// \ttype config struct {\n// \t\tKey1 string\n// \t\tKey3 string\n// \t}\n\n// \tvar conf config\n// \tmd, err := Decode(rawData, &conf)\n// \tif err != nil {\n// \t\tlog.Fatal(err)\n// \t}\n// \tfmt.Printf(\"Undecoded keys: %q\\n\", md.Undecoded())\n// \t// Output:\n// \t// Undecoded keys: [\"key2\"]\n// }\n" }, { "path": "doc.go", "content": "/*\nPackage confl provides facilities for decoding and encoding TOML/NGINX configuration\nfiles via reflection.\n*/\npackage confl\n" }, { "path": "encode.go", "content": "package confl\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"errors\"\n\t\"fmt\"\n\tu \"github.com/araddon/gou\"\n\t\"io\"\n\t\"reflect\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n)\n\ntype encodeError struct{ error }\n\nvar (\n\terrArrayMixedElementTypes = errors.New(\"can't encode array with mixed element types\")\n\terrArrayNilElement = errors.New(\"can't encode array with nil element\")\n\terrNonString = errors.New(\"can't encode a map with non-string key type\")\n\terrAnonNonStruct = errors.New(\"can't encode an anonymous field that is not a struct\")\n\terrArrayNoTable = errors.New(\"array element can't contain a table\")\n\terrNoKey = errors.New(\"top-level values must be a Go map or struct\")\n\terrAnything = errors.New(\"\") // used in testing\n\t_ = u.EMPTY\n)\n\nvar quotedReplacer = strings.NewReplacer(\n\t\"\\t\", \"\\\\t\",\n\t\"\\n\", \"\\\\n\",\n\t\"\\r\", \"\\\\r\",\n\t\"\\\"\", \"\\\\\\\"\",\n\t\"\\\\\", \"\\\\\\\\\",\n)\n\n// Marshall a go struct into bytes\nfunc Marshal(v interface{}) ([]byte, error) {\n\tbuf := bytes.Buffer{}\n\tenc := NewEncoder(&buf)\n\terr := enc.Encode(v)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn buf.Bytes(), nil\n}\n\n// Encoder controls the encoding of Go values to a document to some\n// io.Writer.\n//\n// The indentation level can be controlled with the Indent field.\ntype Encoder struct {\n\t// A single indentation level. By default it is two spaces.\n\tIndent string\n\n\t// hasWritten is whether we have written any output to w yet.\n\thasWritten bool\n\tw *bufio.Writer\n}\n\n// NewEncoder returns a encoder that encodes Go values to the io.Writer\n// given. By default, a single indentation level is 2 spaces.\nfunc NewEncoder(w io.Writer) *Encoder {\n\treturn &Encoder{\n\t\tw: bufio.NewWriter(w),\n\t\tIndent: \" \",\n\t}\n}\n\n// Encode writes a representation of the Go value to the underlying\n// io.Writer. If the value given cannot be encoded to a valid document,\n// then an error is returned.\n//\n// The mapping between Go values and values should be precisely the same\n// as for the Decode* functions. Similarly, the TextMarshaler interface is\n// supported by encoding the resulting bytes as strings. (If you want to write\n// arbitrary binary data then you will need to use something like base64 since\n// does not have any binary types.)\n//\n// When encoding hashes (i.e., Go maps or structs), keys without any\n// sub-hashes are encoded first.\n//\n// If a Go map is encoded, then its keys are sorted alphabetically for\n// deterministic output. More control over this behavior may be provided if\n// there is demand for it.\n//\n// Encoding Go values without a corresponding representation---like map\n// types with non-string keys---will cause an error to be returned. Similarly\n// for mixed arrays/slices, arrays/slices with nil elements, embedded\n// non-struct types and nested slices containing maps or structs.\n// (e.g., [][]map[string]string is not allowed but []map[string]string is OK\n// and so is []map[string][]string.)\nfunc (enc *Encoder) Encode(v interface{}) error {\n\trv := eindirect(reflect.ValueOf(v))\n\tif err := enc.safeEncode(Key([]string{}), rv); err != nil {\n\t\treturn err\n\t}\n\treturn enc.w.Flush()\n}\n\nfunc (enc *Encoder) safeEncode(key Key, rv reflect.Value) (err error) {\n\tdefer func() {\n\t\tif r := recover(); r != nil {\n\t\t\tif terr, ok := r.(encodeError); ok {\n\t\t\t\terr = terr.error\n\t\t\t\treturn\n\t\t\t}\n\t\t\tpanic(r)\n\t\t}\n\t}()\n\tenc.encode(key, rv)\n\treturn nil\n}\n\nfunc (enc *Encoder) encode(key Key, rv reflect.Value) {\n\t// Special case. Time needs to be in ISO8601 format.\n\t// Special case. If we can marshal the type to text, then we used that.\n\t// Basically, this prevents the encoder for handling these types as\n\t// generic structs (or whatever the underlying type of a TextMarshaler is).\n\tswitch rv.Interface().(type) {\n\tcase time.Time, TextMarshaler:\n\t\tenc.keyEqElement(key, rv)\n\t\treturn\n\t}\n\n\tk := rv.Kind()\n\t//u.Debugf(\"key:%v len:%v val=%v\", key.String(), len(key), k.String())\n\tswitch k {\n\tcase reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64,\n\t\treflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32,\n\t\treflect.Uint64,\n\t\treflect.Float32, reflect.Float64, reflect.String, reflect.Bool:\n\t\tenc.keyEqElement(key, rv)\n\tcase reflect.Array, reflect.Slice:\n\t\tif typeEqual(confArrayHash, confTypeOfGo(rv)) {\n\t\t\tenc.eArrayOfTables(key, rv)\n\t\t} else {\n\t\t\tenc.keyEqElement(key, rv)\n\t\t}\n\tcase reflect.Interface:\n\t\tif rv.IsNil() {\n\t\t\treturn\n\t\t}\n\t\tenc.encode(key, rv.Elem())\n\tcase reflect.Map:\n\t\tif rv.IsNil() {\n\t\t\treturn\n\t\t}\n\t\tenc.eTable(key, rv)\n\tcase reflect.Ptr:\n\t\tif rv.IsNil() {\n\t\t\treturn\n\t\t}\n\t\tenc.encode(key, rv.Elem())\n\tcase reflect.Struct:\n\t\tenc.eTable(key, rv)\n\tdefault:\n\t\tpanic(e(\"Unsupported type for key '%s': %s\", key, k))\n\t}\n}\n\n// eElement encodes any value that can be an array element (primitives and\n// arrays).\nfunc (enc *Encoder) eElement(rv reflect.Value) {\n\tswitch v := rv.Interface().(type) {\n\tcase time.Time:\n\t\t// Special case time.Time as a primitive. Has to come before\n\t\t// TextMarshaler below because time.Time implements\n\t\t// encoding.TextMarshaler, but we need to always use UTC.\n\t\tenc.wf(v.In(time.FixedZone(\"UTC\", 0)).Format(\"2006-01-02T15:04:05Z\"))\n\t\treturn\n\tcase TextMarshaler:\n\t\t// Special case. Use text marshaler if it's available for this value.\n\t\tif s, err := v.MarshalText(); err != nil {\n\t\t\tencPanic(err)\n\t\t} else {\n\t\t\tenc.writeQuoted(string(s))\n\t\t}\n\t\treturn\n\t}\n\tswitch rv.Kind() {\n\tcase reflect.Bool:\n\t\tenc.wf(strconv.FormatBool(rv.Bool()))\n\tcase reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:\n\t\tenc.wf(strconv.FormatInt(rv.Int(), 10))\n\tcase reflect.Uint, reflect.Uint8, reflect.Uint16,\n\t\treflect.Uint32, reflect.Uint64:\n\t\tenc.wf(strconv.FormatUint(rv.Uint(), 10))\n\tcase reflect.Float32:\n\t\tenc.wf(floatAddDecimal(strconv.FormatFloat(rv.Float(), 'f', -1, 32)))\n\tcase reflect.Float64:\n\t\tenc.wf(floatAddDecimal(strconv.FormatFloat(rv.Float(), 'f', -1, 64)))\n\tcase reflect.Array, reflect.Slice:\n\t\tenc.eArrayOrSliceElement(rv)\n\tcase reflect.Interface:\n\t\tenc.eElement(rv.Elem())\n\tcase reflect.String:\n\t\tenc.writeQuoted(rv.String())\n\tdefault:\n\t\tpanic(e(\"Unexpected primitive type: %s\", rv.Kind()))\n\t}\n}\n\n// all floats must have a decimal with at least one number on either side.\nfunc floatAddDecimal(fstr string) string {\n\tif !strings.Contains(fstr, \".\") {\n\t\treturn fstr + \".0\"\n\t}\n\treturn fstr\n}\n\nfunc (enc *Encoder) writeQuoted(s string) {\n\tenc.wf(\"\\\"%s\\\"\", quotedReplacer.Replace(s))\n}\n\nfunc (enc *Encoder) eArrayOrSliceElement(rv reflect.Value) {\n\tlength := rv.Len()\n\t//u.Infof(\"arrayorslice? %v\", rv)\n\tenc.wf(\"[\")\n\tfor i := 0; i < length; i++ {\n\t\telem := rv.Index(i)\n\t\tenc.eElement(elem)\n\t\tif i != length-1 {\n\t\t\tenc.wf(\", \")\n\t\t}\n\t}\n\tenc.wf(\"]\")\n}\n\nfunc (enc *Encoder) eArrayOfTables(key Key, rv reflect.Value) {\n\tif len(key) == 0 {\n\t\tencPanic(errNoKey)\n\t}\n\t//u.Debugf(\"eArrayOfTables key=%s key.len=%v rv=%v\", key, len(key), rv)\n\tpanicIfInvalidKey(key, true)\n\t//enc.newline()\n\t//enc.wf(\"%s [\\n%s]]\", enc.indentStr(key), key.String())\n\tnewKey := key.insert(\"_\")\n\tkeyDelta := 0\n\tenc.wf(\"%s%s = [\", enc.indentStrDelta(key, -1), key[len(key)-1])\n\tfor i := 0; i < rv.Len(); i++ {\n\t\ttrv := rv.Index(i)\n\t\tif isNil(trv) {\n\t\t\tcontinue\n\t\t}\n\t\tenc.newline()\n\t\tenc.wf(\"%s{\", enc.indentStrDelta(key, keyDelta))\n\t\tenc.newline()\n\t\t//enc.wf(\"%s{\\n%s\", enc.indentStr(key), key.String())\n\t\t//enc.newline()\n\t\tenc.eMapOrStruct(newKey, trv)\n\t\t//enc.newline()\n\t\tif i == rv.Len()-1 {\n\t\t\tenc.wf(\"%s}\", enc.indentStrDelta(key, keyDelta))\n\t\t} else {\n\t\t\tenc.wf(\"%s},\", enc.indentStrDelta(key, keyDelta))\n\t\t}\n\t}\n\tenc.newline()\n\tenc.wf(\"%s]\", enc.indentStrDelta(key, -1))\n\tenc.newline()\n}\n\nfunc (enc *Encoder) eTable(key Key, rv reflect.Value) {\n\tif len(key) == 1 {\n\t\t// Output an extra new line between top-level tables.\n\t\t// (The newline isn't written if nothing else has been written though.)\n\t\t//enc.newline()\n\t}\n\tif len(key) > 0 {\n\t\tpanicIfInvalidKey(key, true)\n\t\t//u.Infof(\"table? %v %v\", key, rv)\n\t\tenc.wf(\"%s%s {\", enc.indentStrDelta(key, -1), key[len(key)-1])\n\t\tenc.newline()\n\t}\n\tenc.eMapOrStruct(key, rv)\n\n\tif len(key) > 0 {\n\t\tenc.wf(\"%s}\", enc.indentStrDelta(key, -1))\n\t\tenc.newline()\n\t}\n\n}\n\nfunc (enc *Encoder) eMapOrStruct(key Key, rv reflect.Value) {\n\tswitch rv := eindirect(rv); rv.Kind() {\n\tcase reflect.Map:\n\t\tenc.eMap(key, rv)\n\tcase reflect.Struct:\n\t\tenc.eStruct(key, rv)\n\tdefault:\n\t\tpanic(\"eTable: unhandled reflect.Value Kind: \" + rv.Kind().String())\n\t}\n}\n\nfunc (enc *Encoder) eMap(key Key, rv reflect.Value) {\n\trt := rv.Type()\n\tif rt.Key().Kind() != reflect.String {\n\t\tencPanic(errNonString)\n\t}\n\n\t// Sort keys so that we have deterministic output. And write keys directly\n\t// underneath this key first, before writing sub-structs or sub-maps.\n\tvar mapKeysDirect, mapKeysSub []string\n\tfor _, mapKey := range rv.MapKeys() {\n\t\tk := mapKey.String()\n\t\t//u.Infof(\"map key: %v\", k)\n\t\tif typeIsHash(confTypeOfGo(rv.MapIndex(mapKey))) {\n\t\t\t//u.Debugf(\"found sub? %s for %v\", k, confTypeOfGo(rv.MapIndex(mapKey)))\n\t\t\tmapKeysSub = append(mapKeysSub, k)\n\t\t} else {\n\t\t\tmapKeysDirect = append(mapKeysDirect, k)\n\t\t}\n\t}\n\n\tvar writeMapKeys = func(mapKeys []string) {\n\t\tsort.Strings(mapKeys)\n\t\tfor _, mapKey := range mapKeys {\n\t\t\t//u.Infof(\"mapkey: %v\", mapKey)\n\t\t\tmrv := rv.MapIndex(reflect.ValueOf(mapKey))\n\t\t\tif isNil(mrv) {\n\t\t\t\t// Don't write anything for nil fields.\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tenc.encode(key.add(mapKey), mrv)\n\t\t}\n\t}\n\twriteMapKeys(mapKeysDirect)\n\twriteMapKeys(mapKeysSub)\n}\n\nfunc (enc *Encoder) eStruct(key Key, rv reflect.Value) {\n\t// Write keys for fields directly under this key first, because if we write\n\t// a field that creates a new table, then all keys under it will be in that\n\t// table (not the one we're writing here).\n\trt := rv.Type()\n\tvar fieldsDirect, fieldsSub [][]int\n\tvar addFields func(rt reflect.Type, rv reflect.Value, start []int)\n\taddFields = func(rt reflect.Type, rv reflect.Value, start []int) {\n\t\tfor i := 0; i < rt.NumField(); i++ {\n\t\t\tf := rt.Field(i)\n\t\t\t// skip unexporded fields\n\t\t\tif f.PkgPath != \"\" {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tfrv := rv.Field(i)\n\t\t\tif f.Anonymous {\n\t\t\t\tfrv := eindirect(frv)\n\t\t\t\tt := frv.Type()\n\t\t\t\tif t.Kind() != reflect.Struct {\n\t\t\t\t\tencPanic(errAnonNonStruct)\n\t\t\t\t}\n\t\t\t\taddFields(t, frv, f.Index)\n\t\t\t} else if typeIsHash(confTypeOfGo(frv)) {\n\t\t\t\tfieldsSub = append(fieldsSub, append(start, f.Index...))\n\t\t\t} else {\n\t\t\t\tfieldsDirect = append(fieldsDirect, append(start, f.Index...))\n\t\t\t}\n\t\t}\n\t}\n\taddFields(rt, rv, nil)\n\n\tvar writeFields = func(fields [][]int) {\n\t\tfor _, fieldIndex := range fields {\n\t\t\tsft := rt.FieldByIndex(fieldIndex)\n\t\t\tsf := rv.FieldByIndex(fieldIndex)\n\t\t\tif isNil(sf) {\n\t\t\t\t// Don't write anything for nil fields.\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\t\tkeyName := sft.Tag.Get(\"confl\")\n\t\t\tif keyName == \"-\" {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tif keyName == \"\" {\n\t\t\t\tkeyName = sft.Tag.Get(\"json\")\n\t\t\t\tif keyName == \"-\" {\n\t\t\t\t\tcontinue\n\t\t\t\t} else if keyName == \"\" {\n\t\t\t\t\tkeyName = sft.Name\n\t\t\t\t}\n\t\t\t}\n\t\t\t//u.Infof(\"found key: depth?%v keyName='%v'\\t\\tsf=%v\", len(key), keyName, sf)\n\t\t\tenc.encode(key.add(keyName), sf)\n\t\t}\n\t}\n\twriteFields(fieldsDirect)\n\twriteFields(fieldsSub)\n}\n\n// returns the Confl type name of the Go value's type. It is used to\n// determine whether the types of array elements are mixed (which is forbidden).\n// If the Go value is nil, then it is illegal for it to be an array element, and\n// valueIsNil is returned as true.\n\n// Returns the confl type of a Go value. The type may be `nil`, which means\n// no concrete confl type could be found.\nfunc confTypeOfGo(rv reflect.Value) confType {\n\tif isNil(rv) || !rv.IsValid() {\n\t\treturn nil\n\t}\n\n\tswitch rv.Kind() {\n\tcase reflect.Bool:\n\t\treturn confBool\n\tcase reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64,\n\t\treflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32,\n\t\treflect.Uint64:\n\t\treturn confInteger\n\tcase reflect.Float32, reflect.Float64:\n\t\treturn confFloat\n\tcase reflect.Array, reflect.Slice:\n\t\tif typeEqual(confHash, confArrayType(rv)) {\n\t\t\treturn confArrayHash\n\t\t} else {\n\t\t\treturn confArray\n\t\t}\n\tcase reflect.Ptr, reflect.Interface:\n\t\treturn confTypeOfGo(rv.Elem())\n\tcase reflect.String:\n\t\treturn confString\n\tcase reflect.Map:\n\n\t\treturn confHash\n\tcase reflect.Struct:\n\t\tswitch rv.Interface().(type) {\n\t\tcase time.Time:\n\t\t\treturn confDatetime\n\t\tcase TextMarshaler:\n\t\t\treturn confString\n\t\tdefault:\n\t\t\treturn confHash\n\t\t}\n\tdefault:\n\t\tpanic(\"unexpected reflect.Kind: \" + rv.Kind().String())\n\t}\n}\n\n// returns the element type of a array. The type returned\n// may be nil if it cannot be determined (e.g., a nil slice or a zero length\n// slize). This function may also panic if it finds a type that cannot be\n// expressed in (such as nil elements, heterogeneous arrays or directly\n// nested arrays of tables).\nfunc confArrayType(rv reflect.Value) confType {\n\tif isNil(rv) || !rv.IsValid() || rv.Len() == 0 {\n\t\treturn nil\n\t}\n\tfirstType := confTypeOfGo(rv.Index(0))\n\tif firstType == nil {\n\t\tencPanic(errArrayNilElement)\n\t}\n\n\trvlen := rv.Len()\n\tfor i := 1; i < rvlen; i++ {\n\t\telem := rv.Index(i)\n\t\tswitch elemType := confTypeOfGo(elem); {\n\t\tcase elemType == nil:\n\t\t\tencPanic(errArrayNilElement)\n\t\tcase !typeEqual(firstType, elemType):\n\t\t\tencPanic(errArrayMixedElementTypes)\n\t\t}\n\t}\n\t// If we have a nested array, then we must make sure that the nested\n\t// array contains ONLY primitives.\n\t// This checks arbitrarily nested arrays.\n\tif typeEqual(firstType, confArray) || typeEqual(firstType, confArrayHash) {\n\t\tnest := confArrayType(eindirect(rv.Index(0)))\n\t\tif typeEqual(nest, confHash) || typeEqual(nest, confArrayHash) {\n\t\t\tencPanic(errArrayNoTable)\n\t\t}\n\t}\n\treturn firstType\n}\n\nfunc (enc *Encoder) newline() {\n\tif enc.hasWritten {\n\t\tenc.wf(\"\\n\")\n\t}\n}\n\nfunc (enc *Encoder) keyEqElement(key Key, val reflect.Value) {\n\tif len(key) == 0 {\n\t\tencPanic(errNoKey)\n\t}\n\tpanicIfInvalidKey(key, false)\n\t//u.Infof(\"keyEqElement: %v\", key[len(key)-1])\n\tenc.wf(\"%s%s = \", enc.indentStrDelta(key, -1), key[len(key)-1])\n\tenc.eElement(val)\n\tenc.newline()\n}\n\nfunc (enc *Encoder) wf(format string, v ...interface{}) {\n\tif _, err := fmt.Fprintf(enc.w, format, v...); err != nil {\n\t\tencPanic(err)\n\t}\n\tenc.hasWritten = true\n}\n\nfunc (enc *Encoder) indentStr(key Key) string {\n\treturn strings.Repeat(enc.Indent, len(key))\n}\nfunc (enc *Encoder) indentStrDelta(key Key, delta int) string {\n\treturn strings.Repeat(enc.Indent, len(key)+delta)\n}\n\nfunc encPanic(err error) {\n\tpanic(encodeError{err})\n}\n\nfunc eindirect(v reflect.Value) reflect.Value {\n\tswitch v.Kind() {\n\tcase reflect.Ptr, reflect.Interface:\n\t\treturn eindirect(v.Elem())\n\tdefault:\n\t\treturn v\n\t}\n}\n\nfunc isNil(rv reflect.Value) bool {\n\tswitch rv.Kind() {\n\tcase reflect.Interface, reflect.Map, reflect.Ptr, reflect.Slice:\n\t\treturn rv.IsNil()\n\tdefault:\n\t\treturn false\n\t}\n}\n\nfunc panicIfInvalidKey(key Key, hash bool) {\n\tif hash {\n\t\tfor _, k := range key {\n\t\t\tif !isValidTableName(k) {\n\t\t\t\tencPanic(e(\"Key '%s' is not a valid table name. Table names \"+\n\t\t\t\t\t\"cannot contain '[', ']' or '.'.\", key.String()))\n\t\t\t}\n\t\t}\n\t} else {\n\t\tif !isValidKeyName(key[len(key)-1]) {\n\t\t\tencPanic(e(\"Key '%s' is not a name. Key names \"+\n\t\t\t\t\"cannot contain whitespace.\", key.String()))\n\t\t}\n\t}\n}\n\nfunc isValidTableName(s string) bool {\n\tif len(s) == 0 {\n\t\treturn false\n\t}\n\tfor _, r := range s {\n\t\tif r == '[' || r == ']' || r == '.' {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn true\n}\n\nfunc isValidKeyName(s string) bool {\n\tif len(s) == 0 {\n\t\treturn false\n\t}\n\treturn true\n}\n" }, { "path": "encode_test.go", "content": "package confl\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"log\"\n\t\"net\"\n\t\"testing\"\n\t\"time\"\n\n\tu \"github.com/araddon/gou\"\n\t\"github.com/stretchr/testify/assert\"\n)\n\nvar _ = u.EMPTY\n\nfunc TestEncodeRoundTrip(t *testing.T) {\n\ttype Config struct {\n\t\tAge int\n\t\tCats []string\n\t\tPi float64\n\t\tPerfection []int\n\t\tDOB time.Time\n\t\tIpaddress net.IP\n\t}\n\n\tvar inputs = Config{\n\t\t13,\n\t\t[]string{\"one\", \"two\", \"three\"},\n\t\t3.145,\n\t\t[]int{11, 2, 3, 4},\n\t\ttime.Now(),\n\t\tnet.ParseIP(\"192.168.59.254\"),\n\t}\n\n\tvar firstBuffer bytes.Buffer\n\te := NewEncoder(&firstBuffer)\n\terr := e.Encode(inputs)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tvar outputs Config\n\tif _, err := Decode(firstBuffer.String(), &outputs); err != nil {\n\t\tlog.Printf(\"Could not decode:\\n-----\\n%s\\n-----\\n\",\n\t\t\tfirstBuffer.String())\n\t\tt.Fatal(err)\n\t}\n\n\t// could test each value individually, but I'm lazy\n\tvar secondBuffer bytes.Buffer\n\te2 := NewEncoder(&secondBuffer)\n\terr = e2.Encode(outputs)\n\tif err != nil {\n\t\tt.Fatal(err)\n\t}\n\tif firstBuffer.String() != secondBuffer.String() {\n\t\tt.Error(\n\t\t\tfirstBuffer.String(),\n\t\t\t\"\\n\\n is not identical to\\n\\n\",\n\t\t\tsecondBuffer.String())\n\t}\n\n\t// now try with Marshal\n\tmarshalBytes, err := Marshal(&inputs)\n\tassert.Equal(t, nil, err)\n\tassert.True(t, bytes.Equal(marshalBytes, firstBuffer.Bytes()), \"equal?\")\n}\n\nfunc TestEncodeMany(t *testing.T) {\n\ttype Embedded struct {\n\t\tInt int `confl:\"_int\"`\n\t}\n\ttype NonStruct int\n\n\tdate := time.Date(2014, 5, 11, 20, 30, 40, 0, time.FixedZone(\"IST\", 3600))\n\tdateStr := \"2014-05-11T19:30:40Z\"\n\n\ttests := []struct {\n\t\tlabel string\n\t\tinput interface{}\n\t\twantOutput string\n\t\twantError error\n\t}{\n\t\t{label: \"bool field\",\n\t\t\tinput: struct {\n\t\t\t\tBoolTrue bool\n\t\t\t\tBoolFalse bool\n\t\t\t}{true, false},\n\t\t\twantOutput: \"BoolTrue = true\\nBoolFalse = false\\n\",\n\t\t},\n\t\t{label: \"int fields\",\n\t\t\tinput: struct {\n\t\t\t\tInt int\n\t\t\t\tInt8 int8\n\t\t\t\tInt16 int16\n\t\t\t\tInt32 int32\n\t\t\t\tInt64 int64\n\t\t\t}{1, 2, 3, 4, 5},\n\t\t\twantOutput: \"Int = 1\\nInt8 = 2\\nInt16 = 3\\nInt32 = 4\\nInt64 = 5\\n\",\n\t\t},\n\t\t{label: \"uint fields\",\n\t\t\tinput: struct {\n\t\t\t\tUint uint\n\t\t\t\tUint8 uint8\n\t\t\t\tUint16 uint16\n\t\t\t\tUint32 uint32\n\t\t\t\tUint64 uint64\n\t\t\t}{1, 2, 3, 4, 5},\n\t\t\twantOutput: \"Uint = 1\\nUint8 = 2\\nUint16 = 3\\nUint32 = 4\" +\n\t\t\t\t\"\\nUint64 = 5\\n\",\n\t\t},\n\t\t{label: \"float fields\",\n\t\t\tinput: struct {\n\t\t\t\tFloat32 float32\n\t\t\t\tFloat64 float64\n\t\t\t}{1.5, 2.5},\n\t\t\twantOutput: \"Float32 = 1.5\\nFloat64 = 2.5\\n\",\n\t\t},\n\t\t{label: \"string field\",\n\t\t\tinput: struct{ String string }{\"foo\"},\n\t\t\twantOutput: \"String = \\\"foo\\\"\\n\",\n\t\t},\n\t\t{label: \"string field and unexported field\",\n\t\t\tinput: struct {\n\t\t\t\tString string\n\t\t\t\tunexported int\n\t\t\t}{\"foo\", 0},\n\t\t\twantOutput: \"String = \\\"foo\\\"\\n\",\n\t\t},\n\t\t{label: \"datetime field in UTC\",\n\t\t\tinput: struct{ Date time.Time }{date},\n\t\t\twantOutput: fmt.Sprintf(\"Date = %s\\n\", dateStr),\n\t\t},\n\t\t{label: \"datetime field as primitive\",\n\t\t\t// Using a map here to fail if isStructOrMap() returns true for\n\t\t\t// time.Time.\n\t\t\tinput: map[string]interface{}{\n\t\t\t\t\"Date\": date,\n\t\t\t\t\"Int\": 1,\n\t\t\t},\n\t\t\twantOutput: fmt.Sprintf(\"Date = %s\\nInt = 1\\n\", dateStr),\n\t\t},\n\t\t{label: \"array fields\",\n\t\t\tinput: struct {\n\t\t\t\tIntArray0 [0]int\n\t\t\t\tIntArray3 [3]int\n\t\t\t}{[0]int{}, [3]int{1, 2, 3}},\n\t\t\twantOutput: \"IntArray0 = []\\nIntArray3 = [1, 2, 3]\\n\",\n\t\t},\n\t\t{label: \"slice fields\",\n\t\t\tinput: struct{ IntSliceNil, IntSlice0, IntSlice3 []int }{\n\t\t\t\tnil, []int{}, []int{1, 2, 3},\n\t\t\t},\n\t\t\twantOutput: \"IntSlice0 = []\\nIntSlice3 = [1, 2, 3]\\n\",\n\t\t},\n\t\t{label: \"datetime slices\",\n\t\t\tinput: struct{ DatetimeSlice []time.Time }{\n\t\t\t\t[]time.Time{date, date},\n\t\t\t},\n\t\t\twantOutput: fmt.Sprintf(\"DatetimeSlice = [%s, %s]\\n\",\n\t\t\t\tdateStr, dateStr),\n\t\t},\n\t\t{label: \"nested arrays and slices\",\n\t\t\tinput: struct {\n\t\t\t\tSliceOfArrays [][2]int\n\t\t\t\tArrayOfSlices [2][]int\n\t\t\t\tSliceOfArraysOfSlices [][2][]int\n\t\t\t\tArrayOfSlicesOfArrays [2][][2]int\n\t\t\t\tSliceOfMixedArrays [][2]interface{}\n\t\t\t\tArrayOfMixedSlices [2][]interface{}\n\t\t\t}{\n\t\t\t\t[][2]int{{1, 2}, {3, 4}},\n\t\t\t\t[2][]int{{1, 2}, {3, 4}},\n\t\t\t\t[][2][]int{\n\t\t\t\t\t{\n\t\t\t\t\t\t{1, 2}, {3, 4},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\t{5, 6}, {7, 8},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t[2][][2]int{\n\t\t\t\t\t{\n\t\t\t\t\t\t{1, 2}, {3, 4},\n\t\t\t\t\t},\n\t\t\t\t\t{\n\t\t\t\t\t\t{5, 6}, {7, 8},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t\t[][2]interface{}{\n\t\t\t\t\t{1, 2}, {\"a\", \"b\"},\n\t\t\t\t},\n\t\t\t\t[2][]interface{}{\n\t\t\t\t\t{1, 2}, {\"a\", \"b\"},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantOutput: `SliceOfArrays = [[1, 2], [3, 4]]\nArrayOfSlices = [[1, 2], [3, 4]]\nSliceOfArraysOfSlices = [[[1, 2], [3, 4]], [[5, 6], [7, 8]]]\nArrayOfSlicesOfArrays = [[[1, 2], [3, 4]], [[5, 6], [7, 8]]]\nSliceOfMixedArrays = [[1, 2], [\"a\", \"b\"]]\nArrayOfMixedSlices = [[1, 2], [\"a\", \"b\"]]\n`,\n\t\t},\n\t\t{label: \"empty slice\",\n\t\t\tinput: struct{ Empty []interface{} }{[]interface{}{}},\n\t\t\twantOutput: \"Empty = []\\n\",\n\t\t},\n\t\t{label: \"(error) slice with element type mismatch (string and integer)\",\n\t\t\tinput: struct{ Mixed []interface{} }{[]interface{}{1, \"a\"}},\n\t\t\twantError: errArrayMixedElementTypes,\n\t\t},\n\t\t{label: \"(error) slice with element type mismatch (integer and float)\",\n\t\t\tinput: struct{ Mixed []interface{} }{[]interface{}{1, 2.5}},\n\t\t\twantError: errArrayMixedElementTypes,\n\t\t},\n\t\t{label: \"slice with elems of differing Go types, same types\",\n\t\t\tinput: struct {\n\t\t\t\tMixedInts []interface{}\n\t\t\t\tMixedFloats []interface{}\n\t\t\t}{\n\t\t\t\t[]interface{}{\n\t\t\t\t\tint(1), int8(2), int16(3), int32(4), int64(5),\n\t\t\t\t\tuint(1), uint8(2), uint16(3), uint32(4), uint64(5),\n\t\t\t\t},\n\t\t\t\t[]interface{}{float32(1.5), float64(2.5)},\n\t\t\t},\n\t\t\twantOutput: \"MixedInts = [1, 2, 3, 4, 5, 1, 2, 3, 4, 5]\\n\" +\n\t\t\t\t\"MixedFloats = [1.5, 2.5]\\n\",\n\t\t},\n\t\t{label: \"(error) slice w/ element type mismatch (one is nested array)\",\n\t\t\tinput: struct{ Mixed []interface{} }{\n\t\t\t\t[]interface{}{1, []interface{}{2}},\n\t\t\t},\n\t\t\twantError: errArrayMixedElementTypes,\n\t\t},\n\t\t{label: \"(error) slice with 1 nil element\",\n\t\t\tinput: struct{ NilElement1 []interface{} }{[]interface{}{nil}},\n\t\t\twantError: errArrayNilElement,\n\t\t},\n\t\t{label: \"(error) slice with 1 nil element (and other non-nil elements)\",\n\t\t\tinput: struct{ NilElement []interface{} }{\n\t\t\t\t[]interface{}{1, nil},\n\t\t\t},\n\t\t\twantError: errArrayNilElement,\n\t\t},\n\t\t{label: \"simple map\",\n\t\t\tinput: map[string]int{\"a\": 1, \"b\": 2},\n\t\t\twantOutput: \"a = 1\\nb = 2\\n\",\n\t\t},\n\t\t{label: \"map with interface{} value type\",\n\t\t\tinput: map[string]interface{}{\"a\": 1, \"b\": \"c\"},\n\t\t\twantOutput: \"a = 1\\nb = \\\"c\\\"\\n\",\n\t\t},\n\t\t{label: \"map with interface{} value type, some of which are structs\",\n\t\t\tinput: map[string]interface{}{\n\t\t\t\t\"a\": struct{ Int int }{2},\n\t\t\t\t\"b\": 1,\n\t\t\t},\n\t\t\twantOutput: \"b = 1\\na {\\n Int = 2\\n}\\n\",\n\t\t},\n\t\t{label: \"nested map\",\n\t\t\tinput: map[string]map[string]int{\n\t\t\t\t\"a\": {\"b\": 1},\n\t\t\t\t\"c\": {\"d\": 2},\n\t\t\t},\n\t\t\twantOutput: \"a {\\n b = 1\\n}\\nc {\\n d = 2\\n}\\n\",\n\t\t},\n\t\t{label: \"nested struct\",\n\t\t\tinput: struct{ Struct struct{ Int int } }{\n\t\t\t\tstruct{ Int int }{1},\n\t\t\t},\n\t\t\twantOutput: \"Struct {\\n Int = 1\\n}\\n\",\n\t\t},\n\t\t{label: \"nested struct and non-struct field\",\n\t\t\tinput: struct {\n\t\t\t\tStruct struct{ Int int }\n\t\t\t\tBool bool\n\t\t\t}{struct{ Int int }{1}, true},\n\t\t\twantOutput: \"Bool = true\\nStruct {\\n Int = 1\\n}\\n\",\n\t\t},\n\t\t{label: \"2 nested structs\",\n\t\t\tinput: struct{ Struct1, Struct2 struct{ Int int } }{\n\t\t\t\tstruct{ Int int }{1}, struct{ Int int }{2},\n\t\t\t},\n\t\t\twantOutput: \"Struct1 {\\n Int = 1\\n}\\nStruct2 {\\n Int = 2\\n}\\n\",\n\t\t},\n\t\t{label: \"deeply nested structs\",\n\t\t\tinput: struct {\n\t\t\t\tStruct1, Struct2 struct{ Struct3 *struct{ Int int } }\n\t\t\t}{\n\t\t\t\tstruct{ Struct3 *struct{ Int int } }{&struct{ Int int }{1}},\n\t\t\t\tstruct{ Struct3 *struct{ Int int } }{nil},\n\t\t\t},\n\t\t\twantOutput: \"Struct1 {\\n Struct3 {\\n Int = 1\\n }\\n}\\nStruct2 {\\n}\\n\",\n\t\t},\n\t\t{label: \"nested struct with nil struct elem\",\n\t\t\tinput: struct {\n\t\t\t\tStruct struct{ Inner *struct{ Int int } }\n\t\t\t}{\n\t\t\t\tstruct{ Inner *struct{ Int int } }{nil},\n\t\t\t},\n\t\t\twantOutput: \"Struct {\\n}\\n\",\n\t\t},\n\t\t{label: \"nested struct with no fields\",\n\t\t\tinput: struct {\n\t\t\t\tStruct struct{ Inner struct{} }\n\t\t\t}{\n\t\t\t\tstruct{ Inner struct{} }{struct{}{}},\n\t\t\t},\n\t\t\twantOutput: \"Struct {\\n Inner {\\n }\\n}\\n\",\n\t\t},\n\t\t{label: \"struct with tags\",\n\t\t\tinput: struct {\n\t\t\t\tStruct struct {\n\t\t\t\t\tInt int `json:\"_int\"`\n\t\t\t\t} `confl:\"_struct\"`\n\t\t\t\tBool bool `confl:\"_bool\"`\n\t\t\t}{\n\t\t\t\tstruct {\n\t\t\t\t\tInt int `json:\"_int\"`\n\t\t\t\t}{1}, true,\n\t\t\t},\n\t\t\twantOutput: \"_bool = true\\n_struct {\\n _int = 1\\n}\\n\",\n\t\t},\n\t\t{label: \"embedded struct\",\n\t\t\tinput: struct{ Embedded }{Embedded{1}},\n\t\t\twantOutput: \"_int = 1\\n\",\n\t\t},\n\t\t{label: \"embedded *struct\",\n\t\t\tinput: struct{ *Embedded }{&Embedded{1}},\n\t\t\twantOutput: \"_int = 1\\n\",\n\t\t},\n\t\t{label: \"nested embedded struct\",\n\t\t\tinput: struct {\n\t\t\t\tStruct struct{ Embedded } `confl:\"_struct\"`\n\t\t\t}{struct{ Embedded }{Embedded{1}}},\n\t\t\twantOutput: \"_struct {\\n _int = 1\\n}\\n\",\n\t\t},\n\t\t{label: \"nested embedded *struct\",\n\t\t\tinput: struct {\n\t\t\t\tStruct struct{ *Embedded } `confl:\"_struct\"`\n\t\t\t}{struct{ *Embedded }{&Embedded{1}}},\n\t\t\twantOutput: \"_struct {\\n _int = 1\\n}\\n\",\n\t\t},\n\t\t{label: \"array of tables\",\n\t\t\tinput: struct {\n\t\t\t\tStructs []*struct{ Int int } `confl:\"struct\"`\n\t\t\t}{\n\t\t\t\t[]*struct{ Int int }{{1}, {3}},\n\t\t\t},\n\t\t\twantOutput: \"struct = [\\n {\\n Int = 1\\n },\\n {\\n Int = 3\\n }\\n]\\n\",\n\t\t},\n\t\t{label: \"array of tables order\",\n\t\t\tinput: map[string]interface{}{\n\t\t\t\t\"map\": map[string]interface{}{\n\t\t\t\t\t\"zero\": 5,\n\t\t\t\t\t\"arr\": []map[string]int{\n\t\t\t\t\t\tmap[string]int{\n\t\t\t\t\t\t\t\"friend\": 5,\n\t\t\t\t\t\t},\n\t\t\t\t\t},\n\t\t\t\t},\n\t\t\t},\n\t\t\twantOutput: \"map {\\n zero = 5\\n arr = [\\n {\\n friend = 5\\n }\\n ]\\n}\\n\",\n\t\t},\n\t\t{label: \"(error) top-level slice\",\n\t\t\tinput: []struct{ Int int }{{1}, {2}, {3}},\n\t\t\twantError: errNoKey,\n\t\t},\n\t\t{label: \"(error) slice of slice\",\n\t\t\tinput: struct {\n\t\t\t\tSlices [][]struct{ Int int }\n\t\t\t}{\n\t\t\t\t[][]struct{ Int int }{{{1}}, {{2}}, {{3}}},\n\t\t\t},\n\t\t\twantError: errArrayNoTable,\n\t\t},\n\t\t{label: \"(error) map no string key\",\n\t\t\tinput: map[int]string{1: \"\"},\n\t\t\twantError: errNonString,\n\t\t},\n\t\t{label: \"(error) anonymous non-struct\",\n\t\t\tinput: struct{ NonStruct }{5},\n\t\t\twantError: errAnonNonStruct,\n\t\t},\n\t\t{label: \"(error) empty key name\",\n\t\t\tinput: map[string]int{\"\": 1},\n\t\t\twantError: errAnything,\n\t\t},\n\t\t{label: \"(error) empty map name\",\n\t\t\tinput: map[string]interface{}{\n\t\t\t\t\"\": map[string]int{\"v\": 1},\n\t\t\t},\n\t\t\twantError: errAnything,\n\t\t},\n\t}\n\tfor idx, test := range tests {\n\t\tu.Debugf(\"starting test: #%d %v\", idx, test.label)\n\t\tencodeExpected(t, fmt.Sprintf(\"#%d: %s\", idx, test.label), test.input,\n\t\t\ttest.wantOutput, test.wantError)\n\t}\n}\n\nfunc TestEncodeNestedTableArrays(t *testing.T) {\n\ttype song struct {\n\t\tName string `confl:\"name\"`\n\t}\n\ttype album struct {\n\t\tName string `confl:\"name\"`\n\t\tSongs []song `confl:\"songs\"`\n\t}\n\ttype springsteen struct {\n\t\tAlbums []album `confl:\"albums\"`\n\t}\n\tvalue := springsteen{\n\t\t[]album{\n\t\t\t{\"Born to Run\",\n\t\t\t\t[]song{{\"Jungleland\"}, {\"Meeting Across the River\"}}},\n\t\t\t{\"Born in the USA\",\n\t\t\t\t[]song{{\"Glory Days\"}, {\"Dancing in the Dark\"}}},\n\t\t},\n\t}\n\texpected := `albums = [\n {\n name = \"Born to Run\"\n songs = [\n {\n name = \"Jungleland\"\n },\n {\n name = \"Meeting Across the River\"\n }\n ]\n },\n {\n name = \"Born in the USA\"\n songs = [\n {\n name = \"Glory Days\"\n },\n {\n name = \"Dancing in the Dark\"\n }\n ]\n }\n]\n`\n\tencodeExpected(t, \"nested table arrays\", value, expected, nil)\n}\n\nfunc TestEncodeArrayHashWithNormalHashOrder(t *testing.T) {\n\ttype Alpha struct {\n\t\tV int\n\t}\n\ttype Beta struct {\n\t\tV int\n\t}\n\ttype Conf struct {\n\t\tV int\n\t\tA Alpha\n\t\tB []Beta\n\t}\n\n\tval := Conf{\n\t\tV: 1,\n\t\tA: Alpha{2},\n\t\tB: []Beta{{3}},\n\t}\n\texpected := \"V = 1\\nA {\\n V = 2\\n}\\nB = [\\n {\\n V = 3\\n }\\n]\\n\"\n\tencodeExpected(t, \"array hash with normal hash order\", val, expected, nil)\n}\n\nfunc encodeExpected(\n\tt *testing.T, label string, val interface{}, wantStr string, wantErr error,\n) {\n\tvar buf bytes.Buffer\n\tenc := NewEncoder(&buf)\n\terr := enc.Encode(val)\n\tif err != wantErr {\n\t\tif wantErr != nil {\n\t\t\tif wantErr == errAnything && err != nil {\n\t\t\t\treturn\n\t\t\t}\n\t\t\tt.Errorf(\"%s: want Encode error %v, got %v\", label, wantErr, err)\n\t\t} else {\n\t\t\tt.Errorf(\"%s: Encode failed: %s\", label, err)\n\t\t}\n\t}\n\tif err != nil {\n\t\treturn\n\t}\n\tif got := buf.String(); wantStr != got {\n\t\tu.Debugf(\"\\n\\n%s wanted: \\n%s\\ngot: \\n%s\", label, wantStr, got)\n\t\tfor pos, r := range wantStr {\n\t\t\tif len(got)-1 <= pos {\n\t\t\t\tu.Warnf(\"len mismatch? %v vs %v\", len(got), len(wantStr))\n\t\t\t} else if r != rune(got[pos]) {\n\t\t\t\tu.Warnf(\"mismatch at position: %v %s!=%s\", pos, string(r), string(got[pos]))\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tt.Fatalf(\"%s: want\\n-----\\n%q\\n-----\\nbut got\\n-----\\n%q\\n-----\\n\", label, wantStr, got)\n\t}\n}\n\nfunc ExampleEncoder_Encode() {\n\tdate, _ := time.Parse(time.RFC822, \"14 Mar 10 18:00 UTC\")\n\tvar config = map[string]interface{}{\n\t\t\"date\": date,\n\t\t\"counts\": []int{1, 1, 2, 3, 5, 8},\n\t\t\"hash\": map[string]string{\n\t\t\t\"key1\": \"val1\",\n\t\t\t\"key2\": \"val2\",\n\t\t},\n\t}\n\tbuf := new(bytes.Buffer)\n\tif err := NewEncoder(buf).Encode(config); err != nil {\n\t\tu.Errorf(\"could not encode: %v\", err)\n\t\tlog.Fatal(err)\n\t}\n\tfmt.Println(buf.String())\n\n\t// Output:\n\t// counts = [1, 1, 2, 3, 5, 8]\n\t// date = 2010-03-14T18:00:00Z\n\t// hash {\n\t// key1 = \"val1\"\n\t// key2 = \"val2\"\n\t// }\n}\n" }, { "path": "encoding_types.go", "content": "// +build go1.2\n\npackage confl\n\n// In order to support Go 1.1, we define our own TextMarshaler and\n// TextUnmarshaler types. For Go 1.2+, we just alias them with the\n// standard library interfaces.\n\nimport (\n\t\"encoding\"\n)\n\n// TextMarshaler is a synonym for encoding.TextMarshaler. It is defined here\n// so that Go 1.1 can be supported.\ntype TextMarshaler encoding.TextMarshaler\n\n// TextUnmarshaler is a synonym for encoding.TextUnmarshaler. It is defined here\n// so that Go 1.1 can be supported.\ntype TextUnmarshaler encoding.TextUnmarshaler\n" }, { "path": "encoding_types_1.1.go", "content": "// +build !go1.2\n\npackage confl\n\n// These interfaces were introduced in Go 1.2, so we add them manually when\n// compiling for Go 1.1.\n\n// TextMarshaler is a synonym for encoding.TextMarshaler. It is defined here\n// so that Go 1.1 can be supported.\ntype TextMarshaler interface {\n\tMarshalText() (text []byte, err error)\n}\n\n// TextUnmarshaler is a synonym for encoding.TextUnmarshaler. It is defined here\n// so that Go 1.1 can be supported.\ntype TextUnmarshaler interface {\n\tUnmarshalText(text []byte) error\n}\n" }, { "path": "fuzz.go", "content": "// +build gofuzz\n\npackage confl\n\n/*\nFuzz testing support files\n\nhttps://github.com/dvyukov/go-fuzz\n\nUsage:\n\n go-fuzz-build github.com/lytics/confl\n mkdir fuzz\n cp _examples/*.conf fuzz/\n go-fuzz -bin=confl-fuzz.zip -workdir=fuzz\n\nSee fuzz/crashers for results.\n*/\n\nfunc Fuzz(data []byte) int {\n\tvar v map[string]interface{}\n\tif err := Unmarshal(data, &v); err != nil {\n\t\treturn 0\n\t}\n\treturn 1\n}\n" }, { "path": "go.test.sh", "content": "#!/usr/bin/env bash\n\nset -e\necho \"\" > coverage.txt\n\nfor d in $(go list ./... | grep -v vendor); do\n go test -race -coverprofile=profile.out -covermode=atomic $d\n if [ -f profile.out ]; then\n cat profile.out >> coverage.txt\n rm profile.out\n fi\ndone" }, { "path": "lex.go", "content": "// Copyright 2013 Apcera Inc. All rights reserved.\n\n// Customized heavily from\n// https://github.com/BurntSushi/toml/blob/master/lex.go, which is based on\n// Rob Pike's talk: http://cuddle.googlecode.com/hg/talk/lex.html\n\n// The format supported is less restrictive than today's formats.\n// Supports mixed Arrays [], nested Maps {}, multiple comment types (# and //)\n// Also supports key value assigments using '=' or ':' or whiteSpace()\n// e.g. foo = 2, foo : 2, foo 2\n// maps can be assigned with no key separator as well\n// semicolons as value terminators in key/value assignments are optional\n//\n// see lex_test.go for more examples.\n\npackage confl\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\t\"unicode\"\n\t\"unicode/utf8\"\n)\n\nvar (\n\t// IdentityChars Which Identity Characters are allowed?\n\tIdentityChars = \"_.\"\n)\n\ntype itemType int\n\nconst (\n\titemError itemType = iota\n\titemNIL // used in the parser to indicate no type\n\titemEOF\n\titemKey\n\titemText\n\titemString\n\titemBool\n\titemInteger\n\titemFloat\n\titemDatetime\n\titemArrayStart\n\titemArrayEnd\n\titemMapStart\n\titemMapEnd\n\titemCommentStart\n)\n\nconst (\n\teof = 0\n\tmapStart = '{'\n\tmapEnd = '}'\n\tkeySepEqual = '='\n\tkeySepColon = ':'\n\tarrayStart = '['\n\tarrayEnd = ']'\n\tarrayValTerm = ','\n\tmapValTerm = ','\n\tcommentHashStart = '#'\n\tcommentSlashStart = '/'\n\tdqStringStart = '\"'\n\tdqStringEnd = '\"'\n\tsqStringStart = '\\''\n\tsqStringEnd = '\\''\n\toptValTerm = ';'\n\tblockStart = '('\n\tblockEnd = ')'\n)\n\ntype stateFn func(lx *lexer) stateFn\n\ntype lexer struct {\n\tinput string\n\tstart int\n\tpos int\n\twidth int\n\tline int\n\tstate stateFn\n\titems chan item\n\tcircuitBreaker int\n\tisEnd func(lx *lexer, r rune) bool\n\n\t// A stack of state functions used to maintain context.\n\t// The idea is to reuse parts of the state machine in various places.\n\t// For example, values can appear at the top level or within arbitrarily\n\t// nested arrays. The last state on the stack is used after a value has\n\t// been lexed. Similarly for comments.\n\tstack []stateFn\n}\n\ntype item struct {\n\ttyp itemType\n\tval string\n\tline int\n}\n\nfunc (lx *lexer) nextItem() item {\n\tfor {\n\t\tselect {\n\t\tcase item := <-lx.items:\n\t\t\treturn item\n\t\tdefault:\n\t\t\tlx.state = lx.state(lx)\n\t\t}\n\t}\n}\n\nfunc lex(input string) *lexer {\n\tlx := &lexer{\n\t\tinput: input,\n\t\tstate: lexTop,\n\t\tline: 1,\n\t\titems: make(chan item, 10),\n\t\tstack: make([]stateFn, 0, 10),\n\t\tisEnd: isEndNormal,\n\t}\n\treturn lx\n}\n\nfunc (lx *lexer) push(state stateFn) {\n\tlx.stack = append(lx.stack, state)\n}\n\nfunc (lx *lexer) pop() stateFn {\n\tif len(lx.stack) == 0 {\n\t\treturn lx.errorf(\"BUG in lexer: no states to pop.\")\n\t}\n\tli := len(lx.stack) - 1\n\tlast := lx.stack[li]\n\tlx.stack = lx.stack[0:li]\n\treturn last\n}\n\nfunc (lx *lexer) emit(typ itemType) {\n\tlx.items <- item{typ, lx.input[lx.start:lx.pos], lx.line}\n\tlx.start = lx.pos\n}\n\nfunc (lx *lexer) next() (r rune) {\n\n\t// stackBuf := make([]byte, 4096)\n\t// stackBufLen := runtime.Stack(stackBuf, false)\n\t// stackTraceStr := string(stackBuf[0:stackBufLen])\n\n\tif lx.pos >= len(lx.input) {\n\t\t//u.Warnf(\"next() pos=%d len=%d %v\", lx.pos, len(lx.input), string(stackTraceStr))\n\t\tlx.width = 0\n\t\t// if lx.circuitBreaker > 0 {\n\t\t// \tpanic(\"hm\")\n\t\t// }\n\t\t// lx.circuitBreaker++\n\t\treturn eof\n\t}\n\n\t//u.Debugf(\"next() pos=%d len=%d %v\", lx.pos, len(lx.input), string(stackTraceStr))\n\n\tif lx.input[lx.pos] == '\\n' {\n\t\tlx.line++\n\t}\n\tr, lx.width = utf8.DecodeRuneInString(lx.input[lx.pos:])\n\tlx.pos += lx.width\n\treturn r\n}\n\n// ignore skips over the pending input before this point.\nfunc (lx *lexer) ignore() {\n\tlx.start = lx.pos\n}\n\n// backup steps back one rune. Can be called only once per call of next.\nfunc (lx *lexer) backup() {\n\t// This backup has a problem, if eof has already been hit\n\t// lx.width will be = 0\n\t// possibly just manually set to 1?\n\tlx.pos -= lx.width\n\tif lx.pos < len(lx.input) && lx.input[lx.pos] == '\\n' {\n\t\tlx.line--\n\t}\n}\n\n// peek returns but does not consume the next rune in the input.\nfunc (lx *lexer) peek() rune {\n\tr := lx.next()\n\tlx.backup()\n\treturn r\n}\n\n// errorf stops all lexing by emitting an error and returning `nil`.\n// Note that any value that is a character is escaped if it's a special\n// character (new lines, tabs, etc.).\nfunc (lx *lexer) errorf(format string, values ...interface{}) stateFn {\n\tfor i, value := range values {\n\t\tif v, ok := value.(rune); ok {\n\t\t\tvalues[i] = escapeSpecial(v)\n\t\t}\n\t}\n\tlx.items <- item{\n\t\titemError,\n\t\tfmt.Sprintf(format, values...),\n\t\tlx.line,\n\t}\n\treturn nil\n}\n\n// lexTop consumes elements at the top level of data.\nfunc lexTop(lx *lexer) stateFn {\n\tr := lx.next()\n\tif r != eof && (isWhitespace(r) || isNL(r)) {\n\t\treturn lexSkip(lx, lexTop)\n\t}\n\tswitch {\n\tcase r == commentHashStart:\n\t\tlx.push(lexTop)\n\t\treturn lexCommentStart\n\tcase r == commentSlashStart:\n\t\trn := lx.next()\n\t\tif rn == commentSlashStart {\n\t\t\tlx.push(lexTop)\n\t\t\treturn lexCommentStart\n\t\t}\n\t\tlx.backup()\n\t\tfallthrough\n\tcase r == eof:\n\t\tif lx.pos > lx.start {\n\t\t\treturn lx.errorf(\"Unexpected EOF.\")\n\t\t}\n\t\tlx.emit(itemEOF)\n\t\treturn nil\n\t}\n\n\t// At this point, the only valid item can be a key, so we back up\n\t// and let the key lexer do the rest.\n\tlx.backup()\n\tlx.push(lexTopValueEnd)\n\treturn lexKeyStart\n}\n\n// lexTopValueEnd is entered whenever a top-level value has been consumed.\n// It must see only whitespace, and will turn back to lexTop upon a new line.\n// If it sees EOF, it will quit the lexer successfully.\nfunc lexTopValueEnd(lx *lexer) stateFn {\n\tr := lx.next()\n\tswitch {\n\tcase r == commentHashStart:\n\t\t// a comment will read to a new line for us.\n\t\tlx.push(lexTop)\n\t\treturn lexCommentStart\n\tcase r == commentSlashStart:\n\t\trn := lx.next()\n\t\tif rn == commentSlashStart {\n\t\t\tlx.push(lexTop)\n\t\t\treturn lexCommentStart\n\t\t}\n\t\tlx.backup()\n\t\tfallthrough\n\tcase isWhitespace(r):\n\t\treturn lexTopValueEnd\n\tcase isNL(r) || r == eof || r == optValTerm:\n\t\tlx.ignore()\n\t\treturn lexTop\n\t}\n\treturn lx.errorf(\"Expected a top-level value to end with a new line, \"+\n\t\t\"comment or EOF, but got '%v' instead.\", r)\n}\n\n// lexKeyStart consumes a key name up until the first non-whitespace character.\n// lexKeyStart will ignore whitespace. It will also eat enclosing quotes.\nfunc lexKeyStart(lx *lexer) stateFn {\n\tr := lx.peek()\n\tswitch {\n\tcase isKeySeparator(r):\n\t\treturn lx.errorf(\"Unexpected key separator '%v'.\", r)\n\tcase isWhitespace(r) || isNL(r):\n\t\t// Most likely this is not-reachable, as the lexTop already checks for it.\n\t\tlx.next()\n\t\treturn lexSkip(lx, lexKeyStart)\n\tcase r == dqStringStart:\n\t\tlx.next()\n\t\treturn lexSkip(lx, lexDubQuotedKey)\n\tcase r == sqStringStart:\n\t\tlx.next()\n\t\treturn lexSkip(lx, lexQuotedKey)\n\tcase !isIdentifierRune(r):\n\t\t// This is not a valid identity/key rune\n\t\tlx.next()\n\t\tlx.ignore()\n\t\treturn lexKeyStart\n\tcase r == eof:\n\t\treturn lexTop\n\t}\n\tlx.ignore()\n\tlx.next()\n\treturn lexKey\n}\n\n// lexDubQuotedKey consumes the text of a key between quotes.\nfunc lexDubQuotedKey(lx *lexer) stateFn {\n\tr := lx.peek()\n\tif r == dqStringEnd {\n\t\tlx.emit(itemKey)\n\t\tlx.next()\n\t\treturn lexSkip(lx, lexKeyEnd)\n\t}\n\tlx.next()\n\treturn lexDubQuotedKey\n}\n\n// lexQuotedKey consumes the text of a key between quotes.\nfunc lexQuotedKey(lx *lexer) stateFn {\n\tr := lx.peek()\n\tif r == sqStringEnd {\n\t\tlx.emit(itemKey)\n\t\tlx.next()\n\t\treturn lexSkip(lx, lexKeyEnd)\n\t}\n\tlx.next()\n\treturn lexQuotedKey\n}\n\n// lexKey consumes the text of a key. Assumes that the first character (which\n// is not whitespace) has already been consumed.\nfunc lexKey(lx *lexer) stateFn {\n\tr := lx.peek()\n\tswitch {\n\tcase r == eof:\n\t\t// Unexpected end, allow lexTop eof/error to handle it\n\t\treturn lexTop\n\tcase isWhitespace(r) || isNL(r) || isKeySeparator(r):\n\t\tlx.emit(itemKey)\n\t\treturn lexKeyEnd\n\t}\n\tlx.next()\n\treturn lexKey\n}\n\n// lexKeyEnd consumes the end of a key (up to the key separator).\n// Assumes that the first whitespace character after a key (or the '=' or ':'\n// separator) has NOT been consumed.\nfunc lexKeyEnd(lx *lexer) stateFn {\n\tr := lx.next()\n\tswitch {\n\tcase isWhitespace(r) || isNL(r):\n\t\treturn lexSkip(lx, lexKeyEnd)\n\tcase isKeySeparator(r):\n\t\treturn lexSkip(lx, lexValue)\n\t}\n\t// We start the value here\n\tlx.backup()\n\treturn lexValue\n}\n\n// lexValue starts the consumption of a value anywhere a value is expected.\n// lexValue will ignore whitespace.\n// After a value is lexed, the last state on the next is popped and returned.\nfunc lexValue(lx *lexer) stateFn {\n\t// We allow whitespace to precede a value, but NOT new lines.\n\t// In array syntax, the array states are responsible for ignoring new lines.\n\tr := lx.next()\n\t//u.Infof(\"lexValue r = %v\", string(r))\n\tif isWhitespace(r) {\n\t\treturn lexSkip(lx, lexValue)\n\t}\n\n\tswitch {\n\tcase r == arrayStart:\n\t\tlx.ignore()\n\t\tlx.emit(itemArrayStart)\n\t\tlx.isEnd = isEndArrayUnQuoted\n\t\treturn lexArrayValue\n\tcase r == mapStart:\n\t\tlx.ignore()\n\t\tlx.emit(itemMapStart)\n\t\treturn lexMapKeyStart\n\tcase r == sqStringStart: // single quote: '\n\t\tlx.ignore() // ignore the \" or '\n\t\treturn lexQuotedString\n\tcase r == dqStringStart: // \"\n\t\tlx.ignore() // ignore the \" or '\n\t\treturn lexDubQuotedString\n\tcase r == '-':\n\t\treturn lexNumberStart\n\tcase r == blockStart:\n\t\tlx.next() // ignore the /n after {\n\t\tlx.ignore() // Ignore the (\n\t\treturn lexBlock\n\tcase isDigit(r):\n\t\tlx.backup() // avoid an extra state and use the same as above\n\t\treturn lexNumberOrDateStart\n\tcase r == '.': // special error case, be kind to users\n\t\treturn lx.errorf(\"Floats must start with a digit, not '.'.\")\n\tcase isNL(r):\n\t\treturn lx.errorf(\"Expected value but found new line\")\n\t}\n\t// we didn't consume it, so backup\n\tlx.backup()\n\treturn lexString\n\t//return lx.errorf(\"Expected value but found '%s' instead.\", r)\n}\n\n// lexArrayValue consumes one value in an array. It assumes that '[' or ','\n// have already been consumed. All whitespace and new lines are ignored.\nfunc lexArrayValue(lx *lexer) stateFn {\n\tr := lx.next()\n\t//u.Infof(\"lexArrayValue r = %v\", string(r))\n\tswitch {\n\tcase isWhitespace(r) || isNL(r):\n\t\treturn lexSkip(lx, lexArrayValue)\n\tcase r == commentHashStart:\n\t\tlx.push(lexArrayValue)\n\t\treturn lexCommentStart\n\tcase r == commentSlashStart:\n\t\trn := lx.next()\n\t\tif rn == commentSlashStart {\n\t\t\tlx.push(lexArrayValue)\n\t\t\treturn lexCommentStart\n\t\t}\n\t\tlx.backup()\n\t\tfallthrough\n\tcase r == arrayValTerm: // , we should not have found comma yet\n\t\treturn lx.errorf(\"Unexpected array value terminator '%v'.\", arrayValTerm)\n\tcase r == arrayEnd:\n\t\treturn lexArrayEnd\n\t}\n\n\tlx.backup()\n\tlx.push(lexArrayValueEnd)\n\treturn lexValue\n}\n\n// lexArrayValueEnd consumes the cruft between values of an array. Namely,\n// it ignores whitespace and expects either a ',' or a ']'.\nfunc lexArrayValueEnd(lx *lexer) stateFn {\n\tr := lx.next()\n\t//u.Infof(\"lexArrayValueEnd r = %v\", string(r))\n\tswitch {\n\tcase isWhitespace(r):\n\t\treturn lexSkip(lx, lexArrayValueEnd)\n\tcase r == commentHashStart:\n\t\tlx.push(lexArrayValueEnd)\n\t\treturn lexCommentStart\n\tcase r == commentSlashStart:\n\t\trn := lx.next()\n\t\tif rn == commentSlashStart {\n\t\t\tlx.push(lexArrayValueEnd)\n\t\t\treturn lexCommentStart\n\t\t}\n\t\tlx.backup()\n\t\tfallthrough\n\tcase r == arrayValTerm || isNL(r):\n\t\treturn lexSkip(lx, lexArrayValue) // Move onto next\n\tcase r == arrayEnd:\n\t\treturn lexArrayEnd\n\t}\n\treturn lx.errorf(\"Expected an array value terminator %q or an array \"+\n\t\t\"terminator %q, but got '%v' instead.\", arrayValTerm, arrayEnd, r)\n}\n\n// lexArrayEnd finishes the lexing of an array. It assumes that a ']' has\n// just been consumed.\nfunc lexArrayEnd(lx *lexer) stateFn {\n\tlx.ignore()\n\tlx.emit(itemArrayEnd)\n\tlx.isEnd = isEndNormal\n\treturn lx.pop()\n}\n\n// lexMapKeyStart consumes a key name up until the first non-whitespace\n// character.\n// lexMapKeyStart will ignore whitespace.\nfunc lexMapKeyStart(lx *lexer) stateFn {\n\tr := lx.peek()\n\tswitch {\n\tcase r == eof:\n\t\treturn lx.errorf(\"Un terminated map\")\n\tcase isKeySeparator(r):\n\t\treturn lx.errorf(\"Unexpected key separator '%v'.\", r)\n\tcase isWhitespace(r) || isNL(r):\n\t\tlx.next()\n\t\treturn lexSkip(lx, lexMapKeyStart)\n\tcase r == mapEnd:\n\t\tlx.next()\n\t\treturn lexSkip(lx, lexMapEnd)\n\tcase r == commentHashStart:\n\t\tlx.next()\n\t\tlx.push(lexMapKeyStart)\n\t\treturn lexCommentStart\n\tcase r == commentSlashStart:\n\t\tlx.next()\n\t\trn := lx.next()\n\t\tif rn == commentSlashStart {\n\t\t\tlx.push(lexMapKeyStart)\n\t\t\treturn lexCommentStart\n\t\t}\n\t\tlx.backup()\n\tcase r == sqStringStart:\n\t\tlx.next()\n\t\treturn lexSkip(lx, lexMapQuotedKey)\n\tcase r == dqStringStart:\n\t\tlx.next()\n\t\treturn lexSkip(lx, lexMapDubQuotedKey)\n\t}\n\tlx.ignore()\n\tlx.next()\n\treturn lexMapKey\n}\n\n// lexMapQuotedKey consumes the text of a key between quotes.\nfunc lexMapQuotedKey(lx *lexer) stateFn {\n\tr := lx.peek()\n\tif r == sqStringEnd {\n\t\tlx.emit(itemKey)\n\t\tlx.next()\n\t\treturn lexSkip(lx, lexMapKeyEnd)\n\t}\n\tlx.next()\n\treturn lexMapQuotedKey\n}\n\n// lexMapQuotedKey consumes the text of a key between quotes.\nfunc lexMapDubQuotedKey(lx *lexer) stateFn {\n\tr := lx.peek()\n\tif r == dqStringEnd {\n\t\tlx.emit(itemKey)\n\t\tlx.next()\n\t\treturn lexSkip(lx, lexMapKeyEnd)\n\t}\n\tlx.next()\n\treturn lexMapDubQuotedKey\n}\n\n// lexMapKey consumes the text of a key. Assumes that the first character (which\n// is not whitespace) has already been consumed.\nfunc lexMapKey(lx *lexer) stateFn {\n\tr := lx.peek()\n\tif isWhitespace(r) || isNL(r) || isKeySeparator(r) {\n\t\tlx.emit(itemKey)\n\t\treturn lexMapKeyEnd\n\t}\n\tlx.next()\n\treturn lexMapKey\n}\n\n// lexMapKeyEnd consumes the end of a key (up to the key separator).\n// Assumes that the first whitespace character after a key (or the '='\n// separator) has NOT been consumed.\nfunc lexMapKeyEnd(lx *lexer) stateFn {\n\tr := lx.next()\n\tswitch {\n\tcase isWhitespace(r) || isNL(r):\n\t\treturn lexSkip(lx, lexMapKeyEnd)\n\tcase isKeySeparator(r):\n\t\treturn lexSkip(lx, lexMapValue)\n\t}\n\t// We start the value here\n\tlx.backup()\n\treturn lexMapValue\n}\n\n// lexMapValue consumes one value in a map. It assumes that '{' or ','\n// have already been consumed. All whitespace and new lines are ignored.\n// Map values can be separated by ',' or simple NLs.\nfunc lexMapValue(lx *lexer) stateFn {\n\tr := lx.next()\n\tswitch {\n\tcase isWhitespace(r) || isNL(r):\n\t\treturn lexSkip(lx, lexMapValue)\n\tcase r == commentHashStart:\n\t\tlx.push(lexMapValue)\n\t\treturn lexCommentStart\n\tcase r == commentSlashStart:\n\t\trn := lx.next()\n\t\tif rn == commentSlashStart {\n\t\t\tlx.push(lexMapValue)\n\t\t\treturn lexCommentStart\n\t\t}\n\t\tlx.backup()\n\t\tfallthrough\n\tcase r == mapValTerm:\n\t\treturn lx.errorf(\"Unexpected map value terminator %q.\", mapValTerm)\n\tcase r == mapEnd:\n\t\treturn lexSkip(lx, lexMapEnd)\n\t}\n\tlx.backup()\n\tlx.push(lexMapValueEnd)\n\treturn lexValue\n}\n\n// lexMapValueEnd consumes the cruft between values of a map. Namely,\n// it ignores whitespace and expects either a ',' or a '}'.\nfunc lexMapValueEnd(lx *lexer) stateFn {\n\tr := lx.next()\n\tswitch {\n\tcase isWhitespace(r):\n\t\treturn lexSkip(lx, lexMapValueEnd)\n\tcase r == commentHashStart:\n\t\tlx.push(lexMapValueEnd)\n\t\treturn lexCommentStart\n\tcase r == commentSlashStart:\n\t\trn := lx.next()\n\t\tif rn == commentSlashStart {\n\t\t\tlx.push(lexMapValueEnd)\n\t\t\treturn lexCommentStart\n\t\t}\n\t\tlx.backup()\n\t\tfallthrough\n\tcase r == optValTerm || r == mapValTerm || isNL(r):\n\t\treturn lexSkip(lx, lexMapKeyStart) // Move onto next\n\tcase r == mapEnd:\n\t\treturn lexSkip(lx, lexMapEnd)\n\t}\n\treturn lx.errorf(\"Expected a map value terminator %q or a map \"+\n\t\t\"terminator %q, but got '%v' instead.\", mapValTerm, mapEnd, r)\n}\n\n// lexMapEnd finishes the lexing of a map. It assumes that a '}' has\n// just been consumed.\nfunc lexMapEnd(lx *lexer) stateFn {\n\tlx.ignore()\n\tlx.emit(itemMapEnd)\n\treturn lx.pop()\n}\n\n// Checks if the unquoted string was actually a boolean\nfunc (lx *lexer) isBool() bool {\n\tstr := lx.input[lx.start:lx.pos]\n\tstr = strings.ToLower(str)\n\treturn str == \"true\" || str == \"false\"\n}\n\n// lexQuotedString consumes the inner contents of a string. It assumes that the\n// beginning '\"' has already been consumed and ignored. It will not interpret any\n// internal contents.\nfunc lexQuotedString(lx *lexer) stateFn {\n\tr := lx.next()\n\tswitch {\n\tcase r == sqStringEnd:\n\t\tlx.backup()\n\t\tlx.emit(itemString)\n\t\tlx.next()\n\t\tlx.ignore()\n\t\treturn lx.pop()\n\t}\n\treturn lexQuotedString\n}\n\n// lexDubQuotedString consumes the inner contents of a string. It assumes that the\n// beginning '\"' has already been consumed and ignored. It will not interpret any\n// internal contents.\nfunc lexDubQuotedString(lx *lexer) stateFn {\n\tr := lx.next()\n\tswitch {\n\tcase r == dqStringEnd:\n\t\tlx.backup()\n\t\tlx.emit(itemString)\n\t\tlx.next()\n\t\tlx.ignore()\n\t\treturn lx.pop()\n\t}\n\treturn lexDubQuotedString\n}\n\n// lexString consumes the inner contents of a string. It assumes that the\n// beginning '\"' has already been consumed and ignored.\nfunc lexString(lx *lexer) stateFn {\n\tr := lx.next()\n\t//u.Infof(\"lexString r = %v\", string(r))\n\tswitch {\n\tcase r == '\\\\':\n\t\treturn lexStringEscape\n\t// Termination of non-quoted strings\n\tcase lx.isEnd(lx, r):\n\t\tlx.backup()\n\t\tif lx.isBool() {\n\t\t\tlx.emit(itemBool)\n\t\t} else {\n\t\t\tlx.emit(itemString)\n\t\t}\n\t\treturn lx.pop()\n\tcase r == sqStringEnd:\n\t\tlx.backup()\n\t\tlx.emit(itemString)\n\t\tlx.next()\n\t\tlx.ignore()\n\t\treturn lx.pop()\n\t}\n\treturn lexString\n}\n\n// lexDubString consumes the inner contents of a string. It assumes that the\n// beginning '\"' has already been consumed and ignored.\nfunc lexDubString(lx *lexer) stateFn {\n\tr := lx.next()\n\tswitch {\n\tcase r == '\\\\':\n\t\treturn lexStringEscape\n\t// Termination of non-quoted strings\n\tcase isNL(r) || r == eof || r == optValTerm || isWhitespace(r):\n\t\tlx.backup()\n\t\tif lx.isBool() {\n\t\t\tlx.emit(itemBool)\n\t\t} else {\n\t\t\tlx.emit(itemString)\n\t\t}\n\t\treturn lx.pop()\n\tcase r == dqStringEnd:\n\t\tlx.backup()\n\t\tlx.emit(itemString)\n\t\tlx.next()\n\t\tlx.ignore()\n\t\treturn lx.pop()\n\t}\n\treturn lexDubString\n}\n\n// lexBlock consumes the inner contents as a string. It assumes that the\n// beginning '(' has already been consumed and ignored. It will continue\n// processing until it finds a ')' on a new line by itself.\nfunc lexBlock(lx *lexer) stateFn {\n\tr := lx.next()\n\n\t//u.Debugf(\"lexBlock() pos=%d len=%d %q\", lx.pos, len(lx.input), lx.input[lx.pos:])\n\n\tswitch {\n\tcase r == blockEnd:\n\n\t\tlx.backup() // unconsume ) we are going to verify below\n\t\tlx.backup() // unconsume previous rune to ensure it is newline\n\n\t\t// Looking for a ')' character on a line by itself, if the previous\n\t\t// character isn't a new line, then break so we keep processing the block.\n\t\tif lx.next() != '\\n' {\n\t\t\tlx.next() // if inline ( this will consume it\n\t\t\tbreak\n\t\t}\n\t\tlx.next()\n\n\t\t// Make sure the next character is a new line or an eof. We want a ')' on a\n\t\t// bare line by itself.\n\t\tswitch r = lx.next(); r {\n\t\tcase '\\n', eof:\n\t\t\tif r == eof {\n\t\t\t\tlx.width = 1\n\t\t\t}\n\t\t\tlx.backup() // unconsume the \\n, or eof\n\t\t\tr = lx.peek()\n\t\t\tif r != ')' { // For some reason on EOF we aren't where we think\n\t\t\t\tlx.backup() // unconsume the )\n\t\t\t}\n\t\t\tlx.backup() // unconsume the \\n\n\t\t\tlx.emit(itemString)\n\t\t\tlx.next() // consume the previous line \\n\n\t\t\tlx.next() // consume the )\n\t\t\tlx.ignore()\n\t\t\treturn lx.pop()\n\t\t}\n\t\tlx.backup()\n\t}\n\treturn lexBlock\n}\n\n// lexStringEscape consumes an escaped character. It assumes that the preceding\n// '\\\\' has already been consumed.\nfunc lexStringEscape(lx *lexer) stateFn {\n\tr := lx.next()\n\tswitch r {\n\tcase 'x':\n\t\treturn lexStringBinary\n\tcase 't':\n\t\tfallthrough\n\tcase 'n':\n\t\tfallthrough\n\tcase 'r':\n\t\tfallthrough\n\tcase '\"':\n\t\tfallthrough\n\tcase '\\\\':\n\t\treturn lexString\n\t}\n\treturn lx.errorf(\"Invalid escape character '%v'. Only the following \"+\n\t\t\"escape characters are allowed: \\\\xXX, \\\\t, \\\\n, \\\\r, \\\\\\\", \\\\\\\\.\", r)\n}\n\n// lexStringBinary consumes two hexadecimal digits following '\\x'. It assumes\n// that the '\\x' has already been consumed.\nfunc lexStringBinary(lx *lexer) stateFn {\n\tr := lx.next()\n\tif !isHexadecimal(r) {\n\t\treturn lx.errorf(\"Expected two hexadecimal digits after '\\\\x', but \"+\n\t\t\t\"got '%v' instead.\", r)\n\t}\n\n\tr = lx.next()\n\tif !isHexadecimal(r) {\n\t\treturn lx.errorf(\"Expected two hexadecimal digits after '\\\\x', but \"+\n\t\t\t\"got '%v' instead.\", r)\n\t}\n\treturn lexString\n}\n\n// lexNumberOrDateStart consumes either a (positive) integer, float or datetime.\n// It assumes that NO negative sign has been consumed.\nfunc lexNumberOrDateStart(lx *lexer) stateFn {\n\tr := lx.next()\n\tif !isDigit(r) {\n\t\tif r == '.' {\n\t\t\treturn lx.errorf(\"Floats must start with a digit, not '.'.\")\n\t\t} else {\n\t\t\treturn lx.errorf(\"Expected a digit but got '%v'.\", r)\n\t\t}\n\t}\n\treturn lexNumberOrDate\n}\n\n// lexNumberOrDate consumes either a (positive) integer, float or datetime.\nfunc lexNumberOrDate(lx *lexer) stateFn {\n\tr := lx.next()\n\tswitch {\n\tcase r == '-':\n\t\tif lx.pos-lx.start != 5 {\n\t\t\treturn lx.errorf(\"All ISO8601 dates must be in full Zulu form.\")\n\t\t}\n\t\treturn lexDateAfterYear\n\tcase isDigit(r):\n\t\treturn lexNumberOrDate\n\tcase r == '.':\n\t\treturn lexFloatStart\n\t}\n\n\tlx.backup()\n\tlx.emit(itemInteger)\n\treturn lx.pop()\n}\n\n// lexDateAfterYear consumes a full Zulu Datetime in ISO8601 format.\n// It assumes that \"YYYY-\" has already been consumed.\nfunc lexDateAfterYear(lx *lexer) stateFn {\n\tformats := []rune{\n\t\t// digits are '0'.\n\t\t// everything else is direct equality.\n\t\t'0', '0', '-', '0', '0',\n\t\t'T',\n\t\t'0', '0', ':', '0', '0', ':', '0', '0',\n\t\t'Z',\n\t}\n\tfor _, f := range formats {\n\t\tr := lx.next()\n\t\tif f == '0' {\n\t\t\tif !isDigit(r) {\n\t\t\t\treturn lx.errorf(\"Expected digit in ISO8601 datetime, \"+\n\t\t\t\t\t\"but found '%v' instead.\", r)\n\t\t\t}\n\t\t} else if f != r {\n\t\t\treturn lx.errorf(\"Expected '%v' in ISO8601 datetime, \"+\n\t\t\t\t\"but found '%v' instead.\", f, r)\n\t\t}\n\t}\n\tlx.emit(itemDatetime)\n\treturn lx.pop()\n}\n\n// lexNumberStart consumes either an integer or a float. It assumes that a\n// negative sign has already been read, but that *no* digits have been consumed.\n// lexNumberStart will move to the appropriate integer or float states.\nfunc lexNumberStart(lx *lexer) stateFn {\n\t// we MUST see a digit. Even floats have to start with a digit.\n\tr := lx.next()\n\tif !isDigit(r) {\n\t\tif r == '.' {\n\t\t\treturn lx.errorf(\"Floats must start with a digit, not '.'.\")\n\t\t} else {\n\t\t\treturn lx.errorf(\"Expected a digit but got '%v'.\", r)\n\t\t}\n\t}\n\treturn lexNumber\n}\n\n// lexNumber consumes an integer or a float after seeing the first digit.\nfunc lexNumber(lx *lexer) stateFn {\n\tr := lx.next()\n\tswitch {\n\tcase isDigit(r):\n\t\treturn lexNumber\n\tcase r == '.':\n\t\treturn lexFloatStart\n\t}\n\n\tlx.backup()\n\tlx.emit(itemInteger)\n\treturn lx.pop()\n}\n\n// lexFloatStart starts the consumption of digits of a float after a '.'.\n// Namely, at least one digit is required.\nfunc lexFloatStart(lx *lexer) stateFn {\n\tr := lx.next()\n\tif !isDigit(r) {\n\t\treturn lx.errorf(\"Floats must have a digit after the '.', but got \"+\n\t\t\t\"'%v' instead.\", r)\n\t}\n\treturn lexFloat\n}\n\n// lexFloat consumes the digits of a float after a '.'.\n// Assumes that one digit has been consumed after a '.' already.\nfunc lexFloat(lx *lexer) stateFn {\n\tr := lx.next()\n\tif isDigit(r) {\n\t\treturn lexFloat\n\t}\n\n\tlx.backup()\n\tlx.emit(itemFloat)\n\treturn lx.pop()\n}\n\n// lexCommentStart begins the lexing of a comment. It will emit\n// itemCommentStart and consume no characters, passing control to lexComment.\nfunc lexCommentStart(lx *lexer) stateFn {\n\tlx.ignore()\n\tlx.emit(itemCommentStart)\n\treturn lexComment\n}\n\n// lexComment lexes an entire comment. It assumes that '#' has been consumed.\n// It will consume *up to* the first new line character, and pass control\n// back to the last state on the stack.\nfunc lexComment(lx *lexer) stateFn {\n\tr := lx.peek()\n\tif isNL(r) || r == eof {\n\t\tlx.emit(itemText)\n\t\treturn lx.pop()\n\t}\n\tlx.next()\n\treturn lexComment\n}\n\n// lexSkip ignores all slurped input and moves on to the next state.\nfunc lexSkip(lx *lexer, nextState stateFn) stateFn {\n\treturn func(lx *lexer) stateFn {\n\t\tlx.ignore()\n\t\treturn nextState\n\t}\n}\n\nfunc isEndNormal(lx *lexer, r rune) bool {\n\treturn (isNL(r) || r == eof || r == optValTerm || isWhitespace(r))\n}\n\nfunc isEndArrayUnQuoted(lx *lexer, r rune) bool {\n\treturn (isNL(r) || r == eof || r == optValTerm || r == arrayEnd || r == arrayValTerm || isWhitespace(r))\n}\n\nfunc isIdentifierRune(r rune) bool {\n\tif unicode.IsLetter(r) || unicode.IsDigit(r) {\n\t\treturn true\n\t}\n\tfor _, allowedRune := range IdentityChars {\n\t\tif allowedRune == r {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\n// Tests for both key separators\nfunc isKeySeparator(r rune) bool {\n\treturn r == keySepEqual || r == keySepColon\n}\n\n// isWhitespace returns true if `r` is a whitespace character according\n// to the spec.\nfunc isWhitespace(r rune) bool {\n\treturn r == '\\t' || r == ' '\n}\n\nfunc isNL(r rune) bool {\n\treturn r == '\\n' || r == '\\r'\n}\n\nfunc isDigit(r rune) bool {\n\treturn r >= '0' && r <= '9'\n}\n\nfunc isHexadecimal(r rune) bool {\n\treturn (r >= '0' && r <= '9') ||\n\t\t(r >= 'a' && r <= 'f') ||\n\t\t(r >= 'A' && r <= 'F')\n}\n\nfunc (item item) String() string {\n\treturn fmt.Sprintf(\"(%T, '%s', %d)\", item.typ, item.val, item.line)\n}\n\nfunc escapeSpecial(c rune) string {\n\tswitch c {\n\tcase '\\n':\n\t\treturn \"\\\\n\"\n\t}\n\treturn string(c)\n}\n" }, { "path": "lex_test.go", "content": "package confl\n\nimport (\n\t\"testing\"\n\n\tu \"github.com/araddon/gou\"\n)\n\nvar _ = u.EMPTY\n\n// Test to make sure we get what we expect.\nfunc expect(t *testing.T, lx *lexer, items []item) {\n\tfor i := 0; i < len(items); i++ {\n\t\titem := lx.nextItem()\n\t\tif item.typ == itemEOF {\n\t\t\tbreak\n\t\t} else if item.typ == itemError {\n\t\t\tt.Fatal(item.val)\n\t\t}\n\t\tif item != items[i] {\n\t\t\t//u.Debugf(\"\\n\\n%s wanted: \\n%s\\ngot: \\n%s\", label, wantStr, got)\n\t\t\twantStr := items[i].val\n\t\t\tgot := item.val\n\t\t\tfor pos, r := range wantStr {\n\t\t\t\tif len(got)-1 < pos {\n\t\t\t\t\tu.Warnf(\"len mismatch? %v vs %v\", len(got), len(wantStr))\n\t\t\t\t} else if r != rune(got[pos]) {\n\t\t\t\t\tu.Warnf(\"mismatch at position: %v %q!=%q\", pos, string(r), string(got[pos]))\n\t\t\t\t\tbreak\n\t\t\t\t} else {\n\t\t\t\t\t//u.Debugf(\"match at position: %v %q=%q\", pos, string(r), string(got[pos]))\n\t\t\t\t}\n\t\t\t}\n\t\t\tt.Fatalf(\"Testing: '%s'\\nExpected %q, received %q\\n\",\n\t\t\t\tlx.input, items[i], item)\n\t\t}\n\t}\n}\n\n// Test to make sure we get what we expect.\nfunc expectError(t *testing.T, conf string) {\n\tlx := lex(conf)\n\tfor {\n\t\titem := lx.nextItem()\n\t\tif item.typ == itemEOF {\n\t\t\tt.Fatalf(\"expected error got EOF for %v\", conf)\n\t\t\tbreak\n\t\t} else if item.typ == itemError {\n\t\t\t// Success\n\t\t\treturn\n\t\t}\n\t}\n}\n\nvar errorVals = []string{\n\t`//comment1\nconfig : {\n\t/ -- bad comment1\n}`,\n\t`//comment2\nconfig : {\n\tport: 8080\n}/ -- bad comment2`,\n\t`/comment3\nport: 8080`,\n\t`port: 8083 /badcomment4`,\n\t`port: 8084 //goodcomment1\n\tportx: 80/badcomment4`,\n\t`:port: 8085`, // Can't start key with key seperator\n\t`=port=8086`, // Can't start key with key seperator\n\t`rate=.55`, // can't start values with .\n\t`rate=\n`, // can't start values with new line.\n}\n\nfunc TestLexErrors(t *testing.T) {\n\tfor _, v := range errorVals {\n\t\texpectError(t, v)\n\t}\n}\n\nfunc TestLexBadConfString(t *testing.T) {\n\tlx := lex(\"\\x0e9\\xbd\\xbf\\xefr\")\n\titem := lx.nextItem()\n\tif item.typ != itemError {\n\t\tt.Fatal(item.val)\n\t}\n}\n\nfunc TestLexSimpleKeyStringValues(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemString, \"bar\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\t// Double quotes\n\tlx := lex(\"foo = \\\"bar\\\"\")\n\texpect(t, lx, expectedItems)\n\t// Single quotes\n\tlx = lex(\"foo = 'bar'\")\n\texpect(t, lx, expectedItems)\n\t// No spaces\n\tlx = lex(\"foo='bar'\")\n\texpect(t, lx, expectedItems)\n\t// NL\n\tlx = lex(\"foo='bar'\\r\\n\")\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexSimpleKeyIntegerValues(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemInteger, \"123\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\tlx := lex(\"foo = 123\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo=123\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo=123\\r\\n\")\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexSimpleKeyFloatValues(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemFloat, \"22.2\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\tlx := lex(\"foo = 22.2\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo=22.2\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo=22.2\\r\\n\")\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexSimpleKeyBoolValues(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemBool, \"true\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\tlx := lex(\"foo = true\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo=true\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo=true\\r\\n\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo=True\")\n\texpect(t, lx, []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemBool, \"True\", 1},\n\t\t{itemEOF, \"\", 1},\n\t})\n}\n\nfunc TestLexComments(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemCommentStart, \"\", 1},\n\t\t{itemText, \" This is a comment\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\tlx := lex(\"# This is a comment\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"# This is a comment\\r\\n\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"// This is a comment\\r\\n\")\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexArrays(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemArrayStart, \"\", 1},\n\t\t{itemInteger, \"1\", 1},\n\t\t{itemInteger, \"2\", 1},\n\t\t{itemInteger, \"3\", 1},\n\t\t{itemString, \"bar\", 1},\n\t\t{itemArrayEnd, \"\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\tlx := lex(\"foo = [1, 2, 3, 'bar']\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo = [1,2,3,'bar']\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo = [1, 2,3,'bar']\")\n\texpect(t, lx, expectedItems)\n}\n\nvar mlArray = `\n# top level comment\nfoo = [\n 1, # One\n 2, // Two\n 3 , // Three\n 'bar' ,\n \"bar\"\n]\n`\n\nfunc TestLexMultilineArrays(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemCommentStart, \"\", 2},\n\t\t{itemText, \" top level comment\", 2},\n\t\t{itemKey, \"foo\", 3},\n\t\t{itemArrayStart, \"\", 3},\n\t\t{itemInteger, \"1\", 4},\n\t\t{itemCommentStart, \"\", 4},\n\t\t{itemText, \" One\", 4},\n\t\t{itemInteger, \"2\", 5},\n\t\t{itemCommentStart, \"\", 5},\n\t\t{itemText, \" Two\", 5},\n\t\t{itemInteger, \"3\", 6},\n\t\t{itemCommentStart, \"\", 6},\n\t\t{itemText, \" Three\", 6},\n\t\t{itemString, \"bar\", 7},\n\t\t{itemString, \"bar\", 8},\n\t\t{itemArrayEnd, \"\", 9},\n\t\t{itemEOF, \"\", 9},\n\t}\n\tlx := lex(mlArray)\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexUnterminated(t *testing.T) {\n\tlx := lex(\"foo {\\n name = 'bill' \\n\")\n\tlx.nextItem() // foo\n\tlx.nextItem() // map start = \"\"\n\tlx.nextItem() // name\n\tlx.nextItem() // bill\n\titem := lx.nextItem()\n\tif item.typ != itemError {\n\t\tt.Errorf(\"Should be error for un terminated map: %v\", item)\n\t}\n}\n\nvar mlArrayNoSep = `\n# top level comment\nfoo = [\n 1\n 2\n 3\n 'bar'\n \"bar\"\n]\n`\n\nfunc TestLexMultilineArraysNoSep(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemCommentStart, \"\", 2},\n\t\t{itemText, \" top level comment\", 2},\n\t\t{itemKey, \"foo\", 3},\n\t\t{itemArrayStart, \"\", 3},\n\t\t{itemInteger, \"1\", 4},\n\t\t{itemInteger, \"2\", 5},\n\t\t{itemInteger, \"3\", 6},\n\t\t{itemString, \"bar\", 7},\n\t\t{itemString, \"bar\", 8},\n\t\t{itemArrayEnd, \"\", 9},\n\t\t{itemEOF, \"\", 9},\n\t}\n\tlx := lex(mlArrayNoSep)\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexSimpleMap(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemMapStart, \"\", 1},\n\t\t{itemKey, \"ip\", 1},\n\t\t{itemString, \"127.0.0.1\", 1},\n\t\t{itemKey, \"port\", 1},\n\t\t{itemInteger, \"4242\", 1},\n\t\t{itemMapEnd, \"\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\n\tlx := lex(\"foo = {ip='127.0.0.1', port = 4242}\")\n\texpect(t, lx, expectedItems)\n}\n\nvar mlMap = `\nfoo = {\n ip = '127.0.0.1'\n # comment1\n\t#comment2\n port= 4242\n // comment3\n rate = 55\n}\n`\n\nfunc TestLexMultilineMap(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 2},\n\t\t{itemMapStart, \"\", 2},\n\t\t{itemKey, \"ip\", 3},\n\t\t{itemString, \"127.0.0.1\", 3},\n\t\t{itemCommentStart, \"\", 4},\n\t\t{itemText, \" comment1\", 4},\n\t\t{itemCommentStart, \"\", 5},\n\t\t{itemText, \"comment2\", 5},\n\t\t{itemKey, \"port\", 6},\n\t\t{itemInteger, \"4242\", 6},\n\t\t{itemCommentStart, \"\", 7},\n\t\t{itemText, \" comment3\", 7},\n\t\t{itemKey, \"rate\", 8},\n\t\t{itemInteger, \"55\", 8},\n\t\t{itemMapEnd, \"\", 9},\n\t\t{itemEOF, \"\", 9},\n\t}\n\n\tlx := lex(mlMap)\n\texpect(t, lx, expectedItems)\n}\n\nvar nestedMap = `\nfoo = {\n host = {\n\tip = '127.0.0.1'\n\tport= 4242\n\t// rate comment\n\trate = 55\n }\n}\n`\n\nfunc TestLexNestedMaps(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 2},\n\t\t{itemMapStart, \"\", 2},\n\t\t{itemKey, \"host\", 3},\n\t\t{itemMapStart, \"\", 3},\n\t\t{itemKey, \"ip\", 4},\n\t\t{itemString, \"127.0.0.1\", 4},\n\t\t{itemKey, \"port\", 5},\n\t\t{itemInteger, \"4242\", 5},\n\t\t{itemCommentStart, \"\", 6},\n\t\t{itemText, \" rate comment\", 6},\n\t\t{itemKey, \"rate\", 7},\n\t\t{itemInteger, \"55\", 7},\n\t\t{itemMapEnd, \"\", 8},\n\t\t{itemMapEnd, \"\", 9},\n\t\t{itemEOF, \"\", 5},\n\t}\n\n\tlx := lex(nestedMap)\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexQuotedKeys(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemInteger, \"123\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\tlx := lex(\"foo : 123\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"'foo' : 123\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"\\\"foo\\\" : 123\")\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexQuotedKeysWithSpace(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \" foo\", 1},\n\t\t{itemInteger, \"123\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\tlx := lex(\"' foo' : 123\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"\\\" foo\\\" : 123\")\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexColonKeySep(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemInteger, \"123\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\tlx := lex(\"foo : 123\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo:123\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo: 123\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo: 123\\r\\n\")\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexWhitespaceKeySep(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemInteger, \"123\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\tlx := lex(\"foo 123\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo 123\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo\\t123\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo\\t\\t123\\r\\n\")\n\texpect(t, lx, expectedItems)\n}\n\nvar nestedWhitespaceMap = `\nfoo {\n host {\n ip = '127.0.0.1'\n port= 4242\n }\n}\n`\n\nfunc TestLexNestedWhitespaceMaps(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 2},\n\t\t{itemMapStart, \"\", 2},\n\t\t{itemKey, \"host\", 3},\n\t\t{itemMapStart, \"\", 3},\n\t\t{itemKey, \"ip\", 4},\n\t\t{itemString, \"127.0.0.1\", 4},\n\t\t{itemKey, \"port\", 5},\n\t\t{itemInteger, \"4242\", 5},\n\t\t{itemMapEnd, \"\", 6},\n\t\t{itemMapEnd, \"\", 7},\n\t\t{itemEOF, \"\", 5},\n\t}\n\n\tlx := lex(nestedWhitespaceMap)\n\texpect(t, lx, expectedItems)\n}\n\n// Not currently supported\nvar tableOfArrays = `\ntable [\n [ 1, 2],\n [ \"a\", \"b\"],\n]\n`\n\nfunc TestLexTableOfArrays(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"table\", 2},\n\t\t{itemArrayStart, \"\", 2},\n\t\t{itemArrayStart, \"\", 3},\n\t\t{itemInteger, \"1\", 3},\n\t\t{itemInteger, \"2\", 3},\n\t\t{itemArrayEnd, \"\", 3},\n\t\t{itemArrayStart, \"\", 4},\n\t\t{itemString, \"a\", 4},\n\t\t{itemString, \"b\", 4},\n\t\t{itemArrayEnd, \"\", 4},\n\t\t{itemArrayEnd, \"\", 5},\n\t\t{itemEOF, \"\", 6},\n\t}\n\n\tlx := lex(tableOfArrays)\n\texpect(t, lx, expectedItems)\n}\n\nvar semicolons = `\nfoo = 123;\nbar = 'baz';\nbaz = 'boo'\nmap {\n id = 1;\n}\n`\n\nfunc TestOptionalSemicolons(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 2},\n\t\t{itemInteger, \"123\", 2},\n\t\t{itemKey, \"bar\", 3},\n\t\t{itemString, \"baz\", 3},\n\t\t{itemKey, \"baz\", 4},\n\t\t{itemString, \"boo\", 4},\n\t\t{itemKey, \"map\", 5},\n\t\t{itemMapStart, \"\", 5},\n\t\t{itemKey, \"id\", 6},\n\t\t{itemInteger, \"1\", 6},\n\t\t{itemMapEnd, \"\", 7},\n\t\t{itemEOF, \"\", 5},\n\t}\n\n\tlx := lex(semicolons)\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexSemicolonChaining(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemString, \"1\", 1},\n\t\t{itemKey, \"bar\", 1},\n\t\t{itemFloat, \"2.2\", 1},\n\t\t{itemKey, \"baz\", 1},\n\t\t{itemBool, \"true\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\n\tlx := lex(\"foo='1'; bar=2.2; baz=true;\")\n\texpect(t, lx, expectedItems)\n}\n\nvar noquotes = `\nfoo = 123\nbar = baz\nbaz=boo\nmap {\n id:one\n id2 : onetwo\n}\nt true\nf false\ntstr \"true\"\ntkey = two\nfkey = five # This should be a string\n`\n\nfunc TestLexNonQuotedStrings(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 2},\n\t\t{itemInteger, \"123\", 2},\n\t\t{itemKey, \"bar\", 3},\n\t\t{itemString, \"baz\", 3},\n\t\t{itemKey, \"baz\", 4},\n\t\t{itemString, \"boo\", 4},\n\t\t{itemKey, \"map\", 5},\n\t\t{itemMapStart, \"\", 5},\n\t\t{itemKey, \"id\", 6},\n\t\t{itemString, \"one\", 6},\n\t\t{itemKey, \"id2\", 7},\n\t\t{itemString, \"onetwo\", 7},\n\t\t{itemMapEnd, \"\", 8},\n\t\t{itemKey, \"t\", 9},\n\t\t{itemBool, \"true\", 9},\n\t\t{itemKey, \"f\", 10},\n\t\t{itemBool, \"false\", 10},\n\t\t{itemKey, \"tstr\", 11},\n\t\t{itemString, \"true\", 11},\n\t\t{itemKey, \"tkey\", 12},\n\t\t{itemString, \"two\", 12},\n\t\t{itemKey, \"fkey\", 13},\n\t\t{itemString, \"five\", 13},\n\t\t{itemCommentStart, \"\", 13},\n\t\t{itemText, \" This should be a string\", 13},\n\n\t\t{itemEOF, \"\", 14},\n\t}\n\tlx := lex(noquotes)\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexMapQuotedKeys(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemMapStart, \"\", 1},\n\t\t{itemKey, \"bar\", 1},\n\t\t{itemInteger, \"4242\", 1},\n\t\t{itemMapEnd, \"\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\tlx := lex(\"foo = {'bar' = 4242}\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo = {\\\"bar\\\" = 4242}\")\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexSpecialCharsMapQuotedKeys(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"foo\", 1},\n\t\t{itemMapStart, \"\", 1},\n\t\t{itemKey, \"bar-1.2.3\", 1},\n\t\t{itemMapStart, \"\", 1},\n\t\t{itemKey, \"port\", 1},\n\t\t{itemInteger, \"4242\", 1},\n\t\t{itemMapEnd, \"\", 1},\n\t\t{itemMapEnd, \"\", 1},\n\t\t{itemEOF, \"\", 1},\n\t}\n\tlx := lex(\"foo = {'bar-1.2.3' = { port:4242 }}\")\n\texpect(t, lx, expectedItems)\n\tlx = lex(\"foo = {\\\"bar-1.2.3\\\" = { port:4242 }}\")\n\texpect(t, lx, expectedItems)\n}\n\nvar mlnestedmap = `\nsystems {\n allinone {\n description: \"This is a description.\"\n }\n}\n`\n\nfunc TestLexDoubleNestedMapsNewLines(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"systems\", 2},\n\t\t{itemMapStart, \"\", 2},\n\t\t{itemKey, \"allinone\", 3},\n\t\t{itemMapStart, \"\", 3},\n\t\t{itemKey, \"description\", 4},\n\t\t{itemString, \"This is a description.\", 4},\n\t\t{itemMapEnd, \"\", 5},\n\t\t{itemMapEnd, \"\", 6},\n\t\t{itemEOF, \"\", 7},\n\t}\n\tlx := lex(mlnestedmap)\n\texpect(t, lx, expectedItems)\n}\n\nvar blockexample = `\nnumbers (\n1234567890\n)\n`\n\nfunc TestLexBlockString(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"numbers\", 2},\n\t\t{itemString, \"1234567890\", 3},\n\t}\n\tlx := lex(blockexample)\n\texpect(t, lx, expectedItems)\n}\n\nfunc TestLexBlockStringEOF(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"numbers\", 2},\n\t\t{itemString, \"1234567890\", 3},\n\t}\n\tblockbytes := []byte(blockexample[0 : len(blockexample)-1])\n\tblockbytes = append(blockbytes, 0)\n\tlx := lex(string(blockbytes))\n\texpect(t, lx, expectedItems)\n}\n\nvar mlblockexample = `\nnumbers (\n 12(34)56\n (\n 7890\n )\n)\n`\n\nvar mlBlockTextVal = ` 12(34)56\n (\n 7890\n )`\n\nfunc TestLexBlockStringMultiLine(t *testing.T) {\n\texpectedItems := []item{\n\t\t{itemKey, \"numbers\", 2},\n\t\t{itemString, mlBlockTextVal, 6},\n\t}\n\tlx := lex(mlblockexample)\n\texpect(t, lx, expectedItems)\n}\n" }, { "path": "parse.go", "content": "// Copyright 2013 Apcera Inc. All rights reserved.\n\n// Conf is a configuration file format used by gnatsd. It is\n// a flexible format that combines the best of traditional\n// configuration formats and newer styles such as JSON and YAML.\npackage confl\n\n// The format supported is less restrictive than today's formats.\n// Supports mixed Arrays [], nested Maps {}, multiple comment types (# and //)\n// Also supports key value assigments using '=' or ':' or whiteSpace()\n// e.g. foo = 2, foo : 2, foo 2\n// maps can be assigned with no key separator as well\n// semicolons as value terminators in key/value assignments are optional\n//\n// see parse_test.go for more examples.\n\nimport (\n\t\"fmt\"\n\t\"log\"\n\t\"strconv\"\n\t\"strings\"\n\t\"time\"\n\t\"unicode/utf8\"\n\n\tu \"github.com/araddon/gou\"\n)\n\nvar _ = u.EMPTY\n\n// Parser will return a map of keys to interface{}, although concrete types\n// underly them. The values supported are string, bool, int64, float64, DateTime.\n// Arrays and nested Maps are also supported.\ntype parser struct {\n\tmapping map[string]interface{}\n\ttypes map[string]confType\n\tlx *lexer\n\n\t// A list of keys in the order that they appear in the data.\n\tordered []Key\n\n\t// the full key for the current hash in scope\n\tcontext Key\n\n\t// the base key name for everything except hashes\n\tcurrentKey string\n\n\t// rough approximation of line number\n\tapproxLine int\n\n\t// The current scoped context, can be array or map\n\tctx interface{}\n\n\t// stack of contexts, either map or array/slice stack\n\tctxs []interface{}\n\n\t// Keys stack\n\tkeys []string\n\n\t// A map of 'key.group.names' to whether they were created implicitly.\n\timplicits map[string]bool\n}\n\ntype parseError string\n\nfunc (pe parseError) Error() string {\n\treturn string(pe)\n}\n\nfunc Parse(data string) (map[string]interface{}, error) {\n\tp, err := parse(data)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn p.mapping, nil\n}\n\nfunc parse(data string) (p *parser, err error) {\n\n\tp = &parser{\n\t\tmapping: make(map[string]interface{}),\n\t\tlx: lex(data),\n\t\tctxs: make([]interface{}, 0, 4),\n\t\tkeys: make([]string, 0, 4),\n\t}\n\tp.pushContext(p.mapping)\n\n\tfor {\n\t\tit := p.next()\n\t\tif it.typ == itemEOF {\n\t\t\tbreak\n\t\t}\n\t\tif err := p.processItem(it); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\treturn p, nil\n}\n\nfunc (p *parser) panicf(format string, v ...interface{}) {\n\tmsg := fmt.Sprintf(\"Near line %d, key '%s': %s\",\n\t\tp.approxLine, p.current(), fmt.Sprintf(format, v...))\n\tpanic(parseError(msg))\n}\n\nfunc (p *parser) next() item {\n\treturn p.lx.nextItem()\n}\n\nfunc (p *parser) bug(format string, v ...interface{}) {\n\tlog.Fatalf(\"BUG: %s\\n\\n\", fmt.Sprintf(format, v...))\n}\n\nfunc (p *parser) expect(typ itemType) item {\n\tit := p.next()\n\tp.assertEqual(typ, it.typ)\n\treturn it\n}\n\nfunc (p *parser) assertEqual(expected, got itemType) {\n\tif expected != got {\n\t\tp.bug(\"Expected '%s' but got '%s'.\", expected, got)\n\t}\n}\n\nfunc (p *parser) pushContext(ctx interface{}) {\n\tp.ctxs = append(p.ctxs, ctx)\n\tp.ctx = ctx\n}\n\nfunc (p *parser) popContext() interface{} {\n\tif len(p.ctxs) == 0 {\n\t\tpanic(\"BUG in parser, context stack empty\")\n\t}\n\tli := len(p.ctxs) - 1\n\tlast := p.ctxs[li]\n\tp.ctxs = p.ctxs[0:li]\n\tp.ctx = p.ctxs[len(p.ctxs)-1]\n\treturn last\n}\n\nfunc (p *parser) pushKey(key string) {\n\tp.keys = append(p.keys, key)\n}\n\nfunc (p *parser) popKey() string {\n\tif len(p.keys) == 0 {\n\t\tpanic(\"BUG in parser, keys stack empty\")\n\t}\n\tli := len(p.keys) - 1\n\tlast := p.keys[li]\n\tp.keys = p.keys[0:li]\n\treturn last\n}\n\nfunc (p *parser) processItem(it item) error {\n\tswitch it.typ {\n\tcase itemError:\n\t\t//panic(\"error\")\n\t\treturn fmt.Errorf(\"Parse error on line %d: '%s'\", it.line, it.val)\n\tcase itemKey:\n\t\tp.pushKey(it.val)\n\tcase itemMapStart:\n\t\tnewCtx := make(map[string]interface{})\n\t\tp.pushContext(newCtx)\n\tcase itemMapEnd:\n\t\tp.setValue(p.popContext())\n\tcase itemString:\n\t\t// FIXME(dlc) sanitize string?\n\t\tp.setValue(maybeRemoveIndents(it.val))\n\tcase itemInteger:\n\t\tnum, err := strconv.ParseInt(it.val, 10, 64)\n\t\tif err != nil {\n\t\t\tif e, ok := err.(*strconv.NumError); ok &&\n\t\t\t\te.Err == strconv.ErrRange {\n\t\t\t\treturn fmt.Errorf(\"Integer '%s' is out of the range.\", it.val)\n\t\t\t} else {\n\t\t\t\treturn fmt.Errorf(\"Expected integer, but got '%s'.\", it.val)\n\t\t\t}\n\t\t}\n\t\tp.setValue(num)\n\tcase itemFloat:\n\t\tnum, err := strconv.ParseFloat(it.val, 64)\n\t\tif err != nil {\n\t\t\tif e, ok := err.(*strconv.NumError); ok &&\n\t\t\t\te.Err == strconv.ErrRange {\n\t\t\t\treturn fmt.Errorf(\"Float '%s' is out of the range.\", it.val)\n\t\t\t} else {\n\t\t\t\treturn fmt.Errorf(\"Expected float, but got '%s'.\", it.val)\n\t\t\t}\n\t\t}\n\t\tp.setValue(num)\n\tcase itemBool:\n\t\tswitch it.val {\n\t\tcase \"true\":\n\t\t\tp.setValue(true)\n\t\tcase \"false\":\n\t\t\tp.setValue(false)\n\t\tdefault:\n\t\t\treturn fmt.Errorf(\"Expected boolean value, but got '%s'.\", it.val)\n\t\t}\n\tcase itemDatetime:\n\t\tdt, err := time.Parse(\"2006-01-02T15:04:05Z\", it.val)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\n\t\t\t\t\"Expected Zulu formatted DateTime, but got '%s'.\", it.val)\n\t\t}\n\t\tp.setValue(dt)\n\tcase itemArrayStart:\n\t\tarray := make([]interface{}, 0)\n\t\tp.pushContext(array)\n\tcase itemArrayEnd:\n\t\tarray := p.ctx\n\t\tp.popContext()\n\t\tp.setValue(array)\n\t}\n\n\treturn nil\n}\n\nfunc (p *parser) setValue(val interface{}) {\n\t// Test to see if we are on an array or a map\n\n\t// Array processing\n\tif ctx, ok := p.ctx.([]interface{}); ok {\n\t\tp.ctx = append(ctx, val)\n\t\tp.ctxs[len(p.ctxs)-1] = p.ctx\n\t}\n\n\t// Map processing\n\tif ctx, ok := p.ctx.(map[string]interface{}); ok {\n\t\tkey := p.popKey()\n\t\t// FIXME(dlc), make sure to error if redefining same key?\n\t\tctx[key] = val\n\t}\n}\n\n// setType sets the type of a particular value at a given key.\n// It should be called immediately AFTER setValue.\n//\n// Note that if `key` is empty, then the type given will be applied to the\n// current context (which is either a table or an array of tables).\nfunc (p *parser) setType(key string, typ confType) {\n\tkeyContext := make(Key, 0, len(p.context)+1)\n\tfor _, k := range p.context {\n\t\tkeyContext = append(keyContext, k)\n\t}\n\tif len(key) > 0 { // allow type setting for hashes\n\t\tkeyContext = append(keyContext, key)\n\t}\n\tp.types[keyContext.String()] = typ\n}\n\n// addImplicit sets the given Key as having been created implicitly.\nfunc (p *parser) addImplicit(key Key) {\n\tp.implicits[key.String()] = true\n}\n\n// removeImplicit stops tagging the given key as having been implicitly created.\nfunc (p *parser) removeImplicit(key Key) {\n\tp.implicits[key.String()] = false\n}\n\n// isImplicit returns true if the key group pointed to by the key was created\n// implicitly.\nfunc (p *parser) isImplicit(key Key) bool {\n\treturn p.implicits[key.String()]\n}\n\n// current returns the full key name of the current context.\nfunc (p *parser) current() string {\n\tif len(p.currentKey) == 0 {\n\t\treturn p.context.String()\n\t}\n\tif len(p.context) == 0 {\n\t\treturn p.currentKey\n\t}\n\treturn fmt.Sprintf(\"%s.%s\", p.context, p.currentKey)\n}\n\n// for multi-line text comments lets remove the Indent\nfunc maybeRemoveIndents(s string) string {\n\tif !strings.Contains(s, \"\\n\") {\n\t\treturn s\n\t}\n\tlines := strings.Split(s, \"\\n\")\n\tindent := 0\nfindIndent:\n\tfor idx, r := range lines[0] {\n\t\tswitch r {\n\t\tcase '\\t', ' ':\n\t\t\t// keep consuming\n\t\tdefault:\n\t\t\t// first non-whitespace we are going to break\n\t\t\t// and use this as indent size. This makes a variety of assumptions\n\t\t\t// - subsequent indents use same mixture of spaces/tabs\n\t\t\tindent = idx\n\t\t\tbreak findIndent\n\t\t}\n\t}\n\n\tfor i, line := range lines {\n\t\t//u.Debugf(\"%v indent=%d line %q\", i, indent, line)\n\t\tif len(line) >= indent {\n\t\t\tlines[i] = line[indent:]\n\t\t}\n\t}\n\treturn strings.Join(lines, \"\\n\")\n}\n\nfunc replaceEscapes(s string) string {\n\treturn strings.NewReplacer(\n\t\t\"\\\\b\", \"\\u0008\",\n\t\t\"\\\\t\", \"\\u0009\",\n\t\t\"\\\\n\", \"\\u000A\",\n\t\t\"\\\\f\", \"\\u000C\",\n\t\t\"\\\\r\", \"\\u000D\",\n\t\t\"\\\\\\\"\", \"\\u0022\",\n\t\t\"\\\\/\", \"\\u002F\",\n\t\t\"\\\\\\\\\", \"\\u005C\",\n\t).Replace(s)\n}\n\nfunc (p *parser) replaceUnicode(s string) string {\n\tindexEsc := func() int {\n\t\treturn strings.Index(s, \"\\\\u\")\n\t}\n\tfor i := indexEsc(); i != -1; i = indexEsc() {\n\t\tasciiBytes := s[i+2 : i+6]\n\t\ts = strings.Replace(s, s[i:i+6], p.asciiEscapeToUnicode(asciiBytes), -1)\n\t}\n\treturn s\n}\n\nfunc (p *parser) asciiEscapeToUnicode(s string) string {\n\thex, err := strconv.ParseUint(strings.ToLower(s), 16, 32)\n\tif err != nil {\n\t\tp.bug(\"Could not parse '%s' as a hexadecimal number, but the \"+\n\t\t\t\"lexer claims it's OK: %s\", s, err)\n\t}\n\n\t// BUG(burntsushi)\n\t// I honestly don't understand how this works. I can't seem\n\t// to find a way to make this fail. I figured this would fail on invalid\n\t// UTF-8 characters like U+DCFF, but it doesn't.\n\tr := string(rune(hex))\n\tif !utf8.ValidString(r) {\n\t\tp.panicf(\"Escaped character '\\\\u%s' is not valid UTF-8.\", s)\n\t}\n\treturn string(r)\n}\n" }, { "path": "parse_test.go", "content": "package confl\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"testing\"\n)\n\n// Test to make sure we get what we expect.\n\nfunc test(t *testing.T, data string, ex map[string]interface{}) {\n\tm, err := Parse(data)\n\tif err != nil {\n\t\tt.Fatalf(\"Received err: %v\\n\", err)\n\t}\n\tif m == nil {\n\t\tt.Fatal(\"Received nil map\")\n\t}\n\n\tif !reflect.DeepEqual(m, ex) {\n\t\tt.Fatalf(\"Not Equal:\\nReceived: '%+v'\\nExpected: '%+v'\\n\", m, ex)\n\t}\n}\n\nfunc TestParseSimpleTopLevel(t *testing.T) {\n\tex := map[string]interface{}{\n\t\t\"foo\": \"1\",\n\t\t\"bar\": float64(2.2),\n\t\t\"baz\": true,\n\t\t\"boo\": int64(22),\n\t}\n\ttest(t, \"foo='1'; bar=2.2; baz=true; boo=22\", ex)\n}\n\nvar sample1 = `\nfoo {\n host {\n ip = '127.0.0.1'\n port = 4242\n }\n servers = [ \"a.com\", \"b.com\", \"c.com\"]\n}\n`\n\nfunc TestParseSample1(t *testing.T) {\n\tex := map[string]interface{}{\n\t\t\"foo\": map[string]interface{}{\n\t\t\t\"host\": map[string]interface{}{\n\t\t\t\t\"ip\": \"127.0.0.1\",\n\t\t\t\t\"port\": int64(4242),\n\t\t\t},\n\t\t\t\"servers\": []interface{}{\"a.com\", \"b.com\", \"c.com\"},\n\t\t},\n\t}\n\ttest(t, sample1, ex)\n}\n\nvar cluster = `\ncluster {\n port: 4244\n\n authorization {\n user: route_user\n password: top_secret\n timeout: 1\n }\n\n # Routes are actively solicited and connected to from this server.\n # Other servers can connect to us if they supply the correct credentials\n # in their routes definitions from above.\n\n // Test both styles of comments\n\n routes = [\n nats-route://foo:bar@apcera.me:4245\n nats-route://foo:bar@apcera.me:4246\n ]\n}\n`\n\nfunc TestParseSample2(t *testing.T) {\n\tex := map[string]interface{}{\n\t\t\"cluster\": map[string]interface{}{\n\t\t\t\"port\": int64(4244),\n\t\t\t\"authorization\": map[string]interface{}{\n\t\t\t\t\"user\": \"route_user\",\n\t\t\t\t\"password\": \"top_secret\",\n\t\t\t\t\"timeout\": int64(1),\n\t\t\t},\n\t\t\t\"routes\": []interface{}{\n\t\t\t\t\"nats-route://foo:bar@apcera.me:4245\",\n\t\t\t\t\"nats-route://foo:bar@apcera.me:4246\",\n\t\t\t},\n\t\t},\n\t}\n\n\ttest(t, cluster, ex)\n}\n\nvar sample3 = `\nfoo {\n expr = '(true == \"false\")'\n text = 'This is a multi-line\ntext block.'\n text2 (\n\t\t\thello world\n\t\t\t this is multi line\n\n\t\t\twith empty line\n)\n}\n`\n\nfunc TestParseSample3(t *testing.T) {\n\tex := map[string]interface{}{\n\t\t\"foo\": map[string]interface{}{\n\t\t\t\"expr\": \"(true == \\\"false\\\")\",\n\t\t\t\"text\": \"This is a multi-line\\ntext block.\",\n\t\t\t\"text2\": \"hello world\\n this is multi line\\n\\nwith empty line\",\n\t\t},\n\t}\n\ttest(t, sample3, ex)\n}\n\nvar sample4 = `\n array [\n { abc: 123 }\n { xyz: \"word\" }\n ]\n`\n\nfunc TestParseSample4(t *testing.T) {\n\tex := map[string]interface{}{\n\t\t\"array\": []interface{}{\n\t\t\tmap[string]interface{}{\"abc\": int64(123)},\n\t\t\tmap[string]interface{}{\"xyz\": \"word\"},\n\t\t},\n\t}\n\ttest(t, sample4, ex)\n}\n\nvar sample5 = `\n table [\n [ 1, 123 ],\n [ \"a\", \"b\", \"c\"],\n ]\n`\n\nfunc TestParseSample5(t *testing.T) {\n\tex := map[string]interface{}{\n\t\t\"table\": []interface{}{\n\t\t\t[]interface{}{int64(1), int64(123)},\n\t\t\t[]interface{}{\"a\", \"b\", \"c\"},\n\t\t},\n\t}\n\ttest(t, sample5, ex)\n}\n\nfunc TestBigSlices(t *testing.T) {\n\ttxt := \"Hosts : [\"\n\tfor i := 0; i < 100; i++ {\n\t\ttxt += fmt.Sprintf(`\"http://192.168.1.%d:9999\", `, i)\n\t}\n\ttxt += `\"http://123.123.123.123:9999\"]` + \"\\n\"\n\n\tx := struct{ Hosts []string }{}\n\tif err := Unmarshal([]byte(txt), &x); err != nil {\n\t\tt.Fatalf(\"error unmarshaling sample: %v\", err)\n\t}\n\tif len(x.Hosts) != 101 {\n\t\tt.Fatalf(\"%d != 101\", len(x.Hosts))\n\t}\n\tfor i, v := range x.Hosts {\n\t\tif i < 100 && v != fmt.Sprintf(\"http://192.168.1.%d:9999\", i) {\n\t\t\tt.Errorf(\"%d unexpected: %s\", i, v)\n\t\t}\n\t}\n\tif x.Hosts[100] != \"http://123.123.123.123:9999\" {\n\t\tt.Errorf(\"unexpected: %s\", x.Hosts[100])\n\t}\n}\n" }, { "path": "type_check.go", "content": "package confl\n\n// represents any Go type that corresponds to a internal type.\ntype confType interface {\n\ttypeString() string\n}\n\n// typeEqual accepts any two types and returns true if they are equal.\nfunc typeEqual(t1, t2 confType) bool {\n\tif t1 == nil || t2 == nil {\n\t\treturn false\n\t}\n\treturn t1.typeString() == t2.typeString()\n}\n\nfunc typeIsHash(t confType) bool {\n\treturn typeEqual(t, confHash) || typeEqual(t, confArrayHash)\n}\n\ntype confBaseType string\n\nfunc (btype confBaseType) typeString() string {\n\treturn string(btype)\n}\n\nfunc (btype confBaseType) String() string {\n\treturn btype.typeString()\n}\n\nvar (\n\tconfInteger confBaseType = \"Integer\"\n\tconfFloat confBaseType = \"Float\"\n\tconfDatetime confBaseType = \"Datetime\"\n\tconfString confBaseType = \"String\"\n\tconfBool confBaseType = \"Bool\"\n\tconfArray confBaseType = \"Array\"\n\tconfHash confBaseType = \"Hash\"\n\tconfArrayHash confBaseType = \"ArrayHash\"\n)\n\n// typeOfPrimitive returns a confType of any primitive value in conf.\n// Primitive values are: Integer, Float, Datetime, String and Bool.\n//\n// Passing a lexer item other than the following will cause a BUG message\n// to occur: itemString, itemBool, itemInteger, itemFloat, itemDatetime.\nfunc (p *parser) typeOfPrimitive(lexItem item) confType {\n\tswitch lexItem.typ {\n\tcase itemInteger:\n\t\treturn confInteger\n\tcase itemFloat:\n\t\treturn confFloat\n\tcase itemDatetime:\n\t\treturn confDatetime\n\tcase itemString:\n\t\treturn confString\n\tcase itemBool:\n\t\treturn confBool\n\t}\n\tp.bug(\"Cannot infer primitive type of lex item '%s'.\", lexItem)\n\tpanic(\"unreachable\")\n}\n\n// typeOfArray returns a confType for an array given a list of types of its\n// values.\n//\n// In the current spec, if an array is homogeneous, then its type is always\n// \"Array\". If the array is not homogeneous, an error is generated.\nfunc (p *parser) typeOfArray(types []confType) confType {\n\t// Empty arrays are cool.\n\tif len(types) == 0 {\n\t\treturn confArray\n\t}\n\n\ttheType := types[0]\n\tfor _, t := range types[1:] {\n\t\tif !typeEqual(theType, t) {\n\t\t\tp.panicf(\"Array contains values of type '%s' and '%s', but arrays \"+\n\t\t\t\t\"must be homogeneous.\", theType, t)\n\t\t}\n\t}\n\treturn confArray\n}\n" }, { "path": "type_fields.go", "content": "package confl\n\n// Struct field handling is adapted from code in encoding/json:\n//\n// Copyright 2010 The Go Authors. All rights reserved.\n// Use of this source code is governed by a BSD-style\n// license that can be found in the Go distribution.\n\nimport (\n\t\"reflect\"\n\t\"sort\"\n\t\"strings\"\n\t\"sync\"\n)\n\n// A field represents a single field found in a struct.\ntype field struct {\n\tname string // the name of the field (`confl` tag included)\n\ttag bool // whether field has a `confl` tag\n\tindex []int // represents the depth of an anonymous field\n\ttyp reflect.Type // the type of the field\n}\n\n// byName sorts field by name, breaking ties with depth,\n// then breaking ties with \"name came from confl tag\", then\n// breaking ties with index sequence.\ntype byName []field\n\nfunc (x byName) Len() int { return len(x) }\n\nfunc (x byName) Swap(i, j int) { x[i], x[j] = x[j], x[i] }\n\nfunc (x byName) Less(i, j int) bool {\n\tif x[i].name != x[j].name {\n\t\treturn x[i].name < x[j].name\n\t}\n\tif len(x[i].index) != len(x[j].index) {\n\t\treturn len(x[i].index) < len(x[j].index)\n\t}\n\tif x[i].tag != x[j].tag {\n\t\treturn x[i].tag\n\t}\n\treturn byIndex(x).Less(i, j)\n}\n\n// byIndex sorts field by index sequence.\ntype byIndex []field\n\nfunc (x byIndex) Len() int { return len(x) }\n\nfunc (x byIndex) Swap(i, j int) { x[i], x[j] = x[j], x[i] }\n\nfunc (x byIndex) Less(i, j int) bool {\n\tfor k, xik := range x[i].index {\n\t\tif k >= len(x[j].index) {\n\t\t\treturn false\n\t\t}\n\t\tif xik != x[j].index[k] {\n\t\t\treturn xik < x[j].index[k]\n\t\t}\n\t}\n\treturn len(x[i].index) < len(x[j].index)\n}\n\n// typeFields returns a list of fields that confl should recognize for the given\n// type. The algorithm is breadth-first search over the set of structs to\n// include - the top struct and then any reachable anonymous structs.\nfunc typeFields(t reflect.Type) []field {\n\t// Anonymous fields to explore at the current level and the next.\n\tcurrent := []field{}\n\tnext := []field{{typ: t}}\n\n\t// Count of queued names for current level and the next.\n\tcount := map[reflect.Type]int{}\n\tnextCount := map[reflect.Type]int{}\n\n\t// Types already visited at an earlier level.\n\tvisited := map[reflect.Type]bool{}\n\n\t// Fields found.\n\tvar fields []field\n\n\tfor len(next) > 0 {\n\t\tcurrent, next = next, current[:0]\n\t\tcount, nextCount = nextCount, map[reflect.Type]int{}\n\n\t\tfor _, f := range current {\n\t\t\tif visited[f.typ] {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tvisited[f.typ] = true\n\n\t\t\t// Scan f.typ for fields to include.\n\t\t\tfor i := 0; i < f.typ.NumField(); i++ {\n\t\t\t\tsf := f.typ.Field(i)\n\t\t\t\tif sf.PkgPath != \"\" { // unexported\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\tname := sf.Tag.Get(\"confl\")\n\t\t\t\tif name == \"-\" {\n\t\t\t\t\tcontinue\n\t\t\t\t} else if name != \"\" {\n\t\t\t\t\t// ClientID string `confl:\"Client_id,omitempty\"`\n\t\t\t\t\tparts := strings.Split(name, \",\")\n\t\t\t\t\tif len(parts) > 1 {\n\t\t\t\t\t\tname = parts[0]\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tif name == \"\" {\n\t\t\t\t\tname = sf.Tag.Get(\"json\")\n\t\t\t\t\tif name == \"-\" {\n\t\t\t\t\t\tcontinue\n\t\t\t\t\t}\n\t\t\t\t\t// ClientID string `json:\"Client_id,omitempty\"`\n\t\t\t\t\tparts := strings.Split(name, \",\")\n\t\t\t\t\tif len(parts) > 1 {\n\t\t\t\t\t\tname = parts[0]\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\tindex := make([]int, len(f.index)+1)\n\t\t\t\tcopy(index, f.index)\n\t\t\t\tindex[len(f.index)] = i\n\n\t\t\t\tft := sf.Type\n\t\t\t\tif ft.Name() == \"\" && ft.Kind() == reflect.Ptr {\n\t\t\t\t\t// Follow pointer.\n\t\t\t\t\tft = ft.Elem()\n\t\t\t\t}\n\n\t\t\t\t// Record found field and index sequence.\n\t\t\t\tif name != \"\" || !sf.Anonymous || ft.Kind() != reflect.Struct {\n\t\t\t\t\ttagged := name != \"\"\n\t\t\t\t\tif name == \"\" {\n\t\t\t\t\t\tname = sf.Name\n\t\t\t\t\t}\n\t\t\t\t\tfields = append(fields, field{name, tagged, index, ft})\n\t\t\t\t\tif count[f.typ] > 1 {\n\t\t\t\t\t\t// If there were multiple instances, add a second,\n\t\t\t\t\t\t// so that the annihilation code will see a duplicate.\n\t\t\t\t\t\t// It only cares about the distinction between 1 or 2,\n\t\t\t\t\t\t// so don't bother generating any more copies.\n\t\t\t\t\t\tfields = append(fields, fields[len(fields)-1])\n\t\t\t\t\t}\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\n\t\t\t\t// Record new anonymous struct to explore in next round.\n\t\t\t\tnextCount[ft]++\n\t\t\t\tif nextCount[ft] == 1 {\n\t\t\t\t\tf := field{name: ft.Name(), index: index, typ: ft}\n\t\t\t\t\tnext = append(next, f)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\n\tsort.Sort(byName(fields))\n\n\t// Delete all fields that are hidden by the Go rules for embedded fields,\n\t// except that fields with tags are promoted.\n\n\t// The fields are sorted in primary order of name, secondary order\n\t// of field index length. Loop over names; for each name, delete\n\t// hidden fields by choosing the one dominant field that survives.\n\tout := fields[:0]\n\tfor advance, i := 0, 0; i < len(fields); i += advance {\n\t\t// One iteration per name.\n\t\t// Find the sequence of fields with the name of this first field.\n\t\tfi := fields[i]\n\t\tname := fi.name\n\t\tfor advance = 1; i+advance < len(fields); advance++ {\n\t\t\tfj := fields[i+advance]\n\t\t\tif fj.name != name {\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tif advance == 1 { // Only one field with this name\n\t\t\tout = append(out, fi)\n\t\t\tcontinue\n\t\t}\n\t\tdominant, ok := dominantField(fields[i : i+advance])\n\t\tif ok {\n\t\t\tout = append(out, dominant)\n\t\t}\n\t}\n\n\tfields = out\n\tsort.Sort(byIndex(fields))\n\n\treturn fields\n}\n\n// dominantField looks through the fields, all of which are known to\n// have the same name, to find the single field that dominates the\n// others using Go's embedding rules, modified by the presence of\n// tags. If there are multiple top-level fields, the boolean\n// will be false: This condition is an error in Go and we skip all\n// the fields.\nfunc dominantField(fields []field) (field, bool) {\n\t// The fields are sorted in increasing index-length order. The winner\n\t// must therefore be one with the shortest index length. Drop all\n\t// longer entries, which is easy: just truncate the slice.\n\tlength := len(fields[0].index)\n\ttagged := -1 // Index of first tagged field.\n\tfor i, f := range fields {\n\t\tif len(f.index) > length {\n\t\t\tfields = fields[:i]\n\t\t\tbreak\n\t\t}\n\t\tif f.tag {\n\t\t\tif tagged >= 0 {\n\t\t\t\t// Multiple tagged fields at the same level: conflict.\n\t\t\t\t// Return no field.\n\t\t\t\treturn field{}, false\n\t\t\t}\n\t\t\ttagged = i\n\t\t}\n\t}\n\tif tagged >= 0 {\n\t\treturn fields[tagged], true\n\t}\n\t// All remaining fields have the same length. If there's more than one,\n\t// we have a conflict (two fields named \"X\" at the same level) and we\n\t// return no field.\n\tif len(fields) > 1 {\n\t\treturn field{}, false\n\t}\n\treturn fields[0], true\n}\n\nvar fieldCache struct {\n\tsync.RWMutex\n\tm map[reflect.Type][]field\n}\n\n// cachedTypeFields is like typeFields but uses a cache to avoid repeated work.\nfunc cachedTypeFields(t reflect.Type) []field {\n\tfieldCache.RLock()\n\tf := fieldCache.m[t]\n\tfieldCache.RUnlock()\n\tif f != nil {\n\t\treturn f\n\t}\n\n\t// Compute fields without lock.\n\t// Might duplicate effort but won't hold other computations back.\n\tf = typeFields(t)\n\tif f == nil {\n\t\tf = []field{}\n\t}\n\n\tfieldCache.Lock()\n\tif fieldCache.m == nil {\n\t\tfieldCache.m = map[reflect.Type][]field{}\n\t}\n\tfieldCache.m[t] = f\n\tfieldCache.Unlock()\n\treturn f\n}\n" } ]