[
  {
    "path": ".luacheckrc",
    "content": "globals = {\n  \"torch\",\n  \"cutorch\",\n  \"nn\",\n  \"paths\",\n  \"cudnn\",\n  \"image\",\n  \"onmt\",\n  \"tds\"\n}\n\nself = false\n"
  },
  {
    "path": "LICENSE.md",
    "content": "MIT License\n\nCopyright (c) 2016 OpenNMT\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n"
  },
  {
    "path": "README.md",
    "content": "# Im2Text\n\nA deep learning-based approach to learning the image-to-text conversion, built on top of the <a href=\"https://opennmt.github.io/\">OpenNMT</a> system. It is completely data-driven, hence can be used for a variety of image-to-text problems, such as image captioning, optical character recognition and LaTeX decompilation. \n\nTake LaTeX decompilation as an example, given a formula image:\n\n<p align=\"center\"><img src=\"http://lstm.seas.harvard.edu/latex/results/website/images/119b93a445-orig.png\"></p>\n\nThe goal is to infer the LaTeX source that can be compiled to such an image:\n\n```\n d s _ { 1 1 } ^ { 2 } = d x ^ { + } d x ^ { - } + l _ { p } ^ { 9 } \\frac { p _ { - } } { r ^ { 7 } } \\delta ( x ^ { - } ) d x ^ { - } d x ^ { - } + d x _ { 1 } ^ { 2 } + \\; \\cdots \\; + d x _ { 9 } ^ { 2 } \n```\n\nThe paper (http://arxiv.org/pdf/1609.04938v1.pdf) provides more technical details of this model.\n\n## Installation\n\nIm2Text is built on top of <a href=\"https://opennmt.github.io/\">OpenNMT</a>, which is packed in this project. It also depends on `tds`, `class`, `cudnn`, `cutorch` and `paths`. Currently we only support **GPU**.\n\n\n## Quick Start\n\nTo get started, we provide a toy Math-to-LaTex example. We assume that the working directory is `Im2Text` throughout this document.\n\nIm2Text consists of two commands:\n\n1) Train the model.\n\n```\nth src/train.lua -phase train -gpu_id 1 -input_feed -model_dir model \\\n-image_dir data/images -data_path data/train.txt -val_data_path data/validate.txt -label_path data/labels.txt -vocab_file data/vocab.txt \\\n-batch_size 20 -beam_size 1 \\\n-max_num_tokens 150 -max_image_width 500 -max_image_height 160 \\\n-max_grad_norm 20.0 -learning_rate 0.1 -decay perplexity_only\n```\n\n2) Translate the images.\n\n```\nth src/train.lua -phase test -gpu_id 1 -load_model -model_dir model \\\n-image_dir data/images -data_path data/test.txt \\\n-output_dir results \\\n-batch_size 2 -beam_size 5 \\\n-max_num_tokens 500 -max_image_width 800 -max_image_height 800\n```\n\nThe above dataset is sampled from the [processed-im2latex-100k-dataset](http://lstm.seas.harvard.edu/latex/processed-im2latex-100k-dataset.tgz). We provide a trained model [[link]](http://lstm.seas.harvard.edu/latex/model_latest) on this dataset. In order to use it, download and put it under `model_dir` before translating the images.\n\n## Data Format\n\n* `-image_dir`: The directory containing the images. Since images of the same size can be batched together, we suggest padding images of similar sizes to the same size in order to facilitate training.\n\n* `-label_path`: The file storing the tokenized labels, one label per line. It shall look like:\n```\n<label0_token0> <label0_token1> ... <label0_tokenN0>\n<label1_token0> <label1_token1> ... <label1_tokenN1>\n<label2_token0> <label2_token1> ... <label2_tokenN2>\n...\n```\n\n* `-data_path`: The file storing the image-label pairs. Each line starts with the path of the image (relative to `image_dir`), followed by the index of the label in `label_path` (index counts from 0). At test time, the label indexes can be omitted.\n```\n<image0_path> <label_index0>\n<image1_path> <label_index1>\n<image2_path> <label_index2>\n...\n```\n\n* `-vocab_file`: The vocabulary file. Each line corresponds to a token. The tokens not in `vocab_file` will be considered unknown (UNK).\n\n\n## Options\n\nFor a complete set of options, run `th src/train.lua -h`.\n"
  },
  {
    "path": "data/labels.txt",
    "content": "d s ^ { 2 } = ( 1 - { \\frac { q c o s \\theta } { r } } ) ^ { \\frac { 2 } { 1 + \\alpha ^ { 2 } } } \\lbrace d r ^ { 2 } + r ^ { 2 } d \\theta ^ { 2 } + r ^ { 2 } s i n ^ { 2 } \\theta d \\varphi ^ { 2 } \\rbrace - { \\frac { d t ^ { 2 } } { ( 1 - { \\frac { q c o s \\theta } { r } } ) ^ { \\frac { 2 } { 1 + \\alpha ^ { 2 } } } } } \\, .\n\\widetilde \\gamma _ { \\mathrm { h o p f } } \\simeq \\sum _ { n > 0 } \\widetilde { G } _ { n } { \\frac { ( - a ) ^ { n } } { 2 ^ { 2 n - 1 } } }\n( { \\cal L } _ { a } g ) _ { i j } = 0 , \\ \\ \\ \\ ( { \\cal L } _ { a } H ) _ { i j k } = 0 ,\nS _ { s t a t } = 2 \\pi \\sqrt { N _ { 5 } ^ { ( 1 ) } N _ { 5 } ^ { ( 2 ) } N _ { 5 } ^ { ( 3 ) } } \\left( \\sqrt { n } + \\sqrt { \\bar { n } } \\right)\n\\hat { N } _ { 3 } = \\sum \\sp f _ { j = 1 } a _ { j } \\sp { \\dagger } a _ { j } \\, .\n+ \\int \\! \\! d ^ { D } \\! z _ { 1 } d ^ { D } \\! z _ { 2 } d ^ { D } \\! z _ { 3 } \\left. \\frac { \\delta ^ { 2 } W } { \\delta j ( x ) \\delta j ( z _ { 1 } ) } \\, \\frac { \\delta ^ { 2 } W } { \\delta j ( x ) \\delta j ( z _ { 2 } ) } \\, \\frac { \\delta ^ { 2 } W } { \\delta j ( x ) \\delta j ( z _ { 3 } ) } \\, \\frac { \\delta ^ { 3 } \\Gamma } { \\delta \\Phi ( z _ { 1 } ) \\delta \\Phi ( z _ { 2 } ) \\delta \\Phi ( z _ { 3 } ) } \\right] ,\n\\, ^ { * } d \\, ^ { * } H = \\kappa \\, ^ { * } d \\phi = J _ { B } .\n{ \\frac { \\phi ^ { \\prime \\prime } } { A } } + { \\frac { 1 } { A } } \\left( - { \\frac { 1 } { 2 } } { \\frac { A ^ { \\prime } } { A } } + 2 { \\frac { B ^ { \\prime } } { B } } + { \\frac { 2 } { r } } \\right) \\phi ^ { \\prime } - { \\frac { 2 } { r ^ { 2 } } } \\phi - \\lambda \\phi ( \\phi ^ { 2 } - \\eta ^ { 2 } ) = 0 \\, .\n\\partial _ { \\mu } ( F ^ { \\mu \\nu } - e j ^ { \\mu } x ^ { \\nu } ) = 0 .\nV _ { n s } ( { \\tilde { x } } ) = \\left( \\frac { { \\tilde { m } } N ^ { 2 } } { 1 6 \\pi } \\right) N g ^ { 2 n s - 1 } { \\tilde { x } } ^ { 2 } \\left\\{ { \\tilde { x } } ^ { 2 } - \\frac { 2 { \\tilde { b } } } { 3 } { \\tilde { x } } + \\frac { { \\tilde { b } } ^ { 2 } } { 3 } - ( - 1 ) ^ { n s } { \\tilde { c } } \\right\\} \\, .\ng _ { i j } ( x ) = { \\frac { 1 } { a ^ { 2 } } } \\, \\delta _ { i j } , ~ ~ \\phi ^ { a } ( x ) = \\phi ^ { a } , \\quad ( a , \\phi ^ { a } \\! : ~ \\mathrm { c o n s t . } )\n\\rho _ { L } ( q ) = \\sum _ { m = 1 } ^ { L } \\ P _ { L } ( m ) \\ { \\frac { 1 } { q ^ { m - 1 } } } \\ \\ .\ne x p \\left( - \\frac { \\partial } { \\partial \\alpha _ { j } } \\theta ^ { j k } \\frac { \\partial } { \\partial \\alpha _ { k } } \\right)\nL _ { 0 } = \\Phi ( w ) = \\bigtriangleup \\Phi ( w ) ,\n\\left( D ^ { * } D ^ { * } + m ^ { 2 } \\right) { \\cal H } = 0\n{ \\frac { d V } { d \\Phi } } = - { \\frac { w \\Phi } { \\Phi _ { \\! _ { 0 } } ^ { 2 } } } \\, .\ng ( z , \\bar { z } ) = - \\frac { 1 } { 2 } \\left[ x ( z , \\bar { z } ) \\, s + x ^ { * } ( z , \\bar { z } ) \\, s ^ { * } + u ^ { * } ( z , \\bar { z } ) \\, t + u ( z , \\bar { z } ) \\, t ^ { * } \\right] ,\nx _ { \\mu } ^ { c } = x _ { \\mu } + A _ { \\mu } .\ns = { \\frac { S } { V } } = { \\frac { A _ { H } } { l _ { p } ^ { 8 } V } } = { \\frac { T ^ { 2 } } { \\gamma } } .\n\\psi ( \\gamma ) = \\operatorname { e x p } { - ( { \\textstyle { \\frac { g ^ { 2 } } { 2 } } } ) \\int _ { \\gamma } d y ^ { a } \\int _ { \\gamma } d y ^ { a ^ { \\prime } } D _ { 1 } ( y - y ^ { \\prime } ) }\nE = E _ { 0 } + \\frac { 1 } { 2 \\operatorname { s i n h } ( \\gamma ( 0 ) / 2 ) } \\operatorname { s i n h } \\left( \\gamma ( 0 ) \\left( \\frac { 1 } { 2 } + c ( 0 ) \\right) \\right) h c \\nu _ { \\mathrm { v i b } }\n\\langle T _ { z z } \\rangle = - 3 \\times \\frac { \\pi ^ { 2 } } { 1 4 4 0 a ^ { 4 } } .\n\\partial _ { u } \\xi _ { z } ^ { ( 1 ) } + { \\frac { 1 } { u } } \\xi _ { z } ^ { ( 1 ) } = { \\frac { 1 } { ( \\pi T R ) ^ { 2 } u } } \\left[ C _ { z } H _ { z z } ^ { \\prime } + C _ { t } H _ { t z } ^ { \\prime } \\right] \\, .\nS \\sim \\tilde { \\psi } Q _ { o } \\tilde { \\psi } + g _ { s } ^ { 1 / 2 } \\tilde { \\psi } ^ { 3 } + \\tilde { \\phi } Q _ { c } \\tilde { \\phi } + g _ { s } \\tilde { \\phi } ^ { 3 } + \\tilde { \\phi } B ( g _ { s } ^ { 1 / 2 } \\tilde { \\psi } ) + \\cdots .\nC ( x ^ { \\prime } , x ^ { \\prime \\prime } ) = C \\Phi ( x ^ { \\prime } , x ^ { \\prime \\prime } ) \\ , \\quad \\Phi ( x ^ { \\prime } , x ^ { \\prime \\prime } ) = \\operatorname { e x p } \\left[ - i e \\int _ { x ^ { \\prime \\prime } } ^ { x ^ { \\prime } } d x ^ { \\mu } A _ { \\mu } ( x ) \\right] \\ ,\n\\tilde { \\alpha } = \\alpha \\beta ^ { - m } = \\left( \\begin{array} { c c c } { \\omega _ { k } ^ { - 2 y } \\omega _ { 2 d } ^ { 2 m } } & { 0 } & { 0 } \\\\ { 0 } & { \\omega _ { k } ^ { y } \\omega _ { 2 d } ^ { - m } } & { 0 } \\\\ { 0 } & { 0 } & { \\omega _ { k } ^ { y } \\omega _ { 2 d } ^ { - m } } \\\\ \\end{array} \\right)\nd s ^ { 2 } = H ^ { - 2 } f ( r ) d t ^ { 2 } + H ^ { 2 / ( n - 1 ) } ( f ( r ) ^ { - 1 } d r ^ { 2 } + r ^ { 2 } d \\Omega _ { n } ^ { 2 } ) ,\ny ^ { 2 } = \\rho \\; \\operatorname { c o s h } \\beta \\; \\operatorname { s i n } \\theta \\; \\operatorname { s i n } \\phi \\qquad \\qquad y ^ { 3 } = \\rho \\; \\operatorname { c o s } \\theta\ne ^ { A } = e ^ { A _ { 0 } } \\left( t _ { 0 } - \\mathrm { s i g n } ( m ) t \\right) ^ { - \\frac { m } { 2 } } \\; , \\; \\; \\; \\; \\chi = \\chi _ { 0 } \\left( t _ { 0 } - \\mathrm { s i g n } ( m ) t \\right) ^ { m } \\; ,\n\\gamma _ { j } { \\cal P } _ { j i } = \\frac { 4 } { 3 } \\{ [ A d \\, T ] [ t _ { 8 } ^ { c } , [ t _ { 8 } ^ { c } , { \\gamma } _ { j } ] ] [ A d \\, T ^ { - 1 } ] \\} { A d \\, { \\hat { g } } } _ { i j } .\nK _ { \\mu \\nu } ~ = ~ \\frac { 1 } { 2 } \\dot { g } _ { \\mu \\nu } .\nX ( u ) = { \\frac { \\left( \\pm i + e ^ { 3 \\eta } \\right) \\left( - 1 + { e ^ { u } } \\right) \\left( 1 + { e ^ { u } } \\right) x _ { 1 } } { 2 { e ^ { u } } \\left( \\pm i + { e ^ { 3 \\eta + u } } \\right) } } ,\n\\beta ( g ) \\frac { \\partial } { \\partial g } = 2 g \\beta ( g ) \\frac { \\partial } { \\partial g ^ { 2 } }\nA = a r ^ { \\beta } , \\quad B = b r ^ { \\beta + 2 } ; \\qquad a / b = c ( \\beta + 2 ) / ( \\beta - 2 ) ,\n\\delta W _ { P \\mu } = A _ { \\mu } \\Phi + B _ { P \\mu } ^ { \\alpha } K _ { P } ^ { \\alpha } \\ .\n\\frac { 1 } { d - 2 } \\tilde { \\Pi } ^ { 2 } - \\tilde { \\Pi } _ { a b } \\tilde { \\Pi } ^ { a b } = \\frac { \\left( d - 1 \\right) \\left( d - 2 \\right) } { \\ell ^ { 2 } } + R\n\\hat { e } = e / \\varepsilon , \\ \\ \\ \\ \\ \\ \\ \\ \\ \\hat { G } _ { 4 } = G _ { 4 } ,\nV _ { ( n , \\, m ) } ( z , \\overline { { z } } ) = : \\operatorname { e x p } i ( p _ { + } \\phi ( z ) + p _ { - } \\bar { \\phi } ( \\overline { { z } } ) ) : \\: .\n\\langle f | g \\rangle _ { { \\cal L } ^ { 1 | 2 } } = \\langle f _ { 0 } | g _ { 0 } \\rangle _ { \\cal L } ^ { s } + \\langle f _ { 1 } | g _ { 1 } \\rangle _ { \\cal L } ^ { s + 1 / 2 } + \\langle f _ { 2 } | g _ { 2 } \\rangle _ { \\cal L } ^ { s + 1 / 2 } + \\langle f _ { 3 } | g _ { 3 } \\rangle _ { \\cal L } ^ { s + 1 } \\, ,\n\\tilde { s } ^ { 0 } ( x , y ) = i e ^ { 2 } \\int \\! d ^ { 4 } \\! z \\, S _ { \\mathrm { F } } ( x , z ) \\, \\gamma ^ { \\mu } \\, S _ { \\mathrm { F } } ( z , y ) \\, [ d _ { \\mu } ( x - z ) + d _ { \\mu } ( z - y ) ]\n\\left\\{ \\begin{array} { l c l l } { \\phi ~ ( \\infty ) } & { = } & { 0 } & { , \\vspace { 3 m m } } \\\\ { \\phi ~ ( 0 ) } & { = } & { 1 } & { . } \\\\ \\end{array} \\right.\n{ \\cal P } _ { \\delta x } \\equiv { \\frac { k ^ { 3 } } { 2 \\pi ^ { 2 } } } | \\delta x | ^ { 2 } \\, ,\n\\psi ( x ) = - 2 \\phi ( x ) + 2 \\phi ( L ) + c ,\n{ } ^ { ( { } ^ { \\scriptstyle x } y ) } ( { } ^ { x } z ) = { } ^ { x } ( { } ^ { y } z ) , \\qquad \\forall x , y , z \\in X .\n\\delta ( L _ { 1 } + L _ { 2 } ) = 2 \\delta \\bar { \\theta } ( 1 + \\gamma ^ { ( p ) } ) T _ { ( p ) } ^ { \\nu } \\partial _ { \\nu } \\theta .\n\\frac { 1 } { 2 \\lambda f ^ { 2 } } \\int \\; d ^ { 4 } X \\, \\frac { d ^ { 4 } q } { \\left( 2 \\pi \\right) ^ { 4 } } \\left( \\varphi ( X ) \\right) ^ { 2 } \\tilde { \\pi } _ { 0 } ( q ) \\left[ \\partial _ { q } ^ { 2 } + \\frac { 4 i \\lambda } { q ^ { 2 } - \\Sigma ^ { 2 } ( q ) } \\right] \\tilde { \\pi } _ { 0 } ( q ) ,\n( K ^ { - 1 } ) _ { S } ^ { U } = - \\frac { z ^ { * 3 } ( 1 - A | z | ^ { 2 } ) P ^ { \\prime } ( y ) } { e ^ { \\tilde { K } / 2 } P ^ { \\prime \\prime } ( y ) } , \\mathrm { ~ } ( K ^ { - 1 } ) _ { T } ^ { U } = \\frac { ( T + T ^ { * } ) z ^ { * 3 } ( 1 - A | z | ^ { 2 } ) } { e ^ { \\tilde { K } / 2 } ( 1 - \\frac { \\bar { n } } { 3 } B ( 1 - A | z | ^ { 2 } ) \\| \\Pi \\| ) } ,\nG ^ { \\mu \\nu \\mu ^ { \\prime } \\nu ^ { \\prime } } = g ^ { \\mu \\mu ^ { \\prime } } g ^ { \\nu \\nu ^ { \\prime } } + g ^ { \\mu \\nu ^ { \\prime } } g ^ { \\nu \\mu ^ { \\prime } } - { \\frac { 2 } { D } } g ^ { \\mu \\nu } g ^ { \\mu ^ { \\prime } \\nu ^ { \\prime } } + C g ^ { \\mu \\nu } g ^ { \\mu ^ { \\prime } \\nu ^ { \\prime } } \\: .\n[ M _ { \\mu \\nu } , M _ { \\rho \\tau } ] = g _ { \\mu \\tau } \\, M _ { \\nu \\rho } - g _ { \\nu \\tau } M _ { \\mu \\rho } + g _ { \\nu \\rho } M _ { \\mu \\tau } - g _ { \\mu \\rho } M _ { \\nu \\tau } \\, ,\nA _ { 0 } = \\pm \\sqrt { { \\frac { 4 } { 3 ( 1 - \\alpha ) } } } e ^ { ( \\alpha - 1 ) \\phi } \\ .\nC _ { m } ( \\mu ) = { \\frac { 1 } { 2 \\pi i } } \\int _ { \\Gamma _ { r } } { \\frac { C _ { m } ( z ) } { z - \\mu } } d z ,\n\\xi = \\alpha ^ { - 1 } \\sqrt { \\rho } \\operatorname { c o s h } ( 2 \\alpha ^ { 2 } t ) \\, , \\quad \\eta = \\alpha ^ { - 1 } \\sqrt { \\rho } \\operatorname { s i n h } ( 2 \\alpha ^ { 2 } t )\na _ { 1 } = \\frac { 2 \\tilde { q } } { \\alpha ^ { 2 } ( D - 2 ) + 2 q \\tilde { q } } , ~ ~ ~ ~ a _ { 2 } = \\frac { \\alpha ^ { 2 } ( D - 2 ) } { \\alpha ^ { 2 } d ( D - 2 ) + 2 \\tilde { d } q ^ { 2 } }\n\\theta \\epsilon ^ { i } = \\zeta ^ { i } \\, ; \\qquad \\theta \\zeta ^ { i } = \\epsilon ^ { i } \\, ; \\qquad \\theta \\eta ^ { i } = - \\eta ^ { i } \\, .\n{ \\cal A } _ { f i } ( s ) = - i \\frac { Q _ { \\mu \\nu } V _ { f } ^ { \\mu } ( \\bar { s } ) V _ { i } ^ { \\nu } ( \\bar { s } ) } { ( s - \\bar { s } ) [ 1 - A ^ { \\prime } ( \\bar { s } ) ] } + N ,\nS = \\frac { 1 } { G } \\int d x d t \\: \\sqrt { - \\bar { g } } \\: e ^ { - 2 \\bar { \\phi } } ( \\bar { R } + 4 ( \\bar { \\nabla } \\bar { \\phi } ) ^ { 2 } + 4 \\lambda ^ { 2 } ) - \\frac { 1 } { 2 } \\int d x d t \\: \\sqrt { - \\bar { g } } \\: \\sum _ { i = 1 } ^ { N } ( \\bar { \\nabla } f _ { i } ) ^ { 2 } ,\n\\phi ^ { A } = \\frac { \\partial F ( \\phi , \\phi ^ { * } ) } { \\partial \\phi _ { A } ^ { * } } \\hspace { 2 c m } \\phi _ { A } ^ { * } = \\frac { \\partial F ( \\phi , \\phi ^ { * } ) } { \\partial \\phi ^ { A } } .\nQ ^ { M } = \\dot { g } _ { 0 } \\int _ { \\phi ( \\Sigma ) } \\sum _ { i = 1 } ^ { l } \\beta _ { i } \\eta _ { i } \\int _ { N _ { i } } \\frac 1 { \\sqrt { g } } \\delta ^ { D } ( x - z _ { i } ( u ) ) \\sqrt { g _ { u } }\n\\psi ( x ) = [ 2 \\pi \\sigma ^ { 2 } ] ^ { - 1 / 4 } \\operatorname { e x p } \\left[ - \\left( \\frac { x - x _ { 0 } } { 2 \\sigma } \\right) ^ { 2 } + i p _ { 0 } x \\right] ,\nA _ { \\mu } = \\bar { A } _ { \\mu } ( \\phi ) + a _ { \\mu } \\ ,\n\\left[ D _ { f } , D \\right] = 0 \\, .\n\\operatorname { e x p } _ { q } A = \\sum _ { n = 0 } ^ { \\infty } \\frac { A ^ { n } } { [ n ] ! }\n\\langle b _ { 1 } ( z , \\bar { z } ) a _ { 1 } ( z ^ { \\prime } , \\bar { z } ^ { \\prime } ) \\rangle = - { \\frac { 1 } { \\pi } } \\partial ~ K _ { 0 } ( d ^ { 2 } m ^ { 2 } ( { \\bf p } ) ) ~ ,\n\\lambda _ { + } = \\frac { 1 + i \\omega } { 2 } , \\quad \\lambda _ { - } = \\frac { 1 - i \\omega } { 2 }\n1 + \\frac { 2 \\pi \\Lambda G } { 9 \\alpha } > 0\ne ^ { - K } = \\pm \\frac { W ^ { 3 / 2 } } { \\omega _ { 1 } \\omega _ { 2 } \\omega _ { 3 } } \\ ,\n{ \\cal { Z } } ( \\tau { } ) = \\sum _ { m } \\int { \\cal { D } } \\Omega { \\cal { D } } V { \\mathrm { V o l } } _ { Z M } { \\mathrm { d e t } } ( d _ { 2 } )\n| 0 ( t ) \\rangle _ { e , \\mu } \\equiv G _ { \\theta } ^ { - 1 } ( t ) | 0 \\rangle _ { 1 , 2 } \\, ,\n\\Gamma _ { i j } ^ { k } = ( \\partial _ { i } G _ { j { \\bar { l } } } ) G ^ { { \\bar { l } } k } \\,\n{ \\cal { F } } : \\ < g > = \\int d ^ { 3 } \\theta \\ g ( { \\vec { \\theta } } ) f ( { \\vec { \\theta } } , t ) \\,\n\\tilde { S } _ { r , \\Lambda } ( t _ { 2 } , t _ { 1 } ; g ) = - 2 i \\delta ( t _ { 2 } - t _ { 1 } ) H _ { I } ^ { ( r ) } ( t _ { 1 } ) + \\tilde { S } _ { r , \\Lambda } ^ { \\prime } ( t _ { 2 } , t _ { 1 } ; g ) ,\n\\nu _ { R } ( E ) = \\int _ { \\mu } ^ { E - \\mu _ { Q } } \\nu ( E ^ { \\prime } ) \\nu _ { Q } ( E - E ^ { \\prime } ) d E ^ { \\prime } ~ ~ ~ .\nZ _ { \\mathrm { F } } [ A ] = \\int \\! D { \\bar { \\psi } } D \\psi e ^ { - \\int \\! d ^ { 2 } x \\, [ \\psi _ { 1 } ^ { \\dagger } i \\partial \\psi _ { 1 } + \\psi _ { 2 } ^ { \\dagger } i { \\bar { \\partial } } \\psi _ { 2 } - \\psi _ { 1 } ^ { \\dagger } A \\psi _ { 1 } - \\psi _ { 2 } ^ { \\dagger } { \\bar { A } } \\psi _ { 2 } ] } \\; ,\nE q ( 5 . 2 ) T _ { R } \\; \\sim \\; 5 \\left( \\frac { m } { \\mathrm { T e V } } \\right) ^ { 3 / 2 } \\; \\; \\; \\mathrm { k e V } .\nP _ { n } ^ { p } = \\frac { ( - 1 ) ^ { p + n } } { n ! ( p - n ) ! } \\mathop { { \\prod } ^ { \\prime } } _ { k = 0 } ^ { p } ( N - k ) , \\quad n = 0 , . . . , p ,\nT _ { { \\cal G } } ( - t , - t ^ { - 1 } ) = \\sum x ^ { i ( B ) } x ^ { - e ( B ) }\n\\left. \\begin{array} { c c c } { S _ { \\sigma \\sigma ^ { \\prime } } \\nonumber } \\\\ { S _ { \\Delta } \\nonumber } \\\\ { \\tilde { S } _ { \\Delta } } \\\\ \\end{array} \\right\\} \\varpi = ( \\varpi + 2 ) \\left\\{ \\begin{array} { c c c } { S _ { \\sigma \\sigma ^ { \\prime } } \\nonumber } \\\\ { S _ { \\Delta } \\nonumber } \\\\ { \\tilde { S } _ { \\Delta } } \\\\ \\end{array} \\right. .\n\\Phi _ { 0 } ( Z ) = \\delta ^ { - 1 } \\alpha , \\qquad \\Phi _ { 0 } ( Z ) = \\alpha ^ { - 1 } \\delta\nb ( k ) b ^ { \\dagger } ( l ) - q _ { e } b ^ { \\dagger } ( l ) b ( k ) = \\delta ( k - l ) ,\n{ \\cal L } = - \\mathrm { \\small ~ \\frac { 1 } { 2 } ~ } f ^ { 2 } \\, \\partial _ { \\mu } \\pi _ { r } \\, \\partial ^ { \\mu } \\pi _ { r } - \\mathrm { \\small ~ \\frac { 1 } { 2 } ~ } f ^ { 2 } \\lambda \\left( \\pi _ { r } \\pi _ { r } - N \\right) \\; ,\nE _ { \\mathrm { q u a s i l o c a l } } = E _ { + } - E _ { - } = \\left( r \\left[ 1 - \\left| r , _ { y } \\right| \\right] \\right) _ { y = y _ { + } } - \\left( r \\left[ 1 - \\left| r , _ { y } \\right| \\right] \\right) _ { y = y _ { - } } .\nI = 2 \\left( { \\frac { \\alpha } { \\operatorname { s i n h } ^ { 2 } A + \\operatorname { s i n } ^ { 2 } { \\frac { \\gamma } { 2 } } } } - { \\frac { \\operatorname { s i n h } { \\frac { 2 A } { \\alpha } } } { \\operatorname { s i n h } { 2 A } } } ~ { \\frac { 1 } { ( \\operatorname { s i n h } ^ { 2 } { \\frac { A } { \\alpha } } + \\operatorname { s i n } ^ { 2 } { \\frac { [ \\gamma ] } { 2 \\alpha } } ) } } \\right) ~ ~ .\nW ( x ) = \\frac { x ^ { 3 } } { 3 } - a ^ { 2 } x ,\nZ _ { M } = \\sum _ { j _ { s } } \\int d U _ { f } \\Pi _ { s } \\left( 2 j _ { s } + 1 \\right) T r _ { j _ { s } } U _ { s }\nf ( x , p ; t ) = \\int \\! d a d b ~ \\tilde { f } ( a , b ) ~ e ^ { i a x ( - t ) } e ^ { i b p ( - t ) } ,\n\\{ \\langle n | O | p \\rangle \\: | n \\in \\Sigma _ { s } ^ { \\prime } \\}\n{ \\cal L } ^ { ( 0 ) } ( B ) = - { \\frac { 1 } { 2 } } B ^ { 2 } \\; ,\n\\ddot { h } = - \\nabla _ { h } \\Phi\n\\langle 0 | T _ { \\mu \\nu } | 0 \\rangle _ { \\mathrm { \\tiny ~ R e n . } } = \\mathrm { d i a g } \\left[ \\langle 0 | T _ { t t } | 0 \\rangle , 0 , \\langle 0 | T _ { l l } | 0 \\rangle , \\langle 0 | T _ { l l } | 0 \\rangle \\right] .\n\\frac { 1 } { 2 4 . 8 \\pi ^ { 2 } } \\, \\int _ { \\cal M } R \\wedge R = \\frac { 1 } { 2 4 . 8 \\pi ^ { 2 } } \\, \\int _ { { \\cal M } } \\mathrm { d } C = \\frac { 1 } { 2 4 . 8 \\pi ^ { 2 } } \\, \\int _ { \\partial { \\cal M } } C \\, ,\n\\quad | A _ { 1 } | ^ { 2 } + | A _ { 2 } | ^ { 2 } - | B _ { 1 } | ^ { 2 } - | B _ { 2 } | ^ { 2 } = 0 .\ne ^ { \\phi _ { c } ^ { 6 } } = - { \\frac { v _ { a } } { \\tilde { v } _ { a } } }\n\\mathrm { \\boldmath ~ \\pi ~ } = \\mu ^ { - 1 } \\, { \\bf p } \\; .\nS = \\int _ { \\mathcal M } \\left( { \\frac { i } { 2 } } \\left[ \\overline { \\psi } \\gamma ^ { a } \\nabla _ { a } \\psi - \\overline { { ( \\nabla _ { a } \\psi ) } } \\gamma ^ { a } \\psi \\right] - m \\overline { \\psi } \\psi \\right) \\, ,\n\\Psi \\sim \\operatorname { e x p } \\left\\{ - \\frac { 1 } { \\hbar } \\int _ { a } ^ { x } \\sqrt { 2 ( v - e ) } \\right\\} \\sim \\operatorname { e x p } \\left\\{ - \\frac { 1 } { \\hbar } x ^ { \\frac { 5 } { 2 } } \\right\\}\n\\ddot { R } ^ { k } ( t ) = \\omega ^ { k l } \\dot { R } ^ { l } ( t ) + O ( \\dot { R } ^ { k } \\dot { R } ^ { k } ) \\; \\; .\n{ \\cal L } \\rightarrow { \\cal L } + \\frac { \\alpha N } { 1 6 \\pi ^ { 2 } } F \\tilde { F } .\n\\partial _ { 0 } { \\cal E } + { \\bf d i v } { \\bf S } = 0 \\, .\n\\phi ( x , y ) = \\lambda ^ { 2 s } \\, \\phi ( \\lambda x , \\lambda y )\nS _ { B } [ A ] \\; = \\; i \\, \\frac { 1 } { \\eta } \\, S _ { C S } [ A ] \\; + \\; R [ \\widetilde { F } ] + \\frac { i } { \\theta } \\int d ^ { 3 } x d ^ { 3 } y j _ { \\mu } ^ { T } ( x ) { \\mathcal K } _ { \\mu \\nu } ( x - y ) j _ { \\nu } ^ { T } ( y ) \\; ,\n{ \\cal L } = \\frac { 1 } { 2 } { \\it i } \\overline { { \\psi } } { \\Gamma } ^ { \\nu } \\hspace { - . 1 5 c m } \\stackrel { \\; \\leftrightarrow } { \\partial } _ { \\! \\nu } \\hspace { - . 1 c m } { \\psi } - \\overline { { \\psi } } M { \\psi } \\quad ,\nm _ { H } \\approx 2 7 2 \\, \\lambda ^ { 1 / 4 } \\ \\mathrm { G e V } .\nM _ { p _ { e } , q _ { m } } = { \\frac { 2 \\pi } { g _ { \\mathrm { Y M } } ^ { 2 } } } { \\frac { q _ { m } u } { \\operatorname { c o s } \\xi } } = u \\sqrt { \\left( \\frac { 2 \\pi q _ { m } } { g _ { Y M } ^ { 2 } } \\right) ^ { 2 } + p _ { e } ^ { 2 } } \\, .\n{ \\frac { d ^ { 2 } \\varphi } { d \\tau ^ { 2 } } } = \\operatorname { s i n } \\varphi \\ ,\n\\hat { I } _ { 1 2 } = - \\frac { 1 } { 9 6 ( 2 \\pi ) ^ { 5 } } \\hat { I } _ { 4 } \\wedge ( \\frac { 1 } { 4 } ( \\hat { I } _ { 4 } ) ^ { 2 } - X _ { 8 } )\n\\frac { 1 } { k ^ { 2 } } ( g ^ { \\mu \\nu } - \\frac { \\tilde { k } _ { \\mu } \\tilde { k } _ { \\nu } } { \\tilde { k } ^ { 2 } } )\n\\begin{array} { l c r } { q J _ { 1 } J _ { 2 } - q ^ { - 1 } J _ { 2 } J _ { 1 } } & { = } & { ( q ^ { 2 } - q ^ { - 2 } ) J _ { 3 } } \\\\ { q J _ { 2 } J _ { 3 } - q ^ { - 1 } J _ { 3 } J _ { 2 } } & { = } & { ( q ^ { 2 } - q ^ { - 2 } ) J _ { 1 } } \\\\ { q J _ { 3 } J _ { 1 } - q ^ { - 1 } J _ { 1 } J _ { 3 } } & { = } & { ( q ^ { 2 } - q ^ { - 2 } ) J _ { 2 } } \\\\ \\end{array}\nS _ { \\Omega ^ { \\prime } , \\Omega } x \\Omega = x ^ { * } \\Omega , \\, \\, x \\in M\na _ { 1 } = - 2 \\pi I _ { 2 \\alpha } ( 0 ) = - \\frac { \\pi } { 3 } \\left( \\frac { \\pi } { \\alpha } - \\frac { \\alpha } { \\pi } \\right) { . }\n{ \\cal L } = { \\frac { 1 } { 2 } } \\partial _ { \\mu } \\phi _ { x } \\partial ^ { \\mu } \\phi _ { x } - { \\frac { 1 } { 2 } } m _ { 0 } ^ { 2 } \\phi _ { x } ^ { 2 } - { \\frac { m ^ { 2 } } { \\beta ^ { 2 } } } ( 1 - c o s ( \\beta \\phi _ { x } ) ) \\equiv { \\frac { 1 } { 2 } } \\partial _ { \\mu } \\phi _ { x } \\partial ^ { \\mu } \\phi _ { x } - U ( \\phi _ { x } ) ,\n\\langle P ^ { \\prime } | J ^ { \\mu } ( 0 ) | P \\rangle = \\bar { u } ( P ^ { \\prime } ) \\, \\Big [ \\, F _ { 1 } ( q ^ { 2 } ) \\gamma ^ { \\mu } + F _ { 2 } ( q ^ { 2 } ) { \\frac { i } { 2 M } } \\sigma ^ { \\mu \\alpha } q _ { \\alpha } \\, \\Big ] \\, u ( P ) \\ ,\nf \\star g = \\operatorname { e x p } \\Bigg [ \\hbar \\Bigg ( \\frac { \\partial ~ } { \\partial q } \\frac { \\partial ~ } { \\partial \\tilde { p } } - \\frac { \\partial ~ } { \\partial p } \\frac { \\partial ~ } { \\partial \\tilde { q } } \\Bigg ) \\Bigg ] f ( { \\bf x } ) g ( { \\bf \\tilde { x } } ) \\vert _ { { \\bf x } = { \\bf \\tilde { x } } } ,\n\\Delta { \\cal A } = \\frac { 1 } { 2 } ( 1 + g ) .\n\\operatorname { e x p } \\left( i p _ { \\mu } X ^ { \\mu } \\right) \\rightarrow \\hat { v } _ { p } = \\hat { h } ^ { k _ { 2 } }\nT _ { [ \\mu \\nu ] } ^ { \\ a } = \\partial _ { \\mu } E _ { \\nu } ^ { \\underline { { a } } } - \\partial _ { \\nu } E _ { \\mu } ^ { \\underline { { a } } }\n\\eta _ { \\mu \\nu } = \\mathrm { d i a g ( - , + ) }\n\\langle f , f \\rangle = \\langle g , g \\rangle = 0 .\n\\Big ( L ^ { \\frac { 3 } { 2 } } \\Big ) _ { + } = p ^ { 3 } + \\frac { 3 } { 2 } u \\star p - \\frac { 2 \\kappa } { 2 } u ^ { ( 1 ) } ,\nQ _ { a , b } ( \\mu , \\nu ; N ) = p _ { a , b } ( \\mu , \\nu ; 0 ; N ) + \\sum _ { \\lambda = 1 } ^ { \\infty } ( - 1 ) ^ { \\lambda } p _ { a , b } ( \\mu , \\nu ; \\lambda ; N ) + \\sum _ { \\lambda = 1 } ^ { \\infty } ( - 1 ) ^ { \\lambda } m _ { a , b } ( \\mu , \\nu ; \\lambda ; N ) .\n\\vec { A } _ { \\mu } ( x ) \\! = \\! \\partial _ { \\mu } \\vec { n } ( x ) \\wedge \\vec { n } ( x ) + C _ { \\mu } ( x ) \\vec { n } ( x ) + \\vec { W } _ { \\mu } ( x ) ,\n{ \\cal E } = \\frac { { \\bf D } ^ { 2 } + { \\bf H } ^ { 2 } } { 2 } - \\left( { \\bf \\theta } \\cdot { \\bf B } \\right) { \\bf B } ^ { 2 } ,\n\\frac 1 { \\nabla ^ { 2 } } \\, \\delta _ { \\Sigma } ^ { ( 2 ) } ( z - z _ { 0 } ) = - \\frac 1 \\pi \\operatorname { l o g } { \\cal E } ( z , z _ { 0 } )\n\\delta _ { A } \\widehat { \\phi } \\smallskip ( \\widehat { x } ) = i [ \\widehat { A } \\smallskip ,\n{ \\cal M } = M ( \\infty ) = 4 \\pi \\int _ { 0 } ^ { \\infty } \\left[ r ^ { 2 } \\left( { \\cal U } ( f , h , u ) + \\alpha ^ { 2 } { \\cal K } ( f , h , u ) \\right) - \\eta ^ { 2 } \\right] e ^ { - P ( r ) } \\ ,\ny \\sqrt { c } \\operatorname { s i n h } \\alpha = \\operatorname { c o s h } \\alpha - \\operatorname { c o s h } ( x \\sqrt { c } - \\alpha ) \\; \\; , \\; \\; \\; \\; \\operatorname { t a n h } \\alpha \\equiv 2 \\sqrt { c } \\; .\n\\lambda t K _ { \\left| n \\right| } ( \\mu t ) I _ { \\left| n \\right| } ( \\mu t ) .\ng x ( \\alpha \\cdot q , \\xi ) \\to \\left\\{ \\begin{array} { c l } { \\mathrm { f i n i t e } , } & { \\mathrm { f o r } \\quad \\pm \\alpha _ { i } \\in \\Pi \\quad ( \\delta \\leq 1 / h ) \\quad \\mathrm { a n d } \\ \\pm \\alpha _ { h } \\quad ( \\delta = 1 / h ) , } \\\\ { 0 , } & { \\mathrm { o t h e r w i s e , } } \\\\ \\end{array} \\right.\n\\dot { G } _ { \\pm } + \\frac { i } { 2 } \\phi ^ { \\pm } D _ { \\pm } ^ { 2 } G _ { \\pm } + \\frac { i } { 4 } D _ { \\pm } \\phi ^ { \\pm } D _ { \\pm } G _ { \\pm } + \\frac { 3 } { 4 } i D _ { \\pm } ^ { 2 } \\phi ^ { \\pm } G _ { \\pm } = \\frac { \\kappa _ { 0 } } { 2 } D _ { \\pm } ^ { 5 } \\phi ^ { \\pm } ~ .\n\\mathcal { L } = \\sum _ { i = 1 } ^ { N } \\bar { \\psi } ^ { i } \\left( \\partial \\! \\! \\! \\slash + \\lambda \\sigma \\right) \\psi ^ { i }\n\\left\\langle \\operatorname { e x p } \\left( \\; 2 \\sum _ { b = 1 } ^ { N } \\beta ^ { ( b ) } \\int _ { \\Lambda } d ^ { 2 } x \\; t ( x ) \\; : \\operatorname { c o s } \\left( 2 \\sqrt { \\pi } \\varphi ^ { ( b ) } ( x ) \\; \\right) : _ { Q ^ { W } } \\; \\right) \\right\\rangle \\; .\n\\chi _ { a } ( A ) = \\varepsilon _ { a b i } \\, A _ { b i } ( x ) = 0 \\, ,\n\\omega ( \\varepsilon ) = { \\frac { e ^ { \\beta _ { H } \\varepsilon } } { \\varepsilon } } ,\nG ^ { ( N , M ) } ( z _ { 1 } , S _ { M + 1 } , \\cdots , S _ { N } ; z _ { 1 } , z _ { 2 } ) = 0 .\n\\varphi \\sim A + B \\mathrm { s i g n } ( t ) | \\vec { k } t | ^ { \\epsilon } .\n\\stackrel { \\rightarrow } { x } ( \\tau ) = m ^ { - 2 } \\stackrel { \\rightarrow } { V }\nR _ { i j , k l } = \\delta _ { i k } \\delta _ { j l } ( 1 + \\delta _ { i j } ( q - 1 ) ) + \\lambda \\delta _ { i l } \\delta _ { j k } \\theta ( i - j ) \\quad i , j . . . = 1 . . . n\nS = - T _ { ( p - 1 ) } \\int d ^ { p } \\sigma \\sqrt { - \\operatorname* { d e t } ( \\mathcal { G } _ { \\alpha \\beta } + 2 \\pi \\alpha ^ { \\prime } \\mathcal { F } _ { \\alpha \\beta } ) } ,\nS _ { p , m } = \\prod _ { A < B } \\prod _ { I < J } ( z _ { I } ^ { A } - z _ { J } ^ { B } ) ^ { K _ { A , B } } \\prod _ { A } \\prod _ { I < J } ( z _ { I } ^ { A } - z _ { J } ^ { A } ) ^ { K _ { A , A } - 1 }\nH _ { L C } = \\int _ { 0 } ^ { l } \\mathcal { H } _ { L C } = \\int _ { 0 } ^ { l } d \\sigma \\, \\frac { G _ { + - } } { 2 \\, \\pi _ { - } } \\left( G ^ { i j } \\pi _ { i } \\pi _ { j } + \\frac { 1 } { ( 2 \\pi \\alpha ^ { \\prime } ) ^ { 2 } } G _ { i j } Z ^ { i \\, \\prime } Z ^ { j \\, \\prime } \\right)\n\\frac { d \\, \\operatorname { l o g } \\widetilde \\mathcal { P } _ { \\{ N _ { \\nu _ { \\alpha _ { 1 } } } ^ { ( \\alpha _ { 1 } ) } , \\dots , \\, N _ { \\nu _ { \\alpha _ { n } } } ^ { ( \\alpha _ { n } ) } \\} } ^ { \\Lambda , \\lambda _ { \\Lambda } } } { d \\, N _ { \\nu _ { \\alpha _ { j } } } ^ { ( \\alpha _ { j } ) } } \\quad = \\quad \\frac { \\int _ { G } d \\mu ( g ) \\bar { \\chi } ^ { ( \\Lambda ) } ( g ) [ D _ { \\nu _ { 1 } \\nu _ { 1 } } ^ { ( \\alpha _ { 1 } ) } ] ^ { N _ { \\nu _ { \\alpha _ { 1 } } } ^ { ( \\alpha _ { 1 } ) } } \\cdots [ D _ { \\nu _ { n } \\nu _ { n } } ^ { ( \\alpha _ { n } ) } ] ^ { N _ { \\nu _ { \\alpha _ { n } } } ^ { ( \\alpha _ { n } ) } } \\operatorname { l o g } [ D _ { \\nu _ { j } \\nu _ { j } } ^ { ( \\alpha _ { j } ) } ] } { \\int _ { G } d \\mu ( g ) \\bar { \\chi } ^ { ( \\Lambda ) } ( g ) [ D _ { \\nu _ { 1 } \\nu _ { 1 } } ^ { ( \\alpha _ { 1 } ) } ] ^ { N _ { \\nu _ { \\alpha _ { 1 } } } ^ { ( \\alpha _ { 1 } ) } } \\cdots [ D _ { \\nu _ { n } \\nu _ { n } } ^ { ( \\alpha _ { n } ) } ] ^ { N _ { \\nu _ { \\alpha _ { n } } } ^ { ( \\alpha _ { n } ) } } } \\, .\na _ { M } = { \\frac { ( - ) ^ { { \\frac { 1 } { 2 } } M ( M + 1 ) } } { M ! [ ( M - 1 ) ! \\ldots 2 \\cdot 1 ] ^ { 2 } } } ~ ~ ,\n{ \\cal H } ( p ) = - 2 i \\varepsilon ^ { \\mu \\nu \\rho } \\varepsilon ^ { \\alpha \\beta \\lambda } g _ { \\nu \\alpha } \\frac { \\mu ^ { \\epsilon } } { ( 2 \\pi ) ^ { d } } \\int { \\cal D } q \\frac { 1 } { ( p + q ) ^ { 2 } q ^ { 2 } } I _ { \\beta \\mu \\lambda \\rho } ( q ) \\; ,\n\\mathrm { T r } _ { N S - R } = \\frac { V _ { 0 } } { 2 \\pi } \\int \\d E \\, \\sum _ { m = 0 } ^ { N - 1 } \\sum _ { w = - \\infty } ^ { \\infty } \\, \\cdots .\nR _ { \\mu \\nu } - \\frac { 1 } { 2 } R g _ { \\mu \\nu } = 0\n{ \\frac { \\alpha } { p ( N ) } } \\ \\left( { \\frac { 1 } { c ( N ) } } - 1 \\right) \\approx { \\frac { 2 A _ { 2 } } { p ( N ) ^ { 2 } } } \\, .\n{ \\cal W } ^ { ( 1 ) } = { \\frac { 1 } { { \\cal Z } ^ { ( 1 ) } N } } \\sum _ { R , S } \\frac { d _ { R } } { d _ { S } } \\operatorname { e x p } \\left[ - { \\frac { g ^ { 2 } A _ { 1 } } { 2 } } C _ { 2 } ( R ) - { \\frac { g ^ { 2 } A _ { 2 } } { 2 } } C _ { 2 } ( S ) \\right] \\int d U \\mathrm { T r } [ U ] \\chi _ { R } ( U ) \\chi _ { S } ^ { \\dagger } ( U ) ,\nA _ { p a r e n t } ^ { \\mathrm { I V } } = - \\frac { T _ { p } } { 2 } \\int d ^ { p + 1 } { \\xi } \\left[ \\sqrt { - g } \\left( - { \\Psi } ^ { ( p + 3 ) / 2 } g _ { i j } h ^ { i j } + ( p + 3 ) { \\Psi } ^ { ( p + 1 ) / 2 } \\right) + { \\Lambda } \\left( { \\Psi } - { \\Phi } \\right) \\right] .\nF ( z ) \\; = \\; F ( 0 ) \\, + \\, \\sum _ { m = 1 } ^ { \\infty } \\, \\left\\{ \\frac { b _ { m } } { z - a _ { m } } + \\frac { b _ { m } } { a _ { m } } \\right\\} \\, + \\, \\sum _ { m = 1 } ^ { \\infty } \\, \\left\\{ \\frac { b _ { - m } } { z - a _ { - m } } + \\frac { b _ { - m } } { a _ { - m } } \\right\\} \\, .\n\\{ Q ^ { \\alpha } , \\bar { Q } _ { \\beta } \\} = - i ( \\Gamma ^ { a } ) _ { \\beta } ^ { \\alpha } P _ { a } - i ( \\Gamma ^ { a b c d e } ) _ { \\beta } ^ { \\alpha } Z _ { a b c d e } ,\n\\frac { \\xi } { \\sqrt { 1 - \\dot { r } ^ { 2 } } } = \\sqrt { x _ { 1 } ^ { 2 } + x _ { 2 } ^ { 2 } } - r ( \\tau ) ,\n\\left( A _ { 0 } , A _ { 1 } , \\phi , \\pi _ { 0 } , \\pi _ { 1 } , \\pi _ { \\phi } \\right) \\longleftrightarrow \\left( \\xi _ { 1 } , . . . , \\xi _ { 6 } \\right) ,\nE ( L ) - E _ { 0 } ( L ) = E _ { R } + E _ { L } + M \\operatorname { c o s h } \\theta \\ ,\nv _ { e f f } = \\left( { \\frac { y _ { i } } { y _ { 0 } } } + 1 \\right) ^ { 1 / 3 } e ^ { - n } v _ { i } \\, .\n\\Pi _ { \\mu \\nu \\rho \\sigma } ^ { r } ( p , q , k ) = \\Pi _ { \\mu \\nu \\rho \\sigma } ( p , q , k ) - \\Pi _ { \\mu \\nu \\rho \\sigma } ( p , 0 , 0 )\n\\frac { t } { d } = \\oint \\frac { d z } { 2 \\pi \\imath } U _ { e f f } ^ { \\prime } \\left( z + \\frac { R ( t ) } { z } \\right) = \\left( \\frac { m _ { 0 } ^ { 2 } } { d } - \\Gamma _ { 2 } \\right) R ( t ) - \\sum _ { n > 2 } \\frac { ( 2 n ) ! } { ( n ! ) ^ { 2 } } \\Gamma _ { 2 n } R ( t ) ^ { n }\nw : = \\left( \\begin{array} { c } { \\tilde { x } _ { 1 } } \\\\ { . } \\\\ { . } \\\\ { . } \\\\ { \\tilde { x } _ { n } } \\\\ { \\hat { y } _ { 1 } } \\\\ { . } \\\\ { . } \\\\ { . } \\\\ { \\hat { y } _ { n } } \\\\ \\end{array} \\right) \\; , \\; z : = \\left( \\begin{array} { c } { \\tilde { y } _ { 1 } } \\\\ { . } \\\\ { . } \\\\ { . } \\\\ { \\tilde { y } _ { n } } \\\\ { \\hat { x } _ { 1 } } \\\\ { . } \\\\ { . } \\\\ { . } \\\\ { \\hat { x } _ { n } } \\\\ \\end{array} \\right) \\; \\; .\np _ { \\nu } ^ { \\prime } T _ { \\mu \\nu } ^ { S \\rightarrow A A } = 2 m i [ T _ { \\mu } ^ { S \\rightarrow A P } ] - 4 m ( k _ { 3 } + k _ { 2 } ) _ { \\alpha } \\triangle _ { \\alpha \\mu } - 4 m ( l _ { 3 } + l _ { 2 } ) _ { \\alpha } \\triangle _ { \\alpha \\mu } ,\n\\sum _ { j = 1 } ^ { n } [ I _ { i j } ] _ { \\hat { q } } \\, m _ { j } = 2 \\operatorname { c o s } \\pi \\left( \\vartheta _ { h } + t _ { i } \\vartheta _ { H } \\right) \\, m _ { i } \\; , \\quad \\hat { q } = e ^ { i \\pi s \\vartheta _ { H } } \\; .\n{ \\tilde { f } } ^ { ( 0 ) } = \\left( \\begin{array} { c c c c c c } { 0 } & { - 1 } & { 0 } & { 0 } & { 0 } & { q \\partial _ { y } } \\\\ { 1 } & { 0 } & { 0 } & { 0 } & { 0 } & { q } \\\\ { 0 } & { 0 } & { 0 } & { 0 } & { 0 } & { 0 } \\\\ { 0 } & { 0 } & { 0 } & { 0 } & { - 1 } & { q ^ { 2 } } \\\\ { 0 } & { 0 } & { 0 } & { 1 } & { 0 } & { \\partial _ { y } } \\\\ { - q \\partial _ { x } } & { - q } & { 0 } & { - q ^ { 2 } } & { - \\partial _ { x } } & { 0 } \\\\ \\end{array} \\right) \\delta ( x - y )\nW ^ { a } = \\frac { 1 } { I E } \\big ( | Z | ^ { 2 } H ^ { a } - A ^ { a } A _ { b } H ^ { b } + | Z | ^ { 2 } T ^ { a b } \\! A _ { b } \\big ) \\, ,\n\\Delta + \\bar { \\Delta } = \\frac { ( m + n k / 4 ) ^ { 2 } } { k } + \\frac { ( m - n k / 4 ) ^ { 2 } } { k } \\, .\n\\Delta _ { a } ^ { \\! l i n } = - \\int \\frac { k ^ { \\prime } } { 2 l } \\, d l - \\frac { 1 } { 2 } V \\, .\n\\Sigma _ { - } = a _ { 1 } ( x ) \\operatorname { e x p } { ( 3 ( \\gamma - 1 ) \\tau ) } , ~ ~ \\Sigma _ { \\times } = a _ { 2 } ( x ) \\operatorname { e x p } { ( 3 ( \\gamma - 1 ) \\tau ) } ,\n\\left[ \\widehat { x } _ { 0 } , \\widehat { x } _ { i } \\right] = { \\frac { i } { \\kappa } } \\widehat { x } _ { i } \\qquad \\left[ \\widehat { x } _ { i } , \\widehat { x } _ { j } \\right] = 0\n\\begin{array} { l l l l l l } { \\displaystyle D } & { = 3 } & { : \\quad } & { S p ( 4 ; R ) } & { \\to } & { O S p ( N ; 1 | R ) } \\\\ { \\displaystyle D } & { = 4 } & { : \\quad } & { S U ( 2 , 2 ) } & { \\to } & { S U ( 2 , 2 | N ) } \\\\ { \\displaystyle D } & { = 6 } & { : \\quad } & { U _ { \\alpha } ( 4 ; H ) } & { \\to } & { U U _ { \\alpha } ( N ; 4 | R ) } \\\\ \\end{array}\n\\int _ { 0 } ^ { \\infty } d k \\sqrt { k ^ { 2 } + 4 } \\left( \\frac { 4 \\operatorname { c o s } \\rho } { k ^ { 2 } + 4 \\operatorname { c o s } ^ { 2 } \\rho } - \\frac { 4 \\operatorname { c o s } \\rho } { k ^ { 2 } + 4 } + \\frac { 4 c _ { + } } { k ^ { 2 } + 4 c _ { + } ^ { 2 } } - \\frac { 4 c _ { + } } { k ^ { 2 } + 4 } \\right) ,\n\\displaystyle { \\frac { 8 } { \\kappa } } V _ { \\mu \\nu , \\, \\alpha \\beta } ^ { ^ 2 g h o s t } ( k _ { 1 } , k _ { 2 } ; \\, p _ { 1 } , p _ { 2 } ) = - 4 \\, p _ { _ 1 \\mu } \\, p _ { _ 2 \\nu } \\, \\eta _ { \\alpha \\beta } + 8 \\, p _ { _ 1 \\beta } \\, p _ { _ 2 \\nu } \\, \\eta _ { \\alpha \\mu } - 2 \\, p _ { 1 } \\cdot p _ { 2 } \\, \\eta _ { \\alpha \\mu } \\, \\eta _ { \\beta \\nu } + p _ { 1 } \\cdot p _ { 2 } \\, \\eta _ { \\alpha \\beta } \\, \\eta _ { \\mu \\nu } .\n\\left( 1 + i \\gamma \\right) ^ { - 1 } \\approx \\left( 1 - i \\gamma \\right) .\n{ \\cal P } : \\quad\n( \\Delta S ) ^ { 0 } = T r \\left[ \\left( \\begin{array} { l l } { K ^ { \\mu } { } _ { \\nu } } & { K ^ { \\mu } { } _ { F } } \\\\ { K ^ { F } { } _ { \\nu } } & { K ^ { F } { } _ { F } } \\\\ \\end{array} \\right) \\operatorname { e x p } \\frac { 1 } { M ^ { 2 } } \\tilde { \\cal R } \\right] \\ ; \\qquad \\tilde { \\cal R } = \\left( \\begin{array} { l l } { S _ { \\mu \\nu } } & { q _ { \\mu } } \\\\ { - \\tilde { \\partial } q _ { \\nu } ^ { T } } & { \\tilde { \\partial } \\tilde { \\nabla } } \\\\ \\end{array} \\right) \\ .\n[ \\Pi ( { \\bf { x } } \\sigma ) , \\rho ( { \\bf { y } } \\sigma ^ { ' } ) ] = i \\mathrm { ~ } \\delta ( { \\bf { x } } - { \\bf { y } } ) \\delta _ { \\sigma , \\sigma ^ { ' } }\nE ^ { ( R ) } = { \\frac { n _ { R } } { | G | } } \\sum _ { g \\in G } \\chi ^ { ( R ) } ( g ) U ( g ) ~ ,\nE _ { \\small C a s i m i r } ^ { \\small b u l k } = { \\frac { 1 } { 8 \\pi ^ { 2 } } } \\; V \\; \\hbar c \\; K ^ { 4 } \\; \\left[ { \\frac { 1 } { n } } - 1 \\right] .\nH _ { 0 } = \\int _ { 0 } ^ { \\infty } d p \\, p [ A ^ { \\dag } ( p ) A ( p ) - B ^ { \\dag } ( p ) B ( p ) ] .\n\\Omega ^ { J } \\equiv ( v _ { i } ^ { J } ) ^ { T } \\frac { \\partial V ( q ) } { \\partial q _ { i } } = 0 ,\n\\{ Q ^ { + } , Q ^ { - } \\} = H _ { n } + k ( R - 1 ) , \\quad [ H _ { n } , Q ^ { \\pm } ] = \\mp 2 k Q ^ { \\pm } , \\quad [ R , Q ^ { \\pm } ] = \\pm 2 Q ^ { \\pm } , \\quad R ^ { 2 } = 1 .\n\\partial _ { \\mu } < T T R > = ( N _ { c } ^ { 2 } - 1 ) \\cdot \\left[ \\mathrm { \\vspace { 2 e x } \\mathrm { 1 } \\vspace { 0 e x } } \\right] + 3 \\cdot ( N _ { c } ^ { 2 } - 1 ) \\cdot \\left[ - \\frac { 1 } { 3 } \\right] = 0\n\\frac { \\partial g _ { k } } { \\partial x _ { j } } = \\frac { \\partial h _ { k } } { \\partial y _ { j } } \\, \\, \\, ; \\, \\, \\, \\frac { \\partial h _ { k } } { \\partial x _ { j } } = - \\frac { \\partial g _ { k } } { \\partial y _ { j } } ,\n{ \\cal D } [ \\partial ] : = 1 + { \\frac { \\partial ^ { 2 } } { \\eta ^ { 2 } } } - { \\frac { \\partial ^ { 4 } } { \\gamma ^ { 4 } } } .\nZ _ { 0 } ^ { F } ( T , L ) \\equiv \\operatorname { e x p } \\left( \\frac { \\pi L T } { 6 } \\right) .\nx \\mapsto \\omega ( \\Xi ) ( x ) , \\quad \\ x \\in { \\cal O } ,\n\\left. \\begin{array} { l } { \\rho ( 1 ) = 1 } \\\\ { \\rho ( 2 ) = 2 } \\\\ { \\ldots } \\\\ { \\rho ( p ) = p } \\\\ \\end{array} \\right\\}\nS ^ { \\mu \\nu } \\equiv { \\frac { \\partial L } { \\partial \\sigma _ { \\mu \\nu } } } = L ^ { - 1 } [ J ^ { 2 } \\sigma ^ { \\mu \\nu } + { \\frac { M J } { { ( { \\frac { a _ { 1 } a _ { 2 } } { 2 } } + a _ { 3 } } ) ^ { \\frac { 1 } { 2 } } } } ( a _ { 1 } \\sigma ^ { \\mu \\nu } + ( u ^ { \\mu } \\sigma ^ { \\nu \\lambda } - u ^ { \\nu } \\sigma ^ { \\mu \\lambda } ) u _ { \\lambda } ) ] .\nH = 1 + Q _ { 1 } \\sum _ { n = - \\infty } ^ { \\infty } \\frac { 1 } { \\mid \\vec { y } - 2 \\pi n a \\hat { z } \\mid ^ { 6 } } ,\nz ( \\varepsilon , \\tau ) \\; = \\; u \\: + \\: 2 \\tau v \\: + \\: ( 1 - \\varepsilon ^ { 2 } ) \\: \\tau ^ { 2 } \\: q ^ { 2 } \\: - \\: \\frac { q ^ { 2 } } { 4 } \\; .\nV ( \\Phi ) = - m ^ { 2 } \\mathrm { T r } [ \\Phi ^ { 2 } ] + h ( \\mathrm { T r } [ \\Phi ^ { 2 } ] ) ^ { 2 } + \\lambda \\mathrm { T r } [ \\Phi ^ { 4 } ] + V _ { 0 }\ng \\approx 3 - \\sqrt 3 - 0 . 9 1 7 7 f _ { 0 } ^ { 2 } \\; .\n\\varepsilon ( p ^ { 0 } ) = \\theta ( p ^ { 0 } ) - \\theta ( - p ^ { 0 } ) = \\frac { p ^ { 0 } } { | p ^ { 0 } | } ,\n\\tilde { \\omega } _ { m a b } = \\hat { \\omega } _ { m a b } - \\frac { i \\kappa ^ { 2 } } { 4 } \\bar { \\psi } _ { c } \\gamma ^ { c d } { } _ { m a b } \\psi _ { d } .\ns p ^ { \\mu } = \\eta ^ { \\alpha } e _ { \\alpha } ^ { \\mu } ( P _ { 0 } )\nq ^ { ( \\frac { L } { 2 } - \\frac { r } { N } ) } \\sum _ { k = l + 1 } ^ { N } q ^ { - m _ { 0 } } B ^ { ( N ) } ( L , r , l + 1 | k ) = \\sum _ { k = l + 1 } ^ { N } B ^ { ( N ) } ( L , r , l | k ) .\n[ \\Phi ( x ; a _ { 1 } , . . . , a _ { n } ) ] _ { R } = \\Phi ( x ; a _ { 1 } , . . . , a _ { n } ) +\nZ = { \\operatorname* { d e t } } _ { T } ^ { - \\frac 1 2 } ( - \\Delta ) \\times { \\operatorname* { d e t } } _ { S } ^ { \\frac 1 2 } ( - \\Delta ) ,\n\\Omega ( z ) = \\int _ { \\c C } d w \\rho ( w ) \\operatorname { l n } ( z - w )\n\\mathbf { J } ^ { 2 } = ( \\mathbf { J } ^ { 2 } ) ^ { \\dagger } = G ( \\mathbf { J } ^ { 2 } ) ^ { + } G ,\nm ^ { 1 ; 1 ; 9 } , \\ m ^ { 1 ; 4 ; 9 } , \\ m ^ { 2 ; 6 ; 9 } , \\ m ^ { 4 ; 7 ; 9 } , \\ m ^ { 7 ; 7 ; 9 } , \\ m ^ { 2 ; 3 ; 9 ; 9 } , \\dots\nE _ { \\pm } \\approx \\pm m e ^ { - \\frac { \\mu ^ { 2 } } { m } \\Delta x } .\nM ( S , R ) _ { n } = \\left( \\begin{array} { l l } { 1 } & { 0 } \\\\ { 1 } & { 1 } \\\\ \\end{array} \\right) ^ { \\otimes ( n - 1 ) } \\ ,\n\\left[ - \\left( 1 - \\frac { 2 M } { \\rho } \\right) \\frac { d ^ { 2 } } { d \\rho ^ { 2 } } - \\frac { 2 ( \\rho - 4 M ) ( 2 \\rho - 3 M ) } { \\rho ^ { 2 } ( \\rho - 3 M ) } \\frac { d } { d \\rho } + \\frac { 8 M } { \\rho ^ { 2 } ( \\rho - 3 M ) } \\right] H _ { 1 } ( \\rho ) = \\lambda H _ { 1 } ( \\rho ) ,\nG _ { \\alpha \\beta } ( \\tau , \\tau ^ { \\prime } ) = \\frac { 1 } { 2 } \\int \\Sigma _ { \\alpha \\beta } ( \\tau , \\lambda ^ { \\prime } ) \\epsilon ( \\lambda ^ { \\prime } - \\tau ^ { \\prime } ) \\; d \\lambda ^ { \\prime }\n\n\\gamma _ { 1 } = - \\gamma - 1 - \\gamma ( - 2 - a - b ) - \\delta ( 2 - a / \\delta - b / \\delta - 2 a - 2 b ) - 9 / 8 8 ( 1 + a + b ) ,\nj ^ { m i } \\equiv \\frac { \\delta I _ { S I } } { \\delta v _ { m } ^ { i } } , \\quad K ^ { m i } \\equiv \\frac { \\delta I _ { S I } } { \\delta A _ { m } ^ { i } } .\n[ \\hat { x } _ { \\mu } , \\hat { x } _ { \\nu } ] = \\frac { i } { \\kappa } ( a _ { \\mu } \\hat { x } _ { \\nu } - \\hat { x } _ { \\mu } a _ { \\nu } )\nG _ { \\mu \\nu } ^ { 0 } = \\frac { \\epsilon _ { \\mu \\nu \\lambda } p _ { \\lambda } + \\delta _ { \\mu \\nu } M + p _ { \\mu } p _ { \\nu } / M } { p ^ { 2 } + M ^ { 2 } } \\; , ~ ~ D _ { \\mu \\nu } ^ { 0 } ( p ) = \\frac { \\epsilon _ { \\mu \\nu \\lambda } p _ { \\lambda } } { p ^ { 2 } } \\; , ~ ~ \\mathrm { a n d } ~ ~ \\Gamma _ { \\mu \\nu \\lambda } ^ { 0 } = g \\epsilon _ { \\mu \\nu \\lambda }\n\\left\\{ Q _ { a } , Q _ { b } \\right\\} _ { { \\footnotesize P B } } = f _ { a b c } Q _ { c } \\; ,\nA d j ( S p ( 2 n _ { H } ) ) \\to A d j ( U ( k ) ) + A d j ( U ( n _ { H } - k ) ) + 2 ( k , n _ { H } - k )\nC _ { 0 } ( k , q ) = < k , q | \\sum _ { j = 0 } ^ { m - 1 } R ^ { - j } \\omega ^ { j s } | \\phi > = \\sum _ { j = 0 } ^ { m - 1 } ( R ^ { j } | k , q > ) ^ { + } \\omega ^ { j s } | \\phi > .\nG ^ { \\dagger } = G , \\; \\; \\; \\; \\; \\; \\tilde { G } ^ { \\dagger } = \\tilde { G }\nI _ { n } ( x ) = \\int _ { 0 } ^ { x } d x _ { n } \\int _ { 0 } ^ { x _ { n } } d x _ { n - 1 } \\cdots \\int _ { 0 } ^ { x _ { 2 } } d x _ { 1 } \\cdot 1 = \\frac { x ^ { n } } { n ! } .\nN ^ { \\mu \\nu } = L ^ { i } n ^ { [ \\mu } { } _ { i } X ^ { \\nu ] } { } ^ { \\prime } = L ^ { * } \\eta _ { 1 } ^ { [ \\mu } X ^ { \\nu ] } { } ^ { \\prime } \\, .\nX ^ { + } = { \\frac { * ( 2 { \\cal D } X ^ { i } \\wedge f ^ { + i } ) } { * ( f ^ { + i } \\wedge f ^ { - i } ) } } = { \\frac { 1 } { \\cal R } } ( { \\cal D } _ { - } X ^ { i } f _ { + } ^ { ~ + i } - { \\cal D } _ { + } X ^ { i } f _ { - } ^ { ~ + i } )\nT ( z ) = L ^ { a b } : J _ { a } ( z ) J _ { b } ( z ) :\nR ( b ) = \\frac { 4 } { k - 2 } \\; \\frac { k ( k - 4 ) + k ( k - 2 ) b } { [ k + 2 + ( k - 2 ) b ] ^ { 2 } }\nn ! \\prod _ { i = 1 } ^ { k } d _ { i } ! { \\frac { 1 } { ( n - m ) ! } } .\n[ R , M ] = - \\mathrm { \\frac { i } { 2 } } \\sum _ { \\rho \\in \\Delta _ { + } } g _ { | \\rho | } \\thinspace { \\frac { | \\rho | ^ { 2 } \\operatorname { c o s } ( \\rho \\cdot q ) } { \\operatorname { s i n } ^ { 2 } ( \\rho \\cdot q ) } } \\thinspace [ \\mathrm { e } ^ { 2 \\mathrm { i } q \\cdot \\hat { H } } , \\hat { s } _ { \\rho } ] .\nl n d e t = \\frac { | e \\Phi | } { 2 \\pi } \\operatorname { l n } ( m a ) + R ( m ) ,\nB ( 0 ) = B _ { \\mathrm { f } } ( 0 ) + B _ { \\mathrm { b } } ( 0 ) = 0 ,\nH \\, { \\cal U } = i { \\frac { \\partial } { \\partial t } } \\, { \\cal U } \\, ,\n\\Sigma _ { \\pm } ( q _ { 3 } , \\left| { \\bf q } \\right| ) = - \\frac { 1 } { 4 \\pi } \\operatorname { l n } \\left[ \\frac { 1 } { 2 m } \\left( - \\frac { 1 } { 2 } \\delta \\pm i q _ { 3 } + \\frac { \\left| { \\bf q } \\right| ^ { 2 } } { 2 m } \\right) \\right] \\, + \\, O \\left( \\frac { 1 } { m ^ { 2 } } \\right) .\n\\frac { g _ { Y M } ^ { 2 } N } { J ^ { 3 } } \\frac { J ^ { 4 } } { N ^ { 2 } } \\frac { g ^ { 2 } N } { J ^ { 2 } }\n{ \\cal L } _ { n } ( H _ { n } ) = ( - T _ { H _ { n } } \\bar { R } _ { H _ { n } } ) \\phi ^ { n } .\n\\fbox { \\begin{array} { l l } { ( x y ) ( a x ) = x ( a y ) x ; } & { ( x y ) _ { L } x _ { R } = x _ { L } x _ { R } y _ { L } ; } \\\\ { ( x a ) ( y x ) = x ( y a ) x ; } & { ( y x ) _ { R } x _ { L } = x _ { L } x _ { R } y _ { R } ; } \\\\ { ( x a x ) y = x ( a ( x y ) ) ; } & { y _ { R } x _ { L } x _ { R } = x _ { L } ( x y ) _ { R } ; } \\\\ { y ( x a x ) = ( ( y x ) a ) x ; } & { y _ { L } x _ { L } x _ { R } = x _ { R } ( y x ) _ { L } . } \\\\ \\end{array} }\n{ \\delta _ { \\rho } } { \\mathcal { F } _ { D } ^ { 1 } } = { \\delta _ { \\rho } } { \\mathcal { F } _ { D } ^ { 0 } } - \\alpha { Q } \\oint { d } \\hat { s } ( \\xi ) \\rho ( \\xi ) { \\partial _ { \\hat { n } } } { \\hat { G } _ { D } } ( \\xi , \\xi ^ { \\prime } ) + ( { \\alpha ^ { 2 } } - \\alpha { Q } ) \\rho ( \\xi ^ { \\prime } ) .\nD \\sum _ { I \\neq J } ( - 1 ) ^ { p _ { I } p _ { J } } e _ { J I } \\otimes e _ { I J } D ^ { - 1 } = \\sum _ { I \\neq J } e _ { J I } \\otimes e _ { I J }\nH ( t ) | n ; t > = E _ { n } ( t ) | n ; t > , ~ ~ < m ; t | n ; t > = \\delta _ { m n }\nE _ { c } = \\frac { 4 k l } { R } \\left( k - \\frac { r _ { + } ^ { 2 } } { l ^ { 2 } } \\right) ,\nP \\sim \\operatorname { e x p } \\left( - y _ { 0 } ^ { 3 } / M _ { p } ^ { 2 } \\ell \\right) ,\n\\tilde { W } [ \\eta | t ] = \\int _ { \\eta _ { 0 } } ^ { \\eta ( t ) } \\delta \\eta ^ { ' \\nu } ( t ) \\, \\tilde { E } _ { \\nu } [ \\eta ^ { \\prime } | t ] ,\n\\partial _ { z } \\partial _ { \\bar { z } } \\theta = \\{ n ( n + 1 ) \\eta ^ { 2 } + a \\} \\theta \\, , \\quad a \\in { \\cal R } \\, , \\quad n = 0 , 1 , 2 , \\ldots\nC _ { 0 } ^ { o p } = g _ { o } ^ { - 2 } \\frac { 1 } { \\left( 2 \\alpha ^ { ^ { \\prime } } \\right) ^ { d / 2 } } ,\n\\alpha ^ { 2 } \\partial _ { x ^ { - } } ^ { 3 } g _ { a } ( x ) = 0 ~ ; \\alpha \\neq 0\nJ _ { g f } = \\left( \\begin{array} { c c } { 0 } & { J _ { + } } \\\\ { 1 } & { 0 } \\\\ \\end{array} \\right) \\quad ; \\quad \\tilde { J } _ { g f } = \\left( \\begin{array} { c c } { 0 } & { 1 } \\\\ { J _ { + } } & { 0 } \\\\ \\end{array} \\right) \\quad ; \\quad g = \\left( \\begin{array} { c c } { - J _ { + } g _ { 2 2 } } & { - g _ { 2 1 } } \\\\ { g _ { 2 1 } } & { g _ { 2 2 } } \\\\ \\end{array} \\right) .\nZ ^ { i n t } ( s ) = \\int { \\cal D } A _ { \\mu } ~ e ^ { - S _ { b o s } ( A ) - i \\int d ^ { 3 } x ~ \\varepsilon ^ { \\mu \\nu \\rho } A _ { \\mu } \\partial _ { \\nu } s _ { \\rho } } ~ ,\n\\hat { \\Gamma } _ { 0 } \\ldots \\hat { \\Gamma } _ { ( 1 0 ) } = 1\n{ \\cal L } = - V ( T ) \\sqrt { - \\operatorname* { d e t } ( g + F ) } { \\cal F } ( z ) ,\nW _ { \\infty } ^ { 3 , D } = { \\frac { 1 } { 1 6 } } ( 1 - \\gamma ) ^ { - 1 } \\sum _ { k = 0 } ^ { \\infty } ( - 1 ) ^ { k } \\beta ^ { k } ( k + 1 ) ( k + 3 ) ( k + 5 ) \\sum _ { m = 0 } ^ { \\infty } ( m ^ { 2 + k } - m ^ { 1 + k } ) \\alpha _ { m } ^ { - k - 7 } .\n\\begin{array} { c } { \\left\\{ M _ { 1 } , M _ { 2 } \\right\\} = a M _ { 1 } M _ { 2 } - M _ { 1 } M _ { 2 } a , } \\\\ { a = \\frac 1 2 \\left( r - r ^ { * } \\right) . } \\\\ \\end{array}\nB ( T ) = - \\; { \\frac { b _ { 2 } ( V , T ) } { [ b _ { 1 } ( V , T ) ] ^ { 2 } } } \\; ,\n{ \\cal H } = { \\cal H } _ { a } \\otimes { \\cal H } _ { A } \\otimes { \\cal H } _ { c } .\nH r ( \\tau ) = \\frac { k } { \\sqrt { 1 + k ^ { 2 } } } \\mid \\mathrm { s n } [ \\frac { \\tau } { \\sqrt { 1 + k ^ { 2 } } } , \\; k ] \\mid .\n\\delta _ { \\beta } \\left( \\alpha \\right) = \\alpha \\wedge \\beta = \\alpha _ { a b } \\cdot \\beta _ { c d } \\left( \\chi ^ { a b } \\otimes \\chi ^ { c d } \\right) A d _ { R }\nT \\sim \\frac { m } { L ^ { p } } \\sim \\frac { 1 } { g \\ell _ { s } ^ { p + 1 } } .\n( - 1 ) ^ { | X | | Z | } [ X , [ Y , Z ] ] + ( - 1 ) ^ { | X | | Y | } [ Y , [ Z , X ] ] + ( - 1 ) ^ { | Y | | Z | } [ Z , [ X , Y ] = 0 ,\n{ \\cal M } _ { b } ^ { u u } = \\frac { i g ^ { 4 } } { 2 } [ T ^ { a } T ^ { b } \\otimes T _ { a } T _ { b } ] \\int \\frac { d ^ { 2 } k } { ( 2 \\pi ) ^ { 2 } } \\left[ \\frac { T ( k , p _ { 1 } ) T ^ { * } ( k , p _ { 3 } ) } { w _ { k } - w _ { p } } \\right]\n\\int _ { C } \\frac { d \\nu \\: \\nu } { i 4 \\sqrt { 2 } \\operatorname { s i n } \\pi \\nu } \\left[ J _ { \\nu } ( z _ { 1 } ) J _ { - \\nu } ( z _ { 2 } ) + J _ { - \\nu } ( z _ { 1 } ) J _ { \\nu } ( z _ { 2 } ) \\right] \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\;\n\\zeta _ { 0 } ( \\nu ) = - { \\frac { \\nu \\varrho ^ { - 2 \\nu } } { \\pi } } \\int _ { \\mu } ^ { \\infty } d \\omega \\int _ { C _ { + } } d z { \\frac { 2 z ^ { 2 } } { ( z ^ { 2 } + \\omega ^ { 2 } ) ^ { \\nu + 1 } } } \\breve { \\Psi } ( \\omega ; z ) e ^ { i \\epsilon z } ~ ~ ~ ,\n\\Psi = { C } \\Psi ^ { \\ast } \\, .\na _ { \\alpha } W _ { i } Z _ { \\alpha } ^ { I } Y _ { 0 } ^ { \\alpha } + W Z _ { \\alpha } ^ { I } Y _ { I } ^ { \\alpha }\nQ _ { A } + ( \\gamma ^ { 6 } \\gamma ^ { 7 } \\gamma ^ { 8 } \\gamma ^ { 9 } ) _ { A } ^ { B } \\tilde { Q } _ { B } \\nonumber\nW _ { ( 1 , 2 \\ell , 1 ) } W _ { ( 1 , 1 , Z _ { 2 } ) } W _ { ( 1 , 2 \\ell - 1 , 1 ) } W _ { ( 1 , 1 , Z _ { 2 } ) }\nV ( z ) = A ^ { 2 } e ^ { - 2 \\alpha z } - 2 A e ^ { - \\alpha z } \\ , \\ A , \\alpha > 0 \\ .\n\\sigma _ { \\pm } = \\frac { \\tau \\pm \\sigma } { 2 } \\ .\nk ^ { u } ( K _ { u v } ) _ { j } ^ { i } = \\partial _ { v } P _ { j } ^ { i } + [ p _ { v } , P ] _ { j } ^ { i } ,\n\\rho _ { 0 } ( \\theta ) = \\frac { 1 } { \\pi } \\sqrt { \\mu - \\frac { 1 } { 4 } \\mu ^ { 2 } \\theta ^ { 2 } } ,\nC _ { A B C } = \\frac { 1 } { 2 4 \\pi } \\big ( A _ { [ A } ^ { i } \\partial _ { B } A _ { C ] } ^ { i } + i \\frac { 2 } { 3 } f ^ { i j k } A _ { [ A } ^ { i } A _ { B } ^ { j } A _ { C ] } ^ { k } \\big ) .\n= g ^ { 2 } \\left( \\begin{array} { c c } { \\delta _ { \\mu \\nu } \\delta _ { A B } \\delta ^ { 4 } ( x - y ) } & { 0 } \\\\ { 0 } & { \\delta _ { A B } \\delta ^ { 4 } ( x - y ) } \\\\ \\end{array} \\right)\n\\Psi , ~ ~ ~ \\Phi , ~ ~ ~ \\Pi , ~ ~ ~ \\Xi , ~ ~ ~ { \\cal A } _ { w } , ~ ~ ~ { \\cal A } _ { \\theta } , ~ ~ ~ { \\cal A } , ~ ~ ~ \\Delta , ~ ~ ~ \\Delta ^ { * } .\nC _ { N _ { P + 1 } + \\dots + N _ { P + K } } ^ { N _ { P + 1 } } C _ { N _ { P + 2 } + \\dots + N _ { P + K } } ^ { N _ { P + 2 } } \\dots C _ { N _ { P + K - 1 } + N _ { P + K } } ^ { N _ { P + K - 1 } }\n( N _ { c } + \\tilde { N } _ { c } , \\overline { { N _ { c } + \\tilde { N } _ { c } } } ) ( 1 , 1 , 0 , \\beta _ { i } ) , \\ \\ \\ i = 1 , 2 , 3\n{ \\cal D } = \\sum _ { i , j } c _ { i , j } \\, \\frac { ( g ^ { 2 } N ) ^ { i } } { N ^ { 2 j } } ,\n{ \\cal R } = { \\cal R } _ { r } \\operatorname { s i n } \\eta , ~ ~ ~ \\tau = { \\cal R } _ { r } ( 1 - \\operatorname { c o s } \\eta ) .\n\\left\\{ \\begin{array} { r c l } { \\displaystyle \\partial ^ { m } h _ { m } - \\frac { 1 } { 2 } \\dot { h } ^ { \\prime } } & { = } & { 0 , \\vspace { 2 m m } } \\\\ { \\displaystyle \\partial ^ { n } h _ { m n } - \\frac { 1 } { 2 } \\partial _ { m } h ^ { \\prime } } & { = } & { 0 , \\vspace { 2 m m } } \\\\ { \\displaystyle \\partial ^ { m } \\theta _ { m } - \\frac { 1 } { 2 } h ^ { \\prime } } & { = } & { 0 . } \\\\ \\end{array} \\right.\nZ ( { j _ { k } } ) = e ^ { i Z _ { c } ( { j _ { k } } ) }\n| \\{ m s s \\} | = \\sum _ { \\nu = 0 } ^ { k } { \\binom { k } { \\nu } } = ( 1 + 1 ) ^ { k } = 2 ^ { k }\nE _ { 4 s } = 2 ( Q + Q ^ { - 1 } ) + \\xi _ { 1 } + \\xi _ { 1 } ^ { - 1 } + \\xi _ { 2 } + \\xi _ { 2 } ^ { - 1 }\n\\stackrel { \\mathrm { G } } { { \\mathcal L } } \\, : = \\frac { 1 } { 2 m } \\left[ ( D _ { \\alpha } \\overline { { \\Psi } } ) D ^ { \\alpha } \\Psi - m ^ { 2 } \\overline { { \\Psi } } \\Psi \\right] \\, .\np ( x ) = \\prod _ { i = 1 } ^ { N + 1 } ( x + \\nu _ { i } ) \\, , \\quad q ( x ) = p ( x ) - e ^ { i \\theta } / r ^ { N + 1 } = \\prod _ { i = 1 } ^ { N + 1 } ( x - x _ { i } ) \\, ,\n\\tau _ { c l } = { \\frac { \\theta _ { 0 } } { 2 \\pi } } + { \\frac { 4 \\pi i } { g _ { 0 } ^ { 2 } } } ,\nt = ( H ^ { - 2 } - r ^ { 2 } ) ^ { 1 / 2 } \\operatorname { s i n h } ( H \\tau ) ; \\quad v = ( H ^ { - 2 } - r ^ { 2 } ) ^ { 1 / 2 } \\operatorname { c o s h } ( H \\tau ) ; \\quad x = r\n\\begin{array} { l l l } { \\langle q _ { i 0 } - Q _ { i 0 } \\rangle } & { = } & { 0 \\, , } \\\\ { \\langle \\dot { q } _ { i 0 } \\rangle } & { = } & { 0 \\, , } \\\\ { m \\omega _ { i } \\omega _ { j } \\langle ( q _ { i 0 } - Q _ { i 0 } ) ( q _ { j 0 } - Q _ { j 0 } ) \\rangle } & { = } & { k _ { B } T \\delta _ { i j } \\, , } \\\\ { m \\langle \\dot { q } _ { i 0 } \\dot { q } _ { j 0 } \\rangle } & { = } & { k _ { B } T \\delta _ { i j } \\, , } \\\\ { \\langle \\dot { q } _ { i 0 } ( q _ { j 0 } - Q _ { j 0 } ) \\rangle } & { = } & { 0 \\, . } \\\\ \\end{array}\ni \\frac { \\partial } { \\partial \\tau _ { 2 } } < m ^ { \\prime } , \\bar { m ^ { \\prime } } ; \\tau _ { 2 } | m , \\bar { m } ; \\tau _ { 1 } > = \\hat { H } ( m ^ { \\prime } , \\bar { m ^ { \\prime } } ) < m ^ { \\prime } , \\bar { m ^ { \\prime } } ; \\tau _ { 2 } | m , \\bar { m } ; \\tau _ { 1 } > .\nf = \\frac { 1 } { 2 } \\partial _ { \\sigma } Z .\n{ \\cal L } _ { e f f } = { \\cal L } ^ { ( 2 ) } + { \\cal L } ^ { ( 4 ) } + { \\cal L } ^ { ( 6 ) } + . . .\n{ \\bf E } _ { \\pm } ( \\xi _ { \\pm } ) = \\mathrm { T } _ { \\pm } ( \\xi _ { \\pm } ) { \\bf E } _ { 0 } \\mathrm { T } _ { \\pm } ^ { \\dagger } ( \\xi _ { \\pm } ) ,\n\\psi ^ { \\prime } ( t , x ) = e ^ { - i e \\epsilon ( t , x ) } \\psi ( t , x ) \\ \\ , \\ \\ A _ { \\mu } ^ { \\prime } ( t , x ) = A _ { \\mu } ( t , x ) + \\partial _ { \\mu } \\epsilon ( t , x ) \\ ,\n\\delta ( f ( x ) ) = \\sum _ { i = 1 } ^ { n } \\frac { 1 } { | f ^ { \\prime } ( x _ { i } ) | } \\delta ( x - x _ { i } )\nV _ { A } = T ^ { a } \\int _ { 0 } ^ { T } d \\tau \\, \\bigl [ \\dot { x } _ { \\mu } \\varepsilon _ { \\mu } - 2 \\mathrm { i } \\psi _ { \\mu } \\psi _ { \\nu } k _ { \\mu } \\varepsilon _ { \\nu } \\bigr ] \\mathrm { e x p } [ i k x ( { \\tau } ) ]\n\\Omega : u \\rightarrow \\ { S _ { \\Omega } } \\left( u \\right) = { { \\Omega } ^ { - 1 } } u .\nt _ { a } ( M ) - t _ { a } ^ { * } ( M ) = t _ { a } ( M ) ( R - R ^ { * } ) t _ { a } ^ { * } ( M )\n- { \\cal L } _ { \\xi } J ^ { \\mu \\nu } = \\hat { \\xi } ^ { \\lambda } \\partial _ { \\lambda } J ^ { \\mu \\nu } - \\partial _ { \\lambda } \\hat { \\xi } ^ { \\mu } J ^ { \\lambda \\nu } - \\partial _ { \\lambda } \\hat { \\xi } ^ { \\nu } J ^ { \\mu \\lambda }\n\\widehat { F } _ { \\mu \\nu } = F _ { \\mu \\nu } + \\frac { 1 } { 4 } \\theta ^ { \\rho \\sigma } \\bigg ( 2 \\left\\{ F _ { \\mu \\rho } , F _ { \\nu \\sigma } \\right\\} - \\left\\{ A _ { \\rho } , D _ { \\sigma } F _ { \\mu \\nu } + \\partial _ { \\sigma } F _ { \\mu \\nu } \\right\\} \\bigg ) + { \\cal O } \\left( \\theta ^ { 2 } \\right) .\n| q ( x , \\xi ) | = | \\chi ( x , \\xi ) a ( x , \\xi ) ^ { - 1 } | \\geq C _ { K } ( 1 + | \\xi | ) ^ { - m } ,\nd s _ { M } ^ { 2 } = g ^ { 4 / 3 } d x _ { 1 1 } ^ { 2 } + g ^ { - 2 / 3 } d s _ { 1 0 } ^ { 2 }\n\nS _ { m a t t e r } = 1 6 \\pi \\int d ^ { 4 } x \\sqrt { - g } \\; e ^ { 2 ( \\sigma - 1 ) \\psi } \\; L _ { m a t t e r } .\nH ^ { ( N ) } ( L , r , l | d , p ) = \\sum _ { k = l + p } ^ { N } q ^ { - \\sum _ { i = d } ^ { p - 1 } m _ { i } - e _ { d - 1 } C ^ { - 1 } m } B ^ { ( N ) } ( L , r , l + 1 | k ) .\n\\sigma = 4 \\pi g ^ { 2 } M ~ ~ \\mathrm { a n d } ~ ~ \\alpha ^ { - 1 } = - \\frac { \\pi g ^ { 2 } } { 2 M } ,\nd _ { \\mathrm { e l } } ^ { - 2 } = \\rho _ { \\mathrm { p o l } } / A _ { 0 } \\sim g ^ { 2 } T a ^ { 2 - d } \\, .\nV ( \\phi ) \\ = \\ < H _ { \\mu \\nu } ^ { 2 } > \\cdot ( 1 - [ \\rho ^ { 2 } + \\sigma ^ { 2 } ] ) ^ { 2 } \\ \\sim \\ \\lambda ( 1 - | \\phi | ^ { 2 } ) ^ { 2 }\n\\int _ { \\Sigma } ( i ( D g ) \\varphi - ( A g + i g ) \\Lambda ) \\Psi _ { 0 } = \\int _ { \\Sigma } g ( i D \\varphi + C \\Lambda - i \\Lambda ) \\Psi _ { 0 }\nV _ { \\mu } ^ { s } ( x , p ) = { \\pi } ^ { - 4 } \\partial _ { \\nu } ^ { ( x ) } \\int d ^ { 4 } q e ^ { 2 i p . q } { \\bar { \\psi } } ( x + q ) { \\frac { \\sigma _ { \\mu \\nu } } { 2 m } } \\psi ( x - q ) .\n- \\left( 2 s + 1 \\right) - \\frac { \\left( 2 s + 1 \\right) ( e \\tau ) ^ { 2 } \\mathcal { F } } 3 \\left[ s ( s + 1 ) \\left( g ^ { 2 } - \\sigma ^ { 2 } \\right) - 1 \\right] \\biggl \\} ,\n\\int _ { 0 } ^ { t } d t ^ { \\prime } \\int d ^ { 2 } \\xi ^ { \\prime } K _ { \\xi \\xi ^ { \\prime } } ^ { t - t ^ { \\prime } } e ^ { - \\phi ( \\xi ) } \\left( - \\frac { 1 } { 2 } ( \\xi ^ { \\prime } - \\xi ) ^ { a } ( \\xi ^ { \\prime } - \\xi ) ^ { b } \\left( \\partial _ { a } \\phi ( \\xi ) \\partial _ { b } \\phi ( \\xi ) - \\partial _ { a } \\partial _ { b } \\phi ( \\xi ) \\right) \\right) \\Delta ^ { \\prime } K _ { \\xi ^ { \\prime } \\xi } ^ { t ^ { \\prime } }\nm \\epsilon _ { l n } F _ { n 0 } - j _ { l } = 0 ,\ni D _ { k } M = [ C _ { k } , M ] = B _ { k l } [ x ^ { l } , M ] + [ \\hat { A } _ { l } , M ] \\ .\n{ \\tilde { W } } \\equiv W - S _ { 0 } ~ , \\quad { \\tilde { X } } \\equiv X + S _ { 0 } ~ .\n\\Delta _ { \\mu \\nu } ( p ) = \\frac { 4 \\pi } { \\kappa } \\frac { \\mu } { p ^ { 2 } ( p ^ { 2 } + \\mu ^ { 2 } ) } ( \\mu \\epsilon _ { \\mu \\nu \\lambda } p ^ { \\lambda } + \\delta _ { \\mu \\nu } p ^ { 2 } - p _ { \\mu } p _ { \\nu } )\n{ \\mathrm { T r } } ( \\gamma _ { g } ) = - 2 ~ , ~ ~ ~ { \\mathrm { T r } } ( \\gamma _ { R _ { s } } ) = { \\mathrm { T r } } ( \\gamma _ { g R _ { s } } ) = 0 ~ .\nG _ { \\alpha } ^ { ( n ) } \\equiv \\sum _ { m = 0 } ^ { n } \\{ \\tau _ { \\alpha } ^ { ( n - m ) } , \\tilde { H } ^ { ( m ) } \\} + \\sum _ { m = 0 } ^ { n - 2 } \\{ \\tau _ { \\alpha } ^ { ( n - m ) } , \\tilde { H } ^ { ( m + 2 ) } \\} _ { ( \\eta ) } + \\{ \\tau _ { \\alpha } ^ { ( n + 1 ) } , \\tilde { H } ^ { ( 1 ) } \\} _ { ( \\eta ) } ; \\hspace { 0 . 5 c m } n \\geq 2 .\n\\tilde { f } _ { 2 } = \\Phi ( x - z _ { 2 } , x - z _ { 2 } )\nD _ { \\mu } D ^ { \\mu } \\phi ^ { a } = f _ { \\phantom { a } b c } ^ { a } \\psi _ { \\mu } ^ { b } \\psi _ { \\mu } ^ { c } .\n\\delta _ { { \\hat { \\xi } } _ { 2 } } L [ { \\hat { \\xi } } _ { 1 } ] = \\left\\{ L [ { \\hat { \\xi } } _ { 2 } ] , L [ { \\hat { \\xi } } _ { 1 } ] \\right\\} = L [ \\{ { \\hat { \\xi } } _ { 1 } , { \\hat { \\xi } } _ { 2 } \\} _ { \\mathrm { \\scriptsize ~ S D } } ] + K [ { \\hat { \\xi } } _ { 1 } , { \\hat { \\xi } } _ { 2 } ]\nU _ { \\bf n } ^ { l } ( a ) = U _ { \\bf n } a \\, , \\qquad ( a ) U _ { \\bf n } ^ { r } = a U _ { A \\bf n } \\, .\n{ \\bf S } = - ( { \\bf r } - { \\bf q } ) \\times m { \\bf u } ,\n{ \\bf W } _ { a a } = - \\sum _ { c \\neq a } { \\mathrm { \\boldmath ~ \\beta ~ } } _ { a } ^ { * } \\cdot { \\mathrm { \\boldmath ~ \\beta ~ } } _ { c } ^ { * } { \\bf w } _ { a c } , \\, \\, \\, \\, { \\bf W } _ { a b } = { \\mathrm { \\boldmath ~ \\beta ~ } } _ { a } ^ { * } \\cdot { \\mathrm { \\boldmath ~ \\beta ~ } } _ { b } ^ { * } { \\bf w } _ { a b } \\quad \\mathrm { \\hspace { ~ 1 c m } ~ i f ~ a \\neq ~ b ~ } .\n\\frac { \\mathrm { e } ^ { - x ^ { 2 } / 2 t } } { \\left( 2 \\pi t \\right) ^ { D / 2 } } = \\int \\frac { \\mathrm { d } ^ { D } p } { \\left( 2 \\pi \\right) ^ { D } } \\; \\mathrm { e } ^ { - i p \\cdot x - \\frac 1 2 p ^ { 2 } t } ,\n{ \\frac { 1 } { 2 } } \\left\\{ \\left[ \\begin{array} { c } { Q _ { \\alpha } } \\\\ { \\tilde { Q } _ { \\alpha } } \\\\ \\end{array} \\right] , \\left[ Q _ { \\beta } \\ \\tilde { Q } _ { \\beta } \\right] \\right\\} = \\left[ \\begin{array} { c c } { 1 } & { 0 } \\\\ { 0 } & { 1 } \\\\ \\end{array} \\right] M \\delta _ { \\alpha \\beta } + \\left[ \\begin{array} { c c } { p } & { q / g _ { s } } \\\\ { q / g _ { s } } & { - p } \\\\ \\end{array} \\right] { \\frac { { L ( \\Gamma ^ { 0 } \\Gamma ^ { 1 } ) } _ { \\alpha \\beta } } { 2 \\pi { \\alpha ^ { \\prime } } } } \\ .\nS = \\int d ^ { d - 1 } x \\sqrt { | g | } \\ \\left[ R + { \\textstyle \\frac { 1 } { 2 } } \\left( \\partial \\varphi \\right) ^ { 2 } - { \\textstyle \\frac { 1 } { 4 } } e ^ { - a \\varphi } F _ { ( 2 ) } ^ { 2 } + { \\textstyle \\frac { 1 } { 2 } } \\left( { \\cal D } \\phi \\right) ^ { 2 } - { \\textstyle \\frac { 1 } { 2 } } m ^ { 2 } e ^ { a \\varphi } \\right] \\, ,\nw _ { 1 } = \\left( \\begin{array} { c c c } { - 1 } & { } & { } \\\\ \\end{array} \\right)\nF _ { \\mu \\rho } ^ { a } F _ { a \\nu } ^ { \\; \\; \\; \\; \\rho } - { \\frac { 1 } { 4 } } g _ { \\mu \\nu } x F _ { \\rho \\sigma } ^ { a } F _ { a } ^ { \\rho \\sigma }\n\\pm 1 = \\frac { 1 } { 2 { \\pi } i } \\int T r { \\cal P } ^ { ( { \\pm } 1 ) } d { \\cal P } ^ { ( { \\pm } 1 ) } { \\wedge } d { \\cal P } ^ { ( { \\pm } 1 ) } .\n\\zeta _ { \\pm } ^ { 1 } ( z ) = N _ { 1 } \\left( \\begin{array} { c } { z ^ { \\frac { M } { 2 } } ( 1 - z ) ^ { - \\frac { M } { 2 } } F ( 1 , - 1 , \\frac { 1 } { 2 } + M ; z ) } \\\\ { \\pm \\frac { i } { \\frac { 1 } { 2 } + M } z ^ { \\frac { 1 + M } { 2 } } ( 1 - z ) ^ { \\frac { M } { 2 } } F ( - \\frac { 1 } { 2 } + M , \\frac { 3 } { 2 } + M , \\frac { 3 } { 2 } + M ; z ) } \\\\ \\end{array} \\right)\nS _ { s h e l l } \\sim ( \\sigma _ { a b } ) ^ { 2 } \\frac { r ^ { 2 } } { L ^ { 2 } } L ^ { 2 } \\sim \\frac { L ^ { 2 } M ^ { 2 } } { r ^ { 2 } ( 1 - 2 M / r ) ^ { 2 } } .\nH _ { m n } \\equiv F _ { m n } , \\, \\quad \\quad B _ { m } \\equiv F _ { 3 m } = - F _ { m 3 } , \\,\n\\delta \\vec { \\alpha } = { \\textstyle \\frac 1 2 } \\, ( a - F ) \\, \\delta \\theta \\, \\vec { \\alpha } \\, , \\qquad \\delta a = { \\textstyle \\frac 1 2 } \\, ( a - F ) \\, \\delta \\theta \\, a \\, .\nW \\to { \\frac { m ^ { 3 } N ^ { 2 } E _ { 2 } ( \\tau ) } { 9 6 \\pi g } } - { \\frac { m ^ { 3 } N ^ { 3 } } { 9 6 \\pi g q ^ { 2 } ( \\tau + k ) ^ { 2 } } } \\ .\nW [ B , J , \\eta ^ { \\ast } , \\eta ] = \\operatorname { l n } Z [ B , J , \\eta ^ { \\ast } , \\eta ] \\quad .\n\\dot { \\omega } + e _ { \\alpha \\beta } \\partial _ { \\alpha } \\psi \\partial _ { \\beta } \\Delta \\psi = \\nu \\Delta \\omega \\ ,\nj _ { ( \\varphi ) } ^ { \\mu i } = \\epsilon ^ { i j } \\partial _ { j } x ^ { \\mu } \\quad , \\quad j ^ { \\alpha i } = ( 1 / 2 ) \\epsilon ^ { i j } \\partial _ { j } \\theta ^ { \\alpha } \\ ,\n\\Delta _ { N , 2 } ^ { \\mu \\nu , \\lambda \\sigma } = \\frac { 1 } { 2 ( p ^ { 2 } - i \\epsilon ) } \\left( g ^ { \\mu \\nu } g ^ { \\lambda \\sigma } + ( g ^ { \\mu \\nu } n ^ { \\lambda } n ^ { \\sigma } + n ^ { \\mu } n ^ { \\nu } g ^ { \\lambda \\sigma } ) \\frac { p ^ { 2 } } { | { \\bf p } | ^ { 2 } } + n ^ { \\mu } n ^ { \\nu } n ^ { \\lambda } n ^ { \\sigma } \\frac { ( p ^ { 2 } ) ^ { 2 } } { | { \\bf p } | ^ { 4 } } \\right) .\nc _ { l , m } = \\eta ( \\tau ) ^ { - 3 } \\sum _ { - | x | < y \\leq | x | } \\mathrm { s i g n } ( x ) ~ q ^ { ( k + 2 ) x ^ { 2 } - k y ^ { 2 } }\ng ^ { 2 } = - \\frac { ( M \\mp m _ { 1 } ) ( M \\mp m _ { 2 } ) } { ( m _ { 2 } - m _ { 1 } ) ( \\mu _ { 1 } I _ { 1 } \\pm M I _ { 0 } ) } \\, .\nS = - \\frac { 1 } { 2 } \\int d ^ { 4 } x \\sqrt { - \\bar { g } } \\left( \\bar { R } - \\frac { 1 } { 2 } ( \\bar { \\nabla } \\phi ) ^ { 2 } \\right)\nR _ { \\gamma \\alpha \\beta } ^ { \\mu } = \\partial _ { \\alpha } \\Gamma _ { \\beta \\gamma } ^ { \\mu } - \\partial _ { \\beta } \\Gamma _ { \\alpha \\gamma } ^ { \\mu } + \\Gamma _ { \\alpha \\lambda } ^ { \\mu } \\Gamma _ { \\beta \\gamma } ^ { \\lambda } - \\Gamma _ { \\beta \\lambda } ^ { \\mu } \\Gamma _ { \\alpha \\gamma } ^ { \\lambda }\n\\begin{array} { r l } { { \\cal L } = } & { \\frac { 1 } { g ^ { 2 } } \\int \\left( - \\frac { 1 } { 4 } F _ { m n } F ^ { m n } + \\frac { 1 } { 2 } D _ { m } \\Phi ^ { a b } D ^ { m } \\Phi _ { a b } + \\frac { 1 } { 4 } \\{ \\Phi ^ { a b } , \\Phi ^ { c d } \\} _ { * } \\{ \\Phi _ { a b } , \\Phi _ { c d } \\} _ { * } \\right. + } \\\\ \\end{array}\n{ } ^ { \\gamma \\gamma } G _ { 1 2 3 4 } ^ { 4 } = 4 \\left\\{ \\frac { \\delta ^ { 2 } W } { \\delta D _ { 1 2 } ^ { - 1 } \\delta D _ { 3 4 } ^ { - 1 } } + \\frac { \\delta W } { \\delta D _ { 1 2 } ^ { - 1 } } \\frac { \\delta W } { \\delta D _ { 3 4 } ^ { - 1 } } \\right\\} = - 2 \\frac { \\delta \\; { } ^ { \\gamma } G _ { 1 2 } ^ { 2 } } { \\delta D _ { 3 4 } ^ { - 1 } } + { } ^ { \\gamma } G _ { 1 2 } ^ { 2 } { } ^ { \\gamma } G _ { 3 4 } ^ { 2 } .\ny _ { H } \\equiv \\alpha ^ { - 1 } \\sqrt { \\mu } r _ { + } , \\ \\ y _ { i } \\equiv \\alpha ^ { - 1 } { { \\tilde { q } } _ { i } } \\ , ( i = 1 , 2 , 3 ) \\ ,\n\\left\\langle { { \\xi } \\left\\vert { \\xi ^ { \\prime } } \\right\\rangle } \\right. = { \\frac { \\left( { 1 - { \\vert \\xi \\vert } ^ { 2 } } \\right) ^ { K } \\left( { 1 - { \\vert \\xi ^ { \\prime } \\vert } ^ { 2 } } \\right) ^ { K } } { \\left( { 1 - \\xi ^ { * } \\xi ^ { \\prime } } \\right) ^ { 2 K } } } \\ ,\nS = \\frac { 1 } { 4 \\ell _ { \\mathrm { P } } ^ { 2 } } \\int d ^ { 4 } x ~ a ^ { 2 } ~ \\eta ^ { \\mu \\nu } \\partial _ { \\mu } h \\partial _ { \\nu } h ,\n{ \\frac { d ^ { 2 } \\tilde { \\psi } _ { \\infty } } { d { \\rho ^ { * } } ^ { 2 } } } - \\left[ { \\frac { 3 } { 4 } } - \\mu \\right] { \\frac { \\rho ^ { 2 } } { R ^ { 4 } } } \\tilde { \\psi } _ { \\infty } = 0 .\n\\begin{array} { l l l } { U ( 3 ) } & { \\longrightarrow } & { U ( 5 ) / U ( 2 ) } \\\\ \\end{array}\n\\zeta ( z , a ) = \\sum _ { n = - \\infty } ^ { \\infty } \\frac { 1 } { ( n ^ { 2 } + a ^ { 2 } ) ^ { z } } \\, \\, \\, \\, a ^ { 2 } > 0 ,\nk = \\pm \\omega \\frac { \\sqrt { 1 + B ^ { 2 } - E ^ { 2 } } \\mp E B } { 1 - E ^ { 2 } } \\, ,\n\\lambda _ { m } ^ { ( n - p - 1 ) } = \\lambda _ { m } ^ { ( p ) }\n\\int [ d \\bar { \\psi } ] [ d \\psi ] \\operatorname { e x p } \\left\\{ - \\int \\bar { \\psi } D \\psi d x \\right\\} = \\prod _ { n } \\lambda _ { n } = { \\tt d e t } D ,\n( t \\partial _ { z } + z ^ { - 1 } ( D _ { \\jmath _ { 1 } } ^ { + } D _ { \\jmath _ { 1 } } ^ { - } - 2 \\jmath _ { 0 } D _ { \\jmath _ { 1 } } ^ { 0 } ) ) - ( \\tilde { J } ^ { + } ( z ) D _ { \\jmath _ { 1 } } ^ { - } + 2 \\tilde { J } ^ { 0 } ( z ) D _ { \\jmath _ { 1 } } ^ { 0 } + \\tilde { J } ^ { - } ( z ) D _ { \\jmath _ { 1 } } ^ { + } )\n( i d \\otimes \\pi _ { - } ) \\delta _ { V } ^ { 2 } ( x _ { 2 } ^ { j } ( \\xi , \\eta ) ) = \\sum _ { k } { \\cal D } ( A _ { 2 } , D _ { 2 } ) _ { k } ^ { j } \\ ( i d \\otimes \\pi _ { - } ) \\Delta ( x _ { 2 } ^ { k } ( \\xi , \\eta ) ) .\nP ^ { a } = { \\frac { 1 } { 2 } } \\pi ^ { 2 } | \\phi _ { 0 } | ^ { 2 } \\; \\zeta ^ { a } .\n\\sigma _ { q } ( X _ { i } + a ) = \\sum _ { r = 0 } ^ { q } \\left( \\begin{array} { c } { p - q + r } \\\\ { p - q } \\\\ \\end{array} \\right) a ^ { r } \\sigma _ { q - r } ( X _ { i } )\n\\theta _ { s } ( \\sum ( x _ { i } - y _ { i } ) / 2 + z - w ) \\prod _ { I } \\theta _ { s , h _ { I } } ( \\sum ( x _ { i } + y _ { i } ) / 2 - \\sum u _ { i , I } ) ~ .\nL _ { m } ^ { ( g / p ) } \\equiv L _ { m } ^ { ( g ) } - L _ { m } ^ { ( p ) }\n\\partial ^ { \\mu } A _ { \\mu } ^ { a } \\left( x \\right) + B ^ { a } \\left( x \\right) = 0 ,\n\\frac { d \\Phi } { d r } = - \\frac { \\sqrt { 3 } } { 4 } \\ell ( r ) ^ { \\Lambda \\Sigma } \\, \\mathrm { I m } { \\cal N } _ { \\Lambda \\Sigma | \\Gamma \\Delta } \\, f _ { ~ ~ A B } ^ { \\Gamma \\Delta } \\, \\frac { e ^ { U } } { r ^ { 2 } } \\, .\n\\hat { B } _ { 1 q _ { + } } ( \\omega ) = { \\frac { 2 \\operatorname { s i n h } \\Bigl ( ( \\nu - q _ { + } ) { \\frac { \\omega } { 2 } } \\Bigr ) \\operatorname { c o s h } \\Bigl ( ( \\nu - 1 ) { \\frac { \\omega } { 2 } } \\Bigr ) } { \\operatorname { s i n h } ( { \\frac { \\nu \\omega } { 2 } } ) } } \\, .\n- \\left[ J ^ { 1 } ( x ) \\right] _ { - L / 2 } ^ { L / 2 } = \\left[ ( / \\hspace { - 0 . 5 e m } \\partial \\phi + U ) \\gamma ^ { 1 } \\psi - ( - / \\hspace { - 0 . 5 e m } \\partial \\phi + U ) \\gamma ^ { 1 } \\gamma ^ { 3 } \\psi \\right] _ { L / 2 } = \\left. ( 1 + \\gamma _ { 3 } ) ( / \\hspace { - 0 . 5 e m } \\partial \\phi + U ) \\gamma ^ { 1 } \\psi \\right| _ { L / 2 } .\nJ = \\left( \\begin{array} { c c } { 1 } & { 0 } \\\\ { A } & { 1 } \\\\ \\end{array} \\right) ,\na _ { 1 } \\, = \\, b _ { 2 } \\, = \\, - \\, 1 \\; \\; ; \\; \\; a _ { 2 } \\, = \\, b _ { 1 } \\, = \\, 1 \\; \\; ; \\; \\; a _ { 3 } \\, = \\, b _ { 3 } \\, = \\, 0 \\; \\; ,\nC ( X ) = c _ { i } ^ { } \\, \\psi _ { i } ^ { } = c _ { i } ^ { } \\, v _ { i } ^ { 1 / 2 } \\, \\psi _ { i } ^ { - } .\ng _ { \\mu \\nu } \\partial _ { n } X ^ { \\nu } + 2 \\pi \\alpha ^ { \\prime } B _ { \\mu \\nu } \\partial _ { s } X ^ { \\nu } | _ { \\partial \\Sigma } = 0 ,\n\\Phi ( x ) = \\left( \\begin{array} { c } { G _ { \\alpha } ^ { \\beta } ( x ) } \\\\ { \\Psi _ { \\alpha } ( x ) } \\\\ \\end{array} \\right)\n\\Lambda _ { \\infty } R = e ^ { a ( \\infty ) } \\Lambda _ { \\overline { { M S } } } R = e ^ { \\gamma _ { E } } r / \\sqrt { 6 }\n\\left. \\left( \\begin{array} { c c c } { \\partial \\beta _ { g } / \\partial g } & { \\partial \\beta _ { g } / \\partial h } & { \\partial \\beta _ { g } / \\partial f } \\\\ { \\partial \\beta _ { h } / \\partial g } & { \\partial \\beta _ { h } / \\partial h } & { \\partial \\beta _ { h } / \\partial f } \\\\ { \\partial \\beta _ { f } / \\partial g } & { \\partial \\beta _ { f } / \\partial h } & { \\partial \\beta _ { f } / \\partial f } \\\\ \\end{array} \\right) \\right| _ { g = g _ { \\ast } , h = h _ { \\ast } , f = f _ { \\ast } }\n\\mathrm { c } = \\frac { \\theta } { n } ,\n{ \\cal K } ^ { + 4 } = \\mathrm { T r } \\left( [ \\tilde { q } ^ { + } , q ^ { + } ] [ \\tilde { q } ^ { + } , q ^ { + } ] \\right) \\, .\n\\theta _ { \\mu } \\, \\to \\, \\theta _ { \\mu } - i { \\tilde { \\phi } } ^ { * } \\partial _ { \\mu } { \\tilde { \\phi } }\nA _ { i } = { \\frac { \\Phi } { 2 \\pi } } \\epsilon _ { i j } { \\frac { x ^ { j } } { { \\vert x \\vert } ^ { 2 } } }\nh : V _ { n } \\mapsto V _ { n } \\ , \\ n = 0 , 1 , 2 , \\ldots\nc ^ { 2 } M _ { 3 } ^ { 2 } - m _ { 2 } ^ { 2 } G ^ { 2 } > 0 ,\nS _ { 1 } = U \\qquad S _ { 2 } = D | _ { \\lambda = 1 } \\ ,\nh _ { c l } = \\vec { p } ^ { ~ 2 } / 2 m + V ( \\vec { r } )\n\\begin{array} { r l } { \\displaystyle Z _ { N } ( \\vartheta ) } & { = 2 N \\operatorname { a r c t a n } \\displaystyle \\frac { \\operatorname { s i n h } \\vartheta } { \\operatorname { c o s h } \\Theta } + \\displaystyle \\sum _ { k = 1 } ^ { N _ { H } } \\chi ( \\vartheta - h _ { k } ) - 2 \\displaystyle \\sum _ { k = 1 } ^ { N _ { S } } \\chi ( \\vartheta - y _ { k } ) - } \\\\ \\end{array}\n\\hat { x } ^ { + } ( \\hat { x } ^ { - } + \\frac { m } { \\lambda ^ { 3 } x _ { 0 } ^ { + } } ) + \\frac { \\kappa } { 4 \\lambda ^ { 2 } } = 0 .\n\\sum _ { n } \\longrightarrow \\int d n = \\frac { L } { 2 \\pi } \\int d k\nB _ { \\mu } = A _ { \\mu } = - 3 G _ { \\mu } ^ { 0 } ,\n\\{ \\{ F , G \\} , H \\} = \\sum I _ { A B } I _ { C D } \\int _ { \\Omega } D _ { I } { \\frac { \\partial h } { \\partial \\phi _ { B } ^ { ( J ) } } }\n\\vec { \\nabla } \\cdot \\vec { E } = g \\vec { \\nabla } f \\cdot \\vec { E } \\ ,\n\\partial _ { \\bar { z } } T _ { 4 } = \\partial _ { z } \\Theta _ { 2 } \\ ,\n{ \\cal S } _ { ( - 1 ) } = { \\cal S } _ { \\mathrm { c u b i c } } + { \\cal S } _ { \\mathrm { q u a r t i c } }\n\\int \\prod _ { i } d Y _ { i } \\delta ( \\sum _ { i = 1 } ^ { 6 } w _ { i } Y _ { i } - t ) e ^ { - \\sum _ { i } \\mu _ { i } Y _ { i } } \\ \\stackrel { Y _ { i } \\rightarrow Y _ { i } + l n \\mu _ { i } } { \\longrightarrow } \\int \\prod _ { i } d Y _ { i } \\delta ( \\sum _ { i = 1 } ^ { 6 } w _ { i } Y _ { i } - t ^ { \\prime } ) e ^ { - \\sum _ { i } Y _ { i } } ,\nh ( \\tau ) = ( g _ { s } M \\alpha ^ { \\prime } ) ^ { 2 } 2 ^ { 2 / 3 } \\varepsilon ^ { - 8 / 3 } \\int _ { \\tau } ^ { \\infty } d x { \\frac { x \\operatorname { c o t h } x - 1 } { \\operatorname { s i n h } ^ { 2 } x } } ( \\operatorname { s i n h } 2 x - 2 x ) ^ { 1 / 3 } ,\n\\delta { \\cal L } _ { \\pm } = J _ { \\pm i } \\dot { \\eta _ { i } }\nP \\sim \\operatorname { e x p } \\Bigl ( - { \\frac { 3 M _ { P } ^ { 4 } } { 8 V ( \\phi ) } } \\Bigr ) \\ ,\n\\mu = A _ { 1 } E ( a , x ) + A _ { 2 } E ^ { * } ( a , x ) ,\nf ( - 2 p )\ng _ { 1 } ( K , \\mu ) = { \\frac { 1 } { 2 } } \\, { \\frac { u _ { 1 } ^ { 2 } C _ { 1 } ( \\mu ) } { ( K + \\sqrt { K ^ { 2 } + \\mu ^ { 2 } } ) } } - { \\frac { 1 } { 2 } } { \\frac { K + \\sqrt { K ^ { 2 } + \\mu ^ { 2 } } } { C _ { 1 } ( \\mu ) } } \\, ,\nV _ { F } ^ { ( d i v ) } ( H ) = - \\frac { N } { 6 4 \\pi ^ { 2 } \\epsilon } \\left( m _ { t } ( H ) ^ { 4 } + \\sum _ { k = 1 } ^ { \\infty } \\left( \\frac { 2 k } { R } + m _ { t } ( H ) \\right) ^ { 4 } + \\sum _ { k = 1 } ^ { \\infty } \\left( - \\frac { 2 k } { R } + m _ { t } ( H ) \\right) ^ { 4 } \\right)\n\\tilde { \\Gamma } _ { 3 } = - \\frac { \\lambda ^ { 3 } } { 6 4 \\pi ^ { 3 } } \\mathrm { l n } \\left( T \\right) .\nf _ { k } ^ { ( D ) } ( \\Omega ^ { ( { D } ) } ) = - \\Gamma _ { { D } } ( k ) \\, \\frac { 1 } { 1 / \\lambda + { \\mathcal G } _ { { D } } ^ { ( + ) } ( { \\bf 0 } ; k ) } \\; ,\n\\tilde { \\Lambda } _ { S U } ^ { 2 N _ { c } ^ { \\prime } - 2 N _ { f } } = ( \\mu ^ { - 1 } a ^ { 2 } ) ^ { 2 N _ { f } - 2 N _ { c } } \\tilde { \\Lambda } ^ { \\prime 4 ( N _ { c } ^ { \\prime } - 1 ) - 2 ( N _ { f } + \\tilde { N } _ { c } ) } .\n\\left( , \\right) = \\left( , \\right) _ { 1 } + \\left( , \\right) _ { 2 } .\nE ( \\alpha = 0 , n , \\omega , \\beta ) = \\frac { 2 \\omega + 3 } { 2 \\omega + 4 } E ( \\alpha = 0 , n , \\omega = \\infty , \\beta ) \\ . \\\n{ \\frac { 1 } { 2 } } \\sum _ { i = 1 } ^ { n } \\mathrm { d e g } ( \\phi _ { i } ) = ( 1 - g ) ( d - 3 ) + n + \\int _ { \\Sigma _ { g } } x ^ { * } ( c _ { 1 } ( X ) )\n( 2 \\pi { \\bar { N } } \\dot { \\xi } ^ { 2 } ( s ) ) ^ { - 1 } { \\cal D } ^ { \\mu } ( s ) F _ { \\mu } [ \\xi | s ] = - [ { \\cal D } ^ { \\mu } ( s ) , { \\cal D } ^ { \\nu } ( s ) ] L _ { \\mu \\nu } [ \\xi | s ] .\nx ^ { D } R ^ { A } { } _ { B C \\bar { D } } = - g ^ { A E } g _ { E B C } \\ .\nd s ^ { 2 } = ( d x ^ { 0 } ) ^ { 2 } + ( x ^ { 0 } ) ^ { 2 } d \\Omega ^ { 2 } , \\quad 0 \\le x ^ { 0 } \\le r ,\nU _ { \\mathrm { A P } } ^ { \\mathrm { b f } } ( \\vec { x } ; C ) = e ^ { - \\mathrm { i } { \\frac { \\theta } { 2 } } \\tau ^ { 3 } } U _ { \\mathrm { A P } } ^ { \\mathrm { b f } } ( D ( e ^ { \\mathrm { i } { \\frac { \\theta } { 2 } } \\tau ^ { 3 } } ) \\cdot \\vec { x } ; e ^ { \\mathrm { i } { \\frac { \\theta } { 2 } } \\tau ^ { 3 } } C e ^ { - \\mathrm { i } { \\frac { \\theta } { 2 } } \\tau ^ { 3 } } ) e ^ { \\mathrm { i } { \\frac { \\theta } { 2 } } \\tau ^ { 3 } }\n\\Delta E _ { \\mathrm { u p } } = \\frac { 1 } { T ^ { ( 0 ) } } \\frac { 4 e ^ { S ^ { ( 0 ) } } \\operatorname { t a n } ^ { 2 } \\frac { 1 } { 2 } ( \\phi _ { \\alpha } + \\phi _ { \\gamma } ) + 4 e ^ { S ^ { ( 0 ) } } } { 4 e ^ { 2 S ^ { ( 0 ) } } \\operatorname { t a n } ^ { 2 } \\frac { 1 } { 2 } ( \\phi _ { \\alpha } + \\phi _ { \\gamma } ) - 1 } ,\nq _ { i + 1 } = { \\frac { 1 } { r _ { i } } } = \\operatorname { e x p } - x _ { i } , \\quad r _ { i + 1 } = r _ { i } [ q _ { i } r _ { i } - \\ddot { \\mathrm { l n } r _ { i } } ]\ny ^ { 2 } = ( - 1 ) ^ { 2 S } ( x - x ^ { g } ) + u ^ { 2 }\n\\sum _ { n _ { 1 } = - \\infty } ^ { \\infty \\prime } { \\frac { e ^ { i \\pi n _ { 1 } ( \\alpha - 2 \\sigma ) } } { n _ { 1 } \\mathrm { S i n } ( \\pi n _ { 1 } \\alpha ) } } \\quad .\n\\left( \\frac { \\sqrt { 2 } { \\bf S } \\cdot { \\bf A } _ { 0 } } { k } \\right) \\phi _ { E \\, \\pm k \\, m } = - \\left( E + \\frac { 1 } { 2 k ^ { 2 } } \\right) ^ { \\mathrm { ~ \\frac { 1 } { 2 } ~ } } \\phi _ { E \\, \\mp k \\, m } .\n\\left. \\frac { \\partial } { \\partial \\zeta _ { N } } T ( \\zeta _ { 1 } , \\ldots , \\zeta _ { N } ) \\right| _ { \\zeta _ { 1 } , \\ldots , \\zeta _ { N } = 1 } \\propto H _ { \\mathrm { \\scriptsize ~ X Y Z } } + \\mathrm { c o n s t . } .\n\\{ \\bar { \\mathcal V } ( x ) , \\Phi ( x ^ { \\prime } ) \\} = ( \\Phi ( x ) + ( \\bar { h } - 1 ) \\Phi ( x ^ { \\prime } ) ) \\, \\delta ^ { \\prime } ( x - x ^ { \\prime } ) + \\ldots .\n\\delta z ^ { M } e _ { M } ^ { \\alpha q } \\equiv \\kappa ^ { \\underline { \\beta } } ( z ) v _ { \\underline { \\beta } } ^ { ~ \\alpha q } .\nM _ { N } ^ { 2 } ( a ) = m _ { 0 } ^ { 2 } + \\frac { ( n _ { 1 } - a _ { 1 } ) ^ { 2 } } { L _ { 1 } ^ { 2 } } + \\frac { ( n _ { 2 } - a _ { 2 } ) ^ { 2 } } { L _ { 2 } ^ { 2 } } ,\nS _ { \\mathrm { Y M } } = - { \\frac { 1 } { 4 e ^ { 2 } } } \\int _ { \\cal M } d ^ { 2 } x \\sqrt { g } g ^ { a c } g ^ { b d } \\mathrm { T r } ( F _ { a b } F _ { c d } ) ,\n\\pi ( x ) = { \\frac { \\partial { \\cal L } _ { \\sigma } } { \\partial \\partial _ { + } \\psi ( x ) } } = i \\bar { \\psi } \\gamma ^ { + } = \\left( \\begin{array} { l l } { \\psi _ { R } ^ { * } } & { \\psi _ { L } ^ { * } } \\\\ \\end{array} \\right) \\left( \\begin{array} { l l } { i \\sqrt { 1 + s } } & { 0 } \\\\ { 0 } & { i \\sqrt { 1 - s } } \\\\ \\end{array} \\right) .\n\\sum _ { i , j , k , l = 1 } ^ { h } a _ { i j } ^ { p } A _ { i k } A _ { j l } \\overline { a } _ { k l } ^ { q } = \\sum _ { i , j , k , l , m , n = 1 } ^ { h } b _ { i j m } ^ { p } A _ { i k } A _ { j l } A _ { m n } \\overline { b } _ { k l n } ^ { q } .\n\\left( ( p + k ) \\: { \\cal { B } } \\: C _ { n - 1 } \\: { \\cal { B } } \\cdots { \\cal { B } } \\: C _ { 1 } \\: { \\cal { B } } \\: ( p - k ) \\right) ( q , y ) \\; = \\; ( p + k ) ( q ) \\: F _ { n } ( q ) \\; \\; \\; .\n\\partial _ { \\alpha } Y ^ { 2 5 } = \\epsilon _ { \\alpha \\beta } \\partial ^ { \\beta } X ^ { 2 5 } .\ny = \\left( \\begin{array} { c c c c } { 0 } & { q _ { 1 } } & { 0 } & { 0 } \\\\ { 0 } & { 0 } & { q _ { 2 } } & { 0 } \\\\ { 0 } & { 0 } & { 0 } & { q _ { 3 } } \\\\ { q _ { 4 } } & { 0 } & { 0 } & { 0 } \\\\ \\end{array} \\right)\n\\frac { 1 } { 2 } \\epsilon _ { \\kappa \\lambda \\sigma \\alpha } \\partial ^ { \\kappa } \\omega ^ { \\sigma \\lambda } \\equiv D _ { \\alpha } = \\left( \\partial _ { \\alpha } h _ { \\beta } ^ { \\beta } - \\partial _ { \\beta } h _ { \\alpha } ^ { \\beta } \\right) \\ .\nQ _ { 0 } ^ { R , L } = \\frac { 1 } { 2 \\pi i } \\int _ { C ^ { R } , C ^ { L } } d z j ( z ) _ { 0 } \\ , R ^ { R , L } = \\frac { 1 } { 2 \\pi i } \\int _ { C ^ { R } , C ^ { L } } d z r ( z ) \\ .\nK = i \\int _ { 0 } ^ { \\infty } d \\lambda ^ { \\prime } \\; s e ^ { i \\lambda ^ { \\prime } ( \\pi ^ { \\prime \\prime } - \\pi ^ { \\prime } ) } e ^ { - i [ s \\lambda ^ { \\prime } ( \\gamma ^ { \\mu } P _ { \\mu } ^ { \\prime } - m ) + { \\frac { i } { s } } ( { \\bar { \\eta } } ^ { \\prime \\prime } - { \\bar { \\eta } } ^ { \\prime } ) ( \\eta ^ { \\prime \\prime } - \\eta ^ { \\prime } ) ] } \\delta ^ { 4 } ( P ^ { \\prime \\prime } - P ^ { \\prime } ) \\, .\n\\epsilon ^ { i j } \\nabla _ { i } ^ { ( p , k ) } { \\cal A } _ { ( p , k ) j } = - \\sum _ { ( p ^ { \\prime } , k ^ { \\prime } ) \\neq ( p , k ) } \\beta _ { p p ^ { \\prime } } \\delta ^ { 2 } ( { \\bf r } _ { k } ^ { ( p ) } - { \\bf r } _ { k ^ { \\prime } } ^ { ( p ^ { \\prime } ) } ) \\, .\n\\delta \\alpha _ { 3 } + \\gamma \\alpha _ { 2 } = \\partial _ { \\mu } \\left( - f _ { a b c } \\eta ^ { * \\mu } \\eta ^ { a } \\eta ^ { b } \\eta ^ { c } \\right) .\n\\psi _ { l } \\left( z \\right) \\psi _ { l } ^ { \\dagger } \\left( w \\right) \\sim \\left( z - w \\right) ^ { - 2 \\Delta _ { l } } \\left[ 1 + \\frac { 2 \\Delta _ { l } } { c _ { p } } \\left( z - w \\right) ^ { 2 } T _ { p } \\left( w \\right) + O \\left( z - w \\right) ^ { 3 } \\right]\nV ( T ) = \\sqrt { 2 } \\tau _ { p } e ^ { - { \\frac { \\kappa T ^ { 2 } } { 2 \\alpha ^ { \\prime } } } } \\ ,\n+ q ( { \\tilde { d } } - 1 ) a _ { 1 } b _ { 1 } + r ( { \\tilde { d } } - 1 ) b _ { 1 } f _ { 1 } + r q f _ { 1 } a _ { 1 } + \\frac { \\phi _ { 1 } ^ { 2 } } { 4 } - \\frac { h ^ { 2 } } { 4 } = 0 ;\nr = \\frac { \\alpha } { \\beta } \\vert \\operatorname { s i n } \\beta \\left( \\sigma - \\sigma _ { 0 } \\right) \\vert\n\\begin{array} { l l } { f ( \\psi ) = 1 + \\frac { 1 } { R ^ { 2 } } \\sum _ { r = 1 } ^ { N } m _ { r } ^ { 2 } ( \\psi _ { r } ) ^ { 2 } + O ( \\psi ^ { 3 } ) } & { \\mathrm { w i t h ~ \\frac { m _ { r } ^ { 2 } } { R ^ { 2 } } ~ \\equiv ~ \\frac { 1 } { 2 } ~ \\partial _ { r } ~ \\partial _ { r } ~ f ~ | _ { \\psi ~ = 0 } ~ } . } \\\\ \\end{array}\ng _ { z \\overline { { z } } } = \\tau _ { 2 } \\eta ^ { 2 } \\overline { { \\eta } } ^ { 2 } \\prod _ { i } ( z - z _ { i } ) ^ { - 1 / 1 2 } \\prod _ { i } ( \\overline { { z } } - \\overline { { z } } _ { i } ) ^ { - 1 / 1 2 } ~ .\n\\Delta ( A ) = A _ { 1 } K A _ { 2 } , \\hspace { 7 m m } \\Delta ( B ) = B _ { 1 } E B _ { 2 } , \\hspace { 7 m m } \\Delta ( C ) = C _ { 1 } Q C _ { 2 }\n\\begin{array} { l l } { \\left| \\ \\! \\pm ; \\pm ; 2 w ; 2 w ; { \\mathbf 2 } , \\ \\ { \\frac { 1 } { 2 } } ; { \\mathbf 1 } , 0 , 0 \\ \\! \\right\\rangle } & { = { \\frac { 1 } { \\sqrt { 2 } } } \\left( \\left| \\ \\! \\underline { { 1 _ { 2 } } } , \\underline { { 0 _ { } } } ; \\underline { { 0 _ { } } } , \\underline { { 0 _ { } } } \\ \\! \\right\\rangle \\pm \\left| \\ \\! \\underline { { 0 _ { } } } , \\underline { { 1 _ { 2 } } } ; \\underline { { 0 _ { } } } , \\underline { { 0 _ { } } } \\ \\! \\right\\rangle \\right) } \\\\ { \\left| \\ \\! \\pm ; \\pm ; 2 w ; 2 w ; { \\mathbf 2 } , - { \\frac { 1 } { 2 } } ; { \\mathbf 1 } , 0 , 0 \\ \\! \\right\\rangle } & { = { \\frac { 1 } { \\sqrt { 2 } } } \\left( \\left| \\ \\! \\underline { { 1 _ { 1 } } } , \\underline { { 0 _ { } } } ; \\underline { { 0 _ { } } } , \\underline { { 0 _ { } } } \\ \\! \\right\\rangle \\pm \\left| \\ \\! \\underline { { 0 _ { } } } , \\underline { { 1 _ { 1 } } } ; \\underline { { 0 _ { } } } , \\underline { { 0 _ { } } } \\ \\! \\right\\rangle \\right) } \\\\ \\end{array} \\hspace { 1 . 0 c m }\n\\tilde { { \\cal K } } = \\frac { ( D - 2 ) { \\cal K } } { D - 2 - { \\cal K } } \\ ,\nS ( x ) = { \\cal S } _ { 0 } ( x ) \\operatorname { e x p } \\left[ - i e ^ { 2 } \\beta ( x ) \\right] \\; ,\n\\begin{array} { r l } { d s ^ { 2 } = \\alpha ^ { \\prime } g _ { 6 } \\sqrt { N _ { 1 } N _ { 5 } } [ } & { u ^ { 2 } ( d x _ { 0 } ^ { 2 } + d x _ { 1 } ^ { 2 } ) + { \\frac { d u ^ { 2 } } { u ^ { 2 } } } + d \\Omega _ { 3 } ^ { 2 } } \\\\ { \\nonumber + } & { \\beta ( N _ { 1 } , N _ { 5 } ) ( d x _ { 2 } ^ { 2 } + . . . + d x _ { 5 } ^ { 2 } ) ] , } \\\\ \\end{array}\n\\widetilde { \\Gamma } _ { k p } \\ = \\ \\frac { 1 } { \\mu _ { k } { - } \\bar { \\mu } _ { p } } \\, T _ { k } ^ { \\dagger } \\, T _ { p } \\quad , \\qquad \\textrm { i . e . } \\qquad \\sum _ { k = 1 } ^ { m } \\, \\Gamma ^ { \\ell k } \\, \\widetilde { \\Gamma } _ { k p } \\ = \\ \\delta _ { \\ p } ^ { \\ell } \\quad .\n\\psi \\longrightarrow e ^ { i \\theta \\gamma _ { 5 } } \\psi \\, .\n\\zeta _ { \\pm } ^ { \\prime } ( 0 | K _ { 0 } ) = \\frac { 1 } { 2 } \\zeta ^ { \\prime } ( 0 | L ) \\pm \\frac { i \\pi } { 2 } \\eta ( 0 | K _ { 0 } ) \\, .\n\n\\widehat { X } = X _ { + } \\otimes \\sigma _ { 3 } + X _ { - } \\otimes i \\sigma _ { 2 } ,\nF \\equiv \\frac { 1 } { 4 } F ^ { a b } F _ { a b } =\nD _ { 1 } \\phi + i \\sigma _ { 1 } D _ { 2 } \\phi = 0 \\, , \\hspace { 5 m m } D _ { 1 } \\chi + i \\sigma _ { 2 } D _ { 2 } \\chi = 0 \\, ,\nz _ { 1 } ^ { P _ { 1 } + 2 } + \\dots + z _ { P _ { s } } ^ { P _ { s } + 2 } = 0 ~ ,\nx \\times y - y \\times x - F ( x , y ) \\; , \\quad x , y \\in E \\; .\ng \\, e ^ { { \\vec { \\xi } } . { \\vec { X } } } = e ^ { { \\vec { \\xi } } \\, ^ { \\prime } . { \\vec { X } } } e ^ { { \\vec { u } } \\, ^ { \\prime } . { \\vec { T } } } ,\nT _ { \\bar { \\imath } \\bar { \\jmath } k } = T _ { i \\bar { \\jmath } \\bar { k } } = T _ { \\bar { \\imath } j \\bar { k } } = m \\epsilon _ { i j k } \\ .\n\\phi ^ { ( 1 ) } ( a , b , c | u , v , w + 1 / 4 ) = C ( u , v , w ) \\, f ( u , v , w ) \\, W ( a , b | A _ { - 1 } ) \\, \\overline { { W } } ( b , c | B _ { - 1 } ) ,\n{ \\cal D } _ { b } \\psi ^ { a b } = 0 , \\quad \\quad \\psi ^ { a b } = \\psi ^ { b a } ,\n\\{ \\phi , \\psi \\} = \\{ \\phi , \\psi \\} _ { \\mathrm { K i r } } + \\partial ^ { i } \\phi L _ { i } \\psi - \\partial ^ { i } \\psi L _ { i } \\phi , ~ ~ ~ \\phi , \\psi \\in \\mathrm { F u n } ( T ^ { * } G ) .\n[ \\partial _ { x } - \\sum _ { s = 1 } ^ { m _ { 1 } } A ^ { - s } , \\partial _ { y } - ( \\rho h ) - \\sum _ { s = 1 } ^ { m _ { 2 } } A ^ { + s } ] = 0\n\\vec { E } ( \\tau ) = \\left[ { \\frac { d ^ { 2 } } { d \\tau ^ { 2 } } } - ( 2 / \\tau ) { \\frac { d } { d \\tau } } \\right] \\vec { G } ( \\tau )\nA ( b , \\theta ) = A ( b , \\theta ) ^ { \\geq 0 } + A ( b , \\theta ) ^ { < 0 } .\n\\nabla _ { \\nu } \\nabla ^ { \\nu } A _ { \\mu } = - I _ { A \\mu } = 4 e \\Phi ^ { 2 } ( \\chi _ { A , \\mu } + e A _ { \\mu } ) .\n( 1 + \\theta + \\omega + \\theta \\omega ) ( 1 + \\alpha \\Omega ^ { \\prime } )\n\\operatorname { l o g } \\, Z \\left( \\beta \\right) = \\frac { f \\left( 0 \\right) } { 2 } - I \\left( 0 \\right) - 2 \\sum _ { n = 1 } ^ { \\infty } I \\left( n \\right) ,\nH ( \\omega , 0 ) = \\frac { \\phi _ { 1 } f _ { 2 } \\partial _ { r } ( \\phi _ { 2 } ^ { ( - ) } / A _ { - } ) - ( \\phi _ { 2 } ^ { ( - ) } / A _ { - } ) f _ { 1 } \\partial _ { r } \\phi _ { 1 } } { \\phi _ { 1 } f _ { 2 } \\partial _ { r } ( \\phi _ { 2 } ^ { ( + ) } / A _ { + } ) - ( \\phi _ { 2 } ^ { ( + ) } / A _ { + } ) f _ { 1 } \\partial _ { r } \\phi _ { 1 } } \\ .\n\\frac { d \\phi ^ { i } } { d t } = - g ^ { i j } \\frac { \\partial W } { \\partial \\phi ^ { j } } \\ ,\nL _ { 1 } = \\mathcal { \\vartheta } _ { 3 } ^ { 4 } ( 0 | \\theta ) - \\mathcal { \\vartheta }\np _ { \\mu } ^ { i } \\left( \\kappa \\right) = P _ { \\mu } \\, \\, v ^ { i } \\left( \\kappa \\right) + \\varepsilon _ { a b } W _ { \\mu } ^ { a } \\, \\lambda ^ { i b } \\left( \\kappa \\right)\nI _ { 1 } ( y , \\mp r ) = \\int _ { 0 } ^ { \\infty } \\frac { d x } { \\sqrt { x ^ { 2 } + y ^ { 2 } } } \\frac { 1 } { \\operatorname { e x p } ( \\sqrt { x ^ { 2 } + y ^ { 2 } } \\mp r ) + 1 } ,\nr \\rightarrow r _ { 0 } , f \\rightarrow 1 + s i n h ^ { 2 } \\gamma = c o s h ^ { 2 } \\gamma \\equiv \\lambda ^ { 3 }\n\\gamma _ { 1 } = \\left\\langle \\sum _ { X _ { i } \\in ( { \\it G } _ { 1 } ) _ { X _ { i ^ { \\prime } } ^ { \\prime } } } \\sum _ { \\stackrel { i _ { \\alpha _ { 1 } } < j _ { \\alpha _ { 1 } } } { \\left\\{ i _ { \\alpha _ { 1 } } , j _ { \\alpha _ { 1 } } \\right\\} \\in J _ { X _ { i } } } } t _ { i _ { \\alpha _ { 1 } } } t _ { j _ { \\alpha _ { 1 } } } { \\bf 1 } _ { ( w ( t _ { i _ { \\alpha _ { 1 } } } ) = w ( t _ { j _ { \\alpha _ { 1 } } } ) ) } \\chi \\left( \\left( i _ { \\alpha _ { 1 } } , j _ { \\alpha _ { 1 } } \\right) \\in i _ { \\alpha _ { 1 } } ^ { \\prime } \\right) \\right\\rangle _ { w ^ { \\prime } } ^ { l . c . } \\; \\mathrm { ~ a n d }\nq _ { j } ( x _ { 0 } ) = A _ { j + 1 } , \\quad q _ { j } ^ { \\prime } ( x _ { 0 } ) = B _ { j + 1 } , \\quad j = 0 , 1 , \\ldots , n - 1 .\n{ \\frac { 1 } { 2 \\pi } } \\sum _ { l = 1 / 2 } ^ { \\infty } 2 l e ^ { - { \\frac { l ^ { 2 } } { \\rho ^ { 2 } } } t } = { \\frac { \\rho ^ { 2 } } { 4 \\pi t } } + { \\frac { \\rho ^ { 2 } } { ( 4 \\pi t ) ^ { \\frac { 3 } { 2 } } } } \\mathrm { P } \\int _ { - \\infty } ^ { \\infty } d x ( { \\frac { x } { 2 \\rho } } \\mathrm { c o s e c } { \\frac { x } { 2 \\rho } } - 1 ) e ^ { - { \\frac { x ^ { 2 } } { 4 t } } } ,\n\\partial ^ { 2 } \\bar { \\Psi } ^ { \\alpha } - i g \\; \\partial _ { x + i y } \\Phi \\; \\partial _ { 0 - z } \\bar { \\Psi } ^ { \\alpha } + i g \\; \\partial _ { 0 - z } \\Phi \\; \\partial _ { x + i y } \\bar { \\Psi } ^ { \\alpha } = 0 \\, ,\nQ = \\frac { 1 } { 2 \\pi } 2 \\pi \\theta \\sum _ { k = 0 } ^ { \\infty } \\left\\langle k \\right| \\left( - \\theta ^ { - 1 } n \\left| 0 \\right\\rangle \\left\\langle 0 \\right| \\right) \\left| k \\right\\rangle = - n ,\n\n[ \\: x ^ { \\lambda } , \\eta _ { \\mu \\nu } \\, \\dot { x } ^ { \\mu } \\, \\dot { x } ^ { \\nu } \\: ] = 0 .\nm _ { h } = \\frac { g _ { s m } ^ { 2 } | \\psi ( 0 ) | ^ { 2 } } { M _ { G } ^ { 2 } } = \\frac { M _ { G } } { \\sqrt { \\pi } c _ { f } \\alpha _ { s } \\sqrt { N _ { s d } } }\n{ \\bf \\Phi } _ { 1 } ^ { ( n _ { \\rho } , m ) } = { \\bf \\Phi } _ { 1 } ^ { ( n _ { \\rho } , m ) } ( \\rho , k ) = \\left( \\begin{array} { c } { \\phi _ { 1 } ^ { ( n _ { \\rho } , m ) } ( \\rho , k ) } \\\\ { \\phi _ { 2 } ^ { ( n _ { \\rho } , m ) } ( \\rho , k ) } \\\\ \\end{array} \\right)\n\\frac { d \\lambda ^ { \\gamma } } { d u } = { \\tilde { f } } _ { \\gamma } ( u ) \\lambda _ { 1 }\nu _ { 2 } = \\xi _ { 0 } + i \\xi _ { 3 } = \\rho ~ \\frac { e ^ { i \\psi } } { \\sqrt { 1 + z \\bar { z } } }\n\\tilde { h } = \\frac { 1 } { 1 + ( a u ) ^ { 7 - p } } , \\ \\ \\ a ^ { 7 - p } = \\tilde { b } ^ { 2 } / R ^ { 7 - p } , \\ \\ \\ R ^ { 7 - p } = \\frac { 1 } { 2 } ( 2 \\pi ) ^ { 6 - p } \\pi ^ { - ( 7 - p ) / 2 } \\Gamma [ ( 7 - p ) / 2 ] \\hat { g } _ { s } N .\n\\phi ( \\Gamma ) ( m ; p _ { i } ) = \\int _ { 0 } ^ { \\infty } \\frac { F _ { \\Gamma } ( r ; m ; p _ { i } ) } { r } d r ,\nk ^ { 2 } = \\frac { b ^ { 2 } } { 4 } + \\frac { \\pi ^ { 2 } } { y _ { c } ^ { 2 } } n _ { k } ^ { 2 } ~ ,\n\\delta \\hat { \\vec { B } } \\; = \\; d \\hat { \\vec { \\cal N } } \\; \\rightarrow \\; \\delta \\hat { \\vec { B } } \\, = \\, { \\cal D } \\hat { \\vec { \\cal N } } \\, = \\, d \\hat { \\vec { \\cal N } } \\, + \\, \\hat { \\cal E } \\wedge \\hat { \\vec { \\cal N } } \\; ,\n\\partial _ { + } g _ { L } g _ { L } ^ { - 1 } = \\left( \\begin{array} { c c } { v _ { L } \\partial _ { + } u _ { L } + b _ { L } \\partial _ { + } a _ { L } } & { u _ { L } \\partial _ { + } a _ { L } - a _ { L } \\partial _ { + } u _ { L } } \\\\ { b _ { L } \\partial _ { + } v _ { L } - v _ { L } \\partial _ { + } b _ { L } } & { u _ { L } \\partial _ { + } v _ { L } + a _ { L } \\partial _ { + } b _ { L } } \\\\ \\end{array} \\right) ,\n\\langle \\Theta \\rangle = c ~ R ^ { ( 2 ) } + \\beta ^ { i } \\langle V _ { i } \\rangle\ng _ { 2 } \\equiv r + \\frac { \\lambda v ^ { 2 } } { 2 } + \\frac { w v ^ { 4 } } { 2 4 } ,\n\\begin{array} { r c l } { \\psi _ { \\uparrow } ^ { ( + ) } } & { = } & { \\sqrt { \\rho } e ^ { \\gamma _ { 5 } \\pi / 4 } e ^ { - \\gamma _ { 2 1 } M c t / \\hbar } \\, ; } \\\\ { \\psi _ { \\downarrow } ^ { ( + ) } } & { = } & { \\sqrt { \\rho } e ^ { \\gamma _ { 5 } \\pi / 4 } \\gamma _ { 3 1 } e ^ { - \\gamma _ { 2 1 } M c t / \\hbar } \\, ; } \\\\ { \\psi _ { \\uparrow } ^ { ( - ) } } & { = } & { \\sqrt { \\rho } e ^ { \\gamma _ { 5 } 3 \\pi / 4 } e ^ { - \\gamma _ { 2 1 } M c t / \\hbar } ) \\, ; } \\\\ { \\psi _ { \\downarrow } ^ { ( - ) } } & { = } & { \\sqrt { \\rho } e ^ { \\gamma _ { 5 } 3 \\pi / 4 } ( \\gamma _ { 1 2 } e ^ { - \\gamma _ { 2 1 } M c t / \\hbar } ) \\, . } \\\\ \\end{array}\nH = \\frac { 1 } { 2 } \\int ( { \\cal H } + \\bar { \\cal H } ) d x ,\nG _ { \\mathrm { c o m p a c t } } = S O \\left( 2 \\right) _ { E } \\oplus S O \\left( 3 \\right) _ { S } \\oplus S O \\left( 2 \\right) _ { R } \\subset G _ { \\mathrm { e v e n } } .\n\\partial _ { \\mu } ( \\sqrt { g } R ^ { \\mu } ) = - \\frac { 1 } { 3 g ^ { 3 } } \\widetilde { \\beta } ( g ) ( F \\widetilde { F } ) - \\frac { \\tilde { b } ( g ) } { 4 8 \\pi ^ { 2 } } ( B \\widetilde { B } ) + \\frac { \\tilde { c } ( g ) - a ( g ) } { 2 4 \\pi ^ { 2 } } R \\widetilde { R } + \\frac { 5 a ( g ) - 3 \\tilde { c } ( g ) } { 9 \\pi ^ { 2 } } ( V \\widetilde { V } )\n\\tilde { \\cal E } _ { q } [ \\phi ( x ) ] \\sim \\frac { 1 } { 2 } \\left( \\sum _ { \\tilde { E } _ { j } } | \\tilde { E } _ { j } | - \\sum _ { \\tilde { E } _ { j } ^ { 0 } } | \\tilde { E } _ { j } ^ { 0 } | \\right) + \\tilde { \\cal E } _ { c t } [ \\phi ( x ) ]\n\\zeta = \\kappa \\operatorname { c o s } \\theta , \\quad k = \\kappa \\operatorname { s i n } \\theta ,\nS = t r H ^ { 4 } - 2 { \\mu } ^ { 2 } t r H ^ { 2 } + n { \\mu } ^ { 4 } .\nS ^ { - 1 } ( p ) = i \\gamma \\cdot p A ( p ^ { 2 } ) + B ( p ^ { 2 } ) ,\nx ^ { \\mu } = R ^ { \\mu } ( \\tau , \\sigma ) + \\rho ^ { \\alpha } n _ { ( \\alpha ) } ^ { \\mu } ( \\tau , \\sigma ) \\; \\; ,\nT ^ { \\prime i j } = T ^ { i j } + S ^ { i k j } { } _ { ; k } ,\nt ^ { 2 } - v ^ { 2 } t + \\Lambda ^ { 2 } v ^ { 2 } = 0 .\nL = g ^ { - 1 } C g , \\quad L \\in G ^ { * } , \\quad g \\in G ,\nc _ { n } \\equiv t r _ { q } K ^ { n } \\quad , \\quad K c _ { n } = c _ { n } K \\quad .\n{ \\bf e ^ { a } } = 2 ( 2 \\alpha ^ { \\prime } ) ^ { - 1 / 2 } { \\bf w ^ { a } } \\ ,\n\\psi _ { c o v a r } ( p , g ) = F ( L ^ { - 1 } ( p ) g ) \\psi ( p )\nH = \\frac { 1 } { 4 \\pi } \\left( P ^ { 2 } + 4 \\omega ^ { 2 } \\, e ^ { 2 \\gamma Q } \\right)\n{ \\tilde { \\epsilon } } = \\int _ { r _ { h } } ^ { r _ { h } + \\epsilon } d r \\, \\Delta ^ { - \\frac { 1 } { 2 } } = 2 \\epsilon ^ { \\frac { 1 } { 2 } } \\left( \\Delta _ { h } ^ { \\prime } \\right) ^ { - \\frac { 1 } { 2 } } + O ( \\epsilon ^ { \\frac { 3 } { 2 } } ) ,\n0 = \\frac { \\partial \\left\\langle V , V \\right\\rangle } { \\partial \\lambda _ { m } }\nS ( \\mathrm { i } \\pi - \\theta ) = - \\mathrm { i } t a n h ( \\frac { \\theta } { 2 } ) \\; \\; \\; \\bar { S } ( \\theta )\n( - ) ^ { { \\bar { \\alpha } } + { \\bar { \\beta } } + \\bar { \\alpha } \\bar { \\beta } } \\frac { { \\bar { \\vartheta } } [ _ { \\bar { \\beta } } ^ { \\bar { \\alpha } } ] } { { \\bar { \\eta } } } \\rightarrow \\frac { { \\bar { \\vartheta } [ _ { \\bar { \\beta } } ^ { \\bar { \\alpha } } ] } ^ { 1 3 } } { { \\bar { \\eta } } ^ { 1 3 } } .\nR _ { 1 } ^ { \\mathbf { I I B } } = \\frac { R } { \\sqrt { 2 } N _ { 2 } } \\quad \\textrm { a n d } \\quad \\omega _ { n } = \\sqrt { \\mu ^ { 2 } + \\frac { n ^ { 2 } } { l _ { s } ^ { 4 } ( p ^ { + } ) ^ { 2 } } } .\n\\begin{array} { c l } { F ( x , e ) } & { = F ^ { r } ( x , r ) } \\\\ \\end{array}\n\\delta _ { i = j } = \\delta _ { + - } = \\delta _ { - + } = 1 , \\quad \\delta _ { i \\neq j } = \\delta _ { i + } = \\delta _ { i - } = \\delta _ { + + } = \\delta _ { -- } = 0 .\nS _ { \\mathrm { b o u n d a r y } } ^ { \\mathrm { q u i n t i c } } = \\int d x ^ { 0 } \\frac { \\theta } { 4 \\pi r } \\sum _ { i } \\left( \\overline { { \\psi } } _ { + i } \\psi _ { + i } + \\overline { { \\psi } } _ { - i } \\psi _ { - i } \\right)\n\\eta = \\int ^ { X ^ { 0 } } \\frac { d X ^ { 0 } } { R ( X ^ { 0 } ) } ~ ,\nE ^ { f K K } = - { \\cal { F } } \\left[ \\frac { \\sqrt { { \\bf p } ^ { 2 } + a ^ { 2 } k ^ { 2 } x ^ { 2 } } } { 2 } \\operatorname { t a n h } \\left( \\frac { \\beta } { 2 } \\sqrt { { \\bf p } ^ { 2 } + a ^ { 2 } k ^ { 2 } x ^ { 2 } } \\right) \\right] .\nW ^ { \\mathrm { \\scriptsize ~ r e n } } = - \\frac 1 2 \\zeta ^ { D } ( 0 ) ^ { \\prime } - \\operatorname { l o g } ( \\mu ) \\zeta ^ { D } ( 0 ) \\, .\n\\nu _ { - a b } \\equiv n _ { - a b } \\circ \\varphi _ { + } , \\qquad \\eta _ { - a b } \\equiv h _ { - a b } \\circ \\varphi _ { + } .\n\\delta B = { \\frac { 1 } { 2 } } \\delta \\bar { \\theta } \\Gamma _ { M N } \\theta ( d X ^ { M } d X ^ { N } + \\bar { \\theta } \\Gamma ^ { M } d \\theta d X ^ { N } + { \\frac { 1 } { 3 } } \\bar { \\theta } \\Gamma ^ { M } d \\theta \\bar { \\theta } \\Gamma ^ { N } d \\theta )\nT \\equiv \\frac { 1 } { \\sqrt { 1 - y _ { 3 } ^ { 2 } } } \\left( y _ { 2 } { \\bf 1 } _ { 2 } + i { \\bf y } \\cdot \\mathrm { \\boldmath ~ \\sigma ~ } \\right) \\; \\; .\nM _ { i j } \\partial ^ { ( j } A ^ { i ) } = 0 , M _ { i j } \\partial ^ { i } \\partial ^ { j } A _ { k } = 0 .\nN = \\sum _ { n > 0 } \\alpha _ { - n } ^ { i } ( E ) G _ { i j } \\alpha _ { n } ^ { j } ( E ) , \\, \\tilde { N } = \\sum _ { n > 0 } \\tilde { \\alpha } _ { - n } ^ { i } ( E ) G _ { i j } \\tilde { \\alpha } _ { n } ^ { j } ( E )\nc ( i , j , k ) = ( - ) ^ { B ( \\beta _ { i } , \\beta _ { j } ) + B ( \\beta _ { i } , \\beta _ { k } ) + B ( \\beta _ { j } , \\beta _ { k } ) + B ( \\beta _ { 0 } , \\omega ) + 1 }\n\\omega = \\phi ^ { \\alpha } \\cdot \\sum _ { i _ { 1 } < \\cdots < i _ { k } } a _ { i _ { 1 } \\cdots i _ { k } } ( v ) d v ^ { i _ { 1 } } \\wedge \\cdots \\wedge d v ^ { i _ { n } }\ne _ { \\mu } ^ { f } = \\left( { \\frac { 1 } { 2 } } \\left( e _ { \\mu } ^ { [ + 2 ] } + e _ { \\mu } ^ { [ - 2 ] } \\right) , { \\frac { 1 } { 2 } } \\left( e _ { \\mu } ^ { [ + 2 ] } - e _ { \\mu } ^ { [ - 2 ] } \\right) \\right)\nf ( \\mu ) = \\pi - f ( \\pi - \\mu ) .\n\\Sigma _ { \\alpha \\beta } = V ^ { - 1 } ~ \\sigma _ { \\alpha \\beta } V\n{ \\cal H } _ { T } = { \\cal H } _ { 1 } + \\Sigma _ { i = 1 } ^ { 3 } c ^ { i } \\, \\phi _ { i } , \\; \\; \\; \\; \\; { \\cal H } _ { 1 } = - \\frac { 1 } { 2 } r ^ { 2 } q _ { x x } - p \\, q _ { x x }\n{ \\frac { \\partial ^ { 2 } \\varphi } { \\partial \\tau ^ { 2 } } } + { \\frac { \\partial ^ { 2 } \\varphi } { \\partial r ^ { 2 } } } + { \\frac { 2 } { r } } { \\frac { \\partial \\varphi } { \\partial r } } = { \\frac { \\partial U ( \\varphi , T ) } { \\partial \\varphi } } ,\nV _ { \\mu } \\rightarrow \\frac { i } { g } ~ \\theta ( \\mathrm { \\boldmath ~ r ~ } ) ~ \\partial _ { \\mu } ~ \\theta ( \\mathrm { \\boldmath ~ r ~ } ) + \\theta ( \\mathrm { \\boldmath ~ r ~ } ) ~ V _ { \\mu } ~ \\theta ^ { \\dag } ( \\mathrm { \\boldmath ~ r ~ } )\n[ a _ { 1 } , a _ { 2 } , \\cdots ~ , a _ { r } ] \\rightarrow [ a _ { 1 } , a _ { 2 } , \\cdots ~ , ( a _ { j } + 1 ) , 1 , ( a _ { j } + 1 ) , \\cdots ~ , a _ { r } ]\n[ Q _ { i } , \\; Q _ { j } ] _ { + } = \\sum _ { n = 0 } ^ { \\infty } \\; A _ { M _ { 1 } } ^ { \\dagger } \\ldots A _ { M _ { n } } ^ { \\dagger } ( A ^ { \\dagger } [ \\Gamma _ { i } , \\; \\Gamma _ { j } ] _ { + } A ) A _ { M _ { n } } \\ldots A _ { M _ { 1 } } .\n\\beta ( g ) = - g \\frac { b _ { 0 } + b _ { 1 } g + b _ { 2 } g ^ { 2 } } { 1 + a _ { 1 } g + a _ { 2 } g ^ { 2 } }\nA ^ { \\mathrm { T } } = - A , \\qquad M \\eta A + A \\eta M = 0\nJ _ { \\mu } = - { \\frac { i e } { 2 } } \\bigg [ \\Phi ^ { \\ast } D _ { \\mu } \\Phi - \\Phi \\big ( D _ { \\mu } \\Phi \\big ) ^ { \\ast } \\bigg ] \\, .\n\\gamma _ { i } ( \\beta \\sqrt { - \\triangle } ) = \\int _ { 0 } ^ { \\infty } \\! \\! { \\mathrm d } t \\, \\, g _ { i } ( t ) \\left[ \\frac { 2 \\pi } { \\beta { \\sqrt { - \\triangle } } \\, t } \\, \\frac { 1 } { { \\mathrm s h } ( 2 \\pi t / ( \\beta { \\sqrt { - \\triangle } } ) ) } - \\frac { 1 } { t ^ { 2 } } \\right] ,\n\\xi _ { 1 } ^ { m } ( u ) = - \\frac { \\partial \\xi _ { 2 } ^ { 1 } ( u ) } { \\partial x ^ { m } } + \\sum _ { i = 1 } ^ { n } x ^ { i } ( u ) \\frac { \\partial \\xi _ { 1 } ^ { i } ( u ) } { \\partial x ^ { m } } .\n\\sigma _ { \\mathrm { a b s } } ^ { ( 1 , 0 ) } = \\frac { { \\cal A } _ { H } } { 2 } .\n\\psi ( x ) \\mapsto \\psi ^ { \\prime } ( x ) \\equiv e ^ { i \\theta ( x ) } \\psi ( x ) \\ , \\hspace { 2 e x } \\overline { \\psi } ( x ) \\mapsto \\overline { { \\psi ^ { \\prime } } } ( x ) \\equiv \\overline { \\psi } ( x ) e ^ { - i \\theta ( x ) } \\ .\nI ( x ^ { - } , x ^ { + } ) = I ^ { ( 1 ) } ( x ^ { - } , x ^ { + } ) + I ^ { ( 2 ) } ( x ^ { - } , x ^ { + } )\nV ( \\phi ) = { \\frac { \\lambda } { 2 } } \\left( \\phi ^ { a } \\phi ^ { a } - v ^ { 2 } \\right) ^ { 2 } \\ .\n{ \\widetilde \\Phi } ^ { A ^ { \\prime } } ( \\gamma ) \\equiv \\gamma ^ { A ^ { \\prime } C ^ { \\prime } C } \\; { _ { e } n _ { C C ^ { \\prime } } } = 0 \\; .\n\\phi = \\pm \\imath \\operatorname { l o g } \\frac { 2 } { \\beta } \\left( 1 + \\sqrt { 1 - \\frac { \\beta ^ { 2 } } { 4 } } \\right) ,\nN \\approx L ^ { 2 } \\left[ 1 + \\frac { \\gamma \\beta L } { 2 ( 1 - \\gamma L ) ^ { 2 } } \\right] .\n- \\left( \\frac { \\partial } { \\partial \\xi } , \\frac { \\partial } { \\partial \\eta } \\right) \\Gamma ( \\xi , \\eta ) \\left( \\begin{array} { c } { \\frac { \\partial } { \\partial \\xi } } \\\\ { \\frac { \\partial } { \\partial \\eta } } \\\\ \\end{array} \\right)\ni \\frac { \\partial \\psi } { \\partial t } = - i \\mathrm { \\boldmath ~ \\alpha ~ } \\cdot \\nabla \\psi + \\beta m \\psi - q \\mathrm { \\boldmath ~ \\alpha ~ } \\cdot { \\bf A } \\psi + q A _ { 0 } \\psi\nD G _ { R } = 0 , ~ ~ ~ ~ ~ ~ G _ { L } \\overleftarrow { D } = 0\n\\left\\{ - \\sum _ { i = 1 } ^ { 3 } \\frac { \\partial ^ { 2 } } { \\partial x _ { i } ^ { 2 } } + \\sum _ { \\stackrel { i , j = 1 } { i \\neq j } } ^ { 3 } \\left( { \\frac { \\omega ^ { 2 } } { 8 } } \\left( x _ { i } - x _ { j } \\right) ^ { 2 } + { \\frac { 2 \\alpha } { \\left( x _ { i } - x _ { j } \\right) ^ { 2 } } } + \\Omega \\left( \\frac { x _ { i } - x _ { j } } { \\sqrt { 2 } } \\right) \\right) \\right\\} \\psi = E \\psi ~ ,\n\\Pi _ { b } ^ { \\mu \\nu } ( p ) = \\int \\frac { d ^ { 4 } k } { ( 2 \\pi ) ^ { 4 } } t r \\left\\{ \\gamma ^ { \\mu } S ( l ) \\gamma ^ { \\nu } G _ { b } ( l + p ) + \\gamma ^ { \\mu } G _ { b } ( l + p ) \\gamma ^ { \\nu } S ( l ) \\right\\} .\n{ \\frac { \\tau _ { j k } } { c _ { 2 } \\tau \\sp 2 _ { j m } } } =\nM ( E ) = \\left( \\begin{array} { l l } { g - B g ^ { - 1 } B } & { B g ^ { - 1 } } \\\\ { - g ^ { - 1 } B } & { g ^ { - 1 } } \\\\ \\end{array} \\right) ,\nI _ { e f f } ^ { \\mathrm { C . S } } = \\frac { m } { | m | } \\theta ( m ^ { 2 } - \\mu ^ { 2 } ) \\pi W [ A ] ,\n\\delta S _ { W Z N W } = \\frac { k } { 2 \\pi } \\int _ { \\partial M } ( \\partial _ { \\tau } ( U ^ { - 1 } \\partial _ { \\phi } U ) \\delta U .\n\\frac { 1 } { 2 } \\rho \\xi ^ { - 2 } = Y ^ { 2 } \\pm U \\mp V - X ^ { 2 } - Z \\geq 0 .\n{ F } = { \\sum _ { m = 1 / 2 } ^ { \\infty } \\psi _ { - m } \\cdot \\psi _ { m } } - 1 ~ ~ , ~ ~ { G } = { - \\sum _ { m = 1 / 2 } ^ { \\infty } \\left( \\gamma _ { - m } \\beta _ { m } + \\beta _ { - m } \\gamma _ { m } \\right) } ~ ~ .\ne _ { 1 } = \\left| \\begin{array} { c } { \\overrightarrow { e _ { 1 } } } \\\\ { 0 } \\\\ \\end{array} \\right| , \\quad e _ { 2 } = \\left| \\begin{array} { c } { \\overrightarrow { e _ { 2 } } } \\\\ { 0 } \\\\ \\end{array} \\right| , \\quad e _ { 3 } = \\left| \\begin{array} { c } { \\overrightarrow { e _ { 3 } } } \\\\ { 0 } \\\\ \\end{array} \\right| , \\quad e _ { 4 } = \\left| \\begin{array} { c } { 0 } \\\\ { 1 } \\\\ \\end{array} \\right| .\n\\frac { d ( T _ { \\eta } ^ { \\eta } c ( \\eta ) } { d \\eta } = \\frac { \\dot { c } ( \\eta ) } { 2 } T _ { \\beta } ^ { \\beta } .\nR _ { \\pm } ( d T ) _ { 1 } ( d T ) _ { 2 } = - ( d T ) _ { 2 } ( d T ) _ { 1 } R _ { \\mp } .\nI _ { m n } = \\int d \\bar { z } d z \\ e _ { q } ^ { - \\bar { z } z } z ^ { n } \\bar { z } ^ { m } = \\delta _ { m n } [ n ] ! \\ ,\n( \\gamma / \\nu d _ { H } ) _ { e f f } \\equiv \\operatorname { l n } \\left( { \\frac { \\chi _ { 2 N } } { \\chi _ { N } } } \\right) / \\operatorname { l n } 2 .\nF = S T U + f ( T , U ) + f ^ { n o n - p e r t } .\n[ A ( m ) , \\bar { A } ( m ^ { \\prime } ) ] = \\delta _ { m m ^ { \\prime } } \\ \\ \\ \\mathrm { a n d \\ \\ o t h e r s } = 0 .\n\\Lambda ^ { 0 } = ( \\partial _ { i } F ^ { i o } - \\rho ) \\dot { { \\bar { c } } } - b \\nabla ^ { 2 } { \\bar { c } } ,\n\\Psi _ { F Q H E } ^ { m } ( Z ) = \\prod _ { i < j } ^ { N } ( Z _ { i } - Z _ { j } ) ^ { m } \\operatorname { e x p } \\{ - \\sum _ { k = 1 } ^ { N } | Z _ { k } | ^ { 2 } \\} ,\n\\mathbf { \\gamma } _ { b } ^ { a } = - ( \\alpha _ { b c } ^ { a } \\mathbf { \\omega } ^ { c } + \\beta _ { b } ^ { a c } \\mathbf { \\omega } _ { c } )\n\\xi ^ { ( i ) } ( x _ { + } ) \\equiv \\langle \\lambda ^ { i } | T _ { L } , { } ~ ~ ~ ~ \\bar { \\xi } ^ { ( i ) } ( x _ { - } ) \\equiv T _ { R } ^ { - 1 } | \\lambda ^ { i } \\rangle\n\\operatorname* { l i m } _ { q \\to 1 } { \\frac { 1 } { 2 } } \\Big [ Z ^ { a | a } ( L ; q ) - Z ^ { a } ( L ; q ) \\Big ] = \\left\\{ \\begin{array} { l l } { \\frac { 1 } { 2 } \\left( 2 ^ { L - 1 } - 2 ^ { \\lfloor \\frac { L } { 2 } \\rfloor } \\right) , } & { a = 1 o r + } \\\\ { \\frac { 1 } { 2 } \\left( 2 ^ { L } - 2 ^ { \\lfloor \\frac { L + 1 } { 2 } \\rfloor } \\right) ule { 0 in } { 0.25 in } , } & { a = 2 o r F } \\\\ \\end{array} \\right.\nt ^ { t t } \\approx \\ell ^ { 2 } \\delta ^ { - 2 } \\, \\Rightarrow \\, < 0 | T ^ { t t } | 0 > \\approx \\hbar \\ell ^ { 2 } \\delta ^ { - 6 } \\approx \\hbar \\ell ^ { - 1 } ( \\ell - T ) ^ { - 3 } .\nN = \\frac { 1 } { 2 } ( 2 \\times 2 4 - 2 \\times 2 2 - 2 ) = 1 \\ ,\nH _ { \\mu \\nu } = \\mu \\delta _ { \\mu , \\nu } , \\quad E ( \\alpha ) _ { \\mu \\nu } = \\delta _ { \\mu - \\nu , \\alpha } .\n\\Lambda = \\frac { d Y } { d { \\cal G } } \\, { \\cal G } - Y\nY ^ { \\mu } = \\frac { x _ { 1 3 } ^ { \\mu } } { x _ { 1 3 } ^ { 2 } } - \\frac { x _ { 2 3 } ^ { \\mu } } { x _ { 2 3 } ^ { 2 } }\n{ } ~ ~ ~ ~ ~ ~ = \\Bigg \\{ \\sum _ { \\sigma = 0 } ^ { N - 1 } { \\frac { w ( x _ { 3 } , x _ { 1 3 } , x _ { 1 } | \\sigma + e - h ) w ( x _ { 4 } , x _ { 2 4 } , x _ { 2 } | \\sigma + f - b ) s ( \\sigma , b - g ) } { w ( x _ { 4 } , x _ { 1 4 } , x _ { 1 } | \\sigma + e - h ) w ( x _ { 3 } , x _ { 2 3 } , \\omega x _ { 2 } | \\sigma + f - b ) } } \\Bigg \\} _ { 0 } ,\n\\overline { { \\sigma } } _ { 3 } ^ { \\prime } ( s ) = - 4 \\frac { \\mu ( t ) + \\mu ( t ^ { 2 } ) } { \\sqrt { s ( s - 4 ) } } \\, \\Theta ( s - 4 )\nF _ { \\theta \\phi } ^ { I } = k q ^ { I } f ( \\theta ) , \\qquad A _ { \\phi } ^ { I } = k q ^ { I } \\intf ( \\theta ) d \\theta .\nL ^ { d = 4 } = \\frac { 1 } { 4 e ^ { 2 } } \\int \\left( W ^ { 2 } d ^ { 2 } \\theta + \\bar { W } ^ { 2 } d ^ { 2 } \\bar { \\theta } \\right) + \\frac { 1 } { e ^ { 2 } } \\sum _ { r , s , t = 0 } ^ { \\infty } a _ { r s t } \\int d ^ { 4 } \\theta \\left( W ^ { 2 } \\bar { W } ^ { 2 } \\right) ^ { r } X ^ { s } Y ^ { t }\n\\operatorname* { s u p } _ { x } \\frac { | f ( x ) - f ( p ) | } { \\mathrm { d i s t } ( x , p ) } \\leq \\operatorname* { s u p } _ { x } \\operatorname* { l i m } _ { p \\to x } \\frac { | f ( x ) - f ( p ) | } { \\mathrm { d i s t } ( p , q ) } ~ .\n\\operatorname* { d e t } V _ { \\pm } [ \\alpha ] \\operatorname* { d e t } V _ { \\pm } [ \\beta ] = \\operatorname* { d e t } V _ { \\pm } [ \\alpha + \\beta ] \\; ,\n\\epsilon ( t , z , r , \\theta , \\phi ) = e ^ { { \\frac { i } { 2 } } \\Gamma _ { 4 } \\theta } e ^ { { \\frac { 1 } { 2 } } \\Gamma _ { 3 } \\Gamma _ { 4 } \\phi } \\Big ( ( 1 + \\Gamma _ { 0 } \\Gamma _ { 1 } ) r ^ { \\frac { - 1 } { 4 } } - { \\frac { ( t - z ) } { 2 ( Z _ { m } ) _ { \\mathrm { c r } } ^ { \\frac { 3 } { 2 } } } } r ^ { \\frac { 1 } { 4 } } ( \\Gamma _ { 0 } - \\Gamma _ { 1 } ) \\Gamma _ { 2 } \\Big ) \\kappa _ { 0 } .\nJ ^ { M } = - \\frac { 1 } { \\sqrt { g _ { 5 } } } \\frac { \\delta S _ { b u l k } } { \\delta \\partial _ { M } \\phi } .\n\\left. { \\delta I / { \\delta \\phi } } \\right| _ { \\phi = \\phi _ { 0 } } = 0 ~ ~ ~ .\n( z _ { 1 } , { \\cal Z } _ { 1 } , { \\cal F } _ { 1 } ) \\times ( z _ { 2 } , { \\cal Z } _ { 2 } , { \\cal F } _ { 2 } ) = ( z _ { 1 } z _ { 2 } , \\, z _ { 1 } { \\cal Z } _ { 2 } + z _ { 2 } { \\cal Z } _ { 1 } , \\, z _ { 1 } { \\cal F } _ { 2 } + z _ { 2 } { \\cal F } _ { 1 } - 2 { \\cal Z } _ { 1 } { \\cal Z } _ { 2 } ) .\nb _ { 0 } = b _ { 0 , r e n } + { \\frac { m ^ { 2 } ( 1 / 6 - \\xi ) } { 1 6 \\pi ^ { 2 } \\epsilon } }\n\\dot { G } _ { B } ( \\tau , \\tau ) = 0 , \\quad \\dot { G } _ { B } ^ { 2 } ( \\tau , \\tau ) = 1\nh _ { \\ell } = - \\frac { d ^ { 2 } } { d r ^ { 2 } } + \\frac { \\ell ( \\ell + 1 ) } { r ^ { 2 } } + \\frac { \\gamma } { r }\nE \\rightarrow E ^ { \\prime } = ( \\alpha E + \\beta ) ( \\gamma E + \\delta ) ^ { - 1 } ~ .\n\\dot { \\phi } _ { i } ~ = ~ g _ { i } ( \\phi _ { i } , { \\frac { \\delta S } { \\delta \\phi _ { i } } } , \\rho ) ~ = ~ f _ { i } ( \\phi _ { i } , \\rho ) .\nW = \\prod _ { a = 1 } ^ { p - 1 } \\left( \\frac { n ^ { \\frac { 1 } { l } } } { L _ { a } } \\right) \\int _ { 0 } ^ { \\infty } \\frac { d s } { s } \\left( \\frac { \\pi } { 2 s } \\right) ^ { \\frac { p - 1 } { 2 } } e ^ { - r ^ { 2 } s } \\frac { 4 \\operatorname { s i n h } ^ { 2 } ( \\omega _ { l } s \\operatorname { s i n } { \\frac { \\phi } { 2 } } ) - \\operatorname { s i n h } ^ { 2 } ( 2 \\omega _ { l } s \\operatorname { s i n } { \\frac { \\phi } { 2 } } ) } { \\operatorname { c o s } { \\frac { \\phi } { 2 } } ~ \\operatorname { s i n h } ( 2 \\omega _ { l } s \\operatorname { s i n } { \\frac { \\phi } { 2 } } ) } \\, .\n\\hat { \\theta } = \\hat { \\theta } ( x , y , \\tilde { x } , \\tilde { y } ) = \\hat { \\theta } ^ { + } .\nW _ { p , N _ { r } } ^ { ( o ) } ( t ) = ( - 1 ) ^ { e ( N _ { r } ) } \\sum _ { N _ { o } , N _ { o } ^ { \\prime } } \\left[ E _ { p , N _ { o } } ^ { ( - 2 ) } \\left( Y _ { b , N _ { r } } ( t ) \\right) + \\sum _ { N } [ \\partial q _ { N } Y _ { p , N _ { r } } ( t ) ] B _ { N N _ { o } } \\right] A _ { N _ { o } N _ { o } ^ { \\prime } } ^ { - 1 } \\chi _ { N _ { o } ^ { \\prime } } ( t )\n\\Omega _ { U V } \\Omega _ { V W } \\Omega _ { W U } = \\mathrm { i d }\n\\left\\{ Q _ { n } ^ { - } , Q _ { n } ^ { + } \\right\\} _ { P B } = - i ( H _ { n } ) ^ { n } , \\qquad \\{ Q _ { n } ^ { \\pm } , H _ { n } \\} _ { P B } = 0 .\nL = \\sum _ { i = 1 } ^ { N } [ { \\frac { 1 } { 2 } } ( P _ { i } ^ { 2 } + { e ^ { - ( Q _ { i + 1 } - Q _ { i } ) } } ) { \\dot { Q } } _ { i } + \\pi _ { i } ( P ) { \\dot { P } } _ { i } ] - H ( Q , P )\n( 1 - P ) \\, a \\, T \\ = \\ 0 \\quad ,\nx ^ { 0 } ( \\tau , \\sigma ) = \\frac M L \\tau , \\quad \\quad x ^ { 1 } ( \\tau , \\sigma ) = q + \\frac M { 2 L } \\, [ f ( \\tau + \\sigma ) + g ( \\tau - \\sigma ) ] ,\n| D \\rangle _ { 0 } = { \\frac { 1 } { \\sqrt { M } } } | { \\widetilde D } \\rangle _ { 0 } ~ .\n\\overline { { \\Delta _ { A Y ^ { \\prime } } ^ { a } } } = \\epsilon ^ { A B } \\epsilon ^ { Y ^ { \\prime } Z ^ { \\prime } } \\Delta _ { B Z ^ { \\prime } } ^ { a } \\; .\n\\psi \\left( x \\right) = \\psi _ { o u t } \\left( x \\right) + \\int S _ { A } \\left( x - y \\right) j \\left( y \\right) d ^ { 4 } y\nf _ { g } ( \\mu ) = \\frac { 1 + ( 1 - g ) \\mu } { 1 - g \\mu } \\, ,\n\\Delta a _ { j } ^ { i } ( x , y , z ) = a _ { j } ^ { i } ( x , y , z ) = \\sum _ { k , l } a _ { k } ^ { i } ( x ) a _ { l } ^ { k } ( y ) a _ { j } ^ { l } ( z )\n\\phi _ { n } = \\phi _ { n } ^ { \\left( 0 \\right) } - T \\sum _ { m = - \\infty } ^ { \\infty } \\Delta _ { F } ^ { n - m }\ne _ { \\mu } ^ { a } \\, e _ { \\nu } ^ { b } \\, \\hat { \\xi } _ { a b } \\, e ^ { c \\, \\mu } \\, e ^ { d \\, \\nu } \\, \\hat { \\xi } _ { c d } = \\Omega ^ { - 1 } \\, \\hat { \\eta } _ { \\mu \\nu } \\, \\hat { \\eta } ^ { \\mu \\nu } \\, \\Omega .\n\\partial _ { a } \\hat { X } ^ { - } = 0 , \\hspace { 1 . 0 c m } \\partial _ { a } \\hat { X } ^ { M } = 0\n\\kappa = \\sqrt { { \\frac { g ( g + 1 ) } { R ^ { 2 } } } + R ^ { 2 } - ( 2 \\epsilon + 2 g - 1 ) } \\approx { \\frac { \\sqrt { g ( g + 1 ) } } { R } }\n\\Gamma [ W , \\bar { W } , q ^ { + } ] = S [ V ^ { + + } , q ^ { + } ] + \\bar { \\Gamma } [ W , \\bar { W } , q ^ { + } ] ,\n{ \\gamma } _ { 1 1 } = { \\gamma } _ { 2 2 } = e ^ { \\Psi } ~ , ~ ~ ~ ~ { \\gamma } _ { 3 3 } = 1 ,\n\\delta _ { l } ^ { ( 2 ) } \\approx \\eta \\left[ \\operatorname { l n } \\left( \\frac { z _ { l } } \\alpha \\right) + \\frac 1 { 2 4 z _ { l } ^ { 2 } } - \\frac { q ^ { 2 } } { 2 z _ { l } ^ { 2 } } + O \\left( \\frac 1 { z _ { l } ^ { 4 } } \\right) \\right] + O ( \\eta ^ { 3 } ) \\ .\nI \\propto \\epsilon _ { \\{ n \\} } \\epsilon _ { \\{ m \\} } \\prod _ { i } \\left\\langle P _ { n _ { i } } ( x _ { i } ) Q _ { m _ { i } } ( y _ { i } ) \\right\\rangle = N ! \\prod _ { n = 1 } ^ { N } h _ { n }\n[ \\Delta _ { \\perp } ^ { d } ] ^ { - 1 } ( x _ { \\perp } ) = - \\int \\frac { d ^ { d } k _ { \\perp } } { ( 2 \\pi ) ^ { d } } \\frac { e ^ { i k _ { \\perp } \\cdot x _ { \\perp } } } { k _ { \\perp } ^ { 2 } } = - \\frac { 1 } { 2 ^ { 2 - d / / 2 } } \\frac { 1 } { ( 2 \\pi ) ^ { d / 2 } } \\frac { 1 } { ( x _ { \\perp } ) ^ { d / 2 - 1 } } \\Gamma ( d / 2 - 1 )\n\\langle \\Theta ( x ) \\ \\Theta ( y ) \\rangle = { \\frac { 2 ^ { 1 3 } \\cdot 3 ^ { 3 } \\cdot 5 } { \\pi ^ { 3 } } } \\, \\, \\frac { \\beta ^ { 2 } [ \\lambda ( t ) ] f [ \\lambda ( t ) ] } { | x - y | ^ { 1 2 } } ~ , ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \\mathrm { f o r } ~ ~ x \\neq y ,\nV _ { \\mathrm { s t a t } } ^ { \\mathrm { L R } } = \\frac { 1 } { 2 } \\sigma \\, ( r _ { 1 2 } + r _ { 2 3 } + r _ { 3 1 } )\nQ ( x , \\psi ) = x _ { 1 } ^ { d _ { 1 } } + \\cdots + x _ { d + 1 } ^ { d _ { d + 1 } } - k \\psi \\cdot x _ { 1 } \\cdots x _ { d + 1 } ,\n\\bar { H } _ { \\mathrm { g } } = H _ { \\mathrm { g } } ( q ^ { r } , p ^ { r } ) + \\lambda _ { a } ^ { ( 1 ) r _ { a } } \\pi _ { a } ^ { r _ { a } } + \\lambda _ { a } ^ { ( 2 ) r _ { a } } \\psi _ { a } ^ { r _ { a } } + \\lambda _ { a } ^ { ( 3 ) } \\theta _ { a } + \\lambda _ { a } ^ { ( 4 ) } \\gamma _ { a } ,\n( a _ { i j } ) = \\left( \\begin{array} { l l } { 2 } & { - 1 } \\\\ { - 1 } & { 0 } \\\\ \\end{array} \\right) .\nd s ^ { 2 } = R _ { h } ^ { 2 } \\left[ d \\chi ^ { 2 } + \\operatorname { s i n h } ^ { 2 } ( \\chi ) \\left( d \\alpha ^ { 2 } + \\operatorname { s i n } ^ { 2 } ( \\beta ) d \\beta ^ { 2 } \\right) \\right]\n\\beta ^ { I } = - G ^ { I J } ( \\phi ) { \\frac { \\partial } { \\partial \\phi ^ { J } } } \\, \\operatorname { l o g } ( { a \\, \\langle T \\rangle + U ( \\phi ) } )\n\\nabla ^ { 2 } \\Phi ^ { X } + \\nabla _ { \\mu } \\psi \\nabla ^ { \\mu } \\Phi ^ { X } = \\frac { ( q - 1 ) } { R ^ { 2 } } ( e ^ { - 2 \\Phi ^ { X } } - e ^ { - 2 \\psi } ) ,\n\\Delta _ { + } = - ( \\beta ^ { - 1 } \\partial _ { \\mu } \\beta ) ( \\beta \\partial _ { \\mu } \\beta ^ { - 1 } ) \\quad \\mathrm { a n d } \\quad \\Delta _ { - } = - ( \\beta \\partial _ { \\mu } \\beta ^ { - 1 } ) ( \\beta ^ { - 1 } \\partial _ { \\mu } \\beta ) \\, ,\n\\frac { \\delta a } { a _ { p h y s } } = \\frac { C _ { 2 } ^ { \\infty } } { C _ { 1 } } ,\n{ \\cal A } ^ { 0 } = - \\frac { 1 } { 1 6 \\pi } \\int d ^ { 4 } x F _ { \\mu \\nu } ( x ) F ^ { \\mu \\nu } ( x ) + \\int d ^ { 4 } x { \\bar { \\psi } } ( x ) ( i \\partial _ { \\mu } \\gamma ^ { \\mu } - m ) \\psi ( x ) .\n\\chi ( \\tilde { \\cal M } _ { k } ) = \\sum _ { i = 0 } ^ { 4 ( k - 1 ) } \\ \\sum _ { p = 0 } ^ { k - 1 } \\delta _ { i , 2 k - ( p , q ) } = k\nT r : e x p i ( 2 \\pi F ) = \\sum _ { j = 0 } ^ { M - 1 } [ e x p i ( { \\frac { 2 \\pi j n } { { M } } } ) ] = 0 , \\quad\n{ \\frac { d { Q } } { d \\operatorname { l n } { \\rho } } } = { \\frac { g _ { s } } { 2 \\pi } } \\, { \\widehat C _ { I J } m ^ { I } m ^ { J } } \\propto \\, { \\beta _ { i } \\bar { \\beta } _ { j } \\, G ^ { i j } } ~ ,\nB = \\int _ { 0 } ^ { x _ { \\mathrm { m i n } } } \\rho _ { B } d x + \\int _ { x _ { \\mathrm { m i n } } } ^ { \\infty } \\rho _ { B } d x \\nonumber\nt _ { E Q } \\sim \\frac { M _ { p l } } { T _ { E Q } ^ { 2 } } \\ .\nx _ { \\mu } ~ \\longrightarrow ~ \\frac { x _ { \\mu } } { x ^ { 2 } }\nM e _ { \\nu } ( z , h ) = \\alpha _ { \\nu } ( h ) M _ { \\nu } ^ { ( 1 ) } ( z , h )\n\\lambda _ { l } ^ { n } = \\lambda _ { c } ^ { n } + i [ n + 1 - 2 l + \\frac { \\pi } { 2 \\gamma } ( 1 - v _ { s } v _ { n } ) ]\n\\left[ { \\alpha } _ { m } ^ { \\mu } , { \\alpha } _ { n } ^ { \\nu } \\right] = m { \\delta } _ { m + n } { \\eta } ^ { \\mu \\nu }\n\\Delta _ { n } ( \\epsilon ) g = \\int _ { \\cal C } { \\frac { d t } { 2 \\pi i } } t ^ { - n - 1 } \\delta ( \\epsilon , t ) g ,\n\\gamma B _ { a } ^ { 0 i } = \\partial _ { j } \\eta _ { a } ^ { ( 2 ) i j } , \\; \\gamma B _ { a } ^ { i j } = \\eta _ { a } ^ { ( 1 ) i j } , \\; \\gamma H _ { i } ^ { a } = C _ { i } ^ { ( 1 ) a } , \\; \\gamma H _ { 0 } ^ { a } = - \\partial ^ { i } C _ { i } ^ { ( 2 ) a } ,\nD _ { i } = \\partial _ { i } + e ( \\partial _ { j } A _ { i } ) \\, \\tilde { \\partial } _ { j }\nH ^ { 2 } = - H \\frac { \\dot { d _ { 0 } } } { d _ { 0 } } + \\frac { \\kappa ^ { 2 } ( \\rho _ { 1 } + \\rho _ { 2 } ) } { 6 d _ { 0 } } ,\n\\partial _ { x _ { _ \\perp } } ^ { 2 } H _ { p } ( x _ { _ \\perp } ) = 0 .\n\\psi = \\psi _ { + } + \\psi _ { - } , \\qquad \\psi _ { \\pm } = P _ { \\pm } \\psi , \\qquad P _ { \\pm } = \\frac { 1 } { 4 } \\gamma _ { \\mp } \\gamma _ { \\pm } \\; ,\nF _ { \\alpha \\beta } = \\partial _ { \\alpha } A _ { \\beta } - \\partial _ { \\beta } A _ { \\alpha } + [ A _ { \\alpha } , A _ { \\beta } ] _ { \\star }\n\\Omega ^ { 2 } = ( c X _ { 1 } X _ { 2 } ) ^ { 1 / 4 } / \\rho , \\hspace { 1 c m } c = \\operatorname { c o s h } m\n\\! \\! \\! \\! I \\! = \\! \\int _ { \\Sigma } \\! d ^ { 3 } \\! x \\! \\sqrt { \\gamma } \\left( \\! R ( \\gamma ) \\! - \\! { \\frac { 1 } { 2 } } \\! \\left( \\left( \\! D U \\right) ^ { 2 } \\! - \\! e ^ { - 2 U } \\! \\! \\left( \\! D V \\right) ^ { 2 } \\right) \\! - \\! { \\frac { 1 } { 2 } } \\! \\left( \\left( \\! D \\Phi \\! \\right) ^ { 2 } \\! - \\! e ^ { - 2 \\Phi } \\! \\left( \\! D \\Psi \\! \\right) ^ { 2 } \\right) \\! - \\! \\left( \\! D T \\right) \\! ^ { 2 } \\! + \\! e ^ { - ( U \\! + \\Phi ) } V ( T ) \\! \\right)\n\\begin{array} { l } { { \\cal H } = \\frac { N } { 2 } \\left( \\frac { P ^ { m } P _ { m } } { \\sqrt { \\beta } } + \\frac { \\beta _ { a b } } { \\sqrt { \\beta } } \\Pi _ { r } ^ { a } \\Pi _ { r } ^ { b } + \\sqrt { \\beta } \\beta ^ { a b } \\partial _ { a } X ^ { m } \\partial _ { b } X _ { m } - \\sqrt { \\beta } + \\frac { 1 } { 2 } \\sqrt { \\beta } \\beta ^ { a c } \\beta ^ { b d } F _ { a b } ^ { r } F _ { c d } ^ { r } \\right) } \\\\ { + \\Pi _ { r } ^ { a } \\partial _ { a } A _ { 0 } ^ { r } + N ^ { a } \\left( \\partial _ { a } X ^ { m } P _ { m } + \\Pi _ { r } ^ { b } F _ { a b } ^ { r } \\right) } \\\\ \\end{array}\n{ \\cal T } _ { \\sigma } = { \\cal T } _ { C } ( X ) \\otimes { \\cal T } _ { C } ( Y )\n( \\alpha _ { 0 } ^ { I } ) _ { i j } = \\frac { 1 } { \\pi \\sqrt { 2 \\alpha ^ { \\prime } } } [ X ^ { I } , \\cdot \\ ] _ { i j } \\ .\n\\frac { 1 } { 2 \\pi } \\int _ { - \\infty } ^ { \\infty } \\operatorname { l n } \\left( 1 - e ^ { - a \\sqrt { 1 + k ^ { 2 } } } \\right) \\frac { d \\delta _ { s _ { \\parallel , \\perp } } ( k ) } { d k } d k\n\\bar { \\partial } _ { \\dot { \\alpha } } ^ { + } \\theta ^ { \\prime } { } ^ { \\beta } = 0\nS \\sim \\left( N ^ { 2 + \\frac { 3 } { 5 } } E ^ { 3 } \\right) ^ { \\frac { 1 } { 4 } } \\; .\n[ \\rho ( f \\bigotimes v ) \\varphi ] _ { n } ( { \\bf { x _ { 1 } } } \\sigma _ { 1 } , { \\bf { x _ { 2 } } } \\sigma _ { 2 } , . . . . , { \\bf { x _ { n } } } \\sigma _ { n } ) = \\sum _ { i = 1 } ^ { n } f ( { \\bf { x _ { i } } } ) a ( \\sigma _ { i } ) \\varphi _ { n } ( { \\bf { x _ { 1 } } } \\sigma _ { 1 } , { \\bf { x _ { 2 } } } \\sigma _ { 2 } , . . . . , { \\bf { x _ { n } } } \\sigma _ { n } )\nb \\omega _ { + } = q \\omega _ { + } b , ~ ~ ~ ~ d \\omega _ { + } = q \\omega _ { + } d + \\mu \\omega _ { 0 } b ,\nr _ { + } ^ { 2 } = \\frac { l ^ { 2 } } { 2 } \\left( 1 \\pm \\sqrt { 1 - \\frac { 4 \\tilde { \\alpha } } { l ^ { 2 } } } \\right) ,\n\\nabla ^ { ( H ) } Z _ { A B } = Z _ { I } P _ { A B } ^ { I }\nd \\omega \\equiv \\left[ g ^ { D } ( d X , d X ) \\right] ^ { \\frac { 1 } { 2 } } .\ns _ { a } = 2 \\frac { ( s \\xi ) } { \\zeta ^ { 2 } } \\xi _ { a } + 2 \\frac { ( s \\bar { \\xi } ) } { \\zeta ^ { 2 } } \\bar { \\xi } _ { a } - ( s n ) n _ { a } , \\qquad \\zeta = 1 - z \\bar { z }\n( \\sum _ { i = 1 } ^ { N } a _ { i } \\theta _ { i } ) ^ { p + 1 } = 0 \\; .\nk _ { 2 } ^ { \\mu } [ T _ { \\mu \\nu } ^ { ( 1 ) a b } + T _ { \\mu \\nu } ^ { ( 2 ) a b } ] = - g ^ { 2 } f ^ { a b c } \\overline { { v } }\nj ( x ; z ) \\Phi _ { h } ( y , \\bar { y } ; w , \\bar { w } ) \\sim \\frac { 1 } { z - w } \\left\\{ ( y - x ) ^ { 2 } \\partial _ { y } + 2 h ( y - x ) \\right\\} \\Phi _ { h } ( y , \\bar { y } ; w , \\bar { w } ) ,\n+ \\sum _ { l = t + 1 } ^ { r - 2 } q ^ { \\theta ( l > t + 1 ) \\sum _ { \\nu - r + t + 1 } ^ { \\nu - r + l - 1 } n _ { i } ^ { \\prime } } \\prod _ { \\stackrel { i = 1 } { i \\neq \\nu - r + t } } ^ { \\nu - 2 } \\left[ \\begin{array} { c } { n _ { i } ^ { \\prime } + \\tilde { n } _ { i } - V _ { i , r } + \\theta ( i > \\nu - r + l ) + \\theta ( t > 0 ) \\sum _ { m = 0 } ^ { t - 1 } \\delta _ { i , \\nu - r + m } } \\\\ { n _ { i } ^ { \\prime } - \\delta _ { i , \\nu - r + l } } \\\\ \\end{array} \\right] _ { q } \\times\n{ F } _ { \\mathrm { r e n } } ^ { \\infty } = - \\frac 1 { 8 \\pi ^ { 2 } } \\int \\! { \\mathrm d } ^ { 3 } x \\, { g } ^ { 1 / 2 } { \\mathrm t r } \\Big \\{ R _ { i j } \\gamma _ { 1 } ( - \\triangle ) R ^ { i j } + R \\gamma _ { 2 } ( - \\triangle ) R + { \\mathrm { O } } [ \\Re ^ { 3 } ] \\Big \\} .\n\\partial _ { \\rho } G _ { \\mu \\nu } = \\partial _ { \\rho } G _ { \\mu \\nu } ^ { ( 0 ) } + \\gamma _ { \\mu \\nu } u _ { \\rho } \\theta ( u ) .\nG _ { \\mathrm { \\tiny ~ m u l t i p l e t } } = U ( 1 ) \\times S U ( 2 ) \\times S U ( 2 j + 1 ) _ { f } \\times C P .\nK _ { I } ^ { ( h ) } ( \\theta ) = \\sqrt { \\ell ^ { \\prime } } \\; \\; \\frac { \\textrm { s n } \\left[ \\frac { \\textbf { K } } { i \\pi } \\left( \\theta - i \\frac { \\pi } { 2 } \\right) \\right] } { \\textrm { c n } \\left[ \\frac { \\textbf { K } } { i \\pi } \\left( \\theta - i \\frac { \\pi } { 2 } \\right) \\right] } \\cdot \\frac { 1 - \\frac { h ^ { 2 } } { 2 \\textrm { M } } - \\textrm { s n } \\left( \\frac { i \\textbf { K } } { \\pi } 2 \\theta \\right) } { 1 - \\frac { h ^ { 2 } } { 2 \\textrm { M } } + \\textrm { s n } \\left( \\frac { i \\textbf { K } } { \\pi } 2 \\theta \\right) } \\, , \\qquad \\sqrt { 2 \\textrm { M } } \\leq h < \\infty \\, ,\nd A = \\sqrt { E G - F ^ { 2 } } \\hspace { 0 . 2 c m } d u \\hspace { 0 . 1 c m } d v = | \\operatorname { s i n } \\phi | \\hspace { 0 . 2 c m } d u \\hspace { 0 . 1 c m } d v\nz _ { * } = f ^ { - 1 } ( 1 / r ) = F ( 1 / r ) \\, , \\quad \\alpha _ { * } = r \\, .\nd S \\equiv 2 i \\, d z \\, d \\bar { z } / \\left( 1 + | z | ^ { 2 } \\right) ^ { 2 }\nz = \\frac { 1 } { a } ( w - w _ { 0 } ) - \\frac { b } { a ^ { 3 } } ( w - w _ { 0 } ) ^ { 2 } - \\frac { 1 } { a ^ { 5 } } ( a c - 2 b ^ { 2 } ) ( w - w _ { 0 } ) ^ { 3 } - \\frac { 1 } { a ^ { 7 } } ( a ^ { 2 } d - 5 a b c + 5 b ^ { 3 } ) ( w - w _ { 0 } ) ^ { 4 } + \\ldots\nD _ { a } X _ { \\mu } ( \\delta _ { \\mu \\nu } I - i [ X _ { \\mu } , X _ { \\nu } ] ) ^ { - 1 } D _ { b } X _ { \\nu } = ( I - [ X , Y ] ^ { 2 } ) ^ { - 1 } ( D _ { a } X _ { \\mu } D _ { b } X _ { \\mu } + i D _ { a } X _ { \\mu } [ X _ { \\mu } , X _ { \\nu } ] D _ { b } X _ { \\nu } ) .\n\\stackrel { . } { \\theta } ^ { a } = \\left\\{ \\theta ^ { a } , H _ { 2 } ^ { \\left( 1 \\right) } \\right\\} = 0 ,\n\\alpha _ { w e a k } ^ { u _ { R } } = \\frac { 4 } { 9 } \\alpha _ { e l } \\frac { \\operatorname { s i n } ^ { 2 } \\theta _ { W } } { \\operatorname { c o s } ^ { 2 } \\theta _ { W } } .\nc ^ { a } = \\hat { \\eta } ^ { a } \\ \\ \\ , \\ \\ \\ b _ { a } = \\frac { i } { \\hbar } \\hat { { \\cal P } } _ { a } \\ \\ \\ , \\ \\ \\ a = 1 , 2 \\ \\ \\ ,\n\\left\\langle \\int d ^ { 2 } x : \\Psi ^ { \\dagger } ( x ) M \\Psi ( x ) : \\right\\rangle = - \\frac { 1 } { 2 } \\left( \\sum _ { \\mathrm { o c c u p i e d } } \\int d ^ { 2 } x \\psi _ { E } ^ { \\dagger } ( x ) M \\psi _ { E } ( x ) - \\sum _ { \\mathrm { u n o c c u p i e d } } \\int d ^ { 2 } x \\psi _ { E } ^ { \\dagger } ( x ) M \\psi _ { E } ( x ) \\right) .\n\\begin{array} { l } { \\delta \\theta ^ { \\alpha } = \\epsilon ^ { \\alpha } , \\qquad \\delta \\bar { \\theta } _ { \\alpha } = \\bar { \\epsilon } _ { \\alpha } , } \\\\ { \\delta x ^ { \\bar { \\mu } } = - i \\bar { \\epsilon } _ { \\alpha } \\tilde { \\Gamma } ^ { { \\bar { \\mu } } \\alpha \\beta } \\bar { \\theta } _ { \\beta } + i \\epsilon ^ { \\alpha } \\Gamma _ { \\alpha \\beta } ^ { \\bar { \\mu } } \\theta ^ { \\beta } , \\qquad \\delta x ^ { 1 1 } = 0 , } \\\\ \\end{array}\nu = \\phi _ { A B } \\phi _ { B A } , v = \\phi _ { A C } \\phi _ { C A } , w = \\phi _ { A D } \\phi _ { D A } , t = \\phi _ { A B } \\phi _ { B D } \\phi _ { D A }\nZ _ { \\alpha _ { 1 } , j } ^ { \\alpha _ { 0 } } R _ { \\beta _ { 0 } } ^ { j } + F _ { \\beta _ { 0 } \\gamma _ { 0 } } ^ { \\alpha _ { 0 } } Z _ { \\alpha _ { 1 } } ^ { \\gamma _ { 0 } } = - Z _ { \\gamma _ { 1 } } ^ { \\alpha _ { 0 } } G _ { \\beta _ { 0 } \\alpha _ { 1 } } ^ { \\gamma _ { 1 } } ,\n\\Gamma ^ { \\mu _ { 1 } . . . \\mu _ { k } } \\equiv { \\frac { 1 } { k ! } } \\Gamma ^ { [ \\mu _ { 1 } } . . . \\Gamma ^ { \\mu _ { k } ] }\n\\left( M ^ { T } \\overline { M } \\right) _ { j k } = 2 \\pi D _ { j j } K _ { N } ( \\theta _ { j } , \\theta _ { k } ) \\bar { D } _ { k k }\n\\Psi _ { \\bf D } \\rightarrow \\operatorname { e x p } { ( \\phi _ { 1 } E _ { 1 } - \\theta _ { 2 } E _ { 2 } + \\phi _ { 3 } E _ { 3 } + \\theta _ { 1 } F _ { 1 } + \\phi _ { 2 } F _ { 2 } + \\theta _ { 3 } F _ { 3 } ) } \\cdot \\Psi _ { \\bf D } ,\n\\Delta ( \\varphi ) = \\varphi + \\tilde { \\varphi }\nF _ { 1 2 } = 2 \\pi i / ( L _ { 1 } L _ { 2 } ) \\tau _ { 3 } \\, , \\, F _ { 3 4 } = 2 \\pi i / ( L _ { 3 } L _ { 4 } ) \\tau _ { 3 }\n\\left( d - { \\textstyle \\frac { 1 } { 4 } } \\omega _ { a b } \\gamma ^ { a b } + \\Omega \\right) \\kappa = 0 \\, .\nF _ { \\mu _ { 1 } \\mu _ { 2 } \\mu _ { 3 } \\mu _ { 4 } } = e \\, \\epsilon _ { \\mu _ { 1 } \\mu _ { 2 } \\mu _ { 3 } \\mu _ { 4 } }\nw _ { R } ( E ) \\simeq { \\frac { 1 } { \\sigma _ { R } \\sqrt { \\pi } } } e ^ { - { \\frac { ( E - E _ { R } ) ^ { 2 } } { \\sigma _ { R } ^ { 2 } } } } ~ ~ ~ .\n{ \\cal H } = { \\frac { 1 } { 2 } } ( \\pi _ { \\phi } - e A _ { 1 } ) ^ { 2 } + { \\frac { 1 } { 2 } } \\pi _ { 1 } ^ { 2 } + { \\frac { 1 } { 2 } } \\phi ^ { 2 } + \\pi _ { 1 } A _ { 0 } ^ { \\prime } + e \\phi ^ { \\prime } A _ { 0 } + { \\frac { 1 } { 2 } } a e ^ { 2 } ( A _ { 0 } ^ { 2 } - A _ { 1 } ^ { 2 } ) .\n\\sum _ { i = 1 } ^ { 3 } T r ( \\gamma _ { \\theta , 7 _ { i } } ) + 3 T r ( \\gamma _ { \\theta , 3 } ) = 0 .\n| \\Upsilon | = M ^ { - 1 } | Z _ { 1 } Z _ { 2 } | \\, .\nd s ^ { 2 } \\rightarrow - K _ { \\tau } e ^ { ( \\lambda _ { 1 } - \\lambda _ { 3 } - 6 ) \\tau } d \\tau ^ { 2 } + K _ { x } e ^ { - ( \\lambda _ { 3 } + 2 ) \\tau } d \\vec { x } ^ { 2 } + K _ { y } e ^ { ( \\lambda _ { 1 } - 2 ) \\tau } d \\vec { y } ^ { 2 }\n\\hat { \\varrho } _ { \\mathrm { f l d } } = \\hat { \\varrho } _ { m } \\left[ 1 + \\zeta \\left( { \\frac { \\hat { c } } { c } } \\right) ^ { 2 } \\right] + \\hat { \\varrho } _ { r } \\left( { \\frac { \\hat { c } } { c } } \\right) ^ { 2 } , \\ \\ \\ \\ \\ \\ \\ \\hat { \\wp } _ { \\mathrm { f l d } } = { \\frac { 1 } { 3 } } \\hat { \\varrho } _ { r } \\left( { \\frac { \\hat { c } } { c } } \\right) ^ { 2 } ,\nd e t \\left| \\begin{array} { c c c } { Q ( \\lambda - 3 i / 2 ) } & { Q ( \\lambda \\pm i / 2 ) } & { Q ( \\lambda + 3 i / 2 ) } \\\\ { P ( \\lambda - 3 i / 2 ) } & { P ( \\lambda \\pm i / 2 ) } & { P ( \\lambda + 3 i / 2 ) } \\\\ { R ( \\lambda - 3 i / 2 ) } & { R ( \\lambda \\pm i / 2 ) } & { R ( \\lambda + 3 i / 2 ) } \\\\ \\end{array} \\right| = T ^ { \\pm } ( \\lambda ) .\n\\left[ - [ f ( p ) \\partial _ { p } ] ^ { 2 } - \\xi ^ { 2 } + 2 \\, a ( i \\, f ( p ) \\partial _ { p } - \\xi ) + 2 \\, b ( v ( p ) - \\gamma ) - \\mu ^ { 2 } \\right] \\Psi ( p ) = 0\n\\left( \\frac { i \\phi } { { 2 \\pi } } \\right) ^ { 2 n } \\int _ { M } \\mathrm { e } ^ { i \\phi H } \\frac { \\omega ^ { n } } { n ! } = \\sum _ { d H = 0 } \\frac { \\mathrm { e } ^ { i \\phi H } { { \\sqrt { \\operatorname* { d e t } \\omega } } } } { \\sqrt { \\operatorname* { d e t } \\mathrm { P f } H } } .\nB \\times J = \\pi \\frac { 2 + 2 \\operatorname { c o s } 2 v s + 2 \\operatorname { c o s h } 2 s c _ { 1 } + 2 \\operatorname { c o s h } 2 s c _ { 2 } - 8 \\operatorname { c o s } v s ( \\operatorname { c o s h } s c _ { 1 } \\operatorname { c o s h } s c _ { 2 } ) } { 4 \\operatorname { s i n h } s c _ { 1 } \\operatorname { s i n h } s c _ { 2 } \\operatorname { s i n } s v }\n\\left. \\prod _ { p \\in Y _ { j } } \\, \\frac { e \\beta } { 1 - \\frac { e \\beta } { 2 } } \\leq e ^ { - N _ { p } ( Y _ { j } ) \\, \\operatorname { l o g } ( \\frac { 1 } { 2 e \\beta } ) } \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\mathrm { f o r } \\ \\ \\ \\beta < \\frac { 1 } { e } \\ , \\right.\n\\frac { \\partial \\Sigma } { \\partial m } = \\mathcal { B } _ { \\Sigma } \\Lambda \\; ,\nd s ^ { 2 } / \\alpha ^ { \\prime } = \\frac { U } { R _ { 5 } f _ { 1 } ^ { 1 / 2 } } ( - d t ^ { 2 } + d x _ { 1 } ^ { 2 } ) + \\frac { U f _ { 1 } ^ { 1 / 2 } } { R _ { 5 } } ( d x _ { 2 } ^ { 2 } + \\cdots + d x _ { 5 } ^ { 2 } ) + U R _ { 5 } f _ { 1 } ^ { 1 / 2 } \\left( \\frac { d U ^ { 2 } } { U ^ { 2 } } + d \\Omega _ { 3 } ^ { 2 } \\right) .\nL _ { 0 } = \\mathrm { D i a g o n a l } ( 0 , 2 , 3 , 4 , 4 , 5 , 5 , 6 , 6 , 6 , 6 , 7 , 7 , 7 , 7 , 8 , 8 , 8 , 8 , 8 , 8 , 8 , \\ldots )\n\\phi _ { i } = \\left( \\begin{array} { c c c c c } { a _ { i } ^ { 1 } } & { } & { } & { } & { } \\\\ \\end{array} \\right) ,\nz _ { \\nu } ^ { ( 3 ) } + p \\ddot { z } _ { \\nu } + q \\dot { z } _ { \\nu } + r \\lambda _ { \\nu } = 0 \\; .\n\\partial _ { \\mu } V _ { \\mu } ^ { a } = 0 , ~ ~ ~ ~ ~ ~ \\partial _ { \\mu } W _ { 5 \\mu } ^ { a } = 0 .\nu _ { t _ { 2 n + 1 } } = { \\cal K } _ { 2 n + 1 } , ( n = 1 , 2 , \\cdots ) .\nt _ { 2 } = \\operatorname { t a n } \\left( \\frac { \\pi } { 2 4 } \\right) T _ { 0 } T _ { 2 } T _ { 3 } / \\sqrt { 3 } = \\operatorname { t a n } \\left( \\frac { \\pi } { 2 4 } \\right) \\operatorname { t a n } \\left( \\frac { 5 \\pi } { 2 4 } \\right) \\operatorname { t a n } \\left( \\frac { \\pi } { 1 2 } \\right)\n\\theta ^ { \\prime } ( u ^ { \\prime } ) = D ( \\Lambda , u ) \\theta ( u ) D ^ { - 1 } ( \\Lambda , u ) .\ny _ { \\alpha _ { 1 } } = \\left( \\begin{array} { c } { \\pi } \\\\ { \\varphi } \\\\ \\end{array} \\right) ,\n\\mu ^ { \\prime } = \\frac { 1 } { 2 } N x ^ { 2 } \\phi ^ { 2 } + e ^ { 2 \\gamma \\phi } \\left[ N ( K ^ { 2 } + H ^ { 2 } ) + \\frac { 1 } { 8 x ^ { 2 } } \\left( \\left( K ^ { 2 } + H ^ { 2 } - 4 \\right) ^ { 2 } + 1 2 K ^ { 2 } H ^ { 2 } \\right) \\right] \\ ,\ng | \\Lambda \\rangle = \\xi h | \\Lambda \\rangle = \\xi | \\Lambda \\rangle e ^ { i \\phi ( h ) } .\n\\langle 0 | ( X _ { 1 1 } ^ { 1 } ) ^ { 2 } | 0 \\rangle \\langle 0 | ( P _ { 1 1 } ^ { 1 } ) ^ { 2 } | 0 \\rangle \\geq 1 \\ .\n\\sum _ { i = 1 } ^ { d + 2 } { x _ { i } } ^ { ( d + 2 ) } - ( d + 2 ) \\, \\psi \\, { x _ { 1 } x _ { 2 } \\cdot \\cdot \\cdot x _ { d + 2 } } = 0\n\\Delta s = \\left( \\frac { g _ { \\star \\mathrm { R H } } } { g _ { \\star \\mathrm { d o m } } } \\right) ^ { 1 / 4 } \\, \\left( \\frac { H _ { \\mathrm { d o m } } } { H _ { \\mathrm { R H } } } \\right) ^ { [ 4 - 3 ( 1 + w ) ] / [ 2 ( 1 + w ) ] } \\, \\Omega _ { r } ^ { - 3 / 4 } .\n\\Delta S _ { M } ^ { \\mathrm { n l } } ( \\lambda = 0 ) = \\frac { 5 M \\kappa ^ { 2 } } { 1 5 3 6 \\pi ^ { 2 } } \\int d ^ { 4 } x \\sqrt { - g } R \\operatorname { l n } ( - \\nabla ^ { 2 } ) \\delta ^ { 3 } ( { \\bf x } ) ,\nC _ { S R G } ^ { ( g ) } = \\frac { 1 } { \\sqrt { X } } X ^ { + } X ^ { - } - 8 \\lambda ^ { 2 } \\sqrt { X } \\, .\n\\Phi ( \\omega ) \\sim { \\frac { 2 \\omega ^ { D - 2 } } { ( 4 \\pi ) ^ { ( D - 1 ) / 2 } } } \\sum _ { n = 0 } ^ { \\infty } { \\frac { c _ { n } } { \\Gamma \\left( { \\frac { D - 1 } { 2 } } - n \\right) } } \\omega ^ { - 2 n } ,\nT ^ { 0 } = \\frac { 1 } { 4 \\pi \\kappa _ { B } k } \\frac { ( d - 2 k - 1 ) } { r _ { + } } .\n\\begin{array} { c } { { \\cal L } = g _ { a { \\bar { b } } } \\left( { \\dot { z } } ^ { a } { \\dot { \\bar { z } } } ^ { b } + \\frac 1 2 \\eta _ { k } ^ { a } \\frac { D { \\bar { \\eta } } _ { k } ^ { \\bar { b } } } { d \\tau } + \\frac 1 2 \\frac { D \\eta _ { k } ^ { a } } { d \\tau } \\bar { \\eta } ^ { \\bar { b } } \\right) - } \\\\ { - g ^ { a { \\bar { b } } } ( G _ { a } G _ { \\bar { b } } + { U } _ { a } { \\bar { U } } _ { \\bar { b } } ) + } \\\\ { + i U _ { a ; b } \\eta _ { 1 } ^ { a } \\eta _ { 2 } ^ { b } - i { \\bar { U } } _ { \\bar { a } ; \\bar { b } } { \\bar { \\eta } } _ { 1 } ^ { \\bar { a } } { \\bar { \\eta } } _ { 2 } ^ { \\bar { b } } + R _ { a \\bar { b } c \\bar { d } } \\eta _ { 1 } ^ { a } \\bar { \\eta } _ { 1 } ^ { b } \\eta _ { 2 } ^ { a } \\bar { \\eta } _ { 2 } ^ { d } . } \\\\ \\end{array}\n\\Lambda = D _ { 0 } \\otimes \\Lambda _ { 2 p } ^ { C } = \\mathrm { d i a g } \\left( d _ { 1 } \\Lambda _ { 2 p } ^ { C } , \\ldots , d _ { s } \\Lambda _ { 2 p } ^ { C } \\right) .\nc ( z ) \\equiv ( a \\ast b ) ( z ) = \\left( e ^ { \\pi i \\theta ^ { j k } \\frac { \\partial } { \\partial x ^ { j } } \\frac { \\partial } { \\partial y ^ { k } } } a ( x ) b ( y ) \\right) _ { x = y = z } \\, .\nJ ~ \\equiv ~ \\epsilon ^ { \\alpha \\beta \\gamma \\delta } \\epsilon _ { a b c d e } \\phi ^ { e } \\nabla _ { \\alpha } \\phi ^ { a } \\nabla _ { \\beta } \\phi ^ { b } \\nabla _ { \\gamma } \\phi ^ { c } \\nabla _ { \\delta } \\phi ^ { d } ~ ,\nI _ { g } = \\frac { 1 } { 2 \\pi \\kappa } \\int d ^ { 2 } x \\epsilon ^ { \\mu \\nu } \\Bigl ( \\eta _ { a } ( \\partial _ { \\mu } e _ { \\nu } ^ { a } + \\omega _ { \\mu } \\epsilon _ { ~ b } ^ { a } e _ { \\nu } ^ { b } ) + \\eta _ { 2 } \\partial _ { \\mu } \\omega _ { \\nu } + \\eta _ { 3 } ( \\partial _ { \\mu } a _ { \\nu } + \\frac { 1 } { 2 } \\epsilon _ { a b } e _ { \\mu } ^ { a } e _ { \\nu } ^ { b } ) \\Bigr ) \\, .\n\\int _ { \\beta } \\Omega ^ { ( 1 , 0 ) } = \\frac { \\sqrt { 6 } } { \\pi } ( \\pm \\sqrt { B ^ { \\prime } } ) ^ { - 1 / 2 } \\, ( \\pm k _ { \\mp } ^ { 2 } ) ^ { - 1 / 2 } \\, \\, { \\bf K } ( k _ { \\mp } ^ { - 2 } )\n{ \\frac { d H _ { 2 } } { d t } } = \\nu \\int \\sp { 1 / a } k \\sp 2 H _ { k } d k \\propto \\nu a \\sp { - 6 - 4 \\Delta _ { \\psi } } \\propto \\nu \\sp { \\frac { 3 + \\Delta _ { \\phi } + \\Delta _ { \\psi } } { \\Delta _ { \\phi } - \\Delta _ { \\psi } } } \\, .\nX \\Psi _ { n _ { 1 } n _ { 2 } m } ( \\mu , \\nu , \\varphi ; \\delta _ { 1 } , \\delta _ { 2 } ) = - \\alpha \\frac { n _ { 1 } - n _ { 2 } + \\frac { \\delta _ { 1 } - \\delta _ { 2 } } { 2 } } { n + \\frac { \\delta _ { 1 } + \\delta _ { 2 } } { 2 } } \\, \\Psi _ { n _ { 1 } n _ { 2 } m } ( \\mu , \\nu , \\varphi ; \\delta _ { 1 } , \\delta _ { 2 } ) ,\n\\left( \\displaystyle \\frac { \\operatorname { s i n h } \\displaystyle \\frac { \\gamma } { \\pi } \\left[ \\vartheta _ { j } + \\Theta + \\displaystyle \\frac { i \\pi } { 2 } \\right] \\operatorname { s i n h } \\displaystyle \\frac { \\gamma } { \\pi } \\left[ \\vartheta _ { j } - \\Theta + \\displaystyle \\frac { i \\pi } { 2 } \\right] } { \\operatorname { s i n h } \\displaystyle \\frac { \\gamma } { \\pi } \\left[ \\vartheta _ { j } + \\Theta - \\displaystyle \\frac { i \\pi } { 2 } \\right] \\operatorname { s i n h } \\displaystyle \\frac { \\gamma } { \\pi } \\left[ \\vartheta _ { j } - \\Theta - \\displaystyle \\frac { i \\pi } { 2 } \\right] } \\right) ^ { N } = - e ^ { 2 i \\omega } \\prod _ { k = 1 } ^ { M } \\displaystyle \\frac { \\operatorname { s i n h } \\displaystyle \\frac { \\gamma } { \\pi } \\left[ \\vartheta _ { j } - \\vartheta _ { k } + i \\pi \\right] } { \\operatorname { s i n h } \\displaystyle \\frac { \\gamma } { \\pi } \\left[ \\vartheta _ { j } - \\vartheta _ { k } - i \\pi \\right] }\nv ( \\infty ) = \\frac { L ^ { 2 } } { 4 \\pi ^ { 2 } } \\int \\mathrm { T r } \\; F _ { ( \\infty ) } ^ { 2 } = 1 .\nV ( \\Phi ) = \\frac { m ^ { 2 } } { 2 } | \\Phi | ^ { 2 } + \\frac { \\lambda } { 4 ! } | \\Phi | ^ { 4 } ,\nR _ { \\hat { g } } ( z , \\bar { z } ) = - 1 \\to R _ { g } ( z , \\bar { z } ) = - e ^ { 4 \\pi G ( z , w ) } \\left( 1 + 8 \\pi e ^ { - \\varphi ( z , \\bar { z } ) } \\delta ^ { ( 2 ) } ( z - w ) + { \\frac { 1 } { 4 \\chi ( \\Sigma ) } } \\right) .\n\\left\\{ \\begin{array} { l } { \\tilde { x } \\, x ^ { \\prime } = \\bar { q } \\, \\overline { { R } } \\, x \\, \\tilde { x } ^ { \\prime } \\, , } \\\\ { d \\tilde { x } \\, x ^ { \\prime } = \\bar { q } \\, \\overline { { R } } \\, x \\, d \\tilde { x } ^ { \\prime } - \\lambda \\, \\bar { q } \\, d x \\, \\tilde { x } ^ { \\prime } \\, , } \\\\ { \\tilde { x } \\, d x ^ { \\prime } = \\bar { q } \\, R \\, d x \\, \\tilde { x } ^ { \\prime } \\, , } \\\\ { d \\tilde { x } \\, d x ^ { \\prime } = - \\bar { q } \\, R \\, d x \\, d \\tilde { x } ^ { \\prime } \\, ; } \\\\ \\end{array} \\right.\nj _ { [ e , m , b ] } ^ { \\nu } = j _ { [ e ] } ^ { \\nu } + j _ { [ m ] } ^ { \\nu } * K _ { [ e ] } + j _ { [ b ] } ^ { \\nu } * \\varepsilon _ { 0 } * K _ { [ v ] }\n- { \\frac { 8 \\pi } { 1 5 } } < \\mathrm { a r g } ( \\eta ) < { \\frac { 8 \\pi } { 1 5 } }\n( c _ { 0 } ^ { ( 1 ) } + c _ { 0 } ^ { ( 2 ) } + c _ { 0 } ^ { ( 3 ) } ) | V _ { 3 } \\rangle = 0 , \\ \\ c _ { 0 } | I \\star A \\rangle = | I \\star ( c _ { 0 } A ) \\rangle , \\ \\ \\forall A\n\\tilde { R } _ { \\xi } ^ { + } = \\operatorname* { l i m } _ { q \\to 1 } \\left[ \\frac { \\alpha ( 2 - \\alpha ) ( 1 - \\alpha ) \\mathcal { A } _ { f } } { ( q - 1 ) } \\right] ^ { \\frac { 1 } { 2 } } \\xi _ { 0 } ^ { + } .\nm _ { j } = 8 \\, p ^ { \\frac { r } { 6 s } } \\left\\{ \\sum _ { a } \\operatorname { s i n } \\frac { a \\pi } { g } + \\frac { 4 } { 3 } ( p ^ { \\frac { r } { 6 s } } ) ^ { 2 } \\sum _ { a } \\operatorname { s i n } ^ { 3 } \\frac { a \\pi } { g } + \\ldots \\right\\} ,\nJ _ { \\mu } = \\epsilon _ { \\mu \\nu \\rho } x ^ { \\nu } p ^ { \\rho } + S c ^ { 2 } \\frac { p _ { \\mu } } { \\sqrt { p ^ { 2 } } }\n{ \\cal F } = - { \\frac { 1 } { \\sqrt { c _ { 0 } } g ^ { 2 } ( 1 + \\gamma ) } } ( 1 - 2 g ^ { 2 } | \\phi | ^ { 2 } ) ^ { ( \\gamma + 1 ) / 2 }\n{ \\Lambda } ( l ) = \\frac { 1 } { N } { \\times } ( ~ M i n i m u m ~ o f ~ s _ { F } )\n\\frac { \\partial S } { \\partial \\xi } = \\int d ^ { 3 } x d ^ { 3 } y \\, A _ { i } ^ { A } ( x ) W ( x - y ) \\partial _ { i } D _ { j } ^ { A B } \\frac { \\delta S } { \\delta A _ { j } ^ { B } ( y ) } .\nG _ { a b } = \\frac { 1 } { \\sqrt { - g } } \\left( \\frac { \\partial { \\cal L } _ { 1 } } { \\partial g ^ { a b } } - \\partial _ { c } \\frac { \\partial { \\cal L } _ { 1 } } { \\partial ~ \\partial _ { c } g ^ { a b } } \\right) .\n( \\mu \\frac { \\partial } { \\partial \\mu } + \\beta _ { g } \\frac { \\partial } { \\partial g } + \\beta _ { \\lambda } \\frac { \\partial } { \\partial \\lambda } - \\gamma _ { M } \\frac { \\partial } { \\partial \\operatorname { l n } M ^ { 2 } } + \\gamma ) F _ { n } ( x _ { 1 } , . . . , x _ { n } ; m , M , g , \\lambda , \\mu ) = 0 ,\n( \\bar { \\varepsilon } _ { + } ^ { a } ) ^ { \\dagger } ( \\bar { \\varepsilon } _ { - } ^ { a } ) = 0 ; \\hspace { 1 . 0 c m } ( \\bar { \\varepsilon } _ { + } ^ { a } ) ^ { \\dagger } ( \\bar { \\varepsilon } _ { + } ^ { a } ) = ( \\bar { \\varepsilon } _ { - } ^ { a } ) ^ { \\dagger } ( \\bar { \\varepsilon } _ { - } ^ { a } ) = 1 .\n[ T _ { i } , T _ { j } ] = c _ { i j k } T _ { k } \\ , \\quad [ T _ { i } , T _ { j } ^ { \\prime } ] = c _ { i j k } T _ { k } ^ { \\prime } \\ , \\quad [ T _ { i } ^ { \\prime } , T _ { j } ^ { \\prime } ] = - c _ { i j k } T _ { k } \\ .\n( 1 - \\gamma ^ { 1 } ) k _ { n } ( 0 , x ^ { \\prime } ) = \\left( \\begin{array} { c c } { 1 } & { i } \\\\ { - i } & { 1 } \\\\ \\end{array} \\right) k _ { n } ( 0 , x ^ { \\prime } ) = 0 .\n\\tilde { \\Gamma } ( x ) = \\frac { ( 2 \\pi ) ^ { 4 } } { ( x ^ { 2 } ) ^ { 2 } }\nf ( Q ) = F ( u ) : = \\frac { 1 } { 4 0 } ( u - 6 ) ( u - 1 2 ) ( u - 1 5 )\n\\tilde { Z } _ { i j } = i \\sigma _ { 2 } \\left( \\begin{array} { l l } { | Z | } & { 0 } \\\\ { 0 } & { | Z | } \\\\ \\end{array} \\right) \\ , \\qquad \\tilde { Z } _ { i j } = i \\sigma _ { 2 } \\left( \\begin{array} { l l l l } { | Z | } & { 0 } & { 0 } & { 0 } \\\\ { 0 } & { | Z | } & { 0 } & { 0 } \\\\ { 0 } & { 0 } & { | Z | } & { 0 } \\\\ { 0 } & { 0 } & { 0 } & { | Z | } \\\\ \\end{array} \\right) \\ ,\n\\vert \\Psi ( t ) \\rangle = \\sum _ { n } C _ { n } \\vert e _ { n } , t \\rangle e ^ { i \\int d t \\langle e _ { n } , t \\vert i \\frac { \\partial } { \\partial t } - \\hat { H } \\vert e _ { n } , t \\rangle } .\nZ _ { 3 } = \\frac { ( m L ) ^ { 3 } } { 3 ! } I _ { 1 } ^ { 4 } - \\frac { ( m L ) ^ { 2 } } { 2 } I _ { 1 } I _ { 2 } + \\frac { ( m L ) } { 3 } I _ { 3 } \\, \\, \\, ,\n\\chi _ { \\Lambda } ( C _ { m } ^ { \\Lambda _ { 0 } } ) = \\sum _ { ( \\lambda + \\lambda _ { 0 } ^ { \\prime } , \\kappa + \\kappa _ { 0 } , - s ) \\in D ^ { + } } \\sum _ { s = 0 } ^ { \\infty } \\, m _ { \\lambda _ { 0 } ^ { \\prime } , s } \\, \\alpha _ { \\lambda _ { 0 } ^ { \\prime } , s } ^ { m } ( \\Lambda ) \\frac { D _ { q } [ ( \\lambda + \\lambda _ { 0 } ^ { \\prime } , \\kappa + \\kappa _ { 0 } , - s ) ] } { D _ { q } [ ( \\lambda , \\kappa , 0 ) ] } \\, , ~ ~ ~ m \\in { \\bf Z } ^ { + }\na _ { i } a _ { j } ^ { \\dagger } - \\hat { q } a _ { j } ^ { \\dagger } a _ { i } = \\delta _ { i j } , \\qquad \\hat { q } | \\pm > = \\pm 1 | \\pm > .\n\\left( \\begin{array} { c c } { - \\operatorname { c o s } 2 \\phi } & { - \\operatorname { s i n } 2 \\phi } \\\\ { \\operatorname { s i n } 2 \\phi } & { - \\operatorname { c o s } 2 \\phi } \\\\ \\end{array} \\right)\n\\tilde { A } _ { l } ( { \\bf x } ) = A _ { l } ( { \\bf x } ) ,\n\\vert \\nu _ { A } \\rangle = { \\frac { 1 } { \\sqrt { 1 + \\vert \\nu _ { A } \\vert ^ { 2 } } } } \\left( \\begin{array} { c } { - \\nu _ { A } } \\\\ { 1 } \\\\ \\end{array} \\right) \\quad , \\quad \\langle \\nu _ { A } \\vert = { \\frac { 1 } { \\sqrt { 1 + \\vert \\nu _ { A } \\vert ^ { 2 } } } } \\Bigl ( - \\nu _ { A } ^ { * } \\; \\quad \\; 1 \\Bigr ) \\; .\nA ( x _ { 1 } ) B ( x _ { 2 } ) = \\cdots + g _ { A B C } \\, \\frac { 1 } { ( x _ { 1 2 } ^ { 2 } ) ^ { \\frac { 1 } { 2 } ( \\eta _ { A } + \\eta _ { B } - \\eta _ { C } ) } } C ^ { \\eta _ { C } , \\eta _ { A } - \\eta _ { B } } ( x _ { 1 2 } , \\partial _ { 2 } ) C ( x _ { 1 } ) + \\cdots { } .\n\\rho ( \\phi ) = \\frac { 1 } { 2 \\pi } \\int _ { 0 } ^ { 1 } d t \\; \\frac { \\theta \\left( 4 R ( t ) - \\phi ^ { 2 } \\right) } { \\sqrt { 4 R ( t ) - \\phi ^ { 2 } } }\n\\rho \\partial _ { \\pm } F = \\pm \\partial _ { \\pm } ( \\tau \\rho F ) .\ne ^ { i \\theta } = { \\frac { p - q \\tau ^ { * } } { | p - q \\tau ^ { * } | } } ,\nT ( \\xi ) = e ^ { i \\xi _ { i } \\Sigma ^ { i } } = e ^ { 2 i \\xi _ { i } { \\cal S } ^ { i } } \\, , \\quad \\xi = \\xi _ { i } f ^ { i } \\in L _ { F } ,\nr _ { 2 } = r _ { 1 } ~ ~ ~ ~ ~ ; ~ ~ ~ ~ ~ t _ { 2 } = t _ { 1 } ~ ~ ~ ~ ~ ; ~ ~ ~ ~ ~ \\phi _ { 2 } = \\phi _ { 1 } + \\gamma ~ ~ ( \\phi _ { 1 } > 0 ) ~ ~ ~ ~ ~ ; ~ ~ ~ ~ ~ \\phi _ { 2 } = \\phi _ { 1 } - \\gamma ~ ~ ( \\phi _ { 1 } < 0 ) .\n( \\sum _ { n = 1 } ^ { 1 5 } ( - 1 ) ^ { n } - 1 ) ( \\frac { g ^ { 2 } } { 3 2 \\pi ^ { 2 } } ) t r \\epsilon ^ { \\mu \\nu \\alpha \\beta } F _ { \\mu \\nu } F _ { \\alpha \\beta } = - 2 ( \\frac { g ^ { 2 } } { 3 2 \\pi ^ { 2 } } ) t r \\epsilon ^ { \\mu \\nu \\alpha \\beta } F _ { \\mu \\nu } F _ { \\alpha \\beta }\n\\Sigma = S ^ { \\mathrm { N = 4 } } + S _ { \\mathrm { e x t } } + S _ { \\mathrm { g f } } \\, \\, ,\n\\tilde { L } _ { 0 } ^ { k , k } ( e ^ { x } ) = \\sum _ { d = 0 } ^ { \\infty } \\frac { ( k d ) ! } { ( d ! ) ^ { k } } e ^ { d x } ,\n\\int d u \\; 1 = 1 , \\ \\ \\int d u \\; u _ { ( A _ { 1 } } ^ { + } \\ldots u _ { A _ { p } } ^ { + } u _ { B _ { 1 } } ^ { - } \\ldots u _ { B _ { q } ) } ^ { - } = 0 \\ \\mathrm { f o r ~ p ~ a n d / o r ~ q > 0 ~ } \\; .\n{ \\displaystyle x = \\frac { 3 / 2 e _ { 1 } ( \\tau ) } { 3 / 2 e _ { 1 } ( \\tau ) - \\hat { u } } , } \\hspace { 1 c m } { \\displaystyle \\sqrt { y } = - \\frac { e _ { 2 } ( \\tau ) - e _ { 3 } ( \\tau ) } { 3 e _ { 1 } ( \\tau ) } , }\n{ \\cal L } = - \\frac { 1 } { 4 } F _ { \\mu \\nu } F ^ { \\mu \\nu } + \\frac { 1 } { 2 } m ^ { 2 } ( A _ { \\mu } + \\partial _ { \\mu } \\theta ) ( A ^ { \\mu } + \\partial ^ { \\mu } \\theta ) + A ^ { \\mu } \\partial _ { \\mu } B - \\partial _ { \\mu } \\bar { \\cal C } \\partial ^ { \\mu } { \\cal C } - \\frac { 1 } { 2 } \\alpha B ^ { 2 } .\nS = \\frac { 1 } { 1 6 \\pi } \\int d ^ { 5 } x \\sqrt { - g } R \\, ,\nd s ^ { 2 } = - \\frac { d W ^ { 2 } } { \\operatorname { s i n h } ^ { 2 } W \\operatorname { l n } ^ { 2 / 3 } T _ { c } ^ { 2 } } + \\operatorname { l n } ^ { 2 / 3 } T _ { c } ^ { 2 } \\sum _ { j = 2 } ^ { 4 } d x _ { j } ^ { 2 } ,\nI = \\int _ { \\Omega } \\frac { 1 } { 2 } \\left( \\epsilon _ { \\mu \\nu \\lambda } B _ { \\mu } ^ { * } ( \\partial _ { \\nu } - i g a _ { \\nu } ) B _ { \\lambda } + M B _ { \\mu } ^ { * } B ^ { \\mu } + \\epsilon _ { \\mu \\nu \\lambda } a _ { \\mu } \\partial _ { \\nu } a _ { \\lambda } \\right) \\; ,\n\\{ \\theta ^ { a } , \\theta ^ { b } \\} = \\delta ^ { a b } \\sqrt { \\hbar \\alpha ^ { \\prime } }\n\\Omega ( \\gamma _ { r } | A _ { + } \\rangle ) = ( - 1 ) ^ { - r - \\frac { 1 } { 2 } } \\gamma _ { r } ( \\Omega | A _ { + } \\rangle ) .\nS [ \\vec { r } ] \\ = \\ \\int d ^ { D } \\! x \\ { \\frac { 1 } { 2 } } ( \\nabla _ { \\! x } \\vec { r } ) ^ { 2 } \\ + \\b \\ \\int d ^ { D } \\! x \\int d ^ { D } \\! y \\ \\delta ^ { d } ( \\vec { r } ( x ) - \\vec { r } ( y ) ) \\ .\n\\rho = \\rho _ { 0 } ~ R ^ { \\, \\gamma ( 1 - D ) } ~ ~ .\n\\Pi _ { \\sigma } ( p )\n\\langle \\! \\langle \\psi _ { i } | \\psi _ { j } \\rangle = \\langle \\psi _ { i } ^ { * } | \\psi _ { j } \\rangle \\ ,\nP ^ { - } = { \\frac { m ^ { 2 } } { 4 \\pi } } \\int _ { 0 } ^ { \\infty } { \\frac { d k } { k ^ { 2 } } } \\left[ b ^ { \\dag } ( k ) b ( k ) + d ^ { \\dag } ( k ) d ( k ) \\right] + { \\frac { 1 } { 2 } } \\int _ { 0 } ^ { \\infty } { \\frac { d p } { p ^ { 2 } } } a ^ { \\dag } ( p ) a ( p ) \\ .\n\\tau = \\frac { \\lambda } { 2 } ( T _ { 0 } ^ { \\prime } - T _ { 0 } )\n\\left( S _ { n } ( \\rho a ) - S _ { n } ^ { ' } ( \\rho a ) \\right) \\left( S _ { n } ( - \\rho R ) - S _ { n } ^ { ' } ( - \\rho R ) \\right) .\n\\begin{array} { l l } { { \\cal L } _ { K } = } & { \\frac { 1 } { 4 } F _ { \\mu \\nu } F ^ { \\mu \\nu } - ( 1 - g ^ { 2 } \\zeta \\eta ^ { 2 } ) \\tilde { G } _ { \\mu } \\tilde { G } ^ { \\mu } + 2 ( m + g h \\zeta \\eta ^ { 2 } ) A _ { \\mu } \\tilde { G } ^ { \\mu } + } \\\\ \\end{array}\nT _ { m } ( x ) \\equiv \\int _ { 0 } ^ { x } d x ^ { \\prime } ( t a n h \\; x ^ { \\prime } ) ^ { m } = - \\frac { ( t a n h \\; x ) ^ { m - 1 } } { m - 1 } + T _ { m - 2 } ( x ) \\; .\n\\frac { \\mathrm { d } ^ { 2 } \\mu } { \\mathrm { d } x ^ { 2 } } + \\biggl ( \\frac { x ^ { 2 } } { 4 } - a \\biggr ) \\mu = 0 , \\quad a \\equiv - \\frac { \\pi } { \\epsilon \\sqrt { b _ { 1 } } } < 0 .\n\\nu = \\frac { g \\beta } { 2 \\pi } \\int E ( x ) d x = - \\frac { g } { 2 \\pi T } [ A _ { 0 } ( \\infty ) - A _ { 0 } ( - \\infty ) ] \\ = ~ 1\n\\sum _ { s = 0 } ^ { n - l - 1 } s _ { s } s _ { 2 n - l - m - s } + s _ { n - l } s _ { n - m } +\n( D _ { 1 } + i D _ { 2 } ) { \\psi } _ { + } ( \\vec { x } ) = 0 .\n\\operatorname { s i n } ^ { 2 } ( \\pi k / N ) \\sum _ { a = 1 } ^ { K } \\, I _ { a b } \\, r _ { a , k } = 0\nD _ { I } = \\sum _ { a = 1 } ^ { N } q _ { I } ^ { a } \\left( | h _ { a } | ^ { 2 } - | \\tilde { h } _ { a } | ^ { 2 } \\right) ,\n\\xi = \\theta ^ { \\alpha } \\lambda _ { \\alpha } \\ .\nj _ { \\alpha } ^ { 0 } = \\left( i \\Gamma ^ { m } \\theta \\right) _ { \\alpha } \\cdot \\Lambda _ { m } + \\Lambda _ { \\alpha } + \\left( e m b ^ { * } \\Delta _ { \\alpha } \\right) _ { \\nu } \\cdot \\Lambda ^ { \\nu } \\quad ;\n- \\frac { R ( P ) } { 2 } | \\Phi ( P ) | ^ { 2 } - \\frac { 1 } { 2 } | \\Phi ( P ) | ^ { 4 } \\ge 0\na _ { H } ^ { i j } ( x ) - \\frac { 1 } { 3 } \\delta ^ { i j } \\delta ^ { k l } a _ { H } ^ { k l } ( x ) = U ( x ^ { o } ) a ^ { i j } ( x ) U ^ { - 1 } ( x ^ { o } ) ,\n{ \\frac { d n _ { i } ( \\omega ) } { d \\omega } } = { \\frac { \\omega } { \\pi } } \\int _ { - \\infty } ^ { + \\infty } d \\alpha ~ e ^ { i \\alpha \\omega ^ { 2 } } \\mathrm { T r } ~ e ^ { - i \\alpha \\bar { H } _ { i } ^ { 2 } } ~ ~ ~ .\nV _ { 1 } ( t ) = \\operatorname { e x p } \\{ i { \\frac { e } { \\hbar c } } \\int \\int d ^ { 2 } x d ^ { 2 } x ^ { \\prime } \\sqrt { g ( x ) } J _ { 0 } ( x ) G ( \\vec { x } , \\vec { x } ^ { \\prime } ) \\partial _ { i } ^ { \\prime } a _ { i } ^ { ( 1 ) } ( x ^ { \\prime } ) \\} .\nA _ { 4 } \\sim g _ { s } ^ { 2 } \\frac { R _ { s } } { N _ { 1 } N _ { 2 } } \\frac { s ^ { 2 } } { t }\nG ^ { ( \\rho ) } ( x , y ) : = \\sum _ { n } \\overline { { P _ { n } ^ { ( \\rho ) } ( x ) } } P ^ { ( \\rho ) n } ( y ) .\n\\mu = 1 / 2 + \\nu , \\mathrm { f o r ~ t y p e ~ I ~ b o u n d a r y ~ c o n d i t i o n s }\n\\cdot \\left( 1 + \\frac { 2 l ^ { 2 } \\dot { l } ^ { 2 } H _ { + } ^ { 2 } } { \\Delta _ { + } ^ { 2 } } + \\frac { 1 } { \\Delta _ { + } } \\left( \\dot { l } ^ { 2 } + l ^ { 2 } H _ { + } ^ { 2 } - \\frac { 2 l \\dot { l } H _ { + } } { \\Delta _ { + } } \\sqrt { l ^ { 2 } \\dot { l } ^ { 2 } H _ { + } ^ { 2 } + \\Delta _ { + } ^ { 2 } + \\Delta _ { + } ( \\dot { l } ^ { 2 } + l ^ { 2 } H _ { + } ^ { 2 } ) } \\right) \\right) \\Bigr \\}\nW [ \\varphi , \\rho ] = W _ { + } ^ { ( 1 ) } [ \\varphi ] + W _ { - } ^ { ( 1 ) } [ \\rho ] + { \\frac { 1 } { 2 \\pi } } \\int d ^ { 2 } x \\, B _ { + } \\, B _ { - }\ne ^ { 2 x } C ( x ) = \\frac { B ( x ) \\lambda _ { 1 } } { 2 } + \\frac { C ( x ) \\lambda _ { 1 } } { 4 } + \\lambda _ { 1 } ,\nV = - g D ^ { 2 } \\Gamma _ { 4 } + H D ^ { 2 } \\Gamma _ { 4 } .\n\n( { } ^ { \\mu _ { - } } \\omega ) _ { + } = 0 , \\qquad ( { } ^ { \\mu _ { + } } \\omega ) _ { - } = 0 .\n: e ^ { i \\hat { \\varphi } ( \\sigma ) } : \\, = e ^ { i \\hat { \\chi } ^ { \\dagger } ( \\sigma ) } \\, e ^ { i \\hat { \\chi } ( \\sigma ) } .\nI _ { 1 1 } = \\int d ^ { 1 1 } x \\sqrt { - g } ~ ( { \\frac { R } { \\kappa ^ { 2 } } } - 3 \\, \\hat { F } _ { [ 4 ] } ^ { 2 } ) + { 2 8 8 } \\sigma \\int \\hat { F } _ { [ 4 ] } \\wedge \\hat { F } _ { [ 4 ] } \\wedge \\hat { A } _ { [ 3 ] }\nP _ { \\lambda m } ( x , y ) = \\left\\{ \\begin{array} { l l } { J _ { m } ( y ) Y _ { m } ( x ) - Y _ { m } ( y ) J _ { m } ( x ) , } & { \\lambda = 1 } \\\\ { J _ { m } ( y ) Y _ { m } ^ { \\prime } ( x ) - Y _ { m } ( y ) J _ { m } ^ { \\prime } ( x ) , } & { \\lambda = 0 } \\\\ \\end{array} \\right.\n\nn ^ { \\mu } \\nabla _ { \\mu } \\phi = 0 \\, ,\nh = \\Pi ^ { \\mu } ( C \\Gamma _ { \\mu } \\Gamma _ { 1 1 } ) _ { \\alpha \\beta } \\Pi ^ { \\alpha } \\Pi ^ { \\beta } { \\cal F } ^ { \\frac { p - 1 } { 2 } } \\quad , \\quad p \\geq 1 \\quad .\n\\delta \\psi ^ { i } = \\epsilon _ { \\alpha } \\; b _ { \\alpha } ^ { i } \\; , \\; \\; \\; \\; \\delta b _ { \\alpha } ^ { i } = \\epsilon _ { \\gamma } \\; ( \\phi _ { [ \\gamma \\alpha ] } ^ { i } + i \\delta _ { \\gamma \\alpha } \\; \\partial _ { - } \\psi ^ { i } ) \\; .\n\\left| < i _ { l + 1 } ^ { 1 } , k _ { l + 1 } ^ { 1 } , i _ { l + 1 } ^ { 2 } , k _ { l + 1 } ^ { 2 } | e ^ { - \\frac { \\varepsilon } { 2 } H _ { 1 , 2 } } | i _ { l } ^ { 1 } , k _ { l } ^ { 1 } , i _ { l } ^ { 2 } , k _ { l } ^ { 2 } > \\right| .\n\\frac { \\overline { { b } } } { \\overline { { d } } } = \\frac { 2 \\overline { { B } } _ { 2 } } { 3 \\overline { { B } } _ { 1 } }\nO _ { K } = \\left( \\frac 1 8 \\epsilon _ { \\mu \\nu \\kappa \\tau } G _ { \\mu \\nu } ^ { a } G _ { \\kappa \\tau } ^ { a } \\right) ^ { 2 }\n\\left[ \\frac { 1 } { \\omega } ( A _ { N - 6 } b ^ { 6 } ) ^ { 5 / 8 } \\right] ^ { 6 } = \\frac { ( A _ { N - 6 } b ^ { 6 } ) ^ { 6 } } { b ^ { 4 N } } \\quad \\Rightarrow \\quad b ^ { 4 N } = 1 .\n{ \\frac { m ! } { ( 2 m - 1 ) ! ! } } \\left\\{ - 1 / 2 \\partial _ { t } ^ { 2 } + { \\cal Z } ^ { F _ { m - 2 } } ( t ) + \\partial _ { t } ^ { - 1 } { \\cal Z } ^ { F _ { m - 2 } } ( t ) \\partial _ { t } \\right\\} ^ { m } \\cdot 1 = t ,\n\\dot { \\Psi } ^ { R } = i [ H ^ { R } , \\Psi ^ { R } ] = \\gamma _ { 0 } ( - i \\gamma _ { i } \\partial _ { i } - e \\gamma _ { i } V _ { i } ^ { R } + m ) \\Psi ^ { R } + { \\frac { e } { 2 } } \\left\\{ \\Psi ^ { R } , { \\frac { 1 } { \\vec { \\partial } ^ { 2 } - M ^ { 2 } } } J _ { 0 } \\right\\} ~ ,\n( l e f t , \\ r i g h t ) = ( N S + , \\ N S + ) \\oplus ( R + , \\ R + ) .\n\\phi ^ { \\prime \\prime } - \\rho ^ { \\prime \\prime } - \\frac { 1 } { 2 } ( T ^ { \\prime } ) ^ { 2 } = 0\n\\lambda ( R , \\Pi ) = R ( \\sum _ { s = 1 } ^ { N } | \\Pi ^ { s } | ^ { 2 } + R ^ { 2 } ) ^ { - 1 / 2 } .\n\\gamma _ { L } = \\left( 1 - L ^ { - 2 } \\right) \\; c _ { \\infty } , \\quad \\delta _ { L } = L ^ { - 1 } .\nA _ { 1 } = - \\frac { i \\sqrt { 2 } G \\hbar D _ { 1 } \\operatorname { s i n } ( \\frac { 1 } { \\hbar } p _ { 3 } ^ { ( 1 ) } L ) } { ( E + p _ { 3 } ^ { ( 1 ) } ) \\left[ e ^ { - \\frac { i } { \\hbar } p _ { 3 } ^ { ( 1 ) } L } - \\frac { ( G \\hbar D _ { 1 } ) ^ { 2 } } { ( E + p _ { 3 } ^ { ( 1 ) } ) ^ { 2 } } e ^ { \\frac { i } { \\hbar } p _ { 3 } ^ { ( 1 ) } L } \\right] }\nY ^ { - } ( t , \\sigma ^ { a } ) = { \\frac { R } { N } } E t + \\xi ( t , \\sigma ^ { a } )\nF ( z , x ) = \\operatorname { e x p } - \\int _ { a } ^ { b } \\frac { d y } { 2 \\pi i } \\frac { 1 } { y - x } \\operatorname { l o g } \\bigg [ \\frac { z - u ( y ) } { z - \\bar { u } ( y ) } \\bigg ] =\nG _ { _ C } ( x , y ; A ) = \\operatorname { e x p } \\left[ - i e \\int \\! \\! d z \\; A _ { \\mu } ( z ) j _ { + } ^ { \\mu } ( z , x , y ) \\right] P _ { + } G _ { F } ( x - y ) + P _ { - } G _ { F } ( x - y )\n\\left[ \\widetilde R _ { 1 N } ^ { ( 2 ) } ( x ) \\right] _ { \\mathrm { ( I I I ) } } \\simeq \\beta ^ { N } \\simeq \\operatorname { e x p } \\left( - { \\frac { N ^ { 1 / 3 } } { \\alpha _ { c } ^ { 2 } g ^ { 2 / 3 } } } \\right) \\rightarrow 0 .\n\\left[ \\mathrm { V } _ { H _ { 1 } } ( \\phi _ { i } ) , \\mathrm { V } _ { H _ { 2 } } ( \\phi _ { j } ) \\right] = \\delta _ { i j } \\mathrm { V } _ { { \\{ H _ { 1 } , H _ { 2 } \\} } _ { \\mathrm { p b } } } ( \\phi _ { i } )\nl ( 0 ) = r ( 0 ) = 0 \\, , \\qquad l ^ { \\prime } ( \\pi / 2 ) = r ^ { \\prime } ( \\pi / 2 ) = 0 \\, .\n\\varepsilon _ { i j } D _ { i } \\phi ^ { a } = \\pm \\varepsilon ^ { a b c } D _ { j } \\phi ^ { b } \\phi ^ { c } ,\n\\Omega ( a _ { 0 } , a _ { 1 } ; b _ { 0 } , b _ { 1 } ) = \\left\\langle { \\frac { \\mathrm { D e t } ( 1 - a _ { 1 } U ) \\, \\mathrm { D e t } ( 1 - b _ { 1 } \\bar { U } ) } { \\mathrm { D e t } ( 1 - a _ { 0 } U ) \\, \\mathrm { D e t } ( 1 - b _ { 0 } \\bar { U } ) } } \\right\\rangle \\; .\nC ( r _ { A } , r _ { + } ) = \\alpha _ { 4 } ( \\alpha _ { 4 } + 2 ) P ^ { 1 3 } ( P ^ { 2 3 } - P ^ { 1 2 } )\nS _ { R S T } = - \\frac { \\kappa } { \\pi } \\int d ^ { 2 } x \\; \\phi \\partial _ { + } \\partial _ { - } \\rho\n\\alpha _ { A } ( \\varphi ) = - \\left[ \\operatorname { l n } { \\frac { \\widehat \\Lambda _ { s } } { m _ { A } } } + { \\frac { 1 } { 2 } } \\right] { \\frac { \\partial \\operatorname { l n } B ( \\varphi ) } { \\partial \\varphi } } = + \\operatorname { l n } { \\frac { \\widehat \\Lambda _ { s } ^ { \\prime } } { m _ { A } } } \\, { \\frac { \\partial \\operatorname { l n } B ^ { - 1 } ( \\varphi ) } { \\partial \\varphi } } \\ ,\n\\sum _ { m = 1 } ^ { \\infty } \\left\\langle \\frac { P _ { n } ( x _ { 1 } ) \\tilde { Q } _ { m } ( y _ { 1 } ) } { x _ { 1 } - \\phi _ { a } } \\right\\rangle \\left\\langle \\frac { P _ { m } ( x _ { 2 } ) \\tilde { Q } _ { n } ( y _ { 2 } ) } { y _ { 2 } - \\chi _ { b } } \\right\\rangle = \\left\\langle \\frac { P _ { n } ( x _ { 1 } ) \\tilde { Q } _ { n } ( y _ { 1 } ) } { ( x _ { 1 } - \\phi _ { a } ) ( y _ { 1 } - \\chi _ { b } ) } \\right\\rangle\n+ e _ { 2 } \\overline { { \\Phi } } ( z , t _ { 2 } | y , t _ { 1 } ) ( { \\gamma } ^ { 0 } \\otimes { \\gamma } ^ { 0 } ) { \\Phi } ( z , t _ { 2 } | y , t _ { 1 } ) \\} ,\nV ( r ) \\sim \\left\\{ \\begin{array} { c l } { - 2 k ^ { 2 } / r ^ { 6 } } & { \\textrm { s c a l a r p e r t u r b a t i o n } } \\\\ { + 4 k ^ { 2 } / r ^ { 6 } } & { \\textrm { e l e c t r o m a g n e t i c - g r a v i t a t i o n a l p e r t u r b a t i o n } } \\\\ \\end{array} \\right.\n\\, \\, \\, \\, \\partial ^ { \\mu } A _ { \\mu } \\, = \\, 0 \\, \\, \\, \\, \\, ; \\, \\, \\, F _ { \\mu \\nu } \\, + \\, { \\tilde { F } } _ { \\mu \\nu } \\, = \\, 0 \\, \\, \\, \\, \\, ; \\, \\, \\, \\partial ^ { \\mu } \\Psi _ { \\mu } \\, = \\, 0\nF = F ( u _ { 1 } + u _ { 3 } u _ { 2 } , u _ { 3 } ) , \\quad \\alpha ( u _ { 1 } + u _ { 3 } u _ { 2 } , \\lambda )\n{ \\cal L } _ { \\mathrm { e f f } } = T \\, ( \\partial _ { \\mu } y ( x ) ) ( \\partial ^ { \\mu } y ( x ) ) \\, .\n( \\gamma ^ { 0 } + \\epsilon ) \\xi = ( \\gamma ^ { 0 } - \\epsilon ) \\eta = 0 .\n\\rho _ { \\nu } ( x ) = \\rho _ { \\nu ^ { \\prime } } ( x ) \\frac { q ^ { C ( \\nu ) / 2 } + \\epsilon ( \\nu ) \\epsilon ( \\nu ^ { \\prime } ) x q ^ { C ( \\nu ^ { \\prime } ) / 2 } } { x q ^ { C ( \\nu ) / 2 } + \\epsilon ( \\nu ) \\epsilon ( \\nu ^ { \\prime } ) q ^ { C ( \\nu ^ { \\prime } ) / 2 } } \\; , ~ ~ ~ ~ \\forall \\nu \\neq \\nu ^ { \\prime } .\nA _ { \\mu } ^ { a } \\frac { \\sigma ^ { a } } { 2 } = \\bar { \\Sigma } _ { \\mu \\nu } J _ { \\nu } [ \\Phi ]\nB _ { a b } ( \\tau _ { 1 } , \\tau _ { 2 } ) = \\Bigl [ \\delta ( \\tau _ { 1 } - \\tau _ { a } ) - \\delta ( \\tau _ { 1 } - \\tau _ { b } ) \\Bigr ] \\Bigl [ \\delta ( \\tau _ { 2 } - \\tau _ { a } ) - \\delta ( \\tau _ { 2 } - \\tau _ { b } ) \\Bigr ] \\quad .\ne _ { f } ^ { \\mu } = \\left( { \\frac { 1 } { 2 } } \\left( e ^ { \\mu [ - 2 ] } + e ^ { \\mu [ + 2 ] } \\right) , { \\frac { 1 } { 2 } } \\left( e ^ { \\mu [ - 2 ] } - e ^ { \\mu [ + 2 ] } \\right) \\right)\nG ( z \\vert \\tau ) = \\operatorname { l o g } \\chi \\ , \\ \\ \\chi = 2 \\pi e ^ { - \\pi y ^ { 2 } / \\tau _ { 2 } } \\Bigl \\vert { \\frac { \\theta _ { 1 } ( z \\vert \\tau ) } { \\theta _ { 1 } ^ { \\prime } ( 0 \\vert \\tau ) } } \\Bigr \\vert \\ \\ ,\n\\langle { \\bf \\Psi } , { \\bf \\Xi } \\rangle = \\frac 1 2 \\int _ { - \\infty } ^ { \\infty } [ \\psi ^ { * } ( x ) \\operatorname { t a n } ( - i \\partial _ { x } ) \\xi ( x ) - \\xi ( x ) \\operatorname { t a n } ( - i \\partial _ { x } ) \\psi ^ { * } ( x ) ] \\; d x\n\\triangle _ { \\alpha \\beta } ^ { a b } ( x , y ) + \\int d ^ { 3 } w ~ d ^ { 3 } z ~ X _ { \\alpha \\gamma } ^ { a c } ( x , w ) \\omega _ { c d } ^ { \\gamma \\delta } ( w , z ) X _ { \\beta \\delta } ^ { b d } ( y , z ) = 0 .\nd s _ { D W } ^ { 2 } = H ( z ) ^ { - \\frac { 2 \\alpha } { \\epsilon } } \\left( d x ^ { \\mu } \\otimes d x ^ { \\nu } \\eta _ { \\mu \\nu } \\right) + H ( z ) ^ { - \\frac { 2 \\beta + \\epsilon } { \\epsilon } - 2 } \\, \\frac { d z ^ { 2 } } { \\epsilon ^ { 2 } }\n1 6 \\pi { \\cal L } = - e ^ { - { \\cal K } } R + \\dots\n\\dot { \\operatorname { l n } { \\frac { [ i ] } { [ i - 1 ] } } } + ( \\operatorname { l n } ( [ i ] [ i - 1 ] ) ^ { \\prime \\prime } + ( ( \\operatorname { l n } { \\frac { [ i ] } { [ i - 1 ] } } ^ { \\prime } ) ^ { 2 } = 2 ( \\bar { \\alpha } _ { i - 1 , i } + \\bar { \\alpha } _ { i , i - 1 } - \\bar { \\alpha } _ { i - 1 } \\bar { \\alpha } _ { i } )\n\\int \\Phi _ { \\Xi } ^ { \\dagger } ( \\xi , \\xi ^ { * } ) \\Phi _ { H } ( \\xi , \\xi ^ { * } ) d v ( \\xi , \\xi ^ { * } ) = \\left( \\Xi \\mid H \\right)\nd s _ { 1 0 } ^ { 2 } = h ^ { - 1 / 2 } ( \\rho ) d x ^ { \\mu } d x ^ { \\mu } + h ^ { 1 / 2 } ( \\rho ) d s _ { 6 } ^ { 2 } ,\n\\epsilon ^ { \\delta \\gamma \\alpha _ { 1 } \\beta _ { 1 } } D _ { \\gamma } ^ { i } \\; { \\cal O } _ { \\alpha _ { 1 } \\ldots \\alpha _ { J _ { 2 } } \\; \\beta _ { 1 } \\ldots \\beta _ { J _ { 2 } } } = 0 \\, .\n\\{ X ^ { M } \\} = \\{ x ^ { \\mu } , y ^ { i } \\} , \\quad D _ { i } ( X ) \\equiv y ^ { i } .\nT _ { n } = \\frac { m ^ { 2 } } { 2 } \\sum _ { k = 1 } ^ { \\infty } \\frac { a _ { n } ^ { \\dagger } ( k ) a _ { n } ( k ) } { k - \\frac { 1 } { 2 } }\n\\sum _ { n = 1 / 2 , \\cdots } { \\frac { \\Delta p ^ { + } } { p _ { n } ^ { + } } } = \\sum _ { n = n _ { \\mathrm { I R } } + 1 } ^ { n _ { \\mathrm { U V } } - 1 } \\frac { 1 } { n } \\simeq \\mathrm { l n } \\frac { n _ { \\mathrm { U V } } } { n _ { \\mathrm { I R } } } \\simeq \\mathrm { l n } \\frac { 2 \\Lambda ^ { 2 } } { M ^ { 2 } + { \\bf p } _ { \\bot } ^ { 2 } } ,\n\\Gamma ( s + 2 n - 1 ) \\zeta _ { R } ( s ) \\zeta ( \\frac { s + 2 n - 1 } { 2 } | \\L _ { 3 } ) \\; \\beta ^ { - s } \\: , \\nonumber\n\\left\\{ \\begin{array} { c c c } { - \\lambda ^ { 2 } x ^ { + } \\left( x ^ { - } + \\Delta \\right) } & { = } & { { \\frac { \\kappa } { 4 } } } \\\\ { - \\lambda ^ { 2 } x ^ { + } x ^ { - } } & { = } & { { \\frac { \\kappa } { 4 } } \\, e ^ { \\frac { 4 M } { \\kappa \\lambda } } \\; . } \\\\ \\end{array} \\right.\n: \\psi _ { r , a } ^ { * } \\psi _ { s , b } : = \\psi _ { r , a } ^ { * } \\psi _ { s b } - \\langle 0 | \\psi _ { r , a } ^ { * } \\psi _ { s b } | 0 \\rangle ,\n{ \\cal M } ( \\theta ) = \\left( \\begin{array} { c c } { \\operatorname { c o s } \\theta } & { \\operatorname { s i n } \\theta } \\\\ { - \\operatorname { s i n } \\theta } & { \\operatorname { c o s } \\theta } \\\\ \\end{array} \\right)\nd s ^ { 2 } = ( 1 - \\frac { N ^ { 4 } } { r ^ { 4 } } ) ( \\frac { r } { 8 N } ) ^ { 2 } ( d \\tau + 4 N \\operatorname { c o s } \\theta d \\phi ) ^ { 2 } + ( 1 - \\frac { N ^ { 4 } } { r ^ { 4 } } ) ^ { - 1 } d r ^ { 2 } + \\frac { 1 } { 4 } r ^ { 2 } d \\Omega ^ { 2 } .\nx _ { 1 } = \\frac { 1 } { 2 } \\left( x _ { + } + x _ { - } \\right) , \\quad x _ { 2 } = \\frac { \\imath } { 2 } \\left( x _ { - } - x _ { + } \\right) , \\quad x _ { 3 } = \\frac { 1 } { 2 } h ,\n\\phi ( \\sigma ) = U \\, e ^ { i \\, ( N \\sigma + \\varphi ( \\sigma ) ) } ,\nE _ { | 2 } \\left( m , \\lambda \\right) = 2 \\frac V { 2 \\pi ^ { 2 } } \\sum _ { l = 0 } ^ { \\infty } \\sum _ { i = 1 } ^ { 2 } \\int _ { 0 } ^ { \\infty } d p p ^ { 2 } \\sqrt { E _ { i } ^ { 2 } \\left( p , m , l \\right) } ,\nI ( d , a , c ) = \\int _ { 0 } ^ { \\infty } { \\frac { x \\; d x } { ( x ^ { 2 } + d ^ { 2 } ) + a ( x ^ { 2 } + d ^ { 2 } ) ^ { 1 / 2 } } } { \\frac { 1 } { x ^ { 2 } + c ^ { 2 } } }\n\\triangle ^ { s y m } ( t ) = - t ^ { * } \\times 1 + 1 \\times t ,\nR _ { k } ^ { ( N _ { f } ) } ( x _ { 1 } , \\ldots , x _ { k } , M ) \\ = \\ \\operatorname* { d e t } \\left[ K _ { N } ^ { ( N _ { f } ) } ( x _ { i } , x _ { j } , M ) \\right] _ { i , j = 1 , \\ldots , k }\n\\Omega _ { 3 } = { \\frac { 1 } { 3 R ^ { 2 } } } \\varepsilon _ { \\hat { m } \\hat { n } \\hat { p } \\hat { q } } X ^ { \\hat { m } } d X ^ { \\hat { n } } d X ^ { \\hat { p } } d X ^ { \\hat { q } } \\, , \\qquad \\tilde { \\Omega } _ { 3 } = { \\frac { 1 } { 3 R ^ { 2 } } } \\varepsilon _ { \\hat { m } ^ { \\prime } \\hat { n } ^ { \\prime } \\hat { p } ^ { \\prime } \\hat { q } ^ { \\prime } } X ^ { \\hat { m } ^ { \\prime } } d X ^ { \\hat { n } ^ { \\prime } } d X ^ { \\hat { p } ^ { \\prime } } d X ^ { \\hat { q } ^ { \\prime } } \\, ,\nx = \\beta | \\stackrel { \\rightharpoonup } { l } | , \\ y = \\beta m \\ \\mathrm { a n d } \\ r = \\beta \\mu .\n\\tilde { \\Psi } _ { \\nu } ^ { \\prime } = - i \\bigl ( \\nu + { \\bf M } \\delta ( \\sigma ) \\bigr ) \\tilde { \\Psi } _ { \\nu } .\nE = { \\frac { 1 } { n ! } } \\int d \\Omega \\left[ E _ { 1 } \\left| Y _ { a _ { 1 } \\cdots a _ { n } } ^ { ( l ) } \\right| ^ { 2 } + E _ { 2 } \\left| Y _ { a _ { 2 } \\cdots a _ { n } } ^ { ( l ) } \\right| ^ { 2 } + E _ { 3 } \\left| Y _ { a _ { 3 } \\cdots a _ { n } } ^ { ( l ) } \\right| ^ { 2 } \\right] .\n{ H _ { \\bar { \\partial } } ^ { p , p } } ( { { \\mathcal P } ^ { n } } ) \\cong { H _ { D R } ^ { 2 p } } \\cong C\nR _ { i j k l } ^ { ( + ) } = \\frac { 1 } { 2 } g _ { i m } g _ { j n } \\phi ^ { m n p q } R _ { p q k l } ^ { ( + ) } .\n- i A ^ { i j } E ^ { i j } = S q r t { 2 } ( A _ { L } ^ { l } \\alpha _ { L } ^ { l } + A _ { R } ^ { r } \\alpha _ { R } ^ { r } )\nM _ { 1 1 } ^ { \\beta \\gamma } = \\frac { T ^ { \\beta \\gamma } } { 1 - C I ^ { \\beta \\gamma \\, - 1 } T ^ { \\beta \\gamma } + T ^ { \\beta \\gamma \\, 2 } } = \\frac { ( C I ^ { \\beta \\gamma } ) ^ { 5 } - C I ^ { \\beta \\gamma } } { ( 1 - ( C I ^ { \\beta \\gamma } ) ^ { 2 } ) ( 1 - 3 ( C I ^ { \\beta \\gamma } ) ^ { 2 } ) }\nV ( \\varphi _ { 0 } ) = 4 \\sum _ { s = 1 } ^ { \\infty } \\int \\frac { d ^ { 4 } p } { ( 2 \\pi ) ^ { 4 } } \\frac { ( - 1 ) ^ { s } } { 2 s } \\frac { ( M + g \\varphi _ { 0 } ) ^ { 2 s } } { ( p _ { E } ^ { 2 } ) ^ { s } } .\nF ( \\phi ) = \\int \\frac { d ^ { D } p } { ( 2 \\pi ) ^ { D } } \\frac { 1 } { p ^ { 2 } + M ^ { 2 } ( \\phi ) } ,\n\\delta \\Psi _ { \\hat { a } } = \\partial _ { \\hat { a } } \\epsilon + { \\frac { 1 } { 8 } } H ^ { - 1 } ( \\partial _ { \\hat { a } } H ) \\epsilon + { \\frac { 1 } { 8 } } W \\gamma ^ { \\underline { { u x y } } } \\gamma _ { \\underline { { \\hat { a } } } } ( i \\sigma _ { 2 } ) \\epsilon = 0 \\,\nI _ { G } = \\int \\sum _ { p = 0 } ^ { [ d / 2 ] } \\alpha _ { p } L ^ { ( p ) } ,\n\\Pi _ { \\mu \\nu } ^ { ( I ) } = { \\frac { e ^ { 2 } C ^ { ( I ) } } { 2 } } \\, \\int { \\frac { d ^ { n } k } { ( 2 \\pi ) ^ { n } } } \\, { \\frac { ( 1 - \\operatorname { c o s } p \\times k ) } { k ^ { 2 } ( k + p ) ^ { 2 } } } \\, N _ { \\mu \\nu } ^ { ( I ) } ; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; I = a , b , c\n\\Big \\langle \\frac { 1 } { 4 } \\int d ^ { 4 } x \\, d ^ { 4 } \\theta \\, \\Big ( \\phi ^ { * } e ^ { 2 V } \\phi + \\tilde { \\phi } ^ { * } e ^ { - 2 V } \\tilde { \\phi } \\Big ) \\Big \\rangle = 0 .\n{ \\cal L } = \\Psi = P \\{ \\mu \\} \\ .\nq _ { ( 1 ) } = q _ { i } \\delta _ { i } ^ { j } \\qquad \\bar { q } _ { ( 1 ) } = \\bar { q } _ { i } \\delta _ { i } ^ { j } \\qquad T _ { ( 1 ) } = T _ { i } \\delta _ { i } ^ { j }\nY = 2 [ ( x _ { 1 2 } x _ { 3 4 } ) - 2 \\frac { ( x _ { 1 2 } x _ { 2 4 } ) ( x _ { 3 4 } x _ { 2 4 } ) } { x _ { 2 4 } ^ { 2 } } ] \\frac { 1 } { x _ { 2 4 } ^ { 2 } } ( 1 + \\mathcal { O } ( x _ { 1 2 } , x _ { 3 4 } ) )\n{ \\cal S } _ { E } = \\int _ { 0 } ^ { \\tau } d t \\int _ { 0 } ^ { \\pi } d \\sigma \\left( \\frac { \\rho } { 2 } \\left( \\frac { \\partial X } { \\partial t } \\right) ^ { 2 } + \\frac { \\nu } { 2 } \\left( \\frac { \\partial X } { \\partial \\sigma } \\right) ^ { 2 } \\right) .\n\\delta B _ { \\hat { 0 } \\alpha } ^ { g } = \\frac { 1 } { \\sqrt { a } } { \\bar { e } } _ { \\alpha } ^ { \\; \\; i } \\left( \\delta { \\beta } _ { o i } ^ { g } - v _ { i } \\delta \\beta ^ { a } \\right) .\n\\begin{array} { r l } { \\hat { U } ^ { - 1 } ( k _ { 1 } , k _ { 2 } ) } & { | A _ { \\bar { z } } ( z , \\bar { z } ) > = } \\\\ { = } & { e ^ { i \\pi k k _ { 1 } k _ { 2 } } \\, e ^ { - \\frac { 1 } { 2 } i k \\left[ ( k _ { 1 } \\tau - k _ { 2 } ) \\overline { { A _ { \\bar { z } } ^ { + + } ( 0 , 0 ) } } + ( k _ { 1 } \\bar { \\tau } - k _ { 2 } ) A _ { \\bar { z } } ^ { + + } ( 0 , 0 ) \\right] } \\| A _ { \\bar { z } } ( z , \\bar { z } ) - \\frac { i \\pi } { \\tau _ { 2 } } ( \\tau k _ { 1 } - k _ { 2 } ) > \\ \\ \\ , } \\\\ \\end{array}\n\\chi [ { \\cal M } _ { \\alpha } ] = { \\frac { 1 } { 3 2 \\pi ^ { 2 } } } \\int _ { { \\cal M } _ { \\alpha } / \\Sigma } ( R ^ { 2 } - 4 R _ { \\mu \\nu } ^ { 2 } + R _ { \\mu \\nu \\alpha \\beta } ^ { 2 } ) + ( 1 - \\alpha ) \\chi [ \\Sigma ] ~ ~ ~ ,\nS = \\frac { - 8 } { \\kappa ^ { 2 } } \\int d t d ^ { 2 } \\theta d ^ { 2 } \\overline { { \\theta } } { I \\! \\! N } ^ { - 1 } A ( { I \\! \\! N } { \\bf Q } ^ { \\mu } ) ,\nH _ { Q ( t , z ) } ^ { S , l } \\cong H _ { Q ( t ) } ^ { S } \\otimes C [ [ z ] ] \\cong \\hat { H } _ { Q ( t , z ) } ^ { S } ,\nn ( \\omega ) = { \\frac { 1 + e ^ { - \\alpha ^ { 2 } \\omega ^ { 2 } } } { 1 - e ^ { - \\alpha ^ { 2 } \\omega ^ { 2 } } } } .\nQ = \\frac { 1 } { C } \\int _ { T _ { 2 } } W ^ { * } \\omega ,\na + b - 1 + 2 \\sum _ { j = 1 } ^ { r - 2 } j \\, n _ { j } = L + 1 .\n\\hat { R } _ { 1 2 } ( q ^ { - 1 } ) K _ { 1 } ^ { \\prime } \\hat { R } _ { 1 2 } ( q ^ { - 1 } ) K _ { 1 } ^ { \\prime } = K _ { 1 } ^ { \\prime } \\hat { R } _ { 1 2 } ( q ^ { - 1 } ) K _ { 1 } ^ { \\prime } \\hat { R } _ { 1 2 } ( q ^ { - 1 } ) \\quad ,\n\\psi = \\sqrt \\rho _ { 0 } e ^ { i n \\theta } , \\quad A _ { 0 } = 0 , \\quad { \\bf A } = \\frac { n } { e } \\frac { { \\bf e } _ { \\theta } } { r }\nT ^ { M } \\cdot T ^ { N } - ( M \\leftrightarrow N ) = f ^ { M N K } T ^ { K }\nQ _ { U } = \\int _ { U } j _ { p } = \\int _ { U } d ( d C _ { p } ) = \\int _ { \\partial U } d C _ { p } = \\int _ { \\partial U } G _ { p + 1 } - H \\wedge C _ { p - 2 } .\n( \\partial p ) ^ { * } \\partial \\Theta , \\partial g\n\\beta F _ { \\mathrm { b o s e } } = 9 N ^ { 2 } \\operatorname { l o g } [ 2 \\operatorname { s i n h } ( 2 \\beta R _ { \\mathrm { r m s } } ) ]\nS _ { \\mathrm { m a s s i v e \\ I I A } } ( C ^ { ( 9 ) } ) \\sim \\int d ^ { 1 0 } x { \\sqrt { | g | } } \\left\\{ e ^ { - 2 \\phi } \\left[ R - 4 ( \\partial \\phi ) ^ { 2 } \\right] - \\frac { 1 } { 2 \\times 1 0 ! } ( G ^ { ( 1 0 ) } ) ^ { 2 } + \\cdots \\right\\} \\, .\nm , n = 1 \\ldots d ; \\; \\; \\; \\; \\; \\; a , b , c = 1 \\ldots g ; \\; \\; \\; \\; \\; i , j , k = 1 \\ldots g \\; \\; \\; \\; \\; \\; ( g \\leq d )\n[ \\hat { \\eta } _ { m } ^ { ( i ) \\mu } , \\hat { \\eta } _ { n } ^ { \\dagger ( j ) \\nu } ] = G ^ { \\mu \\nu } \\delta ^ { i , j } \\delta _ { m , n } ~ .\n\\Phi ^ { 2 } = 2 ( { \\frac { d - 3 } { d - 2 } } ) q \\Phi _ { h } \\phi ~ ~ , ~ ~ \\gamma _ { a b } = ( { \\frac { \\phi _ { h } } { \\phi } } ) ^ { \\frac { d - 3 } { d - 2 } } e ^ { { \\frac { 2 } { q \\Phi _ { h } } } \\phi } \\bar { \\gamma } _ { a b } ~ ~\nC ( \\theta ) = \\left\\{ \\begin{array} { l } { \\operatorname { c o t } { \\theta } \\hspace { 0 . 5 c m } \\mathrm { f o r } \\hspace { 0 . 3 c m } k = + 1 , } \\\\ { 0 \\hspace { 0 . 5 c m } \\mathrm { f o r } \\hspace { 0 . 3 c m } k = 0 , } \\\\ { \\operatorname { c o t h } { \\theta } \\hspace { 0 . 5 c m } \\mathrm { f o r } \\hspace { 0 . 3 c m } k = - 1 , } \\\\ \\end{array} \\right. \\\n\\eta ^ { l m } \\partial _ { i } \\partial _ { j } \\partial _ { m } F _ { M } \\partial _ { l } \\partial _ { k } \\partial _ { n } F _ { M } = \\eta ^ { l m } \\partial _ { j } \\partial _ { k } \\partial _ { m } F _ { M } \\partial _ { i } \\partial _ { l } \\partial _ { n } F _ { M }\nd s _ { n + 3 } ^ { 2 } = e ^ { - 2 n L \\rho } d x ^ { 2 } + e ^ { 4 L \\rho } ( 2 d x d t + \\sum _ { i = 1 } ^ { n } { d y ^ { i } } ^ { 2 } ) + d \\rho ^ { 2 }\nJ _ { 5 } ^ { \\mu } = - \\frac { e ^ { 2 } } { 8 \\pi ^ { 2 } } \\epsilon ^ { \\mu \\nu \\lambda \\sigma } A _ { \\nu } F _ { \\lambda \\sigma }\n\\hat { D } ( \\hat { u } ) = \\lambda ^ { - 1 } \\hat { Q } ( \\hat { u } )\nh ( r ) = \\eta \\left( 1 - \\sqrt { \\frac { \\lambda _ { 2 } } { 2 \\lambda _ { 1 } } } \\eta r K _ { 1 } ( 2 \\sqrt { \\frac { \\lambda _ { 2 } } { \\lambda _ { 1 } } } \\eta r ) \\right) ,\n\\partial _ { \\tau } \\langle x , \\tau \\vert x ^ { \\prime } , 0 \\rangle = - \\langle x , \\tau \\vert H \\vert x ^ { \\prime } , 0 \\rangle \\; ,\n{ \\cal L } _ { S G } = \\frac { 1 } { 2 } \\partial _ { \\mu } \\varphi \\partial _ { \\mu } \\varphi -\nE [ f ^ { ( 0 ) } + f ^ { ( 1 ) } ] = E [ f ^ { ( 0 ) } ] + \\frac { \\delta E } { \\delta f } [ f ^ { ( 0 ) } ] \\cdot f ^ { ( 1 ) } + \\frac { 1 } { 2 } \\frac { \\delta ^ { 2 } E } { \\delta f ^ { 2 } } [ f ^ { ( 0 ) } ] \\cdot ( f ^ { ( 1 ) } ) ^ { 2 } + \\cdots ,\n{ \\cal L } _ { c l a s s } = { \\cal L } _ { c l a s s } ( \\phi , \\alpha ) = \\sum _ { i } \\lambda _ { i } ( \\alpha ) \\, { \\cal G } _ { i } .\n\\Psi _ { E } ^ { ( - ) } = \\left( \\begin{array} { l } { e ^ { i \\frac { \\theta } { 2 } } \\sum f _ { n } e ^ { i \\left( n + \\frac { 1 } { 2 } \\right) \\theta } } \\\\ { e ^ { - i \\frac { \\theta } { 2 } } \\sum g _ { n } e ^ { i \\left( n + \\frac { 1 } { 2 } \\right) \\theta } } \\\\ \\end{array} \\right)\n[ \\partial _ { x } , \\eta ] = 0 \\ \\ \\ \\ [ \\partial _ { y } , \\xi ] = h ^ { \\prime } \\xi \\partial _ { x } + h h ^ { \\prime } \\eta \\partial _ { x } + h \\eta \\partial _ { y }\n\n\\Gamma _ { \\underline { \\alpha } \\underline { \\beta } } ^ { i } = \\left( \\begin{array} { c c } { 0 } & { \\delta _ { \\beta } ^ { \\alpha } ( \\gamma ^ { i } ) _ { q } ^ { ~ r } } \\\\ { - \\delta _ { \\alpha } ^ { \\beta } ( \\gamma ^ { i } ) _ { ~ r } ^ { q } } & { 0 } \\\\ \\end{array} \\right) \\, , \\quad i = 1 , . . . , 5 \\, ,\nR _ { i j } = \\frac { E ( z _ { i } , z _ { j } ) E ( w _ { i } , w _ { j } ) } { E ( z _ { i } , w _ { j } ) E ( w _ { i } z _ { j } ) }\n\\Delta _ { \\varepsilon , \\overline { { \\varepsilon } } } ^ { ( 1 ) } = \\frac { \\sigma ^ { 2 } } 2 \\cdot \\frac { m - i \\varepsilon \\widehat { p } } { 2 m } \\cdot \\frac { m - i \\overline { { \\varepsilon } } \\overline { { p } } } { 2 m } \\cdot \\left( 1 - \\sigma _ { p } ^ { 2 } \\right) = \\Psi _ { \\varepsilon , \\overline { { \\varepsilon } } } \\cdot \\overline { { \\Psi } } _ { \\varepsilon , \\overline { { \\varepsilon } } }\n\\gamma _ { 5 } \\Gamma _ { 5 } \\gamma _ { 5 } D + D \\Gamma _ { 5 } = 0\n\\vert j , L = M = 0 \\rangle = { \\frac { 1 } { \\sqrt { 2 j + 1 } } } \\; \\sum _ { m \\; = - j } ^ { j } \\vert j , m , - m \\rangle\nc ^ { ( 0 ) } \\delta ( \\nu ) \\equiv \\operatorname* { l i m } _ { t \\to 0 } { \\frac { c ^ { ( 0 ) } ( \\nu , t ) } { \\nu ^ { n - 2 } } } \\quad .\nJ _ { 0 } ^ { \\pm } | j m > = \\sqrt { m ( m \\pm 1 ) - j ( j + 1 ) } \\; | j m \\pm 1 >\n\\delta m = \\sqrt { \\frac { j ( j + 1 ) } { d } } = \\frac { \\sqrt { j ( j + 1 ) } } { r } ,\n\\tilde { G } _ { \\mu \\nu } ^ { 0 A } - \\tilde { G } _ { \\mu \\nu } ^ { 0 L } = - { \\frac { 1 } { k ^ { 2 } } } k _ { \\mu } k _ { \\nu } \\biggl ( { \\frac { ( \\lambda k ^ { 2 } + \\eta ^ { 2 } ) } { ( \\eta \\cdot k ) ^ { 2 } } } + { \\frac { ( 1 - \\lambda ) } { k ^ { 2 } } } \\biggr ) + { \\frac { k _ { [ \\mu } \\eta _ { \\nu ] _ { + } } } { k ^ { 2 } \\eta \\cdot k } } .\nS ^ { 2 } ( p , q ) \\, = \\, f ( p , q ) \\, = \\, - ( p ^ { 0 } q _ { 0 } - p ^ { 1 } q _ { 1 } + p ^ { 2 } q _ { 2 } + p ^ { 3 } q _ { 3 } ) ^ { 2 } + 4 ( p ^ { 2 } p ^ { 3 } - p ^ { 0 } q _ { 1 } ) ( p ^ { 1 } q _ { 0 } + q _ { 2 } q _ { 3 } )\n\\widetilde { \\Omega } _ { i } ^ { ( 1 ) } ( x ) = \\int d y X _ { i j } ( x , y ) \\Phi ^ { j } ( y ) .\nR _ { S } ^ { 0 } ( F ^ { \\otimes n } , G ) = \\int d ^ { n } x ( { \\mathcal R } ( x _ { \\pi _ { t } ( 1 ) } ) \\ldots { \\mathcal R } ( x _ { \\pi _ { t } ( n ) } ) G ) _ { S } \\ .\nZ ^ { I } = \\sum _ { a = 1 } ^ { 1 6 } { \\frac { q ^ { I } { } _ { a } } { | \\vec { x } - \\vec { x } _ { a } | } } \\, \\ , \\qquad \\sum _ { a = 1 } ^ { 1 6 } q ^ { I } { } _ { a } = 0 \\ ,\n\\beta _ { i \\bar { \\jmath } } = - \\frac 1 { 2 \\pi } R _ { i \\bar { \\jmath } } - \\frac { 4 \\zeta ( 3 ) } { 3 ( 4 \\pi ) ^ { 4 } } \\partial _ { i } \\partial _ { \\bar { \\jmath } } \\left[ R _ { k \\bar { l } m \\bar { n } } R ^ { p \\bar { l } q \\bar { n } } R _ { p } { } ^ { k } { } _ { q } { } ^ { m } - R _ { k \\bar { l } m \\bar { n } } R ^ { m \\bar { n } p \\bar { q } } R _ { p \\bar { q } } { } ^ { k \\bar { l } } \\right] + \\ldots\n\\{ F ^ { \\sigma \\nu } , x ^ { \\mu } \\} - \\{ F ^ { \\sigma \\mu } , x ^ { \\nu } \\} = \\{ F ^ { \\mu \\nu } , x ^ { \\sigma } \\} .\nM _ { U \\ \\scriptstyle \\mathrm { m i n } } ^ { E _ { 8 } - E _ { 6 } } \\approx 5 . 4 9 \\times 1 0 ^ { 1 7 } \\times g _ { U } ^ { E _ { 8 } - E _ { 6 } } \\times \\frac { 1 } { \\sqrt { 1 + 0 . 0 8 \\times \\left( g _ { U } ^ { E _ { 8 } - E _ { 6 } } \\right) ^ { 2 } } } \\, \\mathrm { G e V } \\, .\n\n\\left. \\begin{array} { l } { ( \\gamma ) = { } ^ { g _ { 4 } } [ S _ { 4 } ] \\hspace { 2 m m } { } ^ { g _ { 3 } } [ S _ { 3 } ] \\hspace { 2 m m } { } ^ { g _ { 2 } } [ S _ { 2 } ] \\hspace { 2 m m } { } ^ { g _ { 1 } } [ S _ { 1 } ] \\hspace { 2 m m } [ \\xi ] } \\\\ { \\hspace { 6 m m } = [ \\xi ] \\hspace { 2 m m } { } ^ { h _ { 4 } } [ S _ { 4 } ] \\hspace { 2 m m } { } ^ { h _ { 3 } } [ S _ { 3 } ] \\hspace { 2 m m } { } ^ { h _ { 2 } } [ S _ { 2 } ] \\hspace { 2 m m } { } ^ { h _ { 1 } } [ S _ { 1 } ] , } \\\\ \\end{array} \\right.\n\\frac { \\partial \\xi _ { 1 } } { \\partial x _ { 0 } } + \\xi _ { 1 } \\frac { \\partial \\xi _ { 1 } } { \\partial x _ { 1 } } = 0 , \\quad x _ { 1 } - \\xi _ { 1 } x _ { 0 } = F ( \\xi _ { 1 } )\nd f _ { i } = d Q _ { i } \\frac { \\partial f ( Q _ { i } ) } { \\partial Q _ { i } } \\; .\n\\phi _ { a } \\phi _ { a } = \\phi _ { 1 } ^ { 2 } + \\phi _ { 2 } ^ { 2 } + \\phi _ { 3 } ^ { 2 } = 1 .\n\\left( \\tilde { \\alpha } _ { J } \\right) _ { k } = \\frac { 1 } { \\sqrt { n } } \\sum _ { i = 1 } ^ { n }\nT _ { i k } = ( 1 - 2 \\zeta ) \\partial _ { i } \\varphi \\partial _ { k } \\varphi + ( 2 \\zeta - 1 / 2 ) g _ { i k } \\partial ^ { l } \\varphi \\partial _ { l } \\varphi - 2 \\zeta \\varphi \\nabla _ { i } \\nabla _ { k } \\varphi + ( 1 / 2 - 2 \\zeta ) m ^ { 2 } g _ { i k } \\varphi ^ { 2 } .\n[ X _ { J } , [ X ^ { J } , X _ { [ K } ] ] n _ { L ] } + { \\frac { 1 } { 4 } } \\{ \\bar { \\lambda } , \\Gamma _ { K L } \\lambda \\} = 0 .\nC ( k , p ) = \\left( \\begin{array} { c c c c c c c } { 0 } & { 0 } & { i k \\delta ( \\bar { k } + \\bar { p } ) } & { 0 } & { 0 } & { 0 } & { 0 } \\\\ { 0 } & { - 2 i \\delta ( \\bar { k } + \\bar { p } ) } & { 0 } & { 0 } & { 0 } & { 0 } & { 0 } \\\\ { i \\delta ( \\bar { k } + \\bar { p } ) } & { 0 } & { 0 } & { 0 } & { 0 } & { 0 } & { 0 } \\\\ { 0 } & { 0 } & { 0 } & { 0 } & { 0 } & { - \\delta ( 0 ) } & { 0 } \\\\ { 0 } & { 0 } & { 0 } & { 0 } & { 0 } & { 0 } & { - \\delta ( 0 ) } \\\\ { 0 } & { 0 } & { 0 } & { \\delta ( 0 ) } & { 0 } & { 0 } & { 0 } \\\\ { 0 } & { 0 } & { 0 } & { 0 } & { \\delta ( 0 ) } & { 0 } & { 0 } \\\\ \\end{array} \\right) \\, ,\n\\Gamma _ { \\mathrm { e f f } } ^ { \\mathrm { P V } } = \\Gamma _ { \\mathrm { e f f } } ^ { \\mathrm { P V } , ( 1 ) } + \\Gamma _ { \\mathrm { e f f } } ^ { \\mathrm { P V } , \\, ( \\vec { E } ) }\nD _ { a } D ^ { a } \\phi + \\cdots = \\frac { \\partial V ( T ) } { \\partial T }\n\\partial _ { t } Q = \\frac { \\delta { \\cal H } ( Q , P ) } { \\delta P } \\quad \\partial _ { t } P = - \\frac { \\delta { \\cal H } ( Q , P ) } { \\delta Q } .\n2\n\\left[ m _ { 0 } ^ { D - 2 } - K _ { c r } ^ { \\frac { D - 2 } { 2 } } \\frac { 1 } { \\sqrt { \\pi } } \\Gamma ( \\frac { D - 1 } { 2 } ) \\Gamma ( \\frac { D } { 2 } ) \\right] \\Gamma ( 1 - \\frac { D } { 2 } ) = 0 .\n\\dot { b } = \\frac { 1 } { e ^ { t ( | y | ) - N + 3 A } } = \\frac { 1 } { e ^ { t ( | y | ) + 2 k F + 3 J } } ~ ,\n\\Phi _ { 1 } ^ { G } = e + \\zeta \\pi _ { 0 } ^ { - 1 } \\; , \\; \\; \\Phi _ { 2 } ^ { G } = \\chi \\; , \\; \\; \\Phi _ { 3 \\mu } ^ { G } = b _ { \\mu } \\; , \\; \\; \\Phi _ { 4 } ^ { G } = x _ { 0 } - \\zeta \\tau \\; , \\; \\; \\Phi _ { 5 } ^ { G } = \\psi ^ { 0 } \\; , \\; \\;\n{ \\bf P } \\left( \\hat { a } _ { a _ { 1 } } ^ { \\dagger } \\ldots \\hat { a } _ { a _ { s } + 1 } ^ { \\dagger } A _ { \\; \\; \\quad \\quad b _ { 1 } \\ldots b _ { s } } ^ { a _ { 1 } \\ldots a _ { s + 1 } } \\hat { a } ^ { b _ { 1 } } \\ldots \\hat { a } ^ { b _ { s } } \\right) = \\hat { a } _ { a } ^ { \\dagger } K ^ { a }\ne _ { n } = \\gamma _ { n } \\, ; \\quad e _ { 7 } = i \\gamma _ { 5 } \\, ; \\quad e _ { n + 3 } = e _ { n } e _ { 7 } = i \\gamma _ { n } \\gamma _ { 5 } \\, ; \\qquad n = 1 , 2 , 3 \\, .\n\\langle \\Psi | \\Phi \\rangle = \\int { \\overline { \\Psi } } \\Phi e ^ { - \\langle Z , Z \\rangle } { \\cal D } Z { \\cal D } { \\overline { Z } } \\, .\n{ \\cal F } ( a , \\Lambda ) = { 8 \\pi i b _ { 1 } } \\Lambda ^ { 2 } a ^ { 2 } \\int _ { 4 \\Lambda / \\pi } ^ { a } d x { \\cal G } _ { 1 } ( x ) x ^ { - 3 } - { \\frac { i b _ { 1 } \\pi ^ { 3 } } { 4 } } a ^ { 2 } ,\n\\sum _ { B } \\, \\Phi _ { A B } \\ast L _ { B } ^ { \\pm } ( \\theta , r ) = r \\, M _ { A } \\operatorname { c o s h } \\theta + \\operatorname { l n } ( 1 - \\operatorname { e x p } ( - L _ { A } ^ { \\pm } ( \\theta , r ) ) ) \\quad \\, .\nA _ { \\mu } = \\frac { 1 } { 2 q } \\frac { i ( \\overline { { \\psi } } \\gamma _ { \\mu } \\slash { \\partial } \\psi - \\overline { { \\psi } } \\overleftarrow { \\slash { \\partial } } \\gamma _ { \\mu } \\psi ) - 2 m j _ { \\mu } } { \\overline { { \\psi } } \\psi } .\nx = a + b \\overline { { \\psi } } _ { 0 } \\psi _ { 0 } , \\qquad \\psi = c \\psi _ { 0 } + d \\overline { { \\psi } } _ { 0 } , \\qquad \\overline { { \\psi } } = c ^ { * } \\overline { { \\psi } } _ { 0 } + d ^ { * } \\psi _ { 0 } ,\n\\partial ^ { m } \\bar { \\chi } _ { m \\dot { \\alpha } } = \\partial ^ { m } \\chi _ { m \\alpha } = 0 ,\nU _ { x , i j } = \\operatorname { e x p } \\left( - \\textstyle { { \\frac { 1 } { 2 } } } i g a ^ { 2 } \\epsilon _ { i j k } B _ { x , k } ^ { a } \\tau ^ { a } \\right) \\, .\n\\zeta _ { s } ( \\Delta _ { X } ) - \\zeta _ { s } ( \\Delta _ { Y } ) + \\zeta _ { s } ( \\Delta _ { Z } ) = 0 , \\quad \\forall s .\nP _ { M } ^ { N } ( z ) = \\frac { \\sum _ { n = 0 } ^ { N } A _ { n } z ^ { n } } { \\sum _ { n = 0 } ^ { M } B _ { n } z ^ { n } }\nT ( u ) = \\left( \\begin{array} { c c } { 1 } & { - u ^ { 0 } \\vec { u } ^ { \\mathrm { T } } } \\\\ { \\hline 0 } & { I } \\\\ \\end{array} \\right)\np _ { \\mu } T _ { \\mu \\nu } ^ { S \\rightarrow A V } = 2 m i T _ { \\nu } ^ { S \\rightarrow P V }\n\\nabla _ { \\mu } \\phi _ { \\mu _ { 1 } \\dots \\mu _ { k } } = ( \\partial _ { \\mu } + \\omega _ { \\mu } ) \\phi _ { \\mu _ { 1 } \\dots \\mu _ { k } } - \\sum _ { i = 1 } ^ { k } \\Gamma _ { \\mu _ { i } \\mu } ^ { \\lambda } \\phi _ { \\mu _ { 1 } \\dots \\mu _ { i - 1 } \\lambda \\mu _ { i + 1 } \\dots \\mu _ { k } }\n\\vert \\mathrm { o p e n } > = J _ { - 1 } ^ { a } \\vert j >\nM = \\left( \\begin{array} { c c } { C } & { \\hat { 0 } } \\\\ { \\hat { 0 } } & { \\bar { C } } \\\\ \\end{array} \\right) ,\n\\{ \\hat { X } ^ { \\mu } , \\hat { S } ^ { \\nu \\lambda } \\} ^ { \\prime } = 2 \\frac { \\bar { P } ^ { \\nu } \\hat { S } ^ { \\lambda \\mu } - \\bar { P } ^ { \\lambda } \\hat { S } ^ { \\nu \\mu } } { ( P + \\bar { P } ) ^ { 2 } } - \\frac { 4 P ^ { \\mu } } { ( P + \\bar { P } ) ^ { 2 } } ( \\bar { P } ^ { \\lambda } D ^ { \\nu \\rho } A _ { \\rho } - \\bar { P } ^ { \\nu } D ^ { \\lambda \\rho } A _ { \\rho } ) . \\nonumber \\,\nT ^ { 0 i } = B \\tilde { E } _ { i } - ( \\partial _ { j } E _ { j } ) A _ { i } - \\frac { 2 < B ^ { 2 } > } { \\Lambda ^ { 3 } \\kappa } \\partial _ { i } ( A E ) + \\frac { M } { 8 } \\partial _ { j } ( A _ { i } \\tilde { A } _ { j } + A _ { j } \\tilde { A } _ { i } ) ;\n\\langle | \\Phi _ { + } ( y ) | \\rangle = { \\frac { A } { \\operatorname { c o s } [ a y + b + c _ { 0 } \\mathrm { s g n } ( y ) + c _ { \\pi } \\mathrm { s g n } ( y - \\pi R ) ] } } .\n\\sqrt { \\kappa ^ { 2 } + | \\alpha | ^ { 2 } } = \\frac { \\pi \\lambda ^ { 2 } } { L _ { 1 } L _ { 2 } } .\nR _ { 2 } ( y _ { i } , y _ { j } ) = f _ { 1 , 1 } ( y _ { i } , y _ { j } ) + { \\frac { \\kappa } { 2 } } \\left( f _ { 2 , 1 } ( y _ { i } , y _ { j } ) + f _ { 1 , 2 } ( y _ { i } , y _ { j } ) \\right) + O ( \\kappa ^ { 2 } ) ,\n\\Delta ^ { ( 1 ) } ( \\beta _ { R } , \\beta _ { I } , k , z _ { 1 } ) = u ~ P ( \\beta _ { R } | \\beta _ { I } , k , \\mathrm { R e } ( z _ { 1 } ) , \\mathrm { I m } ( z _ { 1 } ) )\n\\eta ^ { \\mu } ( \\sigma ) = \\hat { \\xi } ^ { \\mu } ( \\sigma , r ( \\sigma ) ) \\ .\nm ^ { 2 } ( t ) = m ^ { 2 } \\operatorname { e x p } ( ( 2 - 2 \\alpha + \\frac { 2 \\alpha ^ { 2 } } { Q ^ { 2 } } ) t )\nd _ { p _ { 1 } } C _ { p _ { 1 } } ^ { a _ { 1 } } = H _ { 1 } ^ { a _ { 1 } } , \\; d _ { p _ { 1 } } H _ { 1 } ^ { a _ { 1 } } = 0 ,\n\\partial _ { 5 } \\left\\langle { \\Phi } \\right\\rangle = - g _ { 5 } ^ { 2 } \\phi ^ { \\dagger } \\phi \\left( \\delta ( x _ { 5 } ) - { \\frac { 1 } { 2 \\pi R } } \\right) \\ .\n\\sigma _ { a b } = m ^ { 2 } \\delta _ { a b } + v _ { a b } ,\n\\Pi ^ { - } = 2 i \\theta ^ { - } P _ { p } \\sigma ^ { p } \\ , \\qquad \\bar { \\Pi } ^ { - } = 2 i P _ { p } \\sigma ^ { p } \\bar { \\theta } ^ { - }\n\\widehat { p } ^ { a } \\left| B _ { X } \\right\\rangle \\! = \\left( \\widehat { q } ^ { i } - y ^ { i } \\right) \\left| B _ { X } \\right\\rangle \\! = 0 ,\n{ \\frac { m } { 2 } } \\left[ \\mathrm { T r } \\left( \\Phi ^ { 2 } \\right) - \\mathrm { T r } \\left( { \\widetilde \\Phi } ^ { 2 } \\right) \\right] ~ ,\n\\phi _ { i } ^ { \\prime } ( x ) = \\phi _ { i } ( x ) + \\delta _ { \\mathrm { B R S } } \\phi _ { i } ( x ) \\Theta [ \\phi ] ,\n{ \\frac { \\partial W } { \\partial \\bar { q } } } = 0 \\qquad \\Longrightarrow \\qquad q = Q ,\n\\Omega \\to \\Omega + 1 \\qquad \\mathrm { a n d } \\qquad \\Omega \\to - 1 / \\Omega \\ .\n\\Psi ( x ) = ( 2 \\pi ) ^ { - 3 / 2 } \\int \\left[ \\Psi ^ { + } ( \\mathbf { p } ) e ^ { i p x } + \\Psi ^ { - } ( \\mathbf { p } ) e ^ { - i p x } \\right] d ^ { 3 } p ,\n\\begin{array} { l } { \\Phi _ { 2 } = ( \\phi _ { 2 } ^ { + } ) ^ { - 1 } \\{ D _ { 2 } \\} _ { p , q } , } \\\\ { \\bar { \\Phi } _ { 2 } = \\phi _ { 2 } ^ { + } , } \\\\ { \\Phi _ { 1 } = ( \\phi _ { 1 } ^ { + } ) ^ { - 1 } \\{ D _ { 1 } \\} _ { p , q } p ^ { D _ { 2 } } , } \\\\ { \\bar { \\Phi } _ { 1 } = \\phi _ { 1 } ^ { + } q ^ { D _ { 2 } } . } \\\\ \\end{array}\n\\delta { \\cal L } _ { r e d } ~ ~ \\sim ~ ~ \\partial _ { ( + } \\left[ ~ { \\bar { \\epsilon } } ( D _ { - ) } \\psi ) \\phi ^ { * } ~ + ~ c . c . \\right] ~ ~ ,\n\\epsilon _ { \\mathbf { p } } = \\pm \\sqrt { \\left( E _ { \\mathbf { p } } \\pm \\mu \\right) ^ { 2 } + \\chi ^ { 2 } } .\n\\int d ^ { 2 p - 2 } x \\sqrt { \\operatorname* { d e t } ( \\delta _ { i j } + 2 \\pi \\alpha ^ { \\prime } F _ { i j } ) } \\ .\nQ = \\frac { 1 } { 2 \\pi R } \\int _ { 0 } ^ { L } \\partial _ { x } \\varphi ( x , \\, t ) d x \\, .\ni \\hat { { \\cal V } } ( x , y ) \\; ( \\equiv i { \\cal V } ( x , y ) \\, \\hat { A } ) \\to 2 \\int \\frac { d ^ { \\, 4 } P } { ( 2 \\pi ) ^ { 4 } } \\, e ^ { - i P \\cdot ( x - y ) } \\left[ P \\cdot \\partial _ { X } f ( X ; P ) \\right] \\hat { A } .\n\nf ( n _ { 1 } n _ { 2 } n k ) { \\bar { f } } ( n _ { 1 } , n _ { 2 } , n k ) = 1\n{ \\cal D } = \\left( \\begin{array} { c c } { - \\frac { d ^ { 2 } } { d x ^ { 2 } } + 4 } & { 0 } \\\\ { 0 } & { - \\frac { d ^ { 2 } } { d x ^ { 2 } } + 4 - \\frac { 6 } { \\operatorname { c o s h } ^ { 2 } x } } \\\\ \\end{array} \\right) \\qquad .\nS _ { b o s } \\, = \\, \\int d ^ { 3 } x \\, \\left( \\pm \\frac { i } { 2 } A _ { \\mu } \\epsilon _ { \\mu \\nu \\lambda } \\partial _ { \\nu } A _ { \\lambda } \\, - \\, \\frac { i } { \\sqrt { 4 \\pi } } s _ { \\mu } \\epsilon _ { \\mu \\nu \\lambda } \\partial _ { \\nu } A _ { \\lambda } \\right) ~ ~ ~ ,\n\\omega _ { \\pm } ( p , \\sigma ) = \\sqrt { C ( p ) ^ { 2 } + m _ { \\sigma } ^ { 2 } + ( B ( p ) \\pm \\Lambda ) ^ { 2 } }\n2 i \\pi \\Theta [ - \\epsilon ( z ) ] [ \\; ] { \\cal B } ( - \\alpha , \\beta + \\alpha + 1 ) \\left[ e ^ { i \\pi \\epsilon ( z ) \\alpha } - e ^ { i \\pi \\alpha } \\right] z ^ { - \\alpha - \\beta - 1 } =\n{ \\cal L } = - { \\frac { 1 } { 2 } } \\left\\{ U \\partial _ { \\mu } { \\bf X } \\cdot \\partial ^ { \\mu } { \\bf X } + U ^ { - 1 } { \\cal D } _ { \\mu } \\varphi { \\cal D } ^ { \\mu } \\varphi + \\mu ^ { 2 } U ^ { - 1 } \\right\\} ,\n\\begin{array} { r c l l } { \\langle \\mu _ { \\lambda _ { 1 } } \\mu _ { \\lambda _ { 2 } } \\mu _ { \\lambda _ { 3 } } \\mu _ { \\lambda _ { 4 } } \\rangle } & { = } & { \\mathcal { F } _ { \\lambda _ { 1 } , \\lambda _ { 2 } , \\lambda _ { 3 } , \\lambda _ { 4 } } \\left| \\prod _ { i < j } z _ { i j } ^ { \\lambda _ { i } \\lambda _ { j } } \\right| ^ { 2 } } & { \\mathrm { f o r ~ \\lambda _ 1 + \\lambda _ 2 + \\lambda _ 3 + \\lambda _ 4 = 1 ~ } \\, , } \\\\ \\end{array}\nW _ { i } ( \\xi ) = \\int _ { 0 } ^ { \\infty } i \\Delta ^ { + } ( \\xi , \\kappa ) \\rho _ { i } ( \\kappa ) d \\kappa , \\quad i = 1 , 2\n{ \\bf P } = { \\bf p } , \\quad { \\bf M } = { \\bf p } \\times { \\bf x } + \\sum _ { i = 1 } ^ { N } { \\bf p } _ { i } \\times { \\bf e } _ { i } = { \\bf p } \\times { \\bf x } + \\sum _ { i = 1 } ^ { N } { { \\bf p } ^ { \\bot } } _ { i } \\times { \\bf e } _ { i } - \\frac 1 2 \\sum _ { i , j = 1 } ^ { N } { \\phi } _ { i j } { \\bf e } _ { i } \\times { \\bf e } _ { j } .\n{ \\cal E } ^ { B } = \\frac 1 { 1 2 \\pi } \\int _ { - \\infty } ^ { \\infty } [ d \\tau \\, a ^ { 2 } \\sqrt { f ^ { \\prime } } - d ( a \\sqrt { f ^ { \\prime } } ) ] ,\nm ^ { 2 } ( a ^ { 2 } / 2 , \\chi ) = \\frac { \\mu ^ { 2 } a ^ { 4 } } { 4 }\n\\tilde { D } = ( D \\tilde { \\theta } ) ^ { - 1 } D\n{ \\cal D } _ { 2 } = { \\cal D } _ { 3 } | _ { r = { \\rho } } + \\frac { i { \\gamma } ^ { 3 } } { \\rho } .\n\\tilde { a } _ { 1 } = 2 a _ { 1 } + a _ { 3 } , \\qquad \\tilde { a } _ { 2 } = 2 a _ { 2 } + a _ { 3 } .\n\\lbrack 1 , 1 \\rbrack _ { Z _ { 2 } } = \\lbrack 1 , \\gamma \\rbrack _ { Z _ { 2 } } = 0 , \\quad \\lbrack \\gamma , \\gamma \\rbrack _ { Z _ { 2 } } = - 2 .\n\\frac { i } { 2 \\xi } \\int d ^ { 4 } x ( \\partial . { \\hat { A } } ^ { a } ) ^ { 2 }\nd s ^ { 2 } = ( r ^ { 2 } / R ^ { 2 } - 1 ) d t _ { s } ^ { 2 } - { \\frac { d r ^ { 2 } } { r ^ { 2 } / R ^ { 2 } - 1 } } + r ^ { 2 } d \\Omega _ { D - 2 } ^ { 2 } \\; ,\n{ \\bf E } _ { \\infty } \\simeq \\frac { Q \\, { \\bf r } } { 4 \\pi r ^ { 3 } } \\, , \\ \\, , \\ \\ { \\bf B } _ { \\infty } \\simeq \\frac { { \\bf m } \\wedge { \\bf r } } { 4 \\pi r ^ { 3 } } \\, .\nE ( l ) = \\sum _ { k = 1 } ^ { l } E ( l , k ) = \\frac { 1 } { l } \\sum _ { d | l } \\mu ( l / d ) \\left\\{ F _ { d + 1 } + F _ { d - 1 } \\right\\} \\, ,\n\\begin{array} { c c l } { \\vspace { . 2 c m } } & { } & { \\omega ^ { 2 } ( x ) = - \\mathrm { T r } \\Big [ \\Big ( W ^ { \\prime } ( \\hat { \\Phi } ) + [ \\hat { X } , \\hat { Y } ] \\Big ) \\frac 1 { x - \\hat { \\Phi } } \\Big ] \\ , } \\\\ \\end{array}\nQ _ { 5 } = - i \\int d ^ { 3 } x \\langle j _ { 4 } ^ { 5 } \\rangle\nD _ { t } = \\{ x | f ^ { 2 } ( x ) < 2 \\gamma t \\}\n\\widetilde { \\cal F } ^ { ( 1 ) } ( x ) = - \\int d y \\int d z \\int d w \\Phi ^ { j } ( y ) \\omega _ { j k } ( y , z ) X ^ { k l } ( z , w ) \\{ \\Omega _ { l } ( w ) , { \\cal F } ( x ) \\} _ { ( { \\cal F } ) } .\n\\frac { \\delta m _ { i } } { m _ { i } } = \\frac { \\beta ^ { 2 } } { 2 h } \\operatorname { c o t } \\frac { \\pi } { h } \\, \\, .\n2 \\ell \\rightleftharpoons \\frac { 1 } { T } \\, ; \\hspace { 2 c m } p \\rightleftharpoons - u .\n\\operatorname { e x p } \\left( { \\frac { \\Phi } { 2 } } \\right) = { \\frac { C _ { 0 } } { \\operatorname { l o g } { \\rho / \\rho _ { 0 } } } } ,\n\\epsilon ^ { * } = - { \\frac { 2 c _ { V } ( G ) } { \\pi k } } ,\n\\frac { S } { \\tilde { S } ^ { ( n ) } } = O \\left( \\frac { 1 } { \\lambda ^ { n + 1 } { \\cal T } ^ { 3 } \\, V } \\right) .\nG ( \\mu ) = G _ { 0 } l ^ { 1 - d } W ^ { \\Delta } F \\left( \\Delta , \\Delta - \\frac { d } { 2 } + \\frac { 1 } { 2 } , 2 \\Delta - d + 1 , W \\right) ,\n\\hat { \\lambda } _ { j } \\cdot \\hat { \\sigma } _ { q } ^ { x } \\hat { \\gamma } _ { i } ^ { + } = \\hat { \\lambda }\n\\epsilon ^ { \\mu \\nu } = { \\frac { 1 } { 2 } } e ( e ^ { \\mu { [ + 2 ] } } e ^ { \\nu { [ - 2 ] } } - e ^ { \\mu { [ - 2 ] } } e ^ { \\nu { [ + 2 ] } } ) , \\qquad \\left( \\epsilon ^ { { 0 1 } } = - \\epsilon _ { { 0 1 } } = 1 \\right) ,\n{ \\cal L } ^ { ( 2 ) } = \\pi ^ { a } \\partial _ { 0 } { n } ^ { a } - \\Omega _ { 1 } \\partial _ { 0 } { \\rho } - \\Omega _ { 2 } \\partial _ { 0 } { \\sigma } - { \\cal H } ^ { ( 2 ) } ,\n\\left( \\begin{array} { c } { \\delta _ { \\; \\; \\beta } ^ { \\alpha } } \\\\ { - \\delta _ { \\; \\; \\beta } ^ { \\alpha } } \\\\ \\end{array} \\right) .\ne ^ { i \\Gamma [ A ] } = \\int { \\cal D } \\psi { \\cal D } \\overline { \\psi } e ^ { i \\int d ^ { 2 } x \\overline { \\psi } i D \\! \\! \\! / \\psi } ,\nF ^ { T } \\rightarrow - { \\frac { 1 } { T ^ { 2 } } } ~ F ^ { T } \\; .\n\\Pi ( q ) \\equiv \\prod _ { i < j } \\sigma ( q _ { i j } ) , \\hspace { 1 c m } \\sqrt { g } = - \\frac { l } { n } \\sigma ^ { \\prime } ( 0 ) ,\n\\int A _ { ( 3 ) } \\wedge X _ { 8 } \\ , \\qquad \\mathrm { w i t h } \\quad X _ { 8 } = { \\frac { 1 } { 2 4 } } \\left( p _ { 2 } - { \\frac { 1 } { 4 } } p _ { 1 } \\wedge p _ { 1 } \\right) \\ .\n\\begin{array} { l } { t ^ { \\prime } e ^ { \\prime } { t ^ { \\prime } } ^ { - 1 } = q ^ { 2 } e ^ { \\prime } ~ , ~ ~ ~ ~ t ^ { \\prime } f ^ { \\prime } { t ^ { \\prime } } ^ { - 1 } = q ^ { - 2 } f ^ { \\prime } ~ , } \\\\ { [ e ^ { \\prime } , f ^ { \\prime } ] = \\displaystyle \\frac { t ^ { \\prime } - { t ^ { \\prime } } ^ { - 1 } } { q - q ^ { - 1 } } ~ , } \\\\ { \\Delta ( e ^ { \\prime } ) = e ^ { \\prime } \\otimes { t ^ { \\prime } } ^ { - n } + { t ^ { \\prime } } ^ { 1 - n } \\otimes e ^ { \\prime } ~ , } \\\\ { \\Delta ( f ^ { \\prime } ) = f ^ { \\prime } \\otimes { t ^ { \\prime } } ^ { n - 1 } + { t ^ { \\prime } } ^ { n } \\otimes f ~ , } \\\\ { \\Delta ( t ^ { \\prime } ) = t ^ { \\prime } \\otimes t ^ { \\prime } ~ , } \\\\ { \\epsilon ( e ^ { \\prime } ) = 0 = \\epsilon ( f ^ { \\prime } ) ~ , ~ ~ ~ ~ \\epsilon ( t ^ { \\prime } ) = 1 ~ , } \\\\ { S ( e ^ { \\prime } ) = - t ^ { - 1 } e ^ { \\prime } ~ , ~ ~ ~ ~ S ( f ^ { \\prime } ) = - f ^ { \\prime } t ^ { \\prime } ~ , ~ ~ ~ ~ S ( t ^ { \\prime } ) = t ^ { - 1 } ~ . } \\\\ \\end{array}\nF _ { i j } = \\partial _ { i } A _ { j } - \\partial _ { j } A _ { i } - i A _ { i } \\star A _ { j } + i A _ { j } \\star A _ { i }\n\\mu _ { + } ^ { - 1 } \\mu _ { - } = \\nu _ { - } \\eta \\nu _ { + } ^ { - 1 } ,\nf ( \\rho ) = \\frac { C } { \\rho } , \\qquad a ( \\rho ) = \\frac { D } { \\rho ^ { \\beta } }\n\\rho = \\frac { 1 } { 2 } \\left( \\frac { x _ { i } ^ { 2 } } { m } + k q _ { i } ^ { 2 } \\right)\n{ \\cal Q } _ { k } = \\oint _ { \\partial \\Sigma } d \\Sigma _ { \\mu \\nu } \\gamma ^ { \\mu \\nu \\lambda } \\hat { \\nabla } _ { \\lambda } \\epsilon _ { k } = 0 \\ .\nq _ { - } ^ { \\prime } - \\frac { 1 } { \\Delta _ { \\perp } } \\ast \\rho _ { 2 } \\ ,\n{ \\bf J } = { \\frac { 1 } { 2 } } \\left( \\begin{array} { c c } { p ^ { \\dagger } } & { q ^ { \\dagger } } \\\\ \\end{array} \\right) { \\bf \\sigma } \\left( \\begin{array} { c } { p } \\\\ { q } \\\\ \\end{array} \\right) + { \\frac { 1 } { 2 } } \\left( \\begin{array} { c c } { r ^ { \\dagger } } & { s ^ { \\dagger } } \\\\ \\end{array} \\right) { \\bf \\sigma } \\left( \\begin{array} { c } { r } \\\\ { s } \\\\ \\end{array} \\right)\nm _ { l } / M = \\mathrm { L o g } \\left( \\frac { F _ { l } } { F _ { 0 } } \\right) .\n\\begin{array} { c } { s c = c \\partial c \\; , } \\\\ { s b = - ( \\partial b ) c - 2 b \\partial c \\; . } \\\\ \\end{array}\nE _ { \\mathrm { i n } } = - \\frac { 1 } { 2 \\pi a }\n\\begin{array} { l l } { z ^ { \\prime } = z + \\varepsilon \\zeta \\theta ; } & { \\theta ^ { \\prime } = \\theta + \\varepsilon \\zeta } \\\\ { \\bar { z } ^ { \\prime } = \\bar { z } - \\bar { \\varepsilon } \\bar { \\zeta } \\bar { \\theta } ; } & { \\bar { \\theta } ^ { \\prime } = \\bar { \\theta } + \\bar { \\varepsilon } \\bar { \\zeta } . } \\\\ \\end{array}\n- \\frac { 1 } { 2 } { \\bf \\nabla } _ { \\rho } ^ { 2 } \\psi ( \\rho ) = E \\psi ( \\rho ) .\nF ( x ) \\equiv \\int _ { 0 } ^ { x } d x \\, \\sqrt { x ^ { 2 } + 1 } + { \\frac { 1 } { i \\alpha } } \\operatorname { l n } ( x ^ { 2 } + 1 ) , \\quad \\alpha \\equiv { \\frac { 2 \\beta } { \\hbar v g } } = { \\frac { 4 } { \\hbar v g ^ { 2 } [ V _ { 1 } ^ { \\prime } ( 0 ) - V _ { 2 } ^ { \\prime } ( 0 ) ] } } > 0 .\nE ( l , k ) = \\delta _ { l , 2 } \\delta _ { k , 1 } + \\frac { 2 } { l + k } \\sum _ { 2 d | l \\pm k } \\mu ( d ) { \\binom { \\frac { l + k } { 2 d } } { \\frac { l - k } { 2 d } } } \\, ,\n\\alpha _ { 1 } ^ { r } \\gamma _ { 1 } + \\dots + \\alpha _ { N } ^ { r } \\gamma _ { N } = 0 \\quad ( r = 1 , . . . , R ) \\; ,\n\\eta = - \\frac { 1 } { 2 } \\operatorname { l n } \\left( \\frac { \\operatorname { c o s h } \\left( \\sqrt { 2 } b _ { \\infty } \\sqrt { 1 + \\alpha ^ { 2 } } \\; y - \\mathrm { a r c s i n h } \\; \\alpha \\right) } { \\sqrt { 1 + \\alpha ^ { 2 } } } \\right)\nP _ { ( 2 ) } ^ { - } = \\int \\beta d \\beta d ^ { 9 } p d ^ { 8 } \\lambda \\Phi ( - p , - \\lambda ) \\left( - \\frac { p ^ { I } p ^ { I } } { 2 \\beta } \\right) \\Phi ( p , \\lambda ) \\, .\n\\Gamma ( z + 1 ) = \\int _ { 0 } ^ { \\infty } \\, \\, d x \\, \\, e ^ { - x } x ^ { z } .\n\\frac { d } { d s } { \\bf C } _ { i } = \\frac { 1 } { 2 } \\epsilon _ { i j k } { \\bf C } _ { j } \\times { \\bf C } _ { k } \\, .\nZ = \\sum _ { s p i n s } \\prod _ { c u b e s } W ( a | e , f , g | b , c , d | h ) ,\n\\left\\{ Q ^ { i } , Q ^ { j } \\right\\} = c ^ { i j } \\Gamma ^ { M } C P _ { M } + C c ^ { i j } Z ,\n\\breve { c } _ { n , \\nu } = \\sum _ { m = n } ^ { 2 n } { \\frac { \\Gamma \\left( \\nu + m - { \\frac { D - 1 } { 2 } } \\right) } { \\Gamma \\left( \\nu + n - { \\frac { D - 1 } { 2 } } \\right) } } ~ \\breve { a } _ { 2 ( m - n ) , m } ~ ~ ~ .\nR ( g ) = - f \\left[ 3 \\left[ ( \\operatorname { l n } f ) ^ { \\prime } \\right] ^ { 2 } + \\frac { \\Lambda ( x ^ { 5 } ) } { M ^ { 3 } } \\right] \\; ,\n{ \\frac { d } { d s } } { \\frac { 1 } { \\Gamma ( - s ) } } \\bigg | _ { s = 0 } = - 1 ,\n\\dot { z } _ { 1 } = - N ^ { z } ( z _ { 1 } ) = - g ( z _ { 1 } ) = - \\frac { z _ { 1 } } { P _ { z } ( z _ { 2 } - z _ { 1 } ) } ; ~ ~ ~ \\dot { z } _ { 2 } = - \\frac { z _ { 2 } } { P _ { z } ( z _ { 2 } - z _ { 1 } ) }\nc _ { \\alpha } = \\sum _ { \\beta \\in \\Lambda _ { R } } \\epsilon ( \\alpha , \\beta ) | \\beta + \\bar { p } > < \\beta + \\bar { p } |\n{ \\cal L } = - { \\frac { 1 } { 4 } } F _ { \\mu \\nu } F ^ { \\mu \\nu } + { \\bar { \\psi } } ( i \\gamma ^ { \\mu } D _ { \\mu } - m ) \\psi \\, ,\ne ^ { i { \\bf k \\cdot r } } = e ^ { i k r \\operatorname { c o s } ( \\theta - \\Theta ) } = \\sum _ { l = - \\infty } ^ { \\infty } i ^ { l } \\, J _ { l } ( k r ) \\, e ^ { i l ( \\theta - \\Theta ) } \\, ,\ni \\sqrt { 2 } \\partial _ { - } \\chi - g [ \\phi , \\psi ] = 0 , \\quad \\partial _ { - } ^ { 2 } \\bar { A } _ { + } - g ^ { 2 } J ^ { + } = 0 .\n\\Omega _ { k } ^ { ( l ) } = \\sum _ { s = 0 } \\int d ^ { 3 } y \\left( ( - 1 ) ^ { s + 1 } \\frac { d ^ { s } } { d t ^ { s } } \\phi _ { k } ^ { i ( s ) } ( x , y ) L _ { i } ^ { ( 0 ) } ( y ) \\right) .\nL _ { g } ^ { ' } \\Bigl ( v ( h ) \\Bigr ) = v ( L _ { g } h ) = v ( g h ) \\, , \\, \\, \\, \\forall g , h \\in G ,\n\\xi ^ { 2 } = \\left( \\frac { \\varepsilon _ { 1 } - \\varepsilon _ { 2 } } { \\varepsilon _ { 1 } + \\varepsilon _ { 2 } } \\right) ^ { 2 } = \\left( \\frac { \\mu _ { 1 } - \\mu _ { 2 } } { \\mu _ { 1 } + \\mu _ { 2 } } \\right) ^ { 2 } ,\nR ( e _ { 1 } ) = \\epsilon ^ { - J _ { 6 7 } + J _ { 8 9 } } , \\quad R ( e _ { 2 } ) = \\epsilon ^ { J _ { 4 5 } - J _ { 8 9 } } .\n{ \\tilde { \\cal { E } } } _ { m < 0 } = { \\cal { E } } _ { m < 0 } ( B ) - { \\cal { E } } ( 0 ) = \\frac { B ^ { 2 } } { 2 } + \\frac { ( e B ) ^ { \\frac { 3 } { 2 } } } { 2 \\pi } g \\left( \\frac { e B } { m ^ { 2 } } \\right) \\, ,\n\\hat { O } _ { 2 } ^ { r } \\mid 1 > _ { ( 0 ) } = { O } _ { 2 } ^ { r } \\mid 0 > _ { ( 0 ) } .\nI ^ { c } = \\mp { \\frac { \\pi b \\sqrt { 1 - \\Lambda a ^ { 2 } } } { 2 G } } \\ \\ ,\ng _ { n } ^ { > } ( r , r ^ { \\prime } ) = E _ { n } K _ { | n / \\alpha | } ( \\beta r ) , \\quad \\mathrm { f o r ~ r > r ^ { ' } ~ . }\nR ^ { \\frac { 1 } { 2 } } ( \\theta ) _ { \\left| \\left. a _ { k } \\dots \\frac { 1 } { 2 } , a _ { 1 } , \\frac { 1 } { 2 } \\right| n _ { k } \\dots , m _ { 1 } , n _ { 1 } \\right\\rangle } ^ { \\left| \\left. b _ { k } \\dots \\frac { 1 } { 2 } , b _ { 1 } , \\frac { 1 } { 2 } \\right| n _ { k } \\dots , m _ { 1 } , n _ { 1 } \\right\\rangle } = R _ { a _ { 1 } b _ { 1 } } ^ { \\frac { 1 } { 2 } } ( \\theta ) \\prod _ { i = 1 } ^ { k - 1 } f _ { b _ { i } b _ { i + 1 } } ^ { a _ { i } a _ { i + 1 } } ( w _ { m _ { i } } , \\nu _ { n _ { i + 1 } } , \\theta )\nQ _ { 1 } ^ { a b } ( x , y ) \\equiv Q _ { 1 } ^ { a b } + x \\, J _ { 1 } ^ { a b } + y \\, K _ { 1 } ^ { a b } ,\n\\left\\{ \\begin{array} { c } { \\partial _ { \\tau } R + \\vec { \\nabla } \\cdot \\left( \\vec { \\nabla } \\Theta \\, \\sqrt { \\displaystyle \\frac { R ^ { 2 } + a ^ { 2 } } { 1 + ( \\vec { \\nabla } \\Theta ) ^ { 2 } } } \\right) = 0 , \\hfill } \\\\ { \\partial _ { \\tau } \\Theta + R \\sqrt { \\displaystyle \\frac { 1 + ( \\vec { \\nabla } \\Theta ) ^ { 2 } } { R ^ { 2 } + a ^ { 2 } } } = 0 . \\hfill } \\\\ \\end{array} \\right.\n\\Delta ^ { ( N , 0 ) } ( s ) = - \\sum _ { n > 0 , \\vec { n } ^ { 2 } < N } \\left[ J ( z _ { n } ) - 2 + 2 J ( y _ { n } ) + \\frac { J ^ { 2 } ( y _ { n } ) } { 2 ( 1 - y _ { n } ) } - J ( \\tilde { z } _ { n } ) - 2 J ( \\tilde { y } _ { n } ) \\right] \\ ,\n\\left\\{ \\Psi \\circ \\mu , f \\right\\} = ( \\overline { X } _ { i } f ) \\, ( Y ^ { i } \\Psi ) \\circ \\mu \\, ,\nF _ { n } ^ { \\mathcal { O } | \\mu _ { 1 } \\ldots \\mu _ { n } } ( \\theta _ { 1 } + \\lambda , \\ldots , \\theta _ { n } + \\lambda ) = e ^ { s \\lambda } F _ { n } ^ { \\mathcal { O } | \\mu _ { 1 } \\ldots \\mu _ { n } } ( \\theta _ { 1 } , \\ldots , \\theta _ { n } ) \\, \\, ,\nS = S _ { P h y s . } ( \\Phi ^ { a } , \\Phi ^ { \\ast a } ) + S _ { T } ( \\vartheta ^ { b } , \\vartheta ^ { \\ast b } , c ^ { \\alpha } )\n\\mathcal { A } \\equiv \\operatorname { e x p } \\left[ \\int _ { 0 } ^ { \\lambda } d \\tilde { \\lambda } \\, \\theta ( \\tilde { \\lambda } ) \\right] \\, .\nF _ { - { \\frac { 1 } { 2 } } } ( x ) = \\bar { \\epsilon } _ { 0 } S ( x ) e ^ { - 1 / 2 \\phi ( x ) } \\; , \\qquad F _ { \\frac { 1 } { 2 } } ( x ) = \\bar { \\epsilon } _ { 0 } \\gamma _ { \\mu } S ( x ) \\partial X ^ { \\mu } ( x ) e ^ { 1 / 2 \\phi ( x ) } ,\n\\rho ^ { 0 } = \\left( \\begin{array} { c c } { 0 } & { - i } \\\\ { i } & { 0 } \\\\ \\end{array} \\right) \\, \\, \\, \\mathrm { a n d } \\, \\, \\, \\rho ^ { 1 } = \\left( \\begin{array} { c c } { 0 } & { i } \\\\ { i } & { 0 } \\\\ \\end{array} \\right) .\n\\psi = \\sum _ { i = 0 } ^ { 3 } ( \\psi _ { i } ^ { A } + ( \\psi _ { i } ^ { A } ) ^ { c } ) T ^ { A }\nG = \\! e ^ { i \\tau L _ { - 1 } } e ^ { i U ^ { ( 1 ) } L _ { 1 } } e ^ { i U ^ { ( 2 ) } L _ { 2 } } e ^ { i U ^ { ( 3 ) } L _ { 3 } } \\ldots \\! e ^ { i { U ^ { ( 0 ) } } L _ { 0 } } ,\nV ( z , \\bar { z } ) = e ^ { - q \\Phi ( z ) } e ^ { i \\alpha \\cdot H } e ^ { i ( P _ { R } \\cdot X _ { R } - P _ { L } \\cdot X _ { L } ) } \\; ,\n\\epsilon _ { i } = \\tau _ { i } + \\rho _ { i } + \\rho _ { i - 1 } , \\quad ( \\tau _ { 3 } = 0 , \\: \\rho _ { 0 } = \\rho _ { 4 } )\ns _ { \\infty } ( k ^ { 2 } ) - s _ { J _ { \\operatorname* { m a x } } } ( k ^ { 2 } ) \\sim O ( J _ { \\operatorname* { m a x } } ^ { - 2 } ) .\nA ( u ) ~ = ~ \\mathrm { R e s } \\vert _ { v = u } ^ { } \\left( { \\frac { 1 } { v - u } } \\, R ( u , v ) \\cdot L ( v ) \\right) + \\, { \\textstyle { \\frac { 1 } { 2 } } } \\, \\zeta ( 2 u ) \\, L ( u )\n\\partial _ { a } ^ { m } \\Gamma _ { i } = \\frac { \\Gamma ^ { n } } { \\lambda _ { i } } \\{ \\delta _ { n m } \\psi _ { a } ^ { i } - \\phi _ { b } ^ { n } \\phi _ { c } ^ { m } \\psi _ { b } ^ { i } \\psi _ { c } ^ { i } \\frac { \\psi _ { a } ^ { i } } { \\lambda _ { i } ^ { 2 } } + \\phi _ { b } ^ { n } \\phi _ { c } ^ { m } \\sum _ { j \\neq i } \\psi _ { b } ^ { j } \\frac { ( \\psi _ { c } ^ { i } \\psi _ { a } ^ { j } + \\psi _ { a } ^ { i } \\psi _ { c } ^ { j } ) } { ( \\lambda _ { i } ^ { 2 } - \\lambda _ { j } ^ { 2 } ) } \\} \\vspace { - 1 2 p t }\n\\int \\mathrm { d } ^ { 4 } x _ { 1 } ~ \\cdots ~ \\mathrm { d } ^ { 4 } x _ { n } ~ P _ { 4 } ( x _ { 1 } , \\cdots , x _ { n } ) ~ \\Gamma _ { x _ { 1 } \\cdots x _ { n } 0 } = 0\nL = \\frac { \\dot { x } _ { \\mu } ^ { 2 } } { 2 e } + \\frac { \\lambda } { l } ( e - M ^ { - 1 } \\dot { x } { } ^ { 0 } ) ,\nJ _ { 2 } ( z ) \\times X ^ { + } ( w ) \\rightarrow 0 .\nF ( z _ { 1 2 } ^ { \\prime } ) = \\bar { K } ( z _ { 2 } ; g ) F ( z _ { 1 2 } ) K ( z _ { 1 } ; g )\n{ \\xi } _ { i } ^ { \\ast } , { p } _ { i } ^ { \\ast } , \\quad i = 2 , \\dots , l + 1\n\\varrho _ { L } - { \\cal L } _ { E } = [ 2 \\dot { \\Phi } ^ { 2 } ] \\; K ^ { \\prime } ( \\dot { \\Phi } ^ { 2 } , \\Phi ) - K ( \\dot { \\Phi } ^ { 2 } , \\Phi ) + K ( - \\dot { \\Phi } ^ { 2 } , \\Phi ) .\nK ^ { \\prime } = \\sqrt { c - 2 f } \\ , \\ \\ \\ K ^ { \\prime \\prime } = - \\frac { 1 } { \\sqrt { c - 2 f } } \\ ,\n\\kappa _ { \\omega } = \\frac { 2 \\Gamma ( \\Delta _ { \\omega } ) } { \\pi \\Gamma ( 1 - \\Delta _ { \\omega } ) } \\left( \\frac { \\sqrt { \\pi } \\Gamma \\left( \\frac { 1 } { 2 - 2 \\Delta _ { \\omega } } \\right) } { 2 \\Gamma \\left( \\frac { \\Delta _ { \\omega } } { 2 - 2 \\Delta _ { \\omega } } \\right) } \\right) ^ { 2 - 2 \\Delta _ { \\omega } } \\, .\n< \\frac { 1 } { 2 } , m _ { s } | { \\psi } _ { - } ^ { ( \\frac { 1 } { 2 } ) } ( g ) > \\equiv D _ { m _ { s } - \\frac { 1 } { 2 } } ^ { ( \\frac { 1 } { 2 } ) } ( g ) = < g , l + \\frac { 1 } { 2 } | T _ { m _ { s } - } ^ { \\frac { 1 } { 2 } } | g , l > .\n\\sum _ { l , n } \\frac { \\mu _ { p - 1 } \\lambda ^ { k + n + l } i ^ { k } p ! } { k ! n ! l ! ( p - l ) ! } \\partial _ { x ^ { i _ { 1 } } } \\ldots \\partial _ { x ^ { i _ { n } } } C _ { i _ { 1 } ^ { \\prime } \\ldots i _ { 2 k } ^ { \\prime } j _ { 1 } \\ldots j _ { l } [ a _ { l + 1 } \\ldots a _ { p } } ^ { 0 } S t r \\left( \\partial _ { a _ { 1 } } \\phi ^ { j _ { 1 } } \\ldots \\partial _ { a _ { l } ] } \\phi ^ { j _ { l } } \\phi ^ { i _ { 1 } } \\ldots \\phi ^ { i _ { n } } \\phi ^ { i _ { 2 k } ^ { \\prime } } \\phi ^ { i _ { 2 k - 1 } ^ { \\prime } } \\ldots \\right)\nD ^ { \\mu } \\frac { \\delta f ( A _ { \\nu } ) } { \\delta A _ { \\mu } } = D _ { \\mu } \\partial ^ { \\mu } ( \\partial _ { \\nu } A ^ { \\nu } )\n\\delta \\chi _ { \\mu \\nu } = i b _ { \\mu \\nu } , \\qquad \\delta b _ { \\mu \\nu } = 0 .\nV _ { a b \\ \\ m n } ^ { k } = \\frac { 1 } { g } \\ E _ { a } ^ { r } \\ E _ { b } ^ { s } \\epsilon _ { r s ( m } \\ \\delta _ { n ) } ^ { i } .\nf ( r ) = \\left( 1 - \\frac { m } { 2 r ^ { n - 1 } } \\right) ^ { 2 } + \\frac { r ^ { 2 } } { l ^ { 2 } } .\nE _ { 1 2 } ~ ~ \\Phi = 2 \\sqrt { ( m + \\frac { 1 } { 2 } b r ) ^ { 2 } + p _ { r } ^ { 2 } + \\frac { \\ell ( \\ell + 1 ) } { r ^ { 2 } } } ~ ~ \\Phi ,\nT _ { \\mathit { G } } ( - t , - t ^ { - 1 } ) = T _ { \\mathit { G } ^ { \\ast } } ( - t ^ { - 1 } , - t )\nd s _ { 1 1 } ^ { 2 } = d x ^ { + } d x ^ { - } + l _ { p } ^ { 9 } \\frac { p _ { - } } { r ^ { 7 } } \\delta ( x ^ { - } ) d x ^ { - } d x ^ { - } + d x _ { 1 } ^ { 2 } + \\ \\cdots \\ + d x _ { 9 } ^ { 2 }\nF _ { a b } = { \\frac { 1 } { 2 } } \\epsilon _ { a b c d } F ^ { c d }\n2 f ^ { 2 } - 4 f ^ { 2 } - g ^ { 2 } ( 1 - \\Gamma ) \\, ,\n( a ^ { \\dagger } L _ { m n } a ) = a _ { k } ^ { \\dagger } ( L _ { m n } ) _ { k l } a _ { l } = i a _ { [ m } ^ { \\dagger } a _ { n ] } , \\; \\; \\; \\; \\; \\; ( L _ { m n } ) _ { k l } = i ( \\delta _ { m k } \\delta _ { n l } - \\delta _ { n k } \\delta _ { m l } )\n\\int d t d ^ { 3 } x \\bar { \\lambda } \\partial ^ { \\mu } \\gamma _ { \\mu } \\lambda ,\nh = { \\frac { s \\lambda } { 1 + 2 n + s N + | N | } } ,\nQ = c \\sum _ { i } f _ { i } ^ { \\prime } p ^ { i } + \\sum _ { k } c _ { k } p ^ { k } f _ { k } + i n f i n i t e \\: m o r e .\n\\mathrm { T r } \\, \\operatorname { l o g } ( 1 - \\sum _ { i = 0 } ^ { N } A _ { i } ) ~ = ~ \\mathrm { T r } \\, \\operatorname { l o g } ( 1 - \\sum _ { k = 1 } ^ { N } \\sum _ { m = 0 } ^ { k - 1 } A _ { k } \\phi ^ { m } ) + \\mathrm { T r } \\, \\operatorname { l o g } ( 1 - \\phi ) ~ .\nH _ { i j } ^ { a } = F _ { i j } ^ { a } - g f _ { \\; \\; b c } ^ { a } A _ { i } ^ { b } A _ { j } ^ { c } ,\n{ \\tilde { \\rho } } _ { { \\bf { q } } } = \\sum _ { { \\bf { k } } } [ \\Lambda _ { { \\bf { k } } } ( { \\bf { q } } ) a _ { { \\bf { k } } } ( - { \\bf { q } } ) + \\Lambda _ { { \\bf { k } } } ( - { \\bf { q } } ) a _ { { \\bf { k } } } ^ { \\dagger } ( { \\bf { q } } ) ]\n{ \\bf N } ( { \\bf p } , { \\bf s } ) : = i p _ { 0 } { \\nabla } _ { \\bf p } - \\frac { { \\bf s } \\times { \\bf p } } { p _ { 0 } + m } , \\quad { \\bf J } ( { \\bf p } , { \\bf s } ) : = - i { \\bf p } \\times { \\bf \\nabla } _ { \\bf p } + { \\bf s } : = { \\bf L } ( { \\bf p } ) + { \\bf s } ,\nA _ { \\mu } \\; = \\; \\partial _ { \\mu } \\varphi + \\epsilon _ { \\mu \\nu } \\, \\partial _ { \\nu } \\sigma \\; .\nC _ { J } ( \\nu _ { 1 } , \\nu _ { 2 } ) = ( 2 J + \\nu _ { 1 } + \\nu _ { 2 } + 1 ) \\frac { { \\mit \\Gamma } ( J + 1 ) { \\mit \\Gamma } ( J + \\nu _ { 1 } + \\nu _ { 2 } + 1 ) } { { \\mit \\Gamma } ( J + \\nu _ { 1 } + 1 ) { \\mit \\Gamma } ( J + \\nu _ { 2 } + 1 ) } \\, .\n( \\psi \\otimes _ { \\zeta , z } \\chi ) \\mapsto ( e ^ { - u L _ { - 1 } } \\psi \\otimes _ { \\zeta + u , z + v } e ^ { - v L _ { - 1 } } \\chi ) ,\nu _ { 0 } ( k , r ) = \\sqrt { { \\displaystyle { \\frac { \\pi } { 2 } } } } \\, i \\sqrt { r } \\, J _ { 0 } ( k r ) - \\sqrt { { \\displaystyle { \\frac { \\pi } { 2 } } } } \\, A ( k ) \\sqrt { k r } \\, H _ { 0 } ^ { ( 1 ) } ( k r ) .\nJ _ { k } = \\oint p _ { k } d q _ { k } , ~ ~ k = r , ~ \\theta , ~ \\phi ,\n\\delta _ { \\perp } \\kappa _ { 1 } = \\kappa _ { 3 } \\kappa _ { 2 } \\Psi _ { 3 } + 2 \\kappa _ { 2 } \\Psi _ { 2 } { } ^ { \\prime } + \\kappa _ { 2 } ^ { \\prime } \\Psi _ { 2 } + \\Psi _ { 1 } { } ^ { \\prime \\prime } - \\left( \\kappa _ { 1 } ^ { 2 } + \\kappa _ { 2 } { } ^ { 2 } \\right) \\Psi _ { 1 } \\, .\n\\left( \\gamma _ { \\mu } \\partial _ { \\mu } + m \\right) \\psi ^ { ( b ) } ( x ) = 0 , \\hspace { 0 . 5 i n } b = 1 , 2 , 3 , 4\nf _ { \\alpha } ( x ) = \\left( 4 \\operatorname { s i n } ^ { 2 } \\frac { x } { 2 } \\right) ^ { \\alpha } .\nr _ { h } ^ { 2 } = \\frac { l ^ { 2 } } { 2 } ( \\sqrt { K ^ { 2 } + 4 l ^ { - 2 } \\mu } - K ) .\n\\mu ^ { \\prime \\prime } + \\biggl [ n ^ { 2 } - \\frac { a ^ { \\prime \\prime } } { a } \\biggr ] \\mu = 0 .\nx _ { \\overline { m } } = { \\frac { 1 } { 2 } } ( x _ { m } + x _ { m + 1 } ) ,\nS _ { i j } \\left( \\theta \\right) = \\prod _ { x , y } \\left[ x , y \\right] _ { \\theta }\nA _ { d } ( p ^ { 2 } + \\omega _ { n } ^ { 2 } ) ^ { \\frac { 1 } { 2 } d - 2 } \\left[ \\left( 1 + v ^ { 2 } \\right) ^ { \\frac { 1 } { 2 } d - \\frac { 3 } { 2 } } + \\frac { \\Gamma ( \\frac { 1 } { 2 } d - \\frac { 1 } { 2 } ) } { \\sqrt { \\pi } \\Gamma ( d ) } \\frac { ( v ^ { 2 } ) ^ { \\frac { d } { 2 } - 1 } } { 1 + v ^ { 2 } } { } _ { 2 } F _ { 1 } \\left( \\textstyle { { \\frac { 1 } { 2 } d - \\frac { 1 } { 2 } , 1 ; \\frac { 1 } { 2 } d ; \\frac { v ^ { 2 } } { 1 + v ^ { 2 } } } } \\right) \\right] \\, \\ \\cdot\n\\psi _ { c } ( x ) = \\gamma ^ { 1 } \\psi ^ { * } ( x ) \\ ,\nL = L ^ { \\Lambda } { \\bf T } _ { \\Lambda } = d Z ^ { M } L _ { M } { } ^ { \\Lambda } { \\bf T } _ { \\Lambda } \\, .\nz _ { t , 0 } ^ { ' ( r ) } \\quad = \\quad z _ { t , 0 } ^ { ' ^ { \\prime } ( r ) } \\quad = \\quad 0 \\qquad ( 1 \\le t \\le r , \\, \\, \\mathrm { { a l l } } \\, \\, r ) \\, ;\nd T ( x ) = \\left( \\begin{array} { c c } { \\delta ( x ) 1 _ { N - k } } & { 0 } \\\\ { 0 } & { - \\delta ( x ) 1 _ { k } } \\\\ \\end{array} \\right) d x\n1 - \\frac { 2 G M } { \\rho } = ( \\nabla \\rho ) ^ { 2 } \\equiv f\nM _ { g } = M _ { c _ { 1 } } M _ { c _ { 2 } } M _ { c _ { 3 } } M _ { c _ { 4 } } M _ { c _ { 5 } } M _ { r = \\infty } = 1\n\\delta F \\left( \\operatorname { s i n } \\theta \\, d x ^ { 0 } d x ^ { 1 } , 0 , 0 , \\epsilon \\, d x ^ { 0 } \\cdots d x ^ { 3 } \\right) .\nS _ { \\mathrm { p a r t } , 0 } ^ { ( \\mathrm { N } ) } = \\int d t \\, \\sum _ { \\alpha = 1 } ^ { N } \\left( \\xi _ { \\alpha } ^ { \\underline { { a } } } \\left( E _ { j , \\alpha } ^ { \\underline { { a } } } \\dot { x } _ { \\alpha } ^ { j } + E _ { 0 , \\alpha } ^ { \\underline { { a } } } \\right) - { \\frac { 1 } { 2 } } \\xi _ { \\alpha } ^ { \\underline { { a } } } \\xi _ { \\alpha } ^ { \\underline { { a } } } \\right) \\, .\n\\xi = v _ { 1 } \\left( u _ { 1 } - \\kappa v _ { 2 } \\right) + v _ { 2 } \\left( u _ { 2 } - \\kappa v _ { 1 } \\right) .\nU ( r ) = U ( r _ { 0 } ) + 4 \\pi ^ { 2 } K A ( d , \\sigma ) \\int _ { r _ { 0 } } ^ { r } d s \\frac { s ^ { \\sigma - d - 1 } } { \\varepsilon ( s ) } .\n| 0 \\rangle \\rightarrow | 0 \\rangle _ { \\beta } = ( 1 + \\mathrm { e } ^ { - \\beta \\epsilon } ) ^ { - 1 / 2 } \\{ | 0 \\rangle _ { a } \\otimes | 0 \\rangle _ { b } + \\mathrm { e } ^ { - \\beta \\epsilon / 2 } a ^ { \\dagger } { \\tilde { a } } | 0 \\rangle _ { a } \\otimes b ^ { \\dagger } { \\tilde { b } } | 0 \\rangle _ { b } \\} .\nS _ { E } = \\int _ { 0 } ^ { \\tau } d \\tau \\left( { \\frac { 1 } { 2 } } x _ { \\tau } ^ { 2 } + { \\frac { 1 } { 2 } } W ^ { 2 } ( x ) - \\psi ^ { \\ast } [ \\partial _ { \\tau } - W ^ { \\prime } ( x ) ] \\psi \\right)\nR _ { \\mu \\nu \\, \\, b } ^ { \\quad a } = \\partial _ { \\mu } \\omega _ { \\nu \\, \\, b } ^ { \\, a } - \\partial _ { \\nu } \\omega _ { \\mu \\, \\, b } ^ { \\, a } + \\omega _ { \\mu \\, \\, c } ^ { \\, a } \\ast \\omega _ { \\nu \\, \\, b } ^ { \\, c } - \\omega _ { \\nu \\, \\, c } ^ { \\, a } \\ast \\omega _ { \\mu \\, \\, b } ^ { \\, c }\nM = \\int _ { r \\rightarrow \\infty } d ^ { p } x r ^ { p / 2 } f ^ { - 1 / 2 } T _ { t t } = \\frac { p m V _ { p } } { 1 6 \\pi G _ { p + 2 } } .\nj _ { H W } ( x ) = W _ { i } ( x ) T ^ { i } \\; , \\; T ^ { i } \\in k e r ( A d ( M _ { - } ) )\nk _ { 0 } \\sim \\omega \\sqrt { \\frac { g \\phi _ { 0 } } { 2 M ^ { 2 } } } \\ll \\omega\nY ( T , U ) = \\int _ { \\cal F } \\frac { d ^ { 2 } \\tau } { \\Im \\tau } \\Gamma _ { 2 , 2 } ( T , U ) \\left( - 6 \\left[ { \\overline { { \\Omega } } } _ { 2 } ^ { \\phantom { 2 } } - \\frac { 1 } { 8 \\pi \\Im \\tau } \\right] \\frac { \\overline { { \\Omega } } } { \\overline { { \\eta } } ^ { 2 4 } } - \\frac { \\overline { { j } } } { 8 } + 1 2 6 \\right) \\ ,\nP _ { 0 } S ^ { * } P _ { 0 } S P _ { 0 } = P _ { 0 } = P _ { 0 } S P _ { 0 } S ^ { * } P _ { 0 } \\quad \\Longleftrightarrow \\quad ( S _ { 0 0 } ) ^ { * } = ( S _ { 0 0 } ) ^ { - 1 } \\quad \\mathrm { o n } \\quad \\mathcal { H } _ { \\mathrm { p h y s } } .\n\\langle \\psi _ { F a } ^ { 1 - a } \\mid \\phi _ { F a ^ { \\prime } } ^ { 1 - a ^ { \\prime } } \\rangle _ { t } = \\frac { 1 } { 2 } \\delta ( a - a ^ { \\prime } ) \\theta ( t - 1 + a ) \\theta ( t - 1 + a ^ { \\prime } )\nL ( z , u _ { a } , D ) \\equiv \\int _ { 0 } ^ { \\infty } d \\hat { T } \\, J ( z , u _ { a } , \\hat { T } , D ) = L _ { 0 2 } ( z , u _ { a } , D ) + g ( z , D ) G _ { B a b } ^ { 1 - { \\frac { D } { 2 } } } + O \\bigl ( z ^ { 4 } , G _ { B a b } ^ { 2 - { \\frac { D } { 2 } } } \\bigr ) \\nonumber \\,\n\\int _ { - \\epsilon } ^ { \\infty } d l \\: \\mathrm { e } ^ { - l \\zeta } \\int _ { - \\epsilon } ^ { \\infty } d l ^ { \\prime } \\mathrm { e } ^ { - l ^ { \\prime } \\zeta } l l ^ { \\prime } { \\frac { l ^ { \\prime } - l } { l + l ^ { \\prime } } } \\{ 3 \\, \\delta ^ { \\prime \\prime } ( l ) - { \\frac { 3 } { 4 } } t \\, \\delta ( l ) \\} = 0 .\n[ { \\bar { K } } _ { a } ^ { - } ( p ) , { \\bar { K } } _ { b } ^ { - } ( k ) ] = i f _ { a b c } { \\bar { K } } _ { c } ^ { - } ( p + k ) - 2 { \\pi } ^ { 2 } C p { \\delta } _ { a b } { \\delta } _ { p + k , 0 } .\nE ( v ) = \\frac { d } { d t } E ( q ) \\; \\; \\; \\; \\; \\; \\; \\forall t \\, .\n\\begin{array} { l l } { \\sigma ^ { 0 } = - { \\bf 1 } _ { 2 \\times 2 } = { \\bar { \\sigma } } ^ { 0 } , } & { \\sigma ^ { 1 } = \\tau _ { z } = - { \\bar { \\sigma } } ^ { 1 } , } \\\\ { \\sigma ^ { 2 } = \\tau _ { x } = - { \\bar { \\sigma } } ^ { 2 } , } & { \\sigma ^ { 3 } = \\tau _ { y } = - { \\bar { \\sigma } } ^ { 3 } , \\quad ( \\mathrm { o r } \\quad \\sigma ^ { y } = \\tau _ { y } = - { \\bar { \\sigma } } ^ { y } ) , } \\\\ \\end{array}\n{ \\frac { 1 } { L ^ { 2 } } } \\prod _ { i = 1 } ^ { 3 } ( r _ { + } ^ { 2 } + q _ { i } ) - \\mu r _ { + } ^ { 2 } \\sim 0 \\ .\n\\Lambda _ { - 1 , 1 } = \\lambda ^ { - 1 } ( e _ { 1 , 2 p } - e _ { 2 , 2 p + 1 } ) + ( e _ { 2 , 1 } - e _ { 2 p + 1 , 2 p } ) + 2 \\sum _ { k = 1 } ^ { p - 1 } e _ { 2 + k , 1 + k } - 2 \\sum _ { k = 1 } ^ { p - 1 } e _ { p + 1 + k , p + k } ,\nx ^ { I } ( \\sigma + 2 \\pi , \\tau ) = x ^ { I } ( \\sigma , \\tau ) + 2 \\pi L ^ { I } .\n\\xi _ { \\sigma } ( \\sigma ) = 1 - \\frac { \\pi e ^ { \\sigma } } { 3 } + 8 \\pi e ^ { \\sigma } \\sum _ { n = 1 } ^ { \\infty } \\frac { n e ^ { - 2 n \\pi e ^ { \\sigma } } } { 1 - e ^ { - 2 n \\pi e ^ { \\sigma } } }\nm _ { 0 } ^ { 2 } \\varphi + { \\frac { \\mu ^ { 2 } - m _ { 0 } ^ { 2 } } { \\beta } } \\operatorname { t a n } ( \\beta \\varphi ) = 0\ne _ { \\bot } ( x _ { \\bot } ) = g \\nabla _ { \\bot } \\int d y ^ { - } d y _ { \\bot } d ( x _ { \\bot } - y _ { \\bot } ) \\left\\{ f ^ { 3 a b } A _ { \\bot } ^ { a } ( y _ { \\bot } , y ^ { - } ) \\Pi _ { \\bot } ^ { b } ( y _ { \\bot } , y ^ { - } ) + \\rho _ { m } ^ { 3 } ( y _ { \\bot } , y ^ { - } ) \\right\\} \\frac { \\tau ^ { 3 } } { 2 } \\, .\n{ \\frac { d z } { d r } } = e ^ { 2 A } , \\hspace { 1 c m } \\psi \\rightarrow e ^ { - 3 A / 2 } \\psi\nF _ { A _ { 1 } } \\left( q ^ { a _ { 1 } } , \\dot { q } ^ { a _ { 1 } } \\right) = 0 \\, , \\; F _ { A _ { 0 } } \\left( q ^ { a _ { 1 } } \\right) = 0 \\, .\n1 - \\frac { \\delta \\phi } { 2 \\pi } = L ^ { \\prime } ( \\infty ) = \\frac { 2 G W + 1 } { N ^ { 2 } ( \\infty ) } .\n\\varphi ^ { \\prime \\prime } + { \\frac { 1 } { r } } \\varphi ^ { \\prime } + \\lambda \\left( \\varphi - \\varphi ^ { 3 } \\right) = 0 ,\nx ^ { N } + y ^ { N } = z ^ { N } , ~ ~ \\Phi ( l ) = \\omega ^ { l ( l + N ) / 2 } , ~ ~ \\omega ^ { 1 / 2 } = \\operatorname { e x p } ( \\pi i / N ) .\n{ e _ { \\hat { \\mu } } ^ { \\ \\hat { m } } = \\left( \\begin{array} { l l } { e _ { \\sigma } ^ { \\ s } } & { 0 } \\\\ { 0 } & { e _ { \\alpha } ^ { \\ a } } \\\\ \\end{array} \\right) }\n\\phi = \\left( \\begin{array} { c } { 0 } \\\\ { \\phi _ { 0 } } \\\\ \\end{array} \\right)\nS = { \\frac { \\pi ^ { 3 } R ^ { 5 } } { 2 \\kappa _ { 1 0 } ^ { 2 } } } \\left[ \\; \\int _ { 0 } ^ { 1 } \\! d u \\, d ^ { 4 } x \\, \\sqrt { - g } \\left( { \\cal R } - 2 \\Lambda \\right) + 2 \\int \\! d ^ { 4 } x \\, \\sqrt { - h } K + a \\int \\! d ^ { 4 } x \\, \\sqrt { - h } \\; \\right] \\, .\ng _ { J _ { 1 } \\, J _ { 2 } } ^ { J } \\bigl ( J _ { 1 } , M _ { 1 } ; J _ { 2 } , M _ { 2 } \\vert J _ { 1 } , J _ { 2 } ; J , M _ { 1 } + M _ { 2 } \\bigr ) \\, \\Bigl ( \\xi _ { M _ { 1 } + M _ { 2 } } ^ { ( J ) } ( \\sigma _ { 1 } ) + \\mathrm { d e s c e n d a n t s } \\Bigr ) \\Bigr \\} ,\n\\mathrm { T r } ( { \\bf M } _ { + } { \\bf M } _ { - } ) = 0 \\ ,\n\\int _ { { \\cal C } _ { i } } B _ { 2 } = 4 \\pi ^ { 2 } \\alpha ^ { \\prime } \\frac { 1 } { 2 } \\equiv b _ { 0 }\n\\alpha ^ { 2 } = { \\frac { Q _ { c } ^ { 2 } } { a } } ,\n\\sum _ { j = 1 } ^ { r + 1 } \\bar { x } _ { j } ^ { 2 } = { \\frac { r ( r + 1 ) } { 2 } } ,\nR ^ { ( 1 ) } = R _ { 1 2 } \\, \\mathrm { o r } \\, R _ { 2 1 } ^ { - 1 } \\, , \\quad R ^ { ( 2 ) } = R _ { 2 1 } \\, , \\quad R ^ { ( 3 ) } = R _ { 1 2 } \\, , \\quad R ^ { ( 4 ) } = R _ { 1 2 } ^ { - 1 } \\, \\mathrm { o r } \\, R _ { 2 1 } \\, .\n\\left( \\alpha ^ { \\prime } \\right) ^ { - 1 } e ^ { - \\phi / 2 } f _ { 1 6 } \\, s i m \\ g ^ { - 2 4 } \\sqrt { N } \\ ,\n( R _ { a b } R ^ { a b } - \\frac { n } { 4 ( n - 1 ) } R ^ { 2 } ) \\phi ^ { \\frac { 2 ( n - 4 ) } { n - 2 } }\nj _ { 1 } ^ { k } = \\omega _ { 1 } ^ { k - 2 } \\subseteq \\omega _ { 1 } ^ { k }\nV _ { e f f } ( z ) = \\frac { 3 ( 5 \\alpha + 4 ) k ^ { 2 } } { 6 4 \\left( { \\frac { k z } { 2 } } + 1 \\right) ^ { 2 } } + { \\frac { l ^ { 2 } } { 2 R ^ { 2 } } } \\left( { \\frac { k z } { 2 } } + 1 \\right) ^ { { \\frac { 5 } { 2 } } ( \\alpha - 2 ) } - { \\frac { 3 k } { 4 } } \\delta ( z )\nR : ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \\alpha ^ { \\prime } M ^ { 2 } = - 1 , 0 , 1 , 2 , 3 , . . . . . . . . . . .\n\\hat { \\mathrm { P } } _ { \\mathrm { R } } ^ { s } = \\frac { 1 } { 2 } \\hbar \\int d x ( { \\psi } _ { \\mathrm { R } } ^ { \\dagger s } ( - i \\partial _ { x } ) { \\psi } _ { \\mathrm { R } } ^ { s } - { \\psi } _ { \\mathrm { R } } ^ { s } ( i \\partial _ { x } ) { \\psi } _ { \\mathrm { R } } ^ { \\dagger s } ) - \\int d x \\hat { \\mathrm { E } } { \\partial _ { 1 } } A _ { 1 } .\nK ^ { \\prime } = \\left( \\begin{array} { c c } { K _ { 1 1 } } & { \\begin{array} { c c } { \\cline { 1 - 1 } \\mathrm { \\hspace { 2 p t } ~ \\small ~ - g ~ \\hspace { 1 p t } } } & { \\vline \\hfill } \\\\ \\end{array} } \\\\ { \\hline \\begin{array} { c c } { \\hfill \\vline } & { \\mathrm { \\hspace { 2 p t } ~ \\small ~ - g ~ \\hspace { 1 p t } } } \\\\ { \\cline { 2 - 2 } \\hspace { 4 p t } 0 \\hspace { 4 p t } } & { } \\\\ \\end{array} } & { K _ { 2 2 } ^ { \\prime } } \\\\ \\end{array} \\right) \\, .\na ( \\xi ) = \\frac { \\xi ^ { - 1 } - \\xi } { q ^ { - 1 } \\xi ^ { - 1 } - q \\xi } \\, , \\quad b ( \\xi ) = \\frac { q ^ { - 1 } - q } { q ^ { - 1 } \\xi ^ { - 1 } - q \\xi } \\, .\n\\phi ( \\rho ) = \\left\\{ \\begin{array} { l l } { \\phi ^ { ( 1 ) } ( \\rho ) } & { i n r e g i o n 1 } \\\\ { \\bar { \\phi } ^ { ( 1 ) } ( \\rho ) } & { i n r e g i o n 1 ^ { * } } \\\\ { ( - 1 ) ^ { - d _ { \\phi } } \\, \\phi ^ { ( 2 ) } ( - \\rho ) } & { i n r e g i o n 2 } \\\\ { ( - 1 ) ^ { - d _ { \\phi } } \\, \\bar { \\phi } ^ { ( 2 ) } ( - \\rho ) } & { i n r e g i o n 2 ^ { * } . } \\\\ \\end{array} \\right.\nx ^ { \\pm } = \\frac { \\left( x ^ { 0 } \\pm x ^ { 1 } \\right) } { \\sqrt { 2 } }\nF _ { i _ { 1 } , \\dots , i _ { n } } ( \\beta _ { 1 } , \\dots , \\beta _ { n } + 2 \\pi i ) = \\operatorname { e x p } [ 2 \\pi i \\omega ( { \\cal O } , \\Psi ) ] F _ { i _ { n } , i _ { 1 } , \\dots , i _ { n - 1 } } ( \\beta _ { n } , \\beta _ { 1 } , \\dots , \\beta _ { n - 1 } ) .\na _ { 0 } [ [ a _ { 0 } , a _ { n } ] , a _ { 0 } ] = a _ { 0 } [ [ a _ { 0 } , a _ { n } ] _ { 1 } , a _ { 0 } ] \\hbar + a _ { 0 } [ [ a _ { 0 } , a _ { n } ] _ { 2 } , a _ { 0 } ] \\hbar ^ { 2 } .\ng _ { i j } = \\bar { g } _ { i j } + h _ { i j } ,\nU ( \\vec { x } ; A , B ) = A U _ { \\mathrm { S } } ( D ( B ) ^ { - 1 } \\cdot \\vec { x } ) A ^ { \\dagger }\n{ \\frac { \\partial \\bar { g } _ { \\mu \\nu } } { \\partial l _ { i } } } \\in \\mathrm { K e r } ( \\bar { F } ^ { \\dagger } )\nJ _ { \\mu } = i \\overline { { \\Psi } } ( x ) \\Gamma _ { \\mu } \\Psi ( x ) , \\hspace { 0 . 3 i n } K _ { \\mu } = \\overline { { \\Psi } } ( x ) \\Gamma _ { \\mu } \\overline { { \\Gamma } } _ { 5 } \\Psi ( x ) , \\hspace { 0 . 3 i n } R _ { \\mu \\alpha } = \\overline { { \\Psi } } ( x ) \\Gamma _ { \\mu } \\overline { { \\Gamma } } _ { 5 } \\overline { { \\Gamma } } _ { \\alpha } \\Psi ( x ) ,\n{ \\cal D } ( g ) = \\left( \\begin{array} { c c c } { z _ { 1 } ^ { \\prime } } & { - \\bar { z } _ { 2 } ^ { \\prime } } & { - \\chi } \\\\ { z _ { 2 } ^ { \\prime } } & { \\bar { z } _ { 1 } ^ { \\prime } } & { - \\bar { \\chi } } \\\\ { \\theta ^ { \\prime } } & { - \\bar { \\theta } ^ { \\prime } } & { \\lambda } \\\\ \\end{array} \\right) \\, , \\quad \\quad \\sum _ { i } | z _ { i } ^ { \\prime } | ^ { 2 } + \\bar { \\theta } ^ { \\prime } \\theta ^ { \\prime } = 1\nL _ { a } = \\frac { 1 } { 1 6 \\pi } \\mathrm { I m } \\int d ^ { 2 } \\theta \\, W ^ { 2 } \\tau ( z ^ { - 1 / 4 } )\n+ \\, \\, \\mathrm { e } ^ { \\varphi } \\, { F } _ { [ 3 ] } ^ { R R } \\wedge \\star { F } _ { [ 3 ] } ^ { R R } \\, + \\, \\frac { 1 } { 2 } \\, { F } _ { [ 5 ] } ^ { R R } \\wedge \\star { F } _ { [ 5 ] } ^ { R R } \\, - \\, C _ { [ 4 ] } \\wedge F _ { [ 3 ] } ^ { N S } \\wedge F _ { [ 3 ] } ^ { R R } \\Big ] \\Bigg \\}\nS [ \\phi ^ { \\dagger } , \\phi ] \\; = \\; \\int d ^ { 3 } x \\, { \\cal L } ( \\phi ^ { \\dagger } , \\phi ; \\partial \\phi ^ { \\dagger } , \\partial \\phi ) \\; \\; ,\nG ^ { K } \\, = \\, G ^ { 0 } + G ^ { 0 } K ^ { R } G ^ { 0 } + G ^ { 0 } K ^ { R } G ^ { 0 } K ^ { R } G ^ { 0 } + \\cdots \\equiv \\, G ^ { 0 } + G ^ { K R } .\n\\partial _ { \\mu } = \\frac { \\partial } { \\partial \\xi ^ { \\mu } }\n( \\partial _ { \\gamma } , \\partial _ { \\gamma } ) _ { ( 1 0 ) } = \\sqrt { \\tilde { \\Delta } } e ^ { 2 U }\n3 . 8 G = \\eta _ { a b } d z ^ { a } \\otimes d z ^ { b } = - d z ^ { 0 } \\otimes d z ^ { 0 } + \\sum _ { i = 1 } ^ { n - 1 } d z ^ { i } \\otimes d z ^ { i } ,\n\\int _ { - \\infty } ^ { \\infty } P _ { r } ^ { \\bf V } ( x ) P _ { s } ^ { \\bf V } ( x ) \\, e ^ { - x ^ { 2 } } \\, d x \\, \\propto \\delta _ { r \\, s } .\n\n\\Phi ^ { i } = \\frac { 1 } { 2 } \\pi ^ { i } - \\int d u \\; \\; \\epsilon ( x - u ) \\; \\theta ^ { i } ( u )\n{ \\cal H } _ { i n t } ( x ) = - e \\overline { { \\psi } } ^ { ( 0 ) } ( x ) \\beta ^ { \\mu } A _ { \\mu } ^ { ( 0 ) } ( x ) [ I + \\frac { e } { m } ( I - \\beta _ { 0 } ^ { 2 } ) \\beta ^ { \\nu } A _ { \\nu } ^ { ( 0 ) } ( x ) ] \\psi ^ { ( 0 ) } ( x ) \\, \\, ,\nQ = ( b + 1 / b ) \\rho , \\qquad \\rho = \\frac 1 2 \\sum _ { \\alpha > 0 } \\alpha ,\n{ h _ { 0 } ^ { 0 } } ( x ^ { \\mu } ) = \\sum { b _ { 0 } ^ { 0 } } ( t , z ) Y ( x ^ { i } ) , \\quad { h _ { z } ^ { 0 } } ( x ^ { \\mu } ) = \\sum { b _ { z } ^ { 0 } } ( t , z ) Y ( x ^ { i } ) ,\nP ^ { 2 } = \\bigg ( \\frac { 4 } { \\pi h ^ { 2 } N } \\bigg ) ^ { 2 } \\bigg [ 1 + O \\bigg ( \\frac { 1 } { P ^ { 2 } } \\bigg ) \\bigg ] , \\; \\; \\pi h ^ { 2 } N < < 1 .\nV _ { \\mathrm { e f f } } = \\operatorname* { l i m } _ { \\lambda \\rightarrow 0 } \\left( V ^ { g e n } + V ^ { P T } \\right) .\n2 \\mu { \\frac { \\sqrt { J _ { 1 } J _ { 2 } J _ { 3 } } } { N } } \\big ( \\bar { O } ^ { \\prime J _ { 3 } } \\Pi ^ { \\prime J _ { 1 } } \\bar { \\Pi } ^ { \\prime J _ { 2 } } \\delta _ { J _ { 3 } + J _ { 1 } , J _ { 2 } } + O ^ { J _ { 3 } } \\Pi ^ { \\prime J _ { 1 } } \\bar { \\Pi } ^ { \\prime J _ { 2 } } \\delta _ { J _ { 3 } + J _ { 2 } , J _ { 1 } } \\big ) .\n{ \\theta _ { n } ^ { \\alpha \\Lambda } } { ^ \\dagger } = \\theta _ { - n } ^ { \\alpha \\Lambda } ,\n{ \\cal S } ^ { \\mu \\nu } \\equiv \\epsilon ^ { \\mu \\nu \\rho \\sigma } \\theta _ { \\rho } P _ { \\sigma } = 0 \\ \\ ,\n\\operatorname { e x p } \\left( \\sum _ { g = 0 } ^ { \\infty } x ^ { 2 g - 2 } \\mathcal { F } _ { g } \\right) = e ^ { - \\frac { D ( 0 , 0 , 0 ) } { 2 } \\zeta ( 1 ) } \\prod _ { ( \\ell , n , \\gamma , j ) > 0 } ( 1 - p ^ { \\ell } q ^ { n } \\zeta ^ { \\gamma } y ^ { j } ) ^ { D ( \\ell n , \\gamma , j ) } \\, .\nM _ { P } ^ { 2 } = M _ { s } ^ { 3 } \\int d z e ^ { - 2 \\sigma ( z ) }\n\\begin{array} { c } { p ^ { A } = p _ { 0 } ^ { A } } \\\\ { x ^ { A } \\left( \\tau \\right) = x _ { 0 } ^ { A } + 2 \\left( E _ { 1 } + E _ { 3 } \\delta _ { 2 } ^ { 2 } \\right) p _ { 0 } ^ { A } - m ^ { - 2 } V ^ { A } \\left( \\tau \\right) } \\\\ { V ^ { A } \\left( \\tau \\right) = V _ { 1 } ^ { A } \\operatorname { c o s } \\left( 2 m ^ { 2 } E _ { 3 } \\delta _ { 2 } \\right) + V _ { 2 } ^ { A } \\operatorname { s i n } \\left( 2 m ^ { 2 } E _ { 3 } \\delta _ { 2 } \\right) } \\\\ \\end{array}\n\\phi \\Big [ \\beta , a , b , c , d \\Big ] = \\phi _ { 0 } ^ { \\beta } + a \\, \\phi _ { 0 } ^ { \\beta + 1 } + b \\, \\phi _ { 0 } ^ { \\beta + 2 } + c \\, \\phi _ { 0 } ^ { \\beta + 3 } + d \\, \\phi _ { 0 } ^ { \\beta + 4 } \\; .\nt _ { 1 } = T ~ ~ , \\hspace { 2 c m } t _ { 2 } = - \\frac { S } { 2 \\pi i } ,\nB _ { p } ( \\beta ) = \\operatorname { d i m } \\{ \\operatorname { k e r } \\Delta _ { \\beta } \\cap \\Lambda ^ { p } \\} .\n\\frac { 1 } { 2 } \\left( 1 + \\sqrt { \\frac { j \\left( j + 1 \\right) - \\alpha ^ { 2 } } { 4 } } \\right) \\mp \\frac { i \\alpha E } { \\sqrt { E ^ { 2 } - \\frac { m ^ { 2 } } { 4 } } } = - n\n\\hat { g } = - ( d x ^ { 0 } ) ^ { 2 } + ( d x ^ { 1 } ) ^ { 2 } + \\Big ( d r ^ { 2 } + r ^ { 2 } d \\Omega _ { 8 } ^ { 2 } \\Big ) \\; ,\n\\rho _ { \\alpha \\beta } ^ { ( 1 ) } = \\left( \\begin{array} { c c c c c } { 0 } & { - m \\delta _ { i j } } & { e \\delta _ { i j } \\delta ( \\vec { y } - \\vec { q } ) } & { 0 } & { 0 } \\\\ { m \\delta _ { i j } } & { 0 } & { 0 } & { 0 } & { 0 } \\\\ { - e \\delta _ { i j } \\delta ( \\vec { y } - \\vec { q } ) } & { 0 } & { 0 } & { \\theta \\delta _ { i j } \\delta ( \\vec { x } - \\vec { y } ) } & { 0 } \\\\ { 0 } & { 0 } & { - \\theta \\delta _ { i j } \\delta ( \\vec { x } - \\vec { y } ) } & { 0 } & { 2 \\theta \\partial _ { i } \\delta ( \\vec { x } - \\vec { y } ) } \\\\ { 0 } & { 0 } & { 0 } & { - 2 \\theta \\partial _ { j } \\delta ( \\vec { x } - \\vec { y } ) } & { 0 } \\\\ \\end{array} \\right)\n( - 1 ) ^ { m \\cdot | a | } [ a , b \\cup c ] = [ a , b ] \\cup c + ( - 1 ) ^ { ( | b | \\cdot | c | + m ) } [ a , c ] \\cup b .\n\\operatorname* { l i m } _ { t \\rightarrow - \\infty } ( f ( x ) , \\phi ( x - t ) ) = L _ { 0 } ^ { - 1 } [ f ] \\int _ { - \\infty } ^ { \\infty } \\phi ( x ) d x\nI ( a ) = \\kappa \\int _ { 0 } ^ { 2 a } { \\frac { u ^ { 3 } } { 2 } } g _ { 2 } ( u )\nE ^ { 2 } ( R > > k ) = \\frac { 2 } { k } [ j ( j + 1 ) - m ^ { 2 } + ( m + \\frac { k M } { 2 } ) ^ { 2 } \\frac { 1 } { R ^ { 2 } } ]\nm _ { C } ^ { 2 } = \\langle g ^ { 2 } A _ { \\mu } ^ { a } A _ { \\mu } ^ { a } \\rangle + \\left( { \\frac { 1 } { 2 } } \\alpha + \\beta \\right) \\langle i g ^ { 2 } \\bar { C } ^ { a } C ^ { a } \\rangle .\ng _ { L } ^ { 2 } V _ { L } ( \\alpha \\cdot q ) { { } \\atop { \\longrightarrow \\atop { \\omega _ { 3 } \\to + \\infty } } } \\left\\{ \\begin{array} { c l } { m _ { L } ^ { 2 } \\, e ^ { \\alpha \\cdot Q } } & { \\mathrm { f o r ~ s u c h ~ \\( \\alpha \\in \\Delta _ + \\) ~ t h a t ~ \\( \\alpha \\cdot ~ v \\) ~ i s ~ m i n i m u m , } } \\\\ { m _ { L } ^ { 2 } \\, e ^ { - \\alpha \\cdot Q } } & { \\mathrm { f o r ~ s u c h ~ \\( \\alpha \\in \\Delta _ + \\) ~ t h a t ~ \\( \\alpha \\cdot ~ v \\) ~ i s ~ m a x i m u m , } } \\\\ { 0 } & { \\mathrm { o t h e r w i s e , } } \\\\ \\end{array} \\right.\nr a n k ( b ( x ; \\xi ) \\ q ( x ; \\xi ) ) = r a n k ( q ( x ; \\xi ) ) = r \\ ,\n= \\int d ^ { n } x \\left\\{ \\frac { 1 } { 2 } F _ { \\alpha \\beta } F ^ { \\alpha \\beta } + \\frac { 1 } { 2 } G _ { \\alpha \\beta } G ^ { \\alpha \\beta } + 2 D _ { \\alpha } \\phi ^ { \\dagger } D ^ { \\alpha } \\phi + 2 ( m ^ { 2 } - \\phi ^ { \\dagger } \\phi ) ^ { 2 } \\right\\}\n- \\frac { T } { 8 \\pi } \\left( M _ { W } \\left( 9 M _ { W } ^ { 2 } + \\frac { 3 } { 2 } M _ { H } ^ { 2 } \\right) + \\frac { 3 } { 2 } M _ { H } ^ { 3 } \\right) \\int d ^ { 3 } x ( H _ { 0 } ^ { 2 } - 1 ) .\nV = i \\left[ \\begin{array} { c c } { B } & { 2 \\overline { { A } } } \\\\ { - 2 A } & { - B } \\\\ \\end{array} \\right]\n\\sum _ { { \\lambda } = 1 } ^ { 2 } { \\epsilon } _ { \\mu } ^ { ( { \\lambda } ) } ( k ) { \\epsilon } _ { \\nu } ^ { ( { \\lambda } ) } ( k ) = - g _ { { \\mu } { \\nu } } + \\frac { n _ { \\mu } k _ { \\nu } + n _ { \\nu } k _ { \\mu } } { k _ { - } } - n ^ { 2 } \\frac { k _ { \\mu } k _ { \\nu } } { k _ { - } ^ { 2 } } .\n\\epsilon _ { \\mathrm { v a c } } = \\left< \\frac { \\beta ( g ) } { 8 g } G \\cdot G \\right>\n\\tau _ { j } = 1 - y \\operatorname { e x p } ( i ( \\mu _ { a } ( 2 j ) - \\varepsilon ) ) .\n\\alpha = \\sum _ { a = 1 } ^ { J } \\alpha ( a ; \\infty ) .\nE _ { A D M } = \\frac { 1 } { 1 6 \\pi G _ { 1 0 } } \\oint _ { \\infty } d \\Sigma ^ { m } \\lbrace { } ^ { \\circ } D _ { n } g _ { m p } - { } ^ { \\circ } D _ { m } g _ { n p } \\rbrace { } ^ { \\circ } g ^ { n p }\nJ ( x , y , M ) = \\mathrm { t r } \\left\\langle x \\left| \\Gamma _ { 5 } \\Gamma \\frac { 1 } { \\Theta + M } \\right| y \\right\\rangle .\n\\chi _ { h } ^ { N S } ( q ) = \\sum S _ { h } ^ { h ^ { \\prime } } \\chi _ { h ^ { \\prime } } ^ { N S } ( \\tilde { q } )\n\\mathrm { S p D } _ { 5 6 } \\left( E ^ { { \\vec { \\alpha } } _ { 5 } } \\right) = \\left( \\begin{array} { l l } { { \\bf 0 } } & { B [ { { \\vec { \\alpha } } _ { 5 } } ] } \\\\ { C [ { { \\vec { \\alpha } } _ { 5 } } ] } & { { \\bf 0 } } \\\\ \\end{array} \\right)\n\\phi _ { \\epsilon } ( \\beta ) = { \\sqrt { s } } ( Q - i \\beta P ) + \\sum _ { k \\neq 0 } { \\frac { a _ { k } } { i \\operatorname { s i n h } ( \\pi k \\epsilon ) } } \\operatorname { e x p } ( i k \\epsilon \\beta ) ,\nr = \\rho \\; , \\; \\; f = 1 - \\frac { \\rho _ { 0 } } { \\rho } - \\frac { \\Lambda } { 6 } \\rho ^ { 2 } \\; .\n{ \\cal U } ^ { \\prime \\prime } + { \\frac { 2 } { r } } { \\cal U } ^ { \\prime } - \\left( { \\cal U } ^ { \\prime } \\right) ^ { 2 } = { \\frac { 1 } { 4 } } \\, \\left( h _ { 1 } ^ { \\prime } \\right) ^ { 2 } + { \\frac { 1 } { 4 } } \\left( h _ { 2 } ^ { \\prime } \\right) ^ { 2 } ,\n\\mathbf { \\nabla \\cdot } { \\mathbf { K } } ^ { C } = 4 \\pi { \\mathbf { J } } ^ { C } \\qquad \\mathbf { \\nabla \\cdot } \\; ^ { * } { \\mathbf { K } } ^ { C } = 0\n\\not \\! \\! D _ { V } : { \\cal C } ^ { \\infty } ( { \\cal S } ^ { \\pm } ) \\otimes { \\cal C } ^ { \\infty } ( V ) \\longrightarrow { \\cal C } ^ { \\infty } ( { \\cal S } ^ { \\mp } ) \\otimes { \\cal C } ^ { \\infty } ( V )\n{ A _ { 4 } ^ { Y M } = \\langle T _ { \\mu _ { 1 } \\nu _ { 1 } } ( 1 ) T _ { \\mu _ { 2 } \\nu _ { 2 } } ( 2 ) T _ { \\mu _ { 3 } \\nu _ { 3 } } ( 3 ) T _ { \\mu _ { 4 } \\nu _ { 4 } } ( 4 ) \\rangle = N ^ { 2 } A _ { 4 } ^ { ( 1 ) } + \\tilde { k } N ^ { 1 / 2 } f ^ { ( 0 , 0 ) } ( S , \\bar { S } ) A _ { 4 } ^ { ( 2 ) } + \\dots , }\nS \\left( \\hat { A } + \\delta A \\right) = S \\left( \\hat { A } \\right)\np _ { 1 } ^ { \\mu } \\, = \\, m \\, \\frac { q _ { 2 } ^ { \\mu } } { \\sqrt { q _ { 2 } ^ { 2 } } } \\, - \\, ( p _ { 1 } b ) \\, b ^ { \\mu } .\n\\int _ { - \\infty } ^ { \\infty } \\frac { d x } { ( \\gamma + x ^ { 2 } ) ( \\delta + x ^ { 2 } ) } = \\frac { \\pi } { \\sqrt { \\gamma \\delta } ( \\sqrt { \\gamma } + \\sqrt { \\delta } ) } = \\frac { \\pi } { \\sqrt { \\gamma \\delta } } \\frac { \\sqrt { \\delta } - \\sqrt { \\gamma } } { \\delta - \\gamma } ,\n\\delta _ { \\epsilon } \\Psi = \\epsilon \\star \\Psi - \\Psi \\star \\epsilon \\, \\, .\n| \\varphi _ { k } \\dot { \\varphi } _ { k ^ { \\prime } } - \\dot { \\varphi } _ { k } \\varphi _ { k ^ { \\prime } } | = | \\dot { \\varphi } _ { k } | ^ { 2 } | \\varphi _ { k ^ { \\prime } } | ^ { 2 } + | \\dot { \\varphi } _ { k } | ^ { 2 } | \\varphi _ { k ^ { \\prime } } | ^ { 2 } + \\frac 1 2 ( \\dot { \\varphi } _ { k } \\varphi _ { k } ^ { * } + \\varphi _ { k } \\dot { \\varphi } _ { k } ^ { * } ) ( \\dot { \\varphi } _ { k ^ { \\prime } } \\varphi _ { k ^ { \\prime } } ^ { * } + \\varphi _ { k ^ { \\prime } } \\dot { \\varphi } _ { k ^ { \\prime } } ^ { * } ) + 2\n\\left[ a , b , c \\right] = a b c + b c a + c a b - c b a - a c b - b a c ,\nZ _ { R } ^ { ( m , n ) } = \\frac { 1 } { 4 } \\sum _ { e } C _ { e } ( m , n ) \\frac { \\theta ^ { 4 } [ e ] } { \\eta ^ { 1 2 } } \\Theta _ { ( m , n ) } ( 0 | \\tau ) ;\n{ d s } ^ { 2 } = - V ( U ) { d t } ^ { 2 } + \\frac { { d U } ^ { 2 } } { V ( U ) } + \\frac { U ^ { 2 } } { R ^ { 2 } } \\sum _ { i = 1 } ^ { n - 1 } { d x } _ { i } ^ { 2 } ,\n"
  },
  {
    "path": "data/test.txt",
    "content": "7944775fc9.png 1000\n78228211ca.png 1001\n15b9034ba8.png 1002\n6968dfca15.png 1003\n6cead0df53.png 1004\n5381b22df4.png 1005\n27f2b37ce9.png 1006\n51a257cdf5.png 1007\n5108925e21.png 1008\n3882dd3d43.png 1009\n566cf0c6f5.png 1010\n7d1fe2cc05.png 1011\n450b24df87.png 1012\n667ff49bc5.png 1013\n61928de22b.png 1014\n4cd65285c9.png 1015\n12697ce419.png 1016\na8ec0c091c.png 1017\n72a80f57d9.png 1018\n330f27c566.png 1019\n58be3470dc.png 1020\n3e82680317.png 1021\n31068cb86d.png 1022\n431dd6944e.png 1023\n54a7b9d7f8.png 1024\n632e971eb8.png 1025\n399e18a85c.png 1026\n707b5988e2.png 1027\n25fe4d51bf.png 1028\n3dc7799669.png 1029\n76d30658bb.png 1030\n6a366e1f12.png 1031\nde8a312222.png 1032\n2b891b21ac.png 1033\n72e168fb21.png 1034\n3d129cfe77.png 1035\n6a85896075.png 1036\n79edbca78a.png 1037\n20032b2645.png 1038\n3d15b5c484.png 1039\n2608ceb605.png 1040\n146a5fa39e.png 1041\n159bf72783.png 1042\n1e3aab9a4f.png 1043\n62409f879c.png 1044\n6beab42e50.png 1045\n105ccc7946.png 1046\n6df7276525.png 1047\n65d07ed733.png 1048\n34173474c4.png 1049\n1a79f53af4.png 1050\n57e32e5b33.png 1051\n7e1098abc4.png 1052\n5ada9733aa.png 1053\n5b109d24dc.png 1054\n5193ae2c89.png 1055\n119b93a445.png 1056\n4fa61dbf37.png 1057\n25765b9391.png 1058\n276c373567.png 1059\n3fd05b449f.png 1060\n6b2c7f0c1a.png 1061\n7c2f256525.png 1062\n3beaade5a5.png 1063\n3cf56a8338.png 1064\n4aea73b2b8.png 1065\n2e96a960b1.png 1066\n5b10a20227.png 1067\n18049a05a9.png 1068\n17ad05c612.png 1069\n4b69ad5dc8.png 1070\n4acf2a0344.png 1071\n272667a2d1.png 1072\n5a5e2b80dd.png 1073\n6596750444.png 1074\n4370181032.png 1075\n11ff25534a.png 1076\n22f7232e98.png 1077\n3eaf444392.png 1078\n25c3276f55.png 1079\n32ebd66b47.png 1080\n3b014d22b2.png 1081\n6a88fd17f0.png 1082\n4b49a4f210.png 1083\n4b4263156b.png 1084\n48f89a8fc4.png 1085\n8d78ecda53.png 1086\n4015ba8922.png 1087\nacedffb147.png 1088\n4f055acd1f.png 1089\n5ecb739ec1.png 1090\n2beadd086b.png 1091\n2450656988.png 1092\n147dde7fd3.png 1093\n17d9b6a683.png 1094\n55358c150e.png 1095\n13dbb0dd7c.png 1096\n1d796ff39a.png 1097\n6a39a91654.png 1098\n6661b12767.png 1099\n"
  },
  {
    "path": "data/train.txt",
    "content": "60ee748793.png 0\n66667cee5b.png 1\n1cbb05a562.png 2\ned164cc822.png 3\ne265f9dc6b.png 4\n242a58bc3a.png 6\na4d25113b2.png 7\n72f6bc494a.png 8\n3cf9d0b234.png 9\n5be376c443.png 10\n4dd5a0e4ad.png 11\n6e7448ca84.png 12\n5adf6fe332.png 13\n4e9e19a3cf.png 14\n3a950d09c0.png 15\n5d1a1ed037.png 16\n698111df57.png 17\n73fdf824d0.png 18\n35a6b52146.png 19\n6fefdec123.png 20\n2a7a69318b.png 21\n2ce5749395.png 22\n9016b4fca0.png 23\n2cb87ed9c8.png 24\n4765b43e98.png 25\n33978e1330.png 26\n3945cf2343.png 27\n4c6c104eb5.png 28\n5d58861c3f.png 29\n2ae8eccc13.png 30\n56f7827473.png 31\n6176f74d0f.png 32\n662ccce98f.png 33\n2440895f67.png 34\n45b9b7323d.png 35\n672a31c2cc.png 36\n10c37c445e.png 37\n427968501c.png 38\n65d334ea47.png 39\n7ab1fc083e.png 40\n5dab1ec4de.png 41\n7d0867620d.png 42\n67fcffa9a3.png 43\n4b069edf09.png 44\n2b8e14887f.png 45\n1ba2cffb3c.png 47\n7dfe32d42b.png 48\n11421b7af6.png 49\n22a003507e.png 50\n17226e3e67.png 51\n43f707d9b6.png 52\n2df6c7abd6.png 53\n7d791b3d50.png 54\n1a87610486.png 56\n50919ea0ce.png 57\n7891985624.png 58\nae65a915db.png 59\n4375e58a64.png 60\n2c7002c337.png 61\n12521128dc.png 62\n25ad6cf3b0.png 63\n489da689b3.png 64\n224e79c77d.png 65\n4f86c93855.png 66\n5afd606d6c.png 67\n64f51e3fbc.png 68\n1fa9bb5655.png 69\n50c17d7ef4.png 70\n778496aed1.png 71\n5aa85bc04d.png 72\n6c2c59f99d.png 73\n39baa93854.png 74\n4849c785c7.png 75\n46b3873fbc.png 76\n712073fc17.png 77\n6ffa4086e2.png 78\n72e69b3f75.png 79\n2d0a8b7347.png 80\n616706dd5a.png 81\n1dd5d34448.png 82\n3cdc9f09c6.png 83\nc49990f9c9.png 84\n5f83ae277c.png 85\n22692da57d.png 86\ne2da14260e.png 87\n396ff4def5.png 88\n6bc6b638ef.png 89\n7fb58b3ce2.png 90\n1c5f0abd11.png 91\n4552f04e75.png 92\n4031e921fa.png 93\n78a41122f4.png 94\n5a8ba83188.png 95\n795aa0d8b2.png 96\n5fc2dd90f4.png 97\n61ff1fa86d.png 98\n58dfa3dbcf.png 99\n7b701d6aba.png 100\n5f96184332.png 101\n742b30d224.png 102\n67efc34ea9.png 103\n47a37debff.png 104\n2319dc19dd.png 105\n3f4ac552d5.png 107\n5a86fa9b1d.png 108\n7e82073cd5.png 110\n45e5afd956.png 111\n24d85808a1.png 112\n55d99738e5.png 113\n526038ea4c.png 114\n3c0c7b450c.png 115\n490a32bf20.png 116\n2763f1bb43.png 117\n2fd32ca817.png 119\n148efc192d.png 120\n1a9a0575e7.png 121\n7e3b3622f9.png 122\nff5d66560d.png 123\n1cd201bfa8.png 124\n57bb233b98.png 125\n3a13510507.png 127\n2358b58f46.png 128\n24b499dc70.png 129\n67f3e9e236.png 130\n52c55aa311.png 131\n31872ec89d.png 132\n787a1f5717.png 133\n5cdfccedbe.png 134\n55783f169a.png 135\na20e473646.png 136\n58e6560c6c.png 137\n2fdcb5efee.png 138\n121b29a34e.png 140\n4b9c24d197.png 141\n4a4f93cbe5.png 142\n7149ecb438.png 143\n7665dff25e.png 144\n1eb0468393.png 147\n12cc91e6d0.png 148\n3ab85e1c73.png 149\n57d8b6dd4a.png 150\n2350414e35.png 151\nc17e9094b3.png 152\n70eb7647b5.png 153\n6b36374d9d.png 154\n4b79ffccd9.png 156\n1481a60f40.png 157\n44a2ebab25.png 159\n40be12af5f.png 160\n62fd4fd186.png 161\n5b397b1cd0.png 162\ncbd7757104.png 163\n2983f1a42e.png 164\n6da72deb03.png 165\n5a0621bac5.png 167\n68641b1f35.png 168\n1de2776366.png 170\ncb1a57840a.png 171\n707fec2eb3.png 172\n4fe7cc11b0.png 173\n37b3dd5bd8.png 174\n1f1d28bc77.png 176\n5999b32439.png 177\n13ef0a5f7d.png 178\n7330bec29b.png 179\n1ddfa77a15.png 180\n532b6d7730.png 181\n3dddd02569.png 183\n4bb7d69576.png 184\n2350b6d78e.png 185\ndf04b95eb0.png 186\n3f0119b21c.png 187\n3ae7d4c8aa.png 188\n5a6fb6e214.png 189\n20a17f575f.png 190\n294619977e.png 191\n1d883568bc.png 192\n47cf711405.png 193\n4695858a93.png 194\n4a0cc80e74.png 195\n542bfd32ab.png 196\n6dbd31790b.png 197\n77fb781660.png 199\n607ee9e5c2.png 202\n3362c78722.png 203\n4fa16f1d3f.png 205\n7a54e991b0.png 206\n50a1f82ae6.png 207\n256f1e5b2a.png 208\n1c7a4617a5.png 209\n1de6b57b0c.png 210\n315201f078.png 211\n2d2e6105f4.png 212\n4e721cfe49.png 213\n42c766c6dc.png 214\n799f625cf9.png 215\n323c465216.png 216\n226979ee71.png 217\n4d603c88f4.png 218\n16a0874240.png 219\n386bf9a37b.png 220\n7352f31651.png 221\n5d3a4f9d53.png 223\n252f1b9cea.png 224\n4c475e1afd.png 225\n131a79148f.png 226\n33fcdc8633.png 227\n71feac6326.png 228\n1e8595e5e1.png 229\n3bf4f3ec2d.png 230\n2581f66470.png 231\n3b1cebad19.png 233\n395413c90f.png 234\n3b82cc7aee.png 235\n7b00f012b2.png 237\n9bcc0bb538.png 238\n1be344a743.png 239\n562c231867.png 240\n78ef714591.png 241\n173111e861.png 242\n3f0ef5c2d1.png 243\n3ab0d7b424.png 244\n11a8eea207.png 245\n738c6bc090.png 246\n478234cbe5.png 247\n1a882641b7.png 248\n81ba9ce4a9.png 249\n28a38729aa.png 250\n1c2883bfac.png 251\n3fdecf80a1.png 252\n5f9a3eb396.png 253\n580036eb68.png 254\n6157449337.png 255\n505cc30969.png 256\na43fa630b7.png 257\n36fbd5706b.png 258\n70a6e5b4b9.png 259\n1420ac42ed.png 260\n1e1bbaef2b.png 261\n347196fad5.png 262\n689d84bd57.png 263\n7199fd1e3c.png 264\n4f6c0a6083.png 265\n5c01b8b718.png 266\n7893ca4e1b.png 267\n54682978af.png 268\n3d6cbb761f.png 269\n47b0771bd8.png 271\n396bcb28c0.png 272\n629c42b571.png 273\n3bd3e38c6c.png 274\n149b313ecf.png 275\ne94fbdb3c3.png 276\n234d3db18a.png 277\n742d89ddeb.png 278\n6e8fa596b4.png 279\n49ca0559c8.png 280\n534c7843ec.png 281\n19c9f9219d.png 282\n6f49be4f68.png 283\n30a0e5b27c.png 285\n3b4dd4f61d.png 286\n7f22efe073.png 287\n277fac3048.png 288\nf1c1b75b1b.png 289\n5b7bdc01b4.png 290\n42b805b1d8.png 291\n3ca343759c.png 292\n1e216001c4.png 294\n5f71c12252.png 295\n4c32f7004f.png 296\n4b836d0a2c.png 297\ne6c3904a0c.png 298\n7dd35fa179.png 300\n20ceea5819.png 301\n51a6e5dd63.png 302\n177fda0427.png 303\ne6abb29b53.png 304\n33274d5123.png 305\n4e0c819000.png 306\n5c0583ceab.png 308\n1c7d2271ab.png 309\ne566e35a76.png 310\n54e5851527.png 311\n3faaa820d3.png 313\n6b35aaf215.png 314\n394f0f3fe5.png 315\n3d455f1d1c.png 316\n7467513585.png 317\n18b4418f08.png 318\n6f9af303a5.png 319\n71e9f2ef59.png 321\n4623987c43.png 322\n21184c98a6.png 323\n25346c4649.png 324\n5ac8553f89.png 325\n1a6ad5d0f5.png 327\n61f7ecf1bf.png 328\n40dfccc791.png 329\n153407229b.png 330\n2be537b9e3.png 331\n56043a7864.png 332\n531fb2a3ca.png 333\n37e02be5cf.png 334\n77f009c19a.png 335\n5de0c822cf.png 336\n5813db7bf7.png 337\n4ff4ccee85.png 338\n438ce70099.png 339\n3152e5b5ad.png 340\n73fc2e985e.png 341\n6c2ba34b57.png 342\n3d7bf0c732.png 343\n4fe2b3c53c.png 344\n545bdc02ca.png 346\n4e6f914a8d.png 347\n718c18ce99.png 348\n20a6d707a0.png 349\nfbffbcd491.png 350\n4e4c16ef40.png 351\n1015942522.png 352\n11e04d81c5.png 353\n336de40751.png 354\n315c7b3eed.png 355\n119cec3a3d.png 356\n1be1fb7939.png 357\n3e19ce7b87.png 358\n7bdd525b27.png 359\n74e13dac33.png 360\n5d297628db.png 362\n6d498aaaf2.png 363\n37a00adccd.png 364\n61c32b5afd.png 365\n4e6d45eb04.png 366\ncd5afedf4f.png 367\n3670b56b83.png 368\n7c7dde3340.png 370\n412e5c3dcf.png 371\n3dcc7624c2.png 372\n19d67be751.png 373\n38b878380b.png 374\n52861fae05.png 375\n60ee89c612.png 377\n2177654260.png 378\n4a4d202235.png 379\n11b8f564e0.png 380\n772e39cdf2.png 381\n34f9117856.png 382\n2a381e5b6c.png 383\n152f92a6b0.png 384\n5919bec0f8.png 385\n5f994804a7.png 387\n5c9852d327.png 388\n56c9b6c684.png 389\n618271f3cb.png 390\n56faad495b.png 391\n729ab0ab1b.png 392\n18b0d1e4ff.png 393\nb9edb0bb87.png 394\n16a2ab5d25.png 395\n73dd08e58e.png 396\n69c07928bb.png 397\n490dcc5dd6.png 398\n5e732200a9.png 399\n71069928e1.png 400\nedccba9380.png 401\n4e7d3cee1d.png 402\n6c685710b7.png 403\n74bc680700.png 404\n63eb8cad92.png 405\n34f0b42880.png 406\n1b9f888640.png 407\n227187aa68.png 408\nbd4589a12f.png 409\n5953fb43fa.png 410\n3d9a125931.png 411\n47517e0005.png 412\naf4fb44184.png 413\n67e91357d9.png 415\nbdadb47ded.png 416\n3fba654036.png 418\n662cbd36e7.png 419\n6722c940a2.png 420\n72632adbf9.png 422\n5bccec610e.png 423\n24332daee4.png 424\n2a54d2ba3b.png 425\n658e2e67bb.png 426\n4ec63b04f0.png 427\n7655c206d2.png 428\n2c06a9690c.png 430\n63d175a5e0.png 431\n19aa0666cc.png 432\n2e6122dbfc.png 433\n1d95e13fad.png 434\n4c9f09e110.png 435\n74190c896d.png 436\n550610d5d3.png 437\nd5c0896dd6.png 439\n41dbc59f9c.png 440\n7cf89ec296.png 441\n412e20f23f.png 442\n178ed65519.png 443\n40e8f91d3e.png 445\n216b9616b6.png 447\n54a0a20a42.png 448\n56113331d4.png 450\n3982559db9.png 451\n1e1505c92f.png 452\n24d3cd46d2.png 453\n3b7a1a6e70.png 454\n1915502af4.png 456\n6cd24e259e.png 457\n6e33251164.png 458\n795c17720d.png 460\n62a80b376c.png 461\n31546c8b41.png 463\n46bc294f37.png 464\n23d23c19eb.png 466\n7cb32fac57.png 467\n4f1986e78c.png 468\n4c7fe5c962.png 469\n7389651287.png 470\n2d8ebc9df7.png 471\n1955b98ce0.png 472\n56077c7b8d.png 473\n8adfa46d9e.png 474\n14bb617dee.png 475\n76653bc864.png 476\n7620040742.png 477\n20ecceacba.png 478\n190a4d7839.png 479\n2187611cd5.png 480\n2817a56674.png 481\n2dfcef9eb0.png 482\n4e67fd1568.png 484\n47bfa0eec5.png 485\n6fa146daf6.png 486\n7a6498e465.png 487\n4c0346a484.png 488\n667ed1d308.png 489\n18dd5bebfe.png 490\ne7dbdb276c.png 491\n26a03f5f3e.png 492\n1eba968689.png 493\n28bee01ebd.png 494\n1ccca004c3.png 495\n8c0b3f6ff6.png 496\n23ff5d43fa.png 497\n75735bfa6b.png 498\n68844fd309.png 499\n2e35cd83e8.png 500\n7fc9603050.png 501\n7c6531f57f.png 502\n43e44bc1e6.png 503\n134ca7f589.png 504\n124a510572.png 505\n217c11c290.png 506\n6a7d838ceb.png 507\n2af5ce9da2.png 508\n18d380f64a.png 509\n4d1149ae54.png 510\n2812d8bfc3.png 511\n1d680e7a35.png 512\n21af95c2df.png 513\n20d488de0e.png 514\n7d9ab20550.png 515\n2d7d315ab7.png 516\n7c95382414.png 517\nde8534f55a.png 518\n62756b2706.png 520\na6077cbcac.png 521\n17f1c3c228.png 522\n2b8c88ee64.png 523\n3b48db8330.png 524\n931e144505.png 525\n7b549cf22c.png 526\n7222c26c18.png 527\n2eecb90b72.png 528\n688df0df75.png 529\n4a0e64d10d.png 530\n488bd438ba.png 531\n6516a2853c.png 532\nf1959f2ba0.png 533\n3d3af5cf5a.png 534\n684b823b9f.png 535\n6026138ebe.png 536\n58b0af9a15.png 537\n699cbf3379.png 539\n45af0dfcca.png 540\n36a8d9dc06.png 541\n9eac02a9da.png 542\n42b20bc6d2.png 543\n759f4fc7c4.png 545\n7a4a0fbe98.png 546\nf95f5b6ca0.png 547\n77a207b3bd.png 548\n7927cb4cd3.png 549\n50b736b0b3.png 551\n3e30c4b1fe.png 552\n47ab4b9eac.png 554\n7611483793.png 555\n5049616b13.png 556\n2457925124.png 557\n5319a5cd5b.png 558\n47ece0f8c0.png 560\n30a0b199f2.png 562\n59f60efb0b.png 563\nb58eb32942.png 564\n6e3430c217.png 565\n27a2603118.png 566\n28bd09a362.png 567\n783f45a74a.png 568\n40438319b3.png 569\n3bc0cd120b.png 570\n7953333f74.png 571\n12973ae6bf.png 572\n51b9b3d233.png 573\n3d592d51f4.png 574\n2168ed75c2.png 575\n62ebfd5a50.png 576\n5e5e08a0ff.png 577\n20386b7947.png 578\n69f62b882a.png 579\n4c9ed272d9.png 581\n511af7f448.png 582\n4fade9d45b.png 583\n5ced0b6e06.png 584\n2002a31c69.png 585\n65504a130e.png 586\n1de15504a5.png 587\n1e244f5909.png 588\n2feda5cee2.png 589\n34a988525a.png 590\n7cd6b7823a.png 591\n10fd6c9d3b.png 592\n6a1355bfe2.png 593\n6c70a91560.png 594\n77d2fb4eb6.png 595\n4099955659.png 596\n7c77facebb.png 597\n3870f36b0a.png 598\n1321dfa636.png 599\n49ddb5edd3.png 600\n20e5eb8023.png 601\n58522752a3.png 602\n2e80f95614.png 603\n3daffde15e.png 604\na960f0d483.png 605\n78388c122c.png 606\n1e3d03189c.png 607\n7840a3cb63.png 608\n3c69b7b0e2.png 611\n1c30697876.png 612\n3d5173adba.png 613\n66fbeaadc3.png 614\n1532c71612.png 615\n135b7e68ed.png 617\n3d28f96cce.png 618\nacf4358da0.png 619\n74b9a16ada.png 620\n2233f4b015.png 621\n72586ce90c.png 622\n5019c62bf3.png 623\n59d59dd2aa.png 624\n69f7e9ccf2.png 626\n7139e6b6c4.png 627\n33561ebcc0.png 628\n7cf6a5cd8c.png 630\n6fbe5abccb.png 631\n77d8b27e5e.png 632\n55fb69ef75.png 635\n6cfb18298c.png 636\n364367d4ca.png 637\n54ecd49027.png 639\n6bf1f71956.png 640\n1c275ce7be.png 641\n5ea177bd59.png 642\nc5a5e86be4.png 643\n31eb5311ab.png 644\n5554b8bafe.png 645\nae2dfab323.png 646\n57ee5200eb.png 647\nc2815087ca.png 648\nc93811c0a1.png 649\n6b8fd26512.png 650\n3347106f6c.png 651\n380771147a.png 652\n7e29bd9b2b.png 653\n462d961136.png 654\n20ecb2fa17.png 655\n3077d2b5c5.png 656\n26c18afc01.png 657\n3b31ac97eb.png 659\n34ff47642d.png 660\n2bdee6dc8c.png 662\n5aac83a092.png 663\n49619f5e38.png 664\n5ef7f48e2f.png 665\n24d694e788.png 666\n4ebb99e904.png 667\n76d8be8e91.png 668\n6a91cc4050.png 669\n61db95d2d5.png 671\n1b430de29b.png 672\n4c50fe07a5.png 673\n21983b6f45.png 674\n2ee1cccd98.png 675\n68d27e4c6b.png 676\n5c7818e9bb.png 677\n36fabb1629.png 678\n2cb4925807.png 680\n320edfe252.png 681\n18c66a0966.png 682\n432b66ec70.png 684\n34a146d5cb.png 685\n78087b54d2.png 686\n6ca4ebfe75.png 688\n45a0218f89.png 689\n111bd8befd.png 692\n3d0907a003.png 693\n3077534c40.png 694\na7379b2793.png 695\n1c7a5e88b7.png 696\n45deabb336.png 697\n3db78b4714.png 698\n179ee2322b.png 699\n5b6a5cc6a5.png 700\n4b71e59fcd.png 701\n42458ec06a.png 703\n2480e734be.png 704\n688672b558.png 705\n5318f845f6.png 706\nb18a1fb447.png 707\n4b2b463726.png 708\n4ef383bf1c.png 709\n7b2a0c1fb6.png 710\nc58200df88.png 712\n6c24ee49a0.png 713\n497b674d2d.png 714\n1ba3ebbf0c.png 715\n1213b8125f.png 717\n43868b882e.png 718\n2769dd9b8b.png 719\n5619b13ab4.png 720\n6a1b59a078.png 722\n3549abd8d2.png 723\n4fe7a39b99.png 724\n1ccd797c55.png 725\n77d5eedce4.png 726\n78b2f220f2.png 728\n66a04072d7.png 729\n273e688d7e.png 730\n2e86c71271.png 731\n247cffb096.png 732\n7ce85e3026.png 733\n5e9540cb88.png 734\n40caface69.png 735\n4fd43678d5.png 736\n7356a3b589.png 737\n51f7423302.png 738\n615be49b21.png 739\n5c3a48f6b2.png 740\n4b9e2e8659.png 741\n62a8a1936a.png 742\ndfef241e6c.png 743\n1493d744aa.png 744\nc5fb7b4822.png 745\n7cc24f59f3.png 746\n37041d1f7f.png 747\ne3464d818a.png 748\n49b1da67d5.png 749\n1dace34d4a.png 750\n6c1689ec47.png 751\n16e7e8f3f2.png 752\n108453837c.png 753\n54ce26fd6d.png 754\n16cb3262bd.png 755\n7adad81603.png 756\nda56fb536b.png 758\n1da914be75.png 759\n1f2c64922b.png 760\n6597c474ab.png 762\n2a20b5b1cf.png 763\n4736919312.png 764\n2aa5a25f06.png 765\n7e38694661.png 767\n51f0d37078.png 768\n6bfb8ddf8c.png 769\n33249f40f8.png 770\nfb47c8400a.png 771\n38ecfdce7e.png 772\n6c0df8dbd1.png 773\n93b145eb5e.png 774\n374d5dcf51.png 776\n4e40aa7ddc.png 777\n706f13055c.png 778\n355d2bb1b0.png 779\n27b57bcd1f.png 781\n1dcde3138e.png 782\n62ca41ae62.png 784\n3fef6d42a2.png 785\n3b13d3cf38.png 786\n2670b0a52c.png 787\n5a6022ef11.png 788\n76687e1577.png 789\n62263f898d.png 790\n3733924e20.png 791\n5e6f762bde.png 793\n44bbca7814.png 794\n6031ff1d01.png 795\n4bbcdcf152.png 796\n1ce187f40a.png 797\n4ed2183bb6.png 798\n184acbfaf7.png 799\n410a357212.png 800\n377e3a8561.png 801\n40a7157e7d.png 802\n784bed712c.png 803\n2b50c97ed0.png 804\n27f2d26053.png 805\n24c44bb286.png 806\n349d518d66.png 807\n3cfa82266e.png 809\n5eba7d8a44.png 810\n5193b149c1.png 811\n17b91d8c32.png 812\ndeb12268ee.png 813\n74540b5448.png 814\n4972f72854.png 815\n5e0187c848.png 816\n2f18211d63.png 817\n46d0ff38bf.png 818\n2a8ab3e43c.png 819\nca912eb355.png 820\n77eaeae3e4.png 821\nf1a69e859d.png 822\n43ed632fae.png 823\n6bb7790b92.png 824\n58cf29598d.png 825\nefa360c71f.png 827\n7a906e1e29.png 828\n43b53998fc.png 829\n263efbddad.png 830\n6d34f3687e.png 831\n2fc06d5ffa.png 832\n150630c544.png 833\n387f9d88dd.png 834\n7e3fc2cd8a.png 835\n3e4b91167c.png 836\n76d665efaa.png 837\nbc6584aadf.png 838\n566c5e1ba4.png 839\n16e90a4a4c.png 840\n489f58d4e6.png 841\n6922a6c959.png 842\n6d751e80a1.png 843\n7d7f34a745.png 844\n47d61cf5f0.png 846\n41d1835b9d.png 847\n62657b5e68.png 848\n319dd0463f.png 849\n2bceca2af5.png 850\n1fa7f3fa13.png 851\n28a7805724.png 852\n4f62e74af6.png 853\nd41e91c436.png 854\n7f9cb5466e.png 855\n49e0f63e9d.png 856\n18fcea5309.png 857\n4ccb9dd306.png 859\n4ad52b90e0.png 860\n3d7079863e.png 861\n756fcfed82.png 862\n1466d6bf21.png 863\n31da825201.png 864\n164c43af41.png 865\n49e504bde5.png 866\n3fdfd3011e.png 867\n1434096391.png 868\n1671b4d29f.png 869\n6f85e24401.png 870\n71e97fae64.png 871\n1a5e4f1ea5.png 872\n5c5c0536ca.png 873\n7a8a88a457.png 875\n70f7d8cb2f.png 876\n2efe089406.png 877\n536023b45b.png 878\n424dd560d5.png 879\nd72dcd4146.png 880\n163bb3b656.png 881\n4652f0eee3.png 882\n79230f17fe.png 883\n3e4e1d1c9c.png 885\n37122e6049.png 886\n5a32b3ae99.png 887\n4ae0099f82.png 889\n132e4fe98e.png 890\n53848ae56b.png 893\n1a09f0e488.png 894\n22ebca097d.png 895\n5fe9341510.png 896\n8fc62d8cee.png 898\n415254995a.png 900\n2bbcda22c8.png 901\n2a8be07a0d.png 902\n393fe5be69.png 903\n3d674765e3.png 904\n76a11a58c5.png 905\n6c29c29288.png 906\n4a9defb409.png 907\n394b9c6f7a.png 908\n55928482f0.png 909\n66d373e188.png 910\n511e378fdc.png 911\n5a97c95d42.png 912\n6fd193ddff.png 913\n7f66b07a62.png 914\n686db389cb.png 915\n558cb70d57.png 916\n72d002f812.png 917\n23ec2a9866.png 918\n4ebe29dcdc.png 919\necddd11060.png 920\n3185c48802.png 921\n4f6a6bf201.png 922\n3dbf43363a.png 923\n4f8503ee9d.png 924\n538fef06b2.png 925\n7d471c4099.png 926\nf79d80b12c.png 927\n23a0256e05.png 928\n4e01ccc31a.png 929\n2bc4a215e1.png 930\n286b161118.png 931\n267a44a011.png 932\n786c9a4c35.png 933\n64df465ef2.png 934\n4539737602.png 935\n4bdf6fd523.png 936\n8f3748e911.png 937\n795b27411f.png 938\na219448585.png 939\n2336fcc2f1.png 940\n607728aa68.png 941\n30dbabc472.png 942\n285feda2c2.png 943\n329e5a0233.png 944\n795e058d5e.png 945\n6d2a1c8fd2.png 946\n2bcf312d4d.png 948\n2e26b03a14.png 949\n465798ba74.png 950\n10b43f9ed8.png 951\n50f03ab5e7.png 952\n4c3ade589c.png 953\n6535c971df.png 955\n51973bf662.png 957\n31d37292e5.png 958\n21f28c027e.png 959\n471d4175f7.png 960\n42413b33eb.png 961\n34375303b9.png 962\ne8fac80be0.png 963\n3948c68dfe.png 964\naa9c1f0792.png 965\n800beaf2d4.png 966\n6b309bd9e0.png 967\n333eac3e9f.png 968\n2296f849c2.png 969\n30f320a4ae.png 970\nbc13232098.png 971\n40e9bc17aa.png 972\n60577e99a1.png 973\n55141725a8.png 974\n633712725b.png 975\n56d58715d5.png 976\n6d6e4f3920.png 977\n571314b80e.png 978\n14fffb7ef9.png 979\n180d06c7af.png 980\n32a64a6e80.png 981\n3c1d34583b.png 982\n1a2262a3e2.png 983\n200f79bf14.png 984\nef10941593.png 986\n4ba7bc9664.png 987\n5dcd648dea.png 988\n29a11d16e3.png 989\n104c5cec5a.png 990\n66843f7a8a.png 991\n2a2d9f39aa.png 992\n25a3d6b46b.png 993\n60e35372bc.png 994\n20470c3d5f.png 995\n19ac621c4b.png 996\n61767f5294.png 997\n144b2d2e28.png 999\n"
  },
  {
    "path": "data/validate.txt",
    "content": "5abbb9b19f.png 1100\n329a44c373.png 1101\n73b51f198b.png 1102\n6331d9e7fd.png 1104\n91a55d2cb9.png 1106\n408fe63a30.png 1107\n232d6fea7c.png 1108\n702a6a1e50.png 1110\n7a7df95199.png 1111\nd39642fa83.png 1112\n2c65d65f05.png 1113\n3bea2f425a.png 1114\n3e3f5d7c1f.png 1115\n12e9e7dd1f.png 1116\n7c99ded238.png 1119\n652ec3e732.png 1120\n72689463cf.png 1121\n523ad74d44.png 1122\nb7f8cd1f98.png 1123\n3264c226dd.png 1124\nc578283712.png 1125\n672726b185.png 1126\n18457514f2.png 1127\n3603a44b2b.png 1128\n288ea38816.png 1129\n179b37b3d3.png 1130\n64daea677f.png 1131\n7495d151e1.png 1133\naacbc9b7e5.png 1135\n7aed42a53c.png 1136\n2ffc852060.png 1137\n4efc22f0f5.png 1138\n12603c7701.png 1140\n105de61fa3.png 1141\n47696e11d7.png 1142\n53ded8fccd.png 1143\n6b5b65e371.png 1144\n73778ad253.png 1145\n1c6c15eec1.png 1146\n48eb8b68bd.png 1147\n6c4b620ad6.png 1148\n55d0a0fc56.png 1150\n4d1592b40a.png 1151\n2724dd250b.png 1152\n38dbc7d096.png 1153\n15c5d88a1a.png 1154\n4d31b13054.png 1155\n5cb4b5bd0e.png 1156\n7376be8a16.png 1157\n5caa163344.png 1158\n2673247422.png 1159\n6552f4db6d.png 1160\n63a2608987.png 1162\n7a672b3ac0.png 1163\n4c986ebb34.png 1164\n15329206b1.png 1165\n36b2d3fb20.png 1166\n71361ce21f.png 1168\n23d91c1744.png 1169\n31c875f0d6.png 1170\n5a559fe73b.png 1171\n5e1d3cec87.png 1172\n53a1695e86.png 1174\ne6b89266e6.png 1175\n599063d58b.png 1176\n2fdd8e53dc.png 1177\n36f3afe9f3.png 1178\n326031c68d.png 1179\n4cf2740914.png 1180\n486d8b9515.png 1181\nb80ef03527.png 1182\n4d55385c75.png 1183\n438635c3bf.png 1184\n27fa6d2160.png 1185\n7010249b8a.png 1186\n1d320e1c45.png 1187\n5a51edfa94.png 1188\n9cd4f5f61f.png 1189\n746f9895f3.png 1190\n164f87aa84.png 1192\n28e6358ab6.png 1193\n4104802d8a.png 1194\n367fda4d81.png 1195\n5cb46bae1b.png 1197\n4785815348.png 1198\n337e111a56.png 1199\n"
  },
  {
    "path": "data/vocab.txt",
    "content": "!\n&\n'\n(\n)\n*\n+\n,\n-\n.\n/\n0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n:\n;\n<\n=\n>\nA\nB\nC\nD\nE\nF\nG\nH\nI\nJ\nK\nL\nM\nN\nO\nP\nQ\nR\nS\nT\nU\nV\nW\nX\nY\nZ\n[\n\\\n\\!\n\\,\n\\:\n\\;\n\\Big\n\\Bigg\n\\Bigl\n\\Bigr\n\\Delta\n\\Gamma\n\\Lambda\n\\Omega\n\\Phi\n\\Pi\n\\Psi\n\\Rightarrow\n\\Sigma\n\\Theta\n\\Xi\n\\\\\n\\alpha\n\\approx\n\\ast\n\\bar\n\\begin{array}\n\\beta\n\\bf\n\\big\n\\bigg\n\\biggl\n\\biggr\n\\bigl\n\\bigr\n\\binom\n\\boldmath\n\\cal\n\\cdot\n\\cdots\n\\chi\n\\circ\n\\cong\n\\dag\n\\dagger\n\\ddot\n\\delta\n\\displaystyle\n\\dot\n\\dots\n\\ell\n\\end{array}\n\\epsilon\n\\equiv\n\\eta\n\\forall\n\\frac\n\\gamma\n\\geq\n\\hat\n\\hbar\n\\hspace\n\\imath\n\\in\n\\infty\n\\int\n\\jmath\n\\kappa\n\\lambda\n\\langle\n\\lbrack\n\\ldots\n\\le\n\\left(\n\\left.\n\\left[\n\\left\\langle\n\\left\\{\n\\leftrightarrow\n\\left|\n\\leq\n\\longrightarrow\n\\mapsto\n\\mathbf\n\\mathcal\n\\mathrm\n\\mid\n\\mp\n\\mu\n\\nabla\n\\neq\n\\nonumber\n\\nu\n\\oint\n\\omega\n\\operatorname\n\\operatorname*\n\\oplus\n\\otimes\n\\overleftarrow\n\\overline\n\\overrightarrow\n\\partial\n\\perp\n\\phi\n\\pi\n\\pm\n\\prime\n\\prod\n\\propto\n\\psi\n\\qquad\n\\quad\n\\rangle\n\\rbrack\n\\rho\n\\right)\n\\right.\n\\right\\rangle\n\\right\\}\n\\right]\n\\rightarrow\n\\rightleftharpoons\n\\right|\n\\scriptsize\n\\scriptstyle\n\\sigma\n\\sim\n\\simeq\n\\slash\n\\small\n\\smallskip\n\\sp\n\\sqrt\n\\stackrel\n\\star\n\\sum\n\\tau\n\\textrm\n\\textstyle\n\\theta\n\\thinspace\n\\tilde\n\\times\n\\tiny\n\\to\n\\triangle\n\\underline\n\\varepsilon\n\\varphi\n\\varpi\n\\varrho\n\\vartheta\n\\vec\n\\vert\n\\vspace\n\\wedge\n\\widehat\n\\widetilde\n\\xi\n\\zeta\n\\{\n\\}\n]\n^\n_\na\nb\nc\nd\ne\nf\ng\nh\ni\nj\nk\nl\nm\nn\no\np\nq\nr\ns\nt\nu\nv\nw\nx\ny\nz\n{\n|\n}\n~"
  },
  {
    "path": "opennmt/Constants.lua",
    "content": "return {\n  PAD = 1,\n  UNK = 2,\n  BOS = 3,\n  EOS = 4,\n\n  PAD_WORD = '<blank>',\n  UNK_WORD = '<unk>',\n  BOS_WORD = '<s>',\n  EOS_WORD = '</s>'\n}\n"
  },
  {
    "path": "opennmt/Factory.lua",
    "content": "local Factory = torch.class('Factory')\n\nlocal options = {\n  {\n    '-brnn', false,\n    [[Use a bidirectional encoder.]],\n    {\n      structural = 0\n    }\n  },\n  {\n    '-dbrnn', false,\n    [[Use a deep bidirectional encoder.]],\n    {\n      structural = 0\n    }\n  },\n  {\n    '-pdbrnn', false,\n    [[Use a pyramidal deep bidirectional encoder.]],\n    {\n      structural = 0\n    }\n  },\n  {\n    '-attention', 'global',\n    [[Attention model.]],\n    {\n      enum = {'none', 'global'}\n    }\n  }\n}\n\nfunction Factory.declareOpts(cmd)\n  cmd:setCmdLineOptions(options)\n  onmt.BiEncoder.declareOpts(cmd)\n  onmt.DBiEncoder.declareOpts(cmd)\n  onmt.PDBiEncoder.declareOpts(cmd)\n  onmt.GlobalAttention.declareOpts(cmd)\nend\n\n-- Return effective embeddings size based on user options.\nlocal function resolveEmbSizes(opt, dicts, wordSizes)\n  local wordEmbSize\n  local featEmbSizes = {}\n\n  wordSizes = onmt.utils.String.split(tostring(wordSizes), ',')\n\n  if type(opt.word_vec_size) == 'number' and opt.word_vec_size > 0 then\n    wordEmbSize = opt.word_vec_size\n  else\n    wordEmbSize = tonumber(wordSizes[1])\n  end\n\n  for i = 1, #dicts.features do\n    local size\n\n    if i + 1 <= #wordSizes then\n      size = tonumber(wordSizes[i + 1])\n    elseif opt.feat_merge == 'sum' then\n      size = opt.feat_vec_size\n    else\n      size = math.floor(dicts.features[i]:size() ^ opt.feat_vec_exponent)\n    end\n\n    table.insert(featEmbSizes, size)\n  end\n\n  return wordEmbSize, featEmbSizes\nend\n\nlocal function buildInputNetwork(opt, dicts, wordSizes, pretrainedWords, fixWords, verbose)\n  local wordEmbSize, featEmbSizes = resolveEmbSizes(opt, dicts, wordSizes)\n\n  local wordEmbedding = onmt.WordEmbedding.new(dicts.words:size(), -- vocab size\n                                               wordEmbSize,\n                                               pretrainedWords,\n                                               fixWords)\n\n  local inputs\n  local inputSize = wordEmbSize\n\n  local multiInputs = #dicts.features > 0\n\n  if multiInputs then\n    inputs = nn.ParallelTable()\n      :add(wordEmbedding)\n  else\n    inputs = wordEmbedding\n  end\n\n  if verbose then\n    _G.logger:info('   - word embeddings size: ' .. wordEmbSize)\n  end\n\n  -- Sequence with features.\n  if #dicts.features > 0 then\n    if verbose then\n      _G.logger:info('   - features embeddings sizes: ' .. table.concat(featEmbSizes, ', '))\n    end\n\n    local vocabSizes = {}\n    for i = 1, #dicts.features do\n      table.insert(vocabSizes, dicts.features[i]:size())\n    end\n\n    local featEmbedding = onmt.FeaturesEmbedding.new(vocabSizes, featEmbSizes, opt.feat_merge)\n    inputs:add(featEmbedding)\n    inputSize = inputSize + featEmbedding.outputSize\n  end\n\n  local inputNetwork\n\n  if multiInputs then\n    inputNetwork = nn.Sequential()\n      :add(inputs)\n      :add(nn.JoinTable(2, 2))\n  else\n    inputNetwork = inputs\n  end\n\n  inputNetwork.inputSize = inputSize\n\n  return inputNetwork\nend\n\nfunction Factory.getOutputSizes(dicts)\n  local outputSizes = { dicts.words:size() }\n  for i = 1, #dicts.features do\n    table.insert(outputSizes, dicts.features[i]:size())\n  end\n  return outputSizes\nend\n\nfunction Factory.buildEncoder(opt, inputNetwork, verbose)\n\n  local function describeEncoder(name)\n    if verbose then\n      _G.logger:info('   - type: %s', name)\n      _G.logger:info('   - structure: cell = %s; layers = %d; rnn_size = %d; dropout = ' .. opt.dropout,\n                     opt.rnn_type, opt.layers, opt.rnn_size)\n    end\n  end\n\n  if opt.brnn then\n    describeEncoder('bidirectional')\n    return onmt.BiEncoder.new(opt, inputNetwork)\n  elseif opt.dbrnn then\n    describeEncoder('deep bidirectional')\n    return onmt.DBiEncoder.new(opt, inputNetwork)\n  elseif opt.pdbrnn then\n    describeEncoder('pyramidal deep bidirectional')\n    return onmt.PDBiEncoder.new(opt, inputNetwork)\n  else\n    describeEncoder('simple')\n    return onmt.Encoder.new(opt, inputNetwork)\n  end\n\nend\n\nfunction Factory.buildWordEncoder(opt, dicts, verbose)\n  if verbose then\n    _G.logger:info(' * Encoder:')\n  end\n\n  local inputNetwork = buildInputNetwork(opt, dicts,\n                                         opt.src_word_vec_size or opt.word_vec_size,\n                                         opt.pre_word_vecs_enc, opt.fix_word_vecs_enc == 1,\n                                         verbose)\n\n  return Factory.buildEncoder(opt, inputNetwork, verbose)\nend\n\nfunction Factory.loadEncoder(pretrained, clone)\n  if clone then\n    pretrained = onmt.utils.Tensor.deepClone(pretrained)\n  end\n\n  local encoder\n\n  if pretrained.name == 'Encoder' then\n    encoder = onmt.Encoder.load(pretrained)\n  elseif pretrained.name == 'BiEncoder' then\n    encoder = onmt.BiEncoder.load(pretrained)\n  elseif pretrained.name == 'PDBiEncoder' then\n    encoder = onmt.PDBiEncoder.load(pretrained)\n  elseif pretrained.name == 'DBiEncoder' then\n    encoder = onmt.DBiEncoder.load(pretrained)\n  else\n    -- Keep for backward compatibility.\n    local brnn = #pretrained.modules == 2\n    if brnn then\n      encoder = onmt.BiEncoder.load(pretrained)\n    else\n      encoder = onmt.Encoder.load(pretrained)\n    end\n  end\n\n  return encoder\nend\n\nfunction Factory.buildDecoder(opt, inputNetwork, generator, attnModel, verbose)\n  if verbose then\n    _G.logger:info('   - structure: cell = %s; layers = %d; rnn_size = %d; dropout = ' .. opt.dropout,\n                   opt.rnn_type, opt.layers, opt.rnn_size)\n  end\n\n  return onmt.Decoder.new(opt, inputNetwork, generator, attnModel)\nend\n\nfunction Factory.buildWordDecoder(opt, dicts, verbose)\n  if verbose then\n    _G.logger:info(' * Decoder:')\n  end\n\n  local inputNetwork = buildInputNetwork(opt, dicts,\n                                         opt.tgt_word_vec_size or opt.word_vec_size,\n                                         opt.pre_word_vecs_dec, opt.fix_word_vecs_dec == 1,\n                                         verbose)\n\n  local generator = Factory.buildGenerator(opt.rnn_size, dicts)\n  local attnModel = Factory.buildAttention(opt)\n\n  return Factory.buildDecoder(opt, inputNetwork, generator, attnModel, verbose)\nend\n\nfunction Factory.loadDecoder(pretrained, clone)\n  if clone then\n    pretrained = onmt.utils.Tensor.deepClone(pretrained)\n  end\n\n  local decoder = onmt.Decoder.load(pretrained)\n\n  return decoder\nend\n\nfunction Factory.buildGenerator(rnnSize, dicts)\n  if #dicts.features > 0 then\n    return onmt.FeaturesGenerator(rnnSize, Factory.getOutputSizes(dicts))\n  else\n    return onmt.Generator(rnnSize, dicts.words:size())\n  end\nend\n\nfunction Factory.buildAttention(args)\n  if args.attention == 'none' then\n    _G.logger:info('   - attention: none')\n    return onmt.NoAttention(args, args.rnn_size)\n  else\n    _G.logger:info('   - attention: global (%s)', args.global_attention)\n    return onmt.GlobalAttention(args, args.rnn_size)\n  end\nend\n\nfunction Factory.loadGenerator(pretrained, clone)\n  if clone then\n    pretrained = onmt.utils.Tensor.deepClone(pretrained)\n  end\n\n  return pretrained\nend\n\nreturn Factory\n"
  },
  {
    "path": "opennmt/LanguageModel.lua",
    "content": "--[[ Language Model. ]]\nlocal LanguageModel, parent = torch.class('LanguageModel', 'Model')\n\nlocal options = {\n  {\n    '-word_vec_size', '500',\n    [[Comma-separated list of embedding sizes: `word[,feat1[,feat2[,...] ] ]`.]],\n    {\n      structural = 0\n    }\n  },\n  {\n    '-pre_word_vecs_enc', '',\n    [[Path to pretrained word embeddings on the encoder side serialized as a Torch tensor.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists,\n      init_only = true\n    }\n  },\n  {\n    '-fix_word_vecs_enc', 0,\n    [[Fix word embeddings on the encoder side.]],\n    {\n      enum = {0, 1},\n      structural = 1\n    }\n  },\n  {\n    '-feat_merge', 'concat',\n    [[Merge action for the features embeddings.]],\n    {\n      enum = {'concat', 'sum'},\n      structural = 0\n    }\n  },\n  {\n    '-feat_vec_exponent', 0.7,\n    [[When features embedding sizes are not set and using `-feat_merge concat`, their dimension\n      will be set to `N^feat_vec_exponent` where `N` is the number of values the feature takes.]],\n    {\n      structural = 0\n    }\n  },\n  {\n    '-feat_vec_size', 20,\n    [[When features embedding sizes are not set and using `-feat_merge sum`,\n      this is the common embedding size of the features]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt(),\n      structural = 0\n    }\n  }\n}\n\nfunction LanguageModel.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, 'Language Model')\n  onmt.Encoder.declareOpts(cmd)\n  onmt.Factory.declareOpts(cmd)\nend\n\nfunction LanguageModel:__init(args, dicts, verbose)\n  parent.__init(self, args)\n  onmt.utils.Table.merge(self.args, onmt.utils.ExtendedCmdLine.getModuleOpts(args, options))\n\n  self.models.encoder = onmt.Factory.buildWordEncoder(args, dicts.src, verbose)\n  self.models.generator = onmt.Factory.buildGenerator(args.rnn_size, dicts.src)\n\n  self.criterion = onmt.ParallelClassNLLCriterion(onmt.Factory.getOutputSizes(dicts.src))\n\n  self.eosProto = {}\n  for _ = 1, #dicts.src.features + 1 do\n    table.insert(self.eosProto, torch.LongTensor())\n  end\nend\n\nfunction LanguageModel.load(args, models, dicts, isReplica)\n  local self = torch.factory('LanguageModel')()\n\n  parent.__init(self, args)\n  onmt.utils.Table.merge(self.args, onmt.utils.ExtendedCmdLine.getModuleOpts(args, options))\n\n  self.models.encoder = onmt.Factory.loadEncoder(models.encoder, isReplica)\n  self.models.generator = onmt.Factory.loadGenerator(models.generator, isReplica)\n  self.criterion = onmt.ParallelClassNLLCriterion(onmt.Factory.getOutputSizes(dicts.src))\n\n  return self\nend\n\n-- Returns model name.\nfunction LanguageModel.modelName()\n  return 'Language'\nend\n\n-- Returns expected dataMode.\nfunction LanguageModel.dataType()\n  return 'monotext'\nend\n\nfunction LanguageModel:enableProfiling()\n  _G.profiler.addHook(self.models.encoder, 'encoder')\n  _G.profiler.addHook(self.models.generator, 'generator')\n  _G.profiler.addHook(self.criterion, 'criterion')\nend\n\nfunction LanguageModel:getOutput(batch)\n  return batch.sourceInput\nend\n\nfunction LanguageModel:forwardComputeLoss(batch)\n  local _, context = self.models.encoder:forward(batch)\n  local eos = onmt.utils.Tensor.reuseTensorTable(self.eosProto, { batch.size })\n  for i = 1, #eos do\n    eos[i]:fill(onmt.Constants.EOS)\n  end\n\n  local loss = 0\n\n  for t = 1, batch.sourceLength do\n    local genOutputs = self.models.generator:forward(context:select(2, t))\n\n    -- LanguageModel is supposed to predict the following word.\n    local output\n    if t ~= batch.sourceLength then\n      output = batch:getSourceInput(t + 1)\n    else\n      output = eos\n    end\n\n    -- Same format with and without features.\n    if torch.type(output) ~= 'table' then output = { output } end\n\n    loss = loss + self.criterion:forward(genOutputs, output)\n  end\n\n  return loss\nend\n\nfunction LanguageModel:trainNetwork(batch)\n  local loss = 0\n\n  local _, context = self.models.encoder:forward(batch)\n\n  local gradContexts = context:clone():zero()\n  local eos = onmt.utils.Tensor.reuseTensorTable(self.eosProto, { batch.size })\n  for i = 1, #eos do\n    eos[i]:fill(onmt.Constants.EOS)\n  end\n\n  -- For each word of the sentence, generate target.\n  for t = 1, batch.sourceLength do\n    local genOutputs = self.models.generator:forward(context:select(2, t))\n\n    -- LanguageModel is supposed to predict following word.\n    local output\n    if t ~= batch.sourceLength then\n      output = batch:getSourceInput(t + 1)\n    else\n      output = eos\n    end\n\n    -- Same format with and without features.\n    if torch.type(output) ~= 'table' then output = { output } end\n\n    loss = loss + self.criterion:forward(genOutputs, output)\n\n    local genGradOutput = self.criterion:backward(genOutputs, output)\n    for j = 1, #genGradOutput do\n      genGradOutput[j]:div(batch.totalSize)\n    end\n\n    gradContexts[{{}, t}]:copy(self.models.generator:backward(context:select(2, t), genGradOutput))\n  end\n\n  self.models.encoder:backward(batch, nil, gradContexts)\n\n  return loss\nend\n\nreturn LanguageModel\n"
  },
  {
    "path": "opennmt/Model.lua",
    "content": "--[[ Generic Model class. ]]\nlocal Model = torch.class('Model')\n\nlocal options = {\n  {\n    '-model_type', 'seq2seq',\n    [[Type of model to train. This option impacts all options choices.]],\n    {\n      enum = {'lm', 'seq2seq', 'seqtagger'},\n      structural = 0\n    }\n  },\n  {\n    '-param_init', 0.1,\n    [[Parameters are initialized over uniform distribution with support (-`param_init`, `param_init`).]],\n    {\n      valid = function(v) return v >= 0 and v <= 1 end,\n      init_only = true\n    }\n  }\n}\n\nfunction Model.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, 'Model')\nend\n\nfunction Model:__init(args)\n  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)\n  self.args.train_from = args.train_from\n  self.models = {}\nend\n\n-- Dynamically change parameters in the graph.\nfunction Model:changeParameters(changes)\n  _G.logger:info('Applying new parameters:')\n\n  for k, v in pairs(changes) do\n    _G.logger:info(' * %s = ' .. v, k)\n\n    for _, model in pairs(self.models) do\n      model:apply(function(m)\n        if k == 'dropout' and torch.typename(m) == 'nn.Dropout' then\n          m:setp(v)\n        elseif k:find('fix_word_vecs') and torch.typename(m) == 'onmt.WordEmbedding' then\n          local enc = k == 'fix_word_vecs_enc' and torch.typename(model):find('Encoder')\n          local dec = k == 'fix_word_vecs_dec' and torch.typename(model):find('Decoder')\n          if enc or dec then\n            m:fixEmbeddings(v == 1)\n          end\n        end\n      end)\n    end\n  end\n\nend\n\nfunction Model:getInputLabelsCount(batch)\n  return batch.sourceInput:ne(onmt.Constants.PAD):sum()\nend\n\nfunction Model:getOutputLabelsCount(batch)\n  return self:getOutput(batch):ne(onmt.Constants.PAD):sum()\nend\n\nfunction Model:evaluate()\n  for _, m in pairs(self.models) do\n    m:evaluate()\n  end\nend\n\nfunction Model:training()\n  for _, m in pairs(self.models) do\n    m:training()\n  end\nend\n\nfunction Model:initParams(verbose)\n  local numParams = 0\n  local params = {}\n  local gradParams = {}\n\n  if verbose then\n    _G.logger:info('Initializing parameters...')\n  end\n\n  -- Order the model table because we need all replicas to have the same order.\n  local orderedIndex = {}\n  for key in pairs(self.models) do\n    table.insert(orderedIndex, key)\n  end\n  table.sort(orderedIndex)\n\n  for _, key in ipairs(orderedIndex) do\n    local mod = self.models[key]\n    local p, gp = mod:getParameters()\n\n    if self.args.train_from:len() == 0 then\n      p:uniform(-self.args.param_init, self.args.param_init)\n\n      mod:apply(function (m)\n        if m.postParametersInitialization then\n          m:postParametersInitialization()\n        end\n      end)\n    end\n\n    numParams = numParams + p:size(1)\n    table.insert(params, p)\n    table.insert(gradParams, gp)\n  end\n\n  if verbose then\n    _G.logger:info(' * number of parameters: ' .. numParams)\n  end\n\n  return params, gradParams\nend\n\nreturn Model\n"
  },
  {
    "path": "opennmt/ModelSelector.lua",
    "content": "return function(modelType)\n  if modelType == 'seq2seq' then\n    return onmt.Seq2Seq\n  elseif modelType == 'lm' then\n    return onmt.LanguageModel\n  elseif modelType == 'seqtagger' then\n    return onmt.SeqTagger\n  else\n    error('invalid model type ' .. modelType)\n  end\nend\n"
  },
  {
    "path": "opennmt/Seq2Seq.lua",
    "content": "--[[ Sequence to sequence model with attention. ]]\nlocal Seq2Seq, parent = torch.class('Seq2Seq', 'Model')\n\nlocal options = {\n  {\n    '-word_vec_size', 0,\n    [[Shared word embedding size. If set, this overrides `-src_word_vec_size` and `-tgt_word_vec_size`.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt(),\n      structural = 0\n    }\n  },\n  {\n    '-src_word_vec_size', '500',\n    [[Comma-separated list of source embedding sizes: `word[,feat1[,feat2[,...] ] ]`.]],\n    {\n      structural = 0\n    }\n  },\n  {\n    '-tgt_word_vec_size', '500',\n    [[Comma-separated list of target embedding sizes: `word[,feat1[,feat2[,...] ] ]`.]],\n    {\n      structural = 0\n    }\n  },\n  {\n    '-pre_word_vecs_enc', '',\n    [[Path to pretrained word embeddings on the encoder side serialized as a Torch tensor.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists,\n      init_only = true\n    }\n  },\n  {\n    '-pre_word_vecs_dec', '',\n    [[Path to pretrained word embeddings on the decoder side serialized as a Torch tensor.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists,\n      init_only = true\n    }\n  },\n  {\n    '-fix_word_vecs_enc', 0,\n    [[Fix word embeddings on the encoder side.]],\n    {\n      enum = {0, 1},\n      structural = 1\n    }\n  },\n  {\n    '-fix_word_vecs_dec', 0,\n    [[Fix word embeddings on the decoder side.]],\n    {\n      enum = {0, 1},\n      structural = 1\n    }\n  },\n  {\n    '-feat_merge', 'concat',\n    [[Merge action for the features embeddings.]],\n    {\n      enum = {'concat', 'sum'},\n      structural = 0\n    }\n  },\n  {\n    '-feat_vec_exponent', 0.7,\n    [[When features embedding sizes are not set and using `-feat_merge concat`, their dimension\n      will be set to `N^feat_vec_exponent` where `N` is the number of values the feature takes.]],\n    {\n      structural = 0\n    }\n  },\n  {\n    '-feat_vec_size', 20,\n    [[When features embedding sizes are not set and using `-feat_merge sum`,\n      this is the common embedding size of the features]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt(),\n      structural = 0\n    }\n  }\n}\n\nfunction Seq2Seq.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, Seq2Seq.modelName())\n  onmt.Encoder.declareOpts(cmd)\n  onmt.Decoder.declareOpts(cmd)\n  onmt.Factory.declareOpts(cmd)\nend\n\nfunction Seq2Seq:__init(args, dicts, verbose)\n  parent.__init(self, args)\n  onmt.utils.Table.merge(self.args, onmt.utils.ExtendedCmdLine.getModuleOpts(args, options))\n  self.args.uneven_batches = args.uneven_batches\n\n  self.models.encoder = onmt.Factory.buildWordEncoder(args, dicts.src, verbose)\n  self.models.decoder = onmt.Factory.buildWordDecoder(args, dicts.tgt, verbose)\n  self.criterion = onmt.ParallelClassNLLCriterion(onmt.Factory.getOutputSizes(dicts.tgt))\nend\n\nfunction Seq2Seq.load(args, models, dicts, isReplica)\n  local self = torch.factory('Seq2Seq')()\n\n  parent.__init(self, args)\n  onmt.utils.Table.merge(self.args, onmt.utils.ExtendedCmdLine.getModuleOpts(args, options))\n  self.args.uneven_batches = args.uneven_batches\n\n  self.models.encoder = onmt.Factory.loadEncoder(models.encoder, isReplica)\n  self.models.decoder = onmt.Factory.loadDecoder(models.decoder, isReplica)\n  self.criterion = onmt.ParallelClassNLLCriterion(onmt.Factory.getOutputSizes(dicts.tgt))\n\n  return self\nend\n\n-- Returns model name.\nfunction Seq2Seq.modelName()\n  return 'Sequence to Sequence with Attention'\nend\n\n-- Returns expected dataMode.\nfunction Seq2Seq.dataType()\n  return 'bitext'\nend\n\nfunction Seq2Seq:returnIndividualLosses(enable)\n  if not self.models.decoder.returnIndividualLosses then\n    _G.logger:info('Current Seq2Seq model does not support training with sample_w_ppl option')\n    return false\n  else\n    self.models.decoder:returnIndividualLosses(enable)\n  end\n  return true\nend\n\nfunction Seq2Seq:enableProfiling()\n  _G.profiler.addHook(self.models.encoder, 'encoder')\n  _G.profiler.addHook(self.models.decoder, 'decoder')\n  _G.profiler.addHook(self.models.decoder.modules[2], 'generator')\n  _G.profiler.addHook(self.criterion, 'criterion')\nend\n\nfunction Seq2Seq:getOutput(batch)\n  return batch.targetOutput\nend\n\nfunction Seq2Seq:maskPadding(batch)\n  if self.args.uneven_batches then\n    self.models.encoder:maskPadding()\n    if batch.uneven then\n      self.models.decoder:maskPadding(batch.sourceSize, batch.sourceLength)\n    else\n      self.models.decoder:maskPadding()\n    end\n  end\nend\n\nfunction Seq2Seq:forwardComputeLoss(batch)\n  self:maskPadding(batch)\n  local encoderStates, context = self.models.encoder:forward(batch)\n  return self.models.decoder:computeLoss(batch, encoderStates, context, self.criterion)\nend\n\nfunction Seq2Seq:trainNetwork(batch, dryRun)\n  self:maskPadding(batch)\n\n  local encStates, context = self.models.encoder:forward(batch)\n\n  local decOutputs = self.models.decoder:forward(batch, encStates, context)\n\n  if dryRun then\n    decOutputs = onmt.utils.Tensor.recursiveClone(decOutputs)\n  end\n\n  local encGradStatesOut, gradContext, loss, indvLoss = self.models.decoder:backward(batch, decOutputs, self.criterion)\n  self.models.encoder:backward(batch, encGradStatesOut, gradContext)\n\n  return loss, indvLoss\nend\n\nreturn Seq2Seq\n"
  },
  {
    "path": "opennmt/SeqTagger.lua",
    "content": "--[[ Sequence to sequence model with attention. ]]\nlocal SeqTagger, parent = torch.class('SeqTagger', 'Model')\n\nlocal options = {\n  {\n    '-word_vec_size', '500',\n    [[Comma-separated list of embedding sizes: `word[,feat1[,feat2[,...] ] ]`.]],\n    {\n      structural = 0\n    }\n  },\n  {\n    '-pre_word_vecs_enc', '',\n    [[Path to pretrained word embeddings on the encoder side serialized as a Torch tensor.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists,\n      init_only = true\n    }\n  },\n  {\n    '-fix_word_vecs_enc', 0,\n    [[Fix word embeddings on the encoder side.]],\n    {\n      enum = {0, 1},\n      structural = 1\n    }\n  },\n  {\n    '-feat_merge', 'concat',\n    [[Merge action for the features embeddings.]],\n    {\n      enum = {'concat', 'sum'},\n      structural = 0\n    }\n  },\n  {\n    '-feat_vec_exponent', 0.7,\n    [[When features embedding sizes are not set and using `-feat_merge concat`, their dimension\n      will be set to `N^feat_vec_exponent` where `N` is the number of values the feature takes.]],\n    {\n      structural = 0\n    }\n  },\n  {\n    '-feat_vec_size', 20,\n    [[When features embedding sizes are not set and using `-feat_merge sum`,\n      this is the common embedding size of the features]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt(),\n      structural = 0\n    }\n  }\n}\n\n\nfunction SeqTagger.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, SeqTagger.modelName())\n  onmt.Encoder.declareOpts(cmd)\n  onmt.Factory.declareOpts(cmd)\nend\n\nfunction SeqTagger:__init(args, dicts, verbose)\n  parent.__init(self, args)\n  onmt.utils.Table.merge(self.args, onmt.utils.ExtendedCmdLine.getModuleOpts(args, options))\n\n  self.models.encoder = onmt.Factory.buildWordEncoder(self.args, dicts.src, verbose)\n  self.models.generator = onmt.Factory.buildGenerator(self.args.rnn_size, dicts.tgt)\n  self.criterion = onmt.ParallelClassNLLCriterion(onmt.Factory.getOutputSizes(dicts.tgt))\nend\n\nfunction SeqTagger.load(args, models, dicts, isReplica)\n  local self = torch.factory('SeqTagger')()\n\n  parent.__init(self, args)\n  onmt.utils.Table.merge(self.args, onmt.utils.ExtendedCmdLine.getModuleOpts(args, options))\n\n  self.models.encoder = onmt.Factory.loadEncoder(models.encoder, isReplica)\n  self.models.generator = onmt.Factory.loadGenerator(models.generator, isReplica)\n  self.criterion = onmt.ParallelClassNLLCriterion(onmt.Factory.getOutputSizes(dicts.tgt))\n\n  return self\nend\n\n-- Returns model name.\nfunction SeqTagger.modelName()\n  return 'Sequence Tagger'\nend\n\n-- Returns expected dataMode\nfunction SeqTagger.dataType()\n  return 'bitext'\nend\n\nfunction SeqTagger:enableProfiling()\n  _G.profiler.addHook(self.models.encoder, 'encoder')\n  _G.profiler.addHook(self.models.generator, 'generator')\n  _G.profiler.addHook(self.criterion, 'criterion')\nend\n\nfunction SeqTagger:getOutput(batch)\n  return batch.targetOutput\nend\n\nfunction SeqTagger:forwardComputeLoss(batch)\n  local _, context = self.models.encoder:forward(batch)\n\n  local loss = 0\n\n  for t = 1, batch.sourceLength do\n    local genOutputs = self.models.generator:forward(context:select(2, t))\n\n    local output = batch:getTargetOutput(t)\n\n    -- Same format with and without features.\n    if torch.type(output) ~= 'table' then output = { output } end\n\n    loss = loss + self.criterion:forward(genOutputs, output)\n  end\n\n  return loss\nend\n\nfunction SeqTagger:trainNetwork(batch)\n  local loss = 0\n\n  local _, context = self.models.encoder:forward(batch)\n\n  local gradContexts = context:clone():zero()\n\n  -- For each word of the sentence, generate target.\n  for t = 1, batch.sourceLength do\n    local genOutputs = self.models.generator:forward(context:select(2, t))\n\n    local output = batch:getTargetOutput(t)\n\n    -- Same format with and without features.\n    if torch.type(output) ~= 'table' then output = { output } end\n\n    loss = loss + self.criterion:forward(genOutputs, output)\n\n    local genGradOutput = self.criterion:backward(genOutputs, output)\n    for j = 1, #genGradOutput do\n      genGradOutput[j]:div(batch.totalSize)\n    end\n\n    gradContexts[{{}, t}]:copy(self.models.generator:backward(context:select(2, t), genGradOutput))\n  end\n\n  self.models.encoder:backward(batch, nil, gradContexts)\n\n  return loss\nend\n\nreturn SeqTagger\n"
  },
  {
    "path": "opennmt/data/AliasMultinomial.lua",
    "content": "\n--[[\n Copied with small adjustments from:\n    https://github.com/nicholas-leonard/torchx/blob/master/AliasMultinomial.lua (7eeb6ae)\n]]\n\n--[[\nCopyright (c) 2014, Nikopia (Nicholas Léonard)\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions are met:\n\n* Redistributions of source code must retain the above copyright notices, this\n  list of conditions and the following disclaimer.\n\n* Redistributions in binary form must reproduce the above copyright notice,\n  this list of conditions and the following disclaimer in the documentation\n  and/or other materials provided with the distribution.\n\n* Neither the name of the Nikopia nor the names of its\n  contributors may be used to endorse or promote products derived from\n  this software without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\nAND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\nIMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\nDISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE\nFOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\nDAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\nSERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\nCAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\nOR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\nOF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n]]\n\n-- ref.: https://hips.seas.harvard.edu/blog/2013/03/03/the-alias-method-efficient-sampling-with-many-discrete-outcomes/\nlocal AM = torch.class(\"AliasMultinomial\")\n\nfunction AM:__init(probs)\n   self.J, self.q = self:setup(probs)\nend\n\nfunction AM:setup(probs)\n   assert(probs:dim() == 1)\n   local K = probs:nElement()\n   local q = probs.new(K):zero()\n   local J = torch.LongTensor(K):zero()\n\n   -- Sort the data into the outcomes with probabilities\n   -- that are larger and smaller than 1/K.\n   local smaller, larger = {}, {}\n--   local maxk, maxp = 0, -1\n   for kk = 1,K do\n      local prob = probs[kk]\n      q[kk] = K*prob\n      if q[kk] < 1 then\n         table.insert(smaller, kk)\n      else\n         table.insert(larger, kk)\n      end\n--      if maxk > maxp then\n--\n--      end\n   end\n\n   -- Loop through and create little binary mixtures that\n   -- appropriately allocate the larger outcomes over the\n   -- overall uniform mixture.\n   while #smaller > 0 and #larger > 0 do\n      local small = table.remove(smaller)\n      local large = table.remove(larger)\n\n      J[small] = large\n      q[large] = q[large] - (1.0 - q[small])\n\n      if q[large] < 1.0 then\n         table.insert(smaller,large)\n      else\n         table.insert(larger,large)\n      end\n   end\n   assert(q:min() >= 0)\n   if q:max() > 1 then\n      q:div(q:max())\n   end\n   assert(q:max() <= 1)\n   if J:min() <= 0 then\n      -- sometimes an large index isn't added to J.\n      -- fix it by making the probability 1 so that J isn't indexed.\n      local i = 0\n      J:apply(function(x)\n         i = i + 1\n         if x <= 0 then\n            q[i] = 1\n         end\n      end)\n   end\n   return J, q\nend\n\nfunction AM:draw()\n   local J = self.J\n   local q = self.q\n   local K  = J:nElement()\n\n   -- Draw from the overall uniform mixture.\n   local kk = math.random(1,K)\n\n   -- Draw from the binary mixture, either keeping the\n   -- small one, or choosing the associated larger one.\n   if math.random() < q[kk] then\n      return kk\n   else\n      return J[kk]\n   end\nend\n\nfunction AM:batchdraw(output)\n   assert(torch.type(output) == 'torch.LongTensor')\n   assert(output:nElement() > 0)\n   local J = self.J\n   local K  = J:nElement()\n\n   self._kk = self._kk or output.new()\n   self._kk:resizeAs(output):random(1,K)\n\n   self._q = self._q or self.q.new()\n   self._q:index(self.q, 1, self._kk:view(-1))\n\n   self._mask = self._b or torch.LongTensor()\n   self._mask:resize(self._q:size()):bernoulli(self._q)\n\n   self.__kk = self.__kk or output.new()\n   self.__kk:resize(self._kk:size()):copy(self._kk)\n   self.__kk:cmul(self._mask)\n\n   -- if mask == 0 then output[i] = J[kk[i]] else output[i] = 0\n\n   self._mask:add(-1):mul(-1) -- (1,0) - > (0,1)\n   output:view(-1):index(J, 1, self._kk:view(-1))\n   output:cmul(self._mask)\n\n   -- elseif mask == 1 then output[i] = kk[i]\n\n   output:add(self.__kk)\n\n   return output\nend\n\nreturn AM\n"
  },
  {
    "path": "opennmt/data/Batch.lua",
    "content": "--[[ Return the maxLength, sizes, and non-zero count\n  of a batch of `seq`s ignoring `ignore` words.\n--]]\nlocal function getLength(seq, ignore)\n  local sizes = torch.IntTensor(#seq):zero()\n  local max = 0\n  local uneven = false\n\n  for i = 1, #seq do\n    local len = seq[i]:size(1)\n    if ignore ~= nil then\n      len = len - ignore\n    end\n    if max == 0 or len > max then\n      max = len\n    end\n    if i > 1 and sizes[i - 1] ~= len then\n      uneven = true\n    end\n    sizes[i] = len\n  end\n  return max, sizes, uneven\nend\n\n--[[ Data management and batch creation.\n\nBatch interface reference [size]:\n\n  * size: number of sentences in the batch [1]\n  * sourceLength: max length in source batch [1]\n  * sourceSize:  lengths of each source [batch x 1]\n  * sourceInput:  left-padded idx's of source (PPPPPPABCDE) [batch x max]\n  * sourceInputFeatures: table of source features sequences\n  * sourceInputRev: right-padded  idx's of source rev (EDCBAPPPPPP) [batch x max]\n  * sourceInputRevFeatures: table of reversed source features sequences\n  * targetLength: max length in source batch [1]\n  * targetSize: lengths of each source [batch x 1]\n  * targetInput: input idx's of target (SABCDEPPPPPP) [batch x max]\n  * targetInputFeatures: table of target input features sequences\n  * targetOutput: expected output idx's of target (ABCDESPPPPPP) [batch x max]\n  * targetOutputFeatures: table of target output features sequences\n\n TODO: change name of size => maxlen\n--]]\n\n\n--[[ A batch of sentences to translate and targets. Manages padding,\n  features, and batch alignment (for efficiency).\n\n  Used by the decoder and encoder objects.\n--]]\nlocal Batch = torch.class('Batch')\n\n--[[ Create a batch object.\n\nParameters:\n\n  * `src` - 2D table of source batch indices\n  * `srcFeatures` - 2D table of source batch features (opt)\n  * `tgt` - 2D table of target batch indices\n  * `tgtFeatures` - 2D table of target batch features (opt)\n--]]\nfunction Batch:__init(src, srcFeatures, tgt, tgtFeatures)\n  src = src or {}\n  srcFeatures = srcFeatures or {}\n  tgtFeatures = tgtFeatures or {}\n\n  if tgt ~= nil then\n    assert(#src == #tgt, \"source and target must have the same batch size\")\n  end\n\n  self.size = #src\n  self.totalSize = self.size -- updated when this batch is part of a larger one (data parallelism).\n\n  self.sourceLength, self.sourceSize, self.uneven = getLength(src)\n\n  local sourceSeq = torch.LongTensor(self.sourceLength, self.size):fill(onmt.Constants.PAD)\n  self.sourceInput = sourceSeq:clone()\n  self.sourceInputRev = sourceSeq:clone()\n\n  self.sourceInputFeatures = {}\n  self.sourceInputRevFeatures = {}\n  -- will be used to return extra padded value\n  self.padTensor = torch.LongTensor(self.size):fill(onmt.Constants.PAD)\n\n  if #srcFeatures > 0 then\n    for _ = 1, #srcFeatures[1] do\n      table.insert(self.sourceInputFeatures, sourceSeq:clone())\n      table.insert(self.sourceInputRevFeatures, sourceSeq:clone())\n    end\n  end\n\n  if tgt ~= nil then\n    self.targetLength, self.targetSize = getLength(tgt, 1)\n\n    local targetSeq = torch.LongTensor(self.targetLength, self.size):fill(onmt.Constants.PAD)\n    self.targetInput = targetSeq:clone()\n    self.targetOutput = targetSeq:clone()\n\n    self.targetInputFeatures = {}\n    self.targetOutputFeatures = {}\n\n    if #tgtFeatures > 0 then\n      for _ = 1, #tgtFeatures[1] do\n        table.insert(self.targetInputFeatures, targetSeq:clone())\n        table.insert(self.targetOutputFeatures, targetSeq:clone())\n      end\n    end\n  end\n\n  for b = 1, self.size do\n    local sourceOffset = self.sourceLength - self.sourceSize[b] + 1\n    local sourceInput = src[b]\n    local sourceInputRev = src[b]:index(1, torch.linspace(self.sourceSize[b], 1, self.sourceSize[b]):long())\n\n    -- Source input is left padded [PPPPPPABCDE] .\n    self.sourceInput[{{sourceOffset, self.sourceLength}, b}]:copy(sourceInput)\n    self.sourceInputPadLeft = true\n\n    -- Rev source input is right padded [EDCBAPPPPPP] .\n    self.sourceInputRev[{{1, self.sourceSize[b]}, b}]:copy(sourceInputRev)\n    self.sourceInputRevPadLeft = false\n\n    for i = 1, #self.sourceInputFeatures do\n      local sourceInputFeatures = srcFeatures[b][i]\n      local sourceInputRevFeatures = srcFeatures[b][i]:index(1, torch.linspace(self.sourceSize[b], 1, self.sourceSize[b]):long())\n\n      self.sourceInputFeatures[i][{{sourceOffset, self.sourceLength}, b}]:copy(sourceInputFeatures)\n      self.sourceInputRevFeatures[i][{{1, self.sourceSize[b]}, b}]:copy(sourceInputRevFeatures)\n    end\n\n    if tgt ~= nil then\n      -- Input: [<s>ABCDE]\n      -- Ouput: [ABCDE</s>]\n      local targetLength = tgt[b]:size(1) - 1\n      local targetInput = tgt[b]:narrow(1, 1, targetLength)\n      local targetOutput = tgt[b]:narrow(1, 2, targetLength)\n\n      -- Target is right padded [<S>ABCDEPPPPPP] .\n      self.targetInput[{{1, targetLength}, b}]:copy(targetInput)\n      self.targetOutput[{{1, targetLength}, b}]:copy(targetOutput)\n\n      for i = 1, #self.targetInputFeatures do\n        local targetInputFeatures = tgtFeatures[b][i]:narrow(1, 1, targetLength)\n        local targetOutputFeatures = tgtFeatures[b][i]:narrow(1, 2, targetLength)\n\n        self.targetInputFeatures[i][{{1, targetLength}, b}]:copy(targetInputFeatures)\n        self.targetOutputFeatures[i][{{1, targetLength}, b}]:copy(targetOutputFeatures)\n      end\n    end\n  end\nend\n\n--[[ Set source input directly,\n\nParameters:\n\n  * `sourceInput` - a Tensor of size (sequence_length, batch_size, feature_dim)\n  ,or a sequence of size (sequence_length, batch_size). Be aware that sourceInput is not cloned here.\n\n--]]\nfunction Batch:setSourceInput(sourceInput)\n  assert (sourceInput:dim() >= 2, 'The sourceInput tensor should be of size (seq_len, batch_size, ...)')\n  self.size = sourceInput:size(2)\n  self.sourceLength = sourceInput:size(1)\n  self.sourceInputFeatures = {}\n  self.sourceInputRevReatures = {}\n  self.sourceInput = sourceInput\n  self.sourceInputRev = self.sourceInput:index(1, torch.linspace(self.sourceLength, 1, self.sourceLength):long())\n  return self\nend\n\n--[[ Set target input directly.\n\nParameters:\n\n  * `targetInput` - a tensor of size (sequence_length, batch_size). Padded with onmt.Constants.PAD. Be aware that targetInput is not cloned here.\n--]]\nfunction Batch:setTargetInput(targetInput)\n  assert (targetInput:dim() == 2, 'The targetInput tensor should be of size (seq_len, batch_size)')\n  self.targetInput = targetInput\n  self.size = targetInput:size(2)\n  self.totalSize = self.size\n  self.targetLength = targetInput:size(1)\n  self.targetInputFeatures = {}\n  self.targetSize = torch.sum(targetInput:transpose(1,2):ne(onmt.Constants.PAD), 2):view(-1):double()\n  return self\nend\n\n--[[ Set target output directly.\n\nParameters:\n\n  * `targetOutput` - a tensor of size (sequence_length, batch_size). Padded with onmt.Constants.PAD.  Be aware that targetOutput is not cloned here.\n--]]\nfunction Batch:setTargetOutput(targetOutput)\n  assert (targetOutput:dim() == 2, 'The targetOutput tensor should be of size (seq_len, batch_size)')\n  self.targetOutput = targetOutput\n  self.targetOutputFeatures = {}\n  return self\nend\n\nlocal function addInputFeatures(inputs, featuresSeq, t)\n  local features = {}\n  for j = 1, #featuresSeq do\n    local feat\n    if t > featuresSeq[j]:size(1) then\n      feat = onmt.Constants.PAD\n    else\n      feat = featuresSeq[j][t]\n    end\n    table.insert(features, feat)\n  end\n  if #features > 1 then\n    table.insert(inputs, features)\n  else\n    onmt.utils.Table.append(inputs, features)\n  end\nend\n\n--[[ Get source input batch at timestep `t`. --]]\nfunction Batch:getSourceInput(t)\n  local inputs\n\n  -- If a regular input, return word id, otherwise a table with features.\n  if t > self.sourceInput:size(1) then\n    inputs = self.padTensor\n  else\n    inputs = self.sourceInput[t]\n  end\n\n  if #self.sourceInputFeatures > 0 then\n    inputs = { inputs }\n    addInputFeatures(inputs, self.sourceInputFeatures, t)\n  end\n\n  return inputs\nend\n\n--[[ Get target input batch at timestep `t`. --]]\nfunction Batch:getTargetInput(t)\n  -- If a regular input, return word id, otherwise a table with features.\n  local inputs = self.targetInput[t]\n\n  if #self.targetInputFeatures > 0 then\n    inputs = { inputs }\n    addInputFeatures(inputs, self.targetInputFeatures, t)\n  end\n\n  return inputs\nend\n\n--[[ Get target output batch at timestep `t` (values t+1). --]]\nfunction Batch:getTargetOutput(t)\n  -- If a regular input, return word id, otherwise a table with features.\n  local outputs = { self.targetOutput[t] }\n\n  for j = 1, #self.targetOutputFeatures do\n    table.insert(outputs, self.targetOutputFeatures[j][t])\n  end\n\n  return outputs\nend\n\nreturn Batch\n"
  },
  {
    "path": "opennmt/data/BatchTensor.lua",
    "content": "local BatchTensor = torch.class('BatchTensor')\n\n--[[\n  Take Batch x TimeStep x layer size tensors\n]]\nfunction BatchTensor:__init(T, sizes)\n  self.size = T:size()[1]\n  self.sourceLength = T:size()[2]\n\n  self.sourceSize = sizes or torch.LongTensor(self.size):fill(self.sourceLength)\n\n  self.sourceInput = T\n  self.sourceInputPadLeft = true\n\n  self.sourceInputRev = self.sourceInput\n    :index(2, torch.linspace(self.sourceLength, 1, self.sourceLength):long())\n  self.sourceInputRevPadLeft = false\nend\n\nfunction BatchTensor:getSourceInput(t)\n  return self.sourceInput:select(2, t)\nend\n\nreturn BatchTensor\n"
  },
  {
    "path": "opennmt/data/Dataset.lua",
    "content": "--[[ Data management and batch creation. Handles data created by `preprocess.lua`. ]]\nlocal Dataset = torch.class(\"Dataset\")\n\n--[[ Initialize a data object given aligned tables of IntTensors `srcData`\n  and `tgtData`.\n--]]\nfunction Dataset:__init(srcData, tgtData)\n\n  self.src = srcData.words\n  self.srcFeatures = srcData.features\n\n  if tgtData ~= nil then\n    self.tgt = tgtData.words\n    self.tgtFeatures = tgtData.features\n  end\nend\n\n--[[ Setup up the training data to respect `maxBatchSize`.\n     If uneven_batches - then build up batches with different lengths ]]\nfunction Dataset:setBatchSize(maxBatchSize, uneven_batches)\n\n  self.batchRange = {}\n  self.maxSourceLength = 0\n  self.maxTargetLength = 0\n\n  local batchesCapacity = 0\n  local batchesOccupation = 0\n\n  -- Prepares batches in terms of range within self.src and self.tgt.\n  local offset = 0\n  local batchSize = 1\n  local maxSourceLength = 0\n  local targetLength = 0\n\n  for i = 1, #self.src do\n    -- Set up the offsets to make same source size batches of the\n    -- correct size.\n    local sourceLength = self.src[i]:size(1)\n    if batchSize == maxBatchSize or i == 1 or\n        (not(uneven_batches) and self.src[i]:size(1) ~= maxSourceLength) then\n      if i > 1 then\n        batchesCapacity = batchesCapacity + batchSize * maxSourceLength\n        table.insert(self.batchRange, { [\"begin\"] = offset, [\"end\"] = i - 1 })\n      end\n\n      offset = i\n      batchSize = 1\n      targetLength = 0\n      maxSourceLength = 0\n    else\n      batchSize = batchSize + 1\n    end\n    batchesOccupation = batchesOccupation + sourceLength\n    maxSourceLength = math.max(maxSourceLength, sourceLength)\n\n    self.maxSourceLength = math.max(self.maxSourceLength, sourceLength)\n\n    if self.tgt ~= nil then\n      -- Target contains <s> and </s>.\n      local targetSeqLength = self.tgt[i]:size(1) - 1\n      targetLength = math.max(targetLength, targetSeqLength)\n      self.maxTargetLength = math.max(self.maxTargetLength, targetSeqLength)\n    end\n  end\n\n  -- Catch last batch.\n  batchesCapacity = batchesCapacity + batchSize * maxSourceLength\n  table.insert(self.batchRange, { [\"begin\"] = offset, [\"end\"] = #self.src })\n  return #self.batchRange, batchesOccupation/batchesCapacity\nend\n\n--[[ Return number of batches. ]]\nfunction Dataset:batchCount()\n  if self.batchRange == nil then\n    if #self.src > 0 then\n      return 1\n    else\n      return 0\n    end\n  end\n  return #self.batchRange\nend\n\n--[[ Get `Batch` number `idx`. If nil make a batch of all the data. ]]\nfunction Dataset:getBatch(idx)\n  if #self.src == 0 then\n    return nil\n  end\n\n  if idx == nil or self.batchRange == nil then\n    return onmt.data.Batch.new(self.src, self.srcFeatures, self.tgt, self.tgtFeatures)\n  end\n\n  local rangeStart = self.batchRange[idx][\"begin\"]\n  local rangeEnd = self.batchRange[idx][\"end\"]\n\n  local src = {}\n  local tgt\n\n  if self.tgt ~= nil then\n    tgt = {}\n  end\n\n  local srcFeatures = {}\n  local tgtFeatures = {}\n\n  for i = rangeStart, rangeEnd do\n    table.insert(src, self.src[i])\n\n    if self.srcFeatures[i] then\n      table.insert(srcFeatures, self.srcFeatures[i])\n    end\n\n    if self.tgt ~= nil then\n      table.insert(tgt, self.tgt[i])\n\n      if self.tgtFeatures[i] then\n        table.insert(tgtFeatures, self.tgtFeatures[i])\n      end\n    end\n  end\n\n  return onmt.data.Batch.new(src, srcFeatures, tgt, tgtFeatures)\nend\n\nreturn Dataset\n"
  },
  {
    "path": "opennmt/data/Preprocessor.lua",
    "content": "--[[ Data Preparation functions. ]]\n\nlocal function vecToTensor(vec)\n  local t = torch.Tensor(#vec)\n  for i = 1, #vec do\n    t[i] = vec[i]\n  end\n  return t\nend\n\nlocal Preprocessor = torch.class('Preprocessor')\nlocal tds\n\nlocal bitextOptions = {\n  {\n    '-train_src', '',\n    [[Path to the training source data.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileExists\n    }\n  },\n  {\n    '-train_tgt', '',\n    [[Path to the training target data.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileExists\n    }\n  },\n  {\n    '-valid_src', '',\n    [[Path to the validation source data.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileExists\n    }\n  },\n  {\n    '-valid_tgt', '',\n    [[Path to the validation target data.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileExists\n    }\n  },\n  {\n    '-src_vocab', '',\n    [[Path to an existing source vocabulary.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists\n    }\n  },\n  {\n    '-tgt_vocab', '',\n    [[Path to an existing target vocabulary.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists\n    }\n  },\n  {\n    '-src_vocab_size', '50000',\n    [[Comma-separated list of source vocabularies size: `word[,feat1[,feat2[,...] ] ]`.\n      If = 0, vocabularies are not pruned.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.listUInt\n    }\n  },\n  {\n    '-tgt_vocab_size', '50000',\n    [[Comma-separated list of target vocabularies size: `word[,feat1[,feat2[,...] ] ]`.\n      If = 0, vocabularies are not pruned.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.listUInt\n    }\n  },\n  {\n    '-src_words_min_frequency', '0',\n    [[Comma-separated list of source words min frequency: `word[,feat1[,feat2[,...] ] ]`.\n      If = 0, vocabularies are pruned by size.]],\n    {\n      valid=onmt.utils.ExtendedCmdLine.listUInt\n    }\n  },\n  {\n    '-tgt_words_min_frequency', '0',\n    [[Comma-separated list of target words min frequency: `word[,feat1[,feat2[,...] ] ]`.\n      If = 0, vocabularies are pruned by size.]],\n    {\n      valid=onmt.utils.ExtendedCmdLine.listUInt\n    }\n  },\n  {\n    '-src_seq_length', 50,\n    [[Maximum source sequence length.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt\n    }\n  },\n  {\n    '-tgt_seq_length', 50,\n    [[Maximum target sequence length.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt\n    }\n  }\n}\n\nlocal monotextOptions = {\n  {\n    '-train', '',\n    [[Path to the training source data.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileExists\n    }\n  },\n  {\n    '-valid', '',\n    [[Path to the validation source data.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileExists\n    }\n  },\n  {\n    '-vocab', '',\n    [[Path to an existing source vocabulary.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists\n    }\n  },\n  {\n    '-vocab_size', '50000',\n    [[Comma-separated list of source vocabularies size: `word[,feat1[,feat2[,...] ] ]`.\n      If = 0, vocabularies are not pruned.]],\n    {\n      valid=onmt.utils.ExtendedCmdLine.listUInt\n    }\n  },\n  {\n    '-words_min_frequency', '0',\n    [[Comma-separated list of source words min frequency: `word[,feat1[,feat2[,...] ] ]`.\n      If = 0, vocabularies are pruned by size.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.listUInt\n    }\n  },\n  {\n    '-seq_length', 50,\n    [[Maximum source sequence length.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt()\n    }\n  }\n}\n\nlocal commonOptions = {\n  {\n    '-features_vocabs_prefix', '',\n    [[Path prefix to existing features vocabularies.]]\n  },\n  {\n    '-time_shift_feature', 1,\n    [[Time shift features on the decoder side.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isInt(0, 1)\n    }\n  },\n  {\n    '-sort', 1,\n    [[If = 1, sort the sentences by size to build batches without source padding.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isInt(0, 1)\n    }\n  },\n  {\n    '-shuffle', 1,\n    [[If = 1, shuffle data (prior sorting).]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isInt(0,1)\n    }\n  },\n  {\n    '-report_every', 100000,\n    [[Report status every this many sentences.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt()\n    }\n  }\n}\n\nfunction Preprocessor.declareOpts(cmd, mode)\n  mode = mode or 'bitext'\n  local options\n  if mode == 'bitext' then\n    options = bitextOptions\n  else\n    options = monotextOptions\n  end\n  for _, v in ipairs(commonOptions) do\n    table.insert(options, v)\n  end\n  cmd:setCmdLineOptions(options, 'Data')\nend\n\nfunction Preprocessor:__init(args, mode)\n  tds = require('tds')\n\n  mode = mode or 'bitext'\n  local options\n  if mode == 'bitext' then\n    options = bitextOptions\n  else\n    options = monotextOptions\n  end\n  for _, v in ipairs(commonOptions) do\n    table.insert(options, v)\n  end\n  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)\nend\n\nfunction Preprocessor:makeBilingualData(srcFile, tgtFile, srcDicts, tgtDicts, isValid)\n    -- sentence length distribution\n  local srcSentenceDist = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}\n  local tgtSentenceDist = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}\n\n  local src = tds.Vec()\n  local srcFeatures = tds.Vec()\n\n  local tgt = tds.Vec()\n  local tgtFeatures = tds.Vec()\n\n  local sizes = tds.Vec()\n\n  local count = 0\n  local ignored = 0\n  local avgSrcLength = 0\n  local avgTgtLength = 0\n  local prunedRatioSrc = 0\n  local prunedRatioTgt = 0\n\n  local srcReader = onmt.utils.FileReader.new(srcFile)\n  local tgtReader = onmt.utils.FileReader.new(tgtFile)\n\n  while true do\n    local srcTokens = srcReader:next()\n    local tgtTokens = tgtReader:next()\n\n    if srcTokens == nil or tgtTokens == nil then\n      if srcTokens == nil and tgtTokens ~= nil or srcTokens ~= nil and tgtTokens == nil then\n        _G.logger:warning('source and target do not have the same number of sentences')\n      end\n      break\n    end\n\n    local idxRange = math.floor(#srcTokens/10)+1\n    if idxRange > #srcSentenceDist then\n      idxRange = #srcSentenceDist\n    end\n    srcSentenceDist[idxRange] = srcSentenceDist[idxRange]+1\n    idxRange = math.floor(#tgtTokens/10)+1\n    if idxRange > #tgtSentenceDist then\n      idxRange = #tgtSentenceDist\n    end\n    tgtSentenceDist[idxRange] = tgtSentenceDist[idxRange]+1\n\n    if isValid(srcTokens, self.args.src_seq_length) and isValid(tgtTokens, self.args.tgt_seq_length) then\n      avgSrcLength = avgSrcLength * (#src / (#src + 1)) + #srcTokens / (#src + 1)\n      avgTgtLength = avgTgtLength * (#tgt / (#tgt + 1)) + #tgtTokens / (#tgt + 1)\n\n      local srcWords, srcFeats = onmt.utils.Features.extract(srcTokens)\n      local tgtWords, tgtFeats = onmt.utils.Features.extract(tgtTokens)\n\n      local srcVec = srcDicts.words:convertToIdx(srcWords, onmt.Constants.UNK_WORD)\n      local tgtVec = tgtDicts.words:convertToIdx(tgtWords,\n                                                 onmt.Constants.UNK_WORD,\n                                                 onmt.Constants.BOS_WORD,\n                                                 onmt.Constants.EOS_WORD)\n\n      local srcPruned = srcVec:eq(onmt.Constants.UNK):sum() / srcVec:size(1)\n      local tgtPruned = tgtVec:eq(onmt.Constants.UNK):sum() / tgtVec:size(1)\n\n      prunedRatioSrc = prunedRatioSrc * (#src / (#src + 1)) + srcPruned / (#src + 1)\n      prunedRatioTgt = prunedRatioTgt * (#tgt / (#tgt + 1)) + tgtPruned / (#tgt + 1)\n\n      src:insert(srcVec)\n      tgt:insert(tgtVec)\n\n      if #srcDicts.features > 0 then\n        srcFeatures:insert(onmt.utils.Features.generateSource(srcDicts.features, srcFeats, true))\n      end\n      if #tgtDicts.features > 0 then\n        tgtFeatures:insert(onmt.utils.Features.generateTarget(tgtDicts.features, tgtFeats, true, self.args.time_shift_feature))\n      end\n\n      sizes:insert(#srcWords)\n    else\n      ignored = ignored + 1\n    end\n\n    count = count + 1\n\n    if count % self.args.report_every == 0 then\n      _G.logger:info('... ' .. count .. ' sentences prepared')\n    end\n  end\n\n  for i=1, #srcSentenceDist do\n    srcSentenceDist[i] = srcSentenceDist[i]/count\n    tgtSentenceDist[i] = tgtSentenceDist[i]/count\n  end\n\n\n  srcReader:close()\n  tgtReader:close()\n\n  local function reorderData(perm)\n    src = onmt.utils.Table.reorder(src, perm, true)\n    tgt = onmt.utils.Table.reorder(tgt, perm, true)\n\n    if #srcDicts.features > 0 then\n      srcFeatures = onmt.utils.Table.reorder(srcFeatures, perm, true)\n    end\n    if #tgtDicts.features > 0 then\n      tgtFeatures = onmt.utils.Table.reorder(tgtFeatures, perm, true)\n    end\n  end\n\n  if self.args.shuffle == 1 then\n    _G.logger:info('... shuffling sentences')\n    local perm = torch.randperm(#src)\n    sizes = onmt.utils.Table.reorder(sizes, perm, true)\n    reorderData(perm)\n  end\n\n  if self.args.sort == 1 then\n    _G.logger:info('... sorting sentences by size')\n    local _, perm = torch.sort(vecToTensor(sizes))\n    reorderData(perm)\n  end\n\n  _G.logger:info('Prepared %d sentences:', #src)\n  _G.logger:info(' * %d sequences ignored due to source length > %d or target length > %d',\n                 ignored,\n                 self.args.src_seq_length,\n                 self.args.tgt_seq_length)\n  _G.logger:info(' * average sequence length: source = %.1f, target = %.1f',\n                 avgSrcLength,\n                 avgTgtLength)\n  _G.logger:info(' * %% of unkown words: source = %.1f%%, target = %.1f%%',\n                 prunedRatioSrc * 100,\n                 prunedRatioTgt * 100)\n\n  local dist='[ '\n  for i=1,#srcSentenceDist do\n    if i>1 then\n      dist = dist..' ; '\n    end\n    dist = dist..math.floor(srcSentenceDist[i]*100)..'%'\n  end\n  dist = dist..' ]'\n  _G.logger:info(' * Source Sentence Length (range of 10): '..dist)\n  dist='[ '\n  for i=1,#tgtSentenceDist do\n    if i>1 then\n      dist = dist..' ; '\n    end\n    dist = dist..math.floor(tgtSentenceDist[i]*100)..'%'\n  end\n  dist = dist..' ]'\n  _G.logger:info(' * Target Sentence Length (range of 10): '..dist)\n\n  local srcData = {\n    words = src,\n    features = srcFeatures\n  }\n\n  local tgtData = {\n    words = tgt,\n    features = tgtFeatures\n  }\n\n  return srcData, tgtData\nend\n\nfunction Preprocessor:makeMonolingualData(file, dicts, isValid)\n  local dataset = tds.Vec()\n  local features = tds.Vec()\n\n  local sizes = tds.Vec()\n\n  local count = 0\n  local ignored = 0\n  local avgLength = 0\n  local prunedRatio = 0\n\n  local reader = onmt.utils.FileReader.new(file)\n\n  while true do\n    local tokens = reader:next()\n\n    if tokens == nil then\n      break\n    end\n\n    if isValid(tokens, self.args.seq_length) then\n      avgLength = avgLength * (#dataset / (#dataset + 1)) + #tokens / (#dataset + 1)\n\n      local words, feats = onmt.utils.Features.extract(tokens)\n      local vec = dicts.words:convertToIdx(words, onmt.Constants.UNK_WORD)\n      local pruned = vec:eq(onmt.Constants.UNK):sum() / vec:size(1)\n\n      prunedRatio = prunedRatio * (#dataset / (#dataset + 1)) + pruned / (#dataset + 1)\n\n      dataset:insert(vec)\n\n      if #dicts.features > 0 then\n        features:insert(onmt.utils.Features.generateSource(dicts.features, feats, true))\n      end\n\n      sizes:insert(#words)\n    else\n      ignored = ignored + 1\n    end\n\n    count = count + 1\n\n  end\n\n  reader:close()\n\n  local function reorderData(perm)\n    dataset = onmt.utils.Table.reorder(dataset, perm, true)\n\n    if #dicts.features > 0 then\n      features = onmt.utils.Table.reorder(features, perm, true)\n    end\n  end\n\n  if self.args.shuffle == 1 then\n    _G.logger:info('... shuffling sentences')\n    local perm = torch.randperm(#dataset)\n    sizes = onmt.utils.Table.reorder(sizes, perm, true)\n    reorderData(perm)\n  end\n\n  if self.args.sort == 1 then\n    _G.logger:info('... sorting sentences by size')\n    local _, perm = torch.sort(vecToTensor(sizes))\n    reorderData(perm)\n  end\n\n  _G.logger:info('Prepared %d sentences:', #dataset)\n  _G.logger:info(' * %d sequences ignored due to length > %d', ignored, self.args.seq_length)\n  _G.logger:info(' * average sequence length = %.1f', avgLength)\n  _G.logger:info(' * %% of unkown words = %.1f%%', prunedRatio * 100)\n\n  local data = {\n    words = dataset,\n    features = features\n  }\n\n  return data\nend\n\nreturn Preprocessor\n"
  },
  {
    "path": "opennmt/data/SampledDataset.lua",
    "content": "--[[ Data management and batch creation. Handles data created by `preprocess.lua`. ]]\n\nlocal SampledDataset = torch.class(\"SampledDataset\")\n\nlocal options = {\n  {\n    '-sample', 0,\n    [[Number of instances to sample from train data in each epoch.]]\n  },\n  {\n    '-sample_w_ppl', false,\n    [[If set, ese perplexity as a probability distribution when sampling.]]\n  },\n  {\n    '-sample_w_ppl_init', 15,\n    [[Start perplexity-based sampling when average train perplexity per batch\n      falls below this value.]]\n  },\n  {\n    '-sample_w_ppl_max', -1.5,\n    [[When greater than 0, instances with perplexity above this value will be\n      considered as noise and ignored; when less than 0, mode + `-sample_w_ppl_max` * stdev\n      will be used as threshold.]]\n  }\n}\n\nfunction SampledDataset.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, 'Sampled dataset')\nend\n\n--[[ Initialize a data object given aligned tables of IntTensors `srcData`\n  and `tgtData`.\n--]]\nfunction SampledDataset:__init(srcData, tgtData, opt)\n\n  self.src = srcData.words or srcData.vectors\n  self.srcFeatures = srcData.features\n\n  if tgtData ~= nil then\n    self.tgt = tgtData.words  or tgtData.vectors\n    self.tgtFeatures = tgtData.features\n  end\n\n  self.samplingSize = opt.sample\n  self.sample_w_ppl = opt.sample_w_ppl\n  self.sample_w_ppl_init = opt.sample_w_ppl_init\n  self.sample_w_ppl_max = opt.sample_w_ppl_max\n  self.startedPplSampling = false\n\n  _G.logger:info(' * sampling ' .. opt.sample .. ' instances from ' .. #self.src .. ' at each epoch')\n  if opt.sample_w_ppl then\n    _G.logger:info(' * using train data perplexity as probability distribution when sampling')\n    _G.logger:info(' * sample_w_ppl_init: ' .. opt.sample_w_ppl_init)\n    _G.logger:info(' * sample_w_ppl_max: ' .. opt.sample_w_ppl_max)\n  end\n\n  if self.sample_w_ppl then\n    self.samplingProb = torch.Tensor(#self.src)\n    self.samplingProb:fill(self.sample_w_ppl_init)\n    self.ppl = torch.Tensor(#self.src)\n    self.ppl:fill(self.sample_w_ppl_init)\n  else\n    self.samplingProb = torch.ones(#self.src)\n  end\n\n  self.sampled = nil\n  self.sampledCnt = torch.zeros(#self.src)\nend\n\nfunction SampledDataset:checkModel(model)\n  if self:needIndividualLosses() and (not model.returnIndividualLosses or model:returnIndividualLosses(true) == false) then\n    _G.logger:info('Current model does not support training with invididual losses; Sampling with individual loss will be disabled.')\n    self.sample_w_ppl = false\n    self.samplingProb = torch.ones(#self.src)\n    self.ppl = nil\n  else\n    _G.logger:info('Current sampling does not require individual loss')\n    if model.returnIndividualLosses then\n      model:returnIndividualLosses(false)\n    end\n  end\nend\n\nfunction SampledDataset:needIndividualLosses()\n  return self.sample_w_ppl\nend\n\n--[[ Initiate sampling. ]]\nfunction SampledDataset:sample(logLevel)\n\n  logLevel = logLevel or 'INFO'\n\n  _G.logger:log('Sampling...', logLevel)\n\n  -- Populate self.samplingProb with self.ppl if average ppl is below self.sample_w_ppl_init.\n  if self.sample_w_ppl and not self.startedPplSampling then\n    local avgPpl = torch.sum(self.ppl)\n    avgPpl = avgPpl / self.ppl:size(1)\n    if avgPpl < self.sample_w_ppl_init then\n      _G.logger:log('Beginning to sample with ppl as probability distribution...', logLevel)\n      self.startedPplSampling = true\n    end\n  end\n\n  if self.startedPplSampling then\n    local threshold = self.sample_w_ppl_max\n\n    if self.sample_w_ppl_max < 0 then\n      -- Use mode (instead of mean) and stdev of samples with ppl >= mode to\n      -- find max ppl to consider (mode + x * stdev). when x is:\n      --      x: 1 ~ 100% - 31.7%/2 of train data are not included (divide by 2 because we cut only one-tail)\n      --      x: 2 ~ 100% - 4.55%/2\n      --      x: 3 ~ 100% - 0.270%/2\n      --      x: 4 ~ 100% - 0.00633%/2\n      --      x: 5 ~ 100% - 0.0000573%/2\n      --      x: 6 ~ 100% - 0.000000197%/2\n      --  (https://en.wikipedia.org/wiki/Standard_deviation)\n      -- We are using mode instead of average, and only samples above mode to calculate stdev, so\n      -- this is not really theoretically valid numbers, but more for emperical uses.\n\n      local x = math.abs(self.sample_w_ppl_max)\n\n      -- Find mode.\n      local pplRounded = torch.round(self.ppl * 100) / 100 -- keep up to the second decimal point\n      local bin = {}\n      for i = 1, pplRounded:size(1) do\n        if self.ppl[i] ~=  self.sample_w_ppl_init then\n          local idx = pplRounded[i]\n          if bin[idx] == nil then\n            bin[idx] = 0\n          end\n          bin[idx] = bin[idx] + 1\n        end\n      end\n      local mode = nil\n      for key, value in pairs(bin) do\n        if mode == nil or bin[mode] < value then\n          mode = key\n        end\n      end\n\n      -- stdev with mode only using samples with ppl >= mode\n      local sum = 0\n      local cnt = 0\n      for i = 1, self.ppl:size(1) do\n        if self.ppl[i] > mode and self.ppl[i] ~= self.sample_w_ppl_init then\n          sum = math.pow(self.ppl[i] - mode, 2)\n          cnt = cnt + 1\n        end\n      end\n      local stdev = math.sqrt(sum / (cnt - 1))\n\n      threshold = mode + x * stdev\n\n      _G.logger:log('Sampler count: ' .. cnt, logLevel)\n      _G.logger:log('Sampler mode: ' .. mode, logLevel)\n      _G.logger:log('Sampler stdev: ' .. stdev, logLevel)\n      _G.logger:log('Sampler threshold: ' .. threshold, logLevel)\n    end\n\n    for i = 1, self.ppl:size(1) do\n      if self.ppl[i] ~= self.sample_w_ppl_init and self.ppl[i] > threshold then\n        -- Assign low value to instances with ppl above threshold (outliers).\n        self.samplingProb[i] = 1\n      else\n        self.samplingProb[i] = self.ppl[i]\n      end\n    end\n  end\n\n  local sampler = onmt.data.AliasMultinomial.new(self.samplingProb)\n  _G.logger:log('Created sampler...', logLevel)\n  self.sampled = torch.LongTensor(self.samplingSize)\n  self.sampled = sampler:batchdraw(self.sampled)\n\n  self.sampledCnt:zero()\n  for i = 1, self.sampled:size(1) do\n    self.sampledCnt[self.sampled[i]] = self.sampledCnt[self.sampled[i]] + 1\n  end\n\n  _G.logger:log('Sampled ' .. self.sampled:size(1) .. ' instances', logLevel)\n\n  -- Prepares batches in terms of range within self.src and self.tgt.\n  local batchesCapacity = 0\n  local batchesOccupation = 0\n  self.batchRange = {}\n  local offset = 0\n  local sampleCntBegin = 1\n  local batchSize = 1\n  local maxSourceLength = -1\n  for i = 1, #self.src do\n    for j = 1, self.sampledCnt[i] do\n      local sourceLength = self.src[i]:size(1)\n      if batchSize == self.maxBatchSize or offset == 1 or\n         (not(self.uneven_batches) and self.src[i]:size(1) ~= maxSourceLength) then\n        if offset > 0 then\n          batchesCapacity = batchesCapacity + batchSize * maxSourceLength\n          local batchEnd = (j == 1) and i - 1 or i\n          local sampleCntEnd = (j == 1) and self.sampledCnt[i - 1] or j - 1\n          table.insert(self.batchRange, {\n            [\"begin\"] = offset,\n            [\"end\"] = batchEnd,\n            [\"sampleCntBegin\"] = sampleCntBegin,\n            [\"sampleCntEnd\"] = sampleCntEnd\n          })\n          sampleCntBegin = (j == 1) and 1 or j\n        end\n        offset = i\n        batchSize = 1\n        maxSourceLength = -1\n      else\n        batchSize = batchSize + 1\n      end\n      batchesOccupation = batchesOccupation + sourceLength\n      maxSourceLength = math.max(maxSourceLength, sourceLength)\n    end\n  end\n  -- Catch last batch.\n  if offset < #self.src then\n    batchesCapacity = batchesCapacity + batchSize * maxSourceLength\n    table.insert(self.batchRange, {\n      [\"begin\"] = offset,\n      [\"end\"] = #self.src,\n      [\"sampleCntBegin\"] = sampleCntBegin,\n      [\"sampleCntEnd\"] = self.sampledCnt[#self.src]\n    })\n  end\n\n  _G.logger:log('Prepared ' .. #self.batchRange .. ' batches', logLevel)\n  return #self.batchRange, batchesOccupation / batchesCapacity\nend\n\n--[[ Get perplexity. ]]\nfunction SampledDataset:getPpl()\n  return self.ppl\nend\n\n--[[ Set perplexity. ]]\nfunction SampledDataset:setLoss(batchIdx, loss)\n  assert(self:batchCount() >= batchIdx, \"Batch idx out of range: \" .. batchIdx .. \"/\" .. self:batchCount())\n  local rangeStart = self.batchRange[batchIdx][\"begin\"]\n  local rangeEnd = self.batchRange[batchIdx][\"end\"]\n  local sampleCntBegin = self.batchRange[batchIdx][\"sampleCntBegin\"]\n  local sampleCntEnd =  self.batchRange[batchIdx][\"sampleCntEnd\"]\n  loss = loss:exp()\n  local pplIdx = 1\n  for i = rangeStart, rangeEnd do\n    local jBegin = (i == rangeStart) and sampleCntBegin or 1\n    local jEnd = (i == rangeEnd) and math.min(self.sampledCnt[i], sampleCntEnd) or self.sampledCnt[i]\n    for _ = jBegin, jEnd do\n      self.ppl[i] = loss[pplIdx]\n      pplIdx = pplIdx + 1\n    end\n  end\nend\n\n--[[ Setup up the training data to respect `maxBatchSize`. ]]\nfunction SampledDataset:setBatchSize(maxBatchSize, uneven_batches)\n  self.maxBatchSize = maxBatchSize\n  self.uneven_batches = uneven_batches\n  self.maxSourceLength = 0\n  self.maxTargetLength = 0\n\n  for i = 1, #self.src do\n    self.maxSourceLength = math.max(self.maxSourceLength, self.src[i]:size(1))\n    if self.tgt ~= nil then\n      -- Target contains <s> and </s>.\n      local targetSeqLength = self.tgt[i]:size(1) - 1\n      self.maxTargetLength = math.max(self.maxTargetLength, targetSeqLength)\n    end\n  end\n\n  return self:sample('DEBUG')\nend\n\n--[[ Return number of sampled instances. ]]\nfunction SampledDataset:getNumSampled()\n  return self.sampled:size(1)\nend\n\n--[[ Return number of batches. ]]\nfunction SampledDataset:batchCount()\n  if self.batchRange == nil then\n    if #self.src > 0 then\n      return 1\n    else\n      return 0\n    end\n  end\n  return #self.batchRange\nend\n\n--[[ Get `Batch` number `idx`. If nil make a batch of all the data. ]]\nfunction SampledDataset:getBatch(batchIdx)\n  if #self.src == 0 then\n    return nil\n  end\n  if batchIdx == nil or self.batchRange == nil then\n    return onmt.data.Batch.new(self.src, self.srcFeatures, self.tgt, self.tgtFeatures)\n  end\n\n  assert(self:batchCount() >= batchIdx, \"Batch idx out of range: \" .. batchIdx .. \"/\" .. self:batchCount())\n\n  local rangeStart = self.batchRange[batchIdx][\"begin\"]\n  local rangeEnd = self.batchRange[batchIdx][\"end\"]\n  local sampleCntBegin =  self.batchRange[batchIdx][\"sampleCntBegin\"]\n  local sampleCntEnd =  self.batchRange[batchIdx][\"sampleCntEnd\"]\n\n  local src = {}\n  local tgt\n  if self.tgt ~= nil then\n    tgt = {}\n  end\n\n  local srcFeatures = {}\n  local tgtFeatures = {}\n\n  for i = rangeStart, rangeEnd do\n    local jBegin = (i == rangeStart) and sampleCntBegin or 1\n    local jEnd = (i == rangeEnd) and math.min(self.sampledCnt[i], sampleCntEnd) or self.sampledCnt[i]\n    for _ = jBegin, jEnd do\n      table.insert(src, self.src[i])\n      if self.srcFeatures[i] then\n        table.insert(srcFeatures, self.srcFeatures[i])\n      end\n      if self.tgt ~= nil then\n        table.insert(tgt, self.tgt[i])\n        if self.tgtFeatures[i] then\n          table.insert(tgtFeatures, self.tgtFeatures[i])\n        end\n      end\n    end\n  end\n\n  return onmt.data.Batch.new(src, srcFeatures, tgt, tgtFeatures)\nend\n\nreturn SampledDataset\n"
  },
  {
    "path": "opennmt/data/Vocabulary.lua",
    "content": "local path = require('pl.path')\n\n--[[ Vocabulary management utility functions. ]]\nlocal Vocabulary = torch.class(\"Vocabulary\")\n\nlocal function countFeatures(filename)\n  local reader = onmt.utils.FileReader.new(filename)\n  local _, _, numFeatures = onmt.utils.Features.extract(reader:next())\n  reader:close()\n  return numFeatures\nend\n\nfunction Vocabulary.make(filename, validFunc)\n  local wordVocab = onmt.utils.Dict.new({onmt.Constants.PAD_WORD, onmt.Constants.UNK_WORD,\n                                         onmt.Constants.BOS_WORD, onmt.Constants.EOS_WORD})\n  local featuresVocabs = {}\n\n  local reader = onmt.utils.FileReader.new(filename)\n  local lineId = 0\n\n  while true do\n    local sent = reader:next()\n    if sent == nil then\n      break\n    end\n\n    lineId = lineId + 1\n\n    if validFunc(sent) then\n      local words, features, numFeatures\n      local _, err = pcall(function ()\n        words, features, numFeatures = onmt.utils.Features.extract(sent)\n      end)\n\n      if err then\n        error(err .. ' (' .. filename .. ':' .. lineId .. ')')\n      end\n\n      if #featuresVocabs == 0 and numFeatures > 0 then\n        for j = 1, numFeatures do\n          featuresVocabs[j] = onmt.utils.Dict.new({onmt.Constants.PAD_WORD, onmt.Constants.UNK_WORD,\n                                                   onmt.Constants.BOS_WORD, onmt.Constants.EOS_WORD})\n        end\n      else\n        assert(#featuresVocabs == numFeatures,\n               'all sentences must have the same numbers of additional features (' .. filename .. ':' .. lineId .. ')')\n      end\n\n      for i = 1, #words do\n        wordVocab:add(words[i])\n\n        for j = 1, numFeatures do\n          featuresVocabs[j]:add(features[j][i])\n        end\n      end\n    end\n\n  end\n\n  reader:close()\n\n  return wordVocab, featuresVocabs\nend\n\nfunction Vocabulary.init(name, dataFile, vocabFile, vocabSize, wordsMinFrequency, featuresVocabsFiles, validFunc)\n  local wordVocab\n  local featuresVocabs = {}\n  local numFeatures = countFeatures(dataFile)\n\n  if vocabFile:len() > 0 then\n    -- If given, load existing word dictionary.\n    _G.logger:info('Reading ' .. name .. ' vocabulary from \\'' .. vocabFile .. '\\'...')\n    wordVocab = onmt.utils.Dict.new()\n    wordVocab:loadFile(vocabFile)\n    _G.logger:info('Loaded ' .. wordVocab:size() .. ' ' .. name .. ' words')\n  end\n\n  if featuresVocabsFiles:len() > 0 and numFeatures > 0 then\n    -- If given, discover existing features dictionaries.\n    local j = 1\n\n    while true do\n      local file = featuresVocabsFiles .. '.' .. name .. '_feature_' .. j .. '.dict'\n\n      if not path.exists(file) then\n        break\n      end\n\n      _G.logger:info('Reading ' .. name .. ' feature ' .. j .. ' vocabulary from \\'' .. file .. '\\'...')\n      featuresVocabs[j] = onmt.utils.Dict.new()\n      featuresVocabs[j]:loadFile(file)\n      _G.logger:info('Loaded ' .. featuresVocabs[j]:size() .. ' labels')\n\n      j = j + 1\n    end\n\n    assert(#featuresVocabs > 0,\n           'dictionary \\'' .. featuresVocabsFiles .. '.' .. name .. '_feature_1.dict\\' not found')\n    assert(#featuresVocabs == numFeatures,\n           'the data contains ' .. numFeatures .. ' ' .. name\n             .. ' features but only ' .. #featuresVocabs .. ' dictionaries were found')\n  end\n\n  if wordVocab == nil or (#featuresVocabs == 0 and numFeatures > 0) then\n    -- If a dictionary is still missing, generate it.\n    _G.logger:info('Building ' .. name  .. ' vocabularies...')\n    local genWordVocab, genFeaturesVocabs = Vocabulary.make(dataFile, validFunc)\n\n    local originalSizes = { genWordVocab:size() }\n    for i = 1, #genFeaturesVocabs do\n      table.insert(originalSizes, genFeaturesVocabs[i]:size())\n    end\n\n    local newSizes = onmt.utils.String.split(vocabSize, ',')\n    local minFrequency = onmt.utils.String.split(wordsMinFrequency, ',')\n\n    for i = 1, 1 + #genFeaturesVocabs do\n      newSizes[i] = (newSizes[i] and tonumber(newSizes[i])) or 0\n      minFrequency[i] = (minFrequency[i] and tonumber(minFrequency[i])) or 0\n    end\n\n    if wordVocab == nil then\n      if minFrequency[1] > 0 then\n        wordVocab = genWordVocab:pruneByMinFrequency(minFrequency[1])\n      elseif newSizes[1] > 0 then\n        wordVocab = genWordVocab:prune(newSizes[1])\n      else\n        wordVocab = genWordVocab\n      end\n\n      _G.logger:info('Created word dictionary of size '\n                       .. wordVocab:size() .. ' (pruned from ' .. originalSizes[1] .. ')')\n    end\n\n    if #featuresVocabs == 0 then\n      for i = 1, #genFeaturesVocabs do\n        if minFrequency[i + 1] > 0 then\n          featuresVocabs[i] = genFeaturesVocabs[i]:pruneByMinFrequency(minFrequency[i + 1])\n        elseif newSizes[i + 1] > 0 then\n          featuresVocabs[i] = genFeaturesVocabs[i]:prune(newSizes[i + 1])\n        else\n          featuresVocabs[i] = genFeaturesVocabs[i]\n        end\n\n        _G.logger:info('Created feature ' .. i .. ' dictionary of size '\n                         .. featuresVocabs[i]:size() .. ' (pruned from ' .. originalSizes[i + 1] .. ')')\n\n      end\n    end\n  end\n\n  _G.logger:info('')\n\n  return {\n    words = wordVocab,\n    features = featuresVocabs\n  }\nend\n\nfunction Vocabulary.save(name, vocab, file)\n  _G.logger:info('Saving ' .. name .. ' vocabulary to \\'' .. file .. '\\'...')\n  vocab:writeFile(file)\nend\n\nfunction Vocabulary.saveFeatures(name, vocabs, prefix)\n  for j = 1, #vocabs do\n    local file = prefix .. '.' .. name .. '_feature_' .. j .. '.dict'\n    _G.logger:info('Saving ' .. name .. ' feature ' .. j .. ' vocabulary to \\'' .. file .. '\\'...')\n    vocabs[j]:writeFile(file)\n  end\nend\n\nreturn Vocabulary\n"
  },
  {
    "path": "opennmt/data/init.lua",
    "content": "local data = {}\n\ndata.Dataset = require('opennmt.data.Dataset')\ndata.AliasMultinomial = require('opennmt.data.AliasMultinomial')\ndata.SampledDataset = require('opennmt.data.SampledDataset')\ndata.Batch = require('opennmt.data.Batch')\ndata.BatchTensor = require('opennmt.data.BatchTensor')\ndata.Vocabulary = require('opennmt.data.Vocabulary')\ndata.Preprocessor = require('opennmt.data.Preprocessor')\n\nreturn data\n"
  },
  {
    "path": "opennmt/init.lua",
    "content": "require('torch')\n\nonmt = {}\n\nonmt.utils = require('opennmt.utils.init')\n\nrequire('opennmt.modules.init')\n\nonmt.data = require('opennmt.data.init')\nonmt.train = require('opennmt.train.init')\nonmt.translate = require('opennmt.translate.init')\nonmt.tagger = require('opennmt.tagger.init')\n\nonmt.Constants = require('opennmt.Constants')\nonmt.Factory = require('opennmt.Factory')\nonmt.Model = require('opennmt.Model')\nonmt.Seq2Seq = require('opennmt.Seq2Seq')\nonmt.LanguageModel = require('opennmt.LanguageModel')\nonmt.SeqTagger = require('opennmt.SeqTagger')\nonmt.ModelSelector = require('opennmt.ModelSelector')\n\nreturn onmt\n"
  },
  {
    "path": "opennmt/modules/BiEncoder.lua",
    "content": "---------------------------------------------------------------------------------\n-- Local utility functions\n---------------------------------------------------------------------------------\n\nlocal function reverseInput(batch)\n  batch.sourceInput, batch.sourceInputRev = batch.sourceInputRev, batch.sourceInput\n  batch.sourceInputFeatures, batch.sourceInputRevFeatures = batch.sourceInputRevFeatures, batch.sourceInputFeatures\n  batch.sourceInputPadLeft, batch.sourceInputRevPadLeft = batch.sourceInputRevPadLeft, batch.sourceInputPadLeft\nend\n\n---------------------------------------------------------------------------------\n\n--[[ BiEncoder is a bidirectional Sequencer used for the source language.\n\n\n `netFwd`\n\n    h_1 => h_2 => h_3 => ... => h_n\n     |      |      |             |\n     .      .      .             .\n     |      |      |             |\n    h_1 => h_2 => h_3 => ... => h_n\n     |      |      |             |\n     |      |      |             |\n    x_1    x_2    x_3           x_n\n\n `netBwd`\n\n    h_1 <= h_2 <= h_3 <= ... <= h_n\n     |      |      |             |\n     .      .      .             .\n     |      |      |             |\n    h_1 <= h_2 <= h_3 <= ... <= h_n\n     |      |      |             |\n     |      |      |             |\n    x_1    x_2    x_3           x_n\n\nInherits from [onmt.Sequencer](onmt+modules+Sequencer).\n\n--]]\nlocal BiEncoder, parent = torch.class('onmt.BiEncoder', 'nn.Container')\n\nlocal options = {\n  {\n    '-brnn_merge', 'sum',\n    [[Merge action for the bidirectional states.]],\n    {\n      enum = {'concat', 'sum'},\n      structural = 0\n    }\n  }\n}\n\nfunction BiEncoder.declareOpts(cmd)\n  onmt.Encoder.declareOpts(cmd)\n  cmd:setCmdLineOptions(options)\nend\n\n\n--[[ Create a bi-encoder.\n\nParameters:\n\n  * `input` - input neural network.\n  * `rnn` - recurrent template module.\n  * `merge` - fwd/bwd merge operation {\"concat\", \"sum\"}\n]]\nfunction BiEncoder:__init(args, input)\n  parent.__init(self)\n\n  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)\n\n  local orgRNNSize = args.rnn_size\n\n  -- Compute rnn hidden size depending on hidden states merge action.\n  if self.args.brnn_merge == 'concat' then\n    if args.rnn_size % 2 ~= 0 then\n      error('in concat mode, rnn_size must be divisible by 2')\n    end\n    args.rnn_size = args.rnn_size / 2\n  end\n\n  self.args.rnn_size = args.rnn_size\n\n  self.fwd = onmt.Encoder.new(args, input)\n  self.bwd = onmt.Encoder.new(args, input:clone('weight', 'bias', 'gradWeight', 'gradBias'))\n\n  self.args.numEffectiveLayers = self.fwd.args.numEffectiveLayers\n\n  if self.args.brnn_merge == 'concat' then\n    self.args.hiddenSize = self.args.rnn_size * 2\n  else\n    self.args.hiddenSize = self.args.rnn_size\n  end\n\n  self:add(self.fwd)\n  self:add(self.bwd)\n\n  args.rnn_size = orgRNNSize\n\n  self:resetPreallocation()\nend\n\n--[[ Return a new BiEncoder using the serialized data `pretrained`. ]]\nfunction BiEncoder.load(pretrained)\n  local self = torch.factory('onmt.BiEncoder')()\n\n  parent.__init(self)\n\n  self.fwd = onmt.Encoder.load(pretrained.modules[1])\n  self.bwd = onmt.Encoder.load(pretrained.modules[2])\n  self.args = pretrained.args\n\n  -- backward compatibility\n  self.args.rnn_size = self.args.rnn_size or self.args.rnnSize\n  self.args.brnn_merge = self.args.brnn_merge or self.args.merge\n\n  self:add(self.fwd)\n  self:add(self.bwd)\n\n  self:resetPreallocation()\n\n  return self\nend\n\n--[[ Return data to serialize. ]]\nfunction BiEncoder:serialize()\n  local modulesData = {}\n  for i = 1, #self.modules do\n    table.insert(modulesData, self.modules[i]:serialize())\n  end\n\n  return {\n    name = 'BiEncoder',\n    modules = modulesData,\n    args = self.args\n  }\nend\n\nfunction BiEncoder:resetPreallocation()\n  -- Prototype for preallocated full context vector.\n  self.contextProto = torch.Tensor()\n\n  -- Prototype for preallocated full hidden states tensors.\n  self.stateProto = torch.Tensor()\n\n  -- Prototype for preallocated gradient of the backward context\n  self.gradContextBwdProto = torch.Tensor()\nend\n\nfunction BiEncoder:maskPadding()\n  self.fwd:maskPadding()\n  self.bwd:maskPadding()\nend\n\nfunction BiEncoder:forward(batch, statesFw, statesBw)\n  if self.statesProto == nil then\n    self.statesProto = onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,\n                                                         self.stateProto,\n                                                         { batch.size, self.args.hiddenSize })\n  end\n\n  local states = onmt.utils.Tensor.reuseTensorTable(self.statesProto, { batch.size, self.args.hiddenSize })\n  local context = onmt.utils.Tensor.reuseTensor(self.contextProto,\n                                                { batch.size, batch.sourceLength, self.args.hiddenSize })\n\n  local fwdStates, fwdContext = self.fwd:forward(batch, statesFw)\n  reverseInput(batch)\n  local bwdStates, bwdContext = self.bwd:forward(batch, statesBw)\n  reverseInput(batch)\n\n  if self.args.brnn_merge == 'concat' then\n    for i = 1, #fwdStates do\n      states[i]:narrow(2, 1, self.args.rnn_size):copy(fwdStates[i])\n      states[i]:narrow(2, self.args.rnn_size + 1, self.args.rnn_size):copy(bwdStates[i])\n    end\n    for t = 1, batch.sourceLength do\n      context[{{}, t}]:narrow(2, 1, self.args.rnn_size)\n        :copy(fwdContext[{{}, t}])\n      context[{{}, t}]:narrow(2, self.args.rnn_size + 1, self.args.rnn_size)\n        :copy(bwdContext[{{}, batch.sourceLength - t + 1}])\n    end\n  elseif self.args.brnn_merge == 'sum' then\n    for i = 1, #states do\n      states[i]:copy(fwdStates[i])\n      states[i]:add(bwdStates[i])\n    end\n    for t = 1, batch.sourceLength do\n      context[{{}, t}]:copy(fwdContext[{{}, t}])\n      context[{{}, t}]:add(bwdContext[{{}, batch.sourceLength - t + 1}])\n    end\n  end\n\n  return states, context\nend\n\nfunction BiEncoder:backward(batch, gradStatesOutput, gradContextOutput)\n  gradStatesOutput = gradStatesOutput\n    or onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,\n                                         onmt.utils.Cuda.convert(torch.Tensor()),\n                                         { batch.size, self.args.hiddenSize })\n\n  local gradContextOutputFwd\n  local gradContextOutputBwd\n\n  local gradStatesOutputFwd = {}\n  local gradStatesOutputBwd = {}\n\n  if self.args.brnn_merge == 'concat' then\n    local gradContextOutputSplit = gradContextOutput:chunk(2, 3)\n    gradContextOutputFwd = gradContextOutputSplit[1]\n    gradContextOutputBwd = gradContextOutputSplit[2]\n\n    for i = 1, #gradStatesOutput do\n      local statesSplit = gradStatesOutput[i]:chunk(2, 2)\n      table.insert(gradStatesOutputFwd, statesSplit[1])\n      table.insert(gradStatesOutputBwd, statesSplit[2])\n    end\n  elseif self.args.brnn_merge == 'sum' then\n    gradContextOutputFwd = gradContextOutput\n    gradContextOutputBwd = gradContextOutput\n\n    gradStatesOutputFwd = gradStatesOutput\n    gradStatesOutputBwd = gradStatesOutput\n  end\n\n  local gradInputFwd, gradStateFwd = self.fwd:backward(batch, gradStatesOutputFwd, gradContextOutputFwd)\n\n  -- reverse gradients of the backward context\n  local gradContextBwd = onmt.utils.Tensor.reuseTensor(self.gradContextBwdProto,\n                                                       { batch.size, batch.sourceLength, self.args.rnn_size })\n\n  for t = 1, batch.sourceLength do\n    gradContextBwd[{{}, t}]:copy(gradContextOutputBwd[{{}, batch.sourceLength - t + 1}])\n  end\n\n  local gradInputBwd, gradStateBwd = self.bwd:backward(batch, gradStatesOutputBwd, gradContextBwd)\n\n  for t = 1, batch.sourceLength do\n    onmt.utils.Tensor.recursiveAdd(gradInputFwd[t], gradInputBwd[batch.sourceLength - t + 1])\n  end\n\n  onmt.utils.Table.append(gradStateFwd, gradStateBwd)\n\n  return gradInputFwd, gradStateFwd\nend\n"
  },
  {
    "path": "opennmt/modules/DBiEncoder.lua",
    "content": "--[[ DBiEncoder is a deep bidirectional Sequencer used for the source language.\n\n\n--]]\nlocal DBiEncoder, parent = torch.class('onmt.DBiEncoder', 'nn.Container')\n\nlocal options = {}\n\nfunction DBiEncoder.declareOpts(cmd)\n  onmt.BiEncoder.declareOpts(cmd)\n  cmd:setCmdLineOptions(options)\nend\n\n\n--[[ Create a deep bidirectional encoder - each layers reconnect before starting another bidirectional layer\n\nParameters:\n\n  * `args` - global arguments\n  * `input` - input neural network.\n]]\nfunction DBiEncoder:__init(args, input)\n  parent.__init(self)\n\n  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)\n  self.args.layers = args.layers\n  self.args.dropout = args.dropout\n  local dropout_input = args.dropout_input\n\n  self.layers = {}\n\n  args.layers = 1\n  args.brnn_merge = 'sum'\n\n  for _= 1, self.args.layers do\n    table.insert(self.layers, onmt.BiEncoder(args, input))\n    local identity = nn.Identity()\n    identity.inputSize = args.rnn_size\n    input = identity\n    self:add(self.layers[#self.layers])\n    -- trick to force a dropout on each layer L > 1\n    if #self.layers == 1 and args.dropout > 0 then\n      args.dropout_input = true\n    end\n  end\n  args.layers = self.args.layers\n  self.args.numEffectiveLayers = self.layers[1].args.numEffectiveLayers * self.args.layers\n  self.args.hiddenSize = args.rnn_size\n  args.dropout_input = dropout_input\n\n  self:resetPreallocation()\nend\n\n--[[ Return a new DBiEncoder using the serialized data `pretrained`. ]]\nfunction DBiEncoder.load(pretrained)\n  local self = torch.factory('onmt.DBiEncoder')()\n  parent.__init(self)\n\n  self.layers = {}\n\n  for i=1, #pretrained.layers do\n    self.layers[i] = onmt.BiEncoder.load(pretrained.layers[i])\n    self:add(self.layers[i])\n  end\n\n  self.args = pretrained.args\n\n  self:resetPreallocation()\n  return self\nend\n\n--[[ Return data to serialize. ]]\nfunction DBiEncoder:serialize()\n  local layersData = {}\n\n  for i = 1, #self.layers do\n    table.insert(layersData, self.layers[i]:serialize())\n  end\n\n  return {\n    name = 'DBiEncoder',\n    layers = layersData,\n    args = self.args\n  }\nend\n\nfunction DBiEncoder:resetPreallocation()\n  -- Prototype for preallocated full context vector.\n  self.contextProto = torch.Tensor()\n\n  -- Prototype for preallocated full hidden states tensors.\n  self.stateProto = torch.Tensor()\n\n  -- Prototype for preallocated gradient context\n  self.gradContextProto = torch.Tensor()\nend\n\nfunction DBiEncoder:maskPadding()\n  for _, layer in ipairs(self.layers) do\n    layer:maskPadding()\n  end\nend\n\nfunction DBiEncoder:forward(batch)\n  if self.statesProto == nil then\n    self.statesProto = onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,\n                                                         self.stateProto,\n                                                         { batch.size, self.args.hiddenSize })\n  end\n\n  local states = onmt.utils.Tensor.reuseTensorTable(self.statesProto, { batch.size, self.args.hiddenSize })\n  local context = onmt.utils.Tensor.reuseTensor(self.contextProto,\n                                                { batch.size, batch.sourceLength, self.args.hiddenSize })\n\n  local stateIdx = 1\n  self.inputs = { batch }\n  self.lranges = {}\n  for i = 1,#self.layers do\n    local layerStates, layerContext = self.layers[i]:forward(self.inputs[i])\n    if i ~= #self.layers then\n      table.insert(self.inputs, onmt.data.BatchTensor.new(layerContext, batch.sourceSize))\n    else\n      context:copy(layerContext)\n    end\n    table.insert(self.lranges, {stateIdx, #layerStates})\n    for j = 1,#layerStates do\n      states[stateIdx]:copy(layerStates[j])\n      stateIdx = stateIdx + 1\n    end\n  end\n  return states, context\nend\n\nfunction DBiEncoder:backward(batch, gradStatesOutput, gradContextOutput)\n  local gradInputs\n\n  for i = #self.layers, 1, -1 do\n    local lrange_gradStatesOutput\n    if gradStatesOutput then\n      lrange_gradStatesOutput = gradStatesOutput[{}]\n    end\n    gradInputs = self.layers[i]:backward(self.inputs[i], lrange_gradStatesOutput, gradContextOutput)\n    if i ~= 1 then\n      gradContextOutput = onmt.utils.Tensor.reuseTensor(self.gradContextProto,\n                                              { batch.size, #gradInputs, self.args.hiddenSize })\n      for t = 1, #gradInputs do\n        gradContextOutput[{{},t,{}}]:copy(gradInputs[t])\n      end\n    end\n  end\n\n  return gradInputs\nend\n"
  },
  {
    "path": "opennmt/modules/Decoder.lua",
    "content": "--[[ Unit to decode a sequence of output tokens.\n\n     .      .      .             .\n     |      |      |             |\n    h_1 => h_2 => h_3 => ... => h_n\n     |      |      |             |\n     .      .      .             .\n     |      |      |             |\n    h_1 => h_2 => h_3 => ... => h_n\n     |      |      |             |\n     |      |      |             |\n    x_1    x_2    x_3           x_n\n\nInherits from [onmt.Sequencer](onmt+modules+Sequencer).\n\n--]]\nlocal Decoder, parent = torch.class('onmt.Decoder', 'onmt.Sequencer')\n\nlocal options = {\n  {\n    '-input_feed', 1,\n    [[Feed the context vector at each time step as additional input\n      (via concatenation with the word embeddings) to the decoder.]],\n    {\n      enum = {0, 1},\n      structural = 0\n    }\n  }\n}\n\nfunction Decoder.declareOpts(cmd)\n  cmd:setCmdLineOptions(options)\n  onmt.Encoder.declareOpts(cmd)\nend\n\n--[[ Construct a decoder layer.\n\nParameters:\n\n  * `args` - module arguments\n  * `inputNetwork` - input nn module.\n  * `generator` - an output generator.\n  * `attentionModel` - attention model to apply on source.\n--]]\nfunction Decoder:__init(args, inputNetwork, generator, attentionModel)\n  local RNN = onmt.LSTM\n  if args.rnn_type == 'GRU' then\n    RNN = onmt.GRU\n  end\n\n  -- Input feeding means the decoder takes an extra\n  -- vector each time representing the attention at the\n  -- previous step.\n  local inputSize = inputNetwork.inputSize + (args.input_feed * args.rnn_size)\n\n  local rnn = RNN.new(args.layers, inputSize, args.rnn_size,\n                      args.dropout, args.residual, args.dropout_input)\n\n  self.rnn = rnn\n  self.inputNet = inputNetwork\n\n  self.args = args\n  self.args.rnnSize = self.rnn.outputSize\n  self.args.numEffectiveLayers = self.rnn.numEffectiveLayers\n\n  self.args.inputIndex = {}\n  self.args.outputIndex = {}\n\n  self.args.inputFeed = (args.input_feed == 1)\n\n  parent.__init(self, self:_buildModel(attentionModel))\n\n  -- The generator use the output of the decoder sequencer to generate the\n  -- likelihoods over the target vocabulary.\n  self.generator = generator\n  self:add(self.generator)\n\n  self:resetPreallocation()\nend\n\nfunction Decoder:returnIndividualLosses(enable)\n  self.indvLoss = enable\nend\n\n--[[ Return a new Decoder using the serialized data `pretrained`. ]]\nfunction Decoder.load(pretrained)\n  local self = torch.factory('onmt.Decoder')()\n\n  self.args = pretrained.args\n\n  parent.__init(self, pretrained.modules[1])\n  self.generator = pretrained.modules[2]\n  self:add(self.generator)\n\n  self:resetPreallocation()\n\n  return self\nend\n\n--[[ Return data to serialize. ]]\nfunction Decoder:serialize()\n  return {\n    modules = self.modules,\n    args = self.args\n  }\nend\n\nfunction Decoder:resetPreallocation()\n  if self.args.inputFeed then\n    self.inputFeedProto = torch.Tensor()\n  end\n\n  -- Prototype for preallocated hidden and cell states.\n  self.stateProto = torch.Tensor()\n\n  -- Prototype for preallocated output gradients.\n  self.gradOutputProto = torch.Tensor()\n\n  -- Prototype for preallocated context gradient.\n  self.gradContextProto = torch.Tensor()\nend\n\n--[[ Build a default one time-step of the decoder\n\nReturns: An nn-graph mapping\n\n  $${(c^1_{t-1}, h^1_{t-1}, .., c^L_{t-1}, h^L_{t-1}, x_t, con/H, if) =>\n  (c^1_{t}, h^1_{t}, .., c^L_{t}, h^L_{t}, a)}$$\n\n  Where ${c^l}$ and ${h^l}$ are the hidden and cell states at each layer,\n  ${x_t}$ is a sparse word to lookup,\n  ${con/H}$ is the context/source hidden states for attention,\n  ${if}$ is the input feeding, and\n  ${a}$ is the context vector computed at this timestep.\n--]]\nfunction Decoder:_buildModel(attentionModel)\n  local inputs = {}\n  local states = {}\n\n  -- Inputs are previous layers first.\n  for _ = 1, self.args.numEffectiveLayers do\n    local h0 = nn.Identity()() -- batchSize x rnnSize\n    table.insert(inputs, h0)\n    table.insert(states, h0)\n  end\n\n  local x = nn.Identity()() -- batchSize\n  table.insert(inputs, x)\n  self.args.inputIndex.x = #inputs\n\n  local context = nn.Identity()() -- batchSize x sourceLength x rnnSize\n  table.insert(inputs, context)\n  self.args.inputIndex.context = #inputs\n\n  local inputFeed\n  if self.args.inputFeed then\n    inputFeed = nn.Identity()() -- batchSize x rnnSize\n    table.insert(inputs, inputFeed)\n    self.args.inputIndex.inputFeed = #inputs\n  end\n\n  -- Compute the input network.\n  local input = self.inputNet(x)\n\n  -- If set, concatenate previous decoder output.\n  if self.args.inputFeed then\n    input = nn.JoinTable(2)({input, inputFeed})\n  end\n  table.insert(states, input)\n\n  -- Forward states and input into the RNN.\n  local outputs = self.rnn(states)\n\n  if self.args.numEffectiveLayers > 1 then\n    -- The output of a subgraph is a node: split it to access the last RNN output.\n    outputs = { outputs:split(self.args.numEffectiveLayers) }\n  else\n    outputs = { outputs }\n  end\n\n  -- Compute the attention here using h^L as query.\n  local attnLayer = attentionModel\n  attnLayer.name = 'decoderAttn'\n  local attnInput = {outputs[#outputs], context}\n  local attnOutput = attnLayer(attnInput)\n  if self.rnn.dropout > 0 then\n    attnOutput = nn.Dropout(self.rnn.dropout)(attnOutput)\n  end\n  table.insert(outputs, attnOutput)\n  return nn.gModule(inputs, outputs)\nend\n\nfunction Decoder:findAttentionModel()\n  if not self.decoderAttn then\n    self.network:apply(function (layer)\n      if layer.name == 'decoderAttn' then\n        self.decoderAttn = layer\n      elseif layer.name == 'softmaxAttn' then\n        self.softmaxAttn = layer\n      end\n    end)\n    self.decoderAttnClones = {}\n  end\n  for t = #self.decoderAttnClones+1, #self.networkClones do\n    self:net(t):apply(function (layer)\n      if layer.name == 'decoderAttn' then\n        self.decoderAttnClones[t] = layer\n      elseif layer.name == 'softmaxAttn' then\n        self.decoderAttnClones[t].softmaxAttn = layer\n      end\n    end)\n  end\nend\n\n--[[ Mask padding means that the attention-layer is constrained to\n  give zero-weight to padding. This is done by storing a reference\n  to the softmax attention-layer.\n\n  Parameters:\n\n  * See  [onmt.MaskedSoftmax](onmt+modules+MaskedSoftmax).\n--]]\nfunction Decoder:maskPadding(sourceSizes, sourceLength)\n  self:findAttentionModel()\n\n  local function substituteSoftmax(module)\n    if module.name == 'softmaxAttn' then\n      local mod\n      if sourceSizes ~= nil then\n        mod = onmt.MaskedSoftmax(sourceSizes, sourceLength)\n      else\n        mod = nn.SoftMax()\n      end\n\n      mod.name = 'softmaxAttn'\n      mod:type(module._type)\n      self.softmaxAttn = mod\n      return mod\n    else\n      return module\n    end\n  end\n\n  self.decoderAttn:replace(substituteSoftmax)\n\n  for t = 1, #self.networkClones do\n    self.decoderAttnClones[t]:replace(substituteSoftmax)\n  end\nend\n\n--[[ Run one step of the decoder.\n\nParameters:\n\n  * `input` - input to be passed to inputNetwork.\n  * `prevStates` - stack of hidden states (batch x layers*model x rnnSize)\n  * `context` - encoder output (batch x n x rnnSize)\n  * `prevOut` - previous distribution (batch x #words)\n  * `t` - current timestep\n\nReturns:\n\n 1. `out` - Top-layer hidden state.\n 2. `states` - All states.\n--]]\nfunction Decoder:forwardOne(input, prevStates, context, prevOut, t)\n  local inputs = {}\n\n  -- Create RNN input (see sequencer.lua `buildNetwork('dec')`).\n  onmt.utils.Table.append(inputs, prevStates)\n  table.insert(inputs, input)\n  table.insert(inputs, context)\n  local inputSize\n  if torch.type(input) == 'table' then\n    inputSize = input[1]:size(1)\n  else\n    inputSize = input:size(1)\n  end\n\n  if self.args.inputFeed then\n    if prevOut == nil then\n      table.insert(inputs, onmt.utils.Tensor.reuseTensor(self.inputFeedProto,\n                                                         { inputSize, self.args.rnnSize }))\n    else\n      table.insert(inputs, prevOut)\n    end\n  end\n\n  -- Remember inputs for the backward pass.\n  if self.train then\n    self.inputs[t] = inputs\n  end\n\n  local outputs = self:net(t):forward(inputs)\n\n  -- Make sure decoder always returns table.\n  if type(outputs) ~= \"table\" then outputs = { outputs } end\n\n  local out = outputs[#outputs]\n  local states = {}\n  for i = 1, #outputs - 1 do\n    table.insert(states, outputs[i])\n  end\n\n  return out, states\nend\n\n--[[Compute all forward steps.\n\n  Parameters:\n\n  * `batch` - `Batch` object\n  * `encoderStates` -\n  * `context` -\n  * `func` - Calls `func(out, t)` each timestep.\n--]]\n\nfunction Decoder:forwardAndApply(batch, encoderStates, context, func)\n  -- TODO: Make this a private method.\n\n  if self.statesProto == nil then\n    self.statesProto = onmt.utils.Tensor.initTensorTable(#encoderStates,\n                                                         self.stateProto,\n                                                         { batch.size, self.args.rnnSize })\n  end\n\n  local states = onmt.utils.Tensor.copyTensorTable(self.statesProto, encoderStates)\n\n  local prevOut\n\n  for t = 1, batch.targetLength do\n    prevOut, states = self:forwardOne(batch:getTargetInput(t), states, context, prevOut, t)\n    func(prevOut, t)\n  end\nend\n\n--[[Compute all forward steps.\n\n  Parameters:\n\n  * `batch` - a `Batch` object.\n  * `encoderStates` - a batch of initial decoder states (optional) [0]\n  * `context` - the context to apply attention to.\n\n  Returns: Table of top hidden state for each timestep.\n--]]\nfunction Decoder:forward(batch, encoderStates, context)\n  encoderStates = encoderStates\n    or onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,\n                                         onmt.utils.Cuda.convert(torch.Tensor()),\n                                         { batch.size, self.args.rnnSize })\n  if self.train then\n    self.inputs = {}\n  end\n\n  local outputs = {}\n\n  self:forwardAndApply(batch, encoderStates, context, function (out)\n    table.insert(outputs, out)\n  end)\n\n  return outputs\nend\n\n--[[ Compute the backward update.\n\nParameters:\n\n  * `batch` - a `Batch` object\n  * `outputs` - expected outputs\n  * `criterion` - a single target criterion object\n\n  Note: This code runs both the standard backward and criterion forward/backward.\n  It returns both the gradInputs and the loss.\n  -- ]]\nfunction Decoder:backward(batch, outputs, criterion)\n  if self.gradOutputsProto == nil then\n    self.gradOutputsProto = onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers + 1,\n                                                              self.gradOutputProto,\n                                                              { batch.size, self.args.rnnSize })\n  end\n\n  local gradStatesInput = onmt.utils.Tensor.reuseTensorTable(self.gradOutputsProto,\n                                                             { batch.size, self.args.rnnSize })\n  local gradContextInput = onmt.utils.Tensor.reuseTensor(self.gradContextProto,\n                                                         { batch.size, batch.encoderOutputLength or batch.sourceLength, self.args.rnnSize })\n\n  local loss = 0\n  local indvAvgLoss = torch.zeros(outputs[1]:size(1))\n\n  for t = batch.targetLength, 1, -1 do\n    -- Compute decoder output gradients.\n    -- Note: This would typically be in the forward pass.\n    local pred = self.generator:forward(outputs[t])\n    local output = batch:getTargetOutput(t)\n\n    if self.indvLoss then\n      for i = 1, pred[1]:size(1) do\n        if t <= batch.targetSize[i] then\n          local tmpPred = {}\n          local tmpOutput = {}\n          for j = 1, #pred do\n            table.insert(tmpPred, pred[j][{{i}, {}}])\n            table.insert(tmpOutput, output[j][{{i}}])\n          end\n          local tmpLoss = criterion:forward(tmpPred, tmpOutput)\n          indvAvgLoss[i] = indvAvgLoss[i] + tmpLoss\n          loss = loss + tmpLoss\n        end\n      end\n    else\n      loss = loss + criterion:forward(pred, output)\n    end\n\n    -- Compute the criterion gradient.\n    local genGradOut = criterion:backward(pred, output)\n    for j = 1, #genGradOut do\n      genGradOut[j]:div(batch.totalSize)\n    end\n\n    -- Compute the final layer gradient.\n    local decGradOut = self.generator:backward(outputs[t], genGradOut)\n    gradStatesInput[#gradStatesInput]:add(decGradOut)\n\n    -- Compute the standard backward.\n    local gradInput = self:net(t):backward(self.inputs[t], gradStatesInput)\n\n    -- Accumulate encoder output gradients.\n    gradContextInput:add(gradInput[self.args.inputIndex.context])\n    gradStatesInput[#gradStatesInput]:zero()\n\n    -- Accumulate previous output gradients with input feeding gradients.\n    if self.args.inputFeed and t > 1 then\n      gradStatesInput[#gradStatesInput]:add(gradInput[self.args.inputIndex.inputFeed])\n    end\n\n    -- Prepare next decoder output gradients.\n    for i = 1, #self.statesProto do\n      gradStatesInput[i]:copy(gradInput[i])\n    end\n  end\n\n  if self.indvLoss then\n    indvAvgLoss = torch.cdiv(indvAvgLoss, batch.targetSize:double())\n  end\n\n  return gradStatesInput, gradContextInput, loss, indvAvgLoss\nend\n\n--[[ Compute the loss on a batch.\n\nParameters:\n\n  * `batch` - a `Batch` to score.\n  * `encoderStates` - initialization of decoder.\n  * `context` - the attention context.\n  * `criterion` - a pointwise criterion.\n\n--]]\nfunction Decoder:computeLoss(batch, encoderStates, context, criterion)\n  encoderStates = encoderStates\n    or onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,\n                                         onmt.utils.Cuda.convert(torch.Tensor()),\n                                         { batch.size, self.args.rnnSize })\n\n  local loss = 0\n  self:forwardAndApply(batch, encoderStates, context, function (out, t)\n    local pred = self.generator:forward(out)\n    local output = batch:getTargetOutput(t)\n    loss = loss + criterion:forward(pred, output)\n  end)\n\n  return loss\nend\n\n\n--[[ Compute the score of a batch.\n\nParameters:\n\n  * `batch` - a `Batch` to score.\n  * `encoderStates` - initialization of decoder.\n  * `context` - the attention context.\n\n--]]\nfunction Decoder:computeScore(batch, encoderStates, context)\n  encoderStates = encoderStates\n    or onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,\n                                         onmt.utils.Cuda.convert(torch.Tensor()),\n                                         { batch.size, self.args.rnnSize })\n\n  local score = {}\n\n  self:forwardAndApply(batch, encoderStates, context, function (out, t)\n    local pred = self.generator:forward(out)\n    for b = 1, batch.size do\n      if t <= batch.targetSize[b] then\n        score[b] = (score[b] or 0) + pred[1][b][batch.targetOutput[t][b]]\n      end\n    end\n  end)\n\n  return score\nend\n"
  },
  {
    "path": "opennmt/modules/Encoder.lua",
    "content": "--[[ Encoder is a unidirectional Sequencer used for the source language.\n\n    h_1 => h_2 => h_3 => ... => h_n\n     |      |      |             |\n     .      .      .             .\n     |      |      |             |\n    h_1 => h_2 => h_3 => ... => h_n\n     |      |      |             |\n     |      |      |             |\n    x_1    x_2    x_3           x_n\n\n\nInherits from [onmt.Sequencer](onmt+modules+Sequencer).\n--]]\nlocal Encoder, parent = torch.class('onmt.Encoder', 'onmt.Sequencer')\n\nlocal options = {\n  {\n    '-layers', 2,\n    [[Number of recurrent layers of the encoder and decoder.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt(),\n      structural = 0\n    }\n  },\n  {\n    '-rnn_size', 500,\n    [[Hidden size of the recurrent unit.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt(),\n      structural = 0\n    }\n  },\n  {\n    '-rnn_type', 'LSTM',\n    [[Type of recurrent cell.]],\n    {\n      enum = {'LSTM', 'GRU'},\n      structural = 0\n    }\n  },\n  {\n    '-dropout', 0.3,\n    [[Dropout probability applied between recurrent layers.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isFloat(0, 1),\n      structural = 1\n    }\n  },\n  {\n    '-dropout_input', false,\n    [[Also apply dropout to the input of the recurrent module.]],\n    {\n      structural = 0\n    }\n  },\n  {\n    '-residual', false,\n    [[Add residual connections between recurrent layers.]],\n    {\n      structural = 0\n    }\n  }\n}\n\nfunction Encoder.declareOpts(cmd)\n  cmd:setCmdLineOptions(options)\nend\n\n--[[ Construct an Encoder layer.\n\nParameters:\n\n  * `inputNetwork` - input module.\n  * `rnn` - recurrent module.\n]]\nfunction Encoder:__init(args, inputNetwork)\n  local RNN = onmt.LSTM\n  if args.rnn_type == 'GRU' then\n    RNN = onmt.GRU\n  end\n\n  local rnn = RNN.new(args.layers, inputNetwork.inputSize, args.rnn_size, args.dropout, args.residual, args.dropout_input)\n\n  self.rnn = rnn\n  self.inputNet = inputNetwork\n\n  self.args = {}\n  self.args.rnnSize = self.rnn.outputSize\n  self.args.numEffectiveLayers = self.rnn.numEffectiveLayers\n\n  parent.__init(self, self:_buildModel())\n\n  self:resetPreallocation()\nend\n\n--[[ Return a new Encoder using the serialized data `pretrained`. ]]\nfunction Encoder.load(pretrained)\n  local self = torch.factory('onmt.Encoder')()\n\n  self.args = pretrained.args\n  parent.__init(self, pretrained.modules[1])\n\n  self:resetPreallocation()\n\n  return self\nend\n\n--[[ Return data to serialize. ]]\nfunction Encoder:serialize()\n  return {\n    name = 'Encoder',\n    modules = self.modules,\n    args = self.args\n  }\nend\n\nfunction Encoder:resetPreallocation()\n  -- Prototype for preallocated hidden and cell states.\n  self.stateProto = torch.Tensor()\n\n  -- Prototype for preallocated output gradients.\n  self.gradOutputProto = torch.Tensor()\n\n  -- Prototype for preallocated context vector.\n  self.contextProto = torch.Tensor()\nend\n\nfunction Encoder:maskPadding()\n  self.maskPad = true\nend\n\n--[[ Build one time-step of an Encoder\n\nReturns: An nn-graph mapping\n\n  $${(c^1_{t-1}, h^1_{t-1}, .., c^L_{t-1}, h^L_{t-1}, x_t) =>\n  (c^1_{t}, h^1_{t}, .., c^L_{t}, h^L_{t})}$$\n\n  Where $$c^l$$ and $$h^l$$ are the hidden and cell states at each layer,\n  $$x_t$$ is a sparse word to lookup.\n--]]\nfunction Encoder:_buildModel()\n  local inputs = {}\n  local states = {}\n\n  -- Inputs are previous layers first.\n  for _ = 1, self.args.numEffectiveLayers do\n    local h0 = nn.Identity()() -- batchSize x rnnSize\n    table.insert(inputs, h0)\n    table.insert(states, h0)\n  end\n\n  -- Input word.\n  local x = nn.Identity()() -- batchSize\n  table.insert(inputs, x)\n\n  -- Compute input network.\n  local input = self.inputNet(x)\n  table.insert(states, input)\n\n  -- Forward states and input into the RNN.\n  local outputs = self.rnn(states)\n  return nn.gModule(inputs, { outputs })\nend\n\n--[[Compute the context representation of an input.\n\nParameters:\n\n  * `batch` - as defined in batch.lua.\n\nReturns:\n\n  1. - final hidden states\n  2. - context matrix H\n--]]\nfunction Encoder:forward(batch, states)\n\n  -- TODO: Change `batch` to `input`.\n\n  local finalStates\n  local outputSize = self.args.rnnSize\n\n  if self.statesProto == nil then\n    self.statesProto = onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,\n                                                         self.stateProto,\n                                                         { batch.size, outputSize })\n  end\n\n  -- Make initial states h_0.\n  states = states or onmt.utils.Tensor.reuseTensorTable(self.statesProto, { batch.size, outputSize })\n\n  -- Preallocated output matrix.\n  local context = onmt.utils.Tensor.reuseTensor(self.contextProto,\n                                                { batch.size, batch.sourceLength, outputSize })\n\n  if self.maskPad and not batch.sourceInputPadLeft then\n    finalStates = onmt.utils.Tensor.recursiveClone(states)\n  end\n  if self.train then\n    self.inputs = {}\n  end\n\n  -- Act like nn.Sequential and call each clone in a feed-forward\n  -- fashion.\n  for t = 1, batch.sourceLength do\n\n    -- Construct \"inputs\". Prev states come first then source.\n    local inputs = {}\n    onmt.utils.Table.append(inputs, states)\n    table.insert(inputs, batch:getSourceInput(t))\n\n    if self.train then\n      -- Remember inputs for the backward pass.\n      self.inputs[t] = inputs\n    end\n    states = self:net(t):forward(inputs)\n\n    -- Make sure it always returns table.\n    if type(states) ~= \"table\" then states = { states } end\n\n    -- Special case padding.\n    if self.maskPad then\n      for b = 1, batch.size do\n        if (batch.sourceInputPadLeft and t <= batch.sourceLength - batch.sourceSize[b])\n        or (not batch.sourceInputPadLeft and t > batch.sourceSize[b]) then\n          for j = 1, #states do\n            states[j][b]:zero()\n          end\n        elseif not batch.sourceInputPadLeft and t == batch.sourceSize[b] then\n          for j = 1, #states do\n            finalStates[j][b]:copy(states[j][b])\n          end\n        end\n      end\n    end\n\n    -- Copy output (h^L_t = states[#states]) to context.\n    context[{{}, t}]:copy(states[#states])\n  end\n\n  if finalStates == nil then\n    finalStates = states\n  end\n\n  return finalStates, context\nend\n\n--[[ Backward pass (only called during training)\n\n  Parameters:\n\n  * `batch` - must be same as for forward\n  * `gradStatesOutput` gradient of loss wrt last state - this can be null if states are not used\n  * `gradContextOutput` - gradient of loss wrt full context.\n\n  Returns: `gradInputs` of input network.\n--]]\nfunction Encoder:backward(batch, gradStatesOutput, gradContextOutput)\n  -- TODO: change this to (input, gradOutput) as in nngraph.\n  local outputSize = self.args.rnnSize\n  if self.gradOutputsProto == nil then\n    self.gradOutputsProto = onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,\n                                                              self.gradOutputProto,\n                                                              { batch.size, outputSize })\n  end\n\n  local gradStatesInput\n  if gradStatesOutput then\n    gradStatesInput = onmt.utils.Tensor.copyTensorTable(self.gradOutputsProto, gradStatesOutput)\n  else\n    -- if gradStatesOutput is not defined - start with empty tensor\n    gradStatesInput = onmt.utils.Tensor.reuseTensorTable(self.gradOutputsProto, { batch.size, outputSize })\n  end\n\n  local gradInputs = {}\n  local gradStates = {}\n\n  for t = batch.sourceLength, 1, -1 do\n    -- Add context gradients to last hidden states gradients.\n    gradStatesInput[#gradStatesInput]:add(gradContextOutput[{{}, t}])\n\n    -- nngraph does not accept table of size 1.\n    local timestepGradOutput = #gradStatesInput > 1 and gradStatesInput or gradStatesInput[1]\n\n    local gradInput = self:net(t):backward(self.inputs[t], timestepGradOutput)\n\n    -- Prepare next Encoder output gradients.\n    for i = 1, #gradStatesInput do\n      gradStatesInput[i]:copy(gradInput[i])\n    end\n\n    if t == 1 then\n      gradStates = gradStatesInput\n    end\n\n    -- Gather gradients of all user inputs.\n    gradInputs[t] = {}\n    for i = #gradStatesInput + 1, #gradInput do\n      table.insert(gradInputs[t], gradInput[i])\n    end\n\n    if #gradInputs[t] == 1 then\n      gradInputs[t] = gradInputs[t][1]\n    end\n  end\n  -- TODO: make these names clearer.\n  -- Useful if input came from another network.\n  return gradInputs, gradStates\n\nend\n"
  },
  {
    "path": "opennmt/modules/FeaturesEmbedding.lua",
    "content": "--[[\n  A nngraph unit that maps features ids to embeddings. When using multiple\n  features this can be the concatenation or the sum of each individual embedding.\n]]\nlocal FeaturesEmbedding, parent = torch.class('onmt.FeaturesEmbedding', 'onmt.Network')\n\nfunction FeaturesEmbedding:__init(vocabSizes, vecSizes, merge)\n  assert(#vocabSizes == #vecSizes)\n\n  if merge == 'sum' then\n    for i = 2, #vecSizes do\n      assert(vecSizes[i] == vecSizes[1], 'embeddings must have the same size when merging with a sum')\n    end\n    self.outputSize = vecSizes[1]\n  else\n    self.outputSize = 0\n    for i = 1, #vecSizes do\n      self.outputSize = self.outputSize + vecSizes[i]\n    end\n  end\n\n  parent.__init(self, self:_buildModel(vocabSizes, vecSizes, merge))\nend\n\nfunction FeaturesEmbedding:_buildModel(vocabSizes, vecSizes, merge)\n  local inputs = {}\n  local outputs = {}\n\n  for i = 1, #vocabSizes do\n    table.insert(inputs, nn.Identity()())\n    table.insert(outputs, nn.LookupTable(vocabSizes[i], vecSizes[i])(inputs[#inputs]))\n  end\n\n  local output\n\n  if #outputs == 1 then\n    output = outputs[1]\n  else\n    if merge == 'sum' then\n      output = nn.CAddTable()(outputs)\n    else\n      output = nn.JoinTable(2, 2)(outputs)\n    end\n  end\n\n  return nn.gModule(inputs, {output})\nend\n"
  },
  {
    "path": "opennmt/modules/FeaturesGenerator.lua",
    "content": "--[[ Feature decoder generator. Given RNN state, produce categorical distribution over\ntokens and features.\n\n  Implements $$[softmax(W^1 h + b^1), softmax(W^2 h + b^2), ..., softmax(W^n h + b^n)] $$.\n--]]\nlocal FeaturesGenerator, parent = torch.class('onmt.FeaturesGenerator', 'onmt.Network')\n\n--[[\nParameters:\n\n  * `rnnSize` - Input rnn size.\n  * `outputSizes` - Table of each output size.\n--]]\nfunction FeaturesGenerator:__init(rnnSize, outputSizes)\n  parent.__init(self, self:_buildGenerator(rnnSize, outputSizes))\nend\n\nfunction FeaturesGenerator:_buildGenerator(rnnSize, outputSizes)\n  local generator = nn.ConcatTable()\n\n  for i = 1, #outputSizes do\n    generator:add(nn.Sequential()\n                    :add(nn.Linear(rnnSize, outputSizes[i]))\n                    :add(nn.LogSoftMax()))\n  end\n\n  return generator\nend\n"
  },
  {
    "path": "opennmt/modules/GRU.lua",
    "content": "require('nngraph')\n\n--[[\nImplementation of a single stacked-GRU step as\nan nn unit.\n\n      h^L_{t-1} --- h^L_t\n                 |\n\n\n                 .\n                 |\n             [dropout]\n                 |\n      h^1_{t-1} --- h^1_t\n                 |\n                 |\n                x_t\n\nComputes $$(h_{t-1}, x_t) => (h_{t})$$.\n\n--]]\nlocal GRU, parent = torch.class('onmt.GRU', 'onmt.Network')\n\n--[[\nParameters:\n\n  * `layers` - Number of layers\n  * `inputSize` - Size of input layer\n  * `hiddenSize` - Size of the hidden layers\n  * `dropout` - Dropout rate to use (in $$[0,1]$$ range).\n  * `residual` - Residual connections between layers (boolean)\n  * `dropout_input` - if true, add a dropout layer on the first layer (useful for instance in complex encoders)\n--]]\nfunction GRU:__init(layers, inputSize, hiddenSize, dropout, residual, dropout_input)\n  dropout = dropout or 0\n\n  self.dropout = dropout\n  self.numEffectiveLayers = layers\n  self.outputSize = hiddenSize\n\n  parent.__init(self, self:_buildModel(layers, inputSize, hiddenSize, dropout, residual, dropout_input))\nend\n\n--[[ Stack the GRU units. ]]\nfunction GRU:_buildModel(layers, inputSize, hiddenSize, dropout, residual, dropout_input)\n  -- inputs: { prevOutput L1, ..., prevOutput Ln, input }\n  -- outputs: { output L1, ..., output Ln }\n\n  local inputs = {}\n  local outputs = {}\n\n  for _ = 1, layers do\n    table.insert(inputs, nn.Identity()()) -- h0: batchSize x hiddenSize\n  end\n\n  table.insert(inputs, nn.Identity()()) -- x: batchSize x inputSize\n  local x = inputs[#inputs]\n\n  local prevInput\n  local nextH\n\n  for L = 1, layers do\n    local input\n    local inputDim\n\n    if L == 1 then\n      -- First layer input is x.\n      input = x\n      inputDim = inputSize\n    else\n      inputDim = hiddenSize\n      input = nextH\n      if residual and (L > 2 or inputSize == hiddenSize) then\n        input = nn.CAddTable()({input, prevInput})\n      end\n    end\n    if dropout > 0 and (dropout_input or L > 1) then\n      input = nn.Dropout(dropout)(input)\n    end\n\n    local prevH = inputs[L]\n\n    nextH = self:_buildLayer(inputDim, hiddenSize)({prevH, input})\n    prevInput = input\n\n    table.insert(outputs, nextH)\n  end\n\n  return nn.gModule(inputs, outputs)\nend\n\n--[[ Build a single GRU unit layer.\n    .. math::\n\n            \\begin{array}{ll}\n            r_t = sigmoid(W_{xr} x_t + b_{xr} + W_{hr} h_{(t-1)} + b_{hr}) \\\\\n            i_t = sigmoid(W_{xi} x_t + b_{xi} + W_hi h_{(t-1)} + b_{hi}) \\\\\n            n_t = \\tanh(W_{xn} x_t + b_{xn} + r_t * (W_{hn} h_{(t-1)} + b_{hn}) \\\\\n            h_t = (1 - i_t) * n_t + i_t * h_{(t-1)} = n_t + i_t * (h_{(t-1) - n}) \\\\\n            \\end{array}\n\n    where $$h_t` is the hidden state at time `t`, $$x_t$$ is the hidden\n    state of the previous layer at time `t` or $$input_t$$ for the first layer,\n    and $$r_t$$, $$i_t$$, $$n_t$$ are the reset, input, and new gates, respectively.\n\n    In the function: `prevH`=$$h_{(t-1}}$$, `nextH`=$$h_t$$, `r`=$$r_t$$, `i`=$$i_t$$,\n    `n`=$$n_t$$.\n\n    `x2h_r,x2h_i,x2h_n` are $$W_{xr},W_{xi},W_{xn}$$ and the biases $$b_{xr},b_{xi},b_{xn}$$.\n\n    And `h2h_r,h2h_i,h2h_n` are $$W_{hr},W_{hi},W_{hn}$$ and the biases $$b_{hr},b_{hi},b_{hn}$$.\n\n\n]]\nfunction GRU:_buildLayer(inputSize, hiddenSize)\n  local inputs = {}\n  table.insert(inputs, nn.Identity()())\n  table.insert(inputs, nn.Identity()())\n\n  -- Recurrent input.\n  local prevH = inputs[1]\n  -- Previous layer input.\n  local x = inputs[2]\n\n  -- Evaluate the input sums at once for efficiency.\n  local x2h = nn.Linear(inputSize, 3 * hiddenSize)(x)\n  local h2h = nn.Linear(hiddenSize, 3 * hiddenSize)(prevH)\n\n  -- Extract Wxr.x+bir, Wxi.x+bxi, Wxn.x+bin.\n  local x2h_reshaped = nn.Reshape(3, hiddenSize)(x2h)\n  local x2h_r, x2h_i, x2h_n = nn.SplitTable(2)(x2h_reshaped):split(3)\n\n  -- Extract Whr.x+bhr, Whi.x+bhi, Whn.x+bhn\n  local h2h_reshaped = nn.Reshape(3, hiddenSize)(h2h)\n  local h2h_r, h2h_i, h2h_n = nn.SplitTable(2)(h2h_reshaped):split(3)\n\n  -- Decode the gates.\n  local r = nn.Sigmoid()(nn.CAddTable()({x2h_r, h2h_r}))\n  local i = nn.Sigmoid()(nn.CAddTable()({x2h_i, h2h_i}))\n  local n = nn.Tanh()(nn.CAddTable()({\n    x2h_n, nn.CMulTable()({r, h2h_n})\n  }))\n\n  -- Perform the GRU update.\n  local nextH = nn.CAddTable()({\n    n,\n    nn.CMulTable()({i, nn.CAddTable()({prevH, nn.MulConstant(-1)(n)})})\n  })\n\n  return nn.gModule(inputs, {nextH})\nend\n"
  },
  {
    "path": "opennmt/modules/Generator.lua",
    "content": "--[[ Default decoder generator. Given RNN state, produce categorical distribution.\n\nSimply implements $$softmax(W h + b)$$.\n--]]\nlocal Generator, parent = torch.class('onmt.Generator', 'onmt.Network')\n\n\nfunction Generator:__init(rnnSize, outputSize)\n  parent.__init(self, self:_buildGenerator(rnnSize, outputSize))\nend\n\nfunction Generator:_buildGenerator(rnnSize, outputSize)\n  return nn.Sequential()\n    :add(nn.Linear(rnnSize, outputSize))\n    :add(nn:LogSoftMax())\nend\n\nfunction Generator:updateOutput(input)\n  self.output = {self.net:updateOutput(input)}\n  return self.output\nend\n\nfunction Generator:updateGradInput(input, gradOutput)\n  self.gradInput = self.net:updateGradInput(input, gradOutput[1])\n  return self.gradInput\nend\n\nfunction Generator:accGradParameters(input, gradOutput, scale)\n  self.net:accGradParameters(input, gradOutput[1], scale)\nend\n"
  },
  {
    "path": "opennmt/modules/GlobalAttention.lua",
    "content": "require('nngraph')\n\n--[[ Global attention takes a matrix and a query vector. It\nthen computes a parameterized convex combination of the matrix\nbased on the input query.\n\n\n    H_1 H_2 H_3 ... H_n\n     q   q   q       q\n      |  |   |       |\n       \\ |   |      /\n           .....\n         \\   |  /\n             a\n\nConstructs a unit mapping:\n  $$(H_1 .. H_n, q) => (a)$$\n  Where H is of `batch x n x dim` and q is of `batch x dim`.\n\n  The full function is  $$\\tanh(W_2 [(softmax((W_1 q + b_1) H) H), q] + b_2)$$.\n\n  * dot: $$score(h_t,{\\overline{h}}_s) = h_t^T{\\overline{h}}_s$$\n  * general: $$score(h_t,{\\overline{h}}_s) = h_t^T W_a {\\overline{h}}_s$$\n  * concat: $$score(h_t,{\\overline{h}}_s) = \\nu_a^T tanh(W_a[h_t;{\\overline{h}}_s])$$\n\n--]]\nlocal GlobalAttention, parent = torch.class('onmt.GlobalAttention', 'onmt.Network')\n\nlocal options = {\n  {'-global_attention', 'general',         [[Global attention model type.]],\n        {enum={'general', 'dot', 'concat'}}}\n}\n\nfunction GlobalAttention.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, 'Global Attention Model')\nend\n\n--[[A nn-style module computing attention.\n\n  Parameters:\n\n  * `dim` - dimension of the context vectors.\n--]]\nfunction GlobalAttention:__init(opt, dim)\n  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(opt, options)\n  parent.__init(self, self:_buildModel(dim, self.args.global_attention))\nend\n\nfunction GlobalAttention:_buildModel(dim, global_attention)\n  local inputs = {}\n  table.insert(inputs, nn.Identity()())\n  table.insert(inputs, nn.Identity()())\n\n  local ht = inputs[1]\n  local context = inputs[2] -- batchL x sourceTimesteps x dim\n\n  -- Get attention.\n  local score_ht_hs\n  if global_attention ~= 'concat' then\n    if global_attention == 'general' then\n      ht = nn.Linear(dim, dim, false)(ht) -- batchL x dim\n    end\n    score_ht_hs = nn.MM()({context, nn.Replicate(1,3)(ht)}) -- batchL x sourceL x 1\n  else\n    local ht2 = nn.Replicate(1,2)(ht) -- batchL x 1 x dim\n    local ht_hs = onmt.JoinReplicateTable(2,3)({ht2, context})\n    local Wa_ht_hs = nn.Bottle(nn.Linear(dim*2, dim, false),2)(ht_hs)\n    local tanh_Wa_ht_hs = nn.Tanh()(Wa_ht_hs)\n    score_ht_hs = nn.Bottle(nn.Linear(dim,1),2)(tanh_Wa_ht_hs)\n  end\n  local attn = nn.Sum(3)(score_ht_hs) -- batchL x sourceL\n  local softmaxAttn = nn.SoftMax()\n  softmaxAttn.name = 'softmaxAttn'\n  attn = softmaxAttn(attn)\n  attn = nn.Replicate(1,2)(attn) -- batchL x 1 x sourceL\n\n  -- Apply attention to context.\n  local contextCombined = nn.MM()({attn, context}) -- batchL x 1 x dim\n  contextCombined = nn.Sum(2)(contextCombined) -- batchL x dim\n  contextCombined = nn.JoinTable(2)({contextCombined, inputs[1]}) -- batchL x dim*2\n  local contextOutput = nn.Tanh()(nn.Linear(dim*2, dim, false)(contextCombined))\n\n  return nn.gModule(inputs, {contextOutput})\nend\n"
  },
  {
    "path": "opennmt/modules/JoinReplicateTable.lua",
    "content": "local JoinReplicateTable, parent = torch.class('onmt.JoinReplicateTable', 'nn.Module')\n\nfunction JoinReplicateTable:__init(dimensionReplicate, dimensionJoin, nInputDims)\n  parent.__init(self)\n  self.size = torch.LongStorage()\n  self.dimensionReplicate = dimensionReplicate\n  self.dimensionJoin = dimensionJoin\n  self.gradInput = {}\n  self.nInputDims = nInputDims\nend\n\nfunction JoinReplicateTable:_getPositiveDimensions(input)\n  local dimensionReplicate = self.dimensionReplicate\n  local dimensionJoin = self.dimensionJoin\n  if dimensionReplicate < 0 then\n    dimensionReplicate = input[1]:dim() + dimensionReplicate + 1\n  elseif self.nInputDims and input[1]:dim()==(self.nInputDims+1) then\n    dimensionReplicate = dimensionReplicate + 1\n  end\n  if dimensionJoin < 0 then\n    dimensionJoin = input[1]:dim() + dimensionJoin + 1\n  elseif self.nInputDims and input[1]:dim()==(self.nInputDims+1) then\n    dimensionJoin = dimensionJoin + 1\n  end\n  return dimensionReplicate, dimensionJoin\nend\n\nfunction JoinReplicateTable:_replicate(input, dim, dryRun)\n  -- first replicate along dimensionReplicate, we assert all dimensions are the same but some can be one\n  local maxSize = 0\n  local hasOne = {}\n\n  for i = 1, #input do\n    local size = input[i]:size(dim)\n    assert(maxSize == 0 or size == maxSize or size == 1 or maxSize == 1, \"incorrect tensor size for replicate dimension\")\n    if size > maxSize then\n      maxSize = size\n    end\n    if size == 1 then\n      table.insert(hasOne, i)\n    end\n  end\n\n  -- remember strides to restore after joining operation\n  local memStrides = {}\n  if maxSize > 1 and #hasOne > 0 then\n    for i = 1, #hasOne do\n      local idx = hasOne[i]\n      local sz = input[idx]:size()\n      sz[dim] = maxSize\n      local st = input[idx]:stride()\n      memStrides[idx] = st[dim]\n      if not dryRun then\n        st[dim] = 0\n        input[idx]:set(input[idx]:storage(), input[idx]:storageOffset(), sz, st)\n      end\n    end\n  end\n\n  return memStrides\nend\n\nfunction JoinReplicateTable:_dereplicate(input, dim, memStrides)\n  for idx, stval in ipairs(memStrides) do\n    local sz = input[idx]:size()\n    sz[dim] = 1\n    local st = input[idx]:stride()\n    st[dim] = stval\n    input[idx]:set(input[idx]:storage(), input[idx]:storageOffset(), sz, st)\n  end\nend\n\nfunction JoinReplicateTable:updateOutput(input)\n  local dimensionReplicate, dimensionJoin = self:_getPositiveDimensions(input)\n\n  local memStrides = self:_replicate(input, dimensionReplicate)\n\n  for i=1,#input do\n    local currentOutput = input[i]\n    if i == 1 then\n      self.size:resize(currentOutput:dim()):copy(currentOutput:size())\n    else\n      self.size[dimensionJoin] = self.size[dimensionJoin]\n        + currentOutput:size(dimensionJoin)\n    end\n  end\n  self.output:resize(self.size)\n\n  local offset = 1\n  for i=1,#input do\n    local currentOutput = input[i]\n    self.output:narrow(dimensionJoin, offset,\n      currentOutput:size(dimensionJoin)):copy(currentOutput)\n    offset = offset + currentOutput:size(dimensionJoin)\n  end\n\n  self:_dereplicate(input, dimensionReplicate, memStrides)\n\n  return self.output\nend\n\nfunction JoinReplicateTable:updateGradInput(input, gradOutput)\n  local dimensionReplicate, dimensionJoin = self:_getPositiveDimensions(input)\n\n  local memStrides = self:_replicate(input, dimensionReplicate, true)\n\n  for i=1,#input do\n    if self.gradInput[i] == nil then\n      self.gradInput[i] = input[i].new()\n    end\n    self.gradInput[i]:resizeAs(input[i])\n  end\n\n  -- clear out invalid gradInputs\n  for i=#input+1, #self.gradInput do\n    self.gradInput[i] = nil\n  end\n\n  local offset = 1\n  for i=1,#input do\n    local currentOutput = input[i]\n    local currentGradInput = gradOutput:narrow(dimensionJoin, offset,\n               currentOutput:size(dimensionJoin))\n    if memStrides[i] then\n      -- sum along the replicated dimension\n      self.gradInput[i]:copy(currentGradInput:sum(dimensionReplicate))\n    else\n      self.gradInput[i]:copy(currentGradInput)\n    end\n\n    offset = offset + currentOutput:size(dimensionJoin)\n  end\n\n  return self.gradInput\nend\n\nfunction JoinReplicateTable:type(type, tensorCache)\n  self.gradInput = {}\n  return parent.type(self, type, tensorCache)\nend\n"
  },
  {
    "path": "opennmt/modules/LSTM.lua",
    "content": "require('nngraph')\n\n--[[\nImplementation of a single stacked-LSTM step as\nan nn unit.\n\n      h^L_{t-1} --- h^L_t\n      c^L_{t-1} --- c^L_t\n                 |\n\n\n                 .\n                 |\n             [dropout]\n                 |\n      h^1_{t-1} --- h^1_t\n      c^1_{t-1} --- c^1_t\n                 |\n                 |\n                x_t\n\nComputes $$(c_{t-1}, h_{t-1}, x_t) => (c_{t}, h_{t})$$.\n\n--]]\nlocal LSTM, parent = torch.class('onmt.LSTM', 'onmt.Network')\n\n--[[\nParameters:\n\n  * `layers` - Number of LSTM layers, L.\n  * `inputSize` - Size of input layer\n  * `hiddenSize` - Size of the hidden layers.\n  * `dropout` - Dropout rate to use (in $$[0,1]$$ range).\n  * `residual` - Residual connections between layers.\n  * `dropout_input` - if true, add a dropout layer on the first layer (useful for instance in complex encoders)\n--]]\nfunction LSTM:__init(layers, inputSize, hiddenSize, dropout, residual, dropout_input)\n  dropout = dropout or 0\n\n  self.dropout = dropout\n  self.numEffectiveLayers = 2 * layers\n  self.outputSize = hiddenSize\n\n  parent.__init(self, self:_buildModel(layers, inputSize, hiddenSize, dropout, residual, dropout_input))\nend\n\n--[[ Stack the LSTM units. ]]\nfunction LSTM:_buildModel(layers, inputSize, hiddenSize, dropout, residual, dropout_input)\n  local inputs = {}\n  local outputs = {}\n\n  for _ = 1, layers do\n    table.insert(inputs, nn.Identity()()) -- c0: batchSize x hiddenSize\n    table.insert(inputs, nn.Identity()()) -- h0: batchSize x hiddenSize\n  end\n\n  table.insert(inputs, nn.Identity()()) -- x: batchSize x inputSize\n  local x = inputs[#inputs]\n\n  local prevInput\n  local nextC\n  local nextH\n\n  for L = 1, layers do\n    local input\n    local inputDim\n\n    if L == 1 then\n      -- First layer input is x.\n      input = x\n      inputDim = inputSize\n    else\n      inputDim = hiddenSize\n      input = nextH\n      if residual and (L > 2 or inputSize == hiddenSize) then\n        input = nn.CAddTable()({input, prevInput})\n      end\n    end\n    if dropout > 0 and (dropout_input or L > 1) then\n      input = nn.Dropout(dropout)(input)\n    end\n\n    local prevC = inputs[L*2 - 1]\n    local prevH = inputs[L*2]\n\n    nextC, nextH = self:_buildLayer(inputDim, hiddenSize)({prevC, prevH, input}):split(2)\n    prevInput = input\n\n    table.insert(outputs, nextC)\n    table.insert(outputs, nextH)\n  end\n\n  return nn.gModule(inputs, outputs)\nend\n\n--[[ Build a single LSTM unit layer. ]]\nfunction LSTM:_buildLayer(inputSize, hiddenSize)\n  local inputs = {}\n  table.insert(inputs, nn.Identity()())\n  table.insert(inputs, nn.Identity()())\n  table.insert(inputs, nn.Identity()())\n\n  local prevC = inputs[1]\n  local prevH = inputs[2]\n  local x = inputs[3]\n\n  -- Evaluate the input sums at once for efficiency.\n  local i2h = nn.Linear(inputSize, 4 * hiddenSize)(x)\n  local h2h = nn.Linear(hiddenSize, 4 * hiddenSize)(prevH)\n  local allInputSums = nn.CAddTable()({i2h, h2h})\n\n  local reshaped = nn.Reshape(4, hiddenSize)(allInputSums)\n  local n1, n2, n3, n4 = nn.SplitTable(2)(reshaped):split(4)\n\n  -- Decode the gates.\n  local inGate = nn.Sigmoid()(n1)\n  local forgetGate = nn.Sigmoid()(n2)\n  local outGate = nn.Sigmoid()(n3)\n\n  -- Decode the write inputs.\n  local inTransform = nn.Tanh()(n4)\n\n  -- Perform the LSTM update.\n  local nextC = nn.CAddTable()({\n    nn.CMulTable()({forgetGate, prevC}),\n    nn.CMulTable()({inGate, inTransform})\n  })\n\n  -- Gated cells form the output.\n  local nextH = nn.CMulTable()({outGate, nn.Tanh()(nextC)})\n\n  return nn.gModule(inputs, {nextC, nextH})\nend\n"
  },
  {
    "path": "opennmt/modules/MaskedSoftmax.lua",
    "content": "require('nngraph')\n\n--[[ A batched-softmax wrapper to mask the probabilities of padding.\n\n  For instance there may be a batch of instances where A is padding.\n\n    XXXXAA\n    XXAAAA\n    XXXXXX\n\n  MaskedSoftmax ensures that no probability is given to the A's.\n\n  For this example, `sourceSizes` is {4, 2, 6} and `sourceLength` is 6.\n--]]\nlocal MaskedSoftmax, parent = torch.class('onmt.MaskedSoftmax', 'onmt.Network')\n\n\n--[[ A nn-style module that applies a softmax on input that gives no weight to the left padding.\n\nParameters:\n\n  * `sourceSizes` -  the true lengths (with left padding).\n  * `sourceLength` - the length of the batch.\n--]]\nfunction MaskedSoftmax:__init(sourceSizes, sourceLength)\n  parent.__init(self, self:_buildModel(sourceSizes, sourceLength))\nend\n\nfunction MaskedSoftmax:_buildModel(sourceSizes, sourceLength)\n  local numSents = sourceSizes:size(1)\n  local input = nn.Identity()()\n  local softmax = nn.SoftMax()(input) -- numSents x State.sourceLength\n\n  -- Now we are masking the part of the output we don't need\n  local tab = nn.SplitTable(1)(softmax) -- numSents x { State.sourceLength }\n  local par = nn.ParallelTable()\n\n  for b = 1, numSents do\n    local padLength = sourceLength - sourceSizes[b]\n\n    local seq = nn.Sequential()\n    seq:add(nn.Narrow(1, padLength + 1, sourceSizes[b]))\n    seq:add(nn.Padding(1, -padLength, 1, 0))\n    par:add(seq)\n  end\n\n  local outTab = par(tab) -- numSents x { State.sourceLength }\n  local output = nn.JoinTable(1)(outTab) -- numSents x State.sourceLength\n  output = nn.View(numSents, sourceLength)(output)\n\n  -- Make sure the vector sums to 1 (softmax output)\n  output = nn.Normalize(1)(output)\n\n  return nn.gModule({input}, {output})\nend\n"
  },
  {
    "path": "opennmt/modules/Network.lua",
    "content": "--[[ Wrapper around a single network. ]]\nlocal Network, parent = torch.class('onmt.Network', 'nn.Container')\n\nfunction Network:__init(net)\n  parent.__init(self)\n  self.net = net\n  self:add(net)\nend\n\nfunction Network:updateOutput(input)\n  self.output = self.net:updateOutput(input)\n  return self.output\nend\n\nfunction Network:updateGradInput(input, gradOutput)\n  self.gradInput = self.net:updateGradInput(input, gradOutput)\n  return self.gradInput\nend\n\nfunction Network:accGradParameters(input, gradOutput, scale)\n  self.net:accGradParameters(input, gradOutput, scale)\nend\n"
  },
  {
    "path": "opennmt/modules/NoAttention.lua",
    "content": "require('nngraph')\n\n--[[ No attention module\n\n--]]\nlocal NoAttention, parent = torch.class('onmt.NoAttention', 'onmt.Network')\n\n--[[A nn-style module computing attention.\n\n  Parameters:\n\n  * `dim` - dimension of the context vectors.\n--]]\nfunction NoAttention:__init(_, dim)\n  parent.__init(self, self:_buildModel(dim))\nend\n\nfunction NoAttention:_buildModel(dim)\n  local inputs = {}\n  table.insert(inputs, nn.Identity()())\n  table.insert(inputs, nn.Identity()())\n\n  local lastContext = nn.Select(2,-1)(inputs[2])\n  local contextCombined = nn.JoinTable(2)({lastContext, inputs[1]}) -- batchL x dim*2\n  local contextOutput = nn.Tanh()(nn.Linear(dim*2, dim, false)(contextCombined))\n\n  return nn.gModule(inputs, {contextOutput})\nend\n"
  },
  {
    "path": "opennmt/modules/PDBiEncoder.lua",
    "content": "--[[ PDBiEncoder is a pyramidal deep bidirectional Sequencer used for the source language.\n\n\n--]]\nlocal PDBiEncoder, parent = torch.class('onmt.PDBiEncoder', 'nn.Container')\n\nlocal options = {\n  {\n    '-pdbrnn_reduction', 2,\n    [[Time-reduction factor at each layer.]],\n    {\n      structural = 0\n    }\n  }\n}\n\nfunction PDBiEncoder.declareOpts(cmd)\n  onmt.BiEncoder.declareOpts(cmd)\n  cmd:setCmdLineOptions(options)\nend\n\n\n--[[ Create a pyrimal deep bidirectional encoder - each layers reconnect before starting another bidirectional layer\n\nParameters:\n\n  * `args` - global arguments\n  * `input` - input neural network.\n]]\nfunction PDBiEncoder:__init(args, input)\n  parent.__init(self)\n\n  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)\n  self.args.layers = args.layers\n  self.args.dropout = args.dropout\n  local dropout_input = args.dropout_input\n\n  self.layers = {}\n\n  args.layers = 1\n  args.brnn_merge = 'sum'\n  self.args.multiplier = 1\n  for _ = 1,self.args.layers do\n    table.insert(self.layers, onmt.BiEncoder(args, input))\n    local identity = nn.Identity()\n    identity.inputSize = args.rnn_size\n    input = identity\n    self:add(self.layers[#self.layers])\n    if #self.layers ~= 1 then\n      self.args.multiplier = self.args.multiplier * self.args.pdbrnn_reduction\n    else\n      -- trick to force a dropout on each layer L > 1\n      if args.dropout > 0 then\n        args.dropout_input = true\n      end\n    end\n  end\n  args.layers = self.args.layers\n  args.dropout_input = dropout_input\n  self.args.numEffectiveLayers = self.layers[1].args.numEffectiveLayers * self.args.layers\n  self.args.hiddenSize = args.rnn_size\n\n  self:resetPreallocation()\nend\n\n--[[ Return a new PDBiEncoder using the serialized data `pretrained`. ]]\nfunction PDBiEncoder.load(pretrained)\n  local self = torch.factory('onmt.PDBiEncoder')()\n  parent.__init(self)\n\n  self.layers = {}\n\n  for i=1, #pretrained.layers do\n    self.layers[i] = onmt.BiEncoder.load(pretrained.layers[i])\n    self:add(self.layers[i])\n  end\n\n  self.args = pretrained.args\n\n  self:resetPreallocation()\n  return self\nend\n\n--[[ Return data to serialize. ]]\nfunction PDBiEncoder:serialize()\n  local layersData = {}\n  for i = 1, #self.layers do\n    table.insert(layersData, self.layers[i]:serialize())\n  end\n\n  return {\n    name = 'PDBiEncoder',\n    layers = layersData,\n    args = self.args\n  }\nend\n\nfunction PDBiEncoder:resetPreallocation()\n  -- Prototype for preallocated full context vector.\n  self.contextProto = torch.Tensor()\n\n  -- Prototype for preallocated full hidden states tensors.\n  self.stateProto = torch.Tensor()\n\n  -- Prototype for preallocated gradient context\n  self.gradContextProto = torch.Tensor()\nend\n\nfunction PDBiEncoder:maskPadding()\n  self.layers[1]:maskPadding()\nend\n\nfunction PDBiEncoder:forward(batch)\n  -- adjust batch length so that it can be divided\n  local batch_length = batch.sourceLength\n  batch_length = math.ceil(batch_length/self.args.multiplier)*self.args.multiplier\n  batch.sourceLength = batch_length\n\n  if self.statesProto == nil then\n    self.statesProto = onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,\n                                                         self.stateProto,\n                                                         { batch.size, self.args.hiddenSize })\n  end\n\n  local states = onmt.utils.Tensor.reuseTensorTable(self.statesProto, { batch.size, self.args.hiddenSize })\n  local context\n\n  local stateIdx = 1\n  self.inputs = { batch }\n  self.lranges = {}\n  for i = 1,#self.layers do\n    local layerStates, layerContext = self.layers[i]:forward(self.inputs[i])\n    if i ~= #self.layers then\n      -- compress the layer Context along time dimension\n      local storageOffset = layerContext:storageOffset()\n      local strideReduced = layerContext:stride()\n      strideReduced[2] = strideReduced[2] * self.args.pdbrnn_reduction\n      local sizeReduced = layerContext:size()\n      sizeReduced[2] = math.floor(sizeReduced[2] / self.args.pdbrnn_reduction)\n      local reducedContext = layerContext\n      reducedContext:set(layerContext:storage(), storageOffset, sizeReduced, strideReduced)\n      for j = 1, self.args.pdbrnn_reduction-1 do\n        local to_add = layerContext\n        to_add:set(layerContext:storage(), storageOffset+j, sizeReduced, strideReduced)\n        reducedContext:add(to_add)\n      end\n      table.insert(self.inputs, onmt.data.BatchTensor.new(reducedContext))\n      -- record what is the size of the last reduction\n      batch.encoderOutputLength = sizeReduced[2]\n    else\n      context = onmt.utils.Tensor.reuseTensor(self.contextProto, layerContext:size())\n      context:copy(layerContext)\n    end\n    table.insert(self.lranges, {stateIdx, #layerStates})\n    for j = 1,#layerStates do\n      states[stateIdx]:copy(layerStates[j])\n      stateIdx = stateIdx + 1\n    end\n  end\n  return states, context\nend\n\nfunction PDBiEncoder:backward(batch, gradStatesOutput, gradContextOutput)\n  local gradInputs\n\n  for i = #self.layers, 1, -1 do\n    local lrange_gradStatesOutput\n    if gradStatesOutput then\n      lrange_gradStatesOutput = onmt.utils.Table.subrange(gradStatesOutput, self.lranges[i][1], self.lranges[i][2])\n    end\n    gradInputs = self.layers[i]:backward(self.inputs[i], lrange_gradStatesOutput, gradContextOutput)\n    if i ~= 1 then\n      gradContextOutput = onmt.utils.Tensor.reuseTensor(self.gradContextProto,\n                                              { batch.size, #gradInputs*self.args.pdbrnn_reduction, self.args.hiddenSize })\n      for t = 1, #gradInputs do\n        for j = 1, self.args.pdbrnn_reduction do\n          gradContextOutput[{{},self.args.pdbrnn_reduction*(t-1)+j,{}}]:copy(gradInputs[t])\n        end\n      end\n    end\n  end\n\n  return gradInputs\nend\n"
  },
  {
    "path": "opennmt/modules/ParallelClassNLLCriterion.lua",
    "content": "--[[\n  Define parallel ClassNLLCriterion.\n--]]\nlocal ParallelClassNLLCriterion, parent = torch.class('onmt.ParallelClassNLLCriterion', 'nn.ParallelCriterion')\n\nfunction ParallelClassNLLCriterion:__init(outputSizes)\n  parent.__init(self, false)\n\n  for i = 1, #outputSizes do\n    self:_addCriterion(outputSizes[i])\n  end\nend\n\nfunction ParallelClassNLLCriterion:_addCriterion(size)\n  -- Ignores padding value.\n  local w = torch.ones(size)\n  w[onmt.Constants.PAD] = 0\n\n  local nll = nn.ClassNLLCriterion(w)\n\n  -- Let the training code manage loss normalization.\n  nll.sizeAverage = false\n  self:add(nll)\nend\n"
  },
  {
    "path": "opennmt/modules/Sequencer.lua",
    "content": "require('nngraph')\n\n--[[ Sequencer is the base class for encoder and decoder models.\n  Main task is to manage `self.net(t)`, the unrolled network\n  used during training.\n\n     :net(1) => :net(2) => ... => :net(n-1) => :net(n)\n\n--]]\nlocal Sequencer, parent = torch.class('onmt.Sequencer', 'nn.Container')\n\n--[[\nParameters:\n\n  * `network` - recurrent step template.\n--]]\nfunction Sequencer:__init(network)\n  parent.__init(self)\n\n  self.network = network\n  self:add(self.network)\n\n  self.networkClones = {}\nend\n\nfunction Sequencer:_sharedClone()\n  local clone = self.network:clone('weight', 'gradWeight', 'bias', 'gradBias')\n\n  -- Share intermediate tensors if defined.\n  if self.networkClones[1] then\n    local sharedTensors = {}\n\n    self.networkClones[1]:apply(function(m)\n      if m.gradInputSharedIdx then\n        sharedTensors[m.gradInputSharedIdx] = m.gradInput\n      end\n      if m.outputSharedIdx then\n        sharedTensors[m.outputSharedIdx] = m.output\n      end\n    end)\n\n    clone:apply(function(m)\n      if m.gradInputSharedIdx then\n        m.gradInput = sharedTensors[m.gradInputSharedIdx]\n      end\n      if m.outputSharedIdx then\n        m.output = sharedTensors[m.outputSharedIdx]\n      end\n    end)\n  end\n\n  collectgarbage()\n\n  return clone\nend\n\n--[[Get access to the recurrent unit at a timestep.\n\nParameters:\n  * `t` - timestep.\n\nReturns: The raw network clone at timestep t.\n  When `evaluate()` has been called, cheat and return t=1.\n]]\nfunction Sequencer:net(t)\n  if self.train and t then\n    -- In train mode, the network has to be cloned to remember intermediate\n    -- outputs for each timestep and to allow backpropagation through time.\n    while #self.networkClones < t do\n      table.insert(self.networkClones, self:_sharedClone())\n    end\n    return self.networkClones[t]\n  elseif #self.networkClones > 0 then\n    return self.networkClones[1]\n  else\n    return self.network\n  end\nend\n\n--[[Return the id of the clone to use for timestep t or 0 if not using clones]]\nfunction Sequencer:cloneId(t)\n  if self.train and t then\n    return t\n  elseif #self.networkClones > 0 then\n    return 1\n  else\n    return 0\n  end\nend\n\n\n--[[ Move the network to train mode. ]]\nfunction Sequencer:training()\n  parent.training(self)\n\n  if #self.networkClones > 0 then\n    -- Only first clone was used for evaluation.\n    self.networkClones[1]:training()\n  end\nend\n\n--[[ Move the network to evaluation mode. ]]\nfunction Sequencer:evaluate()\n  parent.evaluate(self)\n\n  if #self.networkClones > 0 then\n    -- We only use the first clone for evaluation.\n    self.networkClones[1]:evaluate()\n  end\nend\n"
  },
  {
    "path": "opennmt/modules/WordEmbedding.lua",
    "content": "--[[ nn unit. Maps from word ids to embeddings. Slim wrapper around\nnn.LookupTable to allow fixed and pretrained embeddings.\n--]]\nlocal WordEmbedding, parent = torch.class('onmt.WordEmbedding', 'onmt.Network')\n\n--[[\nParameters:\n\n  * `vocabSize` - size of the vocabulary\n  * `vecSize` - size of the embedding\n  * `preTrainined` - path to a pretrained vector file\n  * `fix` - keep the weights of the embeddings fixed.\n--]]\nfunction WordEmbedding:__init(vocabSize, vecSize, preTrained, fix)\n  parent.__init(self, nn.LookupTable(vocabSize, vecSize, onmt.Constants.PAD))\n\n  self.preTrained = preTrained\n  if fix then\n    self.net.gradWeight = nil\n  end\nend\n\nfunction WordEmbedding:postParametersInitialization()\n  -- If embeddings are given. Initialize them.\n  if self.preTrained and self.preTrained:len() > 0 then\n    local vecs = torch.load(self.preTrained)\n    self.net.weight:copy(vecs)\n  end\n\n  self.net.weight[onmt.Constants.PAD]:zero()\nend\n\nfunction WordEmbedding:fixEmbeddings(fix)\n  if fix and self.net.gradWeight then\n    self.net.gradWeight = nil\n  elseif not fix and not self.net.gradWeight then\n    self.net.gradWeight = self.net.weight.new(self.net.weight:size()):zero()\n  end\nend\n\nfunction WordEmbedding:accGradParameters(input, gradOutput, scale)\n  if self.net.gradWeight then\n    self.net:accGradParameters(input, gradOutput, scale)\n    self.net.gradWeight[onmt.Constants.PAD]:zero()\n  end\nend\n\nfunction WordEmbedding:parameters()\n  if self.net.gradWeight then\n    return parent.parameters(self)\n  end\nend\n"
  },
  {
    "path": "opennmt/modules/init.lua",
    "content": "onmt = onmt or {}\n\nrequire('opennmt.modules.Sequencer')\nrequire('opennmt.modules.Encoder')\nrequire('opennmt.modules.BiEncoder')\nrequire('opennmt.modules.DBiEncoder')\nrequire('opennmt.modules.PDBiEncoder')\nrequire('opennmt.modules.Decoder')\n\nrequire('opennmt.modules.Network')\n\nrequire('opennmt.modules.GRU')\nrequire('opennmt.modules.LSTM')\n\nrequire('opennmt.modules.MaskedSoftmax')\nrequire('opennmt.modules.WordEmbedding')\nrequire('opennmt.modules.FeaturesEmbedding')\n\nrequire('opennmt.modules.NoAttention')\nrequire('opennmt.modules.GlobalAttention')\n\nrequire('opennmt.modules.Generator')\nrequire('opennmt.modules.FeaturesGenerator')\n\nrequire('opennmt.modules.JoinReplicateTable')\nrequire('opennmt.modules.ParallelClassNLLCriterion')\n\nreturn onmt\n"
  },
  {
    "path": "opennmt/tagger/Tagger.lua",
    "content": "local Tagger = torch.class('Tagger')\n\nlocal options = {\n  {\n    '-model', '',\n    [[Path to the serialized model file.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.nonEmpty\n    }\n  },\n  {\n    '-batch_size', 30,\n    [[Batch size.]]\n  }\n}\n\nfunction Tagger.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, 'Tagger')\nend\n\nfunction Tagger:__init(args)\n  self.opt = args\n\n  _G.logger:info('Loading \\'' .. self.opt.model .. '\\'...')\n  self.checkpoint = torch.load(self.opt.model)\n\n  if not self.checkpoint.options.model_type or self.checkpoint.options.model_type ~= 'seqtagger' then\n    _G.logger:error('Tagger can only process seqtagger models')\n    os.exit(0)\n  end\n\n  self.model = onmt.SeqTagger.load(self.checkpoint.options, self.checkpoint.models, self.checkpoint.dicts)\n  onmt.utils.Cuda.convert(self.model)\n\n  self.dicts = self.checkpoint.dicts\nend\n\nfunction Tagger:buildInput(tokens)\n  local words, features = onmt.utils.Features.extract(tokens)\n\n  local data = {}\n  data.words = words\n\n  if #features > 0 then\n    data.features = features\n  end\n\n  return data\nend\n\nfunction Tagger:buildOutput(data)\n  return table.concat(onmt.utils.Features.annotate(data.words, data.features), ' ')\nend\n\nfunction Tagger:buildData(src)\n  local srcData = {}\n  srcData.words = {}\n  srcData.features = {}\n\n  local ignored = {}\n  local indexMap = {}\n  local index = 1\n\n  for b = 1, #src do\n    if #src[b].words == 0 then\n      table.insert(ignored, b)\n    else\n      indexMap[index] = b\n      index = index + 1\n\n      table.insert(srcData.words,\n                   self.dicts.src.words:convertToIdx(src[b].words, onmt.Constants.UNK_WORD))\n\n      if #self.dicts.src.features > 0 then\n        table.insert(srcData.features,\n                     onmt.utils.Features.generateSource(self.dicts.src.features, src[b].features))\n      end\n    end\n  end\n\n  return onmt.data.Dataset.new(srcData), ignored, indexMap\nend\n\nfunction Tagger:buildTargetWords(pred)\n  local tokens = self.dicts.tgt.words:convertToLabels(pred, onmt.Constants.EOS)\n\n  return tokens\nend\n\nfunction Tagger:buildTargetFeatures(predFeats)\n  local numFeatures = #predFeats[1]\n\n  if numFeatures == 0 then\n    return {}\n  end\n\n  local feats = {}\n  for _ = 1, numFeatures do\n    table.insert(feats, {})\n  end\n\n  for i = 1, #predFeats do\n    for j = 1, numFeatures do\n      table.insert(feats[j], self.dicts.tgt.features[j]:lookup(predFeats[i][j]))\n    end\n  end\n\n  return feats\nend\n\nfunction Tagger:tagBatch(batch)\n  self.model.models.encoder:maskPadding()\n\n  local pred = {}\n  local feats = {}\n  for _ = 1, batch.size do\n    table.insert(pred, {})\n    table.insert(feats, {})\n  end\n  local _, context = self.model.models.encoder:forward(batch)\n\n  for t = 1, batch.sourceLength do\n    local out = self.model.models.generator:forward(context:select(2, t))\n    if type(out[1]) == 'table' then\n      out = out[1]\n    end\n    local _, best = out[1]:max(2)\n    for b = 1, batch.size do\n      if t > batch.sourceLength - batch.sourceSize[b] then\n        pred[b][t - batch.sourceLength + batch.sourceSize[b]] = best[b][1]\n        feats[b][t - batch.sourceLength + batch.sourceSize[b]] = {}\n      end\n    end\n    for j = 2, #out do\n      _, best = out[j]:max(2)\n      for b = 1, batch.size do\n        if t > batch.sourceLength - batch.sourceSize[b] then\n          feats[b][t - batch.sourceLength + batch.sourceSize[b]][j - 1] = best[b][1]\n        end\n      end\n    end\n  end\n\n  return pred, feats\nend\n\n--[[ Tag a batch of source sequences.\n\nParameters:\n\n  * `src` - a batch of tables containing:\n    - `words`: the table of source words\n    - `features`: the table of feaures sequences (`src.features[i][j]` is the value of the ith feature of the jth token)\n\nReturns:\n\n  * `results` - a batch of tables containing:\n      - `words`: the table of target words\n      - `features`: the table of target features sequences\n]]\nfunction Tagger:tag(src)\n  local data, ignored = self:buildData(src)\n\n  local results = {}\n\n  if data:batchCount() > 0 then\n    local batch = data:getBatch()\n\n    local pred, predFeats = self:tagBatch(batch)\n\n    for b = 1, batch.size do\n      results[b] = {}\n      results[b].words = self:buildTargetWords(pred[b])\n      results[b].features = self:buildTargetFeatures(predFeats[b])\n    end\n  end\n\n  for i = 1, #ignored do\n    table.insert(results, ignored[i], {})\n  end\n\n  return results\nend\n\nreturn Tagger\n"
  },
  {
    "path": "opennmt/tagger/init.lua",
    "content": "local tagger = {}\n\ntagger.Tagger = require('opennmt.tagger.Tagger')\n\nreturn tagger\n"
  },
  {
    "path": "opennmt/train/Checkpoint.lua",
    "content": "-- Class for saving and loading models during training.\nlocal Checkpoint = torch.class('Checkpoint')\n\nlocal options = {\n  {\n    '-train_from', '',\n    [[Path to a checkpoint.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists\n    }\n  },\n  {\n    '-continue', false,\n    [[If set, continue the training where it left off.]]\n  }\n}\n\nfunction Checkpoint.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, 'Checkpoint')\nend\n\nfunction Checkpoint:__init(opt, model, optim, dicts)\n  self.options = opt\n  self.model = model\n  self.optim = optim\n  self.dicts = dicts\n\n  self.savePath = self.options.save_model\nend\n\nfunction Checkpoint:save(filePath, info)\n  info.learningRate = self.optim:getLearningRate()\n  info.optimStates = self.optim:getStates()\n  info.rngStates = onmt.utils.Cuda.getRNGStates()\n\n  local data = {\n    models = {},\n    options = self.options,\n    info = info,\n    dicts = self.dicts\n  }\n\n  for k, v in pairs(self.model.models) do\n    if v.serialize then\n      data.models[k] = v:serialize()\n    else\n      data.models[k] = v\n    end\n  end\n\n  torch.save(filePath, data)\nend\n\n--[[ Save the model and data in the middle of an epoch sorting the iteration. ]]\nfunction Checkpoint:saveIteration(iteration, epochState, batchOrder, verbose)\n  local info = {}\n  info.iteration = iteration + 1\n  info.epoch = epochState.epoch\n  info.batchOrder = batchOrder\n\n  local filePath = string.format('%s_checkpoint.t7', self.savePath)\n\n  if verbose then\n    _G.logger:info('Saving checkpoint to \\'' .. filePath .. '\\'...')\n  end\n\n  -- Succeed serialization before overriding existing file\n  self:save(filePath .. '.tmp', info)\n  os.rename(filePath .. '.tmp', filePath)\nend\n\nfunction Checkpoint:saveEpoch(validPpl, epochState, verbose)\n  local info = {}\n  info.validPpl = validPpl\n  info.epoch = epochState.epoch + 1\n  info.iteration = 1\n  info.trainTimeInMinute = epochState:getTime() / 60\n\n  local filePath = string.format('%s_epoch%d_%.2f.t7', self.savePath, epochState.epoch, validPpl)\n\n  if verbose then\n    _G.logger:info('Saving checkpoint to \\'' .. filePath .. '\\'...')\n  end\n\n  self:save(filePath, info)\nend\n\nfunction Checkpoint.loadFromCheckpoint(opt)\n  local checkpoint = {}\n  local paramChanges = {}\n\n  if opt.train_from:len() > 0 then\n    _G.logger:info('Loading checkpoint \\'' .. opt.train_from .. '\\'...')\n\n    checkpoint = torch.load(opt.train_from)\n\n    local function restoreOption(name)\n      if checkpoint.options[name] ~= nil then\n        opt[name] = checkpoint.options[name]\n      end\n    end\n\n    if checkpoint.info.rngStates and not (opt._is_default['seed'] or opt.seed == checkpoint.options.seed) then\n      _G.logger:warning('\\'-seed\\' value has changed - ignoring saved rng states')\n      checkpoint.info.rngStates = nil\n    end\n\n    if opt.continue and (not (opt._is_default['learning_rate'] or opt.learning_rate == checkpoint.options.learning_rate) or\n        not (opt._is_default['start_epoch'] or opt.start_epoch == checkpoint.options.start_epoch)) then\n      _G.logger:warning('\\'-continue\\' option is used but learning_rate or start_epoch are set and different than previous epoch. Ignoring \\'-continue\\'')\n      opt.continue = nil\n    end\n\n    -- Reload and check options.\n    for k, v in pairs(opt) do\n      if k:sub(1, 1) ~= '_' then\n\n        if opt.continue and opt._train_state[k] then\n          -- Training states should be retrieved when continuing a training.\n          restoreOption(k)\n        elseif opt._structural[k] or opt._init_only[k] then\n          -- If an option was set by the user, check that we can actually change it.\n          local valueChanged = not opt._is_default[k] and v ~= checkpoint.options[k]\n\n          if valueChanged then\n            if opt._init_only[k] then\n              _G.logger:warning('Cannot change initialization option -%s. Ignoring.', k)\n              restoreOption(k)\n            elseif opt._structural[k] and opt._structural[k] == 0 then\n              _G.logger:warning('Cannot change dynamically option -%s. Ignoring.', k)\n              restoreOption(k)\n            elseif opt._structural[k] and opt._structural[k] == 1 then\n              paramChanges[k] = v\n            end\n          else\n            restoreOption(k)\n          end\n\n        end\n\n      end\n    end\n\n    if opt.continue then\n      -- When continuing, some options are initialized with their last known value.\n      opt.learning_rate = checkpoint.info.learningRate\n      opt.start_epoch = checkpoint.info.epoch\n      opt.start_iteration = checkpoint.info.iteration\n\n      _G.logger:info('Resuming training from epoch ' .. opt.start_epoch\n                         .. ' at iteration ' .. opt.start_iteration .. '...')\n    else\n      -- Otherwise, we can drop previous training information.\n      checkpoint.info.learningRate = nil\n      checkpoint.info.epoch = nil\n      checkpoint.info.iteration = nil\n    end\n  end\n\n  return checkpoint, opt, paramChanges\nend\n\nreturn Checkpoint\n"
  },
  {
    "path": "opennmt/train/EpochState.lua",
    "content": "--[[ Class for managing the training process by logging and storing\n  the state of the current epoch.\n]]\nlocal EpochState = torch.class('EpochState')\n\n--[[ Initialize for epoch `epoch`]]\nfunction EpochState:__init(epoch, startIterations, numIterations, learningRate)\n  self.epoch = epoch\n  self.iterations = startIterations - 1\n  self.numIterations = numIterations\n  self.learningRate = learningRate\n\n  self.globalTimer = torch.Timer()\n\n  self:reset()\nend\n\nfunction EpochState:reset()\n  self.trainLoss = 0\n  self.sourceWords = 0\n  self.targetWords = 0\n\n  self.timer = torch.Timer()\nend\n\n--[[ Update training status. Takes `batch` (described in data.lua) and last loss.]]\nfunction EpochState:update(model, batch, loss)\n  self.iterations = self.iterations + 1\n  self.trainLoss = self.trainLoss + loss\n  self.sourceWords = self.sourceWords + model:getInputLabelsCount(batch)\n  self.targetWords = self.targetWords + model:getOutputLabelsCount(batch)\nend\n\n--[[ Log to status stdout. ]]\nfunction EpochState:log(iteration)\n  local ppl = math.exp(self.trainLoss / self.targetWords)\n  local tokpersec = self.sourceWords / self.timer:time().real\n  _G.logger:info('Epoch %d ; Iteration %d/%d ; Learning rate %.4f ; Source tokens/s %d ; Perplexity %.2f',\n                  self.epoch,\n                  iteration or self.iterations, self.numIterations,\n                  self.learningRate,\n                  tokpersec,\n                  ppl)\n  if _G.crayon_logger.on == true then\n     _G.crayon_logger.exp:add_scalar_value(\"learning_rate\", self.learningRate)\n     _G.crayon_logger.exp:add_scalar_value(\"perplexity\", ppl)\n     _G.crayon_logger.exp:add_scalar_value(\"token_per_sec\", tokpersec)\n  end\n  self:reset()\nend\n\nfunction EpochState:getTime()\n  return self.globalTimer:time().real\nend\n\nreturn EpochState\n"
  },
  {
    "path": "opennmt/train/Optim.lua",
    "content": "---------------------------------------------------------------------------------\n-- Local utility functions\n---------------------------------------------------------------------------------\n\nlocal function adagradStep(dfdx, lr, state)\n  if not state.var then\n    state.var = torch.Tensor():typeAs(dfdx):resizeAs(dfdx):zero()\n    state.std = torch.Tensor():typeAs(dfdx):resizeAs(dfdx)\n  end\n\n  state.var:addcmul(1, dfdx, dfdx)\n  state.std:sqrt(state.var)\n  dfdx:cdiv(state.std:add(1e-10)):mul(-lr)\nend\n\nlocal function adamStep(dfdx, lr, state)\n  local beta1 = state.beta1 or 0.9\n  local beta2 = state.beta2 or 0.999\n  local eps = state.eps or 1e-8\n\n  state.t = state.t or 0\n  state.m = state.m or dfdx.new(dfdx:size()):zero()\n  state.v = state.v or dfdx.new(dfdx:size()):zero()\n  state.denom = state.denom or dfdx.new(dfdx:size()):zero()\n\n  state.t = state.t + 1\n  state.m:mul(beta1):add(1-beta1, dfdx)\n  state.v:mul(beta2):addcmul(1-beta2, dfdx, dfdx)\n  state.denom:copy(state.v):sqrt():add(eps)\n\n  local bias1 = 1-beta1^state.t\n  local bias2 = 1-beta2^state.t\n  local stepSize = lr * math.sqrt(bias2)/bias1\n\n  dfdx:copy(state.m):cdiv(state.denom):mul(-stepSize)\nend\n\nlocal function adadeltaStep(dfdx, state)\n  local rho = state.rho or 0.9\n  local eps = state.eps or 1e-6\n  state.var = state.var or dfdx.new(dfdx:size()):zero()\n  state.std = state.std or dfdx.new(dfdx:size()):zero()\n  state.delta = state.delta or dfdx.new(dfdx:size()):zero()\n  state.accDelta = state.accDelta or dfdx.new(dfdx:size()):zero()\n  state.var:mul(rho):addcmul(1-rho, dfdx, dfdx)\n  state.std:copy(state.var):add(eps):sqrt()\n  state.delta:copy(state.accDelta):add(eps):sqrt():cdiv(state.std):cmul(dfdx)\n  dfdx:copy(state.delta):mul(-1)\n  state.accDelta:mul(rho):addcmul(1-rho, state.delta, state.delta)\nend\n\n---------------------------------------------------------------------------------\n\nlocal Optim = torch.class('Optim')\n\nlocal options = {\n  {\n    '-max_batch_size', 64,\n    [[Maximum batch size.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt()\n    }\n  },\n  {\n    '-uneven_batches', false,\n    [[If set, batches are filled up to max_batch_size even if source lengths are different.\n      Slower but needed for some tasks.]]\n  },\n  {\n    '-optim', 'sgd',\n    [[Optimization method.]],\n    {\n      enum = {'sgd', 'adagrad', 'adadelta', 'adam'},\n      train_state = true\n    }\n  },\n  {\n    '-learning_rate', 1,\n    [[Starting learning rate. If adagrad or adam is used, then this is the global learning rate.\n      Recommended settings are: sgd = 1, adagrad = 0.1, adam = 0.0002.]],\n    {\n      train_state = true\n    }\n  },\n  {\n    '-min_learning_rate', 0,\n    [[Do not continue the training past this learning rate value.]],\n    {\n      train_state = true\n    }\n  },\n  {\n    '-max_grad_norm', 50,\n    [[Clip the gradients norm to this value.]],\n    {\n      train_state = true\n    }\n  },\n  {\n    '-learning_rate_decay', 0.5,\n    [[Learning rate decay factor: `learning_rate = learning_rate * learning_rate_decay`.]],\n    {\n      train_state = true\n    }\n  },\n  {\n    '-start_decay_at', 9,\n    [[In \"default\" decay mode, start decay after this epoch.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt(),\n      train_state = true\n    }\n  },\n  {\n    '-start_decay_ppl_delta', 0,\n    [[Start decay when validation perplexity improvement is lower than this value.]],\n    {\n      train_state = true\n    }\n  },\n  {\n    '-decay', 'default',\n    [[When to apply learning rate decay.\n      `default`: decay after each epoch past `-start_decay_at` or as soon as the\n      validation perplexity is not improving more than `-start_decay_ppl_delta`,\n      `perplexity_only`: only decay when validation perplexity is not improving more than\n      `-start_decay_ppl_delta`.]],\n    {\n      enum = {'default', 'perplexity_only'},\n      train_state = true\n    }\n  }\n}\n\nfunction Optim.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, 'Optimization')\nend\n\nfunction Optim:__init(args, optimStates)\n  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)\n  self.valPerf = {}\n\n  if self.args.optim == 'sgd' then\n    self.args.start_decay_at = args.start_decay_at\n  else\n    if optimStates ~= nil then\n      self.optimStates = optimStates\n    else\n      self.optimStates = {}\n    end\n  end\nend\n\nfunction Optim:setOptimStates(num)\n  if self.args.optim ~= 'sgd' then\n    for j = 1, num do\n      self.optimStates[j] = {}\n    end\n  end\nend\n\nfunction Optim:zeroGrad(gradParams)\n  for j = 1, #gradParams do\n    gradParams[j]:zero()\n  end\nend\n\nfunction Optim:prepareGrad(gradParams)\n  -- Compute gradients norm.\n  local gradNorm = 0\n  for j = 1, #gradParams do\n    gradNorm = gradNorm + gradParams[j]:norm()^2\n  end\n  gradNorm = math.sqrt(gradNorm)\n\n  local shrinkage = self.args.max_grad_norm / gradNorm\n\n  for j = 1, #gradParams do\n    -- Shrink gradients if needed.\n    if shrinkage < 1 then\n      gradParams[j]:mul(shrinkage)\n    end\n\n    -- Prepare gradients params according to the optimization method.\n    if self.args.optim == 'adagrad' then\n      adagradStep(gradParams[j], self.args.learning_rate, self.optimStates[j])\n    elseif self.args.optim == 'adadelta' then\n      adadeltaStep(gradParams[j], self.optimStates[j])\n    elseif self.args.optim == 'adam' then\n      adamStep(gradParams[j], self.args.learning_rate, self.optimStates[j])\n    else\n      gradParams[j]:mul(-self.args.learning_rate)\n    end\n  end\nend\n\nfunction Optim:updateParams(params, gradParams)\n  for j = 1, #params do\n    params[j]:add(gradParams[j])\n  end\nend\n\n-- decay learning rate if val perf does not improve or we hit the startDecayAt limit\nfunction Optim:updateLearningRate(score, epoch)\n  local function decayLr()\n    self.args.learning_rate = self.args.learning_rate * self.args.learning_rate_decay\n    self.args.learning_rate = math.max(self.args.learning_rate, self.args.min_learning_rate)\n  end\n  if self.args.optim == 'sgd' then\n    self.valPerf[#self.valPerf + 1] = score\n\n    if epoch >= self.args.start_decay_at then\n      self.startDecay = true\n    end\n\n    local decayConditionMet = false\n\n    if self.valPerf[#self.valPerf] ~= nil and self.valPerf[#self.valPerf-1] ~= nil then\n      local currPpl = self.valPerf[#self.valPerf]\n      local prevPpl = self.valPerf[#self.valPerf-1]\n      if prevPpl - currPpl < self.args.start_decay_ppl_delta then\n        self.startDecay = true\n        decayConditionMet = true\n      end\n    end\n\n    if self.args.decay == 'default' and self.startDecay then\n      decayLr()\n    elseif self.args.decay == 'perplexity_only' and decayConditionMet then\n      decayLr()\n    end\n\n    return self.args.learning_rate >= self.args.min_learning_rate\n  end\n\n  return true\nend\n\nfunction Optim:getLearningRate()\n  return self.args.learning_rate\nend\n\nfunction Optim:getStates()\n  return self.optimStates\nend\n\nreturn Optim\n"
  },
  {
    "path": "opennmt/train/Trainer.lua",
    "content": "---------------------------------------------------------------------------------\n-- Local utility functions\n---------------------------------------------------------------------------------\n\nlocal function eval(model, data)\n  local loss = 0\n  local totalWords = 0\n\n  model:evaluate()\n\n  for i = 1, data:batchCount() do\n    local batch = onmt.utils.Cuda.convert(data:getBatch(i))\n    loss = loss + model:forwardComputeLoss(batch)\n    totalWords = totalWords + model:getOutputLabelsCount(batch)\n  end\n\n  model:training()\n\n  return math.exp(loss / totalWords)\nend\n\n---------------------------------------------------------------------------------\n\nlocal Trainer = torch.class('Trainer')\n\nlocal options = {\n  {\n    '-save_every', 5000,\n    [[Save intermediate models every this many iterations within an epoch.\n      If = 0, will not save intermediate models.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt()\n    }\n  },\n  {\n    '-report_every', 50,\n    [[Report progress every this many iterations within an epoch.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt()\n    }\n  },\n  {\n    '-async_parallel', false,\n    [[When training on multiple GPUs, update parameters asynchronously.]]\n  },\n  {\n    '-async_parallel_minbatch', 1000,\n    [[In asynchronous training, minimal number of sequential batches before being parallel.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt()\n    }\n  },\n  {\n    '-start_iteration', 1,\n    [[If loading from a checkpoint, the iteration from which to start.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isInt(1)\n    }\n  },\n  {\n    '-start_epoch', 1,\n    [[If loading from a checkpoint, the epoch from which to start.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isInt(1)\n    }\n  },\n  {\n    '-end_epoch', 13,\n    [[The final epoch of the training.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isInt(1)\n    }\n  },\n  {\n    '-curriculum', 0,\n    [[For this many epochs, order the minibatches based on source length (from smaller to longer).\n      Sometimes setting this to 1 will increase convergence speed.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.isUInt(),\n      train_state = true\n    }\n  }\n}\n\nfunction Trainer.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, 'Trainer')\nend\n\nfunction Trainer:__init(args)\n  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)\n  self.args.profiler = args.profiler\n  self.args.disable_mem_optimization = args.disable_mem_optimization\n\n  -- Make a difference with options which is only used in Checkpoint.\n  self.options = args\nend\n\nfunction Trainer:train(model, optim, trainData, validData, dataset, info)\n  local params, gradParams = {}, {}\n\n  onmt.utils.Parallel.launch(function(idx)\n    -- Only logs information of the first thread.\n    local verbose = idx == 1\n\n    -- Initialize and get model parameters.\n    _G.params, _G.gradParams = _G.model:initParams(verbose)\n\n    -- Switch to training mode.\n    _G.model:training()\n\n    if self.args.profiler then\n      _G.model:enableProfiling()\n    end\n\n    -- optimize memory of the first clone\n    if not self.args.disable_mem_optimization then\n      local batch = onmt.utils.Cuda.convert(trainData:getBatch(1))\n      onmt.utils.Memory.optimize(_G.model, batch, verbose)\n    end\n\n    return idx, _G.params, _G.gradParams\n  end, function(idx, theparams, thegradParams)\n    params[idx] = theparams\n    gradParams[idx] = thegradParams\n  end)\n\n  if info then\n    -- if we had saved a previous rng state - restore it\n    onmt.utils.Cuda.setRNGStates(info.rngStates, true)\n  end\n\n  local checkpoint = onmt.train.Checkpoint.new(self.options, model, optim, dataset.dicts)\n\n  optim:setOptimStates(#params[1])\n\n  local function trainEpoch(epoch, doProfile)\n    local epochProfiler = onmt.utils.Profiler.new(doProfile)\n\n    local startI = self.args.start_iteration\n\n    if trainData.sample then\n      trainData:sample()\n    end\n\n    local numIterations = trainData:batchCount()\n    -- In parallel mode, number of iterations is reduced to reflect larger batch size.\n    if onmt.utils.Parallel.count > 1 and not self.args.async_parallel then\n      numIterations = math.ceil(numIterations / onmt.utils.Parallel.count)\n    end\n\n    local epochState = onmt.train.EpochState.new(epoch, startI, numIterations, optim:getLearningRate())\n    local batchOrder\n\n    if startI > 1 and info ~= nil then\n      batchOrder = info.batchOrder\n    else\n      -- Shuffle mini batch order.\n      batchOrder = torch.randperm(trainData:batchCount())\n    end\n\n    self.args.start_iteration = 1\n    local needLog = false\n\n    if not self.args.async_parallel then\n      -- Synchronous training.\n      local iter = startI\n      for i = startI, trainData:batchCount(), onmt.utils.Parallel.count do\n        local batches = {}\n        local totalSize = 0\n        needLog = true\n\n        for j = 1, math.min(onmt.utils.Parallel.count, trainData:batchCount() - i + 1) do\n          local batchIdx = batchOrder[i + j - 1]\n          if epoch <= self.args.curriculum then\n            batchIdx = i + j - 1\n          end\n          table.insert(batches, trainData:getBatch(batchIdx))\n          totalSize = totalSize + batches[#batches].size\n        end\n\n        local losses = {}\n        local indvAvgLosses = {}\n\n        onmt.utils.Parallel.launch(function(idx)\n          _G.profiler = onmt.utils.Profiler.new(doProfile)\n\n          _G.batch = batches[idx]\n          if _G.batch == nil then\n            return idx, 0, nil, _G.profiler:dump()\n          end\n\n          -- Send batch data to the GPU.\n          onmt.utils.Cuda.convert(_G.batch)\n          _G.batch.totalSize = totalSize\n\n          optim:zeroGrad(_G.gradParams)\n          local loss, indvAvgLoss = _G.model:trainNetwork(_G.batch)\n\n          return idx, loss, indvAvgLoss, _G.profiler:dump()\n        end,\n        function(idx, loss, indvAvgLoss, profile)\n          losses[idx]=loss\n          if trainData.needIndividualLosses and trainData:needIndividualLosses() then\n            indvAvgLosses[idx] = indvAvgLoss\n          end\n          epochProfiler:add(profile)\n        end)\n\n        -- Accumulate the gradients from the different parallel threads.\n        onmt.utils.Parallel.accGradParams(gradParams, batches)\n\n        -- Update the parameters.\n        optim:prepareGrad(gradParams[1])\n        optim:updateParams(params[1], gradParams[1])\n\n        -- Synchronize the parameters with the different parallel threads.\n        onmt.utils.Parallel.syncParams(params)\n\n        for bi = 1, #batches do\n          epochState:update(model, batches[bi], losses[bi])\n          if trainData.needIndividualLosses and trainData:needIndividualLosses() then\n            trainData:setLoss(batchOrder[i + bi - 1], indvAvgLosses[bi])\n          end\n        end\n\n        if iter % self.args.report_every == 0 then\n          epochState:log(iter)\n          needLog = false\n        end\n        if self.args.save_every > 0 and iter % self.args.save_every == 0 then\n          checkpoint:saveIteration(iter, epochState, batchOrder, true)\n        end\n        iter = iter + 1\n      end\n    else\n      -- Asynchronous training.\n      local counter = onmt.utils.Parallel.getCounter()\n      local masterGPU = onmt.utils.Cuda.gpuIds[1]\n      local gradBuffer = onmt.utils.Parallel.gradBuffer\n      local gmutexId = onmt.utils.Parallel.gmutexId()\n\n      local maxConcurrentIter = self.args.report_every\n      if self.args.save_every > 0 and self.args.save_every < maxConcurrentIter then\n        maxConcurrentIter = self.args.save_every\n      end\n      local iter = 0\n\n      counter:set(startI)\n\n      while counter:get() <= trainData:batchCount() do\n        needLog = true\n        local startCounter = counter:get()\n\n        onmt.utils.Parallel.launch(function(idx)\n          _G.profiler = onmt.utils.Profiler.new(doProfile)\n          -- First GPU is only used for master parameters.\n          -- Use 1 GPU only for 1000 first batch.\n          if idx == 1 or (idx > 2 and epoch == 1 and counter:get() < self.args.async_parallel_minbatch) then\n            return\n          end\n\n          local batches = {}\n          local losses = {}\n          local indvAvgLosses = {}\n\n          while true do\n            local i = counter:inc()\n            if i - startCounter >= maxConcurrentIter or i > trainData:batchCount() then\n              return batches, losses, indvAvgLosses, _G.profiler:dump()\n            end\n\n            local batchIdx = batchOrder[i]\n            if epoch <= self.args.curriculum then\n              batchIdx = i\n            end\n\n            _G.batch = trainData:getBatch(batchIdx)\n            table.insert(batches, onmt.utils.Tensor.deepClone(_G.batch))\n            onmt.utils.Cuda.convert(_G.batch)\n\n            optim:zeroGrad(_G.gradParams)\n            local loss, indvAvgLoss = _G.model:trainNetwork(_G.batch)\n            table.insert(losses, loss)\n            if trainData.needIndividualLosses and trainData:needIndividualLosses() then\n              indvAvgLosses[batchIdx] = indvAvgLoss\n            end\n\n            -- Update the parameters.\n            optim:prepareGrad(_G.gradParams)\n\n            -- Add up gradParams to params and synchronize back to this thread.\n            onmt.utils.Parallel.updateAndSync(params[1], _G.gradParams, _G.params, gradBuffer, masterGPU, gmutexId)\n          end\n        end,\n        function(batches, losses, indvAvgLosses, profile)\n          if batches then\n            iter = iter + #batches\n            for i = 1, #batches do\n              epochState:update(model, batches[i], losses[i])\n              if trainData.needIndividualLosses and trainData:needIndividualLosses() then\n                trainData:setLoss(batchOrder[i], indvAvgLosses[batchOrder[i]])\n              end\n            end\n            epochProfiler:add(profile)\n          end\n        end)\n\n        if iter % self.args.report_every == 0 then\n          epochState:log()\n          needLog = false\n        end\n        if iter % self.args.save_every == 0 then\n          checkpoint:saveIteration(iter, epochState, batchOrder, true)\n        end\n      end\n    end\n\n    if needLog then\n      epochState:log(numIterations)\n    end\n\n    return epochState, epochProfiler:dump()\n  end\n\n  _G.logger:info('Start training...')\n\n  for epoch = self.args.start_epoch, self.args.end_epoch do\n    _G.logger:info('')\n\n    local globalProfiler = onmt.utils.Profiler.new(self.args.profiler)\n\n    globalProfiler:start('train')\n    local epochState, epochProfile = trainEpoch(epoch, self.args.profiler)\n    globalProfiler:add(epochProfile)\n    globalProfiler:stop('train')\n\n    globalProfiler:start('valid')\n    local validPpl = eval(model, validData)\n    globalProfiler:stop('valid')\n\n    if self.args.profiler then _G.logger:info('profile: %s', globalProfiler:log()) end\n    _G.logger:info('Validation perplexity: %.2f', validPpl)\n\n    local continue = optim:updateLearningRate(validPpl, epoch)\n\n    checkpoint:saveEpoch(validPpl, epochState, true)\n\n    if not continue then\n      _G.logger:warning('Stopping training due to a too small learning rate value.')\n      break\n    end\n  end\nend\n\nreturn Trainer\n"
  },
  {
    "path": "opennmt/train/init.lua",
    "content": "local train = {}\n\ntrain.Trainer = require('opennmt.train.Trainer')\ntrain.Checkpoint = require('opennmt.train.Checkpoint')\ntrain.EpochState = require('opennmt.train.EpochState')\ntrain.Optim = require('opennmt.train.Optim')\n\nreturn train\n"
  },
  {
    "path": "opennmt/translate/Advancer.lua",
    "content": "--[[ Class for specifying how to advance one step. A beam mainly consists of\n  a list of `tokens` and a `state`. `tokens[t]` stores a flat tensors of size\n  `batchSize * beamSize` representing tokens at step `t`. `state` can be either\n  a tensor with first dimension size `batchSize * beamSize`, or an iterable\n  object containing several such tensors.\n\n  Pseudocode:\n\n      finished = []\n\n      beams = {}\n\n      -- Initialize the beam.\n\n      [ beams[1] ] <-- initBeam()\n\n      FOR t = 1, ... DO\n\n        -- Update beam states based on new tokens.\n\n        update([ beams[t] ])\n\n        -- Expand beams by all possible tokens and return the scores.\n\n        [ [scores] ] <-- expand([ beams[t] ])\n\n        -- Find k best next beams (maintained by BeamSearcher).\n\n        _findKBest([beams], [ [scores] ])\n\n        completed <-- isComplete([ beams[t + 1] ])\n\n        -- Remove completed hypotheses (maintained by BeamSearcher).\n\n        finished += _completeHypotheses([beams], completed)\n\n        IF all(completed) THEN\n\n          BREAK\n\n        END\n\n      ENDWHILE\n\n ==================================================================\n--]]\nlocal Advancer = torch.class('Advancer')\n\n--[[Returns an initial beam.\n\nReturns:\n\n  * `beam` - an `onmt.translate.Beam` object.\n\n]]\nfunction Advancer:initBeam()\nend\n\n--[[Updates beam states given new tokens.\n\nParameters:\n\n  * `beam` - beam with updated token list.\n\n]]\nfunction Advancer:update(beam) -- luacheck: no unused args\nend\n\n--[[Expands beam by all possible tokens and returns the scores.\n\nParameters:\n\n  * `beam` - an `onmt.translate.Beam` object.\n\nReturns:\n\n  * `scores` - a 2D tensor of size `(batchSize * beamSize, numTokens)`.\n\n]]\nfunction Advancer:expand(beam) -- luacheck: no unused args\nend\n\n--[[Checks which hypotheses in the beam are already finished.\n\nParameters:\n\n  * `beam` - an `onmt.translate.Beam` object.\n\nReturns: a binary flat tensor of size `(batchSize * beamSize)`, indicating\n  which hypotheses are finished.\n\n]]\nfunction Advancer:isComplete(beam) -- luacheck: no unused args\nend\n\n--[[Specifies which states to keep track of. After beam search, those states\n  can be retrieved during all steps along with the tokens. This is used\n  for memory efficiency.\n\nParameters:\n\n  * `indexes` - a table of iterators, specifying the indexes in the `states` to track.\n\n]]\nfunction Advancer:setKeptStateIndexes(indexes)\n  self.keptStateIndexes = indexes\nend\n\n--[[Checks which hypotheses in the beam shall be pruned.\n\nParameters:\n\n  * `beam` - an `onmt.translate.Beam` object.\n\nReturns: a binary flat tensor of size `(batchSize * beamSize)`, indicating\n  which beams shall be pruned.\n\n]]\nfunction Advancer:filter()\nend\n\nreturn Advancer\n"
  },
  {
    "path": "opennmt/translate/Beam.lua",
    "content": "--[[ Class for maintaining statistics of each step. A beam mainly consists of\n  a list of tokens `tokens` and a state `state`. `tokens[t]` stores a flat tensor\n  of size `batchSize * beamSize` representing the tokens at step `t`, while\n  `state` can be either a tensor with first dimension size `batchSize * beamSize`,\n  or an iterable object containing several such tensors.\n--]]\nlocal Beam = torch.class('Beam')\n\n--[[Helper function. Recursively convert flat `batchSize * beamSize` tensors\n to 2D `(batchSize, beamSize)` tensors.\n\nParameters:\n\n  * `v` - flat tensor of size `batchSize * beamSize` or a table containing such\n  tensors.\n  * `beamSize` - beam size\n\nReturns: `(batchSize, beamSize)` tensor or a table containing such tensors.\n\n--]]\nlocal function flatToRc(v, beamSize)\n  return onmt.utils.Tensor.recursiveApply(v, function (h)\n    local batchSize = math.floor(h:size(1) / beamSize)\n    local sizes = {}\n    for j = 2, #h:size() do\n      sizes[j - 1] = h:size(j)\n    end\n    return h:view(batchSize, beamSize, table.unpack(sizes))\n  end)\nend\n\n--[[Helper function. Recursively convert 2D `(batchSize, beamSize)` tensors to\n flat `batchSize * beamSize` tensors.\n\nParameters:\n\n  * `v` - flat tensor of size `(batchSize, beamSize)` or a table containing such\n  tensors.\n  * `beamSize` - beam size\n\nReturns: `batchSize * beamSize` tensor or a table containing such tensors.\n\n--]]\nlocal function rcToFlat(v)\n  return onmt.utils.Tensor.recursiveApply(v, function (h)\n    local sizes = {}\n    sizes[1] = h:size(1) * h:size(2)\n    for j = 3, #h:size() do\n      sizes[j - 1] = h:size(j)\n    end\n    return h:view(table.unpack(sizes))\n  end)\nend\n\n--[[Helper function. Recursively select `(batchSize, ...)` tensors by\n  specified batch indexes.\n\nParameters:\n\n  * `v` - tensor of size `(batchSize, ...)` or a table containing such tensors\n  * `indexes` - a table of the desired batch indexes\n\nReturns: Indexed `(newBatchSize, ...)` tensor or a table containing such tensors\n\n--]]\nlocal function selectBatch(v, remaining)\n  return onmt.utils.Tensor.recursiveApply(v, function (h)\n    if not torch.isTensor(remaining) then\n      remaining = torch.LongTensor(remaining)\n    end\n    return h:index(1, remaining)\n  end)\nend\n\n--[[Helper function. Recursively select `(batchSize * beamSize, ...)` tensors by\n  specified batch index and beam index.\n\nParameters:\n\n  * `v` - tensor of size `(batchSize * beamSize, ...)` or a table containing\n  such tensors.\n  * `beamSize` - beam size\n  * `batch` - the desired batch index\n  * `beam` - the desired beam index\n\nReturns: Indexed `(...)` tensor or a table containing such tensors\n\n--]]\nlocal function selectBatchBeam(v, beamSize, batch, beam)\n  return onmt.utils.Tensor.recursiveApply(v, function (h)\n    local batchSize = math.floor(h:size(1) / beamSize)\n    local sizes = {}\n    for j = 2, #h:size() do\n      sizes[j - 1] = h:size(j)\n    end\n    local hOut = h:view(batchSize, beamSize, table.unpack(sizes))\n    return hOut[{batch, beam}]\n  end)\nend\n\n--[[Helper function. Recursively index `(batchSize * beamSize, ...)`\n  tensors by specified indexes.\n\nParameters:\n\n  * `v` - tensor of size `(batchSize * beamSize, ...)` or a table containing\n  such tensors.\n  * `indexes` - a tensor of size `(batchSize, k)` specifying the desired indexes\n  * `beamSize` - beam size\n\nReturns: Indexed `(batchSize * k, ...)` tensor or a table containing such tensors\n\n--]]\nlocal function selectBeam(v, indexes, beamSize)\n  return onmt.utils.Tensor.recursiveApply(v, function (h)\n    local batchSize = indexes:size(1)\n    local k = indexes:size(2)\n    beamSize = beamSize or k\n    local sizes = {}\n    local ones = {}\n    for j = 2, #h:size() do\n      sizes[j - 1] = h:size(j)\n      ones[j - 1] = 1\n    end\n    return h\n      :contiguous()\n      :view(batchSize, beamSize, table.unpack(sizes))\n      :gather(2, indexes\n                :view(batchSize, k, table.unpack(ones))\n                :expand(batchSize, k, table.unpack(sizes)))\n      :view(batchSize * k, table.unpack(sizes))\n  end)\nend\n\n--[[Helper function. Recursively expand `(batchSize, ...)` tensors\n  to `(batchSize * beamSize, ...)` tensors.\n\nParameters:\n\n  * `v` - tensor of size `(batchSize, ...)` or a table containing such tensors\n  * `beamSize` - beam size\n\nReturns: Expanded `(batchSize * beamSize, ...)` tensor or a table containing\n  such tensors\n\n--]]\nlocal function replicateBeam(v, beamSize)\n  return onmt.utils.Tensor.recursiveApply(v, function (h)\n    local batchSize = h:size(1)\n    local sizes = {}\n    for j = 2, #h:size() do\n      sizes[j - 1] = h:size(j)\n    end\n    return h\n      :contiguous()\n      :view(batchSize, 1, table.unpack(sizes))\n      :expand(batchSize, beamSize, table.unpack(sizes))\n      :contiguous()\n      :view(batchSize * beamSize, table.unpack(sizes))\n  end)\nend\n\n\n--[[Constructor. We allow users to either specify all initial hypotheses by\n  passing in `token` and `state` with first dimension `batchSize * beamSize`\n  such that there are `beamSize` initial hypotheses for every sequence in the\n  batch and pass in the number of sequences `batchSize`, or only specify one\n  hypothesis per sequence by passing `token` and `state` with first dimension\n  `batchSize`, and then `onmt.translate.BeamSearcher` will pad with auxiliary\n  hypotheses with scores `-inf` such that each sequence starts with `beamSize`\n  hypotheses as in the former case.\n\nParameters:\n\n  * `token` - tensor of size `(batchSize, vocabSize)` (if start with one initial\n  hypothesis per sequence) or `(batchSize * beamSize, vocabSize)` (if start with\n  `beamSize` initial hypotheses per sequence), or a list of such tensors.\n  * `state` - an iterable object, where the contained tensors should have the\n  same first dimension as `token`.\n  * `batchSize` - optional, number of sentences. Only necessary if\n  start with `beamSize` hypotheses per sequence. [`token:size(1)`]\n\n--]]\nfunction Beam:__init(token, state, params, batchSize)\n  self._remaining = batchSize or token:size(1)\n\n  if torch.type(token) == 'table' then\n    self._tokens = token\n  else\n    self._tokens = { token }\n  end\n  self._state = state\n\n  self._params = {}\n  if params then\n    self._params = params\n  else\n    self._params.length_norm = 0.0\n    self._params.coverage_norm = 0.0\n    self._params.eos_norm = 0.0\n  end\n\n  self._scores = torch.zeros(self._remaining)\n  self._backPointer = nil\n  self._prevBeam = nil\n  self._orig2Remaining = {}\n  self._remaining2Orig = {}\n  self._step = 1\nend\n\n--[[\n\nReturns:\n\n  * `tokens` - a list of tokens. Note that the start-of-sequence symbols are\n  also included. `tokens[t]` stores the tokens at step `t`, which is a tensor\n  of size `batchSize * beamSize`.\n\n--]]\nfunction Beam:getTokens()\n  return self._tokens\nend\n\n--[[\n\nReturns:\n\n  * `state` - an abstract iterable object as passed by constructor. Every tensor\n  inside the `state` has first dimension `batchSize * beamSize`\n\n--]]\nfunction Beam:getState()\n  return self._state\nend\n\n--[[\n\nReturns:\n\n  * `scores` - a flat tensor storing the total scores for each batch. It is of\n  size `batchSize * beamSize`.\n\n--]]\nfunction Beam:getScores()\n  return self._scores\nend\n\n--[[\n\nReturns:\n\n  * `backPointer` - a flat tensor storing the backpointers for each batch. It is\n  of size `batchSize * beamSize`\n\n--]]\nfunction Beam:getBackPointer()\n  return self._backPointer\nend\n\n--[[ Returns the number of unfinished sequences. The finished sequences will be\n  removed from batch.\n\nReturns:\n\n  * `remaining` - the number of unfinished sequences.\n\n--]]\nfunction Beam:getRemaining()\n  return self._remaining\nend\n\n--[[ Since finished sequences are being removed from the batch, this function\n  provides a way to convert the remaining batch id to the original batch id.\n\n--]]\nfunction Beam:remaining2Orig(remainingId)\n  if remainingId then\n    return self._remaining2Orig[remainingId]\n  else\n    return self._remaining2Orig\n  end\nend\n\n--[[ Since finished sequences are being removed from the batch, this function\n  provides a way to convert the original batch id to the remaining batch id.\n\n--]]\nfunction Beam:orig2Remaining(origId)\n  if origId then\n    return self._orig2Remaining[origId]\n  else\n    return self._orig2Remaining\n  end\nend\n\n--[[ Set state.\n--]]\nfunction Beam:setState(state)\n  self._state = state\nend\n\n--[[ Set scores.\n--]]\nfunction Beam:setScores(scores)\n  self._scores = scores:view(-1)\nend\n\n--[[ Set backPointer.\n--]]\nfunction Beam:setBackPointer(backPointer)\n  self._backPointer = backPointer:view(-1)\nend\n\n-- In the first step, if there is only 1 hypothesis per batch, then each\n-- hypothesis is replicated beamSize times to keep consistency with the\n-- following beam search steps, while the scores of the auxiliary hypotheses\n-- are set to -inf.\nfunction Beam:_replicate(beamSize)\n  assert (#self._tokens == 1, 'only the first beam may need replicating!')\n  local token = self._tokens[1]\n  local batchSize = token:size(1)\n  self._tokens[1] = replicateBeam(token, beamSize)\n  self._state = replicateBeam(self._state, beamSize)\n  self._scores = replicateBeam(self._scores, beamSize)\n  local maskScores = self._scores.new():resize(batchSize, beamSize)\n  maskScores:fill(-math.huge)\n  maskScores:select(2,1):fill(0)\n  self._scores:add(maskScores:view(-1))\nend\n\n-- Normalize scores by length and coverage\nfunction Beam:_normalizeScores(scores)\n\n  if #self._state ~= 8 then\n    return scores\n  end\n\n  local function normalizeLength(t)\n    local alpha = self._params.length_norm\n    local norm_term = math.pow((5.0 + t)/6.0, alpha)\n    return norm_term\n  end\n\n  local function normalizeCoverage(ap)\n    local beta = self._params.coverage_norm\n    local result = torch.cmin(ap, 1.0):log1p():sum(3):mul(beta)\n    return result\n  end\n\n  local step = self._step\n  local lengthPenalty = normalizeLength(step)\n\n  local attnProba = self._state[8]:view(self._remaining, scores:size(2), -1)\n  local coveragePenalty = normalizeCoverage(attnProba)\n\n  if (scores:nDimension() > 2) then\n    coveragePenalty =  coveragePenalty:expand(scores:size())\n  else\n    coveragePenalty = coveragePenalty:viewAs(scores)\n  end\n\n  local normScores = torch.add(torch.div(scores, lengthPenalty), coveragePenalty)\n\n  return normScores\n\nend\n\n\n-- Given new scores, combine that with the previous total scores and find the\n-- top K hypotheses to form the next beam.\nfunction Beam:_expandScores(scores, beamSize)\n  local remaining = math.floor(scores:size(1) / beamSize)\n  local vocabSize = scores:size(2)\n\n  if #self._state == 8 and self._params.eos_norm > 0 then\n    local EOS_penalty = torch.div(self._state[6]:view(remaining, beamSize), self._step/self._params.eos_norm)\n    scores:view(remaining, beamSize, -1)[{{},{},onmt.Constants.EOS}]:cmul(EOS_penalty)\n  end\n\n  self._scores = self._scores:typeAs(scores)\n  local expandedScores\n    = (scores:typeAs(self._scores):view(remaining, beamSize, -1)\n         + self._scores:view(remaining, beamSize, 1):expand(remaining, beamSize, vocabSize)\n      )\n\n  local normExpandedScores = self:_normalizeScores(expandedScores)\n  return expandedScores:view(remaining, -1), normExpandedScores:view(remaining, -1)\nend\n\n-- Create a new beam given new token, scores and backpointer.\nfunction Beam:_nextBeam(token, scores, backPointer, beamSize)\n  local remaining = math.floor(token:size(1) / beamSize)\n  local params = self._params\n  local newBeam = Beam.new(self:_nextTokens(token, backPointer, beamSize),\n                           self:_nextState(backPointer, beamSize),\n                           params,\n                           remaining)\n  newBeam:setScores(scores)\n  newBeam:setBackPointer(backPointer)\n  newBeam._prevBeam = self\n  newBeam._step = self._step + 1\n  return newBeam\nend\n\n-- Select the on-beam states using the pointers\nfunction Beam:_nextState(backPointer, beamSize)\n  local nextState = selectBeam(self._state, backPointer, beamSize)\n  return nextState\nend\n\n-- Given backpointers, index the tokens in the history to form the next beam's\n-- token list.\nfunction Beam:_nextTokens(token, backPointer, beamSize)\n  local nextTokens = selectBeam(self._tokens, backPointer, beamSize)\n  nextTokens[#nextTokens + 1] = token\n  return nextTokens\nend\n\n-- Remove finished sequences to save computation.\nfunction Beam:_removeFinishedBatches(remainingIds, beamSize)\n  self._remaining = #remainingIds\n  if #remainingIds > 0 then\n    self._state = rcToFlat(selectBatch(flatToRc(self._state, beamSize), remainingIds))\n    self._tokens = rcToFlat(selectBatch(flatToRc(self._tokens, beamSize), remainingIds))\n    self._scores = rcToFlat(selectBatch(flatToRc(self._scores, beamSize), remainingIds))\n    self._backPointer = rcToFlat(selectBatch(flatToRc(self._backPointer, beamSize), remainingIds))\n  end\nend\n\n-- Index the iterable state object by given batch id and beam id.\nfunction Beam:_indexState(beamSize, batchId, beamId, keptIndexes)\n  keptIndexes = keptIndexes or {}\n  local keptState = {}\n  for _, val in pairs(keptIndexes) do\n    keptState[val] = self._state[val]\n  end\n  return selectBatchBeam(keptState, beamSize, batchId, beamId)\nend\n\n-- Index the last step token by given batch id and beam id.\nfunction Beam:_indexToken(beamSize, batchId, beamId)\n  local token = self._tokens[#self._tokens]\n  return selectBatchBeam(token, beamSize, batchId, beamId)\nend\n\n-- Index backpointer by given batch id and beam id.\nfunction Beam:_indexBackPointer(beamSize, batchId, beamId)\n  if self._backPointer then\n    return selectBatchBeam(self._backPointer, beamSize, batchId, beamId)\n  end\nend\n\n-- To save memory, only states at keptIndexes will be kept, while the rest\n-- are discarded.\nfunction Beam:_cleanUp(keptIndexes)\n  keptIndexes = keptIndexes or {}\n  local keptState = {}\n  for _, val in pairs(keptIndexes) do\n    keptState[val] = self._state[val]\n  end\n  self._state = keptState\nend\n\n-- Add completed hypotheses to buffer.\nfunction Beam:_addCompletedHypotheses(batchId, completed)\n  local origId = self:_getOrigId(batchId)\n  self._remainingId = self._remainingId or 0\n  self._remainingId = self._remainingId + 1\n  self._orig2Remaining[origId] = self._remainingId\n  self._remaining2Orig[self._remainingId] = origId\n  completed = completed:view(self._remaining, -1)\n  local scores = self._scores:view(self._remaining, -1)\n  local normScores = self:_normalizeScores(scores)\n  local tokens = self._tokens[#self._tokens]:view(self._remaining, -1)\n  local backPointers = self._backPointer:view(self._remaining, -1)\n\n  Beam._completed = Beam._completed or {}\n  Beam._completed[origId] = Beam._completed[origId] or {}\n  for k = 1, completed:size(2) do\n    if completed[batchId][k] == 1 then\n      local token = tokens[batchId][k]\n      local backPointer = backPointers[batchId][k]\n      local normScore = normScores[batchId][k]\n      local hypothesis = {origId, normScore, token, backPointer, self._step}\n\n      -- Maintain a sorted list.\n      local id = #Beam._completed[origId] + 1\n      Beam._completed[origId][id] = hypothesis\n      while id > 1 do\n        if Beam._completed[origId][id - 1][2] < normScore then\n          Beam._completed[origId][id - 1], Beam._completed[origId][id]\n            = Beam._completed[origId][id], Beam._completed[origId][id - 1]\n          id = id - 1\n        else\n          break\n        end\n      end\n    end\n  end\nend\n\n-- Free buffer when a sequence is finished.\nfunction Beam._removeCompleted(batchId)\n  if Beam._completed then\n    Beam._completed[batchId] = nil\n  end\nend\n\n-- Get the original if of a sequence given its current position in the batch.\nfunction Beam:_getOrigId(remainingId)\n  local origId\n  if self._step <= 2 then\n    origId = remainingId\n  else\n    origId = self._prevBeam:remaining2Orig(remainingId)\n  end\n  return origId\nend\n\n-- Get nBest hypotheses for a particular sequence in the batch.\nfunction Beam:_getTopHypotheses(remainingId, nBest, completed)\n  local origId = self:_getOrigId(remainingId)\n\n  -- Get previously completed hypotheses of the sequence.\n  local prevCompleted\n  if Beam._completed then\n    prevCompleted = Beam._completed[origId] or {}\n  else\n    prevCompleted = {}\n  end\n\n  local hypotheses = {}\n  local prevId = 1\n  local currId = 1\n  completed = completed:view(self._remaining, -1)\n  local scores = self._scores:view(self._remaining, -1)\n  local normScores = self:_normalizeScores(scores)\n  local tokens = self._tokens[#self._tokens]:view(self._remaining, -1)\n  local backPointers = self._backPointer:view(self._remaining, -1)\n  for _ = 1, nBest do\n    local hypothesis, finished\n    if prevId <= #prevCompleted and prevCompleted[prevId][2] > normScores[remainingId][currId] then\n      hypothesis = prevCompleted[prevId]\n      finished = true\n      prevId = prevId + 1\n    else\n      finished = (completed[remainingId][currId] == 1)\n      if finished then\n        local normScore = normScores[remainingId][currId]\n        local token = tokens[remainingId][currId]\n        local backPointer = backPointers[remainingId][currId]\n        hypothesis = {origId, normScore, token, backPointer, self._step}\n      end\n      currId = currId + 1\n    end\n    table.insert(hypotheses, {hypothesis = hypothesis, finished = finished})\n  end\n  return hypotheses\nend\nreturn Beam\n"
  },
  {
    "path": "opennmt/translate/BeamSearcher.lua",
    "content": "--[[ Class for managing the internals of the beam search process.\n\n\n      hyp1---hyp1---hyp1 -hyp1\n          \\             /\n      hyp2 \\-hyp2 /-hyp2--hyp2\n                 /      \\\n      hyp3---hyp3---hyp3 -hyp3\n      ========================\n\nTakes care of beams.\n--]]\nlocal BeamSearcher = torch.class('BeamSearcher')\n\n--[[Constructor\n\nParameters:\n\n  * `advancer` - an `onmt.translate.Advancer` object.\n\n]]\nfunction BeamSearcher:__init(advancer)\n  self.advancer = advancer\nend\n\n--[[ Performs beam search.\n\nParameters:\n\n  * `beamSize` - beam size. [1]\n  * `nBest` - the `nBest` top hypotheses will be returned after beam search. [1]\n  * `preFilterFactor` - optional, set this only if filter is being used. Before\n  applying filters, hypotheses with top `beamSize * preFilterFactor` scores will\n  be considered. If the returned hypotheses voilate filters, then set this to a\n  larger value to consider more. [1]\n  * `keepInitial` - optional, whether return the initial token or not. [false]\n\nReturns: a table `finished`. `finished[b][n].score`, `finished[b][n].tokens`\nand `finished[b][n].states` describe the n-th best hypothesis for b-th sample\nin the batch.\n\n]]\nfunction BeamSearcher:search(beamSize, nBest, preFilterFactor, keepInitial)\n  self.nBest = nBest or 1\n  self.beamSize = beamSize or 1\n  assert (self.nBest <= self.beamSize)\n  self.preFilterFactor = preFilterFactor or 1\n  self.keepInitial = keepInitial or false\n\n  local beams = {}\n  local finished = {}\n\n  -- Initialize the beam.\n  beams[1] = self.advancer:initBeam()\n  local remaining = beams[1]:getRemaining()\n  if beams[1]:getTokens()[1]:size(1) ~= remaining * beamSize then\n    beams[1]:_replicate(self.beamSize)\n  end\n  local t = 1\n  while remaining > 0 do\n    -- Update beam states based on new tokens.\n    self.advancer:update(beams[t])\n\n    -- Expand beams by all possible tokens and return the scores.\n    local scores = self.advancer:expand(beams[t])\n\n    -- Find k best next beams (maintained by BeamSearcher).\n    self:_findKBest(beams, scores)\n\n    -- Determine which hypotheses are complete.\n    local completed = self.advancer:isComplete(beams[t + 1])\n\n    -- Remove completed hypotheses (maintained by BeamSearcher).\n    local finishedBatches, finishedHypotheses = self:_completeHypotheses(beams, completed)\n\n    for b = 1, #finishedBatches do\n      finished[finishedBatches[b]] = finishedHypotheses[b]\n    end\n    t = t + 1\n    remaining = beams[t]:getRemaining()\n  end\n  return finished\nend\n\n-- Find the top beamSize hypotheses (satisfying filters).\nfunction BeamSearcher:_findKBest(beams, scores)\n\n  local function topk(tensor, ...)\n    if torch.typename(tensor) == 'torch.CudaHalfTensor' then\n      tensor = tensor:cuda()\n    end\n    return tensor:topk(...)\n  end\n\n  local t = #beams\n  local vocabSize = scores:size(2)\n  local expandedScores, expandedNormScores = beams[t]:_expandScores(scores, self.beamSize)\n\n  -- Find top beamSize * preFilterFactor hypotheses.\n  local considered = self.beamSize * self.preFilterFactor\n  local consideredNormScores, consideredIds = topk(expandedNormScores, considered, 2, true, true)\n  local consideredScores = expandedScores:gather(2, consideredIds)\n\n  consideredIds:add(-1)\n\n  local consideredBackPointer = (consideredIds:clone():div(vocabSize)):add(1)\n  local consideredToken = consideredIds:view(-1)\n  if consideredToken.fmod then\n    consideredToken = consideredToken:fmod(vocabSize):add(1)\n  else\n    for i = 1, consideredToken:size(1) do\n      consideredToken[i] = math.fmod(consideredToken[i], vocabSize) + 1\n    end\n  end\n\n  local newBeam = beams[t]:_nextBeam(consideredToken, consideredScores,\n                                    consideredBackPointer, self.beamSize)\n\n  -- Prune hypotheses if necessary.\n  local pruned = self.advancer:filter(newBeam)\n  if pruned and pruned:any() then\n    consideredScores:view(-1):maskedFill(pruned, -math.huge)\n    consideredNormScores:view(-1):maskedFill(pruned, -math.huge)\n  end\n\n  -- Find top beamSize hypotheses.\n  if ((not pruned) or (not pruned:any())) and (self.preFilterFactor == 1) then\n    beams[t + 1] = newBeam\n  else\n    local _, kBestIds = topk(consideredNormScores, self.beamSize, 2, true, true)\n    local kBestScores = consideredScores:gather(2, kBestIds)\n    local backPointer = consideredBackPointer:gather(2, kBestIds)\n    local token = consideredToken\n      :viewAs(consideredIds)\n      :gather(2, kBestIds)\n      :view(-1)\n    newBeam = beams[t]:_nextBeam(token, kBestScores, backPointer, self.beamSize)\n    beams[t + 1] = newBeam\n  end\n\n  -- Cleanup unused memory.\n  beams[t]:_cleanUp(self.advancer.keptStateIndexes)\nend\n\n-- Do a backward pass to get the tokens and states throughout the history.\nfunction BeamSearcher:_retrieveHypothesis(beams, batchId, score, tok, bp, t)\n  local states = {}\n  local tokens = {}\n\n  tokens[t - 1] = tok\n  t = t - 1\n  local remainingId\n  while t > 0 do\n    if t == 1 then\n      remainingId = batchId\n    else\n      remainingId = beams[t]:orig2Remaining(batchId)\n    end\n    assert (remainingId)\n    states[t] = beams[t]:_indexState(self.beamSize, remainingId, bp, self.advancer.keptStateIndexes)\n    tokens[t - 1] = beams[t]:_indexToken(self.beamSize, remainingId, bp)\n    bp = beams[t]:_indexBackPointer(self.beamSize, remainingId, bp)\n    t = t - 1\n  end\n  if not self.keepInitial then\n    tokens[0] = nil\n  end\n\n  -- Transpose states\n  local statesTemp = {}\n    for r = 1, #states do\n      for j, _ in pairs(states[r]) do\n        statesTemp[j] = statesTemp[j] or {}\n        statesTemp[j][r] = states[r][j]\n      end\n    end\n  states = statesTemp\n  return {tokens = tokens, states = states, score = score}\nend\n\n-- Checks which sequences are finished and moves finished hypothese to a buffer.\nfunction BeamSearcher:_completeHypotheses(beams, completed)\n  local t = #beams\n  local batchSize = beams[t]:getRemaining()\n  completed = completed:view(batchSize, -1)\n\n  local finishedBatches = {}\n  local finishedHypotheses = {}\n\n  -- Keep track of unfinished batch ids.\n  local remainingIds = {}\n\n  -- For each sequence in the batch, check whether it is finished or not.\n  for b = 1, batchSize do\n    local batchFinished = true\n    local hypotheses = beams[t]:_getTopHypotheses(b, self.nBest, completed)\n\n    -- Checks whether the top nBest hypotheses are all finished.\n    for k = 1, self.nBest do\n      local hypothesis = hypotheses[k]\n      if not hypothesis.finished then\n        batchFinished = false\n        break\n      end\n    end\n\n    if not batchFinished then\n      -- For incomplete sequences, the complete hypotheses will be removed\n      -- from beam and saved to buffer.\n      table.insert(remainingIds, b)\n      beams[t]:_addCompletedHypotheses(b, completed)\n    else\n      -- For complete sequences, we do a backward pass to retrieve the state\n      -- values and tokens throughout the history.\n      local origId = beams[t]:_getOrigId(b)\n      table.insert(finishedBatches, origId)\n      local hypothesis = {}\n      for k = 1, self.nBest do\n        table.insert(hypothesis, self:_retrieveHypothesis(beams,\n                                                          table.unpack(hypotheses[k].hypothesis)))\n      end\n      table.insert(finishedHypotheses, hypothesis)\n      onmt.translate.Beam._removeCompleted(origId)\n    end\n  end\n\n  beams[t]:getScores():maskedFill(completed:view(-1), -math.huge)\n\n  -- Remove finished sequences from batch.\n  if #remainingIds < batchSize then\n    beams[t]:_removeFinishedBatches(remainingIds, self.beamSize)\n  end\n  return finishedBatches, finishedHypotheses\nend\n\nreturn BeamSearcher\n"
  },
  {
    "path": "opennmt/translate/DecoderAdvancer.lua",
    "content": "--[[ DecoderAdvancer is an implementation of the interface Advancer for\n  specifyinghow to advance one step in decoder.\n--]]\nlocal DecoderAdvancer = torch.class('DecoderAdvancer', 'Advancer')\n\n--[[ Constructor.\n\nParameters:\n\n  * `decoder` - an `onmt.Decoder` object.\n  * `batch` - an `onmt.data.Batch` object.\n  * `context` - encoder output (batch x n x rnnSize).\n  * `max_sent_length` - optional, maximum output sentence length.\n  * `max_num_unks` - optional, maximum number of UNKs.\n  * `decStates` - optional, initial decoder states.\n  * `dicts` - optional, dictionary for additional features.\n\n--]]\nfunction DecoderAdvancer:__init(decoder, batch, context, max_sent_length, max_num_unks, decStates, dicts, length_norm, coverage_norm, eos_norm)\n  self.decoder = decoder\n  self.batch = batch\n  self.context = context\n  self.max_sent_length = max_sent_length or math.huge\n  self.max_num_unks = max_num_unks or math.huge\n  self.length_norm = length_norm or 0.0\n  self.coverage_norm = coverage_norm or 0.0\n  self.eos_norm = eos_norm or 0.0\n  self.decStates = decStates or onmt.utils.Tensor.initTensorTable(\n    decoder.args.numEffectiveLayers,\n    onmt.utils.Cuda.convert(torch.Tensor()),\n    { self.batch.size, decoder.args.rnnSize })\n  self.dicts = dicts\nend\n\n--[[Returns an initial beam.\n\nReturns:\n\n  * `beam` - an `onmt.translate.Beam` object.\n\n--]]\nfunction DecoderAdvancer:initBeam()\n  local tokens = onmt.utils.Cuda.convert(torch.IntTensor(self.batch.size)):fill(onmt.Constants.BOS)\n  local features = {}\n  if self.dicts then\n    for j = 1, #self.dicts.tgt.features do\n      features[j] = torch.IntTensor(self.batch.size):fill(onmt.Constants.EOS)\n    end\n  end\n  local sourceSizes = onmt.utils.Cuda.convert(self.batch.sourceSize)\n  local attnProba = torch.FloatTensor(self.batch.size, self.context:size(2)):fill(0.0001)\n  -- Mask padding\n  for i = 1,self.batch.size do\n    local pad_size = self.context:size(2) - sourceSizes[i]\n    if (pad_size ~= 0) then\n      attnProba[{ i, {1,pad_size} }] = 1.0\n    end\n  end\n\n  -- Define state to be { decoder states, decoder output, context,\n  -- attentions, features, sourceSizes, step, cumulated attention probablities }.\n  local state = { self.decStates, nil, self.context, nil, features, sourceSizes, 1, attnProba }\n  local params = {}\n  params.length_norm = self.length_norm\n  params.coverage_norm = self.coverage_norm\n  params.eos_norm = self.eos_norm\n  return onmt.translate.Beam.new(tokens, state, params)\nend\n\n--[[Updates beam states given new tokens.\n\nParameters:\n\n  * `beam` - beam with updated token list.\n\n]]\nfunction DecoderAdvancer:update(beam)\n  local state = beam:getState()\n  local decStates, decOut, context, _, features, sourceSizes, t, cumAttnProba\n    = table.unpack(state, 1, 8)\n  local tokens = beam:getTokens()\n  local token = tokens[#tokens]\n  local inputs\n  if #features == 0 then\n    inputs = token\n  elseif #features == 1 then\n    inputs = { token, features[1] }\n  else\n    inputs = { token }\n    table.insert(inputs, features)\n  end\n  self.decoder:maskPadding(sourceSizes, self.batch.sourceLength)\n  decOut, decStates = self.decoder:forwardOne(inputs, decStates, context, decOut)\n  t = t + 1\n  local softmaxOut = self.decoder.softmaxAttn.output\n\n  cumAttnProba = cumAttnProba:typeAs(softmaxOut):add(softmaxOut)\n\n  local nextState = {decStates, decOut, context, softmaxOut, nil, sourceSizes, t, cumAttnProba}\n  beam:setState(nextState)\nend\n\n--[[Expand function. Expands beam by all possible tokens and returns the\n  scores.\n\nParameters:\n\n  * `beam` - an `onmt.translate.Beam` object.\n\nReturns:\n\n  * `scores` - a 2D tensor of size `(batchSize * beamSize, numTokens)`.\n\n]]\nfunction DecoderAdvancer:expand(beam)\n  local state = beam:getState()\n  local decOut = state[2]\n  local out = self.decoder.generator:forward(decOut)\n  local features = {}\n  for j = 2, #out do\n    local _, best = out[j]:max(2)\n    features[j - 1] = best:view(-1)\n  end\n  state[5] = features\n  local scores = out[1]\n  return scores\nend\n\n--[[Checks which hypotheses in the beam are already finished. A hypothesis is\n  complete if i) an onmt.Constants.EOS is encountered, or ii) the length of the\n  sequence is greater than or equal to `max_sent_length`.\n\nParameters:\n\n  * `beam` - an `onmt.translate.Beam` object.\n\nReturns: a binary flat tensor of size `(batchSize * beamSize)`, indicating\n  which hypotheses are finished.\n\n]]\nfunction DecoderAdvancer:isComplete(beam)\n  local tokens = beam:getTokens()\n  local seqLength = #tokens - 1\n  local complete = tokens[#tokens]:eq(onmt.Constants.EOS)\n  if seqLength > self.max_sent_length then\n    complete:fill(1)\n  end\n  return complete\nend\n\n--[[Checks which hypotheses in the beam shall be pruned. We disallow empty\n predictions, as well as predictions with more UNKs than `max_num_unks`.\n\nParameters:\n\n  * `beam` - an `onmt.translate.Beam` object.\n\nReturns: a binary flat tensor of size `(batchSize * beamSize)`, indicating\n  which beams shall be pruned.\n\n]]\nfunction DecoderAdvancer:filter(beam)\n  local tokens = beam:getTokens()\n  local numUnks = onmt.utils.Cuda.convert(torch.zeros(tokens[1]:size(1)))\n  for t = 1, #tokens do\n    local token = tokens[t]\n    numUnks:add(onmt.utils.Cuda.convert(token:eq(onmt.Constants.UNK):double()))\n  end\n\n  -- Disallow too many UNKs\n  local pruned = numUnks:gt(self.max_num_unks)\n\n  -- Disallow empty hypotheses\n  if #tokens == 2 then\n    pruned:add(tokens[2]:eq(onmt.Constants.EOS))\n  end\n  return pruned:ge(1)\nend\n\nreturn DecoderAdvancer\n"
  },
  {
    "path": "opennmt/translate/PhraseTable.lua",
    "content": "--[[Parse and lookup a words from a phrase table.\n--]]\nlocal PhraseTable = torch.class('PhraseTable')\n\nfunction PhraseTable:__init(filePath)\n  local f = assert(io.open(filePath, 'r'))\n\n  self.table = {}\n\n  for line in f:lines() do\n    local c = onmt.utils.String.split(line, '|||')\n    assert(#c == 2, 'badly formatted phrase table: ' .. line)\n    self.table[onmt.utils.String.strip(c[1])] = onmt.utils.String.strip(c[2])\n  end\n\n  f:close()\nend\n\n--[[ Return the phrase table match for `word`. ]]\nfunction PhraseTable:lookup(word)\n  return self.table[word]\nend\n\n--[[ Return true if the phrase table contains the source word `word`. ]]\nfunction PhraseTable:contains(word)\n  return self:lookup(word) ~= nil\nend\n\nreturn PhraseTable\n"
  },
  {
    "path": "opennmt/translate/Translator.lua",
    "content": "local Translator = torch.class('Translator')\n\nlocal options = {\n  {\n    '-model', '',\n    [[Path to the serialized model file.]],\n    {\n      valid = onmt.utils.ExtendedCmdLine.nonEmpty\n    }\n  },\n  {\n    '-beam_size', 5,\n    [[Beam size.]]\n  },\n  {\n    '-batch_size', 30,\n    [[Batch size.]]\n  },\n  {\n    '-max_sent_length', 250,\n    [[Maximum output sentence length.]]\n  },\n  {\n    '-replace_unk', false,\n    [[Replace the generated <unk> tokens with the source token that\n      has the highest attention weight. If `-phrase_table` is provided,\n      it will lookup the identified source token and give the corresponding\n      target token. If it is not provided (or the identified source token\n      does not exist in the table) then it will copy the source token]]},\n  {\n    '-phrase_table', '',\n    [[Path to source-target dictionary to replace `<unk>` tokens.]]\n  },\n  {\n    '-n_best', 1,\n    [[If > 1, it will also output an n-best list of decoded sentences.]]\n  },\n  {\n    '-max_num_unks', math.huge,\n    [[All sequences with more `<unk>`s than this will be ignored during beam search.]]\n  },\n  {\n    '-pre_filter_factor', 1,\n    [[Optional, set this only if filter is being used. Before\n      applying filters, hypotheses with top `beam_size * pre_filter_factor`\n      scores will be considered. If the returned hypotheses voilate filters,\n      then set this to a larger value to consider more.]]},\n  {\n    '-length_norm', 0.0,\n    [[Length normalization coefficient (alpha). If set to 0, no length normalization.]]\n  },\n  {\n    '-coverage_norm', 0.0,\n    [[Coverage normalization coefficient (beta).\n      An extra coverage term multiplied by beta is added to hypotheses scores.\n      If is set to 0, no coverage normalization.]]\n  },\n  {\n    '-eos_norm', 0.0,\n    [[End of sentence normalization coefficient (gamma). If set to 0, no EOS normalization.]]\n  },\n  {\n    '-dump_input_encoding', false,\n    [[Instead of generating target tokens conditional on\n    the source tokens, we print the representation\n    (encoding/embedding) of the input.]]\n  }\n}\n\nfunction Translator.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, 'Translator')\nend\n\n\nfunction Translator:__init(args)\n  self.opt = args\n\n  _G.logger:info('Loading \\'' .. self.opt.model .. '\\'...')\n  self.checkpoint = torch.load(self.opt.model)\n\n  if self.checkpoint.options.model_type and self.checkpoint.options.model_type ~= 'seq2seq' then\n    _G.logger:error('Translator can only process seq2seq models')\n    os.exit(0)\n  end\n\n  self.models = {}\n  self.models.encoder = onmt.Factory.loadEncoder(self.checkpoint.models.encoder)\n  self.models.decoder = onmt.Factory.loadDecoder(self.checkpoint.models.decoder)\n\n  self.models.encoder:evaluate()\n  self.models.decoder:evaluate()\n\n  onmt.utils.Cuda.convert(self.models.encoder)\n  onmt.utils.Cuda.convert(self.models.decoder)\n\n  self.dicts = self.checkpoint.dicts\n\n  if self.opt.phrase_table:len() > 0 then\n    self.phraseTable = onmt.translate.PhraseTable.new(self.opt.phrase_table)\n  end\nend\n\nfunction Translator:buildInput(tokens)\n  local words, features = onmt.utils.Features.extract(tokens)\n\n  local data = {}\n  data.words = words\n\n  if #features > 0 then\n    data.features = features\n  end\n\n  return data\nend\n\nfunction Translator:buildOutput(data)\n  return table.concat(onmt.utils.Features.annotate(data.words, data.features), ' ')\nend\n\nfunction Translator:buildData(src, gold)\n  local srcData = {}\n  srcData.words = {}\n  srcData.features = {}\n\n  local goldData\n  if gold then\n    goldData = {}\n    goldData.words = {}\n    goldData.features = {}\n  end\n\n  local ignored = {}\n  local indexMap = {}\n  local index = 1\n\n  for b = 1, #src do\n    if #src[b].words == 0 then\n      table.insert(ignored, b)\n    else\n      indexMap[index] = b\n      index = index + 1\n\n      table.insert(srcData.words,\n                   self.dicts.src.words:convertToIdx(src[b].words, onmt.Constants.UNK_WORD))\n\n      if #self.dicts.src.features > 0 then\n        table.insert(srcData.features,\n                     onmt.utils.Features.generateSource(self.dicts.src.features, src[b].features))\n      end\n\n      if gold then\n        table.insert(goldData.words,\n                     self.dicts.tgt.words:convertToIdx(gold[b].words,\n                                                       onmt.Constants.UNK_WORD,\n                                                       onmt.Constants.BOS_WORD,\n                                                       onmt.Constants.EOS_WORD))\n\n        if #self.dicts.tgt.features > 0 then\n          table.insert(goldData.features,\n                       onmt.utils.Features.generateTarget(self.dicts.tgt.features, gold[b].features))\n        end\n      end\n    end\n  end\n\n  return onmt.data.Dataset.new(srcData, goldData), ignored, indexMap\nend\n\nfunction Translator:buildTargetWords(pred, src, attn)\n  local tokens = self.dicts.tgt.words:convertToLabels(pred, onmt.Constants.EOS)\n\n  if self.opt.replace_unk then\n    for i = 1, #tokens do\n      if tokens[i] == onmt.Constants.UNK_WORD then\n        local _, maxIndex = attn[i]:max(1)\n        local source = src[maxIndex[1]]\n\n        if self.phraseTable and self.phraseTable:contains(source) then\n          tokens[i] = self.phraseTable:lookup(source)\n        else\n          tokens[i] = source\n        end\n      end\n    end\n  end\n\n  return tokens\nend\n\nfunction Translator:buildTargetFeatures(predFeats)\n  local numFeatures = #predFeats[1]\n\n  if numFeatures == 0 then\n    return {}\n  end\n\n  local feats = {}\n  for _ = 1, numFeatures do\n    table.insert(feats, {})\n  end\n\n  for i = 2, #predFeats do\n    for j = 1, numFeatures do\n      table.insert(feats[j], self.dicts.tgt.features[j]:lookup(predFeats[i][j]))\n    end\n  end\n\n  return feats\nend\n\nfunction Translator:translateBatch(batch)\n  self.models.encoder:maskPadding()\n  self.models.decoder:maskPadding()\n\n  local encStates, context = self.models.encoder:forward(batch)\n  if self.opt.dump_input_encoding then\n    return encStates[#encStates]\n  end\n  -- Compute gold score.\n  local goldScore\n  if batch.targetInput ~= nil then\n    if batch.uneven then\n      self.models.decoder:maskPadding(batch.sourceSize, batch.sourceLength)\n    end\n    goldScore = self.models.decoder:computeScore(batch, encStates, context)\n  end\n\n  -- Specify how to go one step forward.\n  local advancer = onmt.translate.DecoderAdvancer.new(self.models.decoder,\n                                                      batch,\n                                                      context,\n                                                      self.opt.max_sent_length,\n                                                      self.opt.max_num_unks,\n                                                      encStates,\n                                                      self.dicts,\n                                                      self.opt.length_norm,\n                                                      self.opt.coverage_norm,\n                                                      self.opt.eos_norm)\n\n  -- Save memory by only keeping track of necessary elements in the states.\n  -- Attentions are at index 4 in the states defined in onmt.translate.DecoderAdvancer.\n  local attnIndex = 4\n\n  -- Features are at index 5 in the states defined in onmt.translate.DecoderAdvancer.\n  local featsIndex = 5\n\n  advancer:setKeptStateIndexes({attnIndex, featsIndex})\n\n  -- Conduct beam search.\n  local beamSearcher = onmt.translate.BeamSearcher.new(advancer)\n  local results = beamSearcher:search(self.opt.beam_size, self.opt.n_best, self.opt.pre_filter_factor)\n\n  local allHyp = {}\n  local allFeats = {}\n  local allAttn = {}\n  local allScores = {}\n\n  for b = 1, batch.size do\n    local hypBatch = {}\n    local featsBatch = {}\n    local attnBatch = {}\n    local scoresBatch = {}\n\n    for n = 1, self.opt.n_best do\n      local result = results[b][n]\n      local tokens = result.tokens\n      local score = result.score\n      local states = result.states\n      local attn = states[attnIndex] or {}\n      local feats = states[featsIndex] or {}\n\n      -- Ignore generated </s>.\n      table.remove(tokens)\n      if #attn > 0 then\n        table.remove(attn)\n      end\n\n      -- Remove unnecessary values from the attention vectors.\n      local size = batch.sourceSize[b]\n      for j = 1, #attn do\n        attn[j] = attn[j]:narrow(1, batch.sourceLength - size + 1, size)\n      end\n\n      table.insert(hypBatch, tokens)\n      if #feats > 0 then\n        table.insert(featsBatch, feats)\n      end\n      table.insert(attnBatch, attn)\n      table.insert(scoresBatch, score)\n    end\n\n    table.insert(allHyp, hypBatch)\n    table.insert(allFeats, featsBatch)\n    table.insert(allAttn, attnBatch)\n    table.insert(allScores, scoresBatch)\n  end\n\n  return allHyp, allFeats, allScores, allAttn, goldScore\nend\n\n--[[ Translate a batch of source sequences.\n\nParameters:\n\n  * `src` - a batch of tables containing:\n    - `words`: the table of source words\n    - `features`: the table of feaures sequences (`src.features[i][j]` is the value of the ith feature of the jth token)\n  * `gold` - gold data to compute confidence score (same format as `src`)\n\nReturns:\n\n  * `results` - a batch of tables containing:\n    - `goldScore`: if `gold` was given, this is the confidence score\n    - `preds`: an array of `opt.n_best` tables containing:\n      - `words`: the table of target words\n      - `features`: the table of target features sequences\n      - `attention`: the attention vectors of each target word over the source words\n      - `score`: the confidence score of the prediction\n]]\nfunction Translator:translate(src, gold)\n  local data, ignored, indexMap = self:buildData(src, gold)\n\n  local results = {}\n\n  if data:batchCount() > 0 then\n    local batch = data:getBatch()\n\n    local encStates = {}\n    local pred = {}\n    local predFeats = {}\n    local predScore = {}\n    local attn = {}\n    local goldScore = {}\n    if self.opt.dump_input_encoding then\n      encStates = self:translateBatch(batch)\n    else\n      pred, predFeats, predScore, attn, goldScore = self:translateBatch(batch)\n    end\n\n    for b = 1, batch.size do\n      if self.opt.dump_input_encoding then\n        results[b] = encStates[b]\n      else\n        results[b] = {}\n\n        results[b].preds = {}\n        for n = 1, self.opt.n_best do\n          results[b].preds[n] = {}\n          results[b].preds[n].words = self:buildTargetWords(pred[b][n], src[indexMap[b]].words, attn[b][n])\n          results[b].preds[n].features = self:buildTargetFeatures(predFeats[b][n])\n          results[b].preds[n].attention = attn[b][n]\n          results[b].preds[n].score = predScore[b][n]\n        end\n      end\n\n      if goldScore and next(goldScore) ~= nil then\n        results[b].goldScore = goldScore[b]\n      end\n    end\n  end\n\n  for i = 1, #ignored do\n    table.insert(results, ignored[i], {})\n  end\n\n  return results\nend\n\nreturn Translator\n"
  },
  {
    "path": "opennmt/translate/init.lua",
    "content": "local translate = {}\n\ntranslate.Advancer = require('opennmt.translate.Advancer')\ntranslate.Beam = require('opennmt.translate.Beam')\ntranslate.BeamSearcher = require('opennmt.translate.BeamSearcher')\ntranslate.DecoderAdvancer = require('opennmt.translate.DecoderAdvancer')\ntranslate.PhraseTable = require('opennmt.translate.PhraseTable')\ntranslate.Translator = require('opennmt.translate.Translator')\n\nreturn translate\n"
  },
  {
    "path": "opennmt/utils/CrayonLogger.lua",
    "content": "local CrayonLogger = torch.class('CrayonLogger')\n\nlocal options = {\n  {\n    '-exp_host', '127.0.0.1',\n    [[Crayon server IP.]]\n  },\n  {\n    '-exp_port', '8889',\n    [[Crayon server port.]]\n  },\n  {\n    '-exp', '',\n    [[Crayon experiment name.]]\n  }\n}\n\nfunction CrayonLogger.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, 'Crayon')\nend\n\nfunction CrayonLogger:__init(args)\n  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)\n  if args.exp ~= '' then\n    self.host = self.args.exp_host\n    self.port = self.args.exp_port\n\n    local crayon = require(\"crayon\")\n    self.cc = crayon.CrayonClient(self.host, self.port)\n    self.exp = self.cc:create_experiment(args.exp)\n    self.on = true\n\n\n  else\n    self.on = false\n  end\nend\n\nreturn CrayonLogger\n"
  },
  {
    "path": "opennmt/utils/Cuda.lua",
    "content": "local ExtendedCmdLine = require('opennmt.utils.ExtendedCmdLine')\n\nlocal Cuda = {\n  fp16 = false,\n  gpuIds = {},\n  activated = false\n}\n\nlocal options = {\n  {\n    '-gpuid', '0',\n    [[List of comma-separated GPU identifiers (1-indexed). CPU is used when set to 0.]],\n    {\n      valid = ExtendedCmdLine.listUInt\n    }\n  },\n  {\n    '-fallback_to_cpu', false,\n    [[If GPU can't be used, rollback on the CPU.]]\n  },\n  {\n    '-fp16', false,\n    [[Use half-precision float on GPU.]]\n  },\n  {\n    '-no_nccl', false,\n    [[Disable usage of nccl in parallel mode.]]\n  }\n}\n\nfunction Cuda.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, 'Cuda')\nend\n\nfunction Cuda.init(opt, masterGPU)\n  for _, val in ipairs(onmt.utils.String.split(opt.gpuid, ',')) do\n    local id = tonumber(val)\n    assert(id ~= nil and id >= 0, 'invalid GPU identifier: ' .. val)\n    if id > 0 then\n      table.insert(Cuda.gpuIds, id)\n    end\n  end\n\n  Cuda.activated = #Cuda.gpuIds > 0\n\n  if Cuda.activated then\n    local _, err = pcall(function()\n      require('cutorch')\n      require('cunn')\n      Cuda.fp16 = opt.fp16\n\n      if masterGPU == nil then\n        masterGPU = 1\n\n        -- Validate GPU identifiers.\n        for i = 1, #Cuda.gpuIds do\n          assert(Cuda.gpuIds[i] <= cutorch.getDeviceCount(),\n                 'GPU ' .. Cuda.gpuIds[i] .. ' is requested but only '\n                   .. cutorch.getDeviceCount() .. ' GPUs are available')\n        end\n\n        _G.logger:info('Using GPU(s): ' .. table.concat(Cuda.gpuIds, ', '))\n\n        if cutorch.isCachingAllocatorEnabled and cutorch.isCachingAllocatorEnabled() then\n          _G.logger:warning('The caching CUDA memory allocator is enabled. This allocator improves performance at the cost of a higher GPU memory usage. To optimize for memory, consider disabling it by setting the environment variable: THC_CACHING_ALLOCATOR=0')\n        end\n\n      end\n\n      cutorch.setDevice(Cuda.gpuIds[masterGPU])\n\n      if opt.seed then\n        cutorch.manualSeed(opt.seed)\n      end\n    end)\n\n    if err then\n      if opt.fallback_to_cpu then\n        _G.logger:warning('Falling back to CPU')\n        Cuda.activated = false\n      else\n        error(err)\n      end\n    end\n    if Cuda.fp16 and not cutorch.hasHalf then\n      error(\"installed cutorch does not support half-tensor\")\n    end\n  end\nend\n\n-- returns RNGState for CPU and enabled GPUs\nfunction Cuda.getRNGStates()\n  local rngStates = { torch.getRNGState() }\n  for _,idx in ipairs(Cuda.gpuIds) do\n    table.insert(rngStates, cutorch.getRNGState(idx))\n  end\n  return rngStates\nend\n\n-- set RNGState from saved state\nfunction Cuda.setRNGStates(rngStates, verbose)\n  if not rngStates then\n    return\n  end\n  if verbose then\n    _G.logger:info(\"Restoring Random Number Generator states\")\n  end\n  torch.setRNGState(rngStates[1])\n  if #rngStates-1 ~= #Cuda.gpuIds then\n    _G.logger:warning('GPU count does not match for resetting Random Number Generator - skipping')\n  else\n    for idx = 2, #rngStates do\n      cutorch.setRNGState(rngStates[idx], idx-1)\n    end\n  end\nend\n\n--[[\n  Recursively move all supported objects in `obj` on the GPU.\n  When using CPU only, converts to float instead of the default double.\n]]\nfunction Cuda.convert(obj)\n  local objtype = torch.typename(obj)\n  if objtype then\n    if Cuda.activated and obj.cuda ~= nil then\n      if objtype:find('torch%..*LongTensor') then\n        return obj:type('torch.CudaLongTensor')\n      elseif Cuda.fp16 then\n        return obj:type('torch.CudaHalfTensor')\n      else\n        return obj:type('torch.CudaTensor')\n      end\n    elseif not Cuda.activated and obj.float ~= nil then\n      -- Defaults to float instead of double.\n      if objtype:find('torch%..*LongTensor') then\n        return obj:type('torch.LongTensor')\n      else\n        return obj:type('torch.FloatTensor')\n      end\n    end\n  end\n\n  if objtype or type(obj) == 'table' then\n    for k, v in pairs(obj) do\n      obj[k] = Cuda.convert(v)\n    end\n  end\n\n  return obj\nend\n\n--[[\n  Synchronize operations on current device if working on GPU.\n  Do nothing otherwise.\n]]\nfunction Cuda.synchronize()\n  if Cuda.activated then cutorch.synchronize() end\nend\n\n--[[\n  Number of available GPU.\n]]\nfunction Cuda.gpuCount()\n  return #Cuda.gpuIds\nend\n\n--[[\n  Free memory on the current GPU device.\n]]\nfunction Cuda.freeMemory()\n  if Cuda.activated then\n    local freeMemory = cutorch.getMemoryUsage(cutorch.getDevice())\n    return freeMemory\n  end\n  return 0\nend\n\nreturn Cuda\n"
  },
  {
    "path": "opennmt/utils/Dict.lua",
    "content": "local Dict = torch.class(\"Dict\")\n\nfunction Dict:__init(data)\n  self.idxToLabel = {}\n  self.labelToIdx = {}\n  self.frequencies = {}\n\n  -- Special entries will not be pruned.\n  self.special = {}\n\n  if data ~= nil then\n    if type(data) == \"string\" then -- File to load.\n      self:loadFile(data)\n    else\n      self:addSpecials(data)\n    end\n  end\nend\n\n--[[ Return the number of entries in the dictionary. ]]\nfunction Dict:size()\n  return #self.idxToLabel\nend\n\n--[[ Load entries from a file. ]]\nfunction Dict:loadFile(filename)\n  local reader = onmt.utils.FileReader.new(filename)\n\n  while true do\n    local fields = reader:next()\n\n    if not fields then\n      break\n    end\n\n    local label = fields[1]\n    local idx = tonumber(fields[2])\n\n    self:add(label, idx)\n  end\n\n  reader:close()\nend\n\n--[[ Write entries to a file. ]]\nfunction Dict:writeFile(filename)\n  local file = assert(io.open(filename, 'w'))\n\n  for i = 1, self:size() do\n    local label = self.idxToLabel[i]\n    file:write(label .. ' ' .. i .. '\\n')\n  end\n\n  file:close()\nend\n\n--[[ Lookup `key` in the dictionary: it can be an index or a string. ]]\nfunction Dict:lookup(key)\n  if type(key) == \"string\" then\n    return self.labelToIdx[key]\n  else\n    return self.idxToLabel[key]\n  end\nend\n\n--[[ Mark this `label` and `idx` as special (i.e. will not be pruned). ]]\nfunction Dict:addSpecial(label, idx)\n  idx = self:add(label, idx)\n  table.insert(self.special, idx)\nend\n\n--[[ Mark all labels in `labels` as specials (i.e. will not be pruned). ]]\nfunction Dict:addSpecials(labels)\n  for i = 1, #labels do\n    self:addSpecial(labels[i])\n  end\nend\n\n--[[ Add `label` in the dictionary. Use `idx` as its index if given. ]]\nfunction Dict:add(label, idx)\n  if idx ~= nil then\n    self.idxToLabel[idx] = label\n    self.labelToIdx[label] = idx\n  else\n    idx = self.labelToIdx[label]\n    if idx == nil then\n      idx = #self.idxToLabel + 1\n      self.idxToLabel[idx] = label\n      self.labelToIdx[label] = idx\n    end\n  end\n\n  if self.frequencies[idx] == nil then\n    self.frequencies[idx] = 1\n  else\n    self.frequencies[idx] = self.frequencies[idx] + 1\n  end\n\n  return idx\nend\n\n--[[ Return a new dictionary with the `size` most frequent entries. ]]\nfunction Dict:prune(size)\n  if size >= self:size() then\n    return self\n  end\n\n  -- Only keep the `size` most frequent entries.\n  local freq = torch.Tensor(self.frequencies)\n  local _, idx = torch.sort(freq, 1, true)\n\n  local newDict = Dict.new()\n\n  -- Add special entries in all cases.\n  for i = 1, #self.special do\n    newDict:addSpecial(self.idxToLabel[self.special[i]])\n  end\n\n  for i = 1, size do\n    newDict:add(self.idxToLabel[idx[i]])\n  end\n\n  return newDict\nend\n\n--[[ Return a new dictionary with entries appearing at least `minFrequency` times. ]]\nfunction Dict:pruneByMinFrequency(minFrequency)\n  if minFrequency < 2 then\n    return self\n  end\n\n  local freq = torch.Tensor(self.frequencies)\n  local sortedFreq, idx = torch.sort(freq, 1, true)\n\n  local newDict = Dict.new()\n\n  -- Add special entries in all cases.\n  for i = 1, #self.special do\n    newDict:addSpecial(self.idxToLabel[self.special[i]])\n  end\n\n  for i = 1, self:size() do\n    if sortedFreq[i] < minFrequency then\n      break\n    end\n    newDict:add(self.idxToLabel[idx[i]])\n  end\n\n  return newDict\nend\n\n--[[\n  Convert `labels` to indices. Use `unkWord` if not found.\n  Optionally insert `bosWord` at the beginning and `eosWord` at the end.\n]]\nfunction Dict:convertToIdx(labels, unkWord, bosWord, eosWord)\n  local vec = {}\n\n  if bosWord ~= nil then\n    table.insert(vec, self:lookup(bosWord))\n  end\n\n  for i = 1, #labels do\n    local idx = self:lookup(labels[i])\n    if idx == nil then\n      idx = self:lookup(unkWord)\n    end\n    table.insert(vec, idx)\n  end\n\n  if eosWord ~= nil then\n    table.insert(vec, self:lookup(eosWord))\n  end\n\n  return torch.IntTensor(vec)\nend\n\n--[[ Convert `idx` to labels. If index `stop` is reached, convert it and return. ]]\nfunction Dict:convertToLabels(idx, stop)\n  local labels = {}\n\n  for i = 1, #idx do\n    table.insert(labels, self:lookup(idx[i]))\n    if idx[i] == stop then\n      break\n    end\n  end\n\n  return labels\nend\n\nreturn Dict\n"
  },
  {
    "path": "opennmt/utils/ExtendedCmdLine.lua",
    "content": "---------------------------------------------------------------------------------\n-- Local utility functions\n---------------------------------------------------------------------------------\n\n--[[ Convert `val` string to its actual type (boolean, number or string). ]]\nlocal function convert(key, val, ref)\n  local new\n\n  if type(val) == type(ref) then\n    new = val\n  elseif type(ref) == 'boolean' then\n    if val == 'true' then\n      new = true\n    elseif val == 'false' then\n      new = false\n    else\n      error('option ' .. key .. ' expects a boolean value [true, false]')\n    end\n  elseif type(ref) == 'number' then\n    new = tonumber(val)\n    assert(new ~= nil, 'option ' .. key .. ' expects a number value')\n  end\n\n  return new\nend\n\nlocal function wrapIndent(text, size, pad)\n  local p = 0\n  while true do\n    local q = text:find(\" \", size+p)\n    if not q then\n      return text\n    end\n    text = text:sub(1, q) .. \"\\n\" ..pad .. text:sub(q+1, #text)\n    p = q+2+#pad\n  end\nend\n\n---------------------------------------------------------------------------------\n\nlocal ExtendedCmdLine, parent, path\n-- the utils function can be run without torch\nif torch then\n  ExtendedCmdLine, parent = torch.class('ExtendedCmdLine', 'torch.CmdLine')\n  path = require('pl.path')\nelse\n  ExtendedCmdLine = {}\nend\n\n--[[\n  Extended handling of command line options - provide validation methods, and utilities for handling options\n  at module level.\n\n  For end-user - provides '-h' for help, with possible '-md' variant for preparing MD ready help file.\n\n  Provides also possibility to read options from config file with '-config file' or save current options to\n  config file with '-save_config file'\n\n  Example:\n\n    cmd = onmt.utils.ExtendedCmdLine.new()\n    local options = {\n      {'-max_batch_size',     64   , 'Maximum batch size', {valid=onmt.utils.ExtendedCmdLine.isUInt()}}\n    }\n\n    cmd:setCmdLineOptions(options, 'Optimization')\n    local opt = cmd:parse(arg)\n\n    local optimArgs = cmd.getModuleOpts(opt, options)\n\n  Additional meta-fields:\n\n  * `valid`: validation method\n  * `enum`: enumeration list\n  * `structural`: if defined, mark a structural parameter - 0 means cannot change value, 1 means that it can change dynamically\n  * `init_only`: if true, mark a parameter that can only be set at init time\n  * `train_state`: if true, the option will be automatically reused when continuing a training\n\n]]\n\nfunction ExtendedCmdLine:__init(script)\n  self.script = script\n  parent.__init(self)\n\n  self:text('')\n  self:option('-h', false, 'This help.')\n  self:option('-md', false, 'Dump help in Markdown format.')\n  self:option('-config', '', 'Load options from this file.', {valid=ExtendedCmdLine.fileNullOrExists})\n  self:option('-save_config', '', 'Save options to this file.')\n\nend\n\nfunction ExtendedCmdLine:help(arg, doMd)\n  if doMd then\n    io.write('`' .. self.script .. '` options:\\n')\n    for _, option in ipairs(self.helplines) do\n      if type(option) == 'table' then\n        io.write('* ')\n        if option.default ~= nil then -- It is an option.\n          local args = type(option.default) == 'boolean' and '' or ' <' .. type(option.default) .. '>'\n          io.write('`' .. option.key .. args ..'`')\n\n          local valInfo = {}\n          if option.meta and option.meta.enum then\n            for k, v in pairs(option.meta.enum) do\n              option.meta.enum[k] = '`' .. v .. '`'\n            end\n            table.insert(valInfo, 'accepted: ' .. table.concat(option.meta.enum, ', '))\n          end\n          if type(option.default) ~= \"boolean\" and option.default ~= '' then\n            table.insert(valInfo, 'default: `' .. tostring(option.default) .. '`')\n          end\n          if #valInfo > 0 then\n            io.write(' (' .. table.concat(valInfo, '; ') .. ')')\n          end\n\n          io.write('<br/>')\n\n          option.help = option.help:gsub(' *\\n   *', ' ')\n          if option.help then\n            io.write(option.help)\n          end\n        else -- It is an argument.\n          io.write('<' .. onmt.utils.String.stripHyphens(option.key) .. '>')\n          if option.help then\n            io.write(' ' .. option.help)\n          end\n        end\n      else\n        local display = option:gsub('%*', '')\n        if display:len() > 0 then\n          io.write('## ')\n        end\n        io.write(display) -- Just some additional help.\n      end\n      io.write('\\n')\n    end\n    io.write('\\n')\n  else\n    if arg then\n      io.write('Usage: ')\n      io.write(arg[0] .. ' ' .. self.script .. ' ')\n      io.write('[options] ')\n      for i = 1, #self.arguments do\n        io.write('<' .. onmt.utils.String.stripHyphens(self.arguments[i].key) .. '>')\n      end\n      io.write('\\n')\n    end\n\n    -- First pass to compute max length.\n    local optsz = 0\n    for _, option in ipairs(self.helplines) do\n      if type(option) == 'table' then\n        if option.default ~= nil then -- It is an option.\n          if #option.key > optsz then\n            optsz = #option.key\n          end\n        else -- It is an argument.\n          local stripOptionKey = onmt.utils.String.stripHyphens(option.key)\n          if #stripOptionKey + 2 > optsz then\n            optsz = #stripOptionKey + 2\n          end\n        end\n      end\n    end\n\n    local padMultiLine = onmt.utils.String.pad('', optsz)\n    -- Second pass to print.\n    for _, option in ipairs(self.helplines) do\n      if type(option) == 'table' then\n        io.write('  ')\n        if option.default ~= nil then -- It is an option.\n          io.write(onmt.utils.String.pad(option.key, optsz))\n          local msg = ''\n          msg = msg .. option.help:gsub('\\n', ' ')\n          local valInfo = {}\n          if option.meta and option.meta.enum then\n            table.insert(valInfo, 'accepted: ' .. table.concat(option.meta.enum, ', '))\n          end\n          if type(option.default) ~= \"boolean\" and option.default ~= '' then\n            table.insert(valInfo, 'default: ' .. tostring(option.default))\n          end\n          if #valInfo > 0 then\n            msg = msg .. ' (' .. table.concat(valInfo, '; ') .. ')'\n          end\n          io.write(' ' .. wrapIndent(msg:gsub('  *', ' '),60,padMultiLine..'     '))\n        else -- It is an argument.\n          io.write(onmt.utils.String.pad('<' .. onmt.utils.String.stripHyphens(option.key) .. '>', optsz))\n          if option.help then\n            io.write(' ' .. option.help)\n          end\n        end\n      else\n        io.write(option) -- Just some additional help.\n      end\n      io.write('\\n')\n    end\n    io.write('\\n')\n  end\nend\n\nfunction ExtendedCmdLine:error(msg)\n   print('ERROR: '..msg)\n   print('> Use -h for help\\n')\n   os.exit(0)\nend\n\nfunction ExtendedCmdLine:option(key, default, help, _meta_)\n  for _,v in ipairs(self.helplines) do\n    if v.key == key then\n      return\n    end\n  end\n  parent.option(self, key, default, help)\n  self.options[key].meta = _meta_\nend\n\n--[[ Override options with option values set in file `filename`. ]]\nfunction ExtendedCmdLine:loadConfig(filename, opt)\n  local file = assert(io.open(filename, 'r'))\n\n  for line in file:lines() do\n    -- Ignore empty or commented out lines.\n    if line:len() > 0 and string.sub(line, 1, 1) ~= '#' then\n      local field = onmt.utils.String.split(line, '=')\n      assert(#field == 2, 'badly formatted config file')\n\n      local key = onmt.utils.String.strip(field[1])\n      local val = onmt.utils.String.strip(field[2])\n\n      assert(opt[key] ~= nil, 'unkown option ' .. key)\n\n      opt[key] = convert(key, val, opt[key])\n      opt._is_default[key] = nil\n\n    end\n  end\n\n  file:close()\n  return opt\nend\n\nfunction ExtendedCmdLine:logConfig(opt)\n  local keys = {}\n  for key in pairs(opt) do\n    table.insert(keys, key)\n  end\n\n  table.sort(keys)\n  _G.logger:debug('Options:')\n\n  for _, key in ipairs(keys) do\n    if key:sub(1, 1) ~= '_' then\n      local val = opt[key]\n      if type(val) == 'string' then\n        val = '\\'' .. val .. '\\''\n      end\n      _G.logger:debug(' * ' .. key .. ' = ' .. tostring(val))\n    end\n  end\nend\n\nfunction ExtendedCmdLine:dumpConfig(opt, filename)\n  local file = assert(io.open(filename, 'w'))\n\n  for key, val in pairs(opt) do\n    if key:sub(1, 1) ~= '_' then\n      file:write(key .. ' = ' .. tostring(val) .. '\\n')\n    end\n  end\n\n  file:close()\nend\n\nfunction ExtendedCmdLine:parse(arg)\n  local i = 1\n\n  -- set default value\n  local params = { _is_default={}, _structural={}, _init_only={}, _train_state={} }\n  for option,v in pairs(self.options) do\n    local soption = onmt.utils.String.stripHyphens(option)\n    params[soption] = v.default\n    params._is_default[soption] = true\n  end\n\n  local nArgument = 0\n\n  local doHelp = false\n  local doMd = false\n  local readConfig\n  local saveConfig\n\n  local cmdlineOptions = {}\n\n  while i <= #arg do\n    if arg[i] == '-help' or arg[i] == '-h' or arg[i] == '--help' then\n      doHelp = true\n      i = i + 1\n    elseif arg[i] == '-md' then\n      doMd = true\n      i = i + 1\n    elseif arg[i] == '-config' then\n      readConfig = arg[i + 1]\n      i = i + 2\n    elseif arg[i] == '-save_config' then\n      saveConfig = arg[i + 1]\n      i = i + 2\n    else\n      local sopt = onmt.utils.String.stripHyphens(arg[i])\n      params._is_default[sopt] = nil\n      if self.options[arg[i]] then\n        if cmdlineOptions[arg[i]] then\n          self:error('duplicate cmdline option: '..arg[i])\n        end\n        cmdlineOptions[arg[i]] = true\n        i = i + self:__readOption__(params, arg, i)\n      else\n        nArgument = nArgument + 1\n        i = i + self:__readArgument__(params, arg, i, nArgument)\n      end\n    end\n  end\n\n  if doHelp then\n    self:help(arg, doMd)\n    os.exit(0)\n  end\n\n  if nArgument ~= #self.arguments then\n    self:error('not enough arguments')\n  end\n\n  if readConfig then\n    params = self:loadConfig(readConfig, params)\n  end\n\n  if saveConfig then\n    self:dumpConfig(params, saveConfig)\n  end\n\n  for k, v in pairs(params) do\n    if k:sub(1, 1) ~= '_' then\n      local K = '-' .. k\n      if not self.options[K] and self.options[k] then\n        K = k\n      end\n      local meta = self.options[K].meta\n      if meta then\n        -- check option validity\n        local isValid = true\n        local reason = nil\n\n        if meta.valid then\n          isValid, reason = meta.valid(v)\n        end\n\n        if not isValid then\n          local msg = 'invalid option -' .. k\n          if reason then\n            msg = msg .. ': ' .. reason\n          end\n          self:error(msg)\n        end\n\n        if meta.enum and not onmt.utils.Table.hasValue(meta.enum, v) then\n          self:error('option -' .. k.. ' is not in accepted values: ' .. table.concat(meta.enum, ', '))\n        end\n        if meta.structural then\n          params._structural[k] = meta.structural\n        end\n        if meta.init_only then\n          params._init_only[k] = meta.init_only\n        end\n        if meta.train_state then\n          params._train_state[k] = meta.train_state\n        end\n      end\n    end\n  end\n\n  return params\nend\n\nfunction ExtendedCmdLine:setCmdLineOptions(moduleOptions, group)\n  if group then\n    self:text('')\n    self:text('**' .. group .. ' options**')\n    self:text('')\n  end\n\n  for i = 1, #moduleOptions do\n    if type(moduleOptions[i]) == 'table' then\n      self:option(table.unpack(moduleOptions[i]))\n    else\n      self:argument(moduleOptions[i])\n    end\n  end\nend\n\nfunction ExtendedCmdLine.getModuleOpts(args, moduleOptions)\n  local moduleArgs = {}\n  for i = 1, #moduleOptions do\n    local optname = moduleOptions[i][1]\n    if optname:sub(1, 1) == '-' then\n      optname = optname:sub(2)\n    end\n    moduleArgs[optname] = args[optname]\n  end\n  return moduleArgs\nend\n\nfunction ExtendedCmdLine.getArgument(args, optName)\n  for i = 1, #args do\n    if args[i] == optName and i < #args then\n      return args[i + 1]\n    end\n  end\n  return nil\nend\n\n---------------------------------------------------------------------------------\n-- Validators\n---------------------------------------------------------------------------------\n\nlocal function buildRangeError(prefix, minValue, maxValue)\n  local err = 'the ' .. prefix .. ' should be'\n  if minValue then\n    err = err .. ' greater than ' .. minValue\n  end\n  if maxValue then\n    if minValue then\n      err = err .. ' and'\n    end\n    err = err .. ' lower than ' .. maxValue\n  end\n  return err\nend\n\n-- Check if is integer between minValue and maxValue.\nfunction ExtendedCmdLine.isInt(minValue, maxValue)\n  return function(v)\n    return (math.floor(v) == v and\n      (not minValue or v >= minValue) and\n      (not maxValue or v <= maxValue)),\n      buildRangeError('integer', minValue, maxValue)\n    end\nend\n\n-- Check if is positive integer.\nfunction ExtendedCmdLine.isUInt(maxValue)\n  return ExtendedCmdLine.isInt(0, maxValue)\nend\n\n-- Check if list of positive integers.\nfunction ExtendedCmdLine.listUInt(v)\n  local sv = tostring(v)\n  local p = 1\n\n  while true do\n    local q\n    p, q = sv:find('%d+',p)\n    if q == #sv then\n      return true\n    end\n    if not p or sv:sub(q+1,q+1) ~= ',' then\n      return false\n    end\n    p = q+2\n  end\nend\n\n-- Check if value between minValue and maxValue.\nfunction ExtendedCmdLine.isFloat(minValue, maxValue)\n  return function(v)\n    return (type(v) == 'number' and\n      (not minValue or v >= minValue) and\n      (not maxValue or v <= maxValue)),\n      buildRangeError('number', minValue, maxValue)\n    end\nend\n\n-- Check if non empty.\nfunction ExtendedCmdLine.nonEmpty(v)\n  return v and v ~= '', 'the argument should not be empty'\nend\n\n-- Check if the corresponding file exists.\nfunction ExtendedCmdLine.fileExists(v)\n  return path.exists(v), 'the file should exist'\nend\n\n-- Check non set or if the corresponding file exists.\nfunction ExtendedCmdLine.fileNullOrExists(v)\n  return v == '' or ExtendedCmdLine.fileExists(v), 'if set, the file should exist'\nend\n\n-- Check it is a directory and some file exists\nfunction ExtendedCmdLine.dirStructure(files)\n  return function(v)\n    for _,f in ipairs(files) do\n      if not path.exists(v..\"/\"..f) then\n        return false, 'the directory should exist'\n      end\n    end\n    return true\n  end\nend\n\nreturn ExtendedCmdLine\n"
  },
  {
    "path": "opennmt/utils/Features.lua",
    "content": "-- tds is lazy loaded.\nlocal tds\n\n--[[ Separate words and features (if any). ]]\nlocal function extract(tokens)\n  local words = {}\n  local features = {}\n  local numFeatures = nil\n\n  for t = 1, #tokens do\n    local field = onmt.utils.String.split(tokens[t], '￨')\n    local word = field[1]\n\n    if word:len() > 0 then\n      table.insert(words, word)\n\n      if numFeatures == nil then\n        numFeatures = #field - 1\n      else\n        assert(#field - 1 == numFeatures,\n               'all words must have the same number of features')\n      end\n\n      if #field > 1 then\n        for i = 2, #field do\n          if features[i - 1] == nil then\n            features[i - 1] = {}\n          end\n          table.insert(features[i - 1], field[i])\n        end\n      end\n    end\n  end\n  return words, features, numFeatures or 0\nend\n\n--[[ Reverse operation: attach features to tokens. ]]\nlocal function annotate(tokens, features)\n  if not features or #features == 0 then\n    return tokens\n  end\n\n  for i = 1, #tokens do\n    for j = 1, #features do\n      tokens[i] = tokens[i] .. '￨' .. features[j][i]\n    end\n  end\n\n  return tokens\nend\n\n--[[ Check that data contains the expected number of features. ]]\nlocal function check(label, dicts, data)\n  local expected = #dicts\n  local got = 0\n  if data ~= nil then\n    got = #data\n  end\n\n  assert(expected == got, \"expected \" .. expected .. \" \" .. label .. \" features, got \" .. got)\nend\n\n--[[ Generate source sequences from labels. ]]\nlocal function generateSource(dicts, src, cdata)\n  check('source', dicts, src)\n\n  local srcId\n  if cdata then\n    if not tds then\n      tds = require('tds')\n    end\n    srcId = tds.Vec()\n  else\n    srcId = {}\n  end\n\n  for j = 1, #dicts do\n    srcId[j] = dicts[j]:convertToIdx(src[j], onmt.Constants.UNK_WORD)\n  end\n\n  return srcId\nend\n\n--[[ Generate target sequences from labels. ]]\nlocal function generateTarget(dicts, tgt, cdata, shift_feature)\n  check('target', dicts, tgt)\n\n  -- back compatibility\n  shift_feature = shift_feature or 1\n  local tgtId\n  if cdata then\n    if not tds then\n      tds = require('tds')\n    end\n    tgtId = tds.Vec()\n  else\n    tgtId = {}\n  end\n\n  for j = 1, #dicts do\n    -- if shift_feature then target features are shifted relative to the target words.\n    -- Use EOS tokens as a placeholder.\n    table.insert(tgt[j], 1, onmt.Constants.BOS_WORD)\n    if shift_feature == 1 then\n      table.insert(tgt[j], 1, onmt.Constants.EOS_WORD)\n    else\n      table.insert(tgt[j], onmt.Constants.EOS_WORD)\n    end\n    tgtId[j] = dicts[j]:convertToIdx(tgt[j], onmt.Constants.UNK_WORD)\n    table.remove(tgt[j], 1)\n    if shift_feature == 1 then\n      table.remove(tgt[j], 1)\n    else\n      table.remove(tgt[j])\n    end\n  end\n\n  return tgtId\nend\n\nreturn {\n  extract = extract,\n  annotate = annotate,\n  generateSource = generateSource,\n  generateTarget = generateTarget\n}\n"
  },
  {
    "path": "opennmt/utils/FileReader.lua",
    "content": "local FileReader = torch.class(\"FileReader\")\n\nfunction FileReader:__init(filename)\n  self.file = assert(io.open(filename, \"r\"))\nend\n\n--[[ Read next line in the file and split it on spaces. If EOF is reached, returns nil. ]]\nfunction FileReader:next()\n  local line = self.file:read()\n\n  if line == nil then\n    return nil\n  end\n\n  local sent = {}\n  for word in line:gmatch'([^%s]+)' do\n    table.insert(sent, word)\n  end\n\n  return sent\nend\n\nfunction FileReader:close()\n  self.file:close()\nend\n\nreturn FileReader\n"
  },
  {
    "path": "opennmt/utils/Logger.lua",
    "content": "--[[ Logger is a class used for maintaining logs in a log file.\n--]]\nlocal Logger = torch.class('Logger')\n\nlocal options = {\n  {\n    '-log_file', '',\n    [[Output logs to a file under this path instead of stdout.]]\n  },\n  {\n    '-disable_logs', false,\n    [[If set, output nothing.]]\n  },\n  {\n    '-log_level', 'INFO',\n    [[Output logs at this level and above.]],\n    {\n      enum = {'DEBUG', 'INFO', 'WARNING', 'ERROR'}\n    }\n  }\n}\n\nfunction Logger.declareOpts(cmd)\n  cmd:setCmdLineOptions(options, 'Logger')\nend\n\n--[[ Construct a Logger object.\n\nParameters:\n  * `logFile` - Outputs logs to a file under this path instead of stdout. ['']\n  * `disableLogs` - If = true, output nothing. [false]\n  * `logLevel` - Outputs logs at this level and above. Possible options are: DEBUG, INFO, WARNING and ERROR. ['INFO']\n\nExample:\n\n    logger = onmt.utils.Logger.new('log.txt')\n    logger:info('%s is an extension of OpenNMT.', 'Im2Text')\n    logger:shutDown()\n\n]]\nfunction Logger:__init(logFile, disableLogs, logLevel)\n  logFile = logFile or ''\n  disableLogs = disableLogs or false\n  logLevel = logLevel or 'INFO'\n\n  self.mute = (logFile:len() > 0)\n  if disableLogs then\n    self:setVisibleLevel('ERROR')\n  else\n    self:setVisibleLevel(logLevel)\n  end\n  local openMode = 'w'\n  local f = io.open(logFile, 'r')\n  if f then\n    f:close()\n    local input = nil\n    while not input do\n      print('Logging file exits. Overwrite(o)? Append(a)? Abort(q)?')\n      input = io.read()\n      if input == 'o' or input == 'O' then\n        openMode = 'w'\n      elseif input == 'a' or input == 'A' then\n        openMode = 'a'\n      elseif input == 'q' or input == 'Q' then\n        os.exit()\n      else\n        openMode = 'a'\n      end\n    end\n  end\n  if string.len(logFile) > 0 then\n    self.logFile = io.open(logFile, openMode)\n  else\n    self.logFile = nil\n  end\n  self.LEVELS = { DEBUG = 0, INFO = 1, WARNING = 2, ERROR = 3 }\nend\n\n--[[ Log a message at a specified level.\n\nParameters:\n  * `message` - the message to log.\n  * `level` - the desired message level. ['INFO']\n\n]]\nfunction Logger:log(message, level)\n  level = level or 'INFO'\n  local timeStamp = os.date('%x %X')\n  local msgFormatted = string.format('[%s %s] %s', timeStamp, level, message)\n  if (not self.mute) and self:_isVisible(level) then\n    print (msgFormatted)\n  end\n  if self.logFile and self:_isVisible(level) then\n    self.logFile:write(msgFormatted .. '\\n')\n    self.logFile:flush()\n  end\nend\n\n--[[ Log a message at 'INFO' level.\n\nParameters:\n  * `message` - the message to log. Supports formatting string.\n\n]]\nfunction Logger:info(...)\n  self:log(self:_format(...), 'INFO')\nend\n\n--[[ Log a message at 'WARNING' level.\n\nParameters:\n  * `message` - the message to log. Supports formatting string.\n\n]]\nfunction Logger:warning(...)\n  self:log(self:_format(...), 'WARNING')\nend\n\n--[[ Log a message at 'ERROR' level.\n\nParameters:\n  * `message` - the message to log. Supports formatting string.\n\n]]\nfunction Logger:error(...)\n  self:log(self:_format(...), 'ERROR')\nend\n\n--[[ Log a message at 'DEBUG' level.\n\nParameters:\n  * `message` - the message to log. Supports formatting string.\n\n]]\nfunction Logger:debug(...)\n  self:log(self:_format(...), 'DEBUG')\nend\n\n--[[ Log a message as exactly it is.\n\nParameters:\n  * `message` - the message to log. Supports formatting string.\n\n]]\nfunction Logger:writeMsg(...)\n  local msg = self:_format(...)\n  if (not self.mute) and self:_isVisible('WARNING') then\n    io.write(msg)\n  end\n  if self.logFile and self:_isVisible('WARNING') then\n    self.logFile:write(msg)\n    self.logFile:flush()\n  end\nend\n\n--[[ Set the visible message level. Lower level messages will be muted.\n\nParameters:\n  * `level` - 'DEBUG', 'INFO', 'WARNING' or 'ERROR'.\n\n]]\nfunction Logger:setVisibleLevel(level)\n  assert (level == 'DEBUG' or level == 'INFO' or\n          level == 'WARNING' or level == 'ERROR')\n  self.level = level\nend\n\n-- Private function for comparing level against visible level.\n-- `level` - 'DEBUG', 'INFO', 'WARNING' or 'ERROR'.\nfunction Logger:_isVisible(level)\n  self.level = self.level or 'INFO'\n  return self.LEVELS[level] >= self.LEVELS[self.level]\nend\n\nfunction Logger:_format(...)\n  if #table.pack(...) == 1 then\n    return ...\n  else\n    return string.format(...)\n  end\nend\n\n--[[ Deconstructor. Close the log file.\n]]\nfunction Logger:shutDown()\n  if self.logFile then\n    self.logFile:close()\n  end\nend\n\nreturn Logger\n"
  },
  {
    "path": "opennmt/utils/Memory.lua",
    "content": "local Memory = {}\n\nlocal options = {\n  {\n    '-disable_mem_optimization', false,\n    [[Disable sharing of internal buffers between clones for visualization or development.]]\n  }\n}\n\nfunction Memory.declareOpts(cmd)\n  cmd:setCmdLineOptions(options)\nend\n\n--[[ Optimize memory usage of Neural Machine Translation.\n\nParameters:\n  * `model` - a table containing encoder and decoder\n  * `batch` - a Batch object\n  * `verbose` - produce output or not\n\nExample:\n\n  local model = {}\n  model.encoder = onmt.Models.buildEncoder(...)\n  model.decoder = onmt.Models.buildDecoder(...)\n  Memory.optimize(model, batch, verbose)\n\n]]\nfunction Memory.optimize(model, batch, verbose)\n\n  if verbose then\n    _G.logger:info('Preparing memory optimization...')\n  end\n\n  -- Prepare memory optimization\n  local memoryOptimizer = onmt.utils.MemoryOptimizer.new(model.models)\n\n  -- Batch of one single word since we optimize the first clone.\n  local realSizes = { sourceLength = batch.sourceLength, targetLength = batch.targetLength, uneven = batch.uneven }\n\n  batch.sourceLength = 1\n  batch.targetLength = 1\n  batch.uneven = false\n\n  model:trainNetwork(batch, true)\n\n  -- mark shared tensors\n  local sharedSize, totSize = memoryOptimizer:optimize()\n\n  if verbose then\n    _G.logger:info(' * sharing %d%% of output/gradInput tensors memory between clones', (sharedSize / totSize)*100)\n  end\n\n  -- Restore batch to be transparent for the calling code.\n  batch.sourceLength = realSizes.sourceLength\n  batch.targetLength = realSizes.targetLength\n  batch.uneven = realSizes.uneven\nend\n\nreturn Memory\n"
  },
  {
    "path": "opennmt/utils/MemoryOptimizer.lua",
    "content": "--[[ MemoryOptimizer is a class used for optimizing memory usage\n--]]\nlocal MemoryOptimizer = torch.class('MemoryOptimizer')\n\n-- We cannot share the output of these modules as they use it in their backward pass.\nlocal protectOutput = {\n  'nn.Sigmoid',\n  'nn.SoftMax',\n  'nn.Tanh'\n}\n\n-- We cannot share the input of these modules as they use it in their backward pass.\nlocal protectInput = {\n  'nn.Linear',\n  'nn.JoinTable',\n  'onmt.JoinReplicateTable',\n  'nn.CMulTable',\n  'nn.MM'\n}\n\nlocal function contains(list, m)\n  for i = 1, #list do\n    if torch.typename(m) == list[i] then\n      return true\n    end\n  end\n  return false\nend\n\nlocal function tensorIncluded(t, l)\n  if torch.isTensor(l) then\n    return torch.pointer(t:storage()) == torch.pointer(l:storage())\n  elseif torch.type(l) == 'table' then\n    for _, m in ipairs(l) do\n      if tensorIncluded(t, m) then\n        return true\n      end\n    end\n  end\n  return false\nend\n\n-- We cannot share a tensor if it is exposed or coming from outside of the net\n-- otherwise we could generate side-effects.\nlocal function canShare(t, net, protected)\n  if torch.isTensor(t) and t:storage() then\n    if not tensorIncluded(t, net.gradInput) and not tensorIncluded(t, net.output) and not tensorIncluded(t, protected) then\n      return true\n    end\n  elseif torch.type(t) == 'table' then\n    for _, m in ipairs(t) do\n      if not canShare(m, net, protected) then\n        return false\n      end\n    end\n    return true\n  end\n  return false\nend\n\nlocal function getSize(t, mempool)\n  local size = 0\n  if torch.isTensor(t) then\n    if t:storage() then\n      if not mempool[torch.pointer(t:storage())] then\n        mempool[torch.pointer(t:storage())] = t:storage():size()*t:elementSize()\n        return mempool[torch.pointer(t:storage())]\n      end\n    end\n  elseif torch.type(t) == 'table' then\n    for _, m in ipairs(t) do\n      size = size + getSize(m, mempool)\n    end\n  end\n  return size\nend\n\n-- Convenience function to register a network to optimize.\nlocal function registerNet(store, net, base)\n  store.net = net\n  store.base = base\n  store.forward = net.forward\n  net.forward = function(network, input)\n    store.input = input\n    return store.forward(network, input)\n  end\n  store.backward = net.backward\n  net.backward = function(network, input, gradOutput)\n    store.gradOutput = gradOutput\n    return store.backward(network, input, gradOutput)\n  end\n\n  -- Add a wrapper around updateOutput to catch the module input.\n  net:apply(function (m)\n    local updateOutput = m.updateOutput\n    m.updateOutput = function (mod, input)\n      mod.input = input\n      return updateOutput(mod, input)\n    end\n  end)\nend\n\n--[[ Construct a MemoryOptimizer object. In this function, forward and backward function will\n--  be overwrited to record input and gradOutput in order to determine which tensors can be shared.\n\nParameters:\n  * `modules` - a list of modules to optimize.\n\nExample:\n\n  local memoryOptimizer = onmt.utils.MemoryOptimizer.new(model) -- prepare memory optimization.\n  model:forward(...) -- initialize output tensors\n  model:backward(...) -- intialize gradInput tensors\n  memoryOptimizer.optimize(model) -- actual optimization by marking shared tensors\n\n]]\nfunction MemoryOptimizer:__init(modules)\n  self.modelDesc = {}\n\n  for name, mod in pairs(modules) do\n    self.modelDesc[name] = {}\n\n    if torch.isTypeOf(mod, 'onmt.Sequencer') then\n      -- If the module directly contains a network, take the first clone.\n      self.modelDesc[name][1] = {}\n      registerNet(self.modelDesc[name][1], mod:net(1), mod.network)\n    elseif mod.modules then\n      -- Otherwise, look in submodules instead.\n      local i = 1\n      mod:apply(function(m)\n        if m.network then\n          self.modelDesc[name][i] = {}\n          registerNet(self.modelDesc[name][i], m:net(1), m.network)\n          i = i + 1\n        end\n      end)\n    end\n  end\nend\n\n--[[ Enable memory optimization by marking tensors to share. Note that the modules must have been initialized\n-- by calling forward() and backward() before calling this function and after calling the MemoryOptimizer constructor.\n\nReturns:\n  1. `sharedSize` - shared tensor size\n  2. `totSize` - total tensor size\n]]\nfunction MemoryOptimizer:optimize()\n  local totSize = 0\n  local sharedSize = 0\n  for _, desc in pairs(self.modelDesc) do\n    for i = 1, #desc do\n      local net = desc[i].net\n      local base = desc[i].base\n      local mempool = {}\n\n      -- Some modules are using output when performing updateGradInput so we cannot share these.\n      local protectedOutput = { desc[i].input }\n      net:apply(function(m)\n        if contains(protectOutput, m) then\n          table.insert(protectedOutput, m.output)\n        end\n        if contains(protectInput, m) then\n          table.insert(protectedOutput, m.input)\n        end\n      end)\n\n      local globalIdx = 1\n      local idx = 1\n\n      local gradInputMap = {}\n      local outputMap = {}\n\n      -- Go over the network to determine which tensors can be shared.\n      net:apply(function(m)\n        local giSize = getSize(m.gradInput, mempool)\n        local oSize = getSize(m.output, mempool)\n        totSize = totSize + giSize\n        totSize = totSize + oSize\n        if canShare(m.gradInput, net, desc[i].gradOutput) then\n          sharedSize = sharedSize + giSize\n          m.gradInputSharedIdx = idx\n          gradInputMap[globalIdx] = idx\n          idx = idx + 1\n        end\n        if canShare(m.output, net, protectedOutput) then\n          sharedSize = sharedSize + oSize\n          m.outputSharedIdx = idx\n          outputMap[globalIdx] = idx\n          idx = idx + 1\n        end\n\n        -- Remove the wrapper around updateOutput to catch the module input.\n        m.updateOutput = nil\n        m.input = nil\n\n        globalIdx = globalIdx + 1\n      end)\n\n      globalIdx = 1\n\n      -- Mark shareable tensors in the base network.\n      base:apply(function (m)\n        if gradInputMap[globalIdx] then\n          m.gradInputSharedIdx = gradInputMap[globalIdx]\n        end\n        if outputMap[globalIdx] then\n          m.outputSharedIdx = outputMap[globalIdx]\n        end\n        globalIdx = globalIdx + 1\n      end)\n\n      -- Restore function on network backward/forward interception input.\n      net.backward = nil\n      net.forward = nil\n    end\n  end\n  return sharedSize, totSize\nend\n\nreturn MemoryOptimizer\n"
  },
  {
    "path": "opennmt/utils/Parallel.lua",
    "content": "--[[\n  This file provides generic parallel class - allowing to run functions\n  in different threads and on different GPU\n]]--\n\nlocal Parallel = {\n  _pool = nil,\n  count = 1,\n  gradBuffer = torch.Tensor()\n}\n\n-- Synchronizes the current stream on dst device with src device. This is only\n-- necessary if we are not on the default stream\nlocal function waitForDevice(dst, src)\n   local stream = cutorch.getStream()\n   if stream ~= 0 then\n      cutorch.streamWaitForMultiDevice(dst, stream, { [src] = {stream} })\n   end\nend\n\nfunction Parallel.getCounter()\n  return Parallel._tds.AtomicCounter()\nend\n\nfunction Parallel.gmutexId()\n  return Parallel._gmutex:id()\nend\n\nfunction Parallel.init(opt)\n  if onmt.utils.Cuda.activated then\n    Parallel.count = onmt.utils.Cuda.gpuCount()\n    Parallel.gradBuffer = onmt.utils.Cuda.convert(Parallel.gradBuffer)\n    Parallel._tds = require('tds')\n\n    if Parallel.count > 1 then\n      local globalLogger = _G.logger\n      local globalProfiler = _G.profiler\n      local threads = require('threads')\n      threads.Threads.serialization('threads.sharedserialize')\n      Parallel._gmutex = threads.Mutex()\n      Parallel._pool = threads.Threads(\n        Parallel.count,\n        function()\n          require('cunn')\n          require('nngraph')\n          require('opennmt.init')\n          _G.threads = require('threads')\n        end,\n        function(threadid)\n          _G.logger = globalLogger\n          _G.profiler = globalProfiler\n          onmt.utils.Cuda.init(opt, threadid)\n        end\n      ) -- dedicate threads to GPUs\n      Parallel._pool:specific(true)\n    end\n\n    if Parallel.count > 1 and not opt.no_nccl and not opt.async_parallel then\n      -- check if we have nccl installed\n      local ret\n      ret, Parallel.usenccl = pcall(require, 'nccl')\n      if not ret then\n        _G.logger:warning(\"For improved efficiency with multiple GPUs, consider installing nccl\")\n        Parallel.usenccl = nil\n      elseif os.getenv('CUDA_LAUNCH_BLOCKING') == '1' then\n        _G.logger:warning(\"CUDA_LAUNCH_BLOCKING set - cannot use nccl\")\n        Parallel.usenccl = nil\n      end\n    end\n\n  end\nend\n\n--[[ Launch function in parallel on different threads. ]]\nfunction Parallel.launch(closure, endCallback)\n  endCallback = endCallback or function() end\n  for j = 1, Parallel.count do\n    if Parallel._pool == nil then\n      endCallback(closure(j))\n    else\n      Parallel._pool:addjob(j, function() return closure(j) end, endCallback)\n    end\n  end\n  if Parallel._pool then\n    Parallel._pool:synchronize()\n  end\nend\n\n--[[ Accumulate the gradient parameters from the different parallel threads. ]]\nfunction Parallel.accGradParams(gradParams, batches)\n  if Parallel.count > 1 then\n    for h = 1, #gradParams[1] do\n      local inputs = { gradParams[1][h] }\n      for j = 2, #batches do\n        if not Parallel.usenccl then\n          -- TODO - this is memory costly since we need to clone full parameters from one GPU to another\n          -- to avoid out-of-memory, we can copy/add by batch\n\n         -- Synchronize before and after copy to ensure that it doesn't overlap\n         -- with this add or previous adds\n          waitForDevice(onmt.utils.Cuda.gpuIds[j], onmt.utils.Cuda.gpuIds[1])\n          local remoteGrads = onmt.utils.Tensor.reuseTensor(Parallel.gradBuffer, gradParams[j][h]:size())\n          remoteGrads:copy(gradParams[j][h])\n          waitForDevice(onmt.utils.Cuda.gpuIds[1], onmt.utils.Cuda.gpuIds[j])\n          gradParams[1][h]:add(remoteGrads)\n        else\n          table.insert(inputs, gradParams[j][h])\n        end\n      end\n      if Parallel.usenccl then\n        Parallel.usenccl.reduce(inputs, nil, true)\n      end\n    end\n  end\nend\n\n-- [[ In async mode, sync the parameters from all replica to master replica. ]]\nfunction Parallel.updateAndSync(masterParams, replicaGradParams, replicaParams, gradBuffer, masterGPU, gmutexId)\n  -- Add a mutex to avoid competition while accessing shared buffer and while updating parameters.\n  local mutex = _G.threads.Mutex(gmutexId)\n  mutex:lock()\n  local device = cutorch.getDevice()\n  cutorch.setDevice(masterGPU)\n  for h = 1, #replicaGradParams do\n    waitForDevice(device, masterGPU)\n    local remoteGrads = onmt.utils.Tensor.reuseTensor(gradBuffer, replicaGradParams[h]:size())\n    remoteGrads:copy(replicaGradParams[h])\n    waitForDevice(masterGPU, device)\n    masterParams[h]:add(remoteGrads)\n  end\n  cutorch.setDevice(device)\n  for h = 1, #replicaGradParams do\n    replicaParams[h]:copy(masterParams[h])\n    waitForDevice(device, masterGPU)\n  end\n  mutex:unlock()\nend\n\n--[[ Sync parameters from main model to different parallel threads. ]]\nfunction Parallel.syncParams(params)\n  if Parallel.count > 1 then\n    if not Parallel.usenccl then\n      for j = 2, Parallel.count do\n        for h = 1, #params[1] do\n          params[j][h]:copy(params[1][h])\n        end\n        waitForDevice(onmt.utils.Cuda.gpuIds[j], onmt.utils.Cuda.gpuIds[1])\n      end\n    else\n      for h = 1, #params[1] do\n        local inputs = { params[1][h] }\n        for j = 2, Parallel.count do\n          table.insert(inputs, params[j][h])\n        end\n        Parallel.usenccl.bcast(inputs, true, 1)\n      end\n    end\n  end\nend\n\nreturn Parallel\n"
  },
  {
    "path": "opennmt/utils/Profiler.lua",
    "content": "--[[ Profile is a class used for generating profiling of a training\n--]]\nlocal Profiler = torch.class('Profiler')\n\nlocal options = {\n  {\n    '-profiler', false,\n    [[Generate profiling logs.]]\n  }\n}\n\nfunction Profiler.declareOpts(cmd)\n  cmd:setCmdLineOptions(options)\nend\n\n--[[ Profiler object\n\nTo avoid concurrency problem for parallel processing, each thread should have its own Profiler.\nProfiles can be embedded.\n\nParameters:\n  * `doProfile` - enable profiling\n\nDocumentation:\n  Profile is recording/aggregating time spent in sections. Sections have hierarchical structure.\n  A section is opened with `P:start(\"name\")` and closed with `P:close(\"name\")`.\n  Start and Stop command can be stacked: `P:stop(\"b\"):start(\"a\")` or combined: `P:start(\"a.b\")`\n\nExample:\n    -- global profiler initialization\n    globalProfiler = onmt.utils.Profiler.new(opt)\n\n    -- thread-specific profiler\n      _G.profiler = onmt.utils.Profiler.new(opt)\n\n      _G.profiler:reset()\n\n      _G.profiler:start(\"encoder\")\n      [...]\n      _G.profiler:stop(\"encoder\"):start(\"decoder\")\n      [...]\n      _G.profiler:stop(\"decoder\")\n\n      local profile = _G.profiler.dump()\n\n    -- adds up thread profile\n    globalProfiler:add(profile)\n\n    Logger:info(globalProfiler:log())\n\n]]\nfunction Profiler:__init(opt)\n  if type(opt) == 'table' then opt=opt.profiler end\n  if not opt then\n    self.disable = true\n  end\n  self:reset()\nend\n\n-- Reset Profiler.\nfunction Profiler:reset()\n  self.profiles = {}\n  self.timers = {}\n  self.stack = {}\nend\n\n-- Start recording a section.\nfunction Profiler:start(name)\n  if self.disable then return self end\n  -- Synchronize current operations on cuda.\n  onmt.utils.Cuda.synchronize()\n  -- if section is multiple, decompose\n  local pos = name:find(\"%.\")\n  if pos then\n    self:start(name:sub(1, pos-1))\n    self:start(name:sub(pos+1))\n  else\n    if not self.timers[name] then self.timers[name] = {} end\n    table.insert(self.timers[name], torch.Timer())\n    table.insert(self.stack, name)\n  end\n  return self\nend\n\n-- Stop recording a section.\nfunction Profiler:stop(name)\n  if self.disable then return self end\n  -- Synchronize current operations on cuda.\n  onmt.utils.Cuda.synchronize()\n  -- if section is multiple, decompose\n  local pos = name:find(\"%.\")\n  if pos then\n    self:stop(name:sub(pos+1))\n    self:stop(name:sub(1, pos-1))\n  else\n    assert(self.stack[#self.stack] == name, 'Invalid profiler stop action: '..name)\n    local path = table.concat(self.stack, \".\")\n    if not self.profiles[path] then self.profiles[path] = 0 end\n    local timer = table.remove(self.timers[name])\n    self.profiles[path] = self.profiles[path] + timer:time().real\n    table.remove(self.stack)\n  end\n  return self\nend\n\n-- Dump profile.\nfunction Profiler:dump()\n  if self.disable then return end\n  return self.profiles\nend\n\n-- Aggregage profiles with a previous dump in the current namespace\nfunction Profiler:add(profile)\n  if self.disable then return end\n  local prefix = table.concat(self.stack, '.')\n  if #prefix > 0 then prefix = prefix .. '.' end\n  for name,v in pairs(profile) do\n    if not self.profiles[prefix..name] then self.profiles[prefix..name] = 0 end\n    self.profiles[prefix..name] = self.profiles[prefix..name] + v\n  end\nend\n\n-- Returns text string with log structured by sub level.\n-- e.g. train:{total:23, encoder_fwd:10, encoder_bwd:14}\nfunction Profiler:log(prefix)\n  prefix = prefix or ''\n  local t = {}\n  for name,v in pairs(self.profiles) do\n    v = string.format(\"%g\", v)\n    if name:sub(1,#prefix) == prefix then\n      local pos = #prefix + 1\n      if not name:sub(pos):find(\"%.\") then\n        local subtree = self:log(name..'.')\n        if #subtree > 0 then\n          v='{total:'..v..','..subtree..'}'\n        end\n        table.insert(t, name:sub(pos)..':'..v)\n      end\n    end\n  end\n  return table.concat(t, \",\")\nend\n\nfunction Profiler.addHook(module, name)\n  module.fwdFunc = module.forward\n  module.bwdFunc = module.backward\n  function module:forward(...)\n    _G.profiler:start(name..\".fwd\")\n    local res, context = self:fwdFunc(...)\n    _G.profiler:stop(name..\".fwd\")\n    return res, context\n  end\n  function module:backward(...)\n    _G.profiler:start(name..\".bwd\")\n    local res, context, loss = self:bwdFunc(...)\n    _G.profiler:stop(name..\".bwd\")\n    return res, context, loss\n  end\nend\n\nreturn Profiler\n"
  },
  {
    "path": "opennmt/utils/String.lua",
    "content": "--[[\n  Split `str` on string or pattern separator `sep`.\n]]\nlocal function split(str, sep)\n  local res = {}\n  local index = 1\n\n  while index <= str:len() do\n    local sepStart, sepEnd = str:find(sep, index)\n\n    local sub\n    if not sepStart then\n      sub = str:sub(index)\n      table.insert(res, sub)\n      index = str:len() + 1\n    else\n      sub = str:sub(index, sepStart - 1)\n      table.insert(res, sub)\n      index = sepEnd + 1\n      if index > str:len() then\n        table.insert(res, '')\n      end\n    end\n  end\n\n  return res\nend\n\n--[[ Remove whitespaces at the start and end of the string `s`. ]]\nlocal function strip(s)\n  return s:gsub('^%s+', ''):gsub('%s+$', '')\nend\n\n--[[ Remove initial hyphen(s). ]]\nlocal function stripHyphens(str)\n   return string.match(str, '%-*(.*)')\nend\n\n--[[ Right pad a strip with spaces. ]]\nlocal function pad(str, sz)\n   return str .. string.rep(' ', sz - #str)\nend\n\n--[[ Convenience function to test `s` for emptiness. ]]\nlocal function isEmpty(s)\n  return s == nil or s == ''\nend\n\nreturn {\n  split = split,\n  strip = strip,\n  isEmpty = isEmpty,\n  pad = pad,\n  stripHyphens = stripHyphens\n}\n"
  },
  {
    "path": "opennmt/utils/Table.lua",
    "content": "-- tds is lazy loaded.\nlocal tds\n\n--[[ Return subset of table ]]\nlocal function subrange(t, first, count)\n  local sub = {}\n  for i=first,first+count-1 do\n    sub[#sub + 1] = t[i]\n  end\n  return sub\nend\n\n--[[ Append table `src` to `dst`. ]]\nlocal function append(dst, src)\n  for i = 1, #src do\n    table.insert(dst, src[i])\n  end\nend\n\n--[[ Merge dict `src` to `dst`. ]]\nlocal function merge(dst, src)\n  for k, v in pairs(src) do\n    dst[k] = v\n  end\nend\n\nlocal function empty (self)\n  if next(self) == nil then\n    return true\n  else\n    return false\n  end\nend\n\n--[[ Reorder table `tab` based on the `index` array. ]]\nlocal function reorder(tab, index, cdata)\n  local newTab\n  if cdata then\n    if not tds then\n      tds = require('tds')\n    end\n    newTab = tds.Vec()\n    newTab:resize(#tab)\n  else\n    newTab = {}\n  end\n\n  for i = 1, #tab do\n    newTab[i] = tab[index[i]]\n  end\n\n  return newTab\nend\n\n--[[ Check if value is part of list/table. ]]\nlocal function hasValue(tab, value)\n  for _, v in ipairs(tab) do\n    if v == value then\n      return true\n    end\n  end\n  return false\nend\n\n\nreturn {\n  subrange = subrange,\n  reorder = reorder,\n  append = append,\n  merge = merge,\n  hasValue = hasValue,\n  empty = empty\n}\n"
  },
  {
    "path": "opennmt/utils/Tensor.lua",
    "content": "--[[ Recursively call `func()` on all tensors within `out`. ]]\nlocal function recursiveApply(out, func, ...)\n  local res\n  if torch.type(out) == 'table' then\n    res = {}\n    for k, v in pairs(out) do\n      res[k] = recursiveApply(v, func, ...)\n    end\n    return res\n  end\n  if torch.isTensor(out) then\n    res = func(out, ...)\n  else\n    res = out\n  end\n  return res\nend\n\n--[[ Recursively call `clone()` on all tensors within `out`. ]]\nlocal function recursiveClone(out)\n  return recursiveApply(out, function (h) return h:clone() end)\nend\n\n--[[ Recursively add `b` tensors into `a`'s. ]]\nlocal function recursiveAdd(a, b)\n  if torch.isTensor(a) then\n    a:add(b)\n  else\n    for i = 1, #a do\n      recursiveAdd(a[i], b[i])\n    end\n  end\n  return a\nend\n\nlocal function recursiveSet(dst, src)\n  if torch.isTensor(dst) then\n    dst:set(src)\n  else\n    for k, _ in ipairs(dst) do\n      recursiveSet(dst[k], src[k])\n    end\n  end\nend\n\n--[[ Clone any serializable Torch object. ]]\nlocal function deepClone(obj)\n  local mem = torch.MemoryFile(\"rw\"):binary()\n  mem:writeObject(obj)\n  mem:seek(1)\n  local clone = mem:readObject()\n  mem:close()\n  return clone\nend\n\n--[[\nReuse Tensor storage.\n\nParameters:\n\n  * `t` - the tensor to be reused\n  * `sizes` - a table or tensor of new sizes\n\nReturns: a view on zero-tensor `t`.\n\n--]]\nlocal function reuseTensor(t, sizes)\n  assert(t ~= nil, 'tensor must not be nil for it to be reused')\n\n  if torch.type(sizes) == 'table' then\n    sizes = torch.LongStorage(sizes)\n  end\n\n  return t:resize(sizes):zero()\nend\n\n--[[\nReuse all Tensors within the table with new sizes.\n\nParameters:\n\n  * `tab` - the table of tensors\n  * `sizes` - a table of new sizes\n\nReturns: a table of tensors using the same storage as `tab`.\n\n--]]\nlocal function reuseTensorTable(tab, sizes)\n  local newTab = {}\n\n  for i = 1, #tab do\n    table.insert(newTab, reuseTensor(tab[i], sizes))\n  end\n\n  return newTab\nend\n\n--[[\nInitialize a table of tensors with the given sizes.\n\nParameters:\n\n  * `size` - the number of clones to create\n  * `proto` - tensor to be clone for each index\n  * `sizes` - a table of new sizes\n\nReturns: an initialized table of tensors.\n\n--]]\nlocal function initTensorTable(size, proto, sizes)\n  local tab = {}\n\n  local base = reuseTensor(proto, sizes)\n\n  for _ = 1, size do\n    table.insert(tab, base:clone())\n  end\n\n  return tab\nend\n\n--[[\nCopy tensors from `src` reusing all tensors from `proto`.\n\nParameters:\n\n  * `proto` - the table of tensors to be reused\n  * `src` - the source table of tensors\n\nReturns: a copy of `src`.\n\n--]]\nlocal function copyTensorTable(proto, src)\n  local tab = reuseTensorTable(proto, src[1]:size())\n\n  for i = 1, #tab do\n    tab[i]:copy(src[i])\n  end\n\n  return tab\nend\n\nreturn {\n  recursiveApply = recursiveApply,\n  recursiveClone = recursiveClone,\n  recursiveAdd = recursiveAdd,\n  recursiveSet = recursiveSet,\n  deepClone = deepClone,\n  reuseTensor = reuseTensor,\n  reuseTensorTable = reuseTensorTable,\n  initTensorTable = initTensorTable,\n  copyTensorTable = copyTensorTable\n}\n"
  },
  {
    "path": "opennmt/utils/init.lua",
    "content": "local utils = {}\n\nutils.Cuda = require('opennmt.utils.Cuda')\nutils.Dict = require('opennmt.utils.Dict')\nutils.FileReader = require('opennmt.utils.FileReader')\nutils.Tensor = require('opennmt.utils.Tensor')\nutils.Table = require('opennmt.utils.Table')\nutils.String = require('opennmt.utils.String')\nutils.Memory = require('opennmt.utils.Memory')\nutils.MemoryOptimizer = require('opennmt.utils.MemoryOptimizer')\nutils.Parallel = require('opennmt.utils.Parallel')\nutils.Features = require('opennmt.utils.Features')\nutils.Logger = require('opennmt.utils.Logger')\nutils.Profiler = require('opennmt.utils.Profiler')\nutils.ExtendedCmdLine = require('opennmt.utils.ExtendedCmdLine')\nutils.CrayonLogger = require('opennmt.utils.CrayonLogger')\n\nreturn utils\n"
  },
  {
    "path": "src/cnn.lua",
    "content": "function createCNNModel()\n  local model = nn.Sequential()\n\n  -- input shape: (batch_size, 1, imgH, imgW)\n  model:add(nn.AddConstant(-128.0))\n  model:add(nn.MulConstant(1.0 / 128))\n\n  model:add(cudnn.SpatialConvolution(1, 64, 3, 3, 1, 1, 1, 1)) -- (batch_size, 64, imgH, imgW)\n  model:add(cudnn.ReLU(true))\n\n  model:add(cudnn.SpatialMaxPooling(2, 2, 2, 2)) -- (batch_size, 64, imgH/2, imgW/2)\n\n  model:add(cudnn.SpatialConvolution(64, 128, 3, 3, 1, 1, 1, 1)) -- (batch_size, 128, imgH/2, imgW/2)\n  model:add(cudnn.ReLU(true))\n\n  model:add(cudnn.SpatialMaxPooling(2, 2, 2, 2)) -- (batch_size, 128, imgH/2/2, imgW/2/2)\n\n  model:add(cudnn.SpatialConvolution(128, 256, 3, 3, 1, 1, 1, 1)) -- (batch_size, 256, imgH/2/2, imgW/2/2)\n  model:add(nn.SpatialBatchNormalization(256))\n  model:add(cudnn.ReLU(true))\n\n  model:add(cudnn.SpatialConvolution(256, 256, 3, 3, 1, 1, 1, 1)) -- (batch_size, 256, imgH/2/2, imgW/2/2)\n  model:add(cudnn.ReLU(true))\n\n  model:add(cudnn.SpatialMaxPooling(1, 2, 1, 2, 0, 0)) -- (batch_size, 256, imgH/2/2/2, imgW/2/2)\n\n  model:add(cudnn.SpatialConvolution(256, 512, 3, 3, 1, 1, 1, 1)) -- (batch_size, 512, imgH/2/2/2, imgW/2/2)\n  model:add(nn.SpatialBatchNormalization(512))\n  model:add(cudnn.ReLU(true))\n\n  model:add(cudnn.SpatialMaxPooling(2, 1, 2, 1, 0, 0)) -- (batch_size, 512, imgH/2/2/2, imgW/2/2/2)\n  model:add(cudnn.SpatialConvolution(512, 512, 3, 3, 1, 1, 1, 1)) -- (batch_size, 512, imgH/2/2/2, imgW/2/2/2)\n  model:add(nn.SpatialBatchNormalization(512))\n  model:add(cudnn.ReLU(true))\n\n  -- (batch_size, 512, H, W)\n  model:add(nn.Transpose({2, 3}, {3,4})) -- (batch_size, H, W, 512)\n  model:add(nn.SplitTable(1, 3)) -- #H list of (batch_size, W, 512)\n\n  return model\nend\n"
  },
  {
    "path": "src/data.lua",
    "content": " --[[ Load data. Adapted from https://github.com/da03/Attention-OCR/blob/master/src/data_util/data_gen.py.\n --  ARGS:\n     * `imageDir`       : string. The base directory of the image path in dataPath.\n     * `dataPath`       : string. The file containing data file names and label indexes. Format per line: imagePath[Space]labelIndex. Note that the imagePath is the relative path to imageDir. LabelIndex counts from 0.\n     * `labelPath`       : string. The file containing labels. Each line corresponds to a label.\n     * `maxImageHeight` : int. Maximum image height. Default: unlimited.\n     * `maxImageWidth`  : int. Maximum image width. Default: unlimited.\n     * `maxNumTokens`   : int. Maximum number of output tokens. Default: unlimited.\n --]]\nrequire 'image'\nrequire 'paths'\nrequire 'class'\nlocal tds = require('tds')\n\nlocal DataLoader = torch.class('DataLoader')\n\nfunction DataLoader:__init(imageDir, dataPath, labelPath, maxImageHeight, maxImageWidth, maxNumTokens)\n  self.imageDir = imageDir\n  self.labelPath = labelPath\n  self.maxImageHeight = maxImageHeight or math.huge\n  self.maxImageWidth = maxImageWidth or math.huge\n  self.maxNumTokens = maxNumTokens or math.huge\n\n  local file, err = io.open(dataPath, \"r\")\n  if err then\n    file, err = io.open(paths.concat(imageDir, dataPath), \"r\")\n    if err then\n      _G.logger:error('Data file %s not found', dataPath)\n      os.exit()\n    end\n  end\n  self.lines = tds.Hash()\n  local idx = 0\n  for line in file:lines() do\n    idx = idx + 1\n    if idx % 1000000 == 0 then\n      _G.logger:info ('%d lines read', idx)\n    end\n    local imagePath, label = unpack(line:split('[%s]+'))\n    self.lines[idx] = tds.Vec({imagePath, label})\n  end\n  self.perm = torch.range(1, #self.lines)\n  self.cursor = 1\n  self.buffer = {}\n  collectgarbage()\nend\n\nfunction DataLoader:shuffle()\n  local perm = torch.randperm(#self.lines)\n  self.lines = onmt.utils.Table.reorder(self.lines, perm)\n  self.perm = self.perm:index(1, perm:type('torch.LongTensor'))\nend\n\nfunction DataLoader:load(dataPath)\n  assert(paths.filep(dataPath), string.format('Data file %s does not exist!', dataPath))\n  local s\n  self.perm, self.cursor, self.buffer, self.epoch, s = table.unpack(torch.load(dataPath))\n  self.lines = onmt.utils.Table.reorder(self.lines, self.perm)\n  torch.setRNGState(s)\nend\n\nfunction DataLoader:save(dataPath)\n  local s = torch.getRNGState()\n  torch.save(dataPath, {self.perm, self.cursor, self.buffer, self.epoch, s})\nend\n\nfunction DataLoader:size()\n  return #self.lines\nend\n\nfunction DataLoader:nextBatch(batchSize)\n  while true do -- accumulate samples of the same size in self.buffer until batchSize or the last data example is reached\n    if self.cursor > #self.lines then\n      break\n    end\n    local imagePath = self.lines[self.cursor][1]\n    local status, imageData = pcall(image.load, paths.concat(self.imageDir, imagePath))\n    if not status then\n      self.cursor = self.cursor + 1\n      _G.logger:warning('Fails to read image file %s', imagePath)\n    else\n      local labelIndex = self.lines[self.cursor][2]\n      local tokenIds = labelIndexToTokenIds(labelIndex, self.labelPath)\n      self.cursor = self.cursor + 1\n      imageData = 255.0*image.rgb2y(imageData) -- convert to greyscale\n      local imageHeight, imageWidth = imageData:size(2), imageData:size(3)\n      if #tokenIds > self.maxNumTokens + 2 then -- truncate target sequence\n        _G.logger:warning('Image %s\\'s target sequence is too long, will be truncated. Consider using a larger maxNumTokens', imagePath)\n        local tokenIdsTemp = {}\n        for i = 1, self.maxNumTokens + 2 do\n          tokenIdsTemp[i] = tokenIds[i]\n        end\n        tokenIds = tokenIdsTemp\n      end\n      if imageHeight <= self.maxImageHeight and imageWidth <= self.maxImageWidth then\n        if self.buffer[imageWidth] == nil then\n          self.buffer[imageWidth] = {}\n        end\n        if self.buffer[imageWidth][imageHeight] == nil then\n          self.buffer[imageWidth][imageHeight] = {}\n        end\n        table.insert(self.buffer[imageWidth][imageHeight], {imageData, tokenIds, imagePath})\n        if #self.buffer[imageWidth][imageHeight] >= batchSize then\n          local images = torch.Tensor(batchSize, 1, imageHeight, imageWidth)\n          local maxTargetLength = -math.huge\n          local imagePaths = {}\n          local offset = #self.buffer[imageWidth][imageHeight] - batchSize\n          for i = 1, batchSize do\n            imagePaths[i] = self.buffer[imageWidth][imageHeight][i+offset][3]\n            images[i]:copy(self.buffer[imageWidth][imageHeight][i+offset][1])\n            maxTargetLength = math.max(maxTargetLength, #self.buffer[imageWidth][imageHeight][i+offset][2])\n          end\n          -- targetInput: used as input to decoder. SOS, tokenId1, tokenId2, ..., tokenIdN\n          local targetInput = torch.IntTensor(batchSize, maxTargetLength-1):fill(onmt.Constants.PAD)\n          -- targetOutput: used for comparing against decoder's output. tokenId1, tokenId2, ..., tokenIdN, EOS\n          local targetOutput = torch.IntTensor(batchSize, maxTargetLength-1):fill(onmt.Constants.PAD)\n          local numNonzeros = 0\n          for i = 1, batchSize do\n            numNonzeros = numNonzeros + #self.buffer[imageWidth][imageHeight][i+offset][2] - 1\n            for j = 1, #self.buffer[imageWidth][imageHeight][i+offset][2]-1 do\n              targetInput[i][j] = self.buffer[imageWidth][imageHeight][i+offset][2][j]\n              targetOutput[i][j] = self.buffer[imageWidth][imageHeight][i+offset][2][j+1]\n            end\n          end\n          if offset == 0 then\n            self.buffer[imageWidth][imageHeight] = nil\n          else\n            for i = 1, batchSize do\n              self.buffer[imageWidth][imageHeight][i+offset] = nil\n            end\n          end\n\n          do return {images, targetInput, targetOutput, numNonzeros, imagePaths} end\n        end\n      else --  not (imageHeight <= self.maxImageHeight and imageWidth <= self.maxImageWidth)\n        _G.logger:warning('Image %s is too large, will be ignored. Consider using a larger maxImageWidth or maxImageHeight'%imagePath)\n      end\n    end\n  end -- cannot accumulate batchSize samples\n\n  -- find if there are any samples left in order to finish the current epoch\n  if next(self.buffer) == nil then\n    self.cursor = 1\n    collectgarbage()\n    return nil\n  end\n  local imageWidth = next(self.buffer)\n  while next(self.buffer[imageWidth]) == nil do\n    if next(self.buffer, imageWidth) == nil then\n      self.cursor = 1\n      collectgarbage()\n      return nil\n    end\n    imageWidth = next(self.buffer, imageWidth)\n  end\n  local imageHeight = next(self.buffer[imageWidth], nil)\n  local actualBatchSize = math.min(batchSize, #self.buffer[imageWidth][imageHeight])\n  local offset = math.max(0, #self.buffer[imageWidth][imageHeight]-batchSize)\n  local images = torch.Tensor(actualBatchSize, 1, imageHeight, imageWidth)\n  local maxTargetLength = -math.huge\n  local imagePaths = {}\n  for i = 1, actualBatchSize do\n    imagePaths[i] = self.buffer[imageWidth][imageHeight][i+offset][3]\n    images[i]:copy(self.buffer[imageWidth][imageHeight][i+offset][1])\n    maxTargetLength = math.max(maxTargetLength, #self.buffer[imageWidth][imageHeight][i+offset][2])\n  end\n  local targetInput = torch.IntTensor(actualBatchSize, maxTargetLength-1):fill(onmt.Constants.PAD)\n  local targetOutput = torch.IntTensor(actualBatchSize, maxTargetLength-1):fill(onmt.Constants.PAD)\n  local numNonzeros = 0\n  for i = 1, actualBatchSize do\n    numNonzeros = numNonzeros + #self.buffer[imageWidth][imageHeight][i+offset][2] - 1\n    for j = 1, #self.buffer[imageWidth][imageHeight][i+offset][2]-1 do\n      targetInput[i][j] = self.buffer[imageWidth][imageHeight][i+offset][2][j]\n      targetOutput[i][j] = self.buffer[imageWidth][imageHeight][i+offset][2][j+1]\n    end\n  end\n  if offset == 0 then\n    self.buffer[imageWidth][imageHeight] = nil\n  else\n    for i = 1, actualBatchSize do\n      self.buffer[imageWidth][imageHeight][i+offset] = nil\n    end\n  end\n  return {images, targetInput, targetOutput, numNonzeros, imagePaths}\nend\n\n-- convert labelIndex to a list of token ids\nfunction labelIndexToTokenIds(labelIndex, labelPath)\n  assert (_G.idToVocab, '_G.idToVocab must be ready before calling labelIndexToTokenIds')\n  if _G.vocabToId == nil then\n    _G.vocabToId = tds.Hash()\n    for i = 1, #_G.idToVocab do\n      _G.vocabToId[_G.idToVocab[i]] = i+4\n    end\n  end\n  if labelLines == nil then\n    labelLines = tds.Hash()\n    local labelFile, err = io.open(labelPath, \"r\")\n    if not err then\n      for line in labelFile:lines() do\n        local tokenList = (onmt.utils.String.strip(line)):split('[%s]+')\n        labelLines[#labelLines+1] = tds.Vec(tokenList)\n      end\n    end\n  end\n  local tokenIds = tds.Hash()\n  tokenIds[1] = onmt.Constants.BOS\n  if labelIndex == nil then\n    tokenIds[#tokenIds+1] = onmt.Constants.EOS\n    do return tokenIds end\n  end\n  local tokens = labelLines[tonumber(labelIndex)+1]\n  if tokens == nil then\n    tokenIds[#tokenIds+1] = onmt.Constants.EOS\n    do return tokenIds end\n  end\n  for i = 1, #tokens do\n    local token = tokens[i]\n    if _G.vocabToId[token] then\n      tokenIds[#tokenIds+1] = _G.vocabToId[token]\n    else\n      tokenIds[#tokenIds+1] = onmt.Constants.UNK -- unknown token\n    end\n  end\n  tokenIds[#tokenIds+1] = onmt.Constants.EOS\n  return tokenIds\nend\n\n-- convert targets tensor to a list of label strings\nfunction targetsTensorToLabelStrings(targets)\n  assert (targets:dim() == 2)\n  local batchSize = targets:size(1)\n  local targetLength = targets:size()[2]\n\n  local labelStrings = {}\n  for b = 1, batchSize do\n    local tokenIds = {}\n    for t = 1, targetLength do\n      local tokenId = targets[b][t]\n      if tokenId == onmt.Constants.EOS then -- ignore tokens after EOS\n        break\n      end\n      table.insert(tokenIds, tokenId)\n    end\n    local labelString = tokenIdsToLabelString(tokenIds)\n    table.insert(labelStrings, labelString)\n  end\n  return labelStrings\nend\n\n-- evaluate the edit distance error rate of the predictions\nfunction evalEditDistanceRate(goldLabelStrings, predLabelStrings)\n  assert(#goldLabelStrings == #predLabelStrings)\n\n  local totalEditDistanceRate = 0.0\n  for b = 1, #goldLabelStrings do\n    local editDistance = string.levenshtein(goldLabelStrings[b], predLabelStrings[b])\n    totalEditDistanceRate = totalEditDistanceRate + editDistance / (string.len(goldLabelStrings[b]) + string.len(predLabelStrings[b]))\n  end\n  return totalEditDistanceRate\nend\n\n-- convert a list of token ids to label string\nfunction tokenIdsToLabelString(tokenIds)\n  local labelString = tds.Vec()\n  for i = 1, #tokenIds do\n    local tokenId = tokenIds[i]\n    if tokenId == onmt.Constants.PAD or tokenId == onmt.Constants.BOS or tokenId == onmt.Constants.EOS then\n      break\n    end\n    local token = _G.idToVocab[tokenId-4]\n    if tokenId == onmt.Constants.UNK then\n      token = 'UNK'\n    end\n    assert (token, 'Make sure your target vocab size is correct!')\n    for c in token:gmatch\".\" do\n      labelString:insert(c)\n    end\n    labelString:insert(' ')\n  end\n  labelString = labelString:concat()\n  return labelString\nend\n\n\n-- https://gist.github.com/Badgerati/3261142\n-- Returns the Levenshtein distance between the two given strings\nfunction string.levenshtein(str1, str2)\n  local len1 = string.len(str1)\n  local len2 = string.len(str2)\n  local matrix = {}\n  local cost\n\n  -- quick cut-offs to save time\n  if (len1 == 0) then\n    return len2\n  elseif (len2 == 0) then\n    return len1\n  elseif (str1 == str2) then\n    return 0\n  end\n\n  -- initialise the base matrix values\n  for i = 0, len1, 1 do\n    matrix[i] = {}\n    matrix[i][0] = i\n  end\n  for j = 0, len2, 1 do\n    matrix[0][j] = j\n  end\n\n  -- actual Levenshtein algorithm\n  for i = 1, len1, 1 do\n    for j = 1, len2, 1 do\n      if (str1:byte(i) == str2:byte(j)) then\n        cost = 0\n      else\n        cost = 1\n      end\n      matrix[i][j] = math.min(matrix[i-1][j] + 1, matrix[i][j-1] + 1, matrix[i-1][j-1] + cost)\n    end\n  end\n\n  -- return the last value - this is the Levenshtein distance\n  return matrix[len1][len2]\nend\n"
  },
  {
    "path": "src/model.lua",
    "content": " --[[ Model, adapted from https://github.com/harvardnlp/seq2seq-attn/blob/master/train.lua\n--]]\nrequire 'nn'\nrequire 'cudnn'\nrequire 'optim'\nrequire 'paths'\nrequire 'src.cnn'\n\nlocal model = torch.class('WYGIWYS')\n\n-- constructor\nfunction model:__init(optim)\n  self.optim = optim\nend\n\n-- in test phase, open a file for predictions\nfunction model:setOutputDirectory(outputDir)\n  if not paths.dirp(outputDir) then\n    paths.mkdir(outputDir)\n  end\n  local outputPath = paths.concat(outputDir, 'results.txt')\n  local outputFile, err = io.open(outputPath, \"w\")\n  if err then\n    _G.logger:error('Output file %s cannot be created', outputPath)\n    os.exit(1)\n  end\n  self.outputFile = outputFile\n  self.outputPath = outputPath\nend\n\n-- load model from model_path\nfunction model:load(modelPath, config)\n  assert(paths.filep(modelPath), string.format('Model file %s does not exist!', modelPath))\n\n  local checkpoint = torch.load(modelPath)\n  local loadedModel, modelConfig = checkpoint[1], checkpoint[2]\n  self.models = {}\n  self.models.cnn = loadedModel[1]:double()\n  self.models.encoder = onmt.BiEncoder.load(loadedModel[2])\n  self.models.decoder = onmt.Decoder.load(loadedModel[3])\n  self.models.posEmbeddingFw, self.models.posEmbeddingBw = loadedModel[4]:double(), loadedModel[5]:double()\n  self.numSteps = checkpoint[3]\n  self.numSamples = checkpoint[4]\n  self.optim = checkpoint[5]\n  _G.idToVocab = checkpoint[6] -- _G.idToVocab is global\n\n  -- Load model structure parameters\n  self.config = {}\n  self.config.phase = config.phase\n  self.config.cnnFeatureSize = modelConfig.cnnFeatureSize\n  self.config.encoderNumHidden = modelConfig.encoderNumHidden\n  self.config.encoderNumLayers = modelConfig.encoderNumLayers\n  self.config.decoderNumHidden = self.config.encoderNumHidden * 2 -- the decoder rnn size is the same as the output size of biEncoder\n  self.config.decoderNumLayers = modelConfig.decoderNumLayers\n  self.config.targetVocabSize = #_G.idToVocab + 4\n  self.config.targetEmbeddingSize = modelConfig.targetEmbeddingSize\n  self.config.inputFeed = modelConfig.inputFeed\n\n  self.config.maxEncoderLengthWidth = config.maxEncoderLengthWidth or modelConfig.maxEncoderLengthWidth\n  self.config.maxEncoderLengthHeight = config.maxEncoderLengthHeight or modelConfig.maxEncoderLengthHeight\n  self.config.maxDecoderLength = config.maxDecoderLength or modelConfig.maxDecoderLength\n  self.config.batchSize = config.batch_size or modelConfig.batchSize\n  self.config.valBatchSize = config.val_batch_size or modelConfig.val_batchSize or self.config.batchSize\n\n  self.config.maxImageWidth = config.max_image_width or modelConfig.max_image_width\n  self.config.maxImageHeight = config.max_image_height or modelConfig.max_image_height\n\n  -- If we want to allow higher images, since the trained positional embeddings are valid only up to modelConfig.maxEncoderLengthHeight, we use the largest available one to initialize the invalid embdddings\n  if self.config.maxEncoderLengthHeight > modelConfig.maxEncoderLengthHeight then\n  local posEmbeddingFw = nn.LookupTable(self.config.maxEncoderLengthHeight, self.config.encoderNumLayers * self.config.encoderNumHidden * 2)\n  local posEmbeddingBw = nn.LookupTable(self.config.maxEncoderLengthHeight, self.config.encoderNumLayers * self.config.encoderNumHidden * 2)\n  for i = 1, self.config.maxEncoderLengthHeight do\n    local j = math.min(i, modelConfig.maxEncoderLengthHeight)\n    posEmbeddingFw.weight[i] = self.models.posEmbeddingFw.weight[j]\n    posEmbeddingBw.weight[i] = self.models.posEmbeddingBw.weight[j]\n  end\n  self.models.posEmbeddingFw, self.models.posEmbeddingBw = posEmbeddingFw, posEmbeddingBw\n  end\n\n  self.config.no_stress_test = config.no_stress_test\n\n  -- build model\n  self:_build()\nend\n\n-- create model with fresh parameters\nfunction model:create(config)\n  self.optim = onmt.train.Optim.new(config)\n  -- set parameters\n  self.config = {}\n  self.config.phase = config.phase\n  self.config.cnnFeatureSize = 512\n  self.config.batchSize = config.batch_size\n  self.config.valBatchSize = config.val_batch_size\n  self.config.inputFeed = config.input_feed\n  self.config.encoderNumHidden = config.encoder_num_hidden\n  self.config.encoderNumLayers = config.encoder_num_layers\n  self.config.decoderNumHidden = config.encoder_num_hidden * 2\n  self.config.decoderNumLayers = config.decoder_num_layers\n  self.config.targetEmbeddingSize = config.target_embedding_size\n  self.config.targetVocabSize = config.targetVocabSize\n  self.config.maxEncoderLengthWidth = config.maxEncoderLengthWidth\n  self.config.maxEncoderLengthHeight = config.maxEncoderLengthHeight\n  self.config.maxDecoderLength = config.maxDecoderLength\n  self.config.maxImageWidth = config.max_image_width\n  self.config.maxImageHeight = config.max_image_height\n\n  -- Create model modules\n  self.models = {}\n  -- positional embeddings\n  self.models.posEmbeddingFw = nn.LookupTable(self.config.maxEncoderLengthHeight, self.config.encoderNumLayers * self.config.encoderNumHidden * 2)\n  self.models.posEmbeddingBw = nn.LookupTable(self.config.maxEncoderLengthHeight, self.config.encoderNumLayers * self.config.encoderNumHidden * 2)\n  -- CNN model, input size: (batchSize, 1, height, width), output size: (batchSize, sequenceLength, cnnFeatureSize)\n  self.models.cnn = createCNNModel()\n  -- biLSTM encoder\n  --local encoderRnn = onmt.LSTM.new(self.config.encoderNumLayers, self.config.cnnFeatureSize, self.config.encoderNumHidden, 0.0)\n  --self.models.encoder = onmt.BiEncoder.new(config, nn.Identity())\n  local encoderConfig = {layers = self.config.encoderNumLayers, rnn_size = self.config.encoderNumHidden*2,\n    brnn=true, brnn_merge = 'concat', dropout = 0, rnn_type = 'LSTM'}\n  local encoderInputNetwork = nn.Identity()\n  encoderInputNetwork.inputSize = self.config.cnnFeatureSize\n  self.models.encoder = onmt.Factory.buildEncoder(encoderConfig, encoderInputNetwork, false)\n\n  -- decoder\n  local inputSize = self.config.targetEmbeddingSize\n  if self.config.inputFeed then\n    inputSize = inputSize + self.config.decoderNumHidden\n  end\n  local generator = onmt.Generator.new(self.config.decoderNumHidden, self.config.targetVocabSize)\n  generator:cuda()\n  local inputNetwork = onmt.WordEmbedding.new(self.config.targetVocabSize, self.config.targetEmbeddingSize)\n  inputNetwork.inputSize = self.config.targetEmbeddingSize\n  local attentionModel = onmt.GlobalAttention({global_attention='general'}, self.config.decoderNumHidden)\n  local inputFeed = 0\n  if self.config.inputFeed then\n    inputFeed = 1\n  end\n  local decoderConfig = {input_feed = inputFeed, layers = self.config.decoderNumLayers, rnn_size = self.config.decoderNumHidden,\n    residual = false, dropout = 0, dropout_input = false}\n  self.models.decoder = onmt.Decoder.new(decoderConfig, inputNetwork, generator, attentionModel)\n\n  self.numSteps = 0\n  self.numSamples = 0\n  self.config.no_stress_test = config.no_stress_test\n  self._init = true\n\n  self:_build()\nend\n\n-- build\nfunction model:_build()\n\n  -- log options\n  for k, v in pairs(self.config) do\n    _G.logger:info('%s: %s', k, v)\n  end\n\n  -- create criterion\n  self.criterion = onmt.ParallelClassNLLCriterion({self.config.targetVocabSize})\n  onmt.utils.Cuda.convert(self.criterion)\n\n  -- convert to cuda\n  self.layers = {self.models.cnn, self.models.encoder, self.models.decoder, self.models.posEmbeddingFw, self.models.posEmbeddingBw}\n  self.models.cnn:cuda()\n  for i = 2, #self.layers do\n    onmt.utils.Cuda.convert(self.layers[i])\n  end\n  onmt.utils.Cuda.convert(self.criterion)\n\n  self.contextProto = onmt.utils.Cuda.convert(torch.zeros(math.max(self.config.batchSize,self.config.valBatchSize or 0), self.config.maxEncoderLengthWidth * self.config.maxEncoderLengthHeight, 2 * self.config.encoderNumHidden))\n  self.cnnGradProto = onmt.utils.Cuda.convert(torch.zeros(self.config.maxEncoderLengthHeight, self.config.batchSize, self.config.maxEncoderLengthWidth, self.config.cnnFeatureSize))\n\n  local numParams = 0\n  self.params, self.gradParams = {}, {}\n  for i = 1, #self.layers do\n    local p, gp = self.layers[i]:getParameters()\n    if self._init then\n      p:uniform(-0.05,0.05)\n    end\n    numParams = numParams + p:size(1)\n    self.params[i] = p\n    self.gradParams[i] = gp\n  end\n  _G.logger:info('Number of parameters: %d', numParams)\n\n  if self.config.phase == 'train' then\n    self:_optimizeMemory()\n    if not self.config.no_stress_test then\n      self:_stressTest()\n    end\n  end\n  collectgarbage()\nend\n\nfunction model:_stressTest()\n  _G.logger:info('Stress Test starts')\n  local s = torch.getRNGState()\n  local images = torch.rand(self.config.batchSize, 1, self.config.maxImageHeight, self.config.maxImageWidth)\n  local targetInput = torch.IntTensor(self.config.batchSize, self.config.maxDecoderLength):fill(onmt.Constants.PAD)\n  local targetOutput = torch.IntTensor(self.config.batchSize, self.config.maxDecoderLength):fill(onmt.Constants.PAD)\n  local numNonzeros = 1\n  local inputBatch = {images, targetInput, targetOutput, numNonzeros, {}}\n  self:step(inputBatch, false, 1)\n  images = torch.rand(self.config.valBatchSize, 1, self.config.maxImageHeight, self.config.maxImageWidth)\n  targetInput = torch.IntTensor(self.config.valBatchSize, self.config.maxDecoderLength):fill(onmt.Constants.PAD)\n  targetOutput = torch.IntTensor(self.config.valBatchSize, self.config.maxDecoderLength):fill(onmt.Constants.PAD)\n  inputBatch = {images, targetInput, targetOutput, numNonzeros, {}}\n  self:step(inputBatch, true, 1, true)\n  torch.setRNGState(s)\n  _G.logger:info('Stress Test ends')\nend\n\n-- one step forward (and optionally backward)\nfunction model:step(inputBatch, isForwardOnly, beamSize, mute)\n  mute = mute or false\n  beamSize = beamSize or 1 -- default greedy decoding\n  assert (beamSize <= self.config.targetVocabSize)\n  --local images = onmt.utils.Cuda.convert(inputBatch[1])\n  local images = inputBatch[1]:cuda()\n  local targetInput = onmt.utils.Cuda.convert(inputBatch[2])\n  local targetOutput = onmt.utils.Cuda.convert(inputBatch[3])\n  local numNonzeros = inputBatch[4]\n  local imagePaths = inputBatch[5]\n\n  local batchSize = images:size(1)\n  local targetLength = targetInput:size(2)\n\n  assert(targetLength <= self.config.maxDecoderLength, string.format('maxDecoderLength (%d) < targetLength (%d)!', self.config.maxDecoderLength, targetLength))\n  -- if isForwardOnly, then re-generate the targetInput with maxDecoderLength for fair evaluation\n  if isForwardOnly then\n    local targetInputTemp = onmt.utils.Cuda.convert(torch.IntTensor(batchSize, self.config.maxDecoderLength)):fill(onmt.Constants.PAD)\n    targetInputTemp[{{}, {1,targetLength}}]:copy(targetInput)\n    targetInput = targetInputTemp\n    local targetOutputTemp = onmt.utils.Cuda.convert(torch.IntTensor(batchSize, self.config.maxDecoderLength)):fill(onmt.Constants.PAD)\n    targetOutputTemp[{{}, {1,targetLength}}]:copy(targetOutput)\n    targetOutput = targetOutputTemp\n    targetLength = self.config.maxDecoderLength\n  end\n\n  -- set phase\n  if not isForwardOnly then\n    self.models.cnn:training()\n    self.models.encoder:training()\n    self.models.decoder:training()\n    self.models.posEmbeddingFw:training()\n    self.models.posEmbeddingBw:training()\n  else\n    self.models.cnn:evaluate()\n    self.models.encoder:evaluate()\n    self.models.decoder:evaluate()\n    self.models.posEmbeddingFw:evaluate()\n    self.models.posEmbeddingBw:evaluate()\n  end\n\n  -- given parameters, evaluate loss (and optionally calculate gradients)\n  local feval = function()\n    local targetIn = targetInput:transpose(1,2)\n    local targetOut = targetOutput:transpose(1,2)\n    local cnnOutputs = self.models.cnn:forward(images) -- list of (batchSize, featureMapWidth, cnnFeatureSize)\n    for i = 1, #cnnOutputs do\n      cnnOutputs[i] = onmt.utils.Cuda.convert(cnnOutputs[i])\n    end\n    local featureMapHeight = #cnnOutputs\n    local featureMapWidth = cnnOutputs[1]:size(2)\n    local context = self.contextProto[{{1, batchSize}, {1, featureMapHeight * featureMapWidth}}]\n    local decoderBatch = Batch():setTargetInput(targetIn):setTargetOutput(targetOut)\n    decoderBatch.sourceLength = context:size(2)\n    decoderBatch.sourceSize = onmt.utils.Cuda.convert(torch.IntTensor(batchSize)):fill(context:size(2))\n    for i = 1, featureMapHeight do\n      local pos = onmt.utils.Cuda.convert(torch.zeros(batchSize)):fill(i)\n      local posEmbeddingFw  = self.models.posEmbeddingFw:forward(pos):view(batchSize, -1) -- (1, batchSize, encoderNumLayers*2*encoderNumHidden)\n      local posEmbeddingBw  = self.models.posEmbeddingBw:forward(pos):view(batchSize, -1)  -- (1, batchSize, encoderNumLayers*2*encoderNumHidden)\n      local cnnOutput = cnnOutputs[i] -- (batchSize, featureMapWidth, cnnFeatureSize)\n      local source = cnnOutput:transpose(1, 2) -- (featureMapWidth, batchSize, cnnFeatureSize)\n      local encoderBatch = Batch():setSourceInput(source)\n      local encoderStatesFw = onmt.utils.Tensor.initTensorTable(self.config.encoderNumLayers*2,\n                                                         onmt.utils.Cuda.convert(torch.Tensor()),\n                                                         { batchSize,  self.config.encoderNumHidden})\n      for k = 1, #encoderStatesFw do\n        encoderStatesFw[k]:copy(posEmbeddingFw[{{}, {(k-1)*self.config.encoderNumHidden+1, k*self.config.encoderNumHidden}}])\n      end\n      local encoderStatesBw = onmt.utils.Tensor.initTensorTable(self.config.encoderNumLayers*2,\n                                                         onmt.utils.Cuda.convert(torch.Tensor()),\n                                                         { batchSize,  self.config.encoderNumHidden})\n      for k = 1, #encoderStatesBw do\n        encoderStatesBw[k]:copy(posEmbeddingBw[{{}, {(k-1)*self.config.encoderNumHidden+1, k*self.config.encoderNumHidden}}])\n      end\n      local _, rowContext = self.models.encoder:forward(encoderBatch, encoderStatesFw, encoderStatesBw)\n      for t = 1, featureMapWidth do\n        local index = (i - 1) * featureMapWidth + t\n        context[{{}, index, {}}]:copy(rowContext[{{}, t, {}}])\n      end\n    end\n\n    -- evaluate loss (and optionally do backward)\n    local loss, numCorrect\n    numCorrect = 0\n    if isForwardOnly then\n      if self.outputFile then\n        -- Specify how to go one step forward.\n        local advancer = onmt.translate.DecoderAdvancer.new(self.models.decoder, decoderBatch, context, self.config.maxDecoderLength)\n        \n        -- Conduct beam search.\n        local beamSearcher = onmt.translate.BeamSearcher.new(advancer)\n        local results = beamSearcher:search(beamSize, 1)\n        local predTarget = onmt.utils.Cuda.convert(torch.zeros(batchSize, targetLength)):fill(onmt.Constants.PAD)\n        for b = 1, batchSize do\n          local tokens = results[b][1].tokens\n          for t = 1, math.min(#tokens, targetLength) do\n            predTarget[b][t] = tokens[t]\n          end\n        end\n        local predLabels = targetsTensorToLabelStrings(predTarget)\n        local goldLabels = targetsTensorToLabelStrings(targetOutput)\n        local editDistanceRate = evalEditDistanceRate(goldLabels, predLabels)\n        numCorrect = batchSize - editDistanceRate\n        if not mute then\n          for i = 1, #imagePaths do\n            _G.logger:info('%s\\t%s\\n', imagePaths[i], predLabels[i])\n            self.outputFile:write(string.format('%s\\t%s\\n', imagePaths[i], predLabels[i]))\n          end\n          self.outputFile:flush()\n        end\n      end\n      -- get loss\n      self.models.decoder:maskPadding()\n      loss = self.models.decoder:computeLoss(decoderBatch, nil, context, self.criterion) / batchSize\n    else -- isForwardOnly == false\n      local decoderOutputs = self.models.decoder:forward(decoderBatch, nil, context)\n      local _, gradContext, totalLoss = self.models.decoder:backward(decoderBatch, decoderOutputs, self.criterion)\n      loss = totalLoss / batchSize\n      gradContext = gradContext:contiguous():view(batchSize, featureMapHeight, featureMapWidth, -1) -- (batchSize, featureMapHeight, featureMapWidth, cnnFeatureSize)\n      local cnnGrad = self.cnnGradProto[{ {1, featureMapHeight}, {1, batchSize}, {1, featureMapWidth}, {} }]\n      for i = 1, featureMapHeight do\n        local cnnOutput = cnnOutputs[i]\n        local source = cnnOutput:transpose(1,2)\n        local pos = onmt.utils.Cuda.convert(torch.zeros(batchSize)):fill(i)\n        local posEmbeddingFw = self.models.posEmbeddingFw:forward(pos):view(batchSize, -1)\n        local posEmbeddingBw = self.models.posEmbeddingBw:forward(pos):view(batchSize, -1)\n        local encoderStatesFw = onmt.utils.Tensor.initTensorTable(self.config.encoderNumLayers*2,\n                                                           onmt.utils.Cuda.convert(torch.Tensor()),\n                                                           { batchSize,  self.config.encoderNumHidden})\n        for k = 1, #encoderStatesFw do\n          encoderStatesFw[k]:copy(posEmbeddingFw[{{}, {(k-1)*self.config.encoderNumHidden+1, k*self.config.encoderNumHidden}}])\n        end\n        local encoderStatesBw = onmt.utils.Tensor.initTensorTable(self.config.encoderNumLayers*2,\n                                                           onmt.utils.Cuda.convert(torch.Tensor()),\n                                                           { batchSize,  self.config.encoderNumHidden})\n        for k = 1, #encoderStatesBw do\n          encoderStatesBw[k]:copy(posEmbeddingBw[{{}, {(k-1)*self.config.encoderNumHidden+1, k*self.config.encoderNumHidden}}])\n        end\n        local encoderBatch = Batch():setSourceInput(source)\n        self.models.encoder:forward(encoderBatch, encoderStatesFw, encoderStatesBw)\n        local rowContextGrad, posEmbeddingGrad = self.models.encoder:backward(encoderBatch, nil, gradContext:select(2,i))\n        for t = 1, featureMapWidth do\n          cnnGrad[{i, {}, t, {}}]:copy(rowContextGrad[t])\n        end\n        local posEmbeddingGradFw = onmt.utils.Cuda.convert(torch.zeros(batchSize, self.config.encoderNumLayers*2*self.config.encoderNumHidden))\n        for k = 1, 2*self.config.encoderNumLayers do\n          posEmbeddingGradFw[{{}, {(k-1)*self.config.encoderNumHidden+1, k*self.config.encoderNumHidden}}]:copy(posEmbeddingGrad[k])\n        end\n        self.models.posEmbeddingFw:backward(pos, posEmbeddingGradFw)\n        for k = 1, 2*self.config.encoderNumLayers do\n          posEmbeddingGradFw[{{}, {(k-1)*self.config.encoderNumHidden+1, k*self.config.encoderNumHidden}}]:copy(posEmbeddingGrad[k+2*self.config.encoderNumLayers])\n        end\n        \n        self.models.posEmbeddingBw:backward(pos, posEmbeddingGradFw)\n      end\n      -- cnn\n      cnnGrad = cnnGrad:split(1, 1)\n      for i = 1, #cnnGrad do\n        cnnGrad[i] = cnnGrad[i]:contiguous():view(batchSize, featureMapWidth, -1):type('torch.CudaTensor')\n      end\n      self.models.cnn:backward(images, cnnGrad)\n      collectgarbage()\n    end\n    return loss, self.gradParams, {numNonzeros, numCorrect}\n  end\n  if not isForwardOnly then\n    -- optimizer\n    self.optim:zeroGrad(self.gradParams)\n    local loss, _, stats = feval(self.params)\n    local flagNan = false\n    for i = 1, #self.gradParams do\n      if self.gradParams[i]:ne(self.gradParams[i]):any() then\n        flagNan = true\n        _G.logger:warning('nans detected in gradients!')\n      end\n    end\n    if not flagNan then\n      self.optim:prepareGrad(self.gradParams)\n      self.optim:updateParams(self.params, self.gradParams)\n    end\n\n    return loss * batchSize, stats\n  else\n    local loss, _, stats = feval(self.params)\n    return loss * batchSize, stats\n  end\nend\n\n-- Optimize Memory Usage by sharing output and gradInput among clones\nfunction model:_optimizeMemory()\n  self.models.encoder:training()\n  self.models.decoder:training()\n  _G.logger:info('Preparing memory optimization...')\n  local memoryOptimizer = onmt.utils.MemoryOptimizer.new({self.models.encoder, self.models.decoder})\n\n  -- Initialize all intermediate tensors with a first batch.\n  local source = onmt.utils.Cuda.convert(torch.zeros(1, 1, self.config.cnnFeatureSize))\n  local targetIn = onmt.utils.Cuda.convert(torch.ones(1, 1))\n  local targetOut = targetIn:clone()\n  local batch = Batch():setSourceInput(source):setTargetInput(targetIn):setTargetOutput(targetOut)\n  self.models.encoder:forward(batch)\n  local context = onmt.utils.Cuda.convert(torch.zeros(1, 1, 2 * self.config.encoderNumHidden))\n  local decOutputs = self.models.decoder:forward(batch, nil, context)\n  decOutputs = onmt.utils.Tensor.recursiveClone(decOutputs)\n  local _, gradContext = self.models.decoder:backward(batch, decOutputs, self.criterion)\n  self.models.encoder:backward(batch, nil, gradContext)\n\n  local sharedSize, totSize = memoryOptimizer:optimize()\n  _G.logger:info(' * sharing %d%% of output/gradInput tensors memory between clones', (sharedSize / totSize)*100)\nend\n\n-- Save model to model_path\nfunction model:save(modelPath)\n  for i = 1, #self.layers do\n    self.layers[i]:clearState()\n  end\n  torch.save(modelPath, {{self.models.cnn, self.models.encoder:serialize(), self.models.decoder:serialize(), self.models.posEmbeddingFw, self.models.posEmbeddingBw}, self.config, self.numSteps, self.numSamples, self.optim, _G.idToVocab})\nend\n\n-- destructor\nfunction model:shutDown()\n  if self.outputFile then\n    self.outputFile:close()\n    _G.logger:info('Results saved to %s.', self.outputPath)\n  end\nend\n"
  },
  {
    "path": "src/train.lua",
    "content": " --[[ Training, adapted from https://github.com/harvardnlp/seq2seq-attn/blob/master/train.lua\n--]]\nrequire 'nn'\nrequire 'nngraph'\nrequire 'cunn'\nrequire 'cutorch'\nrequire 'cudnn'\nrequire 'paths'\nlocal status = pcall(function() require('opennmt.init') end)\nif not status then\n  print('OpenNMT not found. Please enter the path to OpenNMT: ')\n  local onmtPath = io.read()\n  package.path = package.path .. ';' .. paths.concat(onmtPath, '?.lua')\n  status = pcall(function() require('opennmt.init') end)\n  if not status then\n    print ('Error: opennmt not found in the specified path!')\n    os.exit(1)\n  end\nend\ntds = require 'tds'\n\nrequire 'src.model'\nrequire 'src.data'\n\nlocal cmd = onmt.utils.ExtendedCmdLine.new('src/train.lua')\n\n-- Input and Output\ncmd:text('')\ncmd:text('**Control**')\ncmd:text('')\ncmd:option('-phase', 'test', [[train or test]])\ncmd:option('-load_model', false, [[Load model from model_dir or not]])\ncmd:option('-no_stress_test', false, [[]])\ncmd:option('-gpu_id', 1, [[Which gpu to use]])\n\ncmd:text('')\ncmd:text('**Input and Output**')\ncmd:text('')\ncmd:option('-image_dir', '', [[The base directory of the image path in data-path.]])\ncmd:option('-data_path', '', [[The file containing data file names and label indexes. Format per line: image_path label_index. Note that label_index count from 0.]])\ncmd:option('-label_path', '', [[The file containing tokenized labels. Each line corresponds to a label.]])\ncmd:option('-val_data_path', '', [[The path containing validate data file names and labels. Format per line: image_path characters]])\ncmd:option('-vocab_file', '', [[Vocabulary file. A token per line.]])\ncmd:option('-model_dir', 'model', [[The directory for saving and loading model parameters (structure is not stored)]])\ncmd:option('-output_dir', 'results', [[The path to put results]])\n\n-- Logging\ncmd:text('')\ncmd:text('**Display**')\ncmd:text('')\ncmd:option('-steps_per_checkpoint', 100, [[Checkpointing (print perplexity, save model) per how many steps]])\ncmd:option('-log_path', 'log.txt', [[The path to put log]])\n\n-- Optimization\ncmd:text('')\ncmd:text('**Optimization**')\ncmd:text('')\ncmd:option('-num_epochs', 15, [[The number of whole data passes]])\ncmd:option('-batch_size', 1, [[Batch size]])\ncmd:option('-val_batch_size', 10, [[Batch size]])\n\n-- Network\ncmd:text('')\ncmd:text('**Network**')\ncmd:text('')\ncmd:option('-input_feed', false, [[Whether or not use LSTM attention decoder cell]])\ncmd:option('-encoder_num_hidden', 256, [[Number of hidden units in encoder cell]])\ncmd:option('-encoder_num_layers', 1, [[Number of hidden layers in encoder cell]])\ncmd:option('-decoder_num_layers', 1, [[Number of hidden units in decoder cell]])\ncmd:option('-target_embedding_size', 80, [[Embedding dimension for each target]])\n\n-- Other\ncmd:text('')\ncmd:text('**Other**')\ncmd:text('')\ncmd:option('-beam_size', 1, [[Beam size for decoding]])\ncmd:option('-max_num_tokens', 150, [[Maximum number of output tokens]]) -- when evaluate, this is the cut-off length.\ncmd:option('-max_image_width', 500, [[Maximum image width]]) --800/2/2/2\ncmd:option('-max_image_height', 160, [[Maximum image height]]) --80 / (2*2*2)\ncmd:option('-seed', 920110, [[Load model from model_dir or not]])\ncmd:option('-fp16', false, [[Use half-precision float on GPU]])\n\n--onmt.BiEncoder.declareOpts(cmd)\nonmt.utils.Profiler.declareOpts(cmd)\nonmt.train.Optim.declareOpts(cmd)\n\nlocal opt = cmd:parse(arg)\ntorch.manualSeed(opt.seed)\nmath.randomseed(opt.seed)\ncutorch.manualSeed(opt.seed)\n\nlocal function evaluateModel(model, valData, valBatchSize, beamSize, modelDir, epoch, epochEnd)\n  local valLoss, valNumSamples, valNumNonzeros = 0, 0, 0\n  -- Run 1 epoch on validation data\n  while true do\n    local valBatch = valData:nextBatch(valBatchSize)\n    if valBatch == nil then\n      break\n    end\n    local actualBatchSize = valBatch[1]:size(1)\n    local stepLoss, stats = model:step(valBatch, true, beamSize)\n    valLoss = valLoss + stepLoss\n    valNumSamples = valNumSamples + actualBatchSize\n    valNumNonzeros = valNumNonzeros + stats[1]\n  end -- Run 1 epoch\n  local ppl = math.exp(valLoss/valNumNonzeros)\n  if not epochEnd then\n    _G.logger:info('Epoch: %d. Step: %d. #Sample: %d. Val Perplexity: %f', epoch, model.numSteps, model.numSamples, ppl)\n  else\n    _G.logger:info('Epoch (square): %d. Step: %d. #Sample: %d. Val Perplexity: %f', epoch, model.numSteps, model.numSamples, ppl)\n  end\n  return ppl\nend\n\nlocal function saveModel(modelDir, model, trainData)\n  _G.logger:info('Saving Model')\n  local modelPath = paths.concat(modelDir, string.format('model_%d', model.numSteps))\n  local modelPathTemp = paths.concat(modelDir, '.model.tmp')\n  local modelPathLatest = paths.concat(modelDir, 'model_latest')\n  model:save(modelPath)\n  local dataPath = paths.concat(modelDir, string.format('model_%d-data', model.numSteps or 0))\n  trainData:save(dataPath)\n  _G.logger:info('Model saved to %s', modelPath)\n  os.execute(string.format('cp %s %s', modelPath, modelPathTemp))\n  os.execute(string.format('mv %s %s', modelPathTemp, modelPathLatest))\nend\n\nlocal function run(model, phase, batchSize, valBatchSize, numEpochs, trainData, valData, modelDir, stepsPerCheckpoint, beamSize, outputDir, learningRateInit, learningRateDecay)\n  local loss = 0\n  local numSamples = 0\n  local numNonzeros = 0\n\n  assert(phase == 'train' or phase == 'test', 'phase must be either train or test')\n  local isForwardOnly\n  if phase == 'train' then\n    isForwardOnly = false\n  else\n    isForwardOnly = true\n    numEpochs = 1\n    model.numSteps = 0\n    model.numSamples = 0\n    model:setOutputDirectory(outputDir)\n  end\n\n  _G.logger:info('Running...')\n  --local valLosses = {}\n  -- Run numEpochs epochs\n  if not trainData.epoch then\n    trainData.epoch = 1\n    if not isForwardOnly then\n      trainData:shuffle()\n    end\n    _G.logger:info('Learning rate: %f', model.optim.args.learning_rate)\n  end\n\n  local epoch = trainData.epoch\n  while epoch <= numEpochs do\n    -- Run 1 epoch\n    while true do\n      local trainBatch = trainData:nextBatch(batchSize)\n      if trainBatch == nil then\n        break\n      end\n      local actualBatchSize = trainBatch[1]:size(1)\n      local stepLoss, stats = model:step(trainBatch, isForwardOnly, beamSize) -- do one step\n      collectgarbage()\n      if not isForwardOnly then\n        _G.logger:info('step perplexity: %f', math.exp(stepLoss/stats[1]))\n      end\n      numNonzeros = numNonzeros + stats[1]\n      loss = loss + stepLoss\n      model.numSteps = model.numSteps + 1\n      model.numSamples = model.numSamples + actualBatchSize\n      if model.numSteps % stepsPerCheckpoint == 0 then\n        if isForwardOnly then\n          _G.logger:info('Step: %d. Number of samples: %d.', model.numSteps, model.numSamples)\n        else\n          _G.logger:info('Step: %d. #Sample: %d. Training Perplexity: %f', model.numSteps, model.numSamples, math.exp(loss/numNonzeros))\n          --_G.logger:info('Evaluating model on validation data')\n          --local ppl = evaluateModel(model, valData, valBatchSize, beamSize, modelDir, epoch)\n          saveModel(modelDir, model, trainData)\n          loss, numNonzeros = 0, 0\n          collectgarbage()\n        end\n      end\n    end -- Run 1 epoch\n    if not isForwardOnly then\n      trainData:shuffle()\n    end\n    -- After each epoch, evaluate on validation if phase is train\n    if not isForwardOnly then\n      _G.logger:info('Evaluating model on validation data')\n      local ppl = evaluateModel(model, valData, valBatchSize, beamSize, modelDir, epoch, true)\n      local learningRateOld = model.optim.args.learning_rate\n      model.optim:updateLearningRate(ppl, -math.huge)\n      local learningRate = model.optim.args.learning_rate\n      if learningRate < learningRateOld then\n        _G.logger:info('Decay learning rate to %f', learningRate)\n      end\n      saveModel(modelDir, model, trainData)\n    else -- isForwardOnly == true\n      _G.logger:info('Epoch ends. Number of samples: %d.', numSamples)\n    end\n    epoch = epoch + 1\n    trainData.epoch = epoch\n  end -- for epoch\nend -- run function\n\nlocal function main()\n  assert (opt.gpu_id > 0, 'Only support using GPU! Please specify a valid gpu_id.')\n\n  _G.profiler = onmt.utils.Profiler.new(opt.profiler)\n  _G.logger = onmt.utils.Logger.new(opt.log_path)\n  _G.logger.mute = false\n  _G.logger:info('Command Line Arguments: %s', table.concat(arg, ' ') or '')\n\n  local gpuId = opt.gpu_id\n  _G.logger:info('Using CUDA on GPU %d', gpuId)\n  cutorch.setDevice(gpuId)\n  onmt.utils.Cuda.init({gpuid=string.format('%d', gpuId), fp16 = opt.fp16})\n\n  -- Convert Options\n  opt.maxDecoderLength = opt.max_num_tokens + 1 -- since <StartOfSequence> is prepended to the sequence\n  opt.maxEncoderLengthWidth = math.floor(opt.max_image_width / 8.0) -- feature maps after CNN become 8 times smaller\n  opt.maxEncoderLengthHeight = math.floor(opt.max_image_height / 8.0) -- feature maps after CNN become 8 times smaller\n\n  -- Build Model\n  _G.logger:info('Building model')\n  local model = WYGIWYS(optim)\n  local modelPath = paths.concat(opt.model_dir, 'model_latest')\n  if opt.load_model and paths.filep(modelPath) then\n    _G.logger:info('Loading model from %s', modelPath)\n    model:load(modelPath, opt)\n  else\n    -- Load Vocabulary\n    _G.logger:info('Loading vocabulary from %s', opt.vocab_file)\n    _G.idToVocab = tds.Hash() -- vocabulary file is global\n    local file, err = io.open(opt.vocab_file, \"r\")\n    if err then\n      _G.logger:error('Vocabulary file %s does not exist!', opt.vocab_file)\n      os.exit()\n    end\n    for line in file:lines() do\n      local token = onmt.utils.String.strip(line)\n      if onmt.utils.String.isEmpty(token) then\n        token = ' '\n      end\n      _G.idToVocab[#_G.idToVocab+1] = token\n    end\n    opt.targetVocabSize = #_G.idToVocab + 4\n    _G.logger:info('Creating model with fresh parameters')\n    model:create(opt)\n  end\n\n  if not paths.dirp(opt.model_dir) then\n    paths.mkdir(opt.model_dir)\n  end\n\n  -- Load Data\n  _G.logger:info('Image directory: %s', opt.image_dir)\n  _G.logger:info('Loading %s data from %s', opt.phase, opt.data_path)\n  if opt.phase == 'train' and (not paths.filep(opt.label_path)) then\n      _G.logger:error('Label file %s does not exist!', opt.label_path)\n      os.exit(1)\n  end\n  local trainData = DataLoader(opt.image_dir, opt.data_path, opt.label_path, opt.max_image_height, opt.max_image_width, opt.max_num_tokens)\n  local dataPath = paths.concat(opt.model_dir, string.format('model_%d-data', model.numSteps or 0))\n  if opt.load_model and paths.filep(dataPath) and opt.phase == 'train' then\n    _G.logger:info('Loading data state from %s', dataPath)\n    trainData:load(dataPath)\n  end\n  _G.logger:info('Loaded')\n  local valData\n  if opt.phase == 'train' then\n    _G.logger:info('Loading validation data from %s', opt.val_data_path)\n    valData = DataLoader(opt.image_dir, opt.val_data_path, opt.label_path, opt.max_image_height, opt.max_image_width, opt.max_num_tokens)\n    _G.logger:info('Loaded')\n  end\n\n  -- Run Model\n  run(model, opt.phase, opt.batch_size, opt.val_batch_size or opt.batch_size, opt.num_epochs, trainData, valData, opt.model_dir, opt.steps_per_checkpoint, opt.beam_size, opt.output_dir, opt.learning_rate, opt.lr_decay)\n\n  model:shutDown()\n  _G.logger:shutDown()\nend -- function main\n\nmain()\n"
  },
  {
    "path": "tools/README.md",
    "content": "# Tools\n\nThis directory contains additional tools.\n\n## Generate Vocabulary\n\nTo generate the vocabulary:\n\n```\npython tools/generate_vocab.py -data_path data/train.txt -label_path data/labels.txt -vocab_file data/vocab.txt\n```\n\nwhere the options are:\n\n* `-data_path`: Input file containing <image_path> <label_index> per line. This should be the file used for training.\n    \n* `-label_path`: Input file containing a tokenized formula per line.\n\n* `-vocab_file`: Output file for the generated vocabulary. One token per line.\n    \n* `-unk_threshold`: If a token appears less than (including) the threshold, then it will be excluded from the vocabulary.\n"
  },
  {
    "path": "tools/generate_vocab.py",
    "content": "import sys, logging, argparse, os\n\ndef process_args(args):\n    parser = argparse.ArgumentParser(description='Generate vocabulary file.')\n\n    parser.add_argument('-data_path', dest='data_path',\n                        type=str, required=True,\n                        help=('The file containing <image_path> <label_index> per line. This should be the file used for training.'\n                        ))\n    parser.add_argument('-label_path', dest='label_path',\n                        type=str, required=True,\n                        help=('The file containing a tokenized label per line.'\n                        ))\n    parser.add_argument('-vocab_file', dest='vocab_file',\n                        type=str, required=True,\n                        help=('Output file for the generated vocabulary. One token per line.'\n                        ))\n    parser.add_argument('--unk_threshold', dest='unk_threshold',\n                        type=int, default=1,\n                        help=('If the number of occurences of a token is less than (including) the threshold, then it will be excluded from the generated vocabulary.'\n                        ))\n    parser.add_argument('--log-path', dest=\"log_path\",\n                        type=str, default='log.txt',\n                        help=('Log file path, default=log.txt' \n                        ))\n    parameters = parser.parse_args(args)\n    return parameters\n\ndef main(args):\n    parameters = process_args(args)\n    logging.basicConfig(\n        level=logging.INFO,\n        format='%(asctime)-15s %(name)-5s %(levelname)-8s %(message)s',\n        filename=parameters.log_path)\n\n    console = logging.StreamHandler()\n    console.setLevel(logging.INFO)\n    formatter = logging.Formatter('%(asctime)-15s %(name)-5s %(levelname)-8s %(message)s')\n    console.setFormatter(formatter)\n    logging.getLogger('').addHandler(console)\n\n    logging.info('Script being executed: %s'%__file__)\n\n    label_path = parameters.label_path\n    assert os.path.exists(label_path), label_path\n    data_path = parameters.data_path\n    assert os.path.exists(data_path), data_path\n\n    formulas = open(label_path).readlines()\n    vocab = {}\n    max_len = 0\n    with open(data_path) as fin:\n        for line in fin:\n            _, line_idx = line.strip().split()\n            line_strip = formulas[int(line_idx)].strip()\n            tokens = line_strip.split()\n            tokens_out = []\n            for token in tokens:\n                tokens_out.append(token)\n                if token not in vocab:\n                    vocab[token] = 0\n                vocab[token] += 1\n\n    vocab_sort = sorted(list(vocab.keys()))\n    vocab_out = []\n    num_unknown = 0\n    for word in vocab_sort:\n        if vocab[word] > parameters.unk_threshold:\n            vocab_out.append(word)\n        else:\n            num_unknown += 1\n    vocab = [word for word in vocab_out]\n\n    with open(parameters.vocab_file, 'w') as fout:\n        fout.write('\\n'.join(vocab))\n    logging.info('#UNK\\'s: %d'%num_unknown)\n\nif __name__ == '__main__':\n    main(sys.argv[1:])\n    logging.info('Jobs finished')\n"
  }
]