Repository: OpenNMT/Im2Text
Branch: master
Commit: 6184e033c1cd
Files: 78
Total size: 547.8 KB

Directory structure:
gitextract_xkub52gy/

├── .luacheckrc
├── LICENSE.md
├── README.md
├── data/
│   ├── labels.txt
│   ├── test.txt
│   ├── train.txt
│   ├── validate.txt
│   └── vocab.txt
├── opennmt/
│   ├── Constants.lua
│   ├── Factory.lua
│   ├── LanguageModel.lua
│   ├── Model.lua
│   ├── ModelSelector.lua
│   ├── Seq2Seq.lua
│   ├── SeqTagger.lua
│   ├── data/
│   │   ├── AliasMultinomial.lua
│   │   ├── Batch.lua
│   │   ├── BatchTensor.lua
│   │   ├── Dataset.lua
│   │   ├── Preprocessor.lua
│   │   ├── SampledDataset.lua
│   │   ├── Vocabulary.lua
│   │   └── init.lua
│   ├── init.lua
│   ├── modules/
│   │   ├── BiEncoder.lua
│   │   ├── DBiEncoder.lua
│   │   ├── Decoder.lua
│   │   ├── Encoder.lua
│   │   ├── FeaturesEmbedding.lua
│   │   ├── FeaturesGenerator.lua
│   │   ├── GRU.lua
│   │   ├── Generator.lua
│   │   ├── GlobalAttention.lua
│   │   ├── JoinReplicateTable.lua
│   │   ├── LSTM.lua
│   │   ├── MaskedSoftmax.lua
│   │   ├── Network.lua
│   │   ├── NoAttention.lua
│   │   ├── PDBiEncoder.lua
│   │   ├── ParallelClassNLLCriterion.lua
│   │   ├── Sequencer.lua
│   │   ├── WordEmbedding.lua
│   │   └── init.lua
│   ├── tagger/
│   │   ├── Tagger.lua
│   │   └── init.lua
│   ├── train/
│   │   ├── Checkpoint.lua
│   │   ├── EpochState.lua
│   │   ├── Optim.lua
│   │   ├── Trainer.lua
│   │   └── init.lua
│   ├── translate/
│   │   ├── Advancer.lua
│   │   ├── Beam.lua
│   │   ├── BeamSearcher.lua
│   │   ├── DecoderAdvancer.lua
│   │   ├── PhraseTable.lua
│   │   ├── Translator.lua
│   │   └── init.lua
│   └── utils/
│       ├── CrayonLogger.lua
│       ├── Cuda.lua
│       ├── Dict.lua
│       ├── ExtendedCmdLine.lua
│       ├── Features.lua
│       ├── FileReader.lua
│       ├── Logger.lua
│       ├── Memory.lua
│       ├── MemoryOptimizer.lua
│       ├── Parallel.lua
│       ├── Profiler.lua
│       ├── String.lua
│       ├── Table.lua
│       ├── Tensor.lua
│       └── init.lua
├── src/
│   ├── cnn.lua
│   ├── data.lua
│   ├── model.lua
│   └── train.lua
└── tools/
    ├── README.md
    └── generate_vocab.py

================================================
FILE CONTENTS
================================================

================================================
FILE: .luacheckrc
================================================
globals = {
  "torch",
  "cutorch",
  "nn",
  "paths",
  "cudnn",
  "image",
  "onmt",
  "tds"
}

self = false


================================================
FILE: LICENSE.md
================================================
MIT License

Copyright (c) 2016 OpenNMT

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


================================================
FILE: README.md
================================================
# Im2Text

A deep learning-based approach to learning the image-to-text conversion, built on top of the <a href="https://opennmt.github.io/">OpenNMT</a> system. It is completely data-driven, hence can be used for a variety of image-to-text problems, such as image captioning, optical character recognition and LaTeX decompilation. 

Take LaTeX decompilation as an example, given a formula image:

<p align="center"><img src="http://lstm.seas.harvard.edu/latex/results/website/images/119b93a445-orig.png"></p>

The goal is to infer the LaTeX source that can be compiled to such an image:

```
 d s _ { 1 1 } ^ { 2 } = d x ^ { + } d x ^ { - } + l _ { p } ^ { 9 } \frac { p _ { - } } { r ^ { 7 } } \delta ( x ^ { - } ) d x ^ { - } d x ^ { - } + d x _ { 1 } ^ { 2 } + \; \cdots \; + d x _ { 9 } ^ { 2 } 
```

The paper (http://arxiv.org/pdf/1609.04938v1.pdf) provides more technical details of this model.

## Installation

Im2Text is built on top of <a href="https://opennmt.github.io/">OpenNMT</a>, which is packed in this project. It also depends on `tds`, `class`, `cudnn`, `cutorch` and `paths`. Currently we only support **GPU**.


## Quick Start

To get started, we provide a toy Math-to-LaTex example. We assume that the working directory is `Im2Text` throughout this document.

Im2Text consists of two commands:

1) Train the model.

```
th src/train.lua -phase train -gpu_id 1 -input_feed -model_dir model \
-image_dir data/images -data_path data/train.txt -val_data_path data/validate.txt -label_path data/labels.txt -vocab_file data/vocab.txt \
-batch_size 20 -beam_size 1 \
-max_num_tokens 150 -max_image_width 500 -max_image_height 160 \
-max_grad_norm 20.0 -learning_rate 0.1 -decay perplexity_only
```

2) Translate the images.

```
th src/train.lua -phase test -gpu_id 1 -load_model -model_dir model \
-image_dir data/images -data_path data/test.txt \
-output_dir results \
-batch_size 2 -beam_size 5 \
-max_num_tokens 500 -max_image_width 800 -max_image_height 800
```

The above dataset is sampled from the [processed-im2latex-100k-dataset](http://lstm.seas.harvard.edu/latex/processed-im2latex-100k-dataset.tgz). We provide a trained model [[link]](http://lstm.seas.harvard.edu/latex/model_latest) on this dataset. In order to use it, download and put it under `model_dir` before translating the images.

## Data Format

* `-image_dir`: The directory containing the images. Since images of the same size can be batched together, we suggest padding images of similar sizes to the same size in order to facilitate training.

* `-label_path`: The file storing the tokenized labels, one label per line. It shall look like:
```
<label0_token0> <label0_token1> ... <label0_tokenN0>
<label1_token0> <label1_token1> ... <label1_tokenN1>
<label2_token0> <label2_token1> ... <label2_tokenN2>
...
```

* `-data_path`: The file storing the image-label pairs. Each line starts with the path of the image (relative to `image_dir`), followed by the index of the label in `label_path` (index counts from 0). At test time, the label indexes can be omitted.
```
<image0_path> <label_index0>
<image1_path> <label_index1>
<image2_path> <label_index2>
...
```

* `-vocab_file`: The vocabulary file. Each line corresponds to a token. The tokens not in `vocab_file` will be considered unknown (UNK).


## Options

For a complete set of options, run `th src/train.lua -h`.


================================================
FILE: data/labels.txt
================================================
d s ^ { 2 } = ( 1 - { \frac { q c o s \theta } { r } } ) ^ { \frac { 2 } { 1 + \alpha ^ { 2 } } } \lbrace d r ^ { 2 } + r ^ { 2 } d \theta ^ { 2 } + r ^ { 2 } s i n ^ { 2 } \theta d \varphi ^ { 2 } \rbrace - { \frac { d t ^ { 2 } } { ( 1 - { \frac { q c o s \theta } { r } } ) ^ { \frac { 2 } { 1 + \alpha ^ { 2 } } } } } \, .
\widetilde \gamma _ { \mathrm { h o p f } } \simeq \sum _ { n > 0 } \widetilde { G } _ { n } { \frac { ( - a ) ^ { n } } { 2 ^ { 2 n - 1 } } }
( { \cal L } _ { a } g ) _ { i j } = 0 , \ \ \ \ ( { \cal L } _ { a } H ) _ { i j k } = 0 ,
S _ { s t a t } = 2 \pi \sqrt { N _ { 5 } ^ { ( 1 ) } N _ { 5 } ^ { ( 2 ) } N _ { 5 } ^ { ( 3 ) } } \left( \sqrt { n } + \sqrt { \bar { n } } \right)
\hat { N } _ { 3 } = \sum \sp f _ { j = 1 } a _ { j } \sp { \dagger } a _ { j } \, .
+ \int \! \! d ^ { D } \! z _ { 1 } d ^ { D } \! z _ { 2 } d ^ { D } \! z _ { 3 } \left. \frac { \delta ^ { 2 } W } { \delta j ( x ) \delta j ( z _ { 1 } ) } \, \frac { \delta ^ { 2 } W } { \delta j ( x ) \delta j ( z _ { 2 } ) } \, \frac { \delta ^ { 2 } W } { \delta j ( x ) \delta j ( z _ { 3 } ) } \, \frac { \delta ^ { 3 } \Gamma } { \delta \Phi ( z _ { 1 } ) \delta \Phi ( z _ { 2 } ) \delta \Phi ( z _ { 3 } ) } \right] ,
\, ^ { * } d \, ^ { * } H = \kappa \, ^ { * } d \phi = J _ { B } .
{ \frac { \phi ^ { \prime \prime } } { A } } + { \frac { 1 } { A } } \left( - { \frac { 1 } { 2 } } { \frac { A ^ { \prime } } { A } } + 2 { \frac { B ^ { \prime } } { B } } + { \frac { 2 } { r } } \right) \phi ^ { \prime } - { \frac { 2 } { r ^ { 2 } } } \phi - \lambda \phi ( \phi ^ { 2 } - \eta ^ { 2 } ) = 0 \, .
\partial _ { \mu } ( F ^ { \mu \nu } - e j ^ { \mu } x ^ { \nu } ) = 0 .
V _ { n s } ( { \tilde { x } } ) = \left( \frac { { \tilde { m } } N ^ { 2 } } { 1 6 \pi } \right) N g ^ { 2 n s - 1 } { \tilde { x } } ^ { 2 } \left\{ { \tilde { x } } ^ { 2 } - \frac { 2 { \tilde { b } } } { 3 } { \tilde { x } } + \frac { { \tilde { b } } ^ { 2 } } { 3 } - ( - 1 ) ^ { n s } { \tilde { c } } \right\} \, .
g _ { i j } ( x ) = { \frac { 1 } { a ^ { 2 } } } \, \delta _ { i j } , ~ ~ \phi ^ { a } ( x ) = \phi ^ { a } , \quad ( a , \phi ^ { a } \! : ~ \mathrm { c o n s t . } )
\rho _ { L } ( q ) = \sum _ { m = 1 } ^ { L } \ P _ { L } ( m ) \ { \frac { 1 } { q ^ { m - 1 } } } \ \ .
e x p \left( - \frac { \partial } { \partial \alpha _ { j } } \theta ^ { j k } \frac { \partial } { \partial \alpha _ { k } } \right)
L _ { 0 } = \Phi ( w ) = \bigtriangleup \Phi ( w ) ,
\left( D ^ { * } D ^ { * } + m ^ { 2 } \right) { \cal H } = 0
{ \frac { d V } { d \Phi } } = - { \frac { w \Phi } { \Phi _ { \! _ { 0 } } ^ { 2 } } } \, .
g ( z , \bar { z } ) = - \frac { 1 } { 2 } \left[ x ( z , \bar { z } ) \, s + x ^ { * } ( z , \bar { z } ) \, s ^ { * } + u ^ { * } ( z , \bar { z } ) \, t + u ( z , \bar { z } ) \, t ^ { * } \right] ,
x _ { \mu } ^ { c } = x _ { \mu } + A _ { \mu } .
s = { \frac { S } { V } } = { \frac { A _ { H } } { l _ { p } ^ { 8 } V } } = { \frac { T ^ { 2 } } { \gamma } } .
\psi ( \gamma ) = \operatorname { e x p } { - ( { \textstyle { \frac { g ^ { 2 } } { 2 } } } ) \int _ { \gamma } d y ^ { a } \int _ { \gamma } d y ^ { a ^ { \prime } } D _ { 1 } ( y - y ^ { \prime } ) }
E = E _ { 0 } + \frac { 1 } { 2 \operatorname { s i n h } ( \gamma ( 0 ) / 2 ) } \operatorname { s i n h } \left( \gamma ( 0 ) \left( \frac { 1 } { 2 } + c ( 0 ) \right) \right) h c \nu _ { \mathrm { v i b } }
\langle T _ { z z } \rangle = - 3 \times \frac { \pi ^ { 2 } } { 1 4 4 0 a ^ { 4 } } .
\partial _ { u } \xi _ { z } ^ { ( 1 ) } + { \frac { 1 } { u } } \xi _ { z } ^ { ( 1 ) } = { \frac { 1 } { ( \pi T R ) ^ { 2 } u } } \left[ C _ { z } H _ { z z } ^ { \prime } + C _ { t } H _ { t z } ^ { \prime } \right] \, .
S \sim \tilde { \psi } Q _ { o } \tilde { \psi } + g _ { s } ^ { 1 / 2 } \tilde { \psi } ^ { 3 } + \tilde { \phi } Q _ { c } \tilde { \phi } + g _ { s } \tilde { \phi } ^ { 3 } + \tilde { \phi } B ( g _ { s } ^ { 1 / 2 } \tilde { \psi } ) + \cdots .
C ( x ^ { \prime } , x ^ { \prime \prime } ) = C \Phi ( x ^ { \prime } , x ^ { \prime \prime } ) \ , \quad \Phi ( x ^ { \prime } , x ^ { \prime \prime } ) = \operatorname { e x p } \left[ - i e \int _ { x ^ { \prime \prime } } ^ { x ^ { \prime } } d x ^ { \mu } A _ { \mu } ( x ) \right] \ ,
\tilde { \alpha } = \alpha \beta ^ { - m } = \left( \begin{array} { c c c } { \omega _ { k } ^ { - 2 y } \omega _ { 2 d } ^ { 2 m } } & { 0 } & { 0 } \\ { 0 } & { \omega _ { k } ^ { y } \omega _ { 2 d } ^ { - m } } & { 0 } \\ { 0 } & { 0 } & { \omega _ { k } ^ { y } \omega _ { 2 d } ^ { - m } } \\ \end{array} \right)
d s ^ { 2 } = H ^ { - 2 } f ( r ) d t ^ { 2 } + H ^ { 2 / ( n - 1 ) } ( f ( r ) ^ { - 1 } d r ^ { 2 } + r ^ { 2 } d \Omega _ { n } ^ { 2 } ) ,
y ^ { 2 } = \rho \; \operatorname { c o s h } \beta \; \operatorname { s i n } \theta \; \operatorname { s i n } \phi \qquad \qquad y ^ { 3 } = \rho \; \operatorname { c o s } \theta
e ^ { A } = e ^ { A _ { 0 } } \left( t _ { 0 } - \mathrm { s i g n } ( m ) t \right) ^ { - \frac { m } { 2 } } \; , \; \; \; \; \chi = \chi _ { 0 } \left( t _ { 0 } - \mathrm { s i g n } ( m ) t \right) ^ { m } \; ,
\gamma _ { j } { \cal P } _ { j i } = \frac { 4 } { 3 } \{ [ A d \, T ] [ t _ { 8 } ^ { c } , [ t _ { 8 } ^ { c } , { \gamma } _ { j } ] ] [ A d \, T ^ { - 1 } ] \} { A d \, { \hat { g } } } _ { i j } .
K _ { \mu \nu } ~ = ~ \frac { 1 } { 2 } \dot { g } _ { \mu \nu } .
X ( u ) = { \frac { \left( \pm i + e ^ { 3 \eta } \right) \left( - 1 + { e ^ { u } } \right) \left( 1 + { e ^ { u } } \right) x _ { 1 } } { 2 { e ^ { u } } \left( \pm i + { e ^ { 3 \eta + u } } \right) } } ,
\beta ( g ) \frac { \partial } { \partial g } = 2 g \beta ( g ) \frac { \partial } { \partial g ^ { 2 } }
A = a r ^ { \beta } , \quad B = b r ^ { \beta + 2 } ; \qquad a / b = c ( \beta + 2 ) / ( \beta - 2 ) ,
\delta W _ { P \mu } = A _ { \mu } \Phi + B _ { P \mu } ^ { \alpha } K _ { P } ^ { \alpha } \ .
\frac { 1 } { d - 2 } \tilde { \Pi } ^ { 2 } - \tilde { \Pi } _ { a b } \tilde { \Pi } ^ { a b } = \frac { \left( d - 1 \right) \left( d - 2 \right) } { \ell ^ { 2 } } + R
\hat { e } = e / \varepsilon , \ \ \ \ \ \ \ \ \ \hat { G } _ { 4 } = G _ { 4 } ,
V _ { ( n , \, m ) } ( z , \overline { { z } } ) = : \operatorname { e x p } i ( p _ { + } \phi ( z ) + p _ { - } \bar { \phi } ( \overline { { z } } ) ) : \: .
\langle f | g \rangle _ { { \cal L } ^ { 1 | 2 } } = \langle f _ { 0 } | g _ { 0 } \rangle _ { \cal L } ^ { s } + \langle f _ { 1 } | g _ { 1 } \rangle _ { \cal L } ^ { s + 1 / 2 } + \langle f _ { 2 } | g _ { 2 } \rangle _ { \cal L } ^ { s + 1 / 2 } + \langle f _ { 3 } | g _ { 3 } \rangle _ { \cal L } ^ { s + 1 } \, ,
\tilde { s } ^ { 0 } ( x , y ) = i e ^ { 2 } \int \! d ^ { 4 } \! z \, S _ { \mathrm { F } } ( x , z ) \, \gamma ^ { \mu } \, S _ { \mathrm { F } } ( z , y ) \, [ d _ { \mu } ( x - z ) + d _ { \mu } ( z - y ) ]
\left\{ \begin{array} { l c l l } { \phi ~ ( \infty ) } & { = } & { 0 } & { , \vspace { 3 m m } } \\ { \phi ~ ( 0 ) } & { = } & { 1 } & { . } \\ \end{array} \right.
{ \cal P } _ { \delta x } \equiv { \frac { k ^ { 3 } } { 2 \pi ^ { 2 } } } | \delta x | ^ { 2 } \, ,
\psi ( x ) = - 2 \phi ( x ) + 2 \phi ( L ) + c ,
{ } ^ { ( { } ^ { \scriptstyle x } y ) } ( { } ^ { x } z ) = { } ^ { x } ( { } ^ { y } z ) , \qquad \forall x , y , z \in X .
\delta ( L _ { 1 } + L _ { 2 } ) = 2 \delta \bar { \theta } ( 1 + \gamma ^ { ( p ) } ) T _ { ( p ) } ^ { \nu } \partial _ { \nu } \theta .
\frac { 1 } { 2 \lambda f ^ { 2 } } \int \; d ^ { 4 } X \, \frac { d ^ { 4 } q } { \left( 2 \pi \right) ^ { 4 } } \left( \varphi ( X ) \right) ^ { 2 } \tilde { \pi } _ { 0 } ( q ) \left[ \partial _ { q } ^ { 2 } + \frac { 4 i \lambda } { q ^ { 2 } - \Sigma ^ { 2 } ( q ) } \right] \tilde { \pi } _ { 0 } ( q ) ,
( K ^ { - 1 } ) _ { S } ^ { U } = - \frac { z ^ { * 3 } ( 1 - A | z | ^ { 2 } ) P ^ { \prime } ( y ) } { e ^ { \tilde { K } / 2 } P ^ { \prime \prime } ( y ) } , \mathrm { ~ } ( K ^ { - 1 } ) _ { T } ^ { U } = \frac { ( T + T ^ { * } ) z ^ { * 3 } ( 1 - A | z | ^ { 2 } ) } { e ^ { \tilde { K } / 2 } ( 1 - \frac { \bar { n } } { 3 } B ( 1 - A | z | ^ { 2 } ) \| \Pi \| ) } ,
G ^ { \mu \nu \mu ^ { \prime } \nu ^ { \prime } } = g ^ { \mu \mu ^ { \prime } } g ^ { \nu \nu ^ { \prime } } + g ^ { \mu \nu ^ { \prime } } g ^ { \nu \mu ^ { \prime } } - { \frac { 2 } { D } } g ^ { \mu \nu } g ^ { \mu ^ { \prime } \nu ^ { \prime } } + C g ^ { \mu \nu } g ^ { \mu ^ { \prime } \nu ^ { \prime } } \: .
[ M _ { \mu \nu } , M _ { \rho \tau } ] = g _ { \mu \tau } \, M _ { \nu \rho } - g _ { \nu \tau } M _ { \mu \rho } + g _ { \nu \rho } M _ { \mu \tau } - g _ { \mu \rho } M _ { \nu \tau } \, ,
A _ { 0 } = \pm \sqrt { { \frac { 4 } { 3 ( 1 - \alpha ) } } } e ^ { ( \alpha - 1 ) \phi } \ .
C _ { m } ( \mu ) = { \frac { 1 } { 2 \pi i } } \int _ { \Gamma _ { r } } { \frac { C _ { m } ( z ) } { z - \mu } } d z ,
\xi = \alpha ^ { - 1 } \sqrt { \rho } \operatorname { c o s h } ( 2 \alpha ^ { 2 } t ) \, , \quad \eta = \alpha ^ { - 1 } \sqrt { \rho } \operatorname { s i n h } ( 2 \alpha ^ { 2 } t )
a _ { 1 } = \frac { 2 \tilde { q } } { \alpha ^ { 2 } ( D - 2 ) + 2 q \tilde { q } } , ~ ~ ~ ~ a _ { 2 } = \frac { \alpha ^ { 2 } ( D - 2 ) } { \alpha ^ { 2 } d ( D - 2 ) + 2 \tilde { d } q ^ { 2 } }
\theta \epsilon ^ { i } = \zeta ^ { i } \, ; \qquad \theta \zeta ^ { i } = \epsilon ^ { i } \, ; \qquad \theta \eta ^ { i } = - \eta ^ { i } \, .
{ \cal A } _ { f i } ( s ) = - i \frac { Q _ { \mu \nu } V _ { f } ^ { \mu } ( \bar { s } ) V _ { i } ^ { \nu } ( \bar { s } ) } { ( s - \bar { s } ) [ 1 - A ^ { \prime } ( \bar { s } ) ] } + N ,
S = \frac { 1 } { G } \int d x d t \: \sqrt { - \bar { g } } \: e ^ { - 2 \bar { \phi } } ( \bar { R } + 4 ( \bar { \nabla } \bar { \phi } ) ^ { 2 } + 4 \lambda ^ { 2 } ) - \frac { 1 } { 2 } \int d x d t \: \sqrt { - \bar { g } } \: \sum _ { i = 1 } ^ { N } ( \bar { \nabla } f _ { i } ) ^ { 2 } ,
\phi ^ { A } = \frac { \partial F ( \phi , \phi ^ { * } ) } { \partial \phi _ { A } ^ { * } } \hspace { 2 c m } \phi _ { A } ^ { * } = \frac { \partial F ( \phi , \phi ^ { * } ) } { \partial \phi ^ { A } } .
Q ^ { M } = \dot { g } _ { 0 } \int _ { \phi ( \Sigma ) } \sum _ { i = 1 } ^ { l } \beta _ { i } \eta _ { i } \int _ { N _ { i } } \frac 1 { \sqrt { g } } \delta ^ { D } ( x - z _ { i } ( u ) ) \sqrt { g _ { u } }
\psi ( x ) = [ 2 \pi \sigma ^ { 2 } ] ^ { - 1 / 4 } \operatorname { e x p } \left[ - \left( \frac { x - x _ { 0 } } { 2 \sigma } \right) ^ { 2 } + i p _ { 0 } x \right] ,
A _ { \mu } = \bar { A } _ { \mu } ( \phi ) + a _ { \mu } \ ,
\left[ D _ { f } , D \right] = 0 \, .
\operatorname { e x p } _ { q } A = \sum _ { n = 0 } ^ { \infty } \frac { A ^ { n } } { [ n ] ! }
\langle b _ { 1 } ( z , \bar { z } ) a _ { 1 } ( z ^ { \prime } , \bar { z } ^ { \prime } ) \rangle = - { \frac { 1 } { \pi } } \partial ~ K _ { 0 } ( d ^ { 2 } m ^ { 2 } ( { \bf p } ) ) ~ ,
\lambda _ { + } = \frac { 1 + i \omega } { 2 } , \quad \lambda _ { - } = \frac { 1 - i \omega } { 2 }
1 + \frac { 2 \pi \Lambda G } { 9 \alpha } > 0
e ^ { - K } = \pm \frac { W ^ { 3 / 2 } } { \omega _ { 1 } \omega _ { 2 } \omega _ { 3 } } \ ,
{ \cal { Z } } ( \tau { } ) = \sum _ { m } \int { \cal { D } } \Omega { \cal { D } } V { \mathrm { V o l } } _ { Z M } { \mathrm { d e t } } ( d _ { 2 } )
| 0 ( t ) \rangle _ { e , \mu } \equiv G _ { \theta } ^ { - 1 } ( t ) | 0 \rangle _ { 1 , 2 } \, ,
\Gamma _ { i j } ^ { k } = ( \partial _ { i } G _ { j { \bar { l } } } ) G ^ { { \bar { l } } k } \,
{ \cal { F } } : \ < g > = \int d ^ { 3 } \theta \ g ( { \vec { \theta } } ) f ( { \vec { \theta } } , t ) \,
\tilde { S } _ { r , \Lambda } ( t _ { 2 } , t _ { 1 } ; g ) = - 2 i \delta ( t _ { 2 } - t _ { 1 } ) H _ { I } ^ { ( r ) } ( t _ { 1 } ) + \tilde { S } _ { r , \Lambda } ^ { \prime } ( t _ { 2 } , t _ { 1 } ; g ) ,
\nu _ { R } ( E ) = \int _ { \mu } ^ { E - \mu _ { Q } } \nu ( E ^ { \prime } ) \nu _ { Q } ( E - E ^ { \prime } ) d E ^ { \prime } ~ ~ ~ .
Z _ { \mathrm { F } } [ A ] = \int \! D { \bar { \psi } } D \psi e ^ { - \int \! d ^ { 2 } x \, [ \psi _ { 1 } ^ { \dagger } i \partial \psi _ { 1 } + \psi _ { 2 } ^ { \dagger } i { \bar { \partial } } \psi _ { 2 } - \psi _ { 1 } ^ { \dagger } A \psi _ { 1 } - \psi _ { 2 } ^ { \dagger } { \bar { A } } \psi _ { 2 } ] } \; ,
E q ( 5 . 2 ) T _ { R } \; \sim \; 5 \left( \frac { m } { \mathrm { T e V } } \right) ^ { 3 / 2 } \; \; \; \mathrm { k e V } .
P _ { n } ^ { p } = \frac { ( - 1 ) ^ { p + n } } { n ! ( p - n ) ! } \mathop { { \prod } ^ { \prime } } _ { k = 0 } ^ { p } ( N - k ) , \quad n = 0 , . . . , p ,
T _ { { \cal G } } ( - t , - t ^ { - 1 } ) = \sum x ^ { i ( B ) } x ^ { - e ( B ) }
\left. \begin{array} { c c c } { S _ { \sigma \sigma ^ { \prime } } \nonumber } \\ { S _ { \Delta } \nonumber } \\ { \tilde { S } _ { \Delta } } \\ \end{array} \right\} \varpi = ( \varpi + 2 ) \left\{ \begin{array} { c c c } { S _ { \sigma \sigma ^ { \prime } } \nonumber } \\ { S _ { \Delta } \nonumber } \\ { \tilde { S } _ { \Delta } } \\ \end{array} \right. .
\Phi _ { 0 } ( Z ) = \delta ^ { - 1 } \alpha , \qquad \Phi _ { 0 } ( Z ) = \alpha ^ { - 1 } \delta
b ( k ) b ^ { \dagger } ( l ) - q _ { e } b ^ { \dagger } ( l ) b ( k ) = \delta ( k - l ) ,
{ \cal L } = - \mathrm { \small ~ \frac { 1 } { 2 } ~ } f ^ { 2 } \, \partial _ { \mu } \pi _ { r } \, \partial ^ { \mu } \pi _ { r } - \mathrm { \small ~ \frac { 1 } { 2 } ~ } f ^ { 2 } \lambda \left( \pi _ { r } \pi _ { r } - N \right) \; ,
E _ { \mathrm { q u a s i l o c a l } } = E _ { + } - E _ { - } = \left( r \left[ 1 - \left| r , _ { y } \right| \right] \right) _ { y = y _ { + } } - \left( r \left[ 1 - \left| r , _ { y } \right| \right] \right) _ { y = y _ { - } } .
I = 2 \left( { \frac { \alpha } { \operatorname { s i n h } ^ { 2 } A + \operatorname { s i n } ^ { 2 } { \frac { \gamma } { 2 } } } } - { \frac { \operatorname { s i n h } { \frac { 2 A } { \alpha } } } { \operatorname { s i n h } { 2 A } } } ~ { \frac { 1 } { ( \operatorname { s i n h } ^ { 2 } { \frac { A } { \alpha } } + \operatorname { s i n } ^ { 2 } { \frac { [ \gamma ] } { 2 \alpha } } ) } } \right) ~ ~ .
W ( x ) = \frac { x ^ { 3 } } { 3 } - a ^ { 2 } x ,
Z _ { M } = \sum _ { j _ { s } } \int d U _ { f } \Pi _ { s } \left( 2 j _ { s } + 1 \right) T r _ { j _ { s } } U _ { s }
f ( x , p ; t ) = \int \! d a d b ~ \tilde { f } ( a , b ) ~ e ^ { i a x ( - t ) } e ^ { i b p ( - t ) } ,
\{ \langle n | O | p \rangle \: | n \in \Sigma _ { s } ^ { \prime } \}
{ \cal L } ^ { ( 0 ) } ( B ) = - { \frac { 1 } { 2 } } B ^ { 2 } \; ,
\ddot { h } = - \nabla _ { h } \Phi
\langle 0 | T _ { \mu \nu } | 0 \rangle _ { \mathrm { \tiny ~ R e n . } } = \mathrm { d i a g } \left[ \langle 0 | T _ { t t } | 0 \rangle , 0 , \langle 0 | T _ { l l } | 0 \rangle , \langle 0 | T _ { l l } | 0 \rangle \right] .
\frac { 1 } { 2 4 . 8 \pi ^ { 2 } } \, \int _ { \cal M } R \wedge R = \frac { 1 } { 2 4 . 8 \pi ^ { 2 } } \, \int _ { { \cal M } } \mathrm { d } C = \frac { 1 } { 2 4 . 8 \pi ^ { 2 } } \, \int _ { \partial { \cal M } } C \, ,
\quad | A _ { 1 } | ^ { 2 } + | A _ { 2 } | ^ { 2 } - | B _ { 1 } | ^ { 2 } - | B _ { 2 } | ^ { 2 } = 0 .
e ^ { \phi _ { c } ^ { 6 } } = - { \frac { v _ { a } } { \tilde { v } _ { a } } }
\mathrm { \boldmath ~ \pi ~ } = \mu ^ { - 1 } \, { \bf p } \; .
S = \int _ { \mathcal M } \left( { \frac { i } { 2 } } \left[ \overline { \psi } \gamma ^ { a } \nabla _ { a } \psi - \overline { { ( \nabla _ { a } \psi ) } } \gamma ^ { a } \psi \right] - m \overline { \psi } \psi \right) \, ,
\Psi \sim \operatorname { e x p } \left\{ - \frac { 1 } { \hbar } \int _ { a } ^ { x } \sqrt { 2 ( v - e ) } \right\} \sim \operatorname { e x p } \left\{ - \frac { 1 } { \hbar } x ^ { \frac { 5 } { 2 } } \right\}
\ddot { R } ^ { k } ( t ) = \omega ^ { k l } \dot { R } ^ { l } ( t ) + O ( \dot { R } ^ { k } \dot { R } ^ { k } ) \; \; .
{ \cal L } \rightarrow { \cal L } + \frac { \alpha N } { 1 6 \pi ^ { 2 } } F \tilde { F } .
\partial _ { 0 } { \cal E } + { \bf d i v } { \bf S } = 0 \, .
\phi ( x , y ) = \lambda ^ { 2 s } \, \phi ( \lambda x , \lambda y )
S _ { B } [ A ] \; = \; i \, \frac { 1 } { \eta } \, S _ { C S } [ A ] \; + \; R [ \widetilde { F } ] + \frac { i } { \theta } \int d ^ { 3 } x d ^ { 3 } y j _ { \mu } ^ { T } ( x ) { \mathcal K } _ { \mu \nu } ( x - y ) j _ { \nu } ^ { T } ( y ) \; ,
{ \cal L } = \frac { 1 } { 2 } { \it i } \overline { { \psi } } { \Gamma } ^ { \nu } \hspace { - . 1 5 c m } \stackrel { \; \leftrightarrow } { \partial } _ { \! \nu } \hspace { - . 1 c m } { \psi } - \overline { { \psi } } M { \psi } \quad ,
m _ { H } \approx 2 7 2 \, \lambda ^ { 1 / 4 } \ \mathrm { G e V } .
M _ { p _ { e } , q _ { m } } = { \frac { 2 \pi } { g _ { \mathrm { Y M } } ^ { 2 } } } { \frac { q _ { m } u } { \operatorname { c o s } \xi } } = u \sqrt { \left( \frac { 2 \pi q _ { m } } { g _ { Y M } ^ { 2 } } \right) ^ { 2 } + p _ { e } ^ { 2 } } \, .
{ \frac { d ^ { 2 } \varphi } { d \tau ^ { 2 } } } = \operatorname { s i n } \varphi \ ,
\hat { I } _ { 1 2 } = - \frac { 1 } { 9 6 ( 2 \pi ) ^ { 5 } } \hat { I } _ { 4 } \wedge ( \frac { 1 } { 4 } ( \hat { I } _ { 4 } ) ^ { 2 } - X _ { 8 } )
\frac { 1 } { k ^ { 2 } } ( g ^ { \mu \nu } - \frac { \tilde { k } _ { \mu } \tilde { k } _ { \nu } } { \tilde { k } ^ { 2 } } )
\begin{array} { l c r } { q J _ { 1 } J _ { 2 } - q ^ { - 1 } J _ { 2 } J _ { 1 } } & { = } & { ( q ^ { 2 } - q ^ { - 2 } ) J _ { 3 } } \\ { q J _ { 2 } J _ { 3 } - q ^ { - 1 } J _ { 3 } J _ { 2 } } & { = } & { ( q ^ { 2 } - q ^ { - 2 } ) J _ { 1 } } \\ { q J _ { 3 } J _ { 1 } - q ^ { - 1 } J _ { 1 } J _ { 3 } } & { = } & { ( q ^ { 2 } - q ^ { - 2 } ) J _ { 2 } } \\ \end{array}
S _ { \Omega ^ { \prime } , \Omega } x \Omega = x ^ { * } \Omega , \, \, x \in M
a _ { 1 } = - 2 \pi I _ { 2 \alpha } ( 0 ) = - \frac { \pi } { 3 } \left( \frac { \pi } { \alpha } - \frac { \alpha } { \pi } \right) { . }
{ \cal L } = { \frac { 1 } { 2 } } \partial _ { \mu } \phi _ { x } \partial ^ { \mu } \phi _ { x } - { \frac { 1 } { 2 } } m _ { 0 } ^ { 2 } \phi _ { x } ^ { 2 } - { \frac { m ^ { 2 } } { \beta ^ { 2 } } } ( 1 - c o s ( \beta \phi _ { x } ) ) \equiv { \frac { 1 } { 2 } } \partial _ { \mu } \phi _ { x } \partial ^ { \mu } \phi _ { x } - U ( \phi _ { x } ) ,
\langle P ^ { \prime } | J ^ { \mu } ( 0 ) | P \rangle = \bar { u } ( P ^ { \prime } ) \, \Big [ \, F _ { 1 } ( q ^ { 2 } ) \gamma ^ { \mu } + F _ { 2 } ( q ^ { 2 } ) { \frac { i } { 2 M } } \sigma ^ { \mu \alpha } q _ { \alpha } \, \Big ] \, u ( P ) \ ,
f \star g = \operatorname { e x p } \Bigg [ \hbar \Bigg ( \frac { \partial ~ } { \partial q } \frac { \partial ~ } { \partial \tilde { p } } - \frac { \partial ~ } { \partial p } \frac { \partial ~ } { \partial \tilde { q } } \Bigg ) \Bigg ] f ( { \bf x } ) g ( { \bf \tilde { x } } ) \vert _ { { \bf x } = { \bf \tilde { x } } } ,
\Delta { \cal A } = \frac { 1 } { 2 } ( 1 + g ) .
\operatorname { e x p } \left( i p _ { \mu } X ^ { \mu } \right) \rightarrow \hat { v } _ { p } = \hat { h } ^ { k _ { 2 } }
T _ { [ \mu \nu ] } ^ { \ a } = \partial _ { \mu } E _ { \nu } ^ { \underline { { a } } } - \partial _ { \nu } E _ { \mu } ^ { \underline { { a } } }
\eta _ { \mu \nu } = \mathrm { d i a g ( - , + ) }
\langle f , f \rangle = \langle g , g \rangle = 0 .
\Big ( L ^ { \frac { 3 } { 2 } } \Big ) _ { + } = p ^ { 3 } + \frac { 3 } { 2 } u \star p - \frac { 2 \kappa } { 2 } u ^ { ( 1 ) } ,
Q _ { a , b } ( \mu , \nu ; N ) = p _ { a , b } ( \mu , \nu ; 0 ; N ) + \sum _ { \lambda = 1 } ^ { \infty } ( - 1 ) ^ { \lambda } p _ { a , b } ( \mu , \nu ; \lambda ; N ) + \sum _ { \lambda = 1 } ^ { \infty } ( - 1 ) ^ { \lambda } m _ { a , b } ( \mu , \nu ; \lambda ; N ) .
\vec { A } _ { \mu } ( x ) \! = \! \partial _ { \mu } \vec { n } ( x ) \wedge \vec { n } ( x ) + C _ { \mu } ( x ) \vec { n } ( x ) + \vec { W } _ { \mu } ( x ) ,
{ \cal E } = \frac { { \bf D } ^ { 2 } + { \bf H } ^ { 2 } } { 2 } - \left( { \bf \theta } \cdot { \bf B } \right) { \bf B } ^ { 2 } ,
\frac 1 { \nabla ^ { 2 } } \, \delta _ { \Sigma } ^ { ( 2 ) } ( z - z _ { 0 } ) = - \frac 1 \pi \operatorname { l o g } { \cal E } ( z , z _ { 0 } )
\delta _ { A } \widehat { \phi } \smallskip ( \widehat { x } ) = i [ \widehat { A } \smallskip ,
{ \cal M } = M ( \infty ) = 4 \pi \int _ { 0 } ^ { \infty } \left[ r ^ { 2 } \left( { \cal U } ( f , h , u ) + \alpha ^ { 2 } { \cal K } ( f , h , u ) \right) - \eta ^ { 2 } \right] e ^ { - P ( r ) } \ ,
y \sqrt { c } \operatorname { s i n h } \alpha = \operatorname { c o s h } \alpha - \operatorname { c o s h } ( x \sqrt { c } - \alpha ) \; \; , \; \; \; \; \operatorname { t a n h } \alpha \equiv 2 \sqrt { c } \; .
\lambda t K _ { \left| n \right| } ( \mu t ) I _ { \left| n \right| } ( \mu t ) .
g x ( \alpha \cdot q , \xi ) \to \left\{ \begin{array} { c l } { \mathrm { f i n i t e } , } & { \mathrm { f o r } \quad \pm \alpha _ { i } \in \Pi \quad ( \delta \leq 1 / h ) \quad \mathrm { a n d } \ \pm \alpha _ { h } \quad ( \delta = 1 / h ) , } \\ { 0 , } & { \mathrm { o t h e r w i s e , } } \\ \end{array} \right.
\dot { G } _ { \pm } + \frac { i } { 2 } \phi ^ { \pm } D _ { \pm } ^ { 2 } G _ { \pm } + \frac { i } { 4 } D _ { \pm } \phi ^ { \pm } D _ { \pm } G _ { \pm } + \frac { 3 } { 4 } i D _ { \pm } ^ { 2 } \phi ^ { \pm } G _ { \pm } = \frac { \kappa _ { 0 } } { 2 } D _ { \pm } ^ { 5 } \phi ^ { \pm } ~ .
\mathcal { L } = \sum _ { i = 1 } ^ { N } \bar { \psi } ^ { i } \left( \partial \! \! \! \slash + \lambda \sigma \right) \psi ^ { i }
\left\langle \operatorname { e x p } \left( \; 2 \sum _ { b = 1 } ^ { N } \beta ^ { ( b ) } \int _ { \Lambda } d ^ { 2 } x \; t ( x ) \; : \operatorname { c o s } \left( 2 \sqrt { \pi } \varphi ^ { ( b ) } ( x ) \; \right) : _ { Q ^ { W } } \; \right) \right\rangle \; .
\chi _ { a } ( A ) = \varepsilon _ { a b i } \, A _ { b i } ( x ) = 0 \, ,
\omega ( \varepsilon ) = { \frac { e ^ { \beta _ { H } \varepsilon } } { \varepsilon } } ,
G ^ { ( N , M ) } ( z _ { 1 } , S _ { M + 1 } , \cdots , S _ { N } ; z _ { 1 } , z _ { 2 } ) = 0 .
\varphi \sim A + B \mathrm { s i g n } ( t ) | \vec { k } t | ^ { \epsilon } .
\stackrel { \rightarrow } { x } ( \tau ) = m ^ { - 2 } \stackrel { \rightarrow } { V }
R _ { i j , k l } = \delta _ { i k } \delta _ { j l } ( 1 + \delta _ { i j } ( q - 1 ) ) + \lambda \delta _ { i l } \delta _ { j k } \theta ( i - j ) \quad i , j . . . = 1 . . . n
S = - T _ { ( p - 1 ) } \int d ^ { p } \sigma \sqrt { - \operatorname* { d e t } ( \mathcal { G } _ { \alpha \beta } + 2 \pi \alpha ^ { \prime } \mathcal { F } _ { \alpha \beta } ) } ,
S _ { p , m } = \prod _ { A < B } \prod _ { I < J } ( z _ { I } ^ { A } - z _ { J } ^ { B } ) ^ { K _ { A , B } } \prod _ { A } \prod _ { I < J } ( z _ { I } ^ { A } - z _ { J } ^ { A } ) ^ { K _ { A , A } - 1 }
H _ { L C } = \int _ { 0 } ^ { l } \mathcal { H } _ { L C } = \int _ { 0 } ^ { l } d \sigma \, \frac { G _ { + - } } { 2 \, \pi _ { - } } \left( G ^ { i j } \pi _ { i } \pi _ { j } + \frac { 1 } { ( 2 \pi \alpha ^ { \prime } ) ^ { 2 } } G _ { i j } Z ^ { i \, \prime } Z ^ { j \, \prime } \right)
\frac { d \, \operatorname { l o g } \widetilde \mathcal { P } _ { \{ N _ { \nu _ { \alpha _ { 1 } } } ^ { ( \alpha _ { 1 } ) } , \dots , \, N _ { \nu _ { \alpha _ { n } } } ^ { ( \alpha _ { n } ) } \} } ^ { \Lambda , \lambda _ { \Lambda } } } { d \, N _ { \nu _ { \alpha _ { j } } } ^ { ( \alpha _ { j } ) } } \quad = \quad \frac { \int _ { G } d \mu ( g ) \bar { \chi } ^ { ( \Lambda ) } ( g ) [ D _ { \nu _ { 1 } \nu _ { 1 } } ^ { ( \alpha _ { 1 } ) } ] ^ { N _ { \nu _ { \alpha _ { 1 } } } ^ { ( \alpha _ { 1 } ) } } \cdots [ D _ { \nu _ { n } \nu _ { n } } ^ { ( \alpha _ { n } ) } ] ^ { N _ { \nu _ { \alpha _ { n } } } ^ { ( \alpha _ { n } ) } } \operatorname { l o g } [ D _ { \nu _ { j } \nu _ { j } } ^ { ( \alpha _ { j } ) } ] } { \int _ { G } d \mu ( g ) \bar { \chi } ^ { ( \Lambda ) } ( g ) [ D _ { \nu _ { 1 } \nu _ { 1 } } ^ { ( \alpha _ { 1 } ) } ] ^ { N _ { \nu _ { \alpha _ { 1 } } } ^ { ( \alpha _ { 1 } ) } } \cdots [ D _ { \nu _ { n } \nu _ { n } } ^ { ( \alpha _ { n } ) } ] ^ { N _ { \nu _ { \alpha _ { n } } } ^ { ( \alpha _ { n } ) } } } \, .
a _ { M } = { \frac { ( - ) ^ { { \frac { 1 } { 2 } } M ( M + 1 ) } } { M ! [ ( M - 1 ) ! \ldots 2 \cdot 1 ] ^ { 2 } } } ~ ~ ,
{ \cal H } ( p ) = - 2 i \varepsilon ^ { \mu \nu \rho } \varepsilon ^ { \alpha \beta \lambda } g _ { \nu \alpha } \frac { \mu ^ { \epsilon } } { ( 2 \pi ) ^ { d } } \int { \cal D } q \frac { 1 } { ( p + q ) ^ { 2 } q ^ { 2 } } I _ { \beta \mu \lambda \rho } ( q ) \; ,
\mathrm { T r } _ { N S - R } = \frac { V _ { 0 } } { 2 \pi } \int \d E \, \sum _ { m = 0 } ^ { N - 1 } \sum _ { w = - \infty } ^ { \infty } \, \cdots .
R _ { \mu \nu } - \frac { 1 } { 2 } R g _ { \mu \nu } = 0
{ \frac { \alpha } { p ( N ) } } \ \left( { \frac { 1 } { c ( N ) } } - 1 \right) \approx { \frac { 2 A _ { 2 } } { p ( N ) ^ { 2 } } } \, .
{ \cal W } ^ { ( 1 ) } = { \frac { 1 } { { \cal Z } ^ { ( 1 ) } N } } \sum _ { R , S } \frac { d _ { R } } { d _ { S } } \operatorname { e x p } \left[ - { \frac { g ^ { 2 } A _ { 1 } } { 2 } } C _ { 2 } ( R ) - { \frac { g ^ { 2 } A _ { 2 } } { 2 } } C _ { 2 } ( S ) \right] \int d U \mathrm { T r } [ U ] \chi _ { R } ( U ) \chi _ { S } ^ { \dagger } ( U ) ,
A _ { p a r e n t } ^ { \mathrm { I V } } = - \frac { T _ { p } } { 2 } \int d ^ { p + 1 } { \xi } \left[ \sqrt { - g } \left( - { \Psi } ^ { ( p + 3 ) / 2 } g _ { i j } h ^ { i j } + ( p + 3 ) { \Psi } ^ { ( p + 1 ) / 2 } \right) + { \Lambda } \left( { \Psi } - { \Phi } \right) \right] .
F ( z ) \; = \; F ( 0 ) \, + \, \sum _ { m = 1 } ^ { \infty } \, \left\{ \frac { b _ { m } } { z - a _ { m } } + \frac { b _ { m } } { a _ { m } } \right\} \, + \, \sum _ { m = 1 } ^ { \infty } \, \left\{ \frac { b _ { - m } } { z - a _ { - m } } + \frac { b _ { - m } } { a _ { - m } } \right\} \, .
\{ Q ^ { \alpha } , \bar { Q } _ { \beta } \} = - i ( \Gamma ^ { a } ) _ { \beta } ^ { \alpha } P _ { a } - i ( \Gamma ^ { a b c d e } ) _ { \beta } ^ { \alpha } Z _ { a b c d e } ,
\frac { \xi } { \sqrt { 1 - \dot { r } ^ { 2 } } } = \sqrt { x _ { 1 } ^ { 2 } + x _ { 2 } ^ { 2 } } - r ( \tau ) ,
\left( A _ { 0 } , A _ { 1 } , \phi , \pi _ { 0 } , \pi _ { 1 } , \pi _ { \phi } \right) \longleftrightarrow \left( \xi _ { 1 } , . . . , \xi _ { 6 } \right) ,
E ( L ) - E _ { 0 } ( L ) = E _ { R } + E _ { L } + M \operatorname { c o s h } \theta \ ,
v _ { e f f } = \left( { \frac { y _ { i } } { y _ { 0 } } } + 1 \right) ^ { 1 / 3 } e ^ { - n } v _ { i } \, .
\Pi _ { \mu \nu \rho \sigma } ^ { r } ( p , q , k ) = \Pi _ { \mu \nu \rho \sigma } ( p , q , k ) - \Pi _ { \mu \nu \rho \sigma } ( p , 0 , 0 )
\frac { t } { d } = \oint \frac { d z } { 2 \pi \imath } U _ { e f f } ^ { \prime } \left( z + \frac { R ( t ) } { z } \right) = \left( \frac { m _ { 0 } ^ { 2 } } { d } - \Gamma _ { 2 } \right) R ( t ) - \sum _ { n > 2 } \frac { ( 2 n ) ! } { ( n ! ) ^ { 2 } } \Gamma _ { 2 n } R ( t ) ^ { n }
w : = \left( \begin{array} { c } { \tilde { x } _ { 1 } } \\ { . } \\ { . } \\ { . } \\ { \tilde { x } _ { n } } \\ { \hat { y } _ { 1 } } \\ { . } \\ { . } \\ { . } \\ { \hat { y } _ { n } } \\ \end{array} \right) \; , \; z : = \left( \begin{array} { c } { \tilde { y } _ { 1 } } \\ { . } \\ { . } \\ { . } \\ { \tilde { y } _ { n } } \\ { \hat { x } _ { 1 } } \\ { . } \\ { . } \\ { . } \\ { \hat { x } _ { n } } \\ \end{array} \right) \; \; .
p _ { \nu } ^ { \prime } T _ { \mu \nu } ^ { S \rightarrow A A } = 2 m i [ T _ { \mu } ^ { S \rightarrow A P } ] - 4 m ( k _ { 3 } + k _ { 2 } ) _ { \alpha } \triangle _ { \alpha \mu } - 4 m ( l _ { 3 } + l _ { 2 } ) _ { \alpha } \triangle _ { \alpha \mu } ,
\sum _ { j = 1 } ^ { n } [ I _ { i j } ] _ { \hat { q } } \, m _ { j } = 2 \operatorname { c o s } \pi \left( \vartheta _ { h } + t _ { i } \vartheta _ { H } \right) \, m _ { i } \; , \quad \hat { q } = e ^ { i \pi s \vartheta _ { H } } \; .
{ \tilde { f } } ^ { ( 0 ) } = \left( \begin{array} { c c c c c c } { 0 } & { - 1 } & { 0 } & { 0 } & { 0 } & { q \partial _ { y } } \\ { 1 } & { 0 } & { 0 } & { 0 } & { 0 } & { q } \\ { 0 } & { 0 } & { 0 } & { 0 } & { 0 } & { 0 } \\ { 0 } & { 0 } & { 0 } & { 0 } & { - 1 } & { q ^ { 2 } } \\ { 0 } & { 0 } & { 0 } & { 1 } & { 0 } & { \partial _ { y } } \\ { - q \partial _ { x } } & { - q } & { 0 } & { - q ^ { 2 } } & { - \partial _ { x } } & { 0 } \\ \end{array} \right) \delta ( x - y )
W ^ { a } = \frac { 1 } { I E } \big ( | Z | ^ { 2 } H ^ { a } - A ^ { a } A _ { b } H ^ { b } + | Z | ^ { 2 } T ^ { a b } \! A _ { b } \big ) \, ,
\Delta + \bar { \Delta } = \frac { ( m + n k / 4 ) ^ { 2 } } { k } + \frac { ( m - n k / 4 ) ^ { 2 } } { k } \, .
\Delta _ { a } ^ { \! l i n } = - \int \frac { k ^ { \prime } } { 2 l } \, d l - \frac { 1 } { 2 } V \, .
\Sigma _ { - } = a _ { 1 } ( x ) \operatorname { e x p } { ( 3 ( \gamma - 1 ) \tau ) } , ~ ~ \Sigma _ { \times } = a _ { 2 } ( x ) \operatorname { e x p } { ( 3 ( \gamma - 1 ) \tau ) } ,
\left[ \widehat { x } _ { 0 } , \widehat { x } _ { i } \right] = { \frac { i } { \kappa } } \widehat { x } _ { i } \qquad \left[ \widehat { x } _ { i } , \widehat { x } _ { j } \right] = 0
\begin{array} { l l l l l l } { \displaystyle D } & { = 3 } & { : \quad } & { S p ( 4 ; R ) } & { \to } & { O S p ( N ; 1 | R ) } \\ { \displaystyle D } & { = 4 } & { : \quad } & { S U ( 2 , 2 ) } & { \to } & { S U ( 2 , 2 | N ) } \\ { \displaystyle D } & { = 6 } & { : \quad } & { U _ { \alpha } ( 4 ; H ) } & { \to } & { U U _ { \alpha } ( N ; 4 | R ) } \\ \end{array}
\int _ { 0 } ^ { \infty } d k \sqrt { k ^ { 2 } + 4 } \left( \frac { 4 \operatorname { c o s } \rho } { k ^ { 2 } + 4 \operatorname { c o s } ^ { 2 } \rho } - \frac { 4 \operatorname { c o s } \rho } { k ^ { 2 } + 4 } + \frac { 4 c _ { + } } { k ^ { 2 } + 4 c _ { + } ^ { 2 } } - \frac { 4 c _ { + } } { k ^ { 2 } + 4 } \right) ,
\displaystyle { \frac { 8 } { \kappa } } V _ { \mu \nu , \, \alpha \beta } ^ { ^ 2 g h o s t } ( k _ { 1 } , k _ { 2 } ; \, p _ { 1 } , p _ { 2 } ) = - 4 \, p _ { _ 1 \mu } \, p _ { _ 2 \nu } \, \eta _ { \alpha \beta } + 8 \, p _ { _ 1 \beta } \, p _ { _ 2 \nu } \, \eta _ { \alpha \mu } - 2 \, p _ { 1 } \cdot p _ { 2 } \, \eta _ { \alpha \mu } \, \eta _ { \beta \nu } + p _ { 1 } \cdot p _ { 2 } \, \eta _ { \alpha \beta } \, \eta _ { \mu \nu } .
\left( 1 + i \gamma \right) ^ { - 1 } \approx \left( 1 - i \gamma \right) .
{ \cal P } : \quad
( \Delta S ) ^ { 0 } = T r \left[ \left( \begin{array} { l l } { K ^ { \mu } { } _ { \nu } } & { K ^ { \mu } { } _ { F } } \\ { K ^ { F } { } _ { \nu } } & { K ^ { F } { } _ { F } } \\ \end{array} \right) \operatorname { e x p } \frac { 1 } { M ^ { 2 } } \tilde { \cal R } \right] \ ; \qquad \tilde { \cal R } = \left( \begin{array} { l l } { S _ { \mu \nu } } & { q _ { \mu } } \\ { - \tilde { \partial } q _ { \nu } ^ { T } } & { \tilde { \partial } \tilde { \nabla } } \\ \end{array} \right) \ .
[ \Pi ( { \bf { x } } \sigma ) , \rho ( { \bf { y } } \sigma ^ { ' } ) ] = i \mathrm { ~ } \delta ( { \bf { x } } - { \bf { y } } ) \delta _ { \sigma , \sigma ^ { ' } }
E ^ { ( R ) } = { \frac { n _ { R } } { | G | } } \sum _ { g \in G } \chi ^ { ( R ) } ( g ) U ( g ) ~ ,
E _ { \small C a s i m i r } ^ { \small b u l k } = { \frac { 1 } { 8 \pi ^ { 2 } } } \; V \; \hbar c \; K ^ { 4 } \; \left[ { \frac { 1 } { n } } - 1 \right] .
H _ { 0 } = \int _ { 0 } ^ { \infty } d p \, p [ A ^ { \dag } ( p ) A ( p ) - B ^ { \dag } ( p ) B ( p ) ] .
\Omega ^ { J } \equiv ( v _ { i } ^ { J } ) ^ { T } \frac { \partial V ( q ) } { \partial q _ { i } } = 0 ,
\{ Q ^ { + } , Q ^ { - } \} = H _ { n } + k ( R - 1 ) , \quad [ H _ { n } , Q ^ { \pm } ] = \mp 2 k Q ^ { \pm } , \quad [ R , Q ^ { \pm } ] = \pm 2 Q ^ { \pm } , \quad R ^ { 2 } = 1 .
\partial _ { \mu } < T T R > = ( N _ { c } ^ { 2 } - 1 ) \cdot \left[ \mathrm { \vspace { 2 e x } \mathrm { 1 } \vspace { 0 e x } } \right] + 3 \cdot ( N _ { c } ^ { 2 } - 1 ) \cdot \left[ - \frac { 1 } { 3 } \right] = 0
\frac { \partial g _ { k } } { \partial x _ { j } } = \frac { \partial h _ { k } } { \partial y _ { j } } \, \, \, ; \, \, \, \frac { \partial h _ { k } } { \partial x _ { j } } = - \frac { \partial g _ { k } } { \partial y _ { j } } ,
{ \cal D } [ \partial ] : = 1 + { \frac { \partial ^ { 2 } } { \eta ^ { 2 } } } - { \frac { \partial ^ { 4 } } { \gamma ^ { 4 } } } .
Z _ { 0 } ^ { F } ( T , L ) \equiv \operatorname { e x p } \left( \frac { \pi L T } { 6 } \right) .
x \mapsto \omega ( \Xi ) ( x ) , \quad \ x \in { \cal O } ,
\left. \begin{array} { l } { \rho ( 1 ) = 1 } \\ { \rho ( 2 ) = 2 } \\ { \ldots } \\ { \rho ( p ) = p } \\ \end{array} \right\}
S ^ { \mu \nu } \equiv { \frac { \partial L } { \partial \sigma _ { \mu \nu } } } = L ^ { - 1 } [ J ^ { 2 } \sigma ^ { \mu \nu } + { \frac { M J } { { ( { \frac { a _ { 1 } a _ { 2 } } { 2 } } + a _ { 3 } } ) ^ { \frac { 1 } { 2 } } } } ( a _ { 1 } \sigma ^ { \mu \nu } + ( u ^ { \mu } \sigma ^ { \nu \lambda } - u ^ { \nu } \sigma ^ { \mu \lambda } ) u _ { \lambda } ) ] .
H = 1 + Q _ { 1 } \sum _ { n = - \infty } ^ { \infty } \frac { 1 } { \mid \vec { y } - 2 \pi n a \hat { z } \mid ^ { 6 } } ,
z ( \varepsilon , \tau ) \; = \; u \: + \: 2 \tau v \: + \: ( 1 - \varepsilon ^ { 2 } ) \: \tau ^ { 2 } \: q ^ { 2 } \: - \: \frac { q ^ { 2 } } { 4 } \; .
V ( \Phi ) = - m ^ { 2 } \mathrm { T r } [ \Phi ^ { 2 } ] + h ( \mathrm { T r } [ \Phi ^ { 2 } ] ) ^ { 2 } + \lambda \mathrm { T r } [ \Phi ^ { 4 } ] + V _ { 0 }
g \approx 3 - \sqrt 3 - 0 . 9 1 7 7 f _ { 0 } ^ { 2 } \; .
\varepsilon ( p ^ { 0 } ) = \theta ( p ^ { 0 } ) - \theta ( - p ^ { 0 } ) = \frac { p ^ { 0 } } { | p ^ { 0 } | } ,
\tilde { \omega } _ { m a b } = \hat { \omega } _ { m a b } - \frac { i \kappa ^ { 2 } } { 4 } \bar { \psi } _ { c } \gamma ^ { c d } { } _ { m a b } \psi _ { d } .
s p ^ { \mu } = \eta ^ { \alpha } e _ { \alpha } ^ { \mu } ( P _ { 0 } )
q ^ { ( \frac { L } { 2 } - \frac { r } { N } ) } \sum _ { k = l + 1 } ^ { N } q ^ { - m _ { 0 } } B ^ { ( N ) } ( L , r , l + 1 | k ) = \sum _ { k = l + 1 } ^ { N } B ^ { ( N ) } ( L , r , l | k ) .
[ \Phi ( x ; a _ { 1 } , . . . , a _ { n } ) ] _ { R } = \Phi ( x ; a _ { 1 } , . . . , a _ { n } ) +
Z = { \operatorname* { d e t } } _ { T } ^ { - \frac 1 2 } ( - \Delta ) \times { \operatorname* { d e t } } _ { S } ^ { \frac 1 2 } ( - \Delta ) ,
\Omega ( z ) = \int _ { \c C } d w \rho ( w ) \operatorname { l n } ( z - w )
\mathbf { J } ^ { 2 } = ( \mathbf { J } ^ { 2 } ) ^ { \dagger } = G ( \mathbf { J } ^ { 2 } ) ^ { + } G ,
m ^ { 1 ; 1 ; 9 } , \ m ^ { 1 ; 4 ; 9 } , \ m ^ { 2 ; 6 ; 9 } , \ m ^ { 4 ; 7 ; 9 } , \ m ^ { 7 ; 7 ; 9 } , \ m ^ { 2 ; 3 ; 9 ; 9 } , \dots
E _ { \pm } \approx \pm m e ^ { - \frac { \mu ^ { 2 } } { m } \Delta x } .
M ( S , R ) _ { n } = \left( \begin{array} { l l } { 1 } & { 0 } \\ { 1 } & { 1 } \\ \end{array} \right) ^ { \otimes ( n - 1 ) } \ ,
\left[ - \left( 1 - \frac { 2 M } { \rho } \right) \frac { d ^ { 2 } } { d \rho ^ { 2 } } - \frac { 2 ( \rho - 4 M ) ( 2 \rho - 3 M ) } { \rho ^ { 2 } ( \rho - 3 M ) } \frac { d } { d \rho } + \frac { 8 M } { \rho ^ { 2 } ( \rho - 3 M ) } \right] H _ { 1 } ( \rho ) = \lambda H _ { 1 } ( \rho ) ,
G _ { \alpha \beta } ( \tau , \tau ^ { \prime } ) = \frac { 1 } { 2 } \int \Sigma _ { \alpha \beta } ( \tau , \lambda ^ { \prime } ) \epsilon ( \lambda ^ { \prime } - \tau ^ { \prime } ) \; d \lambda ^ { \prime }

\gamma _ { 1 } = - \gamma - 1 - \gamma ( - 2 - a - b ) - \delta ( 2 - a / \delta - b / \delta - 2 a - 2 b ) - 9 / 8 8 ( 1 + a + b ) ,
j ^ { m i } \equiv \frac { \delta I _ { S I } } { \delta v _ { m } ^ { i } } , \quad K ^ { m i } \equiv \frac { \delta I _ { S I } } { \delta A _ { m } ^ { i } } .
[ \hat { x } _ { \mu } , \hat { x } _ { \nu } ] = \frac { i } { \kappa } ( a _ { \mu } \hat { x } _ { \nu } - \hat { x } _ { \mu } a _ { \nu } )
G _ { \mu \nu } ^ { 0 } = \frac { \epsilon _ { \mu \nu \lambda } p _ { \lambda } + \delta _ { \mu \nu } M + p _ { \mu } p _ { \nu } / M } { p ^ { 2 } + M ^ { 2 } } \; , ~ ~ D _ { \mu \nu } ^ { 0 } ( p ) = \frac { \epsilon _ { \mu \nu \lambda } p _ { \lambda } } { p ^ { 2 } } \; , ~ ~ \mathrm { a n d } ~ ~ \Gamma _ { \mu \nu \lambda } ^ { 0 } = g \epsilon _ { \mu \nu \lambda }
\left\{ Q _ { a } , Q _ { b } \right\} _ { { \footnotesize P B } } = f _ { a b c } Q _ { c } \; ,
A d j ( S p ( 2 n _ { H } ) ) \to A d j ( U ( k ) ) + A d j ( U ( n _ { H } - k ) ) + 2 ( k , n _ { H } - k )
C _ { 0 } ( k , q ) = < k , q | \sum _ { j = 0 } ^ { m - 1 } R ^ { - j } \omega ^ { j s } | \phi > = \sum _ { j = 0 } ^ { m - 1 } ( R ^ { j } | k , q > ) ^ { + } \omega ^ { j s } | \phi > .
G ^ { \dagger } = G , \; \; \; \; \; \; \tilde { G } ^ { \dagger } = \tilde { G }
I _ { n } ( x ) = \int _ { 0 } ^ { x } d x _ { n } \int _ { 0 } ^ { x _ { n } } d x _ { n - 1 } \cdots \int _ { 0 } ^ { x _ { 2 } } d x _ { 1 } \cdot 1 = \frac { x ^ { n } } { n ! } .
N ^ { \mu \nu } = L ^ { i } n ^ { [ \mu } { } _ { i } X ^ { \nu ] } { } ^ { \prime } = L ^ { * } \eta _ { 1 } ^ { [ \mu } X ^ { \nu ] } { } ^ { \prime } \, .
X ^ { + } = { \frac { * ( 2 { \cal D } X ^ { i } \wedge f ^ { + i } ) } { * ( f ^ { + i } \wedge f ^ { - i } ) } } = { \frac { 1 } { \cal R } } ( { \cal D } _ { - } X ^ { i } f _ { + } ^ { ~ + i } - { \cal D } _ { + } X ^ { i } f _ { - } ^ { ~ + i } )
T ( z ) = L ^ { a b } : J _ { a } ( z ) J _ { b } ( z ) :
R ( b ) = \frac { 4 } { k - 2 } \; \frac { k ( k - 4 ) + k ( k - 2 ) b } { [ k + 2 + ( k - 2 ) b ] ^ { 2 } }
n ! \prod _ { i = 1 } ^ { k } d _ { i } ! { \frac { 1 } { ( n - m ) ! } } .
[ R , M ] = - \mathrm { \frac { i } { 2 } } \sum _ { \rho \in \Delta _ { + } } g _ { | \rho | } \thinspace { \frac { | \rho | ^ { 2 } \operatorname { c o s } ( \rho \cdot q ) } { \operatorname { s i n } ^ { 2 } ( \rho \cdot q ) } } \thinspace [ \mathrm { e } ^ { 2 \mathrm { i } q \cdot \hat { H } } , \hat { s } _ { \rho } ] .
l n d e t = \frac { | e \Phi | } { 2 \pi } \operatorname { l n } ( m a ) + R ( m ) ,
B ( 0 ) = B _ { \mathrm { f } } ( 0 ) + B _ { \mathrm { b } } ( 0 ) = 0 ,
H \, { \cal U } = i { \frac { \partial } { \partial t } } \, { \cal U } \, ,
\Sigma _ { \pm } ( q _ { 3 } , \left| { \bf q } \right| ) = - \frac { 1 } { 4 \pi } \operatorname { l n } \left[ \frac { 1 } { 2 m } \left( - \frac { 1 } { 2 } \delta \pm i q _ { 3 } + \frac { \left| { \bf q } \right| ^ { 2 } } { 2 m } \right) \right] \, + \, O \left( \frac { 1 } { m ^ { 2 } } \right) .
\frac { g _ { Y M } ^ { 2 } N } { J ^ { 3 } } \frac { J ^ { 4 } } { N ^ { 2 } } \frac { g ^ { 2 } N } { J ^ { 2 } }
{ \cal L } _ { n } ( H _ { n } ) = ( - T _ { H _ { n } } \bar { R } _ { H _ { n } } ) \phi ^ { n } .
\fbox { \begin{array} { l l } { ( x y ) ( a x ) = x ( a y ) x ; } & { ( x y ) _ { L } x _ { R } = x _ { L } x _ { R } y _ { L } ; } \\ { ( x a ) ( y x ) = x ( y a ) x ; } & { ( y x ) _ { R } x _ { L } = x _ { L } x _ { R } y _ { R } ; } \\ { ( x a x ) y = x ( a ( x y ) ) ; } & { y _ { R } x _ { L } x _ { R } = x _ { L } ( x y ) _ { R } ; } \\ { y ( x a x ) = ( ( y x ) a ) x ; } & { y _ { L } x _ { L } x _ { R } = x _ { R } ( y x ) _ { L } . } \\ \end{array} }
{ \delta _ { \rho } } { \mathcal { F } _ { D } ^ { 1 } } = { \delta _ { \rho } } { \mathcal { F } _ { D } ^ { 0 } } - \alpha { Q } \oint { d } \hat { s } ( \xi ) \rho ( \xi ) { \partial _ { \hat { n } } } { \hat { G } _ { D } } ( \xi , \xi ^ { \prime } ) + ( { \alpha ^ { 2 } } - \alpha { Q } ) \rho ( \xi ^ { \prime } ) .
D \sum _ { I \neq J } ( - 1 ) ^ { p _ { I } p _ { J } } e _ { J I } \otimes e _ { I J } D ^ { - 1 } = \sum _ { I \neq J } e _ { J I } \otimes e _ { I J }
H ( t ) | n ; t > = E _ { n } ( t ) | n ; t > , ~ ~ < m ; t | n ; t > = \delta _ { m n }
E _ { c } = \frac { 4 k l } { R } \left( k - \frac { r _ { + } ^ { 2 } } { l ^ { 2 } } \right) ,
P \sim \operatorname { e x p } \left( - y _ { 0 } ^ { 3 } / M _ { p } ^ { 2 } \ell \right) ,
\tilde { W } [ \eta | t ] = \int _ { \eta _ { 0 } } ^ { \eta ( t ) } \delta \eta ^ { ' \nu } ( t ) \, \tilde { E } _ { \nu } [ \eta ^ { \prime } | t ] ,
\partial _ { z } \partial _ { \bar { z } } \theta = \{ n ( n + 1 ) \eta ^ { 2 } + a \} \theta \, , \quad a \in { \cal R } \, , \quad n = 0 , 1 , 2 , \ldots
C _ { 0 } ^ { o p } = g _ { o } ^ { - 2 } \frac { 1 } { \left( 2 \alpha ^ { ^ { \prime } } \right) ^ { d / 2 } } ,
\alpha ^ { 2 } \partial _ { x ^ { - } } ^ { 3 } g _ { a } ( x ) = 0 ~ ; \alpha \neq 0
J _ { g f } = \left( \begin{array} { c c } { 0 } & { J _ { + } } \\ { 1 } & { 0 } \\ \end{array} \right) \quad ; \quad \tilde { J } _ { g f } = \left( \begin{array} { c c } { 0 } & { 1 } \\ { J _ { + } } & { 0 } \\ \end{array} \right) \quad ; \quad g = \left( \begin{array} { c c } { - J _ { + } g _ { 2 2 } } & { - g _ { 2 1 } } \\ { g _ { 2 1 } } & { g _ { 2 2 } } \\ \end{array} \right) .
Z ^ { i n t } ( s ) = \int { \cal D } A _ { \mu } ~ e ^ { - S _ { b o s } ( A ) - i \int d ^ { 3 } x ~ \varepsilon ^ { \mu \nu \rho } A _ { \mu } \partial _ { \nu } s _ { \rho } } ~ ,
\hat { \Gamma } _ { 0 } \ldots \hat { \Gamma } _ { ( 1 0 ) } = 1
{ \cal L } = - V ( T ) \sqrt { - \operatorname* { d e t } ( g + F ) } { \cal F } ( z ) ,
W _ { \infty } ^ { 3 , D } = { \frac { 1 } { 1 6 } } ( 1 - \gamma ) ^ { - 1 } \sum _ { k = 0 } ^ { \infty } ( - 1 ) ^ { k } \beta ^ { k } ( k + 1 ) ( k + 3 ) ( k + 5 ) \sum _ { m = 0 } ^ { \infty } ( m ^ { 2 + k } - m ^ { 1 + k } ) \alpha _ { m } ^ { - k - 7 } .
\begin{array} { c } { \left\{ M _ { 1 } , M _ { 2 } \right\} = a M _ { 1 } M _ { 2 } - M _ { 1 } M _ { 2 } a , } \\ { a = \frac 1 2 \left( r - r ^ { * } \right) . } \\ \end{array}
B ( T ) = - \; { \frac { b _ { 2 } ( V , T ) } { [ b _ { 1 } ( V , T ) ] ^ { 2 } } } \; ,
{ \cal H } = { \cal H } _ { a } \otimes { \cal H } _ { A } \otimes { \cal H } _ { c } .
H r ( \tau ) = \frac { k } { \sqrt { 1 + k ^ { 2 } } } \mid \mathrm { s n } [ \frac { \tau } { \sqrt { 1 + k ^ { 2 } } } , \; k ] \mid .
\delta _ { \beta } \left( \alpha \right) = \alpha \wedge \beta = \alpha _ { a b } \cdot \beta _ { c d } \left( \chi ^ { a b } \otimes \chi ^ { c d } \right) A d _ { R }
T \sim \frac { m } { L ^ { p } } \sim \frac { 1 } { g \ell _ { s } ^ { p + 1 } } .
( - 1 ) ^ { | X | | Z | } [ X , [ Y , Z ] ] + ( - 1 ) ^ { | X | | Y | } [ Y , [ Z , X ] ] + ( - 1 ) ^ { | Y | | Z | } [ Z , [ X , Y ] = 0 ,
{ \cal M } _ { b } ^ { u u } = \frac { i g ^ { 4 } } { 2 } [ T ^ { a } T ^ { b } \otimes T _ { a } T _ { b } ] \int \frac { d ^ { 2 } k } { ( 2 \pi ) ^ { 2 } } \left[ \frac { T ( k , p _ { 1 } ) T ^ { * } ( k , p _ { 3 } ) } { w _ { k } - w _ { p } } \right]
\int _ { C } \frac { d \nu \: \nu } { i 4 \sqrt { 2 } \operatorname { s i n } \pi \nu } \left[ J _ { \nu } ( z _ { 1 } ) J _ { - \nu } ( z _ { 2 } ) + J _ { - \nu } ( z _ { 1 } ) J _ { \nu } ( z _ { 2 } ) \right] \; \; \; \; \; \; \; \; \; \; \; \; \; \; \; \; \; \; \; \; \;
\zeta _ { 0 } ( \nu ) = - { \frac { \nu \varrho ^ { - 2 \nu } } { \pi } } \int _ { \mu } ^ { \infty } d \omega \int _ { C _ { + } } d z { \frac { 2 z ^ { 2 } } { ( z ^ { 2 } + \omega ^ { 2 } ) ^ { \nu + 1 } } } \breve { \Psi } ( \omega ; z ) e ^ { i \epsilon z } ~ ~ ~ ,
\Psi = { C } \Psi ^ { \ast } \, .
a _ { \alpha } W _ { i } Z _ { \alpha } ^ { I } Y _ { 0 } ^ { \alpha } + W Z _ { \alpha } ^ { I } Y _ { I } ^ { \alpha }
Q _ { A } + ( \gamma ^ { 6 } \gamma ^ { 7 } \gamma ^ { 8 } \gamma ^ { 9 } ) _ { A } ^ { B } \tilde { Q } _ { B } \nonumber
W _ { ( 1 , 2 \ell , 1 ) } W _ { ( 1 , 1 , Z _ { 2 } ) } W _ { ( 1 , 2 \ell - 1 , 1 ) } W _ { ( 1 , 1 , Z _ { 2 } ) }
V ( z ) = A ^ { 2 } e ^ { - 2 \alpha z } - 2 A e ^ { - \alpha z } \ , \ A , \alpha > 0 \ .
\sigma _ { \pm } = \frac { \tau \pm \sigma } { 2 } \ .
k ^ { u } ( K _ { u v } ) _ { j } ^ { i } = \partial _ { v } P _ { j } ^ { i } + [ p _ { v } , P ] _ { j } ^ { i } ,
\rho _ { 0 } ( \theta ) = \frac { 1 } { \pi } \sqrt { \mu - \frac { 1 } { 4 } \mu ^ { 2 } \theta ^ { 2 } } ,
C _ { A B C } = \frac { 1 } { 2 4 \pi } \big ( A _ { [ A } ^ { i } \partial _ { B } A _ { C ] } ^ { i } + i \frac { 2 } { 3 } f ^ { i j k } A _ { [ A } ^ { i } A _ { B } ^ { j } A _ { C ] } ^ { k } \big ) .
= g ^ { 2 } \left( \begin{array} { c c } { \delta _ { \mu \nu } \delta _ { A B } \delta ^ { 4 } ( x - y ) } & { 0 } \\ { 0 } & { \delta _ { A B } \delta ^ { 4 } ( x - y ) } \\ \end{array} \right)
\Psi , ~ ~ ~ \Phi , ~ ~ ~ \Pi , ~ ~ ~ \Xi , ~ ~ ~ { \cal A } _ { w } , ~ ~ ~ { \cal A } _ { \theta } , ~ ~ ~ { \cal A } , ~ ~ ~ \Delta , ~ ~ ~ \Delta ^ { * } .
C _ { N _ { P + 1 } + \dots + N _ { P + K } } ^ { N _ { P + 1 } } C _ { N _ { P + 2 } + \dots + N _ { P + K } } ^ { N _ { P + 2 } } \dots C _ { N _ { P + K - 1 } + N _ { P + K } } ^ { N _ { P + K - 1 } }
( N _ { c } + \tilde { N } _ { c } , \overline { { N _ { c } + \tilde { N } _ { c } } } ) ( 1 , 1 , 0 , \beta _ { i } ) , \ \ \ i = 1 , 2 , 3
{ \cal D } = \sum _ { i , j } c _ { i , j } \, \frac { ( g ^ { 2 } N ) ^ { i } } { N ^ { 2 j } } ,
{ \cal R } = { \cal R } _ { r } \operatorname { s i n } \eta , ~ ~ ~ \tau = { \cal R } _ { r } ( 1 - \operatorname { c o s } \eta ) .
\left\{ \begin{array} { r c l } { \displaystyle \partial ^ { m } h _ { m } - \frac { 1 } { 2 } \dot { h } ^ { \prime } } & { = } & { 0 , \vspace { 2 m m } } \\ { \displaystyle \partial ^ { n } h _ { m n } - \frac { 1 } { 2 } \partial _ { m } h ^ { \prime } } & { = } & { 0 , \vspace { 2 m m } } \\ { \displaystyle \partial ^ { m } \theta _ { m } - \frac { 1 } { 2 } h ^ { \prime } } & { = } & { 0 . } \\ \end{array} \right.
Z ( { j _ { k } } ) = e ^ { i Z _ { c } ( { j _ { k } } ) }
| \{ m s s \} | = \sum _ { \nu = 0 } ^ { k } { \binom { k } { \nu } } = ( 1 + 1 ) ^ { k } = 2 ^ { k }
E _ { 4 s } = 2 ( Q + Q ^ { - 1 } ) + \xi _ { 1 } + \xi _ { 1 } ^ { - 1 } + \xi _ { 2 } + \xi _ { 2 } ^ { - 1 }
\stackrel { \mathrm { G } } { { \mathcal L } } \, : = \frac { 1 } { 2 m } \left[ ( D _ { \alpha } \overline { { \Psi } } ) D ^ { \alpha } \Psi - m ^ { 2 } \overline { { \Psi } } \Psi \right] \, .
p ( x ) = \prod _ { i = 1 } ^ { N + 1 } ( x + \nu _ { i } ) \, , \quad q ( x ) = p ( x ) - e ^ { i \theta } / r ^ { N + 1 } = \prod _ { i = 1 } ^ { N + 1 } ( x - x _ { i } ) \, ,
\tau _ { c l } = { \frac { \theta _ { 0 } } { 2 \pi } } + { \frac { 4 \pi i } { g _ { 0 } ^ { 2 } } } ,
t = ( H ^ { - 2 } - r ^ { 2 } ) ^ { 1 / 2 } \operatorname { s i n h } ( H \tau ) ; \quad v = ( H ^ { - 2 } - r ^ { 2 } ) ^ { 1 / 2 } \operatorname { c o s h } ( H \tau ) ; \quad x = r
\begin{array} { l l l } { \langle q _ { i 0 } - Q _ { i 0 } \rangle } & { = } & { 0 \, , } \\ { \langle \dot { q } _ { i 0 } \rangle } & { = } & { 0 \, , } \\ { m \omega _ { i } \omega _ { j } \langle ( q _ { i 0 } - Q _ { i 0 } ) ( q _ { j 0 } - Q _ { j 0 } ) \rangle } & { = } & { k _ { B } T \delta _ { i j } \, , } \\ { m \langle \dot { q } _ { i 0 } \dot { q } _ { j 0 } \rangle } & { = } & { k _ { B } T \delta _ { i j } \, , } \\ { \langle \dot { q } _ { i 0 } ( q _ { j 0 } - Q _ { j 0 } ) \rangle } & { = } & { 0 \, . } \\ \end{array}
i \frac { \partial } { \partial \tau _ { 2 } } < m ^ { \prime } , \bar { m ^ { \prime } } ; \tau _ { 2 } | m , \bar { m } ; \tau _ { 1 } > = \hat { H } ( m ^ { \prime } , \bar { m ^ { \prime } } ) < m ^ { \prime } , \bar { m ^ { \prime } } ; \tau _ { 2 } | m , \bar { m } ; \tau _ { 1 } > .
f = \frac { 1 } { 2 } \partial _ { \sigma } Z .
{ \cal L } _ { e f f } = { \cal L } ^ { ( 2 ) } + { \cal L } ^ { ( 4 ) } + { \cal L } ^ { ( 6 ) } + . . .
{ \bf E } _ { \pm } ( \xi _ { \pm } ) = \mathrm { T } _ { \pm } ( \xi _ { \pm } ) { \bf E } _ { 0 } \mathrm { T } _ { \pm } ^ { \dagger } ( \xi _ { \pm } ) ,
\psi ^ { \prime } ( t , x ) = e ^ { - i e \epsilon ( t , x ) } \psi ( t , x ) \ \ , \ \ A _ { \mu } ^ { \prime } ( t , x ) = A _ { \mu } ( t , x ) + \partial _ { \mu } \epsilon ( t , x ) \ ,
\delta ( f ( x ) ) = \sum _ { i = 1 } ^ { n } \frac { 1 } { | f ^ { \prime } ( x _ { i } ) | } \delta ( x - x _ { i } )
V _ { A } = T ^ { a } \int _ { 0 } ^ { T } d \tau \, \bigl [ \dot { x } _ { \mu } \varepsilon _ { \mu } - 2 \mathrm { i } \psi _ { \mu } \psi _ { \nu } k _ { \mu } \varepsilon _ { \nu } \bigr ] \mathrm { e x p } [ i k x ( { \tau } ) ]
\Omega : u \rightarrow \ { S _ { \Omega } } \left( u \right) = { { \Omega } ^ { - 1 } } u .
t _ { a } ( M ) - t _ { a } ^ { * } ( M ) = t _ { a } ( M ) ( R - R ^ { * } ) t _ { a } ^ { * } ( M )
- { \cal L } _ { \xi } J ^ { \mu \nu } = \hat { \xi } ^ { \lambda } \partial _ { \lambda } J ^ { \mu \nu } - \partial _ { \lambda } \hat { \xi } ^ { \mu } J ^ { \lambda \nu } - \partial _ { \lambda } \hat { \xi } ^ { \nu } J ^ { \mu \lambda }
\widehat { F } _ { \mu \nu } = F _ { \mu \nu } + \frac { 1 } { 4 } \theta ^ { \rho \sigma } \bigg ( 2 \left\{ F _ { \mu \rho } , F _ { \nu \sigma } \right\} - \left\{ A _ { \rho } , D _ { \sigma } F _ { \mu \nu } + \partial _ { \sigma } F _ { \mu \nu } \right\} \bigg ) + { \cal O } \left( \theta ^ { 2 } \right) .
| q ( x , \xi ) | = | \chi ( x , \xi ) a ( x , \xi ) ^ { - 1 } | \geq C _ { K } ( 1 + | \xi | ) ^ { - m } ,
d s _ { M } ^ { 2 } = g ^ { 4 / 3 } d x _ { 1 1 } ^ { 2 } + g ^ { - 2 / 3 } d s _ { 1 0 } ^ { 2 }

S _ { m a t t e r } = 1 6 \pi \int d ^ { 4 } x \sqrt { - g } \; e ^ { 2 ( \sigma - 1 ) \psi } \; L _ { m a t t e r } .
H ^ { ( N ) } ( L , r , l | d , p ) = \sum _ { k = l + p } ^ { N } q ^ { - \sum _ { i = d } ^ { p - 1 } m _ { i } - e _ { d - 1 } C ^ { - 1 } m } B ^ { ( N ) } ( L , r , l + 1 | k ) .
\sigma = 4 \pi g ^ { 2 } M ~ ~ \mathrm { a n d } ~ ~ \alpha ^ { - 1 } = - \frac { \pi g ^ { 2 } } { 2 M } ,
d _ { \mathrm { e l } } ^ { - 2 } = \rho _ { \mathrm { p o l } } / A _ { 0 } \sim g ^ { 2 } T a ^ { 2 - d } \, .
V ( \phi ) \ = \ < H _ { \mu \nu } ^ { 2 } > \cdot ( 1 - [ \rho ^ { 2 } + \sigma ^ { 2 } ] ) ^ { 2 } \ \sim \ \lambda ( 1 - | \phi | ^ { 2 } ) ^ { 2 }
\int _ { \Sigma } ( i ( D g ) \varphi - ( A g + i g ) \Lambda ) \Psi _ { 0 } = \int _ { \Sigma } g ( i D \varphi + C \Lambda - i \Lambda ) \Psi _ { 0 }
V _ { \mu } ^ { s } ( x , p ) = { \pi } ^ { - 4 } \partial _ { \nu } ^ { ( x ) } \int d ^ { 4 } q e ^ { 2 i p . q } { \bar { \psi } } ( x + q ) { \frac { \sigma _ { \mu \nu } } { 2 m } } \psi ( x - q ) .
- \left( 2 s + 1 \right) - \frac { \left( 2 s + 1 \right) ( e \tau ) ^ { 2 } \mathcal { F } } 3 \left[ s ( s + 1 ) \left( g ^ { 2 } - \sigma ^ { 2 } \right) - 1 \right] \biggl \} ,
\int _ { 0 } ^ { t } d t ^ { \prime } \int d ^ { 2 } \xi ^ { \prime } K _ { \xi \xi ^ { \prime } } ^ { t - t ^ { \prime } } e ^ { - \phi ( \xi ) } \left( - \frac { 1 } { 2 } ( \xi ^ { \prime } - \xi ) ^ { a } ( \xi ^ { \prime } - \xi ) ^ { b } \left( \partial _ { a } \phi ( \xi ) \partial _ { b } \phi ( \xi ) - \partial _ { a } \partial _ { b } \phi ( \xi ) \right) \right) \Delta ^ { \prime } K _ { \xi ^ { \prime } \xi } ^ { t ^ { \prime } }
m \epsilon _ { l n } F _ { n 0 } - j _ { l } = 0 ,
i D _ { k } M = [ C _ { k } , M ] = B _ { k l } [ x ^ { l } , M ] + [ \hat { A } _ { l } , M ] \ .
{ \tilde { W } } \equiv W - S _ { 0 } ~ , \quad { \tilde { X } } \equiv X + S _ { 0 } ~ .
\Delta _ { \mu \nu } ( p ) = \frac { 4 \pi } { \kappa } \frac { \mu } { p ^ { 2 } ( p ^ { 2 } + \mu ^ { 2 } ) } ( \mu \epsilon _ { \mu \nu \lambda } p ^ { \lambda } + \delta _ { \mu \nu } p ^ { 2 } - p _ { \mu } p _ { \nu } )
{ \mathrm { T r } } ( \gamma _ { g } ) = - 2 ~ , ~ ~ ~ { \mathrm { T r } } ( \gamma _ { R _ { s } } ) = { \mathrm { T r } } ( \gamma _ { g R _ { s } } ) = 0 ~ .
G _ { \alpha } ^ { ( n ) } \equiv \sum _ { m = 0 } ^ { n } \{ \tau _ { \alpha } ^ { ( n - m ) } , \tilde { H } ^ { ( m ) } \} + \sum _ { m = 0 } ^ { n - 2 } \{ \tau _ { \alpha } ^ { ( n - m ) } , \tilde { H } ^ { ( m + 2 ) } \} _ { ( \eta ) } + \{ \tau _ { \alpha } ^ { ( n + 1 ) } , \tilde { H } ^ { ( 1 ) } \} _ { ( \eta ) } ; \hspace { 0 . 5 c m } n \geq 2 .
\tilde { f } _ { 2 } = \Phi ( x - z _ { 2 } , x - z _ { 2 } )
D _ { \mu } D ^ { \mu } \phi ^ { a } = f _ { \phantom { a } b c } ^ { a } \psi _ { \mu } ^ { b } \psi _ { \mu } ^ { c } .
\delta _ { { \hat { \xi } } _ { 2 } } L [ { \hat { \xi } } _ { 1 } ] = \left\{ L [ { \hat { \xi } } _ { 2 } ] , L [ { \hat { \xi } } _ { 1 } ] \right\} = L [ \{ { \hat { \xi } } _ { 1 } , { \hat { \xi } } _ { 2 } \} _ { \mathrm { \scriptsize ~ S D } } ] + K [ { \hat { \xi } } _ { 1 } , { \hat { \xi } } _ { 2 } ]
U _ { \bf n } ^ { l } ( a ) = U _ { \bf n } a \, , \qquad ( a ) U _ { \bf n } ^ { r } = a U _ { A \bf n } \, .
{ \bf S } = - ( { \bf r } - { \bf q } ) \times m { \bf u } ,
{ \bf W } _ { a a } = - \sum _ { c \neq a } { \mathrm { \boldmath ~ \beta ~ } } _ { a } ^ { * } \cdot { \mathrm { \boldmath ~ \beta ~ } } _ { c } ^ { * } { \bf w } _ { a c } , \, \, \, \, { \bf W } _ { a b } = { \mathrm { \boldmath ~ \beta ~ } } _ { a } ^ { * } \cdot { \mathrm { \boldmath ~ \beta ~ } } _ { b } ^ { * } { \bf w } _ { a b } \quad \mathrm { \hspace { ~ 1 c m } ~ i f ~ a \neq ~ b ~ } .
\frac { \mathrm { e } ^ { - x ^ { 2 } / 2 t } } { \left( 2 \pi t \right) ^ { D / 2 } } = \int \frac { \mathrm { d } ^ { D } p } { \left( 2 \pi \right) ^ { D } } \; \mathrm { e } ^ { - i p \cdot x - \frac 1 2 p ^ { 2 } t } ,
{ \frac { 1 } { 2 } } \left\{ \left[ \begin{array} { c } { Q _ { \alpha } } \\ { \tilde { Q } _ { \alpha } } \\ \end{array} \right] , \left[ Q _ { \beta } \ \tilde { Q } _ { \beta } \right] \right\} = \left[ \begin{array} { c c } { 1 } & { 0 } \\ { 0 } & { 1 } \\ \end{array} \right] M \delta _ { \alpha \beta } + \left[ \begin{array} { c c } { p } & { q / g _ { s } } \\ { q / g _ { s } } & { - p } \\ \end{array} \right] { \frac { { L ( \Gamma ^ { 0 } \Gamma ^ { 1 } ) } _ { \alpha \beta } } { 2 \pi { \alpha ^ { \prime } } } } \ .
S = \int d ^ { d - 1 } x \sqrt { | g | } \ \left[ R + { \textstyle \frac { 1 } { 2 } } \left( \partial \varphi \right) ^ { 2 } - { \textstyle \frac { 1 } { 4 } } e ^ { - a \varphi } F _ { ( 2 ) } ^ { 2 } + { \textstyle \frac { 1 } { 2 } } \left( { \cal D } \phi \right) ^ { 2 } - { \textstyle \frac { 1 } { 2 } } m ^ { 2 } e ^ { a \varphi } \right] \, ,
w _ { 1 } = \left( \begin{array} { c c c } { - 1 } & { } & { } \\ \end{array} \right)
F _ { \mu \rho } ^ { a } F _ { a \nu } ^ { \; \; \; \; \rho } - { \frac { 1 } { 4 } } g _ { \mu \nu } x F _ { \rho \sigma } ^ { a } F _ { a } ^ { \rho \sigma }
\pm 1 = \frac { 1 } { 2 { \pi } i } \int T r { \cal P } ^ { ( { \pm } 1 ) } d { \cal P } ^ { ( { \pm } 1 ) } { \wedge } d { \cal P } ^ { ( { \pm } 1 ) } .
\zeta _ { \pm } ^ { 1 } ( z ) = N _ { 1 } \left( \begin{array} { c } { z ^ { \frac { M } { 2 } } ( 1 - z ) ^ { - \frac { M } { 2 } } F ( 1 , - 1 , \frac { 1 } { 2 } + M ; z ) } \\ { \pm \frac { i } { \frac { 1 } { 2 } + M } z ^ { \frac { 1 + M } { 2 } } ( 1 - z ) ^ { \frac { M } { 2 } } F ( - \frac { 1 } { 2 } + M , \frac { 3 } { 2 } + M , \frac { 3 } { 2 } + M ; z ) } \\ \end{array} \right)
S _ { s h e l l } \sim ( \sigma _ { a b } ) ^ { 2 } \frac { r ^ { 2 } } { L ^ { 2 } } L ^ { 2 } \sim \frac { L ^ { 2 } M ^ { 2 } } { r ^ { 2 } ( 1 - 2 M / r ) ^ { 2 } } .
H _ { m n } \equiv F _ { m n } , \, \quad \quad B _ { m } \equiv F _ { 3 m } = - F _ { m 3 } , \,
\delta \vec { \alpha } = { \textstyle \frac 1 2 } \, ( a - F ) \, \delta \theta \, \vec { \alpha } \, , \qquad \delta a = { \textstyle \frac 1 2 } \, ( a - F ) \, \delta \theta \, a \, .
W \to { \frac { m ^ { 3 } N ^ { 2 } E _ { 2 } ( \tau ) } { 9 6 \pi g } } - { \frac { m ^ { 3 } N ^ { 3 } } { 9 6 \pi g q ^ { 2 } ( \tau + k ) ^ { 2 } } } \ .
W [ B , J , \eta ^ { \ast } , \eta ] = \operatorname { l n } Z [ B , J , \eta ^ { \ast } , \eta ] \quad .
\dot { \omega } + e _ { \alpha \beta } \partial _ { \alpha } \psi \partial _ { \beta } \Delta \psi = \nu \Delta \omega \ ,
j _ { ( \varphi ) } ^ { \mu i } = \epsilon ^ { i j } \partial _ { j } x ^ { \mu } \quad , \quad j ^ { \alpha i } = ( 1 / 2 ) \epsilon ^ { i j } \partial _ { j } \theta ^ { \alpha } \ ,
\Delta _ { N , 2 } ^ { \mu \nu , \lambda \sigma } = \frac { 1 } { 2 ( p ^ { 2 } - i \epsilon ) } \left( g ^ { \mu \nu } g ^ { \lambda \sigma } + ( g ^ { \mu \nu } n ^ { \lambda } n ^ { \sigma } + n ^ { \mu } n ^ { \nu } g ^ { \lambda \sigma } ) \frac { p ^ { 2 } } { | { \bf p } | ^ { 2 } } + n ^ { \mu } n ^ { \nu } n ^ { \lambda } n ^ { \sigma } \frac { ( p ^ { 2 } ) ^ { 2 } } { | { \bf p } | ^ { 4 } } \right) .
c _ { l , m } = \eta ( \tau ) ^ { - 3 } \sum _ { - | x | < y \leq | x | } \mathrm { s i g n } ( x ) ~ q ^ { ( k + 2 ) x ^ { 2 } - k y ^ { 2 } }
g ^ { 2 } = - \frac { ( M \mp m _ { 1 } ) ( M \mp m _ { 2 } ) } { ( m _ { 2 } - m _ { 1 } ) ( \mu _ { 1 } I _ { 1 } \pm M I _ { 0 } ) } \, .
S = - \frac { 1 } { 2 } \int d ^ { 4 } x \sqrt { - \bar { g } } \left( \bar { R } - \frac { 1 } { 2 } ( \bar { \nabla } \phi ) ^ { 2 } \right)
R _ { \gamma \alpha \beta } ^ { \mu } = \partial _ { \alpha } \Gamma _ { \beta \gamma } ^ { \mu } - \partial _ { \beta } \Gamma _ { \alpha \gamma } ^ { \mu } + \Gamma _ { \alpha \lambda } ^ { \mu } \Gamma _ { \beta \gamma } ^ { \lambda } - \Gamma _ { \beta \lambda } ^ { \mu } \Gamma _ { \alpha \gamma } ^ { \lambda }
\begin{array} { r l } { { \cal L } = } & { \frac { 1 } { g ^ { 2 } } \int \left( - \frac { 1 } { 4 } F _ { m n } F ^ { m n } + \frac { 1 } { 2 } D _ { m } \Phi ^ { a b } D ^ { m } \Phi _ { a b } + \frac { 1 } { 4 } \{ \Phi ^ { a b } , \Phi ^ { c d } \} _ { * } \{ \Phi _ { a b } , \Phi _ { c d } \} _ { * } \right. + } \\ \end{array}
{ } ^ { \gamma \gamma } G _ { 1 2 3 4 } ^ { 4 } = 4 \left\{ \frac { \delta ^ { 2 } W } { \delta D _ { 1 2 } ^ { - 1 } \delta D _ { 3 4 } ^ { - 1 } } + \frac { \delta W } { \delta D _ { 1 2 } ^ { - 1 } } \frac { \delta W } { \delta D _ { 3 4 } ^ { - 1 } } \right\} = - 2 \frac { \delta \; { } ^ { \gamma } G _ { 1 2 } ^ { 2 } } { \delta D _ { 3 4 } ^ { - 1 } } + { } ^ { \gamma } G _ { 1 2 } ^ { 2 } { } ^ { \gamma } G _ { 3 4 } ^ { 2 } .
y _ { H } \equiv \alpha ^ { - 1 } \sqrt { \mu } r _ { + } , \ \ y _ { i } \equiv \alpha ^ { - 1 } { { \tilde { q } } _ { i } } \ , ( i = 1 , 2 , 3 ) \ ,
\left\langle { { \xi } \left\vert { \xi ^ { \prime } } \right\rangle } \right. = { \frac { \left( { 1 - { \vert \xi \vert } ^ { 2 } } \right) ^ { K } \left( { 1 - { \vert \xi ^ { \prime } \vert } ^ { 2 } } \right) ^ { K } } { \left( { 1 - \xi ^ { * } \xi ^ { \prime } } \right) ^ { 2 K } } } \ ,
S = \frac { 1 } { 4 \ell _ { \mathrm { P } } ^ { 2 } } \int d ^ { 4 } x ~ a ^ { 2 } ~ \eta ^ { \mu \nu } \partial _ { \mu } h \partial _ { \nu } h ,
{ \frac { d ^ { 2 } \tilde { \psi } _ { \infty } } { d { \rho ^ { * } } ^ { 2 } } } - \left[ { \frac { 3 } { 4 } } - \mu \right] { \frac { \rho ^ { 2 } } { R ^ { 4 } } } \tilde { \psi } _ { \infty } = 0 .
\begin{array} { l l l } { U ( 3 ) } & { \longrightarrow } & { U ( 5 ) / U ( 2 ) } \\ \end{array}
\zeta ( z , a ) = \sum _ { n = - \infty } ^ { \infty } \frac { 1 } { ( n ^ { 2 } + a ^ { 2 } ) ^ { z } } \, \, \, \, a ^ { 2 } > 0 ,
k = \pm \omega \frac { \sqrt { 1 + B ^ { 2 } - E ^ { 2 } } \mp E B } { 1 - E ^ { 2 } } \, ,
\lambda _ { m } ^ { ( n - p - 1 ) } = \lambda _ { m } ^ { ( p ) }
\int [ d \bar { \psi } ] [ d \psi ] \operatorname { e x p } \left\{ - \int \bar { \psi } D \psi d x \right\} = \prod _ { n } \lambda _ { n } = { \tt d e t } D ,
( t \partial _ { z } + z ^ { - 1 } ( D _ { \jmath _ { 1 } } ^ { + } D _ { \jmath _ { 1 } } ^ { - } - 2 \jmath _ { 0 } D _ { \jmath _ { 1 } } ^ { 0 } ) ) - ( \tilde { J } ^ { + } ( z ) D _ { \jmath _ { 1 } } ^ { - } + 2 \tilde { J } ^ { 0 } ( z ) D _ { \jmath _ { 1 } } ^ { 0 } + \tilde { J } ^ { - } ( z ) D _ { \jmath _ { 1 } } ^ { + } )
( i d \otimes \pi _ { - } ) \delta _ { V } ^ { 2 } ( x _ { 2 } ^ { j } ( \xi , \eta ) ) = \sum _ { k } { \cal D } ( A _ { 2 } , D _ { 2 } ) _ { k } ^ { j } \ ( i d \otimes \pi _ { - } ) \Delta ( x _ { 2 } ^ { k } ( \xi , \eta ) ) .
P ^ { a } = { \frac { 1 } { 2 } } \pi ^ { 2 } | \phi _ { 0 } | ^ { 2 } \; \zeta ^ { a } .
\sigma _ { q } ( X _ { i } + a ) = \sum _ { r = 0 } ^ { q } \left( \begin{array} { c } { p - q + r } \\ { p - q } \\ \end{array} \right) a ^ { r } \sigma _ { q - r } ( X _ { i } )
\theta _ { s } ( \sum ( x _ { i } - y _ { i } ) / 2 + z - w ) \prod _ { I } \theta _ { s , h _ { I } } ( \sum ( x _ { i } + y _ { i } ) / 2 - \sum u _ { i , I } ) ~ .
L _ { m } ^ { ( g / p ) } \equiv L _ { m } ^ { ( g ) } - L _ { m } ^ { ( p ) }
\partial ^ { \mu } A _ { \mu } ^ { a } \left( x \right) + B ^ { a } \left( x \right) = 0 ,
\frac { d \Phi } { d r } = - \frac { \sqrt { 3 } } { 4 } \ell ( r ) ^ { \Lambda \Sigma } \, \mathrm { I m } { \cal N } _ { \Lambda \Sigma | \Gamma \Delta } \, f _ { ~ ~ A B } ^ { \Gamma \Delta } \, \frac { e ^ { U } } { r ^ { 2 } } \, .
\hat { B } _ { 1 q _ { + } } ( \omega ) = { \frac { 2 \operatorname { s i n h } \Bigl ( ( \nu - q _ { + } ) { \frac { \omega } { 2 } } \Bigr ) \operatorname { c o s h } \Bigl ( ( \nu - 1 ) { \frac { \omega } { 2 } } \Bigr ) } { \operatorname { s i n h } ( { \frac { \nu \omega } { 2 } } ) } } \, .
- \left[ J ^ { 1 } ( x ) \right] _ { - L / 2 } ^ { L / 2 } = \left[ ( / \hspace { - 0 . 5 e m } \partial \phi + U ) \gamma ^ { 1 } \psi - ( - / \hspace { - 0 . 5 e m } \partial \phi + U ) \gamma ^ { 1 } \gamma ^ { 3 } \psi \right] _ { L / 2 } = \left. ( 1 + \gamma _ { 3 } ) ( / \hspace { - 0 . 5 e m } \partial \phi + U ) \gamma ^ { 1 } \psi \right| _ { L / 2 } .
J = \left( \begin{array} { c c } { 1 } & { 0 } \\ { A } & { 1 } \\ \end{array} \right) ,
a _ { 1 } \, = \, b _ { 2 } \, = \, - \, 1 \; \; ; \; \; a _ { 2 } \, = \, b _ { 1 } \, = \, 1 \; \; ; \; \; a _ { 3 } \, = \, b _ { 3 } \, = \, 0 \; \; ,
C ( X ) = c _ { i } ^ { } \, \psi _ { i } ^ { } = c _ { i } ^ { } \, v _ { i } ^ { 1 / 2 } \, \psi _ { i } ^ { - } .
g _ { \mu \nu } \partial _ { n } X ^ { \nu } + 2 \pi \alpha ^ { \prime } B _ { \mu \nu } \partial _ { s } X ^ { \nu } | _ { \partial \Sigma } = 0 ,
\Phi ( x ) = \left( \begin{array} { c } { G _ { \alpha } ^ { \beta } ( x ) } \\ { \Psi _ { \alpha } ( x ) } \\ \end{array} \right)
\Lambda _ { \infty } R = e ^ { a ( \infty ) } \Lambda _ { \overline { { M S } } } R = e ^ { \gamma _ { E } } r / \sqrt { 6 }
\left. \left( \begin{array} { c c c } { \partial \beta _ { g } / \partial g } & { \partial \beta _ { g } / \partial h } & { \partial \beta _ { g } / \partial f } \\ { \partial \beta _ { h } / \partial g } & { \partial \beta _ { h } / \partial h } & { \partial \beta _ { h } / \partial f } \\ { \partial \beta _ { f } / \partial g } & { \partial \beta _ { f } / \partial h } & { \partial \beta _ { f } / \partial f } \\ \end{array} \right) \right| _ { g = g _ { \ast } , h = h _ { \ast } , f = f _ { \ast } }
\mathrm { c } = \frac { \theta } { n } ,
{ \cal K } ^ { + 4 } = \mathrm { T r } \left( [ \tilde { q } ^ { + } , q ^ { + } ] [ \tilde { q } ^ { + } , q ^ { + } ] \right) \, .
\theta _ { \mu } \, \to \, \theta _ { \mu } - i { \tilde { \phi } } ^ { * } \partial _ { \mu } { \tilde { \phi } }
A _ { i } = { \frac { \Phi } { 2 \pi } } \epsilon _ { i j } { \frac { x ^ { j } } { { \vert x \vert } ^ { 2 } } }
h : V _ { n } \mapsto V _ { n } \ , \ n = 0 , 1 , 2 , \ldots
c ^ { 2 } M _ { 3 } ^ { 2 } - m _ { 2 } ^ { 2 } G ^ { 2 } > 0 ,
S _ { 1 } = U \qquad S _ { 2 } = D | _ { \lambda = 1 } \ ,
h _ { c l } = \vec { p } ^ { ~ 2 } / 2 m + V ( \vec { r } )
\begin{array} { r l } { \displaystyle Z _ { N } ( \vartheta ) } & { = 2 N \operatorname { a r c t a n } \displaystyle \frac { \operatorname { s i n h } \vartheta } { \operatorname { c o s h } \Theta } + \displaystyle \sum _ { k = 1 } ^ { N _ { H } } \chi ( \vartheta - h _ { k } ) - 2 \displaystyle \sum _ { k = 1 } ^ { N _ { S } } \chi ( \vartheta - y _ { k } ) - } \\ \end{array}
\hat { x } ^ { + } ( \hat { x } ^ { - } + \frac { m } { \lambda ^ { 3 } x _ { 0 } ^ { + } } ) + \frac { \kappa } { 4 \lambda ^ { 2 } } = 0 .
\sum _ { n } \longrightarrow \int d n = \frac { L } { 2 \pi } \int d k
B _ { \mu } = A _ { \mu } = - 3 G _ { \mu } ^ { 0 } ,
\{ \{ F , G \} , H \} = \sum I _ { A B } I _ { C D } \int _ { \Omega } D _ { I } { \frac { \partial h } { \partial \phi _ { B } ^ { ( J ) } } }
\vec { \nabla } \cdot \vec { E } = g \vec { \nabla } f \cdot \vec { E } \ ,
\partial _ { \bar { z } } T _ { 4 } = \partial _ { z } \Theta _ { 2 } \ ,
{ \cal S } _ { ( - 1 ) } = { \cal S } _ { \mathrm { c u b i c } } + { \cal S } _ { \mathrm { q u a r t i c } }
\int \prod _ { i } d Y _ { i } \delta ( \sum _ { i = 1 } ^ { 6 } w _ { i } Y _ { i } - t ) e ^ { - \sum _ { i } \mu _ { i } Y _ { i } } \ \stackrel { Y _ { i } \rightarrow Y _ { i } + l n \mu _ { i } } { \longrightarrow } \int \prod _ { i } d Y _ { i } \delta ( \sum _ { i = 1 } ^ { 6 } w _ { i } Y _ { i } - t ^ { \prime } ) e ^ { - \sum _ { i } Y _ { i } } ,
h ( \tau ) = ( g _ { s } M \alpha ^ { \prime } ) ^ { 2 } 2 ^ { 2 / 3 } \varepsilon ^ { - 8 / 3 } \int _ { \tau } ^ { \infty } d x { \frac { x \operatorname { c o t h } x - 1 } { \operatorname { s i n h } ^ { 2 } x } } ( \operatorname { s i n h } 2 x - 2 x ) ^ { 1 / 3 } ,
\delta { \cal L } _ { \pm } = J _ { \pm i } \dot { \eta _ { i } }
P \sim \operatorname { e x p } \Bigl ( - { \frac { 3 M _ { P } ^ { 4 } } { 8 V ( \phi ) } } \Bigr ) \ ,
\mu = A _ { 1 } E ( a , x ) + A _ { 2 } E ^ { * } ( a , x ) ,
f ( - 2 p )
g _ { 1 } ( K , \mu ) = { \frac { 1 } { 2 } } \, { \frac { u _ { 1 } ^ { 2 } C _ { 1 } ( \mu ) } { ( K + \sqrt { K ^ { 2 } + \mu ^ { 2 } } ) } } - { \frac { 1 } { 2 } } { \frac { K + \sqrt { K ^ { 2 } + \mu ^ { 2 } } } { C _ { 1 } ( \mu ) } } \, ,
V _ { F } ^ { ( d i v ) } ( H ) = - \frac { N } { 6 4 \pi ^ { 2 } \epsilon } \left( m _ { t } ( H ) ^ { 4 } + \sum _ { k = 1 } ^ { \infty } \left( \frac { 2 k } { R } + m _ { t } ( H ) \right) ^ { 4 } + \sum _ { k = 1 } ^ { \infty } \left( - \frac { 2 k } { R } + m _ { t } ( H ) \right) ^ { 4 } \right)
\tilde { \Gamma } _ { 3 } = - \frac { \lambda ^ { 3 } } { 6 4 \pi ^ { 3 } } \mathrm { l n } \left( T \right) .
f _ { k } ^ { ( D ) } ( \Omega ^ { ( { D } ) } ) = - \Gamma _ { { D } } ( k ) \, \frac { 1 } { 1 / \lambda + { \mathcal G } _ { { D } } ^ { ( + ) } ( { \bf 0 } ; k ) } \; ,
\tilde { \Lambda } _ { S U } ^ { 2 N _ { c } ^ { \prime } - 2 N _ { f } } = ( \mu ^ { - 1 } a ^ { 2 } ) ^ { 2 N _ { f } - 2 N _ { c } } \tilde { \Lambda } ^ { \prime 4 ( N _ { c } ^ { \prime } - 1 ) - 2 ( N _ { f } + \tilde { N } _ { c } ) } .
\left( , \right) = \left( , \right) _ { 1 } + \left( , \right) _ { 2 } .
E ( \alpha = 0 , n , \omega , \beta ) = \frac { 2 \omega + 3 } { 2 \omega + 4 } E ( \alpha = 0 , n , \omega = \infty , \beta ) \ . \
{ \frac { 1 } { 2 } } \sum _ { i = 1 } ^ { n } \mathrm { d e g } ( \phi _ { i } ) = ( 1 - g ) ( d - 3 ) + n + \int _ { \Sigma _ { g } } x ^ { * } ( c _ { 1 } ( X ) )
( 2 \pi { \bar { N } } \dot { \xi } ^ { 2 } ( s ) ) ^ { - 1 } { \cal D } ^ { \mu } ( s ) F _ { \mu } [ \xi | s ] = - [ { \cal D } ^ { \mu } ( s ) , { \cal D } ^ { \nu } ( s ) ] L _ { \mu \nu } [ \xi | s ] .
x ^ { D } R ^ { A } { } _ { B C \bar { D } } = - g ^ { A E } g _ { E B C } \ .
d s ^ { 2 } = ( d x ^ { 0 } ) ^ { 2 } + ( x ^ { 0 } ) ^ { 2 } d \Omega ^ { 2 } , \quad 0 \le x ^ { 0 } \le r ,
U _ { \mathrm { A P } } ^ { \mathrm { b f } } ( \vec { x } ; C ) = e ^ { - \mathrm { i } { \frac { \theta } { 2 } } \tau ^ { 3 } } U _ { \mathrm { A P } } ^ { \mathrm { b f } } ( D ( e ^ { \mathrm { i } { \frac { \theta } { 2 } } \tau ^ { 3 } } ) \cdot \vec { x } ; e ^ { \mathrm { i } { \frac { \theta } { 2 } } \tau ^ { 3 } } C e ^ { - \mathrm { i } { \frac { \theta } { 2 } } \tau ^ { 3 } } ) e ^ { \mathrm { i } { \frac { \theta } { 2 } } \tau ^ { 3 } }
\Delta E _ { \mathrm { u p } } = \frac { 1 } { T ^ { ( 0 ) } } \frac { 4 e ^ { S ^ { ( 0 ) } } \operatorname { t a n } ^ { 2 } \frac { 1 } { 2 } ( \phi _ { \alpha } + \phi _ { \gamma } ) + 4 e ^ { S ^ { ( 0 ) } } } { 4 e ^ { 2 S ^ { ( 0 ) } } \operatorname { t a n } ^ { 2 } \frac { 1 } { 2 } ( \phi _ { \alpha } + \phi _ { \gamma } ) - 1 } ,
q _ { i + 1 } = { \frac { 1 } { r _ { i } } } = \operatorname { e x p } - x _ { i } , \quad r _ { i + 1 } = r _ { i } [ q _ { i } r _ { i } - \ddot { \mathrm { l n } r _ { i } } ]
y ^ { 2 } = ( - 1 ) ^ { 2 S } ( x - x ^ { g } ) + u ^ { 2 }
\sum _ { n _ { 1 } = - \infty } ^ { \infty \prime } { \frac { e ^ { i \pi n _ { 1 } ( \alpha - 2 \sigma ) } } { n _ { 1 } \mathrm { S i n } ( \pi n _ { 1 } \alpha ) } } \quad .
\left( \frac { \sqrt { 2 } { \bf S } \cdot { \bf A } _ { 0 } } { k } \right) \phi _ { E \, \pm k \, m } = - \left( E + \frac { 1 } { 2 k ^ { 2 } } \right) ^ { \mathrm { ~ \frac { 1 } { 2 } ~ } } \phi _ { E \, \mp k \, m } .
\left. \frac { \partial } { \partial \zeta _ { N } } T ( \zeta _ { 1 } , \ldots , \zeta _ { N } ) \right| _ { \zeta _ { 1 } , \ldots , \zeta _ { N } = 1 } \propto H _ { \mathrm { \scriptsize ~ X Y Z } } + \mathrm { c o n s t . } .
\{ \bar { \mathcal V } ( x ) , \Phi ( x ^ { \prime } ) \} = ( \Phi ( x ) + ( \bar { h } - 1 ) \Phi ( x ^ { \prime } ) ) \, \delta ^ { \prime } ( x - x ^ { \prime } ) + \ldots .
\delta z ^ { M } e _ { M } ^ { \alpha q } \equiv \kappa ^ { \underline { \beta } } ( z ) v _ { \underline { \beta } } ^ { ~ \alpha q } .
M _ { N } ^ { 2 } ( a ) = m _ { 0 } ^ { 2 } + \frac { ( n _ { 1 } - a _ { 1 } ) ^ { 2 } } { L _ { 1 } ^ { 2 } } + \frac { ( n _ { 2 } - a _ { 2 } ) ^ { 2 } } { L _ { 2 } ^ { 2 } } ,
S _ { \mathrm { Y M } } = - { \frac { 1 } { 4 e ^ { 2 } } } \int _ { \cal M } d ^ { 2 } x \sqrt { g } g ^ { a c } g ^ { b d } \mathrm { T r } ( F _ { a b } F _ { c d } ) ,
\pi ( x ) = { \frac { \partial { \cal L } _ { \sigma } } { \partial \partial _ { + } \psi ( x ) } } = i \bar { \psi } \gamma ^ { + } = \left( \begin{array} { l l } { \psi _ { R } ^ { * } } & { \psi _ { L } ^ { * } } \\ \end{array} \right) \left( \begin{array} { l l } { i \sqrt { 1 + s } } & { 0 } \\ { 0 } & { i \sqrt { 1 - s } } \\ \end{array} \right) .
\sum _ { i , j , k , l = 1 } ^ { h } a _ { i j } ^ { p } A _ { i k } A _ { j l } \overline { a } _ { k l } ^ { q } = \sum _ { i , j , k , l , m , n = 1 } ^ { h } b _ { i j m } ^ { p } A _ { i k } A _ { j l } A _ { m n } \overline { b } _ { k l n } ^ { q } .
\left( ( p + k ) \: { \cal { B } } \: C _ { n - 1 } \: { \cal { B } } \cdots { \cal { B } } \: C _ { 1 } \: { \cal { B } } \: ( p - k ) \right) ( q , y ) \; = \; ( p + k ) ( q ) \: F _ { n } ( q ) \; \; \; .
\partial _ { \alpha } Y ^ { 2 5 } = \epsilon _ { \alpha \beta } \partial ^ { \beta } X ^ { 2 5 } .
y = \left( \begin{array} { c c c c } { 0 } & { q _ { 1 } } & { 0 } & { 0 } \\ { 0 } & { 0 } & { q _ { 2 } } & { 0 } \\ { 0 } & { 0 } & { 0 } & { q _ { 3 } } \\ { q _ { 4 } } & { 0 } & { 0 } & { 0 } \\ \end{array} \right)
\frac { 1 } { 2 } \epsilon _ { \kappa \lambda \sigma \alpha } \partial ^ { \kappa } \omega ^ { \sigma \lambda } \equiv D _ { \alpha } = \left( \partial _ { \alpha } h _ { \beta } ^ { \beta } - \partial _ { \beta } h _ { \alpha } ^ { \beta } \right) \ .
Q _ { 0 } ^ { R , L } = \frac { 1 } { 2 \pi i } \int _ { C ^ { R } , C ^ { L } } d z j ( z ) _ { 0 } \ , R ^ { R , L } = \frac { 1 } { 2 \pi i } \int _ { C ^ { R } , C ^ { L } } d z r ( z ) \ .
K = i \int _ { 0 } ^ { \infty } d \lambda ^ { \prime } \; s e ^ { i \lambda ^ { \prime } ( \pi ^ { \prime \prime } - \pi ^ { \prime } ) } e ^ { - i [ s \lambda ^ { \prime } ( \gamma ^ { \mu } P _ { \mu } ^ { \prime } - m ) + { \frac { i } { s } } ( { \bar { \eta } } ^ { \prime \prime } - { \bar { \eta } } ^ { \prime } ) ( \eta ^ { \prime \prime } - \eta ^ { \prime } ) ] } \delta ^ { 4 } ( P ^ { \prime \prime } - P ^ { \prime } ) \, .
\epsilon ^ { i j } \nabla _ { i } ^ { ( p , k ) } { \cal A } _ { ( p , k ) j } = - \sum _ { ( p ^ { \prime } , k ^ { \prime } ) \neq ( p , k ) } \beta _ { p p ^ { \prime } } \delta ^ { 2 } ( { \bf r } _ { k } ^ { ( p ) } - { \bf r } _ { k ^ { \prime } } ^ { ( p ^ { \prime } ) } ) \, .
\delta \alpha _ { 3 } + \gamma \alpha _ { 2 } = \partial _ { \mu } \left( - f _ { a b c } \eta ^ { * \mu } \eta ^ { a } \eta ^ { b } \eta ^ { c } \right) .
\psi _ { l } \left( z \right) \psi _ { l } ^ { \dagger } \left( w \right) \sim \left( z - w \right) ^ { - 2 \Delta _ { l } } \left[ 1 + \frac { 2 \Delta _ { l } } { c _ { p } } \left( z - w \right) ^ { 2 } T _ { p } \left( w \right) + O \left( z - w \right) ^ { 3 } \right]
V ( T ) = \sqrt { 2 } \tau _ { p } e ^ { - { \frac { \kappa T ^ { 2 } } { 2 \alpha ^ { \prime } } } } \ ,
+ q ( { \tilde { d } } - 1 ) a _ { 1 } b _ { 1 } + r ( { \tilde { d } } - 1 ) b _ { 1 } f _ { 1 } + r q f _ { 1 } a _ { 1 } + \frac { \phi _ { 1 } ^ { 2 } } { 4 } - \frac { h ^ { 2 } } { 4 } = 0 ;
r = \frac { \alpha } { \beta } \vert \operatorname { s i n } \beta \left( \sigma - \sigma _ { 0 } \right) \vert
\begin{array} { l l } { f ( \psi ) = 1 + \frac { 1 } { R ^ { 2 } } \sum _ { r = 1 } ^ { N } m _ { r } ^ { 2 } ( \psi _ { r } ) ^ { 2 } + O ( \psi ^ { 3 } ) } & { \mathrm { w i t h ~ \frac { m _ { r } ^ { 2 } } { R ^ { 2 } } ~ \equiv ~ \frac { 1 } { 2 } ~ \partial _ { r } ~ \partial _ { r } ~ f ~ | _ { \psi ~ = 0 } ~ } . } \\ \end{array}
g _ { z \overline { { z } } } = \tau _ { 2 } \eta ^ { 2 } \overline { { \eta } } ^ { 2 } \prod _ { i } ( z - z _ { i } ) ^ { - 1 / 1 2 } \prod _ { i } ( \overline { { z } } - \overline { { z } } _ { i } ) ^ { - 1 / 1 2 } ~ .
\Delta ( A ) = A _ { 1 } K A _ { 2 } , \hspace { 7 m m } \Delta ( B ) = B _ { 1 } E B _ { 2 } , \hspace { 7 m m } \Delta ( C ) = C _ { 1 } Q C _ { 2 }
\begin{array} { l l } { \left| \ \! \pm ; \pm ; 2 w ; 2 w ; { \mathbf 2 } , \ \ { \frac { 1 } { 2 } } ; { \mathbf 1 } , 0 , 0 \ \! \right\rangle } & { = { \frac { 1 } { \sqrt { 2 } } } \left( \left| \ \! \underline { { 1 _ { 2 } } } , \underline { { 0 _ { } } } ; \underline { { 0 _ { } } } , \underline { { 0 _ { } } } \ \! \right\rangle \pm \left| \ \! \underline { { 0 _ { } } } , \underline { { 1 _ { 2 } } } ; \underline { { 0 _ { } } } , \underline { { 0 _ { } } } \ \! \right\rangle \right) } \\ { \left| \ \! \pm ; \pm ; 2 w ; 2 w ; { \mathbf 2 } , - { \frac { 1 } { 2 } } ; { \mathbf 1 } , 0 , 0 \ \! \right\rangle } & { = { \frac { 1 } { \sqrt { 2 } } } \left( \left| \ \! \underline { { 1 _ { 1 } } } , \underline { { 0 _ { } } } ; \underline { { 0 _ { } } } , \underline { { 0 _ { } } } \ \! \right\rangle \pm \left| \ \! \underline { { 0 _ { } } } , \underline { { 1 _ { 1 } } } ; \underline { { 0 _ { } } } , \underline { { 0 _ { } } } \ \! \right\rangle \right) } \\ \end{array} \hspace { 1 . 0 c m }
\tilde { { \cal K } } = \frac { ( D - 2 ) { \cal K } } { D - 2 - { \cal K } } \ ,
S ( x ) = { \cal S } _ { 0 } ( x ) \operatorname { e x p } \left[ - i e ^ { 2 } \beta ( x ) \right] \; ,
\begin{array} { r l } { d s ^ { 2 } = \alpha ^ { \prime } g _ { 6 } \sqrt { N _ { 1 } N _ { 5 } } [ } & { u ^ { 2 } ( d x _ { 0 } ^ { 2 } + d x _ { 1 } ^ { 2 } ) + { \frac { d u ^ { 2 } } { u ^ { 2 } } } + d \Omega _ { 3 } ^ { 2 } } \\ { \nonumber + } & { \beta ( N _ { 1 } , N _ { 5 } ) ( d x _ { 2 } ^ { 2 } + . . . + d x _ { 5 } ^ { 2 } ) ] , } \\ \end{array}
\widetilde { \Gamma } _ { k p } \ = \ \frac { 1 } { \mu _ { k } { - } \bar { \mu } _ { p } } \, T _ { k } ^ { \dagger } \, T _ { p } \quad , \qquad \textrm { i . e . } \qquad \sum _ { k = 1 } ^ { m } \, \Gamma ^ { \ell k } \, \widetilde { \Gamma } _ { k p } \ = \ \delta _ { \ p } ^ { \ell } \quad .
\psi \longrightarrow e ^ { i \theta \gamma _ { 5 } } \psi \, .
\zeta _ { \pm } ^ { \prime } ( 0 | K _ { 0 } ) = \frac { 1 } { 2 } \zeta ^ { \prime } ( 0 | L ) \pm \frac { i \pi } { 2 } \eta ( 0 | K _ { 0 } ) \, .

\widehat { X } = X _ { + } \otimes \sigma _ { 3 } + X _ { - } \otimes i \sigma _ { 2 } ,
F \equiv \frac { 1 } { 4 } F ^ { a b } F _ { a b } =
D _ { 1 } \phi + i \sigma _ { 1 } D _ { 2 } \phi = 0 \, , \hspace { 5 m m } D _ { 1 } \chi + i \sigma _ { 2 } D _ { 2 } \chi = 0 \, ,
z _ { 1 } ^ { P _ { 1 } + 2 } + \dots + z _ { P _ { s } } ^ { P _ { s } + 2 } = 0 ~ ,
x \times y - y \times x - F ( x , y ) \; , \quad x , y \in E \; .
g \, e ^ { { \vec { \xi } } . { \vec { X } } } = e ^ { { \vec { \xi } } \, ^ { \prime } . { \vec { X } } } e ^ { { \vec { u } } \, ^ { \prime } . { \vec { T } } } ,
T _ { \bar { \imath } \bar { \jmath } k } = T _ { i \bar { \jmath } \bar { k } } = T _ { \bar { \imath } j \bar { k } } = m \epsilon _ { i j k } \ .
\phi ^ { ( 1 ) } ( a , b , c | u , v , w + 1 / 4 ) = C ( u , v , w ) \, f ( u , v , w ) \, W ( a , b | A _ { - 1 } ) \, \overline { { W } } ( b , c | B _ { - 1 } ) ,
{ \cal D } _ { b } \psi ^ { a b } = 0 , \quad \quad \psi ^ { a b } = \psi ^ { b a } ,
\{ \phi , \psi \} = \{ \phi , \psi \} _ { \mathrm { K i r } } + \partial ^ { i } \phi L _ { i } \psi - \partial ^ { i } \psi L _ { i } \phi , ~ ~ ~ \phi , \psi \in \mathrm { F u n } ( T ^ { * } G ) .
[ \partial _ { x } - \sum _ { s = 1 } ^ { m _ { 1 } } A ^ { - s } , \partial _ { y } - ( \rho h ) - \sum _ { s = 1 } ^ { m _ { 2 } } A ^ { + s } ] = 0
\vec { E } ( \tau ) = \left[ { \frac { d ^ { 2 } } { d \tau ^ { 2 } } } - ( 2 / \tau ) { \frac { d } { d \tau } } \right] \vec { G } ( \tau )
A ( b , \theta ) = A ( b , \theta ) ^ { \geq 0 } + A ( b , \theta ) ^ { < 0 } .
\nabla _ { \nu } \nabla ^ { \nu } A _ { \mu } = - I _ { A \mu } = 4 e \Phi ^ { 2 } ( \chi _ { A , \mu } + e A _ { \mu } ) .
( 1 + \theta + \omega + \theta \omega ) ( 1 + \alpha \Omega ^ { \prime } )
\operatorname { l o g } \, Z \left( \beta \right) = \frac { f \left( 0 \right) } { 2 } - I \left( 0 \right) - 2 \sum _ { n = 1 } ^ { \infty } I \left( n \right) ,
H ( \omega , 0 ) = \frac { \phi _ { 1 } f _ { 2 } \partial _ { r } ( \phi _ { 2 } ^ { ( - ) } / A _ { - } ) - ( \phi _ { 2 } ^ { ( - ) } / A _ { - } ) f _ { 1 } \partial _ { r } \phi _ { 1 } } { \phi _ { 1 } f _ { 2 } \partial _ { r } ( \phi _ { 2 } ^ { ( + ) } / A _ { + } ) - ( \phi _ { 2 } ^ { ( + ) } / A _ { + } ) f _ { 1 } \partial _ { r } \phi _ { 1 } } \ .
\frac { d \phi ^ { i } } { d t } = - g ^ { i j } \frac { \partial W } { \partial \phi ^ { j } } \ ,
L _ { 1 } = \mathcal { \vartheta } _ { 3 } ^ { 4 } ( 0 | \theta ) - \mathcal { \vartheta }
p _ { \mu } ^ { i } \left( \kappa \right) = P _ { \mu } \, \, v ^ { i } \left( \kappa \right) + \varepsilon _ { a b } W _ { \mu } ^ { a } \, \lambda ^ { i b } \left( \kappa \right)
I _ { 1 } ( y , \mp r ) = \int _ { 0 } ^ { \infty } \frac { d x } { \sqrt { x ^ { 2 } + y ^ { 2 } } } \frac { 1 } { \operatorname { e x p } ( \sqrt { x ^ { 2 } + y ^ { 2 } } \mp r ) + 1 } ,
r \rightarrow r _ { 0 } , f \rightarrow 1 + s i n h ^ { 2 } \gamma = c o s h ^ { 2 } \gamma \equiv \lambda ^ { 3 }
\gamma _ { 1 } = \left\langle \sum _ { X _ { i } \in ( { \it G } _ { 1 } ) _ { X _ { i ^ { \prime } } ^ { \prime } } } \sum _ { \stackrel { i _ { \alpha _ { 1 } } < j _ { \alpha _ { 1 } } } { \left\{ i _ { \alpha _ { 1 } } , j _ { \alpha _ { 1 } } \right\} \in J _ { X _ { i } } } } t _ { i _ { \alpha _ { 1 } } } t _ { j _ { \alpha _ { 1 } } } { \bf 1 } _ { ( w ( t _ { i _ { \alpha _ { 1 } } } ) = w ( t _ { j _ { \alpha _ { 1 } } } ) ) } \chi \left( \left( i _ { \alpha _ { 1 } } , j _ { \alpha _ { 1 } } \right) \in i _ { \alpha _ { 1 } } ^ { \prime } \right) \right\rangle _ { w ^ { \prime } } ^ { l . c . } \; \mathrm { ~ a n d }
q _ { j } ( x _ { 0 } ) = A _ { j + 1 } , \quad q _ { j } ^ { \prime } ( x _ { 0 } ) = B _ { j + 1 } , \quad j = 0 , 1 , \ldots , n - 1 .
{ \frac { 1 } { 2 \pi } } \sum _ { l = 1 / 2 } ^ { \infty } 2 l e ^ { - { \frac { l ^ { 2 } } { \rho ^ { 2 } } } t } = { \frac { \rho ^ { 2 } } { 4 \pi t } } + { \frac { \rho ^ { 2 } } { ( 4 \pi t ) ^ { \frac { 3 } { 2 } } } } \mathrm { P } \int _ { - \infty } ^ { \infty } d x ( { \frac { x } { 2 \rho } } \mathrm { c o s e c } { \frac { x } { 2 \rho } } - 1 ) e ^ { - { \frac { x ^ { 2 } } { 4 t } } } ,
\partial ^ { 2 } \bar { \Psi } ^ { \alpha } - i g \; \partial _ { x + i y } \Phi \; \partial _ { 0 - z } \bar { \Psi } ^ { \alpha } + i g \; \partial _ { 0 - z } \Phi \; \partial _ { x + i y } \bar { \Psi } ^ { \alpha } = 0 \, ,
Q = \frac { 1 } { 2 \pi } 2 \pi \theta \sum _ { k = 0 } ^ { \infty } \left\langle k \right| \left( - \theta ^ { - 1 } n \left| 0 \right\rangle \left\langle 0 \right| \right) \left| k \right\rangle = - n ,

[ \: x ^ { \lambda } , \eta _ { \mu \nu } \, \dot { x } ^ { \mu } \, \dot { x } ^ { \nu } \: ] = 0 .
m _ { h } = \frac { g _ { s m } ^ { 2 } | \psi ( 0 ) | ^ { 2 } } { M _ { G } ^ { 2 } } = \frac { M _ { G } } { \sqrt { \pi } c _ { f } \alpha _ { s } \sqrt { N _ { s d } } }
{ \bf \Phi } _ { 1 } ^ { ( n _ { \rho } , m ) } = { \bf \Phi } _ { 1 } ^ { ( n _ { \rho } , m ) } ( \rho , k ) = \left( \begin{array} { c } { \phi _ { 1 } ^ { ( n _ { \rho } , m ) } ( \rho , k ) } \\ { \phi _ { 2 } ^ { ( n _ { \rho } , m ) } ( \rho , k ) } \\ \end{array} \right)
\frac { d \lambda ^ { \gamma } } { d u } = { \tilde { f } } _ { \gamma } ( u ) \lambda _ { 1 }
u _ { 2 } = \xi _ { 0 } + i \xi _ { 3 } = \rho ~ \frac { e ^ { i \psi } } { \sqrt { 1 + z \bar { z } } }
\tilde { h } = \frac { 1 } { 1 + ( a u ) ^ { 7 - p } } , \ \ \ a ^ { 7 - p } = \tilde { b } ^ { 2 } / R ^ { 7 - p } , \ \ \ R ^ { 7 - p } = \frac { 1 } { 2 } ( 2 \pi ) ^ { 6 - p } \pi ^ { - ( 7 - p ) / 2 } \Gamma [ ( 7 - p ) / 2 ] \hat { g } _ { s } N .
\phi ( \Gamma ) ( m ; p _ { i } ) = \int _ { 0 } ^ { \infty } \frac { F _ { \Gamma } ( r ; m ; p _ { i } ) } { r } d r ,
k ^ { 2 } = \frac { b ^ { 2 } } { 4 } + \frac { \pi ^ { 2 } } { y _ { c } ^ { 2 } } n _ { k } ^ { 2 } ~ ,
\delta \hat { \vec { B } } \; = \; d \hat { \vec { \cal N } } \; \rightarrow \; \delta \hat { \vec { B } } \, = \, { \cal D } \hat { \vec { \cal N } } \, = \, d \hat { \vec { \cal N } } \, + \, \hat { \cal E } \wedge \hat { \vec { \cal N } } \; ,
\partial _ { + } g _ { L } g _ { L } ^ { - 1 } = \left( \begin{array} { c c } { v _ { L } \partial _ { + } u _ { L } + b _ { L } \partial _ { + } a _ { L } } & { u _ { L } \partial _ { + } a _ { L } - a _ { L } \partial _ { + } u _ { L } } \\ { b _ { L } \partial _ { + } v _ { L } - v _ { L } \partial _ { + } b _ { L } } & { u _ { L } \partial _ { + } v _ { L } + a _ { L } \partial _ { + } b _ { L } } \\ \end{array} \right) ,
\langle \Theta \rangle = c ~ R ^ { ( 2 ) } + \beta ^ { i } \langle V _ { i } \rangle
g _ { 2 } \equiv r + \frac { \lambda v ^ { 2 } } { 2 } + \frac { w v ^ { 4 } } { 2 4 } ,
\begin{array} { r c l } { \psi _ { \uparrow } ^ { ( + ) } } & { = } & { \sqrt { \rho } e ^ { \gamma _ { 5 } \pi / 4 } e ^ { - \gamma _ { 2 1 } M c t / \hbar } \, ; } \\ { \psi _ { \downarrow } ^ { ( + ) } } & { = } & { \sqrt { \rho } e ^ { \gamma _ { 5 } \pi / 4 } \gamma _ { 3 1 } e ^ { - \gamma _ { 2 1 } M c t / \hbar } \, ; } \\ { \psi _ { \uparrow } ^ { ( - ) } } & { = } & { \sqrt { \rho } e ^ { \gamma _ { 5 } 3 \pi / 4 } e ^ { - \gamma _ { 2 1 } M c t / \hbar } ) \, ; } \\ { \psi _ { \downarrow } ^ { ( - ) } } & { = } & { \sqrt { \rho } e ^ { \gamma _ { 5 } 3 \pi / 4 } ( \gamma _ { 1 2 } e ^ { - \gamma _ { 2 1 } M c t / \hbar } ) \, . } \\ \end{array}
H = \frac { 1 } { 2 } \int ( { \cal H } + \bar { \cal H } ) d x ,
G _ { \mathrm { c o m p a c t } } = S O \left( 2 \right) _ { E } \oplus S O \left( 3 \right) _ { S } \oplus S O \left( 2 \right) _ { R } \subset G _ { \mathrm { e v e n } } .
\partial _ { \mu } ( \sqrt { g } R ^ { \mu } ) = - \frac { 1 } { 3 g ^ { 3 } } \widetilde { \beta } ( g ) ( F \widetilde { F } ) - \frac { \tilde { b } ( g ) } { 4 8 \pi ^ { 2 } } ( B \widetilde { B } ) + \frac { \tilde { c } ( g ) - a ( g ) } { 2 4 \pi ^ { 2 } } R \widetilde { R } + \frac { 5 a ( g ) - 3 \tilde { c } ( g ) } { 9 \pi ^ { 2 } } ( V \widetilde { V } )
\tilde { \cal E } _ { q } [ \phi ( x ) ] \sim \frac { 1 } { 2 } \left( \sum _ { \tilde { E } _ { j } } | \tilde { E } _ { j } | - \sum _ { \tilde { E } _ { j } ^ { 0 } } | \tilde { E } _ { j } ^ { 0 } | \right) + \tilde { \cal E } _ { c t } [ \phi ( x ) ]
\zeta = \kappa \operatorname { c o s } \theta , \quad k = \kappa \operatorname { s i n } \theta ,
S = t r H ^ { 4 } - 2 { \mu } ^ { 2 } t r H ^ { 2 } + n { \mu } ^ { 4 } .
S ^ { - 1 } ( p ) = i \gamma \cdot p A ( p ^ { 2 } ) + B ( p ^ { 2 } ) ,
x ^ { \mu } = R ^ { \mu } ( \tau , \sigma ) + \rho ^ { \alpha } n _ { ( \alpha ) } ^ { \mu } ( \tau , \sigma ) \; \; ,
T ^ { \prime i j } = T ^ { i j } + S ^ { i k j } { } _ { ; k } ,
t ^ { 2 } - v ^ { 2 } t + \Lambda ^ { 2 } v ^ { 2 } = 0 .
L = g ^ { - 1 } C g , \quad L \in G ^ { * } , \quad g \in G ,
c _ { n } \equiv t r _ { q } K ^ { n } \quad , \quad K c _ { n } = c _ { n } K \quad .
{ \bf e ^ { a } } = 2 ( 2 \alpha ^ { \prime } ) ^ { - 1 / 2 } { \bf w ^ { a } } \ ,
\psi _ { c o v a r } ( p , g ) = F ( L ^ { - 1 } ( p ) g ) \psi ( p )
H = \frac { 1 } { 4 \pi } \left( P ^ { 2 } + 4 \omega ^ { 2 } \, e ^ { 2 \gamma Q } \right)
{ \tilde { \epsilon } } = \int _ { r _ { h } } ^ { r _ { h } + \epsilon } d r \, \Delta ^ { - \frac { 1 } { 2 } } = 2 \epsilon ^ { \frac { 1 } { 2 } } \left( \Delta _ { h } ^ { \prime } \right) ^ { - \frac { 1 } { 2 } } + O ( \epsilon ^ { \frac { 3 } { 2 } } ) ,
0 = \frac { \partial \left\langle V , V \right\rangle } { \partial \lambda _ { m } }
S ( \mathrm { i } \pi - \theta ) = - \mathrm { i } t a n h ( \frac { \theta } { 2 } ) \; \; \; \bar { S } ( \theta )
( - ) ^ { { \bar { \alpha } } + { \bar { \beta } } + \bar { \alpha } \bar { \beta } } \frac { { \bar { \vartheta } } [ _ { \bar { \beta } } ^ { \bar { \alpha } } ] } { { \bar { \eta } } } \rightarrow \frac { { \bar { \vartheta } [ _ { \bar { \beta } } ^ { \bar { \alpha } } ] } ^ { 1 3 } } { { \bar { \eta } } ^ { 1 3 } } .
R _ { 1 } ^ { \mathbf { I I B } } = \frac { R } { \sqrt { 2 } N _ { 2 } } \quad \textrm { a n d } \quad \omega _ { n } = \sqrt { \mu ^ { 2 } + \frac { n ^ { 2 } } { l _ { s } ^ { 4 } ( p ^ { + } ) ^ { 2 } } } .
\begin{array} { c l } { F ( x , e ) } & { = F ^ { r } ( x , r ) } \\ \end{array}
\delta _ { i = j } = \delta _ { + - } = \delta _ { - + } = 1 , \quad \delta _ { i \neq j } = \delta _ { i + } = \delta _ { i - } = \delta _ { + + } = \delta _ { -- } = 0 .
S _ { \mathrm { b o u n d a r y } } ^ { \mathrm { q u i n t i c } } = \int d x ^ { 0 } \frac { \theta } { 4 \pi r } \sum _ { i } \left( \overline { { \psi } } _ { + i } \psi _ { + i } + \overline { { \psi } } _ { - i } \psi _ { - i } \right)
\eta = \int ^ { X ^ { 0 } } \frac { d X ^ { 0 } } { R ( X ^ { 0 } ) } ~ ,
E ^ { f K K } = - { \cal { F } } \left[ \frac { \sqrt { { \bf p } ^ { 2 } + a ^ { 2 } k ^ { 2 } x ^ { 2 } } } { 2 } \operatorname { t a n h } \left( \frac { \beta } { 2 } \sqrt { { \bf p } ^ { 2 } + a ^ { 2 } k ^ { 2 } x ^ { 2 } } \right) \right] .
W ^ { \mathrm { \scriptsize ~ r e n } } = - \frac 1 2 \zeta ^ { D } ( 0 ) ^ { \prime } - \operatorname { l o g } ( \mu ) \zeta ^ { D } ( 0 ) \, .
\nu _ { - a b } \equiv n _ { - a b } \circ \varphi _ { + } , \qquad \eta _ { - a b } \equiv h _ { - a b } \circ \varphi _ { + } .
\delta B = { \frac { 1 } { 2 } } \delta \bar { \theta } \Gamma _ { M N } \theta ( d X ^ { M } d X ^ { N } + \bar { \theta } \Gamma ^ { M } d \theta d X ^ { N } + { \frac { 1 } { 3 } } \bar { \theta } \Gamma ^ { M } d \theta \bar { \theta } \Gamma ^ { N } d \theta )
T \equiv \frac { 1 } { \sqrt { 1 - y _ { 3 } ^ { 2 } } } \left( y _ { 2 } { \bf 1 } _ { 2 } + i { \bf y } \cdot \mathrm { \boldmath ~ \sigma ~ } \right) \; \; .
M _ { i j } \partial ^ { ( j } A ^ { i ) } = 0 , M _ { i j } \partial ^ { i } \partial ^ { j } A _ { k } = 0 .
N = \sum _ { n > 0 } \alpha _ { - n } ^ { i } ( E ) G _ { i j } \alpha _ { n } ^ { j } ( E ) , \, \tilde { N } = \sum _ { n > 0 } \tilde { \alpha } _ { - n } ^ { i } ( E ) G _ { i j } \tilde { \alpha } _ { n } ^ { j } ( E )
c ( i , j , k ) = ( - ) ^ { B ( \beta _ { i } , \beta _ { j } ) + B ( \beta _ { i } , \beta _ { k } ) + B ( \beta _ { j } , \beta _ { k } ) + B ( \beta _ { 0 } , \omega ) + 1 }
\omega = \phi ^ { \alpha } \cdot \sum _ { i _ { 1 } < \cdots < i _ { k } } a _ { i _ { 1 } \cdots i _ { k } } ( v ) d v ^ { i _ { 1 } } \wedge \cdots \wedge d v ^ { i _ { n } }
e _ { \mu } ^ { f } = \left( { \frac { 1 } { 2 } } \left( e _ { \mu } ^ { [ + 2 ] } + e _ { \mu } ^ { [ - 2 ] } \right) , { \frac { 1 } { 2 } } \left( e _ { \mu } ^ { [ + 2 ] } - e _ { \mu } ^ { [ - 2 ] } \right) \right)
f ( \mu ) = \pi - f ( \pi - \mu ) .
\Sigma _ { \alpha \beta } = V ^ { - 1 } ~ \sigma _ { \alpha \beta } V
{ \cal H } _ { T } = { \cal H } _ { 1 } + \Sigma _ { i = 1 } ^ { 3 } c ^ { i } \, \phi _ { i } , \; \; \; \; \; { \cal H } _ { 1 } = - \frac { 1 } { 2 } r ^ { 2 } q _ { x x } - p \, q _ { x x }
{ \frac { \partial ^ { 2 } \varphi } { \partial \tau ^ { 2 } } } + { \frac { \partial ^ { 2 } \varphi } { \partial r ^ { 2 } } } + { \frac { 2 } { r } } { \frac { \partial \varphi } { \partial r } } = { \frac { \partial U ( \varphi , T ) } { \partial \varphi } } ,
V _ { \mu } \rightarrow \frac { i } { g } ~ \theta ( \mathrm { \boldmath ~ r ~ } ) ~ \partial _ { \mu } ~ \theta ( \mathrm { \boldmath ~ r ~ } ) + \theta ( \mathrm { \boldmath ~ r ~ } ) ~ V _ { \mu } ~ \theta ^ { \dag } ( \mathrm { \boldmath ~ r ~ } )
[ a _ { 1 } , a _ { 2 } , \cdots ~ , a _ { r } ] \rightarrow [ a _ { 1 } , a _ { 2 } , \cdots ~ , ( a _ { j } + 1 ) , 1 , ( a _ { j } + 1 ) , \cdots ~ , a _ { r } ]
[ Q _ { i } , \; Q _ { j } ] _ { + } = \sum _ { n = 0 } ^ { \infty } \; A _ { M _ { 1 } } ^ { \dagger } \ldots A _ { M _ { n } } ^ { \dagger } ( A ^ { \dagger } [ \Gamma _ { i } , \; \Gamma _ { j } ] _ { + } A ) A _ { M _ { n } } \ldots A _ { M _ { 1 } } .
\beta ( g ) = - g \frac { b _ { 0 } + b _ { 1 } g + b _ { 2 } g ^ { 2 } } { 1 + a _ { 1 } g + a _ { 2 } g ^ { 2 } }
A ^ { \mathrm { T } } = - A , \qquad M \eta A + A \eta M = 0
J _ { \mu } = - { \frac { i e } { 2 } } \bigg [ \Phi ^ { \ast } D _ { \mu } \Phi - \Phi \big ( D _ { \mu } \Phi \big ) ^ { \ast } \bigg ] \, .
\gamma _ { i } ( \beta \sqrt { - \triangle } ) = \int _ { 0 } ^ { \infty } \! \! { \mathrm d } t \, \, g _ { i } ( t ) \left[ \frac { 2 \pi } { \beta { \sqrt { - \triangle } } \, t } \, \frac { 1 } { { \mathrm s h } ( 2 \pi t / ( \beta { \sqrt { - \triangle } } ) ) } - \frac { 1 } { t ^ { 2 } } \right] ,
\xi _ { 1 } ^ { m } ( u ) = - \frac { \partial \xi _ { 2 } ^ { 1 } ( u ) } { \partial x ^ { m } } + \sum _ { i = 1 } ^ { n } x ^ { i } ( u ) \frac { \partial \xi _ { 1 } ^ { i } ( u ) } { \partial x ^ { m } } .
\sigma _ { \mathrm { a b s } } ^ { ( 1 , 0 ) } = \frac { { \cal A } _ { H } } { 2 } .
\psi ( x ) \mapsto \psi ^ { \prime } ( x ) \equiv e ^ { i \theta ( x ) } \psi ( x ) \ , \hspace { 2 e x } \overline { \psi } ( x ) \mapsto \overline { { \psi ^ { \prime } } } ( x ) \equiv \overline { \psi } ( x ) e ^ { - i \theta ( x ) } \ .
I ( x ^ { - } , x ^ { + } ) = I ^ { ( 1 ) } ( x ^ { - } , x ^ { + } ) + I ^ { ( 2 ) } ( x ^ { - } , x ^ { + } )
V ( \phi ) = { \frac { \lambda } { 2 } } \left( \phi ^ { a } \phi ^ { a } - v ^ { 2 } \right) ^ { 2 } \ .
{ \widetilde \Phi } ^ { A ^ { \prime } } ( \gamma ) \equiv \gamma ^ { A ^ { \prime } C ^ { \prime } C } \; { _ { e } n _ { C C ^ { \prime } } } = 0 \; .
\phi = \pm \imath \operatorname { l o g } \frac { 2 } { \beta } \left( 1 + \sqrt { 1 - \frac { \beta ^ { 2 } } { 4 } } \right) ,
N \approx L ^ { 2 } \left[ 1 + \frac { \gamma \beta L } { 2 ( 1 - \gamma L ) ^ { 2 } } \right] .
- \left( \frac { \partial } { \partial \xi } , \frac { \partial } { \partial \eta } \right) \Gamma ( \xi , \eta ) \left( \begin{array} { c } { \frac { \partial } { \partial \xi } } \\ { \frac { \partial } { \partial \eta } } \\ \end{array} \right)
i \frac { \partial \psi } { \partial t } = - i \mathrm { \boldmath ~ \alpha ~ } \cdot \nabla \psi + \beta m \psi - q \mathrm { \boldmath ~ \alpha ~ } \cdot { \bf A } \psi + q A _ { 0 } \psi
D G _ { R } = 0 , ~ ~ ~ ~ ~ ~ G _ { L } \overleftarrow { D } = 0
\left\{ - \sum _ { i = 1 } ^ { 3 } \frac { \partial ^ { 2 } } { \partial x _ { i } ^ { 2 } } + \sum _ { \stackrel { i , j = 1 } { i \neq j } } ^ { 3 } \left( { \frac { \omega ^ { 2 } } { 8 } } \left( x _ { i } - x _ { j } \right) ^ { 2 } + { \frac { 2 \alpha } { \left( x _ { i } - x _ { j } \right) ^ { 2 } } } + \Omega \left( \frac { x _ { i } - x _ { j } } { \sqrt { 2 } } \right) \right) \right\} \psi = E \psi ~ ,
\Pi _ { b } ^ { \mu \nu } ( p ) = \int \frac { d ^ { 4 } k } { ( 2 \pi ) ^ { 4 } } t r \left\{ \gamma ^ { \mu } S ( l ) \gamma ^ { \nu } G _ { b } ( l + p ) + \gamma ^ { \mu } G _ { b } ( l + p ) \gamma ^ { \nu } S ( l ) \right\} .
{ \frac { \tau _ { j k } } { c _ { 2 } \tau \sp 2 _ { j m } } } =
M ( E ) = \left( \begin{array} { l l } { g - B g ^ { - 1 } B } & { B g ^ { - 1 } } \\ { - g ^ { - 1 } B } & { g ^ { - 1 } } \\ \end{array} \right) ,
I _ { e f f } ^ { \mathrm { C . S } } = \frac { m } { | m | } \theta ( m ^ { 2 } - \mu ^ { 2 } ) \pi W [ A ] ,
\delta S _ { W Z N W } = \frac { k } { 2 \pi } \int _ { \partial M } ( \partial _ { \tau } ( U ^ { - 1 } \partial _ { \phi } U ) \delta U .
\frac { 1 } { 2 } \rho \xi ^ { - 2 } = Y ^ { 2 } \pm U \mp V - X ^ { 2 } - Z \geq 0 .
{ F } = { \sum _ { m = 1 / 2 } ^ { \infty } \psi _ { - m } \cdot \psi _ { m } } - 1 ~ ~ , ~ ~ { G } = { - \sum _ { m = 1 / 2 } ^ { \infty } \left( \gamma _ { - m } \beta _ { m } + \beta _ { - m } \gamma _ { m } \right) } ~ ~ .
e _ { 1 } = \left| \begin{array} { c } { \overrightarrow { e _ { 1 } } } \\ { 0 } \\ \end{array} \right| , \quad e _ { 2 } = \left| \begin{array} { c } { \overrightarrow { e _ { 2 } } } \\ { 0 } \\ \end{array} \right| , \quad e _ { 3 } = \left| \begin{array} { c } { \overrightarrow { e _ { 3 } } } \\ { 0 } \\ \end{array} \right| , \quad e _ { 4 } = \left| \begin{array} { c } { 0 } \\ { 1 } \\ \end{array} \right| .
\frac { d ( T _ { \eta } ^ { \eta } c ( \eta ) } { d \eta } = \frac { \dot { c } ( \eta ) } { 2 } T _ { \beta } ^ { \beta } .
R _ { \pm } ( d T ) _ { 1 } ( d T ) _ { 2 } = - ( d T ) _ { 2 } ( d T ) _ { 1 } R _ { \mp } .
I _ { m n } = \int d \bar { z } d z \ e _ { q } ^ { - \bar { z } z } z ^ { n } \bar { z } ^ { m } = \delta _ { m n } [ n ] ! \ ,
( \gamma / \nu d _ { H } ) _ { e f f } \equiv \operatorname { l n } \left( { \frac { \chi _ { 2 N } } { \chi _ { N } } } \right) / \operatorname { l n } 2 .
F = S T U + f ( T , U ) + f ^ { n o n - p e r t } .
[ A ( m ) , \bar { A } ( m ^ { \prime } ) ] = \delta _ { m m ^ { \prime } } \ \ \ \mathrm { a n d \ \ o t h e r s } = 0 .
\Lambda ^ { 0 } = ( \partial _ { i } F ^ { i o } - \rho ) \dot { { \bar { c } } } - b \nabla ^ { 2 } { \bar { c } } ,
\Psi _ { F Q H E } ^ { m } ( Z ) = \prod _ { i < j } ^ { N } ( Z _ { i } - Z _ { j } ) ^ { m } \operatorname { e x p } \{ - \sum _ { k = 1 } ^ { N } | Z _ { k } | ^ { 2 } \} ,
\mathbf { \gamma } _ { b } ^ { a } = - ( \alpha _ { b c } ^ { a } \mathbf { \omega } ^ { c } + \beta _ { b } ^ { a c } \mathbf { \omega } _ { c } )
\xi ^ { ( i ) } ( x _ { + } ) \equiv \langle \lambda ^ { i } | T _ { L } , { } ~ ~ ~ ~ \bar { \xi } ^ { ( i ) } ( x _ { - } ) \equiv T _ { R } ^ { - 1 } | \lambda ^ { i } \rangle
\operatorname* { l i m } _ { q \to 1 } { \frac { 1 } { 2 } } \Big [ Z ^ { a | a } ( L ; q ) - Z ^ { a } ( L ; q ) \Big ] = \left\{ \begin{array} { l l } { \frac { 1 } { 2 } \left( 2 ^ { L - 1 } - 2 ^ { \lfloor \frac { L } { 2 } \rfloor } \right) , } & { a = 1 o r + } \\ { \frac { 1 } { 2 } \left( 2 ^ { L } - 2 ^ { \lfloor \frac { L + 1 } { 2 } \rfloor } \right) ule { 0 in } { 0.25 in } , } & { a = 2 o r F } \\ \end{array} \right.
t ^ { t t } \approx \ell ^ { 2 } \delta ^ { - 2 } \, \Rightarrow \, < 0 | T ^ { t t } | 0 > \approx \hbar \ell ^ { 2 } \delta ^ { - 6 } \approx \hbar \ell ^ { - 1 } ( \ell - T ) ^ { - 3 } .
N = \frac { 1 } { 2 } ( 2 \times 2 4 - 2 \times 2 2 - 2 ) = 1 \ ,
H _ { \mu \nu } = \mu \delta _ { \mu , \nu } , \quad E ( \alpha ) _ { \mu \nu } = \delta _ { \mu - \nu , \alpha } .
\Lambda = \frac { d Y } { d { \cal G } } \, { \cal G } - Y
Y ^ { \mu } = \frac { x _ { 1 3 } ^ { \mu } } { x _ { 1 3 } ^ { 2 } } - \frac { x _ { 2 3 } ^ { \mu } } { x _ { 2 3 } ^ { 2 } }
{ } ~ ~ ~ ~ ~ ~ = \Bigg \{ \sum _ { \sigma = 0 } ^ { N - 1 } { \frac { w ( x _ { 3 } , x _ { 1 3 } , x _ { 1 } | \sigma + e - h ) w ( x _ { 4 } , x _ { 2 4 } , x _ { 2 } | \sigma + f - b ) s ( \sigma , b - g ) } { w ( x _ { 4 } , x _ { 1 4 } , x _ { 1 } | \sigma + e - h ) w ( x _ { 3 } , x _ { 2 3 } , \omega x _ { 2 } | \sigma + f - b ) } } \Bigg \} _ { 0 } ,
\overline { { \sigma } } _ { 3 } ^ { \prime } ( s ) = - 4 \frac { \mu ( t ) + \mu ( t ^ { 2 } ) } { \sqrt { s ( s - 4 ) } } \, \Theta ( s - 4 )
F _ { \theta \phi } ^ { I } = k q ^ { I } f ( \theta ) , \qquad A _ { \phi } ^ { I } = k q ^ { I } \intf ( \theta ) d \theta .
L ^ { d = 4 } = \frac { 1 } { 4 e ^ { 2 } } \int \left( W ^ { 2 } d ^ { 2 } \theta + \bar { W } ^ { 2 } d ^ { 2 } \bar { \theta } \right) + \frac { 1 } { e ^ { 2 } } \sum _ { r , s , t = 0 } ^ { \infty } a _ { r s t } \int d ^ { 4 } \theta \left( W ^ { 2 } \bar { W } ^ { 2 } \right) ^ { r } X ^ { s } Y ^ { t }
\operatorname* { s u p } _ { x } \frac { | f ( x ) - f ( p ) | } { \mathrm { d i s t } ( x , p ) } \leq \operatorname* { s u p } _ { x } \operatorname* { l i m } _ { p \to x } \frac { | f ( x ) - f ( p ) | } { \mathrm { d i s t } ( p , q ) } ~ .
\operatorname* { d e t } V _ { \pm } [ \alpha ] \operatorname* { d e t } V _ { \pm } [ \beta ] = \operatorname* { d e t } V _ { \pm } [ \alpha + \beta ] \; ,
\epsilon ( t , z , r , \theta , \phi ) = e ^ { { \frac { i } { 2 } } \Gamma _ { 4 } \theta } e ^ { { \frac { 1 } { 2 } } \Gamma _ { 3 } \Gamma _ { 4 } \phi } \Big ( ( 1 + \Gamma _ { 0 } \Gamma _ { 1 } ) r ^ { \frac { - 1 } { 4 } } - { \frac { ( t - z ) } { 2 ( Z _ { m } ) _ { \mathrm { c r } } ^ { \frac { 3 } { 2 } } } } r ^ { \frac { 1 } { 4 } } ( \Gamma _ { 0 } - \Gamma _ { 1 } ) \Gamma _ { 2 } \Big ) \kappa _ { 0 } .
J ^ { M } = - \frac { 1 } { \sqrt { g _ { 5 } } } \frac { \delta S _ { b u l k } } { \delta \partial _ { M } \phi } .
\left. { \delta I / { \delta \phi } } \right| _ { \phi = \phi _ { 0 } } = 0 ~ ~ ~ .
( z _ { 1 } , { \cal Z } _ { 1 } , { \cal F } _ { 1 } ) \times ( z _ { 2 } , { \cal Z } _ { 2 } , { \cal F } _ { 2 } ) = ( z _ { 1 } z _ { 2 } , \, z _ { 1 } { \cal Z } _ { 2 } + z _ { 2 } { \cal Z } _ { 1 } , \, z _ { 1 } { \cal F } _ { 2 } + z _ { 2 } { \cal F } _ { 1 } - 2 { \cal Z } _ { 1 } { \cal Z } _ { 2 } ) .
b _ { 0 } = b _ { 0 , r e n } + { \frac { m ^ { 2 } ( 1 / 6 - \xi ) } { 1 6 \pi ^ { 2 } \epsilon } }
\dot { G } _ { B } ( \tau , \tau ) = 0 , \quad \dot { G } _ { B } ^ { 2 } ( \tau , \tau ) = 1
h _ { \ell } = - \frac { d ^ { 2 } } { d r ^ { 2 } } + \frac { \ell ( \ell + 1 ) } { r ^ { 2 } } + \frac { \gamma } { r }
E \rightarrow E ^ { \prime } = ( \alpha E + \beta ) ( \gamma E + \delta ) ^ { - 1 } ~ .
\dot { \phi } _ { i } ~ = ~ g _ { i } ( \phi _ { i } , { \frac { \delta S } { \delta \phi _ { i } } } , \rho ) ~ = ~ f _ { i } ( \phi _ { i } , \rho ) .
W = \prod _ { a = 1 } ^ { p - 1 } \left( \frac { n ^ { \frac { 1 } { l } } } { L _ { a } } \right) \int _ { 0 } ^ { \infty } \frac { d s } { s } \left( \frac { \pi } { 2 s } \right) ^ { \frac { p - 1 } { 2 } } e ^ { - r ^ { 2 } s } \frac { 4 \operatorname { s i n h } ^ { 2 } ( \omega _ { l } s \operatorname { s i n } { \frac { \phi } { 2 } } ) - \operatorname { s i n h } ^ { 2 } ( 2 \omega _ { l } s \operatorname { s i n } { \frac { \phi } { 2 } } ) } { \operatorname { c o s } { \frac { \phi } { 2 } } ~ \operatorname { s i n h } ( 2 \omega _ { l } s \operatorname { s i n } { \frac { \phi } { 2 } } ) } \, .
\hat { \theta } = \hat { \theta } ( x , y , \tilde { x } , \tilde { y } ) = \hat { \theta } ^ { + } .
W _ { p , N _ { r } } ^ { ( o ) } ( t ) = ( - 1 ) ^ { e ( N _ { r } ) } \sum _ { N _ { o } , N _ { o } ^ { \prime } } \left[ E _ { p , N _ { o } } ^ { ( - 2 ) } \left( Y _ { b , N _ { r } } ( t ) \right) + \sum _ { N } [ \partial q _ { N } Y _ { p , N _ { r } } ( t ) ] B _ { N N _ { o } } \right] A _ { N _ { o } N _ { o } ^ { \prime } } ^ { - 1 } \chi _ { N _ { o } ^ { \prime } } ( t )
\Omega _ { U V } \Omega _ { V W } \Omega _ { W U } = \mathrm { i d }
\left\{ Q _ { n } ^ { - } , Q _ { n } ^ { + } \right\} _ { P B } = - i ( H _ { n } ) ^ { n } , \qquad \{ Q _ { n } ^ { \pm } , H _ { n } \} _ { P B } = 0 .
L = \sum _ { i = 1 } ^ { N } [ { \frac { 1 } { 2 } } ( P _ { i } ^ { 2 } + { e ^ { - ( Q _ { i + 1 } - Q _ { i } ) } } ) { \dot { Q } } _ { i } + \pi _ { i } ( P ) { \dot { P } } _ { i } ] - H ( Q , P )
( 1 - P ) \, a \, T \ = \ 0 \quad ,
x ^ { 0 } ( \tau , \sigma ) = \frac M L \tau , \quad \quad x ^ { 1 } ( \tau , \sigma ) = q + \frac M { 2 L } \, [ f ( \tau + \sigma ) + g ( \tau - \sigma ) ] ,
| D \rangle _ { 0 } = { \frac { 1 } { \sqrt { M } } } | { \widetilde D } \rangle _ { 0 } ~ .
\overline { { \Delta _ { A Y ^ { \prime } } ^ { a } } } = \epsilon ^ { A B } \epsilon ^ { Y ^ { \prime } Z ^ { \prime } } \Delta _ { B Z ^ { \prime } } ^ { a } \; .
\psi \left( x \right) = \psi _ { o u t } \left( x \right) + \int S _ { A } \left( x - y \right) j \left( y \right) d ^ { 4 } y
f _ { g } ( \mu ) = \frac { 1 + ( 1 - g ) \mu } { 1 - g \mu } \, ,
\Delta a _ { j } ^ { i } ( x , y , z ) = a _ { j } ^ { i } ( x , y , z ) = \sum _ { k , l } a _ { k } ^ { i } ( x ) a _ { l } ^ { k } ( y ) a _ { j } ^ { l } ( z )
\phi _ { n } = \phi _ { n } ^ { \left( 0 \right) } - T \sum _ { m = - \infty } ^ { \infty } \Delta _ { F } ^ { n - m }
e _ { \mu } ^ { a } \, e _ { \nu } ^ { b } \, \hat { \xi } _ { a b } \, e ^ { c \, \mu } \, e ^ { d \, \nu } \, \hat { \xi } _ { c d } = \Omega ^ { - 1 } \, \hat { \eta } _ { \mu \nu } \, \hat { \eta } ^ { \mu \nu } \, \Omega .
\partial _ { a } \hat { X } ^ { - } = 0 , \hspace { 1 . 0 c m } \partial _ { a } \hat { X } ^ { M } = 0
\kappa = \sqrt { { \frac { g ( g + 1 ) } { R ^ { 2 } } } + R ^ { 2 } - ( 2 \epsilon + 2 g - 1 ) } \approx { \frac { \sqrt { g ( g + 1 ) } } { R } }
\Gamma [ W , \bar { W } , q ^ { + } ] = S [ V ^ { + + } , q ^ { + } ] + \bar { \Gamma } [ W , \bar { W } , q ^ { + } ] ,
{ \gamma } _ { 1 1 } = { \gamma } _ { 2 2 } = e ^ { \Psi } ~ , ~ ~ ~ ~ { \gamma } _ { 3 3 } = 1 ,
\delta _ { l } ^ { ( 2 ) } \approx \eta \left[ \operatorname { l n } \left( \frac { z _ { l } } \alpha \right) + \frac 1 { 2 4 z _ { l } ^ { 2 } } - \frac { q ^ { 2 } } { 2 z _ { l } ^ { 2 } } + O \left( \frac 1 { z _ { l } ^ { 4 } } \right) \right] + O ( \eta ^ { 3 } ) \ .
I \propto \epsilon _ { \{ n \} } \epsilon _ { \{ m \} } \prod _ { i } \left\langle P _ { n _ { i } } ( x _ { i } ) Q _ { m _ { i } } ( y _ { i } ) \right\rangle = N ! \prod _ { n = 1 } ^ { N } h _ { n }
[ \Delta _ { \perp } ^ { d } ] ^ { - 1 } ( x _ { \perp } ) = - \int \frac { d ^ { d } k _ { \perp } } { ( 2 \pi ) ^ { d } } \frac { e ^ { i k _ { \perp } \cdot x _ { \perp } } } { k _ { \perp } ^ { 2 } } = - \frac { 1 } { 2 ^ { 2 - d / / 2 } } \frac { 1 } { ( 2 \pi ) ^ { d / 2 } } \frac { 1 } { ( x _ { \perp } ) ^ { d / 2 - 1 } } \Gamma ( d / 2 - 1 )
\langle \Theta ( x ) \ \Theta ( y ) \rangle = { \frac { 2 ^ { 1 3 } \cdot 3 ^ { 3 } \cdot 5 } { \pi ^ { 3 } } } \, \, \frac { \beta ^ { 2 } [ \lambda ( t ) ] f [ \lambda ( t ) ] } { | x - y | ^ { 1 2 } } ~ , ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \mathrm { f o r } ~ ~ x \neq y ,
V _ { \mathrm { s t a t } } ^ { \mathrm { L R } } = \frac { 1 } { 2 } \sigma \, ( r _ { 1 2 } + r _ { 2 3 } + r _ { 3 1 } )
Q ( x , \psi ) = x _ { 1 } ^ { d _ { 1 } } + \cdots + x _ { d + 1 } ^ { d _ { d + 1 } } - k \psi \cdot x _ { 1 } \cdots x _ { d + 1 } ,
\bar { H } _ { \mathrm { g } } = H _ { \mathrm { g } } ( q ^ { r } , p ^ { r } ) + \lambda _ { a } ^ { ( 1 ) r _ { a } } \pi _ { a } ^ { r _ { a } } + \lambda _ { a } ^ { ( 2 ) r _ { a } } \psi _ { a } ^ { r _ { a } } + \lambda _ { a } ^ { ( 3 ) } \theta _ { a } + \lambda _ { a } ^ { ( 4 ) } \gamma _ { a } ,
( a _ { i j } ) = \left( \begin{array} { l l } { 2 } & { - 1 } \\ { - 1 } & { 0 } \\ \end{array} \right) .
d s ^ { 2 } = R _ { h } ^ { 2 } \left[ d \chi ^ { 2 } + \operatorname { s i n h } ^ { 2 } ( \chi ) \left( d \alpha ^ { 2 } + \operatorname { s i n } ^ { 2 } ( \beta ) d \beta ^ { 2 } \right) \right]
\beta ^ { I } = - G ^ { I J } ( \phi ) { \frac { \partial } { \partial \phi ^ { J } } } \, \operatorname { l o g } ( { a \, \langle T \rangle + U ( \phi ) } )
\nabla ^ { 2 } \Phi ^ { X } + \nabla _ { \mu } \psi \nabla ^ { \mu } \Phi ^ { X } = \frac { ( q - 1 ) } { R ^ { 2 } } ( e ^ { - 2 \Phi ^ { X } } - e ^ { - 2 \psi } ) ,
\Delta _ { + } = - ( \beta ^ { - 1 } \partial _ { \mu } \beta ) ( \beta \partial _ { \mu } \beta ^ { - 1 } ) \quad \mathrm { a n d } \quad \Delta _ { - } = - ( \beta \partial _ { \mu } \beta ^ { - 1 } ) ( \beta ^ { - 1 } \partial _ { \mu } \beta ) \, ,
\frac { \delta a } { a _ { p h y s } } = \frac { C _ { 2 } ^ { \infty } } { C _ { 1 } } ,
{ \cal A } ^ { 0 } = - \frac { 1 } { 1 6 \pi } \int d ^ { 4 } x F _ { \mu \nu } ( x ) F ^ { \mu \nu } ( x ) + \int d ^ { 4 } x { \bar { \psi } } ( x ) ( i \partial _ { \mu } \gamma ^ { \mu } - m ) \psi ( x ) .
\chi ( \tilde { \cal M } _ { k } ) = \sum _ { i = 0 } ^ { 4 ( k - 1 ) } \ \sum _ { p = 0 } ^ { k - 1 } \delta _ { i , 2 k - ( p , q ) } = k
T r : e x p i ( 2 \pi F ) = \sum _ { j = 0 } ^ { M - 1 } [ e x p i ( { \frac { 2 \pi j n } { { M } } } ) ] = 0 , \quad
{ \frac { d { Q } } { d \operatorname { l n } { \rho } } } = { \frac { g _ { s } } { 2 \pi } } \, { \widehat C _ { I J } m ^ { I } m ^ { J } } \propto \, { \beta _ { i } \bar { \beta } _ { j } \, G ^ { i j } } ~ ,
B = \int _ { 0 } ^ { x _ { \mathrm { m i n } } } \rho _ { B } d x + \int _ { x _ { \mathrm { m i n } } } ^ { \infty } \rho _ { B } d x \nonumber
t _ { E Q } \sim \frac { M _ { p l } } { T _ { E Q } ^ { 2 } } \ .
x _ { \mu } ~ \longrightarrow ~ \frac { x _ { \mu } } { x ^ { 2 } }
M e _ { \nu } ( z , h ) = \alpha _ { \nu } ( h ) M _ { \nu } ^ { ( 1 ) } ( z , h )
\lambda _ { l } ^ { n } = \lambda _ { c } ^ { n } + i [ n + 1 - 2 l + \frac { \pi } { 2 \gamma } ( 1 - v _ { s } v _ { n } ) ]
\left[ { \alpha } _ { m } ^ { \mu } , { \alpha } _ { n } ^ { \nu } \right] = m { \delta } _ { m + n } { \eta } ^ { \mu \nu }
\Delta _ { n } ( \epsilon ) g = \int _ { \cal C } { \frac { d t } { 2 \pi i } } t ^ { - n - 1 } \delta ( \epsilon , t ) g ,
\gamma B _ { a } ^ { 0 i } = \partial _ { j } \eta _ { a } ^ { ( 2 ) i j } , \; \gamma B _ { a } ^ { i j } = \eta _ { a } ^ { ( 1 ) i j } , \; \gamma H _ { i } ^ { a } = C _ { i } ^ { ( 1 ) a } , \; \gamma H _ { 0 } ^ { a } = - \partial ^ { i } C _ { i } ^ { ( 2 ) a } ,
D _ { i } = \partial _ { i } + e ( \partial _ { j } A _ { i } ) \, \tilde { \partial } _ { j }
H ^ { 2 } = - H \frac { \dot { d _ { 0 } } } { d _ { 0 } } + \frac { \kappa ^ { 2 } ( \rho _ { 1 } + \rho _ { 2 } ) } { 6 d _ { 0 } } ,
\partial _ { x _ { _ \perp } } ^ { 2 } H _ { p } ( x _ { _ \perp } ) = 0 .
\psi = \psi _ { + } + \psi _ { - } , \qquad \psi _ { \pm } = P _ { \pm } \psi , \qquad P _ { \pm } = \frac { 1 } { 4 } \gamma _ { \mp } \gamma _ { \pm } \; ,
F _ { \alpha \beta } = \partial _ { \alpha } A _ { \beta } - \partial _ { \beta } A _ { \alpha } + [ A _ { \alpha } , A _ { \beta } ] _ { \star }
\Omega ^ { 2 } = ( c X _ { 1 } X _ { 2 } ) ^ { 1 / 4 } / \rho , \hspace { 1 c m } c = \operatorname { c o s h } m
\! \! \! \! I \! = \! \int _ { \Sigma } \! d ^ { 3 } \! x \! \sqrt { \gamma } \left( \! R ( \gamma ) \! - \! { \frac { 1 } { 2 } } \! \left( \left( \! D U \right) ^ { 2 } \! - \! e ^ { - 2 U } \! \! \left( \! D V \right) ^ { 2 } \right) \! - \! { \frac { 1 } { 2 } } \! \left( \left( \! D \Phi \! \right) ^ { 2 } \! - \! e ^ { - 2 \Phi } \! \left( \! D \Psi \! \right) ^ { 2 } \right) \! - \! \left( \! D T \right) \! ^ { 2 } \! + \! e ^ { - ( U \! + \Phi ) } V ( T ) \! \right)
\begin{array} { l } { { \cal H } = \frac { N } { 2 } \left( \frac { P ^ { m } P _ { m } } { \sqrt { \beta } } + \frac { \beta _ { a b } } { \sqrt { \beta } } \Pi _ { r } ^ { a } \Pi _ { r } ^ { b } + \sqrt { \beta } \beta ^ { a b } \partial _ { a } X ^ { m } \partial _ { b } X _ { m } - \sqrt { \beta } + \frac { 1 } { 2 } \sqrt { \beta } \beta ^ { a c } \beta ^ { b d } F _ { a b } ^ { r } F _ { c d } ^ { r } \right) } \\ { + \Pi _ { r } ^ { a } \partial _ { a } A _ { 0 } ^ { r } + N ^ { a } \left( \partial _ { a } X ^ { m } P _ { m } + \Pi _ { r } ^ { b } F _ { a b } ^ { r } \right) } \\ \end{array}
{ \cal T } _ { \sigma } = { \cal T } _ { C } ( X ) \otimes { \cal T } _ { C } ( Y )
( \alpha _ { 0 } ^ { I } ) _ { i j } = \frac { 1 } { \pi \sqrt { 2 \alpha ^ { \prime } } } [ X ^ { I } , \cdot \ ] _ { i j } \ .
\frac { 1 } { 2 \pi } \int _ { - \infty } ^ { \infty } \operatorname { l n } \left( 1 - e ^ { - a \sqrt { 1 + k ^ { 2 } } } \right) \frac { d \delta _ { s _ { \parallel , \perp } } ( k ) } { d k } d k
\bar { \partial } _ { \dot { \alpha } } ^ { + } \theta ^ { \prime } { } ^ { \beta } = 0
S \sim \left( N ^ { 2 + \frac { 3 } { 5 } } E ^ { 3 } \right) ^ { \frac { 1 } { 4 } } \; .
[ \rho ( f \bigotimes v ) \varphi ] _ { n } ( { \bf { x _ { 1 } } } \sigma _ { 1 } , { \bf { x _ { 2 } } } \sigma _ { 2 } , . . . . , { \bf { x _ { n } } } \sigma _ { n } ) = \sum _ { i = 1 } ^ { n } f ( { \bf { x _ { i } } } ) a ( \sigma _ { i } ) \varphi _ { n } ( { \bf { x _ { 1 } } } \sigma _ { 1 } , { \bf { x _ { 2 } } } \sigma _ { 2 } , . . . . , { \bf { x _ { n } } } \sigma _ { n } )
b \omega _ { + } = q \omega _ { + } b , ~ ~ ~ ~ d \omega _ { + } = q \omega _ { + } d + \mu \omega _ { 0 } b ,
r _ { + } ^ { 2 } = \frac { l ^ { 2 } } { 2 } \left( 1 \pm \sqrt { 1 - \frac { 4 \tilde { \alpha } } { l ^ { 2 } } } \right) ,
\nabla ^ { ( H ) } Z _ { A B } = Z _ { I } P _ { A B } ^ { I }
d \omega \equiv \left[ g ^ { D } ( d X , d X ) \right] ^ { \frac { 1 } { 2 } } .
s _ { a } = 2 \frac { ( s \xi ) } { \zeta ^ { 2 } } \xi _ { a } + 2 \frac { ( s \bar { \xi } ) } { \zeta ^ { 2 } } \bar { \xi } _ { a } - ( s n ) n _ { a } , \qquad \zeta = 1 - z \bar { z }
( \sum _ { i = 1 } ^ { N } a _ { i } \theta _ { i } ) ^ { p + 1 } = 0 \; .
k _ { 2 } ^ { \mu } [ T _ { \mu \nu } ^ { ( 1 ) a b } + T _ { \mu \nu } ^ { ( 2 ) a b } ] = - g ^ { 2 } f ^ { a b c } \overline { { v } }
j ( x ; z ) \Phi _ { h } ( y , \bar { y } ; w , \bar { w } ) \sim \frac { 1 } { z - w } \left\{ ( y - x ) ^ { 2 } \partial _ { y } + 2 h ( y - x ) \right\} \Phi _ { h } ( y , \bar { y } ; w , \bar { w } ) ,
+ \sum _ { l = t + 1 } ^ { r - 2 } q ^ { \theta ( l > t + 1 ) \sum _ { \nu - r + t + 1 } ^ { \nu - r + l - 1 } n _ { i } ^ { \prime } } \prod _ { \stackrel { i = 1 } { i \neq \nu - r + t } } ^ { \nu - 2 } \left[ \begin{array} { c } { n _ { i } ^ { \prime } + \tilde { n } _ { i } - V _ { i , r } + \theta ( i > \nu - r + l ) + \theta ( t > 0 ) \sum _ { m = 0 } ^ { t - 1 } \delta _ { i , \nu - r + m } } \\ { n _ { i } ^ { \prime } - \delta _ { i , \nu - r + l } } \\ \end{array} \right] _ { q } \times
{ F } _ { \mathrm { r e n } } ^ { \infty } = - \frac 1 { 8 \pi ^ { 2 } } \int \! { \mathrm d } ^ { 3 } x \, { g } ^ { 1 / 2 } { \mathrm t r } \Big \{ R _ { i j } \gamma _ { 1 } ( - \triangle ) R ^ { i j } + R \gamma _ { 2 } ( - \triangle ) R + { \mathrm { O } } [ \Re ^ { 3 } ] \Big \} .
\partial _ { \rho } G _ { \mu \nu } = \partial _ { \rho } G _ { \mu \nu } ^ { ( 0 ) } + \gamma _ { \mu \nu } u _ { \rho } \theta ( u ) .
G _ { \mathrm { \tiny ~ m u l t i p l e t } } = U ( 1 ) \times S U ( 2 ) \times S U ( 2 j + 1 ) _ { f } \times C P .
K _ { I } ^ { ( h ) } ( \theta ) = \sqrt { \ell ^ { \prime } } \; \; \frac { \textrm { s n } \left[ \frac { \textbf { K } } { i \pi } \left( \theta - i \frac { \pi } { 2 } \right) \right] } { \textrm { c n } \left[ \frac { \textbf { K } } { i \pi } \left( \theta - i \frac { \pi } { 2 } \right) \right] } \cdot \frac { 1 - \frac { h ^ { 2 } } { 2 \textrm { M } } - \textrm { s n } \left( \frac { i \textbf { K } } { \pi } 2 \theta \right) } { 1 - \frac { h ^ { 2 } } { 2 \textrm { M } } + \textrm { s n } \left( \frac { i \textbf { K } } { \pi } 2 \theta \right) } \, , \qquad \sqrt { 2 \textrm { M } } \leq h < \infty \, ,
d A = \sqrt { E G - F ^ { 2 } } \hspace { 0 . 2 c m } d u \hspace { 0 . 1 c m } d v = | \operatorname { s i n } \phi | \hspace { 0 . 2 c m } d u \hspace { 0 . 1 c m } d v
z _ { * } = f ^ { - 1 } ( 1 / r ) = F ( 1 / r ) \, , \quad \alpha _ { * } = r \, .
d S \equiv 2 i \, d z \, d \bar { z } / \left( 1 + | z | ^ { 2 } \right) ^ { 2 }
z = \frac { 1 } { a } ( w - w _ { 0 } ) - \frac { b } { a ^ { 3 } } ( w - w _ { 0 } ) ^ { 2 } - \frac { 1 } { a ^ { 5 } } ( a c - 2 b ^ { 2 } ) ( w - w _ { 0 } ) ^ { 3 } - \frac { 1 } { a ^ { 7 } } ( a ^ { 2 } d - 5 a b c + 5 b ^ { 3 } ) ( w - w _ { 0 } ) ^ { 4 } + \ldots
D _ { a } X _ { \mu } ( \delta _ { \mu \nu } I - i [ X _ { \mu } , X _ { \nu } ] ) ^ { - 1 } D _ { b } X _ { \nu } = ( I - [ X , Y ] ^ { 2 } ) ^ { - 1 } ( D _ { a } X _ { \mu } D _ { b } X _ { \mu } + i D _ { a } X _ { \mu } [ X _ { \mu } , X _ { \nu } ] D _ { b } X _ { \nu } ) .
\stackrel { . } { \theta } ^ { a } = \left\{ \theta ^ { a } , H _ { 2 } ^ { \left( 1 \right) } \right\} = 0 ,
\alpha _ { w e a k } ^ { u _ { R } } = \frac { 4 } { 9 } \alpha _ { e l } \frac { \operatorname { s i n } ^ { 2 } \theta _ { W } } { \operatorname { c o s } ^ { 2 } \theta _ { W } } .
c ^ { a } = \hat { \eta } ^ { a } \ \ \ , \ \ \ b _ { a } = \frac { i } { \hbar } \hat { { \cal P } } _ { a } \ \ \ , \ \ \ a = 1 , 2 \ \ \ ,
\left\langle \int d ^ { 2 } x : \Psi ^ { \dagger } ( x ) M \Psi ( x ) : \right\rangle = - \frac { 1 } { 2 } \left( \sum _ { \mathrm { o c c u p i e d } } \int d ^ { 2 } x \psi _ { E } ^ { \dagger } ( x ) M \psi _ { E } ( x ) - \sum _ { \mathrm { u n o c c u p i e d } } \int d ^ { 2 } x \psi _ { E } ^ { \dagger } ( x ) M \psi _ { E } ( x ) \right) .
\begin{array} { l } { \delta \theta ^ { \alpha } = \epsilon ^ { \alpha } , \qquad \delta \bar { \theta } _ { \alpha } = \bar { \epsilon } _ { \alpha } , } \\ { \delta x ^ { \bar { \mu } } = - i \bar { \epsilon } _ { \alpha } \tilde { \Gamma } ^ { { \bar { \mu } } \alpha \beta } \bar { \theta } _ { \beta } + i \epsilon ^ { \alpha } \Gamma _ { \alpha \beta } ^ { \bar { \mu } } \theta ^ { \beta } , \qquad \delta x ^ { 1 1 } = 0 , } \\ \end{array}
u = \phi _ { A B } \phi _ { B A } , v = \phi _ { A C } \phi _ { C A } , w = \phi _ { A D } \phi _ { D A } , t = \phi _ { A B } \phi _ { B D } \phi _ { D A }
Z _ { \alpha _ { 1 } , j } ^ { \alpha _ { 0 } } R _ { \beta _ { 0 } } ^ { j } + F _ { \beta _ { 0 } \gamma _ { 0 } } ^ { \alpha _ { 0 } } Z _ { \alpha _ { 1 } } ^ { \gamma _ { 0 } } = - Z _ { \gamma _ { 1 } } ^ { \alpha _ { 0 } } G _ { \beta _ { 0 } \alpha _ { 1 } } ^ { \gamma _ { 1 } } ,
\Gamma ^ { \mu _ { 1 } . . . \mu _ { k } } \equiv { \frac { 1 } { k ! } } \Gamma ^ { [ \mu _ { 1 } } . . . \Gamma ^ { \mu _ { k } ] }
\left( M ^ { T } \overline { M } \right) _ { j k } = 2 \pi D _ { j j } K _ { N } ( \theta _ { j } , \theta _ { k } ) \bar { D } _ { k k }
\Psi _ { \bf D } \rightarrow \operatorname { e x p } { ( \phi _ { 1 } E _ { 1 } - \theta _ { 2 } E _ { 2 } + \phi _ { 3 } E _ { 3 } + \theta _ { 1 } F _ { 1 } + \phi _ { 2 } F _ { 2 } + \theta _ { 3 } F _ { 3 } ) } \cdot \Psi _ { \bf D } ,
\Delta ( \varphi ) = \varphi + \tilde { \varphi }
F _ { 1 2 } = 2 \pi i / ( L _ { 1 } L _ { 2 } ) \tau _ { 3 } \, , \, F _ { 3 4 } = 2 \pi i / ( L _ { 3 } L _ { 4 } ) \tau _ { 3 }
\left( d - { \textstyle \frac { 1 } { 4 } } \omega _ { a b } \gamma ^ { a b } + \Omega \right) \kappa = 0 \, .
F _ { \mu _ { 1 } \mu _ { 2 } \mu _ { 3 } \mu _ { 4 } } = e \, \epsilon _ { \mu _ { 1 } \mu _ { 2 } \mu _ { 3 } \mu _ { 4 } }
w _ { R } ( E ) \simeq { \frac { 1 } { \sigma _ { R } \sqrt { \pi } } } e ^ { - { \frac { ( E - E _ { R } ) ^ { 2 } } { \sigma _ { R } ^ { 2 } } } } ~ ~ ~ .
{ \cal H } = { \frac { 1 } { 2 } } ( \pi _ { \phi } - e A _ { 1 } ) ^ { 2 } + { \frac { 1 } { 2 } } \pi _ { 1 } ^ { 2 } + { \frac { 1 } { 2 } } \phi ^ { 2 } + \pi _ { 1 } A _ { 0 } ^ { \prime } + e \phi ^ { \prime } A _ { 0 } + { \frac { 1 } { 2 } } a e ^ { 2 } ( A _ { 0 } ^ { 2 } - A _ { 1 } ^ { 2 } ) .
\sum _ { i = 1 } ^ { 3 } T r ( \gamma _ { \theta , 7 _ { i } } ) + 3 T r ( \gamma _ { \theta , 3 } ) = 0 .
| \Upsilon | = M ^ { - 1 } | Z _ { 1 } Z _ { 2 } | \, .
d s ^ { 2 } \rightarrow - K _ { \tau } e ^ { ( \lambda _ { 1 } - \lambda _ { 3 } - 6 ) \tau } d \tau ^ { 2 } + K _ { x } e ^ { - ( \lambda _ { 3 } + 2 ) \tau } d \vec { x } ^ { 2 } + K _ { y } e ^ { ( \lambda _ { 1 } - 2 ) \tau } d \vec { y } ^ { 2 }
\hat { \varrho } _ { \mathrm { f l d } } = \hat { \varrho } _ { m } \left[ 1 + \zeta \left( { \frac { \hat { c } } { c } } \right) ^ { 2 } \right] + \hat { \varrho } _ { r } \left( { \frac { \hat { c } } { c } } \right) ^ { 2 } , \ \ \ \ \ \ \ \hat { \wp } _ { \mathrm { f l d } } = { \frac { 1 } { 3 } } \hat { \varrho } _ { r } \left( { \frac { \hat { c } } { c } } \right) ^ { 2 } ,
d e t \left| \begin{array} { c c c } { Q ( \lambda - 3 i / 2 ) } & { Q ( \lambda \pm i / 2 ) } & { Q ( \lambda + 3 i / 2 ) } \\ { P ( \lambda - 3 i / 2 ) } & { P ( \lambda \pm i / 2 ) } & { P ( \lambda + 3 i / 2 ) } \\ { R ( \lambda - 3 i / 2 ) } & { R ( \lambda \pm i / 2 ) } & { R ( \lambda + 3 i / 2 ) } \\ \end{array} \right| = T ^ { \pm } ( \lambda ) .
\left[ - [ f ( p ) \partial _ { p } ] ^ { 2 } - \xi ^ { 2 } + 2 \, a ( i \, f ( p ) \partial _ { p } - \xi ) + 2 \, b ( v ( p ) - \gamma ) - \mu ^ { 2 } \right] \Psi ( p ) = 0
\left( \frac { i \phi } { { 2 \pi } } \right) ^ { 2 n } \int _ { M } \mathrm { e } ^ { i \phi H } \frac { \omega ^ { n } } { n ! } = \sum _ { d H = 0 } \frac { \mathrm { e } ^ { i \phi H } { { \sqrt { \operatorname* { d e t } \omega } } } } { \sqrt { \operatorname* { d e t } \mathrm { P f } H } } .
B \times J = \pi \frac { 2 + 2 \operatorname { c o s } 2 v s + 2 \operatorname { c o s h } 2 s c _ { 1 } + 2 \operatorname { c o s h } 2 s c _ { 2 } - 8 \operatorname { c o s } v s ( \operatorname { c o s h } s c _ { 1 } \operatorname { c o s h } s c _ { 2 } ) } { 4 \operatorname { s i n h } s c _ { 1 } \operatorname { s i n h } s c _ { 2 } \operatorname { s i n } s v }
\left. \prod _ { p \in Y _ { j } } \, \frac { e \beta } { 1 - \frac { e \beta } { 2 } } \leq e ^ { - N _ { p } ( Y _ { j } ) \, \operatorname { l o g } ( \frac { 1 } { 2 e \beta } ) } \ \ \ \ \ \ \ \ \ \ \mathrm { f o r } \ \ \ \beta < \frac { 1 } { e } \ , \right.
\frac { \partial \Sigma } { \partial m } = \mathcal { B } _ { \Sigma } \Lambda \; ,
d s ^ { 2 } / \alpha ^ { \prime } = \frac { U } { R _ { 5 } f _ { 1 } ^ { 1 / 2 } } ( - d t ^ { 2 } + d x _ { 1 } ^ { 2 } ) + \frac { U f _ { 1 } ^ { 1 / 2 } } { R _ { 5 } } ( d x _ { 2 } ^ { 2 } + \cdots + d x _ { 5 } ^ { 2 } ) + U R _ { 5 } f _ { 1 } ^ { 1 / 2 } \left( \frac { d U ^ { 2 } } { U ^ { 2 } } + d \Omega _ { 3 } ^ { 2 } \right) .
L _ { 0 } = \mathrm { D i a g o n a l } ( 0 , 2 , 3 , 4 , 4 , 5 , 5 , 6 , 6 , 6 , 6 , 7 , 7 , 7 , 7 , 8 , 8 , 8 , 8 , 8 , 8 , 8 , \ldots )
\phi _ { i } = \left( \begin{array} { c c c c c } { a _ { i } ^ { 1 } } & { } & { } & { } & { } \\ \end{array} \right) ,
z _ { \nu } ^ { ( 3 ) } + p \ddot { z } _ { \nu } + q \dot { z } _ { \nu } + r \lambda _ { \nu } = 0 \; .
\partial _ { \mu } V _ { \mu } ^ { a } = 0 , ~ ~ ~ ~ ~ ~ \partial _ { \mu } W _ { 5 \mu } ^ { a } = 0 .
u _ { t _ { 2 n + 1 } } = { \cal K } _ { 2 n + 1 } , ( n = 1 , 2 , \cdots ) .
t _ { 2 } = \operatorname { t a n } \left( \frac { \pi } { 2 4 } \right) T _ { 0 } T _ { 2 } T _ { 3 } / \sqrt { 3 } = \operatorname { t a n } \left( \frac { \pi } { 2 4 } \right) \operatorname { t a n } \left( \frac { 5 \pi } { 2 4 } \right) \operatorname { t a n } \left( \frac { \pi } { 1 2 } \right)
\theta ^ { \prime } ( u ^ { \prime } ) = D ( \Lambda , u ) \theta ( u ) D ^ { - 1 } ( \Lambda , u ) .
y _ { \alpha _ { 1 } } = \left( \begin{array} { c } { \pi } \\ { \varphi } \\ \end{array} \right) ,
\mu ^ { \prime } = \frac { 1 } { 2 } N x ^ { 2 } \phi ^ { 2 } + e ^ { 2 \gamma \phi } \left[ N ( K ^ { 2 } + H ^ { 2 } ) + \frac { 1 } { 8 x ^ { 2 } } \left( \left( K ^ { 2 } + H ^ { 2 } - 4 \right) ^ { 2 } + 1 2 K ^ { 2 } H ^ { 2 } \right) \right] \ ,
g | \Lambda \rangle = \xi h | \Lambda \rangle = \xi | \Lambda \rangle e ^ { i \phi ( h ) } .
\langle 0 | ( X _ { 1 1 } ^ { 1 } ) ^ { 2 } | 0 \rangle \langle 0 | ( P _ { 1 1 } ^ { 1 } ) ^ { 2 } | 0 \rangle \geq 1 \ .
\sum _ { i = 1 } ^ { d + 2 } { x _ { i } } ^ { ( d + 2 ) } - ( d + 2 ) \, \psi \, { x _ { 1 } x _ { 2 } \cdot \cdot \cdot x _ { d + 2 } } = 0
\Delta s = \left( \frac { g _ { \star \mathrm { R H } } } { g _ { \star \mathrm { d o m } } } \right) ^ { 1 / 4 } \, \left( \frac { H _ { \mathrm { d o m } } } { H _ { \mathrm { R H } } } \right) ^ { [ 4 - 3 ( 1 + w ) ] / [ 2 ( 1 + w ) ] } \, \Omega _ { r } ^ { - 3 / 4 } .
\Delta S _ { M } ^ { \mathrm { n l } } ( \lambda = 0 ) = \frac { 5 M \kappa ^ { 2 } } { 1 5 3 6 \pi ^ { 2 } } \int d ^ { 4 } x \sqrt { - g } R \operatorname { l n } ( - \nabla ^ { 2 } ) \delta ^ { 3 } ( { \bf x } ) ,
C _ { S R G } ^ { ( g ) } = \frac { 1 } { \sqrt { X } } X ^ { + } X ^ { - } - 8 \lambda ^ { 2 } \sqrt { X } \, .
\Phi ( \omega ) \sim { \frac { 2 \omega ^ { D - 2 } } { ( 4 \pi ) ^ { ( D - 1 ) / 2 } } } \sum _ { n = 0 } ^ { \infty } { \frac { c _ { n } } { \Gamma \left( { \frac { D - 1 } { 2 } } - n \right) } } \omega ^ { - 2 n } ,
T ^ { 0 } = \frac { 1 } { 4 \pi \kappa _ { B } k } \frac { ( d - 2 k - 1 ) } { r _ { + } } .
\begin{array} { c } { { \cal L } = g _ { a { \bar { b } } } \left( { \dot { z } } ^ { a } { \dot { \bar { z } } } ^ { b } + \frac 1 2 \eta _ { k } ^ { a } \frac { D { \bar { \eta } } _ { k } ^ { \bar { b } } } { d \tau } + \frac 1 2 \frac { D \eta _ { k } ^ { a } } { d \tau } \bar { \eta } ^ { \bar { b } } \right) - } \\ { - g ^ { a { \bar { b } } } ( G _ { a } G _ { \bar { b } } + { U } _ { a } { \bar { U } } _ { \bar { b } } ) + } \\ { + i U _ { a ; b } \eta _ { 1 } ^ { a } \eta _ { 2 } ^ { b } - i { \bar { U } } _ { \bar { a } ; \bar { b } } { \bar { \eta } } _ { 1 } ^ { \bar { a } } { \bar { \eta } } _ { 2 } ^ { \bar { b } } + R _ { a \bar { b } c \bar { d } } \eta _ { 1 } ^ { a } \bar { \eta } _ { 1 } ^ { b } \eta _ { 2 } ^ { a } \bar { \eta } _ { 2 } ^ { d } . } \\ \end{array}
\Lambda = D _ { 0 } \otimes \Lambda _ { 2 p } ^ { C } = \mathrm { d i a g } \left( d _ { 1 } \Lambda _ { 2 p } ^ { C } , \ldots , d _ { s } \Lambda _ { 2 p } ^ { C } \right) .
c ( z ) \equiv ( a \ast b ) ( z ) = \left( e ^ { \pi i \theta ^ { j k } \frac { \partial } { \partial x ^ { j } } \frac { \partial } { \partial y ^ { k } } } a ( x ) b ( y ) \right) _ { x = y = z } \, .
J ~ \equiv ~ \epsilon ^ { \alpha \beta \gamma \delta } \epsilon _ { a b c d e } \phi ^ { e } \nabla _ { \alpha } \phi ^ { a } \nabla _ { \beta } \phi ^ { b } \nabla _ { \gamma } \phi ^ { c } \nabla _ { \delta } \phi ^ { d } ~ ,
I _ { g } = \frac { 1 } { 2 \pi \kappa } \int d ^ { 2 } x \epsilon ^ { \mu \nu } \Bigl ( \eta _ { a } ( \partial _ { \mu } e _ { \nu } ^ { a } + \omega _ { \mu } \epsilon _ { ~ b } ^ { a } e _ { \nu } ^ { b } ) + \eta _ { 2 } \partial _ { \mu } \omega _ { \nu } + \eta _ { 3 } ( \partial _ { \mu } a _ { \nu } + \frac { 1 } { 2 } \epsilon _ { a b } e _ { \mu } ^ { a } e _ { \nu } ^ { b } ) \Bigr ) \, .
\int _ { \beta } \Omega ^ { ( 1 , 0 ) } = \frac { \sqrt { 6 } } { \pi } ( \pm \sqrt { B ^ { \prime } } ) ^ { - 1 / 2 } \, ( \pm k _ { \mp } ^ { 2 } ) ^ { - 1 / 2 } \, \, { \bf K } ( k _ { \mp } ^ { - 2 } )
{ \frac { d H _ { 2 } } { d t } } = \nu \int \sp { 1 / a } k \sp 2 H _ { k } d k \propto \nu a \sp { - 6 - 4 \Delta _ { \psi } } \propto \nu \sp { \frac { 3 + \Delta _ { \phi } + \Delta _ { \psi } } { \Delta _ { \phi } - \Delta _ { \psi } } } \, .
X \Psi _ { n _ { 1 } n _ { 2 } m } ( \mu , \nu , \varphi ; \delta _ { 1 } , \delta _ { 2 } ) = - \alpha \frac { n _ { 1 } - n _ { 2 } + \frac { \delta _ { 1 } - \delta _ { 2 } } { 2 } } { n + \frac { \delta _ { 1 } + \delta _ { 2 } } { 2 } } \, \Psi _ { n _ { 1 } n _ { 2 } m } ( \mu , \nu , \varphi ; \delta _ { 1 } , \delta _ { 2 } ) ,
\left( \displaystyle \frac { \operatorname { s i n h } \displaystyle \frac { \gamma } { \pi } \left[ \vartheta _ { j } + \Theta + \displaystyle \frac { i \pi } { 2 } \right] \operatorname { s i n h } \displaystyle \frac { \gamma } { \pi } \left[ \vartheta _ { j } - \Theta + \displaystyle \frac { i \pi } { 2 } \right] } { \operatorname { s i n h } \displaystyle \frac { \gamma } { \pi } \left[ \vartheta _ { j } + \Theta - \displaystyle \frac { i \pi } { 2 } \right] \operatorname { s i n h } \displaystyle \frac { \gamma } { \pi } \left[ \vartheta _ { j } - \Theta - \displaystyle \frac { i \pi } { 2 } \right] } \right) ^ { N } = - e ^ { 2 i \omega } \prod _ { k = 1 } ^ { M } \displaystyle \frac { \operatorname { s i n h } \displaystyle \frac { \gamma } { \pi } \left[ \vartheta _ { j } - \vartheta _ { k } + i \pi \right] } { \operatorname { s i n h } \displaystyle \frac { \gamma } { \pi } \left[ \vartheta _ { j } - \vartheta _ { k } - i \pi \right] }
v ( \infty ) = \frac { L ^ { 2 } } { 4 \pi ^ { 2 } } \int \mathrm { T r } \; F _ { ( \infty ) } ^ { 2 } = 1 .
V ( \Phi ) = \frac { m ^ { 2 } } { 2 } | \Phi | ^ { 2 } + \frac { \lambda } { 4 ! } | \Phi | ^ { 4 } ,
R _ { \hat { g } } ( z , \bar { z } ) = - 1 \to R _ { g } ( z , \bar { z } ) = - e ^ { 4 \pi G ( z , w ) } \left( 1 + 8 \pi e ^ { - \varphi ( z , \bar { z } ) } \delta ^ { ( 2 ) } ( z - w ) + { \frac { 1 } { 4 \chi ( \Sigma ) } } \right) .
\left\{ \begin{array} { l } { \tilde { x } \, x ^ { \prime } = \bar { q } \, \overline { { R } } \, x \, \tilde { x } ^ { \prime } \, , } \\ { d \tilde { x } \, x ^ { \prime } = \bar { q } \, \overline { { R } } \, x \, d \tilde { x } ^ { \prime } - \lambda \, \bar { q } \, d x \, \tilde { x } ^ { \prime } \, , } \\ { \tilde { x } \, d x ^ { \prime } = \bar { q } \, R \, d x \, \tilde { x } ^ { \prime } \, , } \\ { d \tilde { x } \, d x ^ { \prime } = - \bar { q } \, R \, d x \, d \tilde { x } ^ { \prime } \, ; } \\ \end{array} \right.
j _ { [ e , m , b ] } ^ { \nu } = j _ { [ e ] } ^ { \nu } + j _ { [ m ] } ^ { \nu } * K _ { [ e ] } + j _ { [ b ] } ^ { \nu } * \varepsilon _ { 0 } * K _ { [ v ] }
- { \frac { 8 \pi } { 1 5 } } < \mathrm { a r g } ( \eta ) < { \frac { 8 \pi } { 1 5 } }
( c _ { 0 } ^ { ( 1 ) } + c _ { 0 } ^ { ( 2 ) } + c _ { 0 } ^ { ( 3 ) } ) | V _ { 3 } \rangle = 0 , \ \ c _ { 0 } | I \star A \rangle = | I \star ( c _ { 0 } A ) \rangle , \ \ \forall A
\tilde { R } _ { \xi } ^ { + } = \operatorname* { l i m } _ { q \to 1 } \left[ \frac { \alpha ( 2 - \alpha ) ( 1 - \alpha ) \mathcal { A } _ { f } } { ( q - 1 ) } \right] ^ { \frac { 1 } { 2 } } \xi _ { 0 } ^ { + } .
m _ { j } = 8 \, p ^ { \frac { r } { 6 s } } \left\{ \sum _ { a } \operatorname { s i n } \frac { a \pi } { g } + \frac { 4 } { 3 } ( p ^ { \frac { r } { 6 s } } ) ^ { 2 } \sum _ { a } \operatorname { s i n } ^ { 3 } \frac { a \pi } { g } + \ldots \right\} ,
J _ { \mu } = \epsilon _ { \mu \nu \rho } x ^ { \nu } p ^ { \rho } + S c ^ { 2 } \frac { p _ { \mu } } { \sqrt { p ^ { 2 } } }
{ \cal F } = - { \frac { 1 } { \sqrt { c _ { 0 } } g ^ { 2 } ( 1 + \gamma ) } } ( 1 - 2 g ^ { 2 } | \phi | ^ { 2 } ) ^ { ( \gamma + 1 ) / 2 }
{ \Lambda } ( l ) = \frac { 1 } { N } { \times } ( ~ M i n i m u m ~ o f ~ s _ { F } )
\frac { \partial S } { \partial \xi } = \int d ^ { 3 } x d ^ { 3 } y \, A _ { i } ^ { A } ( x ) W ( x - y ) \partial _ { i } D _ { j } ^ { A B } \frac { \delta S } { \delta A _ { j } ^ { B } ( y ) } .
G _ { a b } = \frac { 1 } { \sqrt { - g } } \left( \frac { \partial { \cal L } _ { 1 } } { \partial g ^ { a b } } - \partial _ { c } \frac { \partial { \cal L } _ { 1 } } { \partial ~ \partial _ { c } g ^ { a b } } \right) .
( \mu \frac { \partial } { \partial \mu } + \beta _ { g } \frac { \partial } { \partial g } + \beta _ { \lambda } \frac { \partial } { \partial \lambda } - \gamma _ { M } \frac { \partial } { \partial \operatorname { l n } M ^ { 2 } } + \gamma ) F _ { n } ( x _ { 1 } , . . . , x _ { n } ; m , M , g , \lambda , \mu ) = 0 ,
( \bar { \varepsilon } _ { + } ^ { a } ) ^ { \dagger } ( \bar { \varepsilon } _ { - } ^ { a } ) = 0 ; \hspace { 1 . 0 c m } ( \bar { \varepsilon } _ { + } ^ { a } ) ^ { \dagger } ( \bar { \varepsilon } _ { + } ^ { a } ) = ( \bar { \varepsilon } _ { - } ^ { a } ) ^ { \dagger } ( \bar { \varepsilon } _ { - } ^ { a } ) = 1 .
[ T _ { i } , T _ { j } ] = c _ { i j k } T _ { k } \ , \quad [ T _ { i } , T _ { j } ^ { \prime } ] = c _ { i j k } T _ { k } ^ { \prime } \ , \quad [ T _ { i } ^ { \prime } , T _ { j } ^ { \prime } ] = - c _ { i j k } T _ { k } \ .
( 1 - \gamma ^ { 1 } ) k _ { n } ( 0 , x ^ { \prime } ) = \left( \begin{array} { c c } { 1 } & { i } \\ { - i } & { 1 } \\ \end{array} \right) k _ { n } ( 0 , x ^ { \prime } ) = 0 .
\tilde { \Gamma } ( x ) = \frac { ( 2 \pi ) ^ { 4 } } { ( x ^ { 2 } ) ^ { 2 } }
f ( Q ) = F ( u ) : = \frac { 1 } { 4 0 } ( u - 6 ) ( u - 1 2 ) ( u - 1 5 )
\tilde { Z } _ { i j } = i \sigma _ { 2 } \left( \begin{array} { l l } { | Z | } & { 0 } \\ { 0 } & { | Z | } \\ \end{array} \right) \ , \qquad \tilde { Z } _ { i j } = i \sigma _ { 2 } \left( \begin{array} { l l l l } { | Z | } & { 0 } & { 0 } & { 0 } \\ { 0 } & { | Z | } & { 0 } & { 0 } \\ { 0 } & { 0 } & { | Z | } & { 0 } \\ { 0 } & { 0 } & { 0 } & { | Z | } \\ \end{array} \right) \ ,
\vert \Psi ( t ) \rangle = \sum _ { n } C _ { n } \vert e _ { n } , t \rangle e ^ { i \int d t \langle e _ { n } , t \vert i \frac { \partial } { \partial t } - \hat { H } \vert e _ { n } , t \rangle } .
Z _ { 3 } = \frac { ( m L ) ^ { 3 } } { 3 ! } I _ { 1 } ^ { 4 } - \frac { ( m L ) ^ { 2 } } { 2 } I _ { 1 } I _ { 2 } + \frac { ( m L ) } { 3 } I _ { 3 } \, \, \, ,
\chi _ { \Lambda } ( C _ { m } ^ { \Lambda _ { 0 } } ) = \sum _ { ( \lambda + \lambda _ { 0 } ^ { \prime } , \kappa + \kappa _ { 0 } , - s ) \in D ^ { + } } \sum _ { s = 0 } ^ { \infty } \, m _ { \lambda _ { 0 } ^ { \prime } , s } \, \alpha _ { \lambda _ { 0 } ^ { \prime } , s } ^ { m } ( \Lambda ) \frac { D _ { q } [ ( \lambda + \lambda _ { 0 } ^ { \prime } , \kappa + \kappa _ { 0 } , - s ) ] } { D _ { q } [ ( \lambda , \kappa , 0 ) ] } \, , ~ ~ ~ m \in { \bf Z } ^ { + }
a _ { i } a _ { j } ^ { \dagger } - \hat { q } a _ { j } ^ { \dagger } a _ { i } = \delta _ { i j } , \qquad \hat { q } | \pm > = \pm 1 | \pm > .
\left( \begin{array} { c c } { - \operatorname { c o s } 2 \phi } & { - \operatorname { s i n } 2 \phi } \\ { \operatorname { s i n } 2 \phi } & { - \operatorname { c o s } 2 \phi } \\ \end{array} \right)
\tilde { A } _ { l } ( { \bf x } ) = A _ { l } ( { \bf x } ) ,
\vert \nu _ { A } \rangle = { \frac { 1 } { \sqrt { 1 + \vert \nu _ { A } \vert ^ { 2 } } } } \left( \begin{array} { c } { - \nu _ { A } } \\ { 1 } \\ \end{array} \right) \quad , \quad \langle \nu _ { A } \vert = { \frac { 1 } { \sqrt { 1 + \vert \nu _ { A } \vert ^ { 2 } } } } \Bigl ( - \nu _ { A } ^ { * } \; \quad \; 1 \Bigr ) \; .
A ( x _ { 1 } ) B ( x _ { 2 } ) = \cdots + g _ { A B C } \, \frac { 1 } { ( x _ { 1 2 } ^ { 2 } ) ^ { \frac { 1 } { 2 } ( \eta _ { A } + \eta _ { B } - \eta _ { C } ) } } C ^ { \eta _ { C } , \eta _ { A } - \eta _ { B } } ( x _ { 1 2 } , \partial _ { 2 } ) C ( x _ { 1 } ) + \cdots { } .
\rho ( \phi ) = \frac { 1 } { 2 \pi } \int _ { 0 } ^ { 1 } d t \; \frac { \theta \left( 4 R ( t ) - \phi ^ { 2 } \right) } { \sqrt { 4 R ( t ) - \phi ^ { 2 } } }
\rho \partial _ { \pm } F = \pm \partial _ { \pm } ( \tau \rho F ) .
e ^ { i \theta } = { \frac { p - q \tau ^ { * } } { | p - q \tau ^ { * } | } } ,
T ( \xi ) = e ^ { i \xi _ { i } \Sigma ^ { i } } = e ^ { 2 i \xi _ { i } { \cal S } ^ { i } } \, , \quad \xi = \xi _ { i } f ^ { i } \in L _ { F } ,
r _ { 2 } = r _ { 1 } ~ ~ ~ ~ ~ ; ~ ~ ~ ~ ~ t _ { 2 } = t _ { 1 } ~ ~ ~ ~ ~ ; ~ ~ ~ ~ ~ \phi _ { 2 } = \phi _ { 1 } + \gamma ~ ~ ( \phi _ { 1 } > 0 ) ~ ~ ~ ~ ~ ; ~ ~ ~ ~ ~ \phi _ { 2 } = \phi _ { 1 } - \gamma ~ ~ ( \phi _ { 1 } < 0 ) .
( \sum _ { n = 1 } ^ { 1 5 } ( - 1 ) ^ { n } - 1 ) ( \frac { g ^ { 2 } } { 3 2 \pi ^ { 2 } } ) t r \epsilon ^ { \mu \nu \alpha \beta } F _ { \mu \nu } F _ { \alpha \beta } = - 2 ( \frac { g ^ { 2 } } { 3 2 \pi ^ { 2 } } ) t r \epsilon ^ { \mu \nu \alpha \beta } F _ { \mu \nu } F _ { \alpha \beta }
\Sigma = S ^ { \mathrm { N = 4 } } + S _ { \mathrm { e x t } } + S _ { \mathrm { g f } } \, \, ,
\tilde { L } _ { 0 } ^ { k , k } ( e ^ { x } ) = \sum _ { d = 0 } ^ { \infty } \frac { ( k d ) ! } { ( d ! ) ^ { k } } e ^ { d x } ,
\int d u \; 1 = 1 , \ \ \int d u \; u _ { ( A _ { 1 } } ^ { + } \ldots u _ { A _ { p } } ^ { + } u _ { B _ { 1 } } ^ { - } \ldots u _ { B _ { q } ) } ^ { - } = 0 \ \mathrm { f o r ~ p ~ a n d / o r ~ q > 0 ~ } \; .
{ \displaystyle x = \frac { 3 / 2 e _ { 1 } ( \tau ) } { 3 / 2 e _ { 1 } ( \tau ) - \hat { u } } , } \hspace { 1 c m } { \displaystyle \sqrt { y } = - \frac { e _ { 2 } ( \tau ) - e _ { 3 } ( \tau ) } { 3 e _ { 1 } ( \tau ) } , }
{ \cal L } = - \frac { 1 } { 4 } F _ { \mu \nu } F ^ { \mu \nu } + \frac { 1 } { 2 } m ^ { 2 } ( A _ { \mu } + \partial _ { \mu } \theta ) ( A ^ { \mu } + \partial ^ { \mu } \theta ) + A ^ { \mu } \partial _ { \mu } B - \partial _ { \mu } \bar { \cal C } \partial ^ { \mu } { \cal C } - \frac { 1 } { 2 } \alpha B ^ { 2 } .
S = \frac { 1 } { 1 6 \pi } \int d ^ { 5 } x \sqrt { - g } R \, ,
d s ^ { 2 } = - \frac { d W ^ { 2 } } { \operatorname { s i n h } ^ { 2 } W \operatorname { l n } ^ { 2 / 3 } T _ { c } ^ { 2 } } + \operatorname { l n } ^ { 2 / 3 } T _ { c } ^ { 2 } \sum _ { j = 2 } ^ { 4 } d x _ { j } ^ { 2 } ,
I = \int _ { \Omega } \frac { 1 } { 2 } \left( \epsilon _ { \mu \nu \lambda } B _ { \mu } ^ { * } ( \partial _ { \nu } - i g a _ { \nu } ) B _ { \lambda } + M B _ { \mu } ^ { * } B ^ { \mu } + \epsilon _ { \mu \nu \lambda } a _ { \mu } \partial _ { \nu } a _ { \lambda } \right) \; ,
\{ \theta ^ { a } , \theta ^ { b } \} = \delta ^ { a b } \sqrt { \hbar \alpha ^ { \prime } }
\Omega ( \gamma _ { r } | A _ { + } \rangle ) = ( - 1 ) ^ { - r - \frac { 1 } { 2 } } \gamma _ { r } ( \Omega | A _ { + } \rangle ) .
S [ \vec { r } ] \ = \ \int d ^ { D } \! x \ { \frac { 1 } { 2 } } ( \nabla _ { \! x } \vec { r } ) ^ { 2 } \ + \b \ \int d ^ { D } \! x \int d ^ { D } \! y \ \delta ^ { d } ( \vec { r } ( x ) - \vec { r } ( y ) ) \ .
\rho = \rho _ { 0 } ~ R ^ { \, \gamma ( 1 - D ) } ~ ~ .
\Pi _ { \sigma } ( p )
\langle \! \langle \psi _ { i } | \psi _ { j } \rangle = \langle \psi _ { i } ^ { * } | \psi _ { j } \rangle \ ,
P ^ { - } = { \frac { m ^ { 2 } } { 4 \pi } } \int _ { 0 } ^ { \infty } { \frac { d k } { k ^ { 2 } } } \left[ b ^ { \dag } ( k ) b ( k ) + d ^ { \dag } ( k ) d ( k ) \right] + { \frac { 1 } { 2 } } \int _ { 0 } ^ { \infty } { \frac { d p } { p ^ { 2 } } } a ^ { \dag } ( p ) a ( p ) \ .
\tau = \frac { \lambda } { 2 } ( T _ { 0 } ^ { \prime } - T _ { 0 } )
\left( S _ { n } ( \rho a ) - S _ { n } ^ { ' } ( \rho a ) \right) \left( S _ { n } ( - \rho R ) - S _ { n } ^ { ' } ( - \rho R ) \right) .
\begin{array} { l l } { { \cal L } _ { K } = } & { \frac { 1 } { 4 } F _ { \mu \nu } F ^ { \mu \nu } - ( 1 - g ^ { 2 } \zeta \eta ^ { 2 } ) \tilde { G } _ { \mu } \tilde { G } ^ { \mu } + 2 ( m + g h \zeta \eta ^ { 2 } ) A _ { \mu } \tilde { G } ^ { \mu } + } \\ \end{array}
T _ { m } ( x ) \equiv \int _ { 0 } ^ { x } d x ^ { \prime } ( t a n h \; x ^ { \prime } ) ^ { m } = - \frac { ( t a n h \; x ) ^ { m - 1 } } { m - 1 } + T _ { m - 2 } ( x ) \; .
\frac { \mathrm { d } ^ { 2 } \mu } { \mathrm { d } x ^ { 2 } } + \biggl ( \frac { x ^ { 2 } } { 4 } - a \biggr ) \mu = 0 , \quad a \equiv - \frac { \pi } { \epsilon \sqrt { b _ { 1 } } } < 0 .
\nu = \frac { g \beta } { 2 \pi } \int E ( x ) d x = - \frac { g } { 2 \pi T } [ A _ { 0 } ( \infty ) - A _ { 0 } ( - \infty ) ] \ = ~ 1
\sum _ { s = 0 } ^ { n - l - 1 } s _ { s } s _ { 2 n - l - m - s } + s _ { n - l } s _ { n - m } +
( D _ { 1 } + i D _ { 2 } ) { \psi } _ { + } ( \vec { x } ) = 0 .
\operatorname { s i n } ^ { 2 } ( \pi k / N ) \sum _ { a = 1 } ^ { K } \, I _ { a b } \, r _ { a , k } = 0
D _ { I } = \sum _ { a = 1 } ^ { N } q _ { I } ^ { a } \left( | h _ { a } | ^ { 2 } - | \tilde { h } _ { a } | ^ { 2 } \right) ,
\xi = \theta ^ { \alpha } \lambda _ { \alpha } \ .
j _ { \alpha } ^ { 0 } = \left( i \Gamma ^ { m } \theta \right) _ { \alpha } \cdot \Lambda _ { m } + \Lambda _ { \alpha } + \left( e m b ^ { * } \Delta _ { \alpha } \right) _ { \nu } \cdot \Lambda ^ { \nu } \quad ;
- \frac { R ( P ) } { 2 } | \Phi ( P ) | ^ { 2 } - \frac { 1 } { 2 } | \Phi ( P ) | ^ { 4 } \ge 0
a _ { H } ^ { i j } ( x ) - \frac { 1 } { 3 } \delta ^ { i j } \delta ^ { k l } a _ { H } ^ { k l } ( x ) = U ( x ^ { o } ) a ^ { i j } ( x ) U ^ { - 1 } ( x ^ { o } ) ,
{ \frac { d n _ { i } ( \omega ) } { d \omega } } = { \frac { \omega } { \pi } } \int _ { - \infty } ^ { + \infty } d \alpha ~ e ^ { i \alpha \omega ^ { 2 } } \mathrm { T r } ~ e ^ { - i \alpha \bar { H } _ { i } ^ { 2 } } ~ ~ ~ .
V _ { 1 } ( t ) = \operatorname { e x p } \{ i { \frac { e } { \hbar c } } \int \int d ^ { 2 } x d ^ { 2 } x ^ { \prime } \sqrt { g ( x ) } J _ { 0 } ( x ) G ( \vec { x } , \vec { x } ^ { \prime } ) \partial _ { i } ^ { \prime } a _ { i } ^ { ( 1 ) } ( x ^ { \prime } ) \} .
A _ { 4 } \sim g _ { s } ^ { 2 } \frac { R _ { s } } { N _ { 1 } N _ { 2 } } \frac { s ^ { 2 } } { t }
G ^ { ( \rho ) } ( x , y ) : = \sum _ { n } \overline { { P _ { n } ^ { ( \rho ) } ( x ) } } P ^ { ( \rho ) n } ( y ) .
\mu = 1 / 2 + \nu , \mathrm { f o r ~ t y p e ~ I ~ b o u n d a r y ~ c o n d i t i o n s }
\cdot \left( 1 + \frac { 2 l ^ { 2 } \dot { l } ^ { 2 } H _ { + } ^ { 2 } } { \Delta _ { + } ^ { 2 } } + \frac { 1 } { \Delta _ { + } } \left( \dot { l } ^ { 2 } + l ^ { 2 } H _ { + } ^ { 2 } - \frac { 2 l \dot { l } H _ { + } } { \Delta _ { + } } \sqrt { l ^ { 2 } \dot { l } ^ { 2 } H _ { + } ^ { 2 } + \Delta _ { + } ^ { 2 } + \Delta _ { + } ( \dot { l } ^ { 2 } + l ^ { 2 } H _ { + } ^ { 2 } ) } \right) \right) \Bigr \}
W [ \varphi , \rho ] = W _ { + } ^ { ( 1 ) } [ \varphi ] + W _ { - } ^ { ( 1 ) } [ \rho ] + { \frac { 1 } { 2 \pi } } \int d ^ { 2 } x \, B _ { + } \, B _ { - }
e ^ { 2 x } C ( x ) = \frac { B ( x ) \lambda _ { 1 } } { 2 } + \frac { C ( x ) \lambda _ { 1 } } { 4 } + \lambda _ { 1 } ,
V = - g D ^ { 2 } \Gamma _ { 4 } + H D ^ { 2 } \Gamma _ { 4 } .

( { } ^ { \mu _ { - } } \omega ) _ { + } = 0 , \qquad ( { } ^ { \mu _ { + } } \omega ) _ { - } = 0 .
: e ^ { i \hat { \varphi } ( \sigma ) } : \, = e ^ { i \hat { \chi } ^ { \dagger } ( \sigma ) } \, e ^ { i \hat { \chi } ( \sigma ) } .
I _ { 1 1 } = \int d ^ { 1 1 } x \sqrt { - g } ~ ( { \frac { R } { \kappa ^ { 2 } } } - 3 \, \hat { F } _ { [ 4 ] } ^ { 2 } ) + { 2 8 8 } \sigma \int \hat { F } _ { [ 4 ] } \wedge \hat { F } _ { [ 4 ] } \wedge \hat { A } _ { [ 3 ] }
P _ { \lambda m } ( x , y ) = \left\{ \begin{array} { l l } { J _ { m } ( y ) Y _ { m } ( x ) - Y _ { m } ( y ) J _ { m } ( x ) , } & { \lambda = 1 } \\ { J _ { m } ( y ) Y _ { m } ^ { \prime } ( x ) - Y _ { m } ( y ) J _ { m } ^ { \prime } ( x ) , } & { \lambda = 0 } \\ \end{array} \right.

n ^ { \mu } \nabla _ { \mu } \phi = 0 \, ,
h = \Pi ^ { \mu } ( C \Gamma _ { \mu } \Gamma _ { 1 1 } ) _ { \alpha \beta } \Pi ^ { \alpha } \Pi ^ { \beta } { \cal F } ^ { \frac { p - 1 } { 2 } } \quad , \quad p \geq 1 \quad .
\delta \psi ^ { i } = \epsilon _ { \alpha } \; b _ { \alpha } ^ { i } \; , \; \; \; \; \delta b _ { \alpha } ^ { i } = \epsilon _ { \gamma } \; ( \phi _ { [ \gamma \alpha ] } ^ { i } + i \delta _ { \gamma \alpha } \; \partial _ { - } \psi ^ { i } ) \; .
\left| < i _ { l + 1 } ^ { 1 } , k _ { l + 1 } ^ { 1 } , i _ { l + 1 } ^ { 2 } , k _ { l + 1 } ^ { 2 } | e ^ { - \frac { \varepsilon } { 2 } H _ { 1 , 2 } } | i _ { l } ^ { 1 } , k _ { l } ^ { 1 } , i _ { l } ^ { 2 } , k _ { l } ^ { 2 } > \right| .
\frac { \overline { { b } } } { \overline { { d } } } = \frac { 2 \overline { { B } } _ { 2 } } { 3 \overline { { B } } _ { 1 } }
O _ { K } = \left( \frac 1 8 \epsilon _ { \mu \nu \kappa \tau } G _ { \mu \nu } ^ { a } G _ { \kappa \tau } ^ { a } \right) ^ { 2 }
\left[ \frac { 1 } { \omega } ( A _ { N - 6 } b ^ { 6 } ) ^ { 5 / 8 } \right] ^ { 6 } = \frac { ( A _ { N - 6 } b ^ { 6 } ) ^ { 6 } } { b ^ { 4 N } } \quad \Rightarrow \quad b ^ { 4 N } = 1 .
{ \frac { m ! } { ( 2 m - 1 ) ! ! } } \left\{ - 1 / 2 \partial _ { t } ^ { 2 } + { \cal Z } ^ { F _ { m - 2 } } ( t ) + \partial _ { t } ^ { - 1 } { \cal Z } ^ { F _ { m - 2 } } ( t ) \partial _ { t } \right\} ^ { m } \cdot 1 = t ,
\dot { \Psi } ^ { R } = i [ H ^ { R } , \Psi ^ { R } ] = \gamma _ { 0 } ( - i \gamma _ { i } \partial _ { i } - e \gamma _ { i } V _ { i } ^ { R } + m ) \Psi ^ { R } + { \frac { e } { 2 } } \left\{ \Psi ^ { R } , { \frac { 1 } { \vec { \partial } ^ { 2 } - M ^ { 2 } } } J _ { 0 } \right\} ~ ,
( l e f t , \ r i g h t ) = ( N S + , \ N S + ) \oplus ( R + , \ R + ) .
\phi ^ { \prime \prime } - \rho ^ { \prime \prime } - \frac { 1 } { 2 } ( T ^ { \prime } ) ^ { 2 } = 0
\lambda ( R , \Pi ) = R ( \sum _ { s = 1 } ^ { N } | \Pi ^ { s } | ^ { 2 } + R ^ { 2 } ) ^ { - 1 / 2 } .
\gamma _ { L } = \left( 1 - L ^ { - 2 } \right) \; c _ { \infty } , \quad \delta _ { L } = L ^ { - 1 } .
A _ { 1 } = - \frac { i \sqrt { 2 } G \hbar D _ { 1 } \operatorname { s i n } ( \frac { 1 } { \hbar } p _ { 3 } ^ { ( 1 ) } L ) } { ( E + p _ { 3 } ^ { ( 1 ) } ) \left[ e ^ { - \frac { i } { \hbar } p _ { 3 } ^ { ( 1 ) } L } - \frac { ( G \hbar D _ { 1 } ) ^ { 2 } } { ( E + p _ { 3 } ^ { ( 1 ) } ) ^ { 2 } } e ^ { \frac { i } { \hbar } p _ { 3 } ^ { ( 1 ) } L } \right] }
Y ^ { - } ( t , \sigma ^ { a } ) = { \frac { R } { N } } E t + \xi ( t , \sigma ^ { a } )
F ( z , x ) = \operatorname { e x p } - \int _ { a } ^ { b } \frac { d y } { 2 \pi i } \frac { 1 } { y - x } \operatorname { l o g } \bigg [ \frac { z - u ( y ) } { z - \bar { u } ( y ) } \bigg ] =
G _ { _ C } ( x , y ; A ) = \operatorname { e x p } \left[ - i e \int \! \! d z \; A _ { \mu } ( z ) j _ { + } ^ { \mu } ( z , x , y ) \right] P _ { + } G _ { F } ( x - y ) + P _ { - } G _ { F } ( x - y )
\left[ \widetilde R _ { 1 N } ^ { ( 2 ) } ( x ) \right] _ { \mathrm { ( I I I ) } } \simeq \beta ^ { N } \simeq \operatorname { e x p } \left( - { \frac { N ^ { 1 / 3 } } { \alpha _ { c } ^ { 2 } g ^ { 2 / 3 } } } \right) \rightarrow 0 .
\left[ \mathrm { V } _ { H _ { 1 } } ( \phi _ { i } ) , \mathrm { V } _ { H _ { 2 } } ( \phi _ { j } ) \right] = \delta _ { i j } \mathrm { V } _ { { \{ H _ { 1 } , H _ { 2 } \} } _ { \mathrm { p b } } } ( \phi _ { i } )
l ( 0 ) = r ( 0 ) = 0 \, , \qquad l ^ { \prime } ( \pi / 2 ) = r ^ { \prime } ( \pi / 2 ) = 0 \, .
\varepsilon _ { i j } D _ { i } \phi ^ { a } = \pm \varepsilon ^ { a b c } D _ { j } \phi ^ { b } \phi ^ { c } ,
\Omega ( a _ { 0 } , a _ { 1 } ; b _ { 0 } , b _ { 1 } ) = \left\langle { \frac { \mathrm { D e t } ( 1 - a _ { 1 } U ) \, \mathrm { D e t } ( 1 - b _ { 1 } \bar { U } ) } { \mathrm { D e t } ( 1 - a _ { 0 } U ) \, \mathrm { D e t } ( 1 - b _ { 0 } \bar { U } ) } } \right\rangle \; .
C ( r _ { A } , r _ { + } ) = \alpha _ { 4 } ( \alpha _ { 4 } + 2 ) P ^ { 1 3 } ( P ^ { 2 3 } - P ^ { 1 2 } )
S _ { R S T } = - \frac { \kappa } { \pi } \int d ^ { 2 } x \; \phi \partial _ { + } \partial _ { - } \rho
\alpha _ { A } ( \varphi ) = - \left[ \operatorname { l n } { \frac { \widehat \Lambda _ { s } } { m _ { A } } } + { \frac { 1 } { 2 } } \right] { \frac { \partial \operatorname { l n } B ( \varphi ) } { \partial \varphi } } = + \operatorname { l n } { \frac { \widehat \Lambda _ { s } ^ { \prime } } { m _ { A } } } \, { \frac { \partial \operatorname { l n } B ^ { - 1 } ( \varphi ) } { \partial \varphi } } \ ,
\sum _ { m = 1 } ^ { \infty } \left\langle \frac { P _ { n } ( x _ { 1 } ) \tilde { Q } _ { m } ( y _ { 1 } ) } { x _ { 1 } - \phi _ { a } } \right\rangle \left\langle \frac { P _ { m } ( x _ { 2 } ) \tilde { Q } _ { n } ( y _ { 2 } ) } { y _ { 2 } - \chi _ { b } } \right\rangle = \left\langle \frac { P _ { n } ( x _ { 1 } ) \tilde { Q } _ { n } ( y _ { 1 } ) } { ( x _ { 1 } - \phi _ { a } ) ( y _ { 1 } - \chi _ { b } ) } \right\rangle
+ e _ { 2 } \overline { { \Phi } } ( z , t _ { 2 } | y , t _ { 1 } ) ( { \gamma } ^ { 0 } \otimes { \gamma } ^ { 0 } ) { \Phi } ( z , t _ { 2 } | y , t _ { 1 } ) \} ,
V ( r ) \sim \left\{ \begin{array} { c l } { - 2 k ^ { 2 } / r ^ { 6 } } & { \textrm { s c a l a r p e r t u r b a t i o n } } \\ { + 4 k ^ { 2 } / r ^ { 6 } } & { \textrm { e l e c t r o m a g n e t i c - g r a v i t a t i o n a l p e r t u r b a t i o n } } \\ \end{array} \right.
\, \, \, \, \partial ^ { \mu } A _ { \mu } \, = \, 0 \, \, \, \, \, ; \, \, \, F _ { \mu \nu } \, + \, { \tilde { F } } _ { \mu \nu } \, = \, 0 \, \, \, \, \, ; \, \, \, \partial ^ { \mu } \Psi _ { \mu } \, = \, 0
F = F ( u _ { 1 } + u _ { 3 } u _ { 2 } , u _ { 3 } ) , \quad \alpha ( u _ { 1 } + u _ { 3 } u _ { 2 } , \lambda )
{ \cal L } _ { \mathrm { e f f } } = T \, ( \partial _ { \mu } y ( x ) ) ( \partial ^ { \mu } y ( x ) ) \, .
( \gamma ^ { 0 } + \epsilon ) \xi = ( \gamma ^ { 0 } - \epsilon ) \eta = 0 .
\rho _ { \nu } ( x ) = \rho _ { \nu ^ { \prime } } ( x ) \frac { q ^ { C ( \nu ) / 2 } + \epsilon ( \nu ) \epsilon ( \nu ^ { \prime } ) x q ^ { C ( \nu ^ { \prime } ) / 2 } } { x q ^ { C ( \nu ) / 2 } + \epsilon ( \nu ) \epsilon ( \nu ^ { \prime } ) q ^ { C ( \nu ^ { \prime } ) / 2 } } \; , ~ ~ ~ ~ \forall \nu \neq \nu ^ { \prime } .
A _ { \mu } ^ { a } \frac { \sigma ^ { a } } { 2 } = \bar { \Sigma } _ { \mu \nu } J _ { \nu } [ \Phi ]
B _ { a b } ( \tau _ { 1 } , \tau _ { 2 } ) = \Bigl [ \delta ( \tau _ { 1 } - \tau _ { a } ) - \delta ( \tau _ { 1 } - \tau _ { b } ) \Bigr ] \Bigl [ \delta ( \tau _ { 2 } - \tau _ { a } ) - \delta ( \tau _ { 2 } - \tau _ { b } ) \Bigr ] \quad .
e _ { f } ^ { \mu } = \left( { \frac { 1 } { 2 } } \left( e ^ { \mu [ - 2 ] } + e ^ { \mu [ + 2 ] } \right) , { \frac { 1 } { 2 } } \left( e ^ { \mu [ - 2 ] } - e ^ { \mu [ + 2 ] } \right) \right)
G ( z \vert \tau ) = \operatorname { l o g } \chi \ , \ \ \chi = 2 \pi e ^ { - \pi y ^ { 2 } / \tau _ { 2 } } \Bigl \vert { \frac { \theta _ { 1 } ( z \vert \tau ) } { \theta _ { 1 } ^ { \prime } ( 0 \vert \tau ) } } \Bigr \vert \ \ ,
\langle { \bf \Psi } , { \bf \Xi } \rangle = \frac 1 2 \int _ { - \infty } ^ { \infty } [ \psi ^ { * } ( x ) \operatorname { t a n } ( - i \partial _ { x } ) \xi ( x ) - \xi ( x ) \operatorname { t a n } ( - i \partial _ { x } ) \psi ^ { * } ( x ) ] \; d x
\triangle _ { \alpha \beta } ^ { a b } ( x , y ) + \int d ^ { 3 } w ~ d ^ { 3 } z ~ X _ { \alpha \gamma } ^ { a c } ( x , w ) \omega _ { c d } ^ { \gamma \delta } ( w , z ) X _ { \beta \delta } ^ { b d } ( y , z ) = 0 .
d s _ { D W } ^ { 2 } = H ( z ) ^ { - \frac { 2 \alpha } { \epsilon } } \left( d x ^ { \mu } \otimes d x ^ { \nu } \eta _ { \mu \nu } \right) + H ( z ) ^ { - \frac { 2 \beta + \epsilon } { \epsilon } - 2 } \, \frac { d z ^ { 2 } } { \epsilon ^ { 2 } }
1 6 \pi { \cal L } = - e ^ { - { \cal K } } R + \dots
\dot { \operatorname { l n } { \frac { [ i ] } { [ i - 1 ] } } } + ( \operatorname { l n } ( [ i ] [ i - 1 ] ) ^ { \prime \prime } + ( ( \operatorname { l n } { \frac { [ i ] } { [ i - 1 ] } } ^ { \prime } ) ^ { 2 } = 2 ( \bar { \alpha } _ { i - 1 , i } + \bar { \alpha } _ { i , i - 1 } - \bar { \alpha } _ { i - 1 } \bar { \alpha } _ { i } )
\int \Phi _ { \Xi } ^ { \dagger } ( \xi , \xi ^ { * } ) \Phi _ { H } ( \xi , \xi ^ { * } ) d v ( \xi , \xi ^ { * } ) = \left( \Xi \mid H \right)
d s _ { 1 0 } ^ { 2 } = h ^ { - 1 / 2 } ( \rho ) d x ^ { \mu } d x ^ { \mu } + h ^ { 1 / 2 } ( \rho ) d s _ { 6 } ^ { 2 } ,
\epsilon ^ { \delta \gamma \alpha _ { 1 } \beta _ { 1 } } D _ { \gamma } ^ { i } \; { \cal O } _ { \alpha _ { 1 } \ldots \alpha _ { J _ { 2 } } \; \beta _ { 1 } \ldots \beta _ { J _ { 2 } } } = 0 \, .
\{ X ^ { M } \} = \{ x ^ { \mu } , y ^ { i } \} , \quad D _ { i } ( X ) \equiv y ^ { i } .
T _ { n } = \frac { m ^ { 2 } } { 2 } \sum _ { k = 1 } ^ { \infty } \frac { a _ { n } ^ { \dagger } ( k ) a _ { n } ( k ) } { k - \frac { 1 } { 2 } }
\sum _ { n = 1 / 2 , \cdots } { \frac { \Delta p ^ { + } } { p _ { n } ^ { + } } } = \sum _ { n = n _ { \mathrm { I R } } + 1 } ^ { n _ { \mathrm { U V } } - 1 } \frac { 1 } { n } \simeq \mathrm { l n } \frac { n _ { \mathrm { U V } } } { n _ { \mathrm { I R } } } \simeq \mathrm { l n } \frac { 2 \Lambda ^ { 2 } } { M ^ { 2 } + { \bf p } _ { \bot } ^ { 2 } } ,
\Gamma ( s + 2 n - 1 ) \zeta _ { R } ( s ) \zeta ( \frac { s + 2 n - 1 } { 2 } | \L _ { 3 } ) \; \beta ^ { - s } \: , \nonumber
\left\{ \begin{array} { c c c } { - \lambda ^ { 2 } x ^ { + } \left( x ^ { - } + \Delta \right) } & { = } & { { \frac { \kappa } { 4 } } } \\ { - \lambda ^ { 2 } x ^ { + } x ^ { - } } & { = } & { { \frac { \kappa } { 4 } } \, e ^ { \frac { 4 M } { \kappa \lambda } } \; . } \\ \end{array} \right.
: \psi _ { r , a } ^ { * } \psi _ { s , b } : = \psi _ { r , a } ^ { * } \psi _ { s b } - \langle 0 | \psi _ { r , a } ^ { * } \psi _ { s b } | 0 \rangle ,
{ \cal M } ( \theta ) = \left( \begin{array} { c c } { \operatorname { c o s } \theta } & { \operatorname { s i n } \theta } \\ { - \operatorname { s i n } \theta } & { \operatorname { c o s } \theta } \\ \end{array} \right)
d s ^ { 2 } = ( 1 - \frac { N ^ { 4 } } { r ^ { 4 } } ) ( \frac { r } { 8 N } ) ^ { 2 } ( d \tau + 4 N \operatorname { c o s } \theta d \phi ) ^ { 2 } + ( 1 - \frac { N ^ { 4 } } { r ^ { 4 } } ) ^ { - 1 } d r ^ { 2 } + \frac { 1 } { 4 } r ^ { 2 } d \Omega ^ { 2 } .
x _ { 1 } = \frac { 1 } { 2 } \left( x _ { + } + x _ { - } \right) , \quad x _ { 2 } = \frac { \imath } { 2 } \left( x _ { - } - x _ { + } \right) , \quad x _ { 3 } = \frac { 1 } { 2 } h ,
\phi ( \sigma ) = U \, e ^ { i \, ( N \sigma + \varphi ( \sigma ) ) } ,
E _ { | 2 } \left( m , \lambda \right) = 2 \frac V { 2 \pi ^ { 2 } } \sum _ { l = 0 } ^ { \infty } \sum _ { i = 1 } ^ { 2 } \int _ { 0 } ^ { \infty } d p p ^ { 2 } \sqrt { E _ { i } ^ { 2 } \left( p , m , l \right) } ,
I ( d , a , c ) = \int _ { 0 } ^ { \infty } { \frac { x \; d x } { ( x ^ { 2 } + d ^ { 2 } ) + a ( x ^ { 2 } + d ^ { 2 } ) ^ { 1 / 2 } } } { \frac { 1 } { x ^ { 2 } + c ^ { 2 } } }
\triangle ^ { s y m } ( t ) = - t ^ { * } \times 1 + 1 \times t ,
R _ { k } ^ { ( N _ { f } ) } ( x _ { 1 } , \ldots , x _ { k } , M ) \ = \ \operatorname* { d e t } \left[ K _ { N } ^ { ( N _ { f } ) } ( x _ { i } , x _ { j } , M ) \right] _ { i , j = 1 , \ldots , k }
\Omega _ { 3 } = { \frac { 1 } { 3 R ^ { 2 } } } \varepsilon _ { \hat { m } \hat { n } \hat { p } \hat { q } } X ^ { \hat { m } } d X ^ { \hat { n } } d X ^ { \hat { p } } d X ^ { \hat { q } } \, , \qquad \tilde { \Omega } _ { 3 } = { \frac { 1 } { 3 R ^ { 2 } } } \varepsilon _ { \hat { m } ^ { \prime } \hat { n } ^ { \prime } \hat { p } ^ { \prime } \hat { q } ^ { \prime } } X ^ { \hat { m } ^ { \prime } } d X ^ { \hat { n } ^ { \prime } } d X ^ { \hat { p } ^ { \prime } } d X ^ { \hat { q } ^ { \prime } } \, ,
x = \beta | \stackrel { \rightharpoonup } { l } | , \ y = \beta m \ \mathrm { a n d } \ r = \beta \mu .
\tilde { \Psi } _ { \nu } ^ { \prime } = - i \bigl ( \nu + { \bf M } \delta ( \sigma ) \bigr ) \tilde { \Psi } _ { \nu } .
E = { \frac { 1 } { n ! } } \int d \Omega \left[ E _ { 1 } \left| Y _ { a _ { 1 } \cdots a _ { n } } ^ { ( l ) } \right| ^ { 2 } + E _ { 2 } \left| Y _ { a _ { 2 } \cdots a _ { n } } ^ { ( l ) } \right| ^ { 2 } + E _ { 3 } \left| Y _ { a _ { 3 } \cdots a _ { n } } ^ { ( l ) } \right| ^ { 2 } \right] .
{ H _ { \bar { \partial } } ^ { p , p } } ( { { \mathcal P } ^ { n } } ) \cong { H _ { D R } ^ { 2 p } } \cong C
R _ { i j k l } ^ { ( + ) } = \frac { 1 } { 2 } g _ { i m } g _ { j n } \phi ^ { m n p q } R _ { p q k l } ^ { ( + ) } .
- i A ^ { i j } E ^ { i j } = S q r t { 2 } ( A _ { L } ^ { l } \alpha _ { L } ^ { l } + A _ { R } ^ { r } \alpha _ { R } ^ { r } )
M _ { 1 1 } ^ { \beta \gamma } = \frac { T ^ { \beta \gamma } } { 1 - C I ^ { \beta \gamma \, - 1 } T ^ { \beta \gamma } + T ^ { \beta \gamma \, 2 } } = \frac { ( C I ^ { \beta \gamma } ) ^ { 5 } - C I ^ { \beta \gamma } } { ( 1 - ( C I ^ { \beta \gamma } ) ^ { 2 } ) ( 1 - 3 ( C I ^ { \beta \gamma } ) ^ { 2 } ) }
V ( \varphi _ { 0 } ) = 4 \sum _ { s = 1 } ^ { \infty } \int \frac { d ^ { 4 } p } { ( 2 \pi ) ^ { 4 } } \frac { ( - 1 ) ^ { s } } { 2 s } \frac { ( M + g \varphi _ { 0 } ) ^ { 2 s } } { ( p _ { E } ^ { 2 } ) ^ { s } } .
F ( \phi ) = \int \frac { d ^ { D } p } { ( 2 \pi ) ^ { D } } \frac { 1 } { p ^ { 2 } + M ^ { 2 } ( \phi ) } ,
\delta \Psi _ { \hat { a } } = \partial _ { \hat { a } } \epsilon + { \frac { 1 } { 8 } } H ^ { - 1 } ( \partial _ { \hat { a } } H ) \epsilon + { \frac { 1 } { 8 } } W \gamma ^ { \underline { { u x y } } } \gamma _ { \underline { { \hat { a } } } } ( i \sigma _ { 2 } ) \epsilon = 0 \,
I _ { G } = \int \sum _ { p = 0 } ^ { [ d / 2 ] } \alpha _ { p } L ^ { ( p ) } ,
\Pi _ { \mu \nu } ^ { ( I ) } = { \frac { e ^ { 2 } C ^ { ( I ) } } { 2 } } \, \int { \frac { d ^ { n } k } { ( 2 \pi ) ^ { n } } } \, { \frac { ( 1 - \operatorname { c o s } p \times k ) } { k ^ { 2 } ( k + p ) ^ { 2 } } } \, N _ { \mu \nu } ^ { ( I ) } ; \; \; \; \; \; \; \; \; \; \; \; I = a , b , c
\Big \langle \frac { 1 } { 4 } \int d ^ { 4 } x \, d ^ { 4 } \theta \, \Big ( \phi ^ { * } e ^ { 2 V } \phi + \tilde { \phi } ^ { * } e ^ { - 2 V } \tilde { \phi } \Big ) \Big \rangle = 0 .
{ \cal L } = \Psi = P \{ \mu \} \ .
q _ { ( 1 ) } = q _ { i } \delta _ { i } ^ { j } \qquad \bar { q } _ { ( 1 ) } = \bar { q } _ { i } \delta _ { i } ^ { j } \qquad T _ { ( 1 ) } = T _ { i } \delta _ { i } ^ { j }
Y = 2 [ ( x _ { 1 2 } x _ { 3 4 } ) - 2 \frac { ( x _ { 1 2 } x _ { 2 4 } ) ( x _ { 3 4 } x _ { 2 4 } ) } { x _ { 2 4 } ^ { 2 } } ] \frac { 1 } { x _ { 2 4 } ^ { 2 } } ( 1 + \mathcal { O } ( x _ { 1 2 } , x _ { 3 4 } ) )
{ \cal S } _ { E } = \int _ { 0 } ^ { \tau } d t \int _ { 0 } ^ { \pi } d \sigma \left( \frac { \rho } { 2 } \left( \frac { \partial X } { \partial t } \right) ^ { 2 } + \frac { \nu } { 2 } \left( \frac { \partial X } { \partial \sigma } \right) ^ { 2 } \right) .
\delta B _ { \hat { 0 } \alpha } ^ { g } = \frac { 1 } { \sqrt { a } } { \bar { e } } _ { \alpha } ^ { \; \; i } \left( \delta { \beta } _ { o i } ^ { g } - v _ { i } \delta \beta ^ { a } \right) .
\begin{array} { r l } { \hat { U } ^ { - 1 } ( k _ { 1 } , k _ { 2 } ) } & { | A _ { \bar { z } } ( z , \bar { z } ) > = } \\ { = } & { e ^ { i \pi k k _ { 1 } k _ { 2 } } \, e ^ { - \frac { 1 } { 2 } i k \left[ ( k _ { 1 } \tau - k _ { 2 } ) \overline { { A _ { \bar { z } } ^ { + + } ( 0 , 0 ) } } + ( k _ { 1 } \bar { \tau } - k _ { 2 } ) A _ { \bar { z } } ^ { + + } ( 0 , 0 ) \right] } \| A _ { \bar { z } } ( z , \bar { z } ) - \frac { i \pi } { \tau _ { 2 } } ( \tau k _ { 1 } - k _ { 2 } ) > \ \ \ , } \\ \end{array}
\chi [ { \cal M } _ { \alpha } ] = { \frac { 1 } { 3 2 \pi ^ { 2 } } } \int _ { { \cal M } _ { \alpha } / \Sigma } ( R ^ { 2 } - 4 R _ { \mu \nu } ^ { 2 } + R _ { \mu \nu \alpha \beta } ^ { 2 } ) + ( 1 - \alpha ) \chi [ \Sigma ] ~ ~ ~ ,
S = \frac { - 8 } { \kappa ^ { 2 } } \int d t d ^ { 2 } \theta d ^ { 2 } \overline { { \theta } } { I \! \! N } ^ { - 1 } A ( { I \! \! N } { \bf Q } ^ { \mu } ) ,
H _ { Q ( t , z ) } ^ { S , l } \cong H _ { Q ( t ) } ^ { S } \otimes C [ [ z ] ] \cong \hat { H } _ { Q ( t , z ) } ^ { S } ,
n ( \omega ) = { \frac { 1 + e ^ { - \alpha ^ { 2 } \omega ^ { 2 } } } { 1 - e ^ { - \alpha ^ { 2 } \omega ^ { 2 } } } } .
Q = \frac { 1 } { C } \int _ { T _ { 2 } } W ^ { * } \omega ,
a + b - 1 + 2 \sum _ { j = 1 } ^ { r - 2 } j \, n _ { j } = L + 1 .
\hat { R } _ { 1 2 } ( q ^ { - 1 } ) K _ { 1 } ^ { \prime } \hat { R } _ { 1 2 } ( q ^ { - 1 } ) K _ { 1 } ^ { \prime } = K _ { 1 } ^ { \prime } \hat { R } _ { 1 2 } ( q ^ { - 1 } ) K _ { 1 } ^ { \prime } \hat { R } _ { 1 2 } ( q ^ { - 1 } ) \quad ,
\psi = \sqrt \rho _ { 0 } e ^ { i n \theta } , \quad A _ { 0 } = 0 , \quad { \bf A } = \frac { n } { e } \frac { { \bf e } _ { \theta } } { r }
T ^ { M } \cdot T ^ { N } - ( M \leftrightarrow N ) = f ^ { M N K } T ^ { K }
Q _ { U } = \int _ { U } j _ { p } = \int _ { U } d ( d C _ { p } ) = \int _ { \partial U } d C _ { p } = \int _ { \partial U } G _ { p + 1 } - H \wedge C _ { p - 2 } .
( \partial p ) ^ { * } \partial \Theta , \partial g
\beta F _ { \mathrm { b o s e } } = 9 N ^ { 2 } \operatorname { l o g } [ 2 \operatorname { s i n h } ( 2 \beta R _ { \mathrm { r m s } } ) ]
S _ { \mathrm { m a s s i v e \ I I A } } ( C ^ { ( 9 ) } ) \sim \int d ^ { 1 0 } x { \sqrt { | g | } } \left\{ e ^ { - 2 \phi } \left[ R - 4 ( \partial \phi ) ^ { 2 } \right] - \frac { 1 } { 2 \times 1 0 ! } ( G ^ { ( 1 0 ) } ) ^ { 2 } + \cdots \right\} \, .
m , n = 1 \ldots d ; \; \; \; \; \; \; a , b , c = 1 \ldots g ; \; \; \; \; \; i , j , k = 1 \ldots g \; \; \; \; \; \; ( g \leq d )
[ \hat { \eta } _ { m } ^ { ( i ) \mu } , \hat { \eta } _ { n } ^ { \dagger ( j ) \nu } ] = G ^ { \mu \nu } \delta ^ { i , j } \delta _ { m , n } ~ .
\Phi ^ { 2 } = 2 ( { \frac { d - 3 } { d - 2 } } ) q \Phi _ { h } \phi ~ ~ , ~ ~ \gamma _ { a b } = ( { \frac { \phi _ { h } } { \phi } } ) ^ { \frac { d - 3 } { d - 2 } } e ^ { { \frac { 2 } { q \Phi _ { h } } } \phi } \bar { \gamma } _ { a b } ~ ~
C ( \theta ) = \left\{ \begin{array} { l } { \operatorname { c o t } { \theta } \hspace { 0 . 5 c m } \mathrm { f o r } \hspace { 0 . 3 c m } k = + 1 , } \\ { 0 \hspace { 0 . 5 c m } \mathrm { f o r } \hspace { 0 . 3 c m } k = 0 , } \\ { \operatorname { c o t h } { \theta } \hspace { 0 . 5 c m } \mathrm { f o r } \hspace { 0 . 3 c m } k = - 1 , } \\ \end{array} \right. \
\eta ^ { l m } \partial _ { i } \partial _ { j } \partial _ { m } F _ { M } \partial _ { l } \partial _ { k } \partial _ { n } F _ { M } = \eta ^ { l m } \partial _ { j } \partial _ { k } \partial _ { m } F _ { M } \partial _ { i } \partial _ { l } \partial _ { n } F _ { M }
d s _ { n + 3 } ^ { 2 } = e ^ { - 2 n L \rho } d x ^ { 2 } + e ^ { 4 L \rho } ( 2 d x d t + \sum _ { i = 1 } ^ { n } { d y ^ { i } } ^ { 2 } ) + d \rho ^ { 2 }
J _ { 5 } ^ { \mu } = - \frac { e ^ { 2 } } { 8 \pi ^ { 2 } } \epsilon ^ { \mu \nu \lambda \sigma } A _ { \nu } F _ { \lambda \sigma }
\hat { D } ( \hat { u } ) = \lambda ^ { - 1 } \hat { Q } ( \hat { u } )
h ( r ) = \eta \left( 1 - \sqrt { \frac { \lambda _ { 2 } } { 2 \lambda _ { 1 } } } \eta r K _ { 1 } ( 2 \sqrt { \frac { \lambda _ { 2 } } { \lambda _ { 1 } } } \eta r ) \right) ,
\partial _ { \tau } \langle x , \tau \vert x ^ { \prime } , 0 \rangle = - \langle x , \tau \vert H \vert x ^ { \prime } , 0 \rangle \; ,
{ \cal L } _ { S G } = \frac { 1 } { 2 } \partial _ { \mu } \varphi \partial _ { \mu } \varphi -
E [ f ^ { ( 0 ) } + f ^ { ( 1 ) } ] = E [ f ^ { ( 0 ) } ] + \frac { \delta E } { \delta f } [ f ^ { ( 0 ) } ] \cdot f ^ { ( 1 ) } + \frac { 1 } { 2 } \frac { \delta ^ { 2 } E } { \delta f ^ { 2 } } [ f ^ { ( 0 ) } ] \cdot ( f ^ { ( 1 ) } ) ^ { 2 } + \cdots ,
{ \cal L } _ { c l a s s } = { \cal L } _ { c l a s s } ( \phi , \alpha ) = \sum _ { i } \lambda _ { i } ( \alpha ) \, { \cal G } _ { i } .
\Psi _ { E } ^ { ( - ) } = \left( \begin{array} { l } { e ^ { i \frac { \theta } { 2 } } \sum f _ { n } e ^ { i \left( n + \frac { 1 } { 2 } \right) \theta } } \\ { e ^ { - i \frac { \theta } { 2 } } \sum g _ { n } e ^ { i \left( n + \frac { 1 } { 2 } \right) \theta } } \\ \end{array} \right)
[ \partial _ { x } , \eta ] = 0 \ \ \ \ [ \partial _ { y } , \xi ] = h ^ { \prime } \xi \partial _ { x } + h h ^ { \prime } \eta \partial _ { x } + h \eta \partial _ { y }

\Gamma _ { \underline { \alpha } \underline { \beta } } ^ { i } = \left( \begin{array} { c c } { 0 } & { \delta _ { \beta } ^ { \alpha } ( \gamma ^ { i } ) _ { q } ^ { ~ r } } \\ { - \delta _ { \alpha } ^ { \beta } ( \gamma ^ { i } ) _ { ~ r } ^ { q } } & { 0 } \\ \end{array} \right) \, , \quad i = 1 , . . . , 5 \, ,
R _ { i j } = \frac { E ( z _ { i } , z _ { j } ) E ( w _ { i } , w _ { j } ) } { E ( z _ { i } , w _ { j } ) E ( w _ { i } z _ { j } ) }
\Delta _ { \varepsilon , \overline { { \varepsilon } } } ^ { ( 1 ) } = \frac { \sigma ^ { 2 } } 2 \cdot \frac { m - i \varepsilon \widehat { p } } { 2 m } \cdot \frac { m - i \overline { { \varepsilon } } \overline { { p } } } { 2 m } \cdot \left( 1 - \sigma _ { p } ^ { 2 } \right) = \Psi _ { \varepsilon , \overline { { \varepsilon } } } \cdot \overline { { \Psi } } _ { \varepsilon , \overline { { \varepsilon } } }
\gamma _ { 5 } \Gamma _ { 5 } \gamma _ { 5 } D + D \Gamma _ { 5 } = 0
\vert j , L = M = 0 \rangle = { \frac { 1 } { \sqrt { 2 j + 1 } } } \; \sum _ { m \; = - j } ^ { j } \vert j , m , - m \rangle
c ^ { ( 0 ) } \delta ( \nu ) \equiv \operatorname* { l i m } _ { t \to 0 } { \frac { c ^ { ( 0 ) } ( \nu , t ) } { \nu ^ { n - 2 } } } \quad .
J _ { 0 } ^ { \pm } | j m > = \sqrt { m ( m \pm 1 ) - j ( j + 1 ) } \; | j m \pm 1 >
\delta m = \sqrt { \frac { j ( j + 1 ) } { d } } = \frac { \sqrt { j ( j + 1 ) } } { r } ,
\tilde { G } _ { \mu \nu } ^ { 0 A } - \tilde { G } _ { \mu \nu } ^ { 0 L } = - { \frac { 1 } { k ^ { 2 } } } k _ { \mu } k _ { \nu } \biggl ( { \frac { ( \lambda k ^ { 2 } + \eta ^ { 2 } ) } { ( \eta \cdot k ) ^ { 2 } } } + { \frac { ( 1 - \lambda ) } { k ^ { 2 } } } \biggr ) + { \frac { k _ { [ \mu } \eta _ { \nu ] _ { + } } } { k ^ { 2 } \eta \cdot k } } .
S ^ { 2 } ( p , q ) \, = \, f ( p , q ) \, = \, - ( p ^ { 0 } q _ { 0 } - p ^ { 1 } q _ { 1 } + p ^ { 2 } q _ { 2 } + p ^ { 3 } q _ { 3 } ) ^ { 2 } + 4 ( p ^ { 2 } p ^ { 3 } - p ^ { 0 } q _ { 1 } ) ( p ^ { 1 } q _ { 0 } + q _ { 2 } q _ { 3 } )
\widetilde { \Omega } _ { i } ^ { ( 1 ) } ( x ) = \int d y X _ { i j } ( x , y ) \Phi ^ { j } ( y ) .
R _ { S } ^ { 0 } ( F ^ { \otimes n } , G ) = \int d ^ { n } x ( { \mathcal R } ( x _ { \pi _ { t } ( 1 ) } ) \ldots { \mathcal R } ( x _ { \pi _ { t } ( n ) } ) G ) _ { S } \ .
Z ^ { I } = \sum _ { a = 1 } ^ { 1 6 } { \frac { q ^ { I } { } _ { a } } { | \vec { x } - \vec { x } _ { a } | } } \, \ , \qquad \sum _ { a = 1 } ^ { 1 6 } q ^ { I } { } _ { a } = 0 \ ,
\beta _ { i \bar { \jmath } } = - \frac 1 { 2 \pi } R _ { i \bar { \jmath } } - \frac { 4 \zeta ( 3 ) } { 3 ( 4 \pi ) ^ { 4 } } \partial _ { i } \partial _ { \bar { \jmath } } \left[ R _ { k \bar { l } m \bar { n } } R ^ { p \bar { l } q \bar { n } } R _ { p } { } ^ { k } { } _ { q } { } ^ { m } - R _ { k \bar { l } m \bar { n } } R ^ { m \bar { n } p \bar { q } } R _ { p \bar { q } } { } ^ { k \bar { l } } \right] + \ldots
\{ F ^ { \sigma \nu } , x ^ { \mu } \} - \{ F ^ { \sigma \mu } , x ^ { \nu } \} = \{ F ^ { \mu \nu } , x ^ { \sigma } \} .
M _ { U \ \scriptstyle \mathrm { m i n } } ^ { E _ { 8 } - E _ { 6 } } \approx 5 . 4 9 \times 1 0 ^ { 1 7 } \times g _ { U } ^ { E _ { 8 } - E _ { 6 } } \times \frac { 1 } { \sqrt { 1 + 0 . 0 8 \times \left( g _ { U } ^ { E _ { 8 } - E _ { 6 } } \right) ^ { 2 } } } \, \mathrm { G e V } \, .

\left. \begin{array} { l } { ( \gamma ) = { } ^ { g _ { 4 } } [ S _ { 4 } ] \hspace { 2 m m } { } ^ { g _ { 3 } } [ S _ { 3 } ] \hspace { 2 m m } { } ^ { g _ { 2 } } [ S _ { 2 } ] \hspace { 2 m m } { } ^ { g _ { 1 } } [ S _ { 1 } ] \hspace { 2 m m } [ \xi ] } \\ { \hspace { 6 m m } = [ \xi ] \hspace { 2 m m } { } ^ { h _ { 4 } } [ S _ { 4 } ] \hspace { 2 m m } { } ^ { h _ { 3 } } [ S _ { 3 } ] \hspace { 2 m m } { } ^ { h _ { 2 } } [ S _ { 2 } ] \hspace { 2 m m } { } ^ { h _ { 1 } } [ S _ { 1 } ] , } \\ \end{array} \right.
\frac { \partial \xi _ { 1 } } { \partial x _ { 0 } } + \xi _ { 1 } \frac { \partial \xi _ { 1 } } { \partial x _ { 1 } } = 0 , \quad x _ { 1 } - \xi _ { 1 } x _ { 0 } = F ( \xi _ { 1 } )
d f _ { i } = d Q _ { i } \frac { \partial f ( Q _ { i } ) } { \partial Q _ { i } } \; .
\phi _ { a } \phi _ { a } = \phi _ { 1 } ^ { 2 } + \phi _ { 2 } ^ { 2 } + \phi _ { 3 } ^ { 2 } = 1 .
\left( \tilde { \alpha } _ { J } \right) _ { k } = \frac { 1 } { \sqrt { n } } \sum _ { i = 1 } ^ { n }
T _ { i k } = ( 1 - 2 \zeta ) \partial _ { i } \varphi \partial _ { k } \varphi + ( 2 \zeta - 1 / 2 ) g _ { i k } \partial ^ { l } \varphi \partial _ { l } \varphi - 2 \zeta \varphi \nabla _ { i } \nabla _ { k } \varphi + ( 1 / 2 - 2 \zeta ) m ^ { 2 } g _ { i k } \varphi ^ { 2 } .
[ X _ { J } , [ X ^ { J } , X _ { [ K } ] ] n _ { L ] } + { \frac { 1 } { 4 } } \{ \bar { \lambda } , \Gamma _ { K L } \lambda \} = 0 .
C ( k , p ) = \left( \begin{array} { c c c c c c c } { 0 } & { 0 } & { i k \delta ( \bar { k } + \bar { p } ) } & { 0 } & { 0 } & { 0 } & { 0 } \\ { 0 } & { - 2 i \delta ( \bar { k } + \bar { p } ) } & { 0 } & { 0 } & { 0 } & { 0 } & { 0 } \\ { i \delta ( \bar { k } + \bar { p } ) } & { 0 } & { 0 } & { 0 } & { 0 } & { 0 } & { 0 } \\ { 0 } & { 0 } & { 0 } & { 0 } & { 0 } & { - \delta ( 0 ) } & { 0 } \\ { 0 } & { 0 } & { 0 } & { 0 } & { 0 } & { 0 } & { - \delta ( 0 ) } \\ { 0 } & { 0 } & { 0 } & { \delta ( 0 ) } & { 0 } & { 0 } & { 0 } \\ { 0 } & { 0 } & { 0 } & { 0 } & { \delta ( 0 ) } & { 0 } & { 0 } \\ \end{array} \right) \, ,
\Gamma _ { \mathrm { e f f } } ^ { \mathrm { P V } } = \Gamma _ { \mathrm { e f f } } ^ { \mathrm { P V } , ( 1 ) } + \Gamma _ { \mathrm { e f f } } ^ { \mathrm { P V } , \, ( \vec { E } ) }
D _ { a } D ^ { a } \phi + \cdots = \frac { \partial V ( T ) } { \partial T }
\partial _ { t } Q = \frac { \delta { \cal H } ( Q , P ) } { \delta P } \quad \partial _ { t } P = - \frac { \delta { \cal H } ( Q , P ) } { \delta Q } .
2
\left[ m _ { 0 } ^ { D - 2 } - K _ { c r } ^ { \frac { D - 2 } { 2 } } \frac { 1 } { \sqrt { \pi } } \Gamma ( \frac { D - 1 } { 2 } ) \Gamma ( \frac { D } { 2 } ) \right] \Gamma ( 1 - \frac { D } { 2 } ) = 0 .
\dot { b } = \frac { 1 } { e ^ { t ( | y | ) - N + 3 A } } = \frac { 1 } { e ^ { t ( | y | ) + 2 k F + 3 J } } ~ ,
\Phi _ { 1 } ^ { G } = e + \zeta \pi _ { 0 } ^ { - 1 } \; , \; \; \Phi _ { 2 } ^ { G } = \chi \; , \; \; \Phi _ { 3 \mu } ^ { G } = b _ { \mu } \; , \; \; \Phi _ { 4 } ^ { G } = x _ { 0 } - \zeta \tau \; , \; \; \Phi _ { 5 } ^ { G } = \psi ^ { 0 } \; , \; \;
{ \bf P } \left( \hat { a } _ { a _ { 1 } } ^ { \dagger } \ldots \hat { a } _ { a _ { s } + 1 } ^ { \dagger } A _ { \; \; \quad \quad b _ { 1 } \ldots b _ { s } } ^ { a _ { 1 } \ldots a _ { s + 1 } } \hat { a } ^ { b _ { 1 } } \ldots \hat { a } ^ { b _ { s } } \right) = \hat { a } _ { a } ^ { \dagger } K ^ { a }
e _ { n } = \gamma _ { n } \, ; \quad e _ { 7 } = i \gamma _ { 5 } \, ; \quad e _ { n + 3 } = e _ { n } e _ { 7 } = i \gamma _ { n } \gamma _ { 5 } \, ; \qquad n = 1 , 2 , 3 \, .
\langle \Psi | \Phi \rangle = \int { \overline { \Psi } } \Phi e ^ { - \langle Z , Z \rangle } { \cal D } Z { \cal D } { \overline { Z } } \, .
{ \cal F } ( a , \Lambda ) = { 8 \pi i b _ { 1 } } \Lambda ^ { 2 } a ^ { 2 } \int _ { 4 \Lambda / \pi } ^ { a } d x { \cal G } _ { 1 } ( x ) x ^ { - 3 } - { \frac { i b _ { 1 } \pi ^ { 3 } } { 4 } } a ^ { 2 } ,
\sum _ { B } \, \Phi _ { A B } \ast L _ { B } ^ { \pm } ( \theta , r ) = r \, M _ { A } \operatorname { c o s h } \theta + \operatorname { l n } ( 1 - \operatorname { e x p } ( - L _ { A } ^ { \pm } ( \theta , r ) ) ) \quad \, .
A _ { \mu } = \frac { 1 } { 2 q } \frac { i ( \overline { { \psi } } \gamma _ { \mu } \slash { \partial } \psi - \overline { { \psi } } \overleftarrow { \slash { \partial } } \gamma _ { \mu } \psi ) - 2 m j _ { \mu } } { \overline { { \psi } } \psi } .
x = a + b \overline { { \psi } } _ { 0 } \psi _ { 0 } , \qquad \psi = c \psi _ { 0 } + d \overline { { \psi } } _ { 0 } , \qquad \overline { { \psi } } = c ^ { * } \overline { { \psi } } _ { 0 } + d ^ { * } \psi _ { 0 } ,
\partial ^ { m } \bar { \chi } _ { m \dot { \alpha } } = \partial ^ { m } \chi _ { m \alpha } = 0 ,
U _ { x , i j } = \operatorname { e x p } \left( - \textstyle { { \frac { 1 } { 2 } } } i g a ^ { 2 } \epsilon _ { i j k } B _ { x , k } ^ { a } \tau ^ { a } \right) \, .
\zeta _ { s } ( \Delta _ { X } ) - \zeta _ { s } ( \Delta _ { Y } ) + \zeta _ { s } ( \Delta _ { Z } ) = 0 , \quad \forall s .
P _ { M } ^ { N } ( z ) = \frac { \sum _ { n = 0 } ^ { N } A _ { n } z ^ { n } } { \sum _ { n = 0 } ^ { M } B _ { n } z ^ { n } }
T ( u ) = \left( \begin{array} { c c } { 1 } & { - u ^ { 0 } \vec { u } ^ { \mathrm { T } } } \\ { \hline 0 } & { I } \\ \end{array} \right)
p _ { \mu } T _ { \mu \nu } ^ { S \rightarrow A V } = 2 m i T _ { \nu } ^ { S \rightarrow P V }
\nabla _ { \mu } \phi _ { \mu _ { 1 } \dots \mu _ { k } } = ( \partial _ { \mu } + \omega _ { \mu } ) \phi _ { \mu _ { 1 } \dots \mu _ { k } } - \sum _ { i = 1 } ^ { k } \Gamma _ { \mu _ { i } \mu } ^ { \lambda } \phi _ { \mu _ { 1 } \dots \mu _ { i - 1 } \lambda \mu _ { i + 1 } \dots \mu _ { k } }
\vert \mathrm { o p e n } > = J _ { - 1 } ^ { a } \vert j >
M = \left( \begin{array} { c c } { C } & { \hat { 0 } } \\ { \hat { 0 } } & { \bar { C } } \\ \end{array} \right) ,
\{ \hat { X } ^ { \mu } , \hat { S } ^ { \nu \lambda } \} ^ { \prime } = 2 \frac { \bar { P } ^ { \nu } \hat { S } ^ { \lambda \mu } - \bar { P } ^ { \lambda } \hat { S } ^ { \nu \mu } } { ( P + \bar { P } ) ^ { 2 } } - \frac { 4 P ^ { \mu } } { ( P + \bar { P } ) ^ { 2 } } ( \bar { P } ^ { \lambda } D ^ { \nu \rho } A _ { \rho } - \bar { P } ^ { \nu } D ^ { \lambda \rho } A _ { \rho } ) . \nonumber \,
T ^ { 0 i } = B \tilde { E } _ { i } - ( \partial _ { j } E _ { j } ) A _ { i } - \frac { 2 < B ^ { 2 } > } { \Lambda ^ { 3 } \kappa } \partial _ { i } ( A E ) + \frac { M } { 8 } \partial _ { j } ( A _ { i } \tilde { A } _ { j } + A _ { j } \tilde { A } _ { i } ) ;
\langle | \Phi _ { + } ( y ) | \rangle = { \frac { A } { \operatorname { c o s } [ a y + b + c _ { 0 } \mathrm { s g n } ( y ) + c _ { \pi } \mathrm { s g n } ( y - \pi R ) ] } } .
\sqrt { \kappa ^ { 2 } + | \alpha | ^ { 2 } } = \frac { \pi \lambda ^ { 2 } } { L _ { 1 } L _ { 2 } } .
R _ { 2 } ( y _ { i } , y _ { j } ) = f _ { 1 , 1 } ( y _ { i } , y _ { j } ) + { \frac { \kappa } { 2 } } \left( f _ { 2 , 1 } ( y _ { i } , y _ { j } ) + f _ { 1 , 2 } ( y _ { i } , y _ { j } ) \right) + O ( \kappa ^ { 2 } ) ,
\Delta ^ { ( 1 ) } ( \beta _ { R } , \beta _ { I } , k , z _ { 1 } ) = u ~ P ( \beta _ { R } | \beta _ { I } , k , \mathrm { R e } ( z _ { 1 } ) , \mathrm { I m } ( z _ { 1 } ) )
\eta ^ { \mu } ( \sigma ) = \hat { \xi } ^ { \mu } ( \sigma , r ( \sigma ) ) \ .
m ^ { 2 } ( t ) = m ^ { 2 } \operatorname { e x p } ( ( 2 - 2 \alpha + \frac { 2 \alpha ^ { 2 } } { Q ^ { 2 } } ) t )
d _ { p _ { 1 } } C _ { p _ { 1 } } ^ { a _ { 1 } } = H _ { 1 } ^ { a _ { 1 } } , \; d _ { p _ { 1 } } H _ { 1 } ^ { a _ { 1 } } = 0 ,
\partial _ { 5 } \left\langle { \Phi } \right\rangle = - g _ { 5 } ^ { 2 } \phi ^ { \dagger } \phi \left( \delta ( x _ { 5 } ) - { \frac { 1 } { 2 \pi R } } \right) \ .
\sigma _ { a b } = m ^ { 2 } \delta _ { a b } + v _ { a b } ,
\Pi ^ { - } = 2 i \theta ^ { - } P _ { p } \sigma ^ { p } \ , \qquad \bar { \Pi } ^ { - } = 2 i P _ { p } \sigma ^ { p } \bar { \theta } ^ { - }
\widehat { p } ^ { a } \left| B _ { X } \right\rangle \! = \left( \widehat { q } ^ { i } - y ^ { i } \right) \left| B _ { X } \right\rangle \! = 0 ,
{ \frac { m } { 2 } } \left[ \mathrm { T r } \left( \Phi ^ { 2 } \right) - \mathrm { T r } \left( { \widetilde \Phi } ^ { 2 } \right) \right] ~ ,
\phi _ { i } ^ { \prime } ( x ) = \phi _ { i } ( x ) + \delta _ { \mathrm { B R S } } \phi _ { i } ( x ) \Theta [ \phi ] ,
{ \frac { \partial W } { \partial \bar { q } } } = 0 \qquad \Longrightarrow \qquad q = Q ,
\Omega \to \Omega + 1 \qquad \mathrm { a n d } \qquad \Omega \to - 1 / \Omega \ .
\Psi ( x ) = ( 2 \pi ) ^ { - 3 / 2 } \int \left[ \Psi ^ { + } ( \mathbf { p } ) e ^ { i p x } + \Psi ^ { - } ( \mathbf { p } ) e ^ { - i p x } \right] d ^ { 3 } p ,
\begin{array} { l } { \Phi _ { 2 } = ( \phi _ { 2 } ^ { + } ) ^ { - 1 } \{ D _ { 2 } \} _ { p , q } , } \\ { \bar { \Phi } _ { 2 } = \phi _ { 2 } ^ { + } , } \\ { \Phi _ { 1 } = ( \phi _ { 1 } ^ { + } ) ^ { - 1 } \{ D _ { 1 } \} _ { p , q } p ^ { D _ { 2 } } , } \\ { \bar { \Phi } _ { 1 } = \phi _ { 1 } ^ { + } q ^ { D _ { 2 } } . } \\ \end{array}
\delta { \cal L } _ { r e d } ~ ~ \sim ~ ~ \partial _ { ( + } \left[ ~ { \bar { \epsilon } } ( D _ { - ) } \psi ) \phi ^ { * } ~ + ~ c . c . \right] ~ ~ ,
\epsilon _ { \mathbf { p } } = \pm \sqrt { \left( E _ { \mathbf { p } } \pm \mu \right) ^ { 2 } + \chi ^ { 2 } } .
\int d ^ { 2 p - 2 } x \sqrt { \operatorname* { d e t } ( \delta _ { i j } + 2 \pi \alpha ^ { \prime } F _ { i j } ) } \ .
Q = \frac { 1 } { 2 \pi R } \int _ { 0 } ^ { L } \partial _ { x } \varphi ( x , \, t ) d x \, .
i \hat { { \cal V } } ( x , y ) \; ( \equiv i { \cal V } ( x , y ) \, \hat { A } ) \to 2 \int \frac { d ^ { \, 4 } P } { ( 2 \pi ) ^ { 4 } } \, e ^ { - i P \cdot ( x - y ) } \left[ P \cdot \partial _ { X } f ( X ; P ) \right] \hat { A } .

f ( n _ { 1 } n _ { 2 } n k ) { \bar { f } } ( n _ { 1 } , n _ { 2 } , n k ) = 1
{ \cal D } = \left( \begin{array} { c c } { - \frac { d ^ { 2 } } { d x ^ { 2 } } + 4 } & { 0 } \\ { 0 } & { - \frac { d ^ { 2 } } { d x ^ { 2 } } + 4 - \frac { 6 } { \operatorname { c o s h } ^ { 2 } x } } \\ \end{array} \right) \qquad .
S _ { b o s } \, = \, \int d ^ { 3 } x \, \left( \pm \frac { i } { 2 } A _ { \mu } \epsilon _ { \mu \nu \lambda } \partial _ { \nu } A _ { \lambda } \, - \, \frac { i } { \sqrt { 4 \pi } } s _ { \mu } \epsilon _ { \mu \nu \lambda } \partial _ { \nu } A _ { \lambda } \right) ~ ~ ~ ,
\omega _ { \pm } ( p , \sigma ) = \sqrt { C ( p ) ^ { 2 } + m _ { \sigma } ^ { 2 } + ( B ( p ) \pm \Lambda ) ^ { 2 } }
2 i \pi \Theta [ - \epsilon ( z ) ] [ \; ] { \cal B } ( - \alpha , \beta + \alpha + 1 ) \left[ e ^ { i \pi \epsilon ( z ) \alpha } - e ^ { i \pi \alpha } \right] z ^ { - \alpha - \beta - 1 } =
{ \cal L } = - { \frac { 1 } { 2 } } \left\{ U \partial _ { \mu } { \bf X } \cdot \partial ^ { \mu } { \bf X } + U ^ { - 1 } { \cal D } _ { \mu } \varphi { \cal D } ^ { \mu } \varphi + \mu ^ { 2 } U ^ { - 1 } \right\} ,
\begin{array} { r c l l } { \langle \mu _ { \lambda _ { 1 } } \mu _ { \lambda _ { 2 } } \mu _ { \lambda _ { 3 } } \mu _ { \lambda _ { 4 } } \rangle } & { = } & { \mathcal { F } _ { \lambda _ { 1 } , \lambda _ { 2 } , \lambda _ { 3 } , \lambda _ { 4 } } \left| \prod _ { i < j } z _ { i j } ^ { \lambda _ { i } \lambda _ { j } } \right| ^ { 2 } } & { \mathrm { f o r ~ \lambda _ 1 + \lambda _ 2 + \lambda _ 3 + \lambda _ 4 = 1 ~ } \, , } \\ \end{array}
W _ { i } ( \xi ) = \int _ { 0 } ^ { \infty } i \Delta ^ { + } ( \xi , \kappa ) \rho _ { i } ( \kappa ) d \kappa , \quad i = 1 , 2
{ \bf P } = { \bf p } , \quad { \bf M } = { \bf p } \times { \bf x } + \sum _ { i = 1 } ^ { N } { \bf p } _ { i } \times { \bf e } _ { i } = { \bf p } \times { \bf x } + \sum _ { i = 1 } ^ { N } { { \bf p } ^ { \bot } } _ { i } \times { \bf e } _ { i } - \frac 1 2 \sum _ { i , j = 1 } ^ { N } { \phi } _ { i j } { \bf e } _ { i } \times { \bf e } _ { j } .
{ \cal E } ^ { B } = \frac 1 { 1 2 \pi } \int _ { - \infty } ^ { \infty } [ d \tau \, a ^ { 2 } \sqrt { f ^ { \prime } } - d ( a \sqrt { f ^ { \prime } } ) ] ,
m ^ { 2 } ( a ^ { 2 } / 2 , \chi ) = \frac { \mu ^ { 2 } a ^ { 4 } } { 4 }
\tilde { D } = ( D \tilde { \theta } ) ^ { - 1 } D
{ \cal D } _ { 2 } = { \cal D } _ { 3 } | _ { r = { \rho } } + \frac { i { \gamma } ^ { 3 } } { \rho } .
\tilde { a } _ { 1 } = 2 a _ { 1 } + a _ { 3 } , \qquad \tilde { a } _ { 2 } = 2 a _ { 2 } + a _ { 3 } .
\lbrack 1 , 1 \rbrack _ { Z _ { 2 } } = \lbrack 1 , \gamma \rbrack _ { Z _ { 2 } } = 0 , \quad \lbrack \gamma , \gamma \rbrack _ { Z _ { 2 } } = - 2 .
\frac { i } { 2 \xi } \int d ^ { 4 } x ( \partial . { \hat { A } } ^ { a } ) ^ { 2 }
d s ^ { 2 } = ( r ^ { 2 } / R ^ { 2 } - 1 ) d t _ { s } ^ { 2 } - { \frac { d r ^ { 2 } } { r ^ { 2 } / R ^ { 2 } - 1 } } + r ^ { 2 } d \Omega _ { D - 2 } ^ { 2 } \; ,
{ \bf E } _ { \infty } \simeq \frac { Q \, { \bf r } } { 4 \pi r ^ { 3 } } \, , \ \, , \ \ { \bf B } _ { \infty } \simeq \frac { { \bf m } \wedge { \bf r } } { 4 \pi r ^ { 3 } } \, .
E ( l ) = \sum _ { k = 1 } ^ { l } E ( l , k ) = \frac { 1 } { l } \sum _ { d | l } \mu ( l / d ) \left\{ F _ { d + 1 } + F _ { d - 1 } \right\} \, ,
\begin{array} { c c l } { \vspace { . 2 c m } } & { } & { \omega ^ { 2 } ( x ) = - \mathrm { T r } \Big [ \Big ( W ^ { \prime } ( \hat { \Phi } ) + [ \hat { X } , \hat { Y } ] \Big ) \frac 1 { x - \hat { \Phi } } \Big ] \ , } \\ \end{array}
Q _ { 5 } = - i \int d ^ { 3 } x \langle j _ { 4 } ^ { 5 } \rangle
D _ { t } = \{ x | f ^ { 2 } ( x ) < 2 \gamma t \}
\widetilde { \cal F } ^ { ( 1 ) } ( x ) = - \int d y \int d z \int d w \Phi ^ { j } ( y ) \omega _ { j k } ( y , z ) X ^ { k l } ( z , w ) \{ \Omega _ { l } ( w ) , { \cal F } ( x ) \} _ { ( { \cal F } ) } .
\frac { \delta m _ { i } } { m _ { i } } = \frac { \beta ^ { 2 } } { 2 h } \operatorname { c o t } \frac { \pi } { h } \, \, .
2 \ell \rightleftharpoons \frac { 1 } { T } \, ; \hspace { 2 c m } p \rightleftharpoons - u .
\operatorname { e x p } \left( { \frac { \Phi } { 2 } } \right) = { \frac { C _ { 0 } } { \operatorname { l o g } { \rho / \rho _ { 0 } } } } ,
\epsilon ^ { * } = - { \frac { 2 c _ { V } ( G ) } { \pi k } } ,
\frac { S } { \tilde { S } ^ { ( n ) } } = O \left( \frac { 1 } { \lambda ^ { n + 1 } { \cal T } ^ { 3 } \, V } \right) .
G ( \mu ) = G _ { 0 } l ^ { 1 - d } W ^ { \Delta } F \left( \Delta , \Delta - \frac { d } { 2 } + \frac { 1 } { 2 } , 2 \Delta - d + 1 , W \right) ,
\hat { \lambda } _ { j } \cdot \hat { \sigma } _ { q } ^ { x } \hat { \gamma } _ { i } ^ { + } = \hat { \lambda }
\epsilon ^ { \mu \nu } = { \frac { 1 } { 2 } } e ( e ^ { \mu { [ + 2 ] } } e ^ { \nu { [ - 2 ] } } - e ^ { \mu { [ - 2 ] } } e ^ { \nu { [ + 2 ] } } ) , \qquad \left( \epsilon ^ { { 0 1 } } = - \epsilon _ { { 0 1 } } = 1 \right) ,
{ \cal L } ^ { ( 2 ) } = \pi ^ { a } \partial _ { 0 } { n } ^ { a } - \Omega _ { 1 } \partial _ { 0 } { \rho } - \Omega _ { 2 } \partial _ { 0 } { \sigma } - { \cal H } ^ { ( 2 ) } ,
\left( \begin{array} { c } { \delta _ { \; \; \beta } ^ { \alpha } } \\ { - \delta _ { \; \; \beta } ^ { \alpha } } \\ \end{array} \right) .
e ^ { i \Gamma [ A ] } = \int { \cal D } \psi { \cal D } \overline { \psi } e ^ { i \int d ^ { 2 } x \overline { \psi } i D \! \! \! / \psi } ,
F ^ { T } \rightarrow - { \frac { 1 } { T ^ { 2 } } } ~ F ^ { T } \; .
\Pi ( q ) \equiv \prod _ { i < j } \sigma ( q _ { i j } ) , \hspace { 1 c m } \sqrt { g } = - \frac { l } { n } \sigma ^ { \prime } ( 0 ) ,
\int A _ { ( 3 ) } \wedge X _ { 8 } \ , \qquad \mathrm { w i t h } \quad X _ { 8 } = { \frac { 1 } { 2 4 } } \left( p _ { 2 } - { \frac { 1 } { 4 } } p _ { 1 } \wedge p _ { 1 } \right) \ .
\begin{array} { l } { t ^ { \prime } e ^ { \prime } { t ^ { \prime } } ^ { - 1 } = q ^ { 2 } e ^ { \prime } ~ , ~ ~ ~ ~ t ^ { \prime } f ^ { \prime } { t ^ { \prime } } ^ { - 1 } = q ^ { - 2 } f ^ { \prime } ~ , } \\ { [ e ^ { \prime } , f ^ { \prime } ] = \displaystyle \frac { t ^ { \prime } - { t ^ { \prime } } ^ { - 1 } } { q - q ^ { - 1 } } ~ , } \\ { \Delta ( e ^ { \prime } ) = e ^ { \prime } \otimes { t ^ { \prime } } ^ { - n } + { t ^ { \prime } } ^ { 1 - n } \otimes e ^ { \prime } ~ , } \\ { \Delta ( f ^ { \prime } ) = f ^ { \prime } \otimes { t ^ { \prime } } ^ { n - 1 } + { t ^ { \prime } } ^ { n } \otimes f ~ , } \\ { \Delta ( t ^ { \prime } ) = t ^ { \prime } \otimes t ^ { \prime } ~ , } \\ { \epsilon ( e ^ { \prime } ) = 0 = \epsilon ( f ^ { \prime } ) ~ , ~ ~ ~ ~ \epsilon ( t ^ { \prime } ) = 1 ~ , } \\ { S ( e ^ { \prime } ) = - t ^ { - 1 } e ^ { \prime } ~ , ~ ~ ~ ~ S ( f ^ { \prime } ) = - f ^ { \prime } t ^ { \prime } ~ , ~ ~ ~ ~ S ( t ^ { \prime } ) = t ^ { - 1 } ~ . } \\ \end{array}
F _ { i j } = \partial _ { i } A _ { j } - \partial _ { j } A _ { i } - i A _ { i } \star A _ { j } + i A _ { j } \star A _ { i }
\mu _ { + } ^ { - 1 } \mu _ { - } = \nu _ { - } \eta \nu _ { + } ^ { - 1 } ,
f ( \rho ) = \frac { C } { \rho } , \qquad a ( \rho ) = \frac { D } { \rho ^ { \beta } }
\rho = \frac { 1 } { 2 } \left( \frac { x _ { i } ^ { 2 } } { m } + k q _ { i } ^ { 2 } \right)
{ \cal Q } _ { k } = \oint _ { \partial \Sigma } d \Sigma _ { \mu \nu } \gamma ^ { \mu \nu \lambda } \hat { \nabla } _ { \lambda } \epsilon _ { k } = 0 \ .
q _ { - } ^ { \prime } - \frac { 1 } { \Delta _ { \perp } } \ast \rho _ { 2 } \ ,
{ \bf J } = { \frac { 1 } { 2 } } \left( \begin{array} { c c } { p ^ { \dagger } } & { q ^ { \dagger } } \\ \end{array} \right) { \bf \sigma } \left( \begin{array} { c } { p } \\ { q } \\ \end{array} \right) + { \frac { 1 } { 2 } } \left( \begin{array} { c c } { r ^ { \dagger } } & { s ^ { \dagger } } \\ \end{array} \right) { \bf \sigma } \left( \begin{array} { c } { r } \\ { s } \\ \end{array} \right)
m _ { l } / M = \mathrm { L o g } \left( \frac { F _ { l } } { F _ { 0 } } \right) .
\begin{array} { c } { s c = c \partial c \; , } \\ { s b = - ( \partial b ) c - 2 b \partial c \; . } \\ \end{array}
E _ { \mathrm { i n } } = - \frac { 1 } { 2 \pi a }
\begin{array} { l l } { z ^ { \prime } = z + \varepsilon \zeta \theta ; } & { \theta ^ { \prime } = \theta + \varepsilon \zeta } \\ { \bar { z } ^ { \prime } = \bar { z } - \bar { \varepsilon } \bar { \zeta } \bar { \theta } ; } & { \bar { \theta } ^ { \prime } = \bar { \theta } + \bar { \varepsilon } \bar { \zeta } . } \\ \end{array}
- \frac { 1 } { 2 } { \bf \nabla } _ { \rho } ^ { 2 } \psi ( \rho ) = E \psi ( \rho ) .
F ( x ) \equiv \int _ { 0 } ^ { x } d x \, \sqrt { x ^ { 2 } + 1 } + { \frac { 1 } { i \alpha } } \operatorname { l n } ( x ^ { 2 } + 1 ) , \quad \alpha \equiv { \frac { 2 \beta } { \hbar v g } } = { \frac { 4 } { \hbar v g ^ { 2 } [ V _ { 1 } ^ { \prime } ( 0 ) - V _ { 2 } ^ { \prime } ( 0 ) ] } } > 0 .
E ( l , k ) = \delta _ { l , 2 } \delta _ { k , 1 } + \frac { 2 } { l + k } \sum _ { 2 d | l \pm k } \mu ( d ) { \binom { \frac { l + k } { 2 d } } { \frac { l - k } { 2 d } } } \, ,
\alpha _ { 1 } ^ { r } \gamma _ { 1 } + \dots + \alpha _ { N } ^ { r } \gamma _ { N } = 0 \quad ( r = 1 , . . . , R ) \; ,
\eta = - \frac { 1 } { 2 } \operatorname { l n } \left( \frac { \operatorname { c o s h } \left( \sqrt { 2 } b _ { \infty } \sqrt { 1 + \alpha ^ { 2 } } \; y - \mathrm { a r c s i n h } \; \alpha \right) } { \sqrt { 1 + \alpha ^ { 2 } } } \right)
P _ { ( 2 ) } ^ { - } = \int \beta d \beta d ^ { 9 } p d ^ { 8 } \lambda \Phi ( - p , - \lambda ) \left( - \frac { p ^ { I } p ^ { I } } { 2 \beta } \right) \Phi ( p , \lambda ) \, .
\Gamma ( z + 1 ) = \int _ { 0 } ^ { \infty } \, \, d x \, \, e ^ { - x } x ^ { z } .
\frac { d } { d s } { \bf C } _ { i } = \frac { 1 } { 2 } \epsilon _ { i j k } { \bf C } _ { j } \times { \bf C } _ { k } \, .
Z = \sum _ { s p i n s } \prod _ { c u b e s } W ( a | e , f , g | b , c , d | h ) ,
\left\{ Q ^ { i } , Q ^ { j } \right\} = c ^ { i j } \Gamma ^ { M } C P _ { M } + C c ^ { i j } Z ,
\breve { c } _ { n , \nu } = \sum _ { m = n } ^ { 2 n } { \frac { \Gamma \left( \nu + m - { \frac { D - 1 } { 2 } } \right) } { \Gamma \left( \nu + n - { \frac { D - 1 } { 2 } } \right) } } ~ \breve { a } _ { 2 ( m - n ) , m } ~ ~ ~ .
R ( g ) = - f \left[ 3 \left[ ( \operatorname { l n } f ) ^ { \prime } \right] ^ { 2 } + \frac { \Lambda ( x ^ { 5 } ) } { M ^ { 3 } } \right] \; ,
{ \frac { d } { d s } } { \frac { 1 } { \Gamma ( - s ) } } \bigg | _ { s = 0 } = - 1 ,
\dot { z } _ { 1 } = - N ^ { z } ( z _ { 1 } ) = - g ( z _ { 1 } ) = - \frac { z _ { 1 } } { P _ { z } ( z _ { 2 } - z _ { 1 } ) } ; ~ ~ ~ \dot { z } _ { 2 } = - \frac { z _ { 2 } } { P _ { z } ( z _ { 2 } - z _ { 1 } ) }
c _ { \alpha } = \sum _ { \beta \in \Lambda _ { R } } \epsilon ( \alpha , \beta ) | \beta + \bar { p } > < \beta + \bar { p } |
{ \cal L } = - { \frac { 1 } { 4 } } F _ { \mu \nu } F ^ { \mu \nu } + { \bar { \psi } } ( i \gamma ^ { \mu } D _ { \mu } - m ) \psi \, ,
e ^ { i { \bf k \cdot r } } = e ^ { i k r \operatorname { c o s } ( \theta - \Theta ) } = \sum _ { l = - \infty } ^ { \infty } i ^ { l } \, J _ { l } ( k r ) \, e ^ { i l ( \theta - \Theta ) } \, ,
i \sqrt { 2 } \partial _ { - } \chi - g [ \phi , \psi ] = 0 , \quad \partial _ { - } ^ { 2 } \bar { A } _ { + } - g ^ { 2 } J ^ { + } = 0 .
\Omega _ { k } ^ { ( l ) } = \sum _ { s = 0 } \int d ^ { 3 } y \left( ( - 1 ) ^ { s + 1 } \frac { d ^ { s } } { d t ^ { s } } \phi _ { k } ^ { i ( s ) } ( x , y ) L _ { i } ^ { ( 0 ) } ( y ) \right) .
L _ { g } ^ { ' } \Bigl ( v ( h ) \Bigr ) = v ( L _ { g } h ) = v ( g h ) \, , \, \, \, \forall g , h \in G ,
\xi ^ { 2 } = \left( \frac { \varepsilon _ { 1 } - \varepsilon _ { 2 } } { \varepsilon _ { 1 } + \varepsilon _ { 2 } } \right) ^ { 2 } = \left( \frac { \mu _ { 1 } - \mu _ { 2 } } { \mu _ { 1 } + \mu _ { 2 } } \right) ^ { 2 } ,
R ( e _ { 1 } ) = \epsilon ^ { - J _ { 6 7 } + J _ { 8 9 } } , \quad R ( e _ { 2 } ) = \epsilon ^ { J _ { 4 5 } - J _ { 8 9 } } .
{ \tilde { \cal { E } } } _ { m < 0 } = { \cal { E } } _ { m < 0 } ( B ) - { \cal { E } } ( 0 ) = \frac { B ^ { 2 } } { 2 } + \frac { ( e B ) ^ { \frac { 3 } { 2 } } } { 2 \pi } g \left( \frac { e B } { m ^ { 2 } } \right) \, ,
\hat { O } _ { 2 } ^ { r } \mid 1 > _ { ( 0 ) } = { O } _ { 2 } ^ { r } \mid 0 > _ { ( 0 ) } .
I ^ { c } = \mp { \frac { \pi b \sqrt { 1 - \Lambda a ^ { 2 } } } { 2 G } } \ \ ,
g _ { n } ^ { > } ( r , r ^ { \prime } ) = E _ { n } K _ { | n / \alpha | } ( \beta r ) , \quad \mathrm { f o r ~ r > r ^ { ' } ~ . }
R ^ { \frac { 1 } { 2 } } ( \theta ) _ { \left| \left. a _ { k } \dots \frac { 1 } { 2 } , a _ { 1 } , \frac { 1 } { 2 } \right| n _ { k } \dots , m _ { 1 } , n _ { 1 } \right\rangle } ^ { \left| \left. b _ { k } \dots \frac { 1 } { 2 } , b _ { 1 } , \frac { 1 } { 2 } \right| n _ { k } \dots , m _ { 1 } , n _ { 1 } \right\rangle } = R _ { a _ { 1 } b _ { 1 } } ^ { \frac { 1 } { 2 } } ( \theta ) \prod _ { i = 1 } ^ { k - 1 } f _ { b _ { i } b _ { i + 1 } } ^ { a _ { i } a _ { i + 1 } } ( w _ { m _ { i } } , \nu _ { n _ { i + 1 } } , \theta )
Q _ { 1 } ^ { a b } ( x , y ) \equiv Q _ { 1 } ^ { a b } + x \, J _ { 1 } ^ { a b } + y \, K _ { 1 } ^ { a b } ,
\left\{ \begin{array} { c } { \partial _ { \tau } R + \vec { \nabla } \cdot \left( \vec { \nabla } \Theta \, \sqrt { \displaystyle \frac { R ^ { 2 } + a ^ { 2 } } { 1 + ( \vec { \nabla } \Theta ) ^ { 2 } } } \right) = 0 , \hfill } \\ { \partial _ { \tau } \Theta + R \sqrt { \displaystyle \frac { 1 + ( \vec { \nabla } \Theta ) ^ { 2 } } { R ^ { 2 } + a ^ { 2 } } } = 0 . \hfill } \\ \end{array} \right.
\Delta ^ { ( N , 0 ) } ( s ) = - \sum _ { n > 0 , \vec { n } ^ { 2 } < N } \left[ J ( z _ { n } ) - 2 + 2 J ( y _ { n } ) + \frac { J ^ { 2 } ( y _ { n } ) } { 2 ( 1 - y _ { n } ) } - J ( \tilde { z } _ { n } ) - 2 J ( \tilde { y } _ { n } ) \right] \ ,
\left\{ \Psi \circ \mu , f \right\} = ( \overline { X } _ { i } f ) \, ( Y ^ { i } \Psi ) \circ \mu \, ,
F _ { n } ^ { \mathcal { O } | \mu _ { 1 } \ldots \mu _ { n } } ( \theta _ { 1 } + \lambda , \ldots , \theta _ { n } + \lambda ) = e ^ { s \lambda } F _ { n } ^ { \mathcal { O } | \mu _ { 1 } \ldots \mu _ { n } } ( \theta _ { 1 } , \ldots , \theta _ { n } ) \, \, ,
S = S _ { P h y s . } ( \Phi ^ { a } , \Phi ^ { \ast a } ) + S _ { T } ( \vartheta ^ { b } , \vartheta ^ { \ast b } , c ^ { \alpha } )
\mathcal { A } \equiv \operatorname { e x p } \left[ \int _ { 0 } ^ { \lambda } d \tilde { \lambda } \, \theta ( \tilde { \lambda } ) \right] \, .
F _ { - { \frac { 1 } { 2 } } } ( x ) = \bar { \epsilon } _ { 0 } S ( x ) e ^ { - 1 / 2 \phi ( x ) } \; , \qquad F _ { \frac { 1 } { 2 } } ( x ) = \bar { \epsilon } _ { 0 } \gamma _ { \mu } S ( x ) \partial X ^ { \mu } ( x ) e ^ { 1 / 2 \phi ( x ) } ,
\rho ^ { 0 } = \left( \begin{array} { c c } { 0 } & { - i } \\ { i } & { 0 } \\ \end{array} \right) \, \, \, \mathrm { a n d } \, \, \, \rho ^ { 1 } = \left( \begin{array} { c c } { 0 } & { i } \\ { i } & { 0 } \\ \end{array} \right) .
\psi = \sum _ { i = 0 } ^ { 3 } ( \psi _ { i } ^ { A } + ( \psi _ { i } ^ { A } ) ^ { c } ) T ^ { A }
G = \! e ^ { i \tau L _ { - 1 } } e ^ { i U ^ { ( 1 ) } L _ { 1 } } e ^ { i U ^ { ( 2 ) } L _ { 2 } } e ^ { i U ^ { ( 3 ) } L _ { 3 } } \ldots \! e ^ { i { U ^ { ( 0 ) } } L _ { 0 } } ,
V ( z , \bar { z } ) = e ^ { - q \Phi ( z ) } e ^ { i \alpha \cdot H } e ^ { i ( P _ { R } \cdot X _ { R } - P _ { L } \cdot X _ { L } ) } \; ,
\epsilon _ { i } = \tau _ { i } + \rho _ { i } + \rho _ { i - 1 } , \quad ( \tau _ { 3 } = 0 , \: \rho _ { 0 } = \rho _ { 4 } )
s _ { \infty } ( k ^ { 2 } ) - s _ { J _ { \operatorname* { m a x } } } ( k ^ { 2 } ) \sim O ( J _ { \operatorname* { m a x } } ^ { - 2 } ) .
A ( u ) ~ = ~ \mathrm { R e s } \vert _ { v = u } ^ { } \left( { \frac { 1 } { v - u } } \, R ( u , v ) \cdot L ( v ) \right) + \, { \textstyle { \frac { 1 } { 2 } } } \, \zeta ( 2 u ) \, L ( u )
\partial _ { a } ^ { m } \Gamma _ { i } = \frac { \Gamma ^ { n } } { \lambda _ { i } } \{ \delta _ { n m } \psi _ { a } ^ { i } - \phi _ { b } ^ { n } \phi _ { c } ^ { m } \psi _ { b } ^ { i } \psi _ { c } ^ { i } \frac { \psi _ { a } ^ { i } } { \lambda _ { i } ^ { 2 } } + \phi _ { b } ^ { n } \phi _ { c } ^ { m } \sum _ { j \neq i } \psi _ { b } ^ { j } \frac { ( \psi _ { c } ^ { i } \psi _ { a } ^ { j } + \psi _ { a } ^ { i } \psi _ { c } ^ { j } ) } { ( \lambda _ { i } ^ { 2 } - \lambda _ { j } ^ { 2 } ) } \} \vspace { - 1 2 p t }
\int \mathrm { d } ^ { 4 } x _ { 1 } ~ \cdots ~ \mathrm { d } ^ { 4 } x _ { n } ~ P _ { 4 } ( x _ { 1 } , \cdots , x _ { n } ) ~ \Gamma _ { x _ { 1 } \cdots x _ { n } 0 } = 0
L = \frac { \dot { x } _ { \mu } ^ { 2 } } { 2 e } + \frac { \lambda } { l } ( e - M ^ { - 1 } \dot { x } { } ^ { 0 } ) ,
J _ { 2 } ( z ) \times X ^ { + } ( w ) \rightarrow 0 .
F ( z _ { 1 2 } ^ { \prime } ) = \bar { K } ( z _ { 2 } ; g ) F ( z _ { 1 2 } ) K ( z _ { 1 } ; g )
{ \xi } _ { i } ^ { \ast } , { p } _ { i } ^ { \ast } , \quad i = 2 , \dots , l + 1
\varrho _ { L } - { \cal L } _ { E } = [ 2 \dot { \Phi } ^ { 2 } ] \; K ^ { \prime } ( \dot { \Phi } ^ { 2 } , \Phi ) - K ( \dot { \Phi } ^ { 2 } , \Phi ) + K ( - \dot { \Phi } ^ { 2 } , \Phi ) .
K ^ { \prime } = \sqrt { c - 2 f } \ , \ \ \ K ^ { \prime \prime } = - \frac { 1 } { \sqrt { c - 2 f } } \ ,
\kappa _ { \omega } = \frac { 2 \Gamma ( \Delta _ { \omega } ) } { \pi \Gamma ( 1 - \Delta _ { \omega } ) } \left( \frac { \sqrt { \pi } \Gamma \left( \frac { 1 } { 2 - 2 \Delta _ { \omega } } \right) } { 2 \Gamma \left( \frac { \Delta _ { \omega } } { 2 - 2 \Delta _ { \omega } } \right) } \right) ^ { 2 - 2 \Delta _ { \omega } } \, .
< \frac { 1 } { 2 } , m _ { s } | { \psi } _ { - } ^ { ( \frac { 1 } { 2 } ) } ( g ) > \equiv D _ { m _ { s } - \frac { 1 } { 2 } } ^ { ( \frac { 1 } { 2 } ) } ( g ) = < g , l + \frac { 1 } { 2 } | T _ { m _ { s } - } ^ { \frac { 1 } { 2 } } | g , l > .
\sum _ { l , n } \frac { \mu _ { p - 1 } \lambda ^ { k + n + l } i ^ { k } p ! } { k ! n ! l ! ( p - l ) ! } \partial _ { x ^ { i _ { 1 } } } \ldots \partial _ { x ^ { i _ { n } } } C _ { i _ { 1 } ^ { \prime } \ldots i _ { 2 k } ^ { \prime } j _ { 1 } \ldots j _ { l } [ a _ { l + 1 } \ldots a _ { p } } ^ { 0 } S t r \left( \partial _ { a _ { 1 } } \phi ^ { j _ { 1 } } \ldots \partial _ { a _ { l } ] } \phi ^ { j _ { l } } \phi ^ { i _ { 1 } } \ldots \phi ^ { i _ { n } } \phi ^ { i _ { 2 k } ^ { \prime } } \phi ^ { i _ { 2 k - 1 } ^ { \prime } } \ldots \right)
D ^ { \mu } \frac { \delta f ( A _ { \nu } ) } { \delta A _ { \mu } } = D _ { \mu } \partial ^ { \mu } ( \partial _ { \nu } A ^ { \nu } )
\delta \chi _ { \mu \nu } = i b _ { \mu \nu } , \qquad \delta b _ { \mu \nu } = 0 .
V _ { a b \ \ m n } ^ { k } = \frac { 1 } { g } \ E _ { a } ^ { r } \ E _ { b } ^ { s } \epsilon _ { r s ( m } \ \delta _ { n ) } ^ { i } .
f ( r ) = \left( 1 - \frac { m } { 2 r ^ { n - 1 } } \right) ^ { 2 } + \frac { r ^ { 2 } } { l ^ { 2 } } .
E _ { 1 2 } ~ ~ \Phi = 2 \sqrt { ( m + \frac { 1 } { 2 } b r ) ^ { 2 } + p _ { r } ^ { 2 } + \frac { \ell ( \ell + 1 ) } { r ^ { 2 } } } ~ ~ \Phi ,
T _ { \mathit { G } } ( - t , - t ^ { - 1 } ) = T _ { \mathit { G } ^ { \ast } } ( - t ^ { - 1 } , - t )
d s _ { 1 1 } ^ { 2 } = d x ^ { + } d x ^ { - } + l _ { p } ^ { 9 } \frac { p _ { - } } { r ^ { 7 } } \delta ( x ^ { - } ) d x ^ { - } d x ^ { - } + d x _ { 1 } ^ { 2 } + \ \cdots \ + d x _ { 9 } ^ { 2 }
F _ { a b } = { \frac { 1 } { 2 } } \epsilon _ { a b c d } F ^ { c d }
2 f ^ { 2 } - 4 f ^ { 2 } - g ^ { 2 } ( 1 - \Gamma ) \, ,
( a ^ { \dagger } L _ { m n } a ) = a _ { k } ^ { \dagger } ( L _ { m n } ) _ { k l } a _ { l } = i a _ { [ m } ^ { \dagger } a _ { n ] } , \; \; \; \; \; \; ( L _ { m n } ) _ { k l } = i ( \delta _ { m k } \delta _ { n l } - \delta _ { n k } \delta _ { m l } )
\int d t d ^ { 3 } x \bar { \lambda } \partial ^ { \mu } \gamma _ { \mu } \lambda ,
h = { \frac { s \lambda } { 1 + 2 n + s N + | N | } } ,
Q = c \sum _ { i } f _ { i } ^ { \prime } p ^ { i } + \sum _ { k } c _ { k } p ^ { k } f _ { k } + i n f i n i t e \: m o r e .
\mathrm { T r } \, \operatorname { l o g } ( 1 - \sum _ { i = 0 } ^ { N } A _ { i } ) ~ = ~ \mathrm { T r } \, \operatorname { l o g } ( 1 - \sum _ { k = 1 } ^ { N } \sum _ { m = 0 } ^ { k - 1 } A _ { k } \phi ^ { m } ) + \mathrm { T r } \, \operatorname { l o g } ( 1 - \phi ) ~ .
H _ { i j } ^ { a } = F _ { i j } ^ { a } - g f _ { \; \; b c } ^ { a } A _ { i } ^ { b } A _ { j } ^ { c } ,
{ \tilde { \rho } } _ { { \bf { q } } } = \sum _ { { \bf { k } } } [ \Lambda _ { { \bf { k } } } ( { \bf { q } } ) a _ { { \bf { k } } } ( - { \bf { q } } ) + \Lambda _ { { \bf { k } } } ( - { \bf { q } } ) a _ { { \bf { k } } } ^ { \dagger } ( { \bf { q } } ) ]
{ \bf N } ( { \bf p } , { \bf s } ) : = i p _ { 0 } { \nabla } _ { \bf p } - \frac { { \bf s } \times { \bf p } } { p _ { 0 } + m } , \quad { \bf J } ( { \bf p } , { \bf s } ) : = - i { \bf p } \times { \bf \nabla } _ { \bf p } + { \bf s } : = { \bf L } ( { \bf p } ) + { \bf s } ,
A _ { \mu } \; = \; \partial _ { \mu } \varphi + \epsilon _ { \mu \nu } \, \partial _ { \nu } \sigma \; .
C _ { J } ( \nu _ { 1 } , \nu _ { 2 } ) = ( 2 J + \nu _ { 1 } + \nu _ { 2 } + 1 ) \frac { { \mit \Gamma } ( J + 1 ) { \mit \Gamma } ( J + \nu _ { 1 } + \nu _ { 2 } + 1 ) } { { \mit \Gamma } ( J + \nu _ { 1 } + 1 ) { \mit \Gamma } ( J + \nu _ { 2 } + 1 ) } \, .
( \psi \otimes _ { \zeta , z } \chi ) \mapsto ( e ^ { - u L _ { - 1 } } \psi \otimes _ { \zeta + u , z + v } e ^ { - v L _ { - 1 } } \chi ) ,
u _ { 0 } ( k , r ) = \sqrt { { \displaystyle { \frac { \pi } { 2 } } } } \, i \sqrt { r } \, J _ { 0 } ( k r ) - \sqrt { { \displaystyle { \frac { \pi } { 2 } } } } \, A ( k ) \sqrt { k r } \, H _ { 0 } ^ { ( 1 ) } ( k r ) .
J _ { k } = \oint p _ { k } d q _ { k } , ~ ~ k = r , ~ \theta , ~ \phi ,
\delta _ { \perp } \kappa _ { 1 } = \kappa _ { 3 } \kappa _ { 2 } \Psi _ { 3 } + 2 \kappa _ { 2 } \Psi _ { 2 } { } ^ { \prime } + \kappa _ { 2 } ^ { \prime } \Psi _ { 2 } + \Psi _ { 1 } { } ^ { \prime \prime } - \left( \kappa _ { 1 } ^ { 2 } + \kappa _ { 2 } { } ^ { 2 } \right) \Psi _ { 1 } \, .
\left( \gamma _ { \mu } \partial _ { \mu } + m \right) \psi ^ { ( b ) } ( x ) = 0 , \hspace { 0 . 5 i n } b = 1 , 2 , 3 , 4
f _ { \alpha } ( x ) = \left( 4 \operatorname { s i n } ^ { 2 } \frac { x } { 2 } \right) ^ { \alpha } .
r _ { h } ^ { 2 } = \frac { l ^ { 2 } } { 2 } ( \sqrt { K ^ { 2 } + 4 l ^ { - 2 } \mu } - K ) .
\mu ^ { \prime \prime } + \biggl [ n ^ { 2 } - \frac { a ^ { \prime \prime } } { a } \biggr ] \mu = 0 .
x _ { \overline { m } } = { \frac { 1 } { 2 } } ( x _ { m } + x _ { m + 1 } ) ,
S _ { i j } \left( \theta \right) = \prod _ { x , y } \left[ x , y \right] _ { \theta }
A _ { d } ( p ^ { 2 } + \omega _ { n } ^ { 2 } ) ^ { \frac { 1 } { 2 } d - 2 } \left[ \left( 1 + v ^ { 2 } \right) ^ { \frac { 1 } { 2 } d - \frac { 3 } { 2 } } + \frac { \Gamma ( \frac { 1 } { 2 } d - \frac { 1 } { 2 } ) } { \sqrt { \pi } \Gamma ( d ) } \frac { ( v ^ { 2 } ) ^ { \frac { d } { 2 } - 1 } } { 1 + v ^ { 2 } } { } _ { 2 } F _ { 1 } \left( \textstyle { { \frac { 1 } { 2 } d - \frac { 1 } { 2 } , 1 ; \frac { 1 } { 2 } d ; \frac { v ^ { 2 } } { 1 + v ^ { 2 } } } } \right) \right] \, \ \cdot
\psi _ { c } ( x ) = \gamma ^ { 1 } \psi ^ { * } ( x ) \ ,
L = L ^ { \Lambda } { \bf T } _ { \Lambda } = d Z ^ { M } L _ { M } { } ^ { \Lambda } { \bf T } _ { \Lambda } \, .
z _ { t , 0 } ^ { ' ( r ) } \quad = \quad z _ { t , 0 } ^ { ' ^ { \prime } ( r ) } \quad = \quad 0 \qquad ( 1 \le t \le r , \, \, \mathrm { { a l l } } \, \, r ) \, ;
d T ( x ) = \left( \begin{array} { c c } { \delta ( x ) 1 _ { N - k } } & { 0 } \\ { 0 } & { - \delta ( x ) 1 _ { k } } \\ \end{array} \right) d x
1 - \frac { 2 G M } { \rho } = ( \nabla \rho ) ^ { 2 } \equiv f
M _ { g } = M _ { c _ { 1 } } M _ { c _ { 2 } } M _ { c _ { 3 } } M _ { c _ { 4 } } M _ { c _ { 5 } } M _ { r = \infty } = 1
\delta F \left( \operatorname { s i n } \theta \, d x ^ { 0 } d x ^ { 1 } , 0 , 0 , \epsilon \, d x ^ { 0 } \cdots d x ^ { 3 } \right) .
S _ { \mathrm { p a r t } , 0 } ^ { ( \mathrm { N } ) } = \int d t \, \sum _ { \alpha = 1 } ^ { N } \left( \xi _ { \alpha } ^ { \underline { { a } } } \left( E _ { j , \alpha } ^ { \underline { { a } } } \dot { x } _ { \alpha } ^ { j } + E _ { 0 , \alpha } ^ { \underline { { a } } } \right) - { \frac { 1 } { 2 } } \xi _ { \alpha } ^ { \underline { { a } } } \xi _ { \alpha } ^ { \underline { { a } } } \right) \, .
\xi = v _ { 1 } \left( u _ { 1 } - \kappa v _ { 2 } \right) + v _ { 2 } \left( u _ { 2 } - \kappa v _ { 1 } \right) .
U ( r ) = U ( r _ { 0 } ) + 4 \pi ^ { 2 } K A ( d , \sigma ) \int _ { r _ { 0 } } ^ { r } d s \frac { s ^ { \sigma - d - 1 } } { \varepsilon ( s ) } .
| 0 \rangle \rightarrow | 0 \rangle _ { \beta } = ( 1 + \mathrm { e } ^ { - \beta \epsilon } ) ^ { - 1 / 2 } \{ | 0 \rangle _ { a } \otimes | 0 \rangle _ { b } + \mathrm { e } ^ { - \beta \epsilon / 2 } a ^ { \dagger } { \tilde { a } } | 0 \rangle _ { a } \otimes b ^ { \dagger } { \tilde { b } } | 0 \rangle _ { b } \} .
S _ { E } = \int _ { 0 } ^ { \tau } d \tau \left( { \frac { 1 } { 2 } } x _ { \tau } ^ { 2 } + { \frac { 1 } { 2 } } W ^ { 2 } ( x ) - \psi ^ { \ast } [ \partial _ { \tau } - W ^ { \prime } ( x ) ] \psi \right)
R _ { \mu \nu \, \, b } ^ { \quad a } = \partial _ { \mu } \omega _ { \nu \, \, b } ^ { \, a } - \partial _ { \nu } \omega _ { \mu \, \, b } ^ { \, a } + \omega _ { \mu \, \, c } ^ { \, a } \ast \omega _ { \nu \, \, b } ^ { \, c } - \omega _ { \nu \, \, c } ^ { \, a } \ast \omega _ { \mu \, \, b } ^ { \, c }
M = \int _ { r \rightarrow \infty } d ^ { p } x r ^ { p / 2 } f ^ { - 1 / 2 } T _ { t t } = \frac { p m V _ { p } } { 1 6 \pi G _ { p + 2 } } .
j _ { H W } ( x ) = W _ { i } ( x ) T ^ { i } \; , \; T ^ { i } \in k e r ( A d ( M _ { - } ) )
k _ { 0 } \sim \omega \sqrt { \frac { g \phi _ { 0 } } { 2 M ^ { 2 } } } \ll \omega
Y ( T , U ) = \int _ { \cal F } \frac { d ^ { 2 } \tau } { \Im \tau } \Gamma _ { 2 , 2 } ( T , U ) \left( - 6 \left[ { \overline { { \Omega } } } _ { 2 } ^ { \phantom { 2 } } - \frac { 1 } { 8 \pi \Im \tau } \right] \frac { \overline { { \Omega } } } { \overline { { \eta } } ^ { 2 4 } } - \frac { \overline { { j } } } { 8 } + 1 2 6 \right) \ ,
P _ { 0 } S ^ { * } P _ { 0 } S P _ { 0 } = P _ { 0 } = P _ { 0 } S P _ { 0 } S ^ { * } P _ { 0 } \quad \Longleftrightarrow \quad ( S _ { 0 0 } ) ^ { * } = ( S _ { 0 0 } ) ^ { - 1 } \quad \mathrm { o n } \quad \mathcal { H } _ { \mathrm { p h y s } } .
\langle \psi _ { F a } ^ { 1 - a } \mid \phi _ { F a ^ { \prime } } ^ { 1 - a ^ { \prime } } \rangle _ { t } = \frac { 1 } { 2 } \delta ( a - a ^ { \prime } ) \theta ( t - 1 + a ) \theta ( t - 1 + a ^ { \prime } )
L ( z , u _ { a } , D ) \equiv \int _ { 0 } ^ { \infty } d \hat { T } \, J ( z , u _ { a } , \hat { T } , D ) = L _ { 0 2 } ( z , u _ { a } , D ) + g ( z , D ) G _ { B a b } ^ { 1 - { \frac { D } { 2 } } } + O \bigl ( z ^ { 4 } , G _ { B a b } ^ { 2 - { \frac { D } { 2 } } } \bigr ) \nonumber \,
\int _ { - \epsilon } ^ { \infty } d l \: \mathrm { e } ^ { - l \zeta } \int _ { - \epsilon } ^ { \infty } d l ^ { \prime } \mathrm { e } ^ { - l ^ { \prime } \zeta } l l ^ { \prime } { \frac { l ^ { \prime } - l } { l + l ^ { \prime } } } \{ 3 \, \delta ^ { \prime \prime } ( l ) - { \frac { 3 } { 4 } } t \, \delta ( l ) \} = 0 .
[ { \bar { K } } _ { a } ^ { - } ( p ) , { \bar { K } } _ { b } ^ { - } ( k ) ] = i f _ { a b c } { \bar { K } } _ { c } ^ { - } ( p + k ) - 2 { \pi } ^ { 2 } C p { \delta } _ { a b } { \delta } _ { p + k , 0 } .
E ( v ) = \frac { d } { d t } E ( q ) \; \; \; \; \; \; \; \forall t \, .
\begin{array} { l l } { \sigma ^ { 0 } = - { \bf 1 } _ { 2 \times 2 } = { \bar { \sigma } } ^ { 0 } , } & { \sigma ^ { 1 } = \tau _ { z } = - { \bar { \sigma } } ^ { 1 } , } \\ { \sigma ^ { 2 } = \tau _ { x } = - { \bar { \sigma } } ^ { 2 } , } & { \sigma ^ { 3 } = \tau _ { y } = - { \bar { \sigma } } ^ { 3 } , \quad ( \mathrm { o r } \quad \sigma ^ { y } = \tau _ { y } = - { \bar { \sigma } } ^ { y } ) , } \\ \end{array}
{ \frac { 1 } { L ^ { 2 } } } \prod _ { i = 1 } ^ { 3 } ( r _ { + } ^ { 2 } + q _ { i } ) - \mu r _ { + } ^ { 2 } \sim 0 \ .
\Lambda _ { - 1 , 1 } = \lambda ^ { - 1 } ( e _ { 1 , 2 p } - e _ { 2 , 2 p + 1 } ) + ( e _ { 2 , 1 } - e _ { 2 p + 1 , 2 p } ) + 2 \sum _ { k = 1 } ^ { p - 1 } e _ { 2 + k , 1 + k } - 2 \sum _ { k = 1 } ^ { p - 1 } e _ { p + 1 + k , p + k } ,
x ^ { I } ( \sigma + 2 \pi , \tau ) = x ^ { I } ( \sigma , \tau ) + 2 \pi L ^ { I } .
\xi _ { \sigma } ( \sigma ) = 1 - \frac { \pi e ^ { \sigma } } { 3 } + 8 \pi e ^ { \sigma } \sum _ { n = 1 } ^ { \infty } \frac { n e ^ { - 2 n \pi e ^ { \sigma } } } { 1 - e ^ { - 2 n \pi e ^ { \sigma } } }
m _ { 0 } ^ { 2 } \varphi + { \frac { \mu ^ { 2 } - m _ { 0 } ^ { 2 } } { \beta } } \operatorname { t a n } ( \beta \varphi ) = 0
e _ { \bot } ( x _ { \bot } ) = g \nabla _ { \bot } \int d y ^ { - } d y _ { \bot } d ( x _ { \bot } - y _ { \bot } ) \left\{ f ^ { 3 a b } A _ { \bot } ^ { a } ( y _ { \bot } , y ^ { - } ) \Pi _ { \bot } ^ { b } ( y _ { \bot } , y ^ { - } ) + \rho _ { m } ^ { 3 } ( y _ { \bot } , y ^ { - } ) \right\} \frac { \tau ^ { 3 } } { 2 } \, .
{ \frac { d z } { d r } } = e ^ { 2 A } , \hspace { 1 c m } \psi \rightarrow e ^ { - 3 A / 2 } \psi
F _ { A _ { 1 } } \left( q ^ { a _ { 1 } } , \dot { q } ^ { a _ { 1 } } \right) = 0 \, , \; F _ { A _ { 0 } } \left( q ^ { a _ { 1 } } \right) = 0 \, .
1 - \frac { \delta \phi } { 2 \pi } = L ^ { \prime } ( \infty ) = \frac { 2 G W + 1 } { N ^ { 2 } ( \infty ) } .
\varphi ^ { \prime \prime } + { \frac { 1 } { r } } \varphi ^ { \prime } + \lambda \left( \varphi - \varphi ^ { 3 } \right) = 0 ,
x ^ { N } + y ^ { N } = z ^ { N } , ~ ~ \Phi ( l ) = \omega ^ { l ( l + N ) / 2 } , ~ ~ \omega ^ { 1 / 2 } = \operatorname { e x p } ( \pi i / N ) .
{ e _ { \hat { \mu } } ^ { \ \hat { m } } = \left( \begin{array} { l l } { e _ { \sigma } ^ { \ s } } & { 0 } \\ { 0 } & { e _ { \alpha } ^ { \ a } } \\ \end{array} \right) }
\phi = \left( \begin{array} { c } { 0 } \\ { \phi _ { 0 } } \\ \end{array} \right)
S = { \frac { \pi ^ { 3 } R ^ { 5 } } { 2 \kappa _ { 1 0 } ^ { 2 } } } \left[ \; \int _ { 0 } ^ { 1 } \! d u \, d ^ { 4 } x \, \sqrt { - g } \left( { \cal R } - 2 \Lambda \right) + 2 \int \! d ^ { 4 } x \, \sqrt { - h } K + a \int \! d ^ { 4 } x \, \sqrt { - h } \; \right] \, .
g _ { J _ { 1 } \, J _ { 2 } } ^ { J } \bigl ( J _ { 1 } , M _ { 1 } ; J _ { 2 } , M _ { 2 } \vert J _ { 1 } , J _ { 2 } ; J , M _ { 1 } + M _ { 2 } \bigr ) \, \Bigl ( \xi _ { M _ { 1 } + M _ { 2 } } ^ { ( J ) } ( \sigma _ { 1 } ) + \mathrm { d e s c e n d a n t s } \Bigr ) \Bigr \} ,
\mathrm { T r } ( { \bf M } _ { + } { \bf M } _ { - } ) = 0 \ ,
\int _ { { \cal C } _ { i } } B _ { 2 } = 4 \pi ^ { 2 } \alpha ^ { \prime } \frac { 1 } { 2 } \equiv b _ { 0 }
\alpha ^ { 2 } = { \frac { Q _ { c } ^ { 2 } } { a } } ,
\sum _ { j = 1 } ^ { r + 1 } \bar { x } _ { j } ^ { 2 } = { \frac { r ( r + 1 ) } { 2 } } ,
R ^ { ( 1 ) } = R _ { 1 2 } \, \mathrm { o r } \, R _ { 2 1 } ^ { - 1 } \, , \quad R ^ { ( 2 ) } = R _ { 2 1 } \, , \quad R ^ { ( 3 ) } = R _ { 1 2 } \, , \quad R ^ { ( 4 ) } = R _ { 1 2 } ^ { - 1 } \, \mathrm { o r } \, R _ { 2 1 } \, .
\left( \alpha ^ { \prime } \right) ^ { - 1 } e ^ { - \phi / 2 } f _ { 1 6 } \, s i m \ g ^ { - 2 4 } \sqrt { N } \ ,
( R _ { a b } R ^ { a b } - \frac { n } { 4 ( n - 1 ) } R ^ { 2 } ) \phi ^ { \frac { 2 ( n - 4 ) } { n - 2 } }
j _ { 1 } ^ { k } = \omega _ { 1 } ^ { k - 2 } \subseteq \omega _ { 1 } ^ { k }
V _ { e f f } ( z ) = \frac { 3 ( 5 \alpha + 4 ) k ^ { 2 } } { 6 4 \left( { \frac { k z } { 2 } } + 1 \right) ^ { 2 } } + { \frac { l ^ { 2 } } { 2 R ^ { 2 } } } \left( { \frac { k z } { 2 } } + 1 \right) ^ { { \frac { 5 } { 2 } } ( \alpha - 2 ) } - { \frac { 3 k } { 4 } } \delta ( z )
R : ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ \alpha ^ { \prime } M ^ { 2 } = - 1 , 0 , 1 , 2 , 3 , . . . . . . . . . . .
\hat { \mathrm { P } } _ { \mathrm { R } } ^ { s } = \frac { 1 } { 2 } \hbar \int d x ( { \psi } _ { \mathrm { R } } ^ { \dagger s } ( - i \partial _ { x } ) { \psi } _ { \mathrm { R } } ^ { s } - { \psi } _ { \mathrm { R } } ^ { s } ( i \partial _ { x } ) { \psi } _ { \mathrm { R } } ^ { \dagger s } ) - \int d x \hat { \mathrm { E } } { \partial _ { 1 } } A _ { 1 } .
K ^ { \prime } = \left( \begin{array} { c c } { K _ { 1 1 } } & { \begin{array} { c c } { \cline { 1 - 1 } \mathrm { \hspace { 2 p t } ~ \small ~ - g ~ \hspace { 1 p t } } } & { \vline \hfill } \\ \end{array} } \\ { \hline \begin{array} { c c } { \hfill \vline } & { \mathrm { \hspace { 2 p t } ~ \small ~ - g ~ \hspace { 1 p t } } } \\ { \cline { 2 - 2 } \hspace { 4 p t } 0 \hspace { 4 p t } } & { } \\ \end{array} } & { K _ { 2 2 } ^ { \prime } } \\ \end{array} \right) \, .
a ( \xi ) = \frac { \xi ^ { - 1 } - \xi } { q ^ { - 1 } \xi ^ { - 1 } - q \xi } \, , \quad b ( \xi ) = \frac { q ^ { - 1 } - q } { q ^ { - 1 } \xi ^ { - 1 } - q \xi } \, .
\phi ( \rho ) = \left\{ \begin{array} { l l } { \phi ^ { ( 1 ) } ( \rho ) } & { i n r e g i o n 1 } \\ { \bar { \phi } ^ { ( 1 ) } ( \rho ) } & { i n r e g i o n 1 ^ { * } } \\ { ( - 1 ) ^ { - d _ { \phi } } \, \phi ^ { ( 2 ) } ( - \rho ) } & { i n r e g i o n 2 } \\ { ( - 1 ) ^ { - d _ { \phi } } \, \bar { \phi } ^ { ( 2 ) } ( - \rho ) } & { i n r e g i o n 2 ^ { * } . } \\ \end{array} \right.
x ^ { \pm } = \frac { \left( x ^ { 0 } \pm x ^ { 1 } \right) } { \sqrt { 2 } }
F _ { i _ { 1 } , \dots , i _ { n } } ( \beta _ { 1 } , \dots , \beta _ { n } + 2 \pi i ) = \operatorname { e x p } [ 2 \pi i \omega ( { \cal O } , \Psi ) ] F _ { i _ { n } , i _ { 1 } , \dots , i _ { n - 1 } } ( \beta _ { n } , \beta _ { 1 } , \dots , \beta _ { n - 1 } ) .
a _ { 0 } [ [ a _ { 0 } , a _ { n } ] , a _ { 0 } ] = a _ { 0 } [ [ a _ { 0 } , a _ { n } ] _ { 1 } , a _ { 0 } ] \hbar + a _ { 0 } [ [ a _ { 0 } , a _ { n } ] _ { 2 } , a _ { 0 } ] \hbar ^ { 2 } .
g _ { i j } = \bar { g } _ { i j } + h _ { i j } ,
U ( \vec { x } ; A , B ) = A U _ { \mathrm { S } } ( D ( B ) ^ { - 1 } \cdot \vec { x } ) A ^ { \dagger }
{ \frac { \partial \bar { g } _ { \mu \nu } } { \partial l _ { i } } } \in \mathrm { K e r } ( \bar { F } ^ { \dagger } )
J _ { \mu } = i \overline { { \Psi } } ( x ) \Gamma _ { \mu } \Psi ( x ) , \hspace { 0 . 3 i n } K _ { \mu } = \overline { { \Psi } } ( x ) \Gamma _ { \mu } \overline { { \Gamma } } _ { 5 } \Psi ( x ) , \hspace { 0 . 3 i n } R _ { \mu \alpha } = \overline { { \Psi } } ( x ) \Gamma _ { \mu } \overline { { \Gamma } } _ { 5 } \overline { { \Gamma } } _ { \alpha } \Psi ( x ) ,
{ \cal D } ( g ) = \left( \begin{array} { c c c } { z _ { 1 } ^ { \prime } } & { - \bar { z } _ { 2 } ^ { \prime } } & { - \chi } \\ { z _ { 2 } ^ { \prime } } & { \bar { z } _ { 1 } ^ { \prime } } & { - \bar { \chi } } \\ { \theta ^ { \prime } } & { - \bar { \theta } ^ { \prime } } & { \lambda } \\ \end{array} \right) \, , \quad \quad \sum _ { i } | z _ { i } ^ { \prime } | ^ { 2 } + \bar { \theta } ^ { \prime } \theta ^ { \prime } = 1
L _ { a } = \frac { 1 } { 1 6 \pi } \mathrm { I m } \int d ^ { 2 } \theta \, W ^ { 2 } \tau ( z ^ { - 1 / 4 } )
+ \, \, \mathrm { e } ^ { \varphi } \, { F } _ { [ 3 ] } ^ { R R } \wedge \star { F } _ { [ 3 ] } ^ { R R } \, + \, \frac { 1 } { 2 } \, { F } _ { [ 5 ] } ^ { R R } \wedge \star { F } _ { [ 5 ] } ^ { R R } \, - \, C _ { [ 4 ] } \wedge F _ { [ 3 ] } ^ { N S } \wedge F _ { [ 3 ] } ^ { R R } \Big ] \Bigg \}
S [ \phi ^ { \dagger } , \phi ] \; = \; \int d ^ { 3 } x \, { \cal L } ( \phi ^ { \dagger } , \phi ; \partial \phi ^ { \dagger } , \partial \phi ) \; \; ,
G ^ { K } \, = \, G ^ { 0 } + G ^ { 0 } K ^ { R } G ^ { 0 } + G ^ { 0 } K ^ { R } G ^ { 0 } K ^ { R } G ^ { 0 } + \cdots \equiv \, G ^ { 0 } + G ^ { K R } .
\partial _ { \mu } = \frac { \partial } { \partial \xi ^ { \mu } }
( \partial _ { \gamma } , \partial _ { \gamma } ) _ { ( 1 0 ) } = \sqrt { \tilde { \Delta } } e ^ { 2 U }
3 . 8 G = \eta _ { a b } d z ^ { a } \otimes d z ^ { b } = - d z ^ { 0 } \otimes d z ^ { 0 } + \sum _ { i = 1 } ^ { n - 1 } d z ^ { i } \otimes d z ^ { i } ,
\int _ { - \infty } ^ { \infty } P _ { r } ^ { \bf V } ( x ) P _ { s } ^ { \bf V } ( x ) \, e ^ { - x ^ { 2 } } \, d x \, \propto \delta _ { r \, s } .

\Phi ^ { i } = \frac { 1 } { 2 } \pi ^ { i } - \int d u \; \; \epsilon ( x - u ) \; \theta ^ { i } ( u )
{ \cal H } _ { i n t } ( x ) = - e \overline { { \psi } } ^ { ( 0 ) } ( x ) \beta ^ { \mu } A _ { \mu } ^ { ( 0 ) } ( x ) [ I + \frac { e } { m } ( I - \beta _ { 0 } ^ { 2 } ) \beta ^ { \nu } A _ { \nu } ^ { ( 0 ) } ( x ) ] \psi ^ { ( 0 ) } ( x ) \, \, ,
Q = ( b + 1 / b ) \rho , \qquad \rho = \frac 1 2 \sum _ { \alpha > 0 } \alpha ,
{ h _ { 0 } ^ { 0 } } ( x ^ { \mu } ) = \sum { b _ { 0 } ^ { 0 } } ( t , z ) Y ( x ^ { i } ) , \quad { h _ { z } ^ { 0 } } ( x ^ { \mu } ) = \sum { b _ { z } ^ { 0 } } ( t , z ) Y ( x ^ { i } ) ,
P ^ { 2 } = \bigg ( \frac { 4 } { \pi h ^ { 2 } N } \bigg ) ^ { 2 } \bigg [ 1 + O \bigg ( \frac { 1 } { P ^ { 2 } } \bigg ) \bigg ] , \; \; \pi h ^ { 2 } N < < 1 .
V _ { \mathrm { e f f } } = \operatorname* { l i m } _ { \lambda \rightarrow 0 } \left( V ^ { g e n } + V ^ { P T } \right) .
2 \mu { \frac { \sqrt { J _ { 1 } J _ { 2 } J _ { 3 } } } { N } } \big ( \bar { O } ^ { \prime J _ { 3 } } \Pi ^ { \prime J _ { 1 } } \bar { \Pi } ^ { \prime J _ { 2 } } \delta _ { J _ { 3 } + J _ { 1 } , J _ { 2 } } + O ^ { J _ { 3 } } \Pi ^ { \prime J _ { 1 } } \bar { \Pi } ^ { \prime J _ { 2 } } \delta _ { J _ { 3 } + J _ { 2 } , J _ { 1 } } \big ) .
{ \theta _ { n } ^ { \alpha \Lambda } } { ^ \dagger } = \theta _ { - n } ^ { \alpha \Lambda } ,
{ \cal S } ^ { \mu \nu } \equiv \epsilon ^ { \mu \nu \rho \sigma } \theta _ { \rho } P _ { \sigma } = 0 \ \ ,
\operatorname { e x p } \left( \sum _ { g = 0 } ^ { \infty } x ^ { 2 g - 2 } \mathcal { F } _ { g } \right) = e ^ { - \frac { D ( 0 , 0 , 0 ) } { 2 } \zeta ( 1 ) } \prod _ { ( \ell , n , \gamma , j ) > 0 } ( 1 - p ^ { \ell } q ^ { n } \zeta ^ { \gamma } y ^ { j } ) ^ { D ( \ell n , \gamma , j ) } \, .
M _ { P } ^ { 2 } = M _ { s } ^ { 3 } \int d z e ^ { - 2 \sigma ( z ) }
\begin{array} { c } { p ^ { A } = p _ { 0 } ^ { A } } \\ { x ^ { A } \left( \tau \right) = x _ { 0 } ^ { A } + 2 \left( E _ { 1 } + E _ { 3 } \delta _ { 2 } ^ { 2 } \right) p _ { 0 } ^ { A } - m ^ { - 2 } V ^ { A } \left( \tau \right) } \\ { V ^ { A } \left( \tau \right) = V _ { 1 } ^ { A } \operatorname { c o s } \left( 2 m ^ { 2 } E _ { 3 } \delta _ { 2 } \right) + V _ { 2 } ^ { A } \operatorname { s i n } \left( 2 m ^ { 2 } E _ { 3 } \delta _ { 2 } \right) } \\ \end{array}
\phi \Big [ \beta , a , b , c , d \Big ] = \phi _ { 0 } ^ { \beta } + a \, \phi _ { 0 } ^ { \beta + 1 } + b \, \phi _ { 0 } ^ { \beta + 2 } + c \, \phi _ { 0 } ^ { \beta + 3 } + d \, \phi _ { 0 } ^ { \beta + 4 } \; .
t _ { 1 } = T ~ ~ , \hspace { 2 c m } t _ { 2 } = - \frac { S } { 2 \pi i } ,
B _ { p } ( \beta ) = \operatorname { d i m } \{ \operatorname { k e r } \Delta _ { \beta } \cap \Lambda ^ { p } \} .
\frac { 1 } { 2 } \left( 1 + \sqrt { \frac { j \left( j + 1 \right) - \alpha ^ { 2 } } { 4 } } \right) \mp \frac { i \alpha E } { \sqrt { E ^ { 2 } - \frac { m ^ { 2 } } { 4 } } } = - n
\hat { g } = - ( d x ^ { 0 } ) ^ { 2 } + ( d x ^ { 1 } ) ^ { 2 } + \Big ( d r ^ { 2 } + r ^ { 2 } d \Omega _ { 8 } ^ { 2 } \Big ) \; ,
\rho _ { \alpha \beta } ^ { ( 1 ) } = \left( \begin{array} { c c c c c } { 0 } & { - m \delta _ { i j } } & { e \delta _ { i j } \delta ( \vec { y } - \vec { q } ) } & { 0 } & { 0 } \\ { m \delta _ { i j } } & { 0 } & { 0 } & { 0 } & { 0 } \\ { - e \delta _ { i j } \delta ( \vec { y } - \vec { q } ) } & { 0 } & { 0 } & { \theta \delta _ { i j } \delta ( \vec { x } - \vec { y } ) } & { 0 } \\ { 0 } & { 0 } & { - \theta \delta _ { i j } \delta ( \vec { x } - \vec { y } ) } & { 0 } & { 2 \theta \partial _ { i } \delta ( \vec { x } - \vec { y } ) } \\ { 0 } & { 0 } & { 0 } & { - 2 \theta \partial _ { j } \delta ( \vec { x } - \vec { y } ) } & { 0 } \\ \end{array} \right)
( - 1 ) ^ { m \cdot | a | } [ a , b \cup c ] = [ a , b ] \cup c + ( - 1 ) ^ { ( | b | \cdot | c | + m ) } [ a , c ] \cup b .
\operatorname* { l i m } _ { t \rightarrow - \infty } ( f ( x ) , \phi ( x - t ) ) = L _ { 0 } ^ { - 1 } [ f ] \int _ { - \infty } ^ { \infty } \phi ( x ) d x
I ( a ) = \kappa \int _ { 0 } ^ { 2 a } { \frac { u ^ { 3 } } { 2 } } g _ { 2 } ( u )
E ^ { 2 } ( R > > k ) = \frac { 2 } { k } [ j ( j + 1 ) - m ^ { 2 } + ( m + \frac { k M } { 2 } ) ^ { 2 } \frac { 1 } { R ^ { 2 } } ]
m _ { C } ^ { 2 } = \langle g ^ { 2 } A _ { \mu } ^ { a } A _ { \mu } ^ { a } \rangle + \left( { \frac { 1 } { 2 } } \alpha + \beta \right) \langle i g ^ { 2 } \bar { C } ^ { a } C ^ { a } \rangle .
g _ { L } ^ { 2 } V _ { L } ( \alpha \cdot q ) { { } \atop { \longrightarrow \atop { \omega _ { 3 } \to + \infty } } } \left\{ \begin{array} { c l } { m _ { L } ^ { 2 } \, e ^ { \alpha \cdot Q } } & { \mathrm { f o r ~ s u c h ~ \( \alpha \in \Delta _ + \) ~ t h a t ~ \( \alpha \cdot ~ v \) ~ i s ~ m i n i m u m , } } \\ { m _ { L } ^ { 2 } \, e ^ { - \alpha \cdot Q } } & { \mathrm { f o r ~ s u c h ~ \( \alpha \in \Delta _ + \) ~ t h a t ~ \( \alpha \cdot ~ v \) ~ i s ~ m a x i m u m , } } \\ { 0 } & { \mathrm { o t h e r w i s e , } } \\ \end{array} \right.
r a n k ( b ( x ; \xi ) \ q ( x ; \xi ) ) = r a n k ( q ( x ; \xi ) ) = r \ ,
= \int d ^ { n } x \left\{ \frac { 1 } { 2 } F _ { \alpha \beta } F ^ { \alpha \beta } + \frac { 1 } { 2 } G _ { \alpha \beta } G ^ { \alpha \beta } + 2 D _ { \alpha } \phi ^ { \dagger } D ^ { \alpha } \phi + 2 ( m ^ { 2 } - \phi ^ { \dagger } \phi ) ^ { 2 } \right\}
- \frac { T } { 8 \pi } \left( M _ { W } \left( 9 M _ { W } ^ { 2 } + \frac { 3 } { 2 } M _ { H } ^ { 2 } \right) + \frac { 3 } { 2 } M _ { H } ^ { 3 } \right) \int d ^ { 3 } x ( H _ { 0 } ^ { 2 } - 1 ) .
V = i \left[ \begin{array} { c c } { B } & { 2 \overline { { A } } } \\ { - 2 A } & { - B } \\ \end{array} \right]
\sum _ { { \lambda } = 1 } ^ { 2 } { \epsilon } _ { \mu } ^ { ( { \lambda } ) } ( k ) { \epsilon } _ { \nu } ^ { ( { \lambda } ) } ( k ) = - g _ { { \mu } { \nu } } + \frac { n _ { \mu } k _ { \nu } + n _ { \nu } k _ { \mu } } { k _ { - } } - n ^ { 2 } \frac { k _ { \mu } k _ { \nu } } { k _ { - } ^ { 2 } } .
\epsilon _ { \mathrm { v a c } } = \left< \frac { \beta ( g ) } { 8 g } G \cdot G \right>
\tau _ { j } = 1 - y \operatorname { e x p } ( i ( \mu _ { a } ( 2 j ) - \varepsilon ) ) .
\alpha = \sum _ { a = 1 } ^ { J } \alpha ( a ; \infty ) .
E _ { A D M } = \frac { 1 } { 1 6 \pi G _ { 1 0 } } \oint _ { \infty } d \Sigma ^ { m } \lbrace { } ^ { \circ } D _ { n } g _ { m p } - { } ^ { \circ } D _ { m } g _ { n p } \rbrace { } ^ { \circ } g ^ { n p }
J ( x , y , M ) = \mathrm { t r } \left\langle x \left| \Gamma _ { 5 } \Gamma \frac { 1 } { \Theta + M } \right| y \right\rangle .
\chi _ { h } ^ { N S } ( q ) = \sum S _ { h } ^ { h ^ { \prime } } \chi _ { h ^ { \prime } } ^ { N S } ( \tilde { q } )
\mathrm { S p D } _ { 5 6 } \left( E ^ { { \vec { \alpha } } _ { 5 } } \right) = \left( \begin{array} { l l } { { \bf 0 } } & { B [ { { \vec { \alpha } } _ { 5 } } ] } \\ { C [ { { \vec { \alpha } } _ { 5 } } ] } & { { \bf 0 } } \\ \end{array} \right)
\phi _ { \epsilon } ( \beta ) = { \sqrt { s } } ( Q - i \beta P ) + \sum _ { k \neq 0 } { \frac { a _ { k } } { i \operatorname { s i n h } ( \pi k \epsilon ) } } \operatorname { e x p } ( i k \epsilon \beta ) ,
r = \rho \; , \; \; f = 1 - \frac { \rho _ { 0 } } { \rho } - \frac { \Lambda } { 6 } \rho ^ { 2 } \; .
{ \cal U } ^ { \prime \prime } + { \frac { 2 } { r } } { \cal U } ^ { \prime } - \left( { \cal U } ^ { \prime } \right) ^ { 2 } = { \frac { 1 } { 4 } } \, \left( h _ { 1 } ^ { \prime } \right) ^ { 2 } + { \frac { 1 } { 4 } } \left( h _ { 2 } ^ { \prime } \right) ^ { 2 } ,
\mathbf { \nabla \cdot } { \mathbf { K } } ^ { C } = 4 \pi { \mathbf { J } } ^ { C } \qquad \mathbf { \nabla \cdot } \; ^ { * } { \mathbf { K } } ^ { C } = 0
\not \! \! D _ { V } : { \cal C } ^ { \infty } ( { \cal S } ^ { \pm } ) \otimes { \cal C } ^ { \infty } ( V ) \longrightarrow { \cal C } ^ { \infty } ( { \cal S } ^ { \mp } ) \otimes { \cal C } ^ { \infty } ( V )
{ A _ { 4 } ^ { Y M } = \langle T _ { \mu _ { 1 } \nu _ { 1 } } ( 1 ) T _ { \mu _ { 2 } \nu _ { 2 } } ( 2 ) T _ { \mu _ { 3 } \nu _ { 3 } } ( 3 ) T _ { \mu _ { 4 } \nu _ { 4 } } ( 4 ) \rangle = N ^ { 2 } A _ { 4 } ^ { ( 1 ) } + \tilde { k } N ^ { 1 / 2 } f ^ { ( 0 , 0 ) } ( S , \bar { S } ) A _ { 4 } ^ { ( 2 ) } + \dots , }
S \left( \hat { A } + \delta A \right) = S \left( \hat { A } \right)
p _ { 1 } ^ { \mu } \, = \, m \, \frac { q _ { 2 } ^ { \mu } } { \sqrt { q _ { 2 } ^ { 2 } } } \, - \, ( p _ { 1 } b ) \, b ^ { \mu } .
\int _ { - \infty } ^ { \infty } \frac { d x } { ( \gamma + x ^ { 2 } ) ( \delta + x ^ { 2 } ) } = \frac { \pi } { \sqrt { \gamma \delta } ( \sqrt { \gamma } + \sqrt { \delta } ) } = \frac { \pi } { \sqrt { \gamma \delta } } \frac { \sqrt { \delta } - \sqrt { \gamma } } { \delta - \gamma } ,
\delta _ { \epsilon } \Psi = \epsilon \star \Psi - \Psi \star \epsilon \, \, .
| \varphi _ { k } \dot { \varphi } _ { k ^ { \prime } } - \dot { \varphi } _ { k } \varphi _ { k ^ { \prime } } | = | \dot { \varphi } _ { k } | ^ { 2 } | \varphi _ { k ^ { \prime } } | ^ { 2 } + | \dot { \varphi } _ { k } | ^ { 2 } | \varphi _ { k ^ { \prime } } | ^ { 2 } + \frac 1 2 ( \dot { \varphi } _ { k } \varphi _ { k } ^ { * } + \varphi _ { k } \dot { \varphi } _ { k } ^ { * } ) ( \dot { \varphi } _ { k ^ { \prime } } \varphi _ { k ^ { \prime } } ^ { * } + \varphi _ { k ^ { \prime } } \dot { \varphi } _ { k ^ { \prime } } ^ { * } ) + 2
\left[ a , b , c \right] = a b c + b c a + c a b - c b a - a c b - b a c ,
Z _ { R } ^ { ( m , n ) } = \frac { 1 } { 4 } \sum _ { e } C _ { e } ( m , n ) \frac { \theta ^ { 4 } [ e ] } { \eta ^ { 1 2 } } \Theta _ { ( m , n ) } ( 0 | \tau ) ;
{ d s } ^ { 2 } = - V ( U ) { d t } ^ { 2 } + \frac { { d U } ^ { 2 } } { V ( U ) } + \frac { U ^ { 2 } } { R ^ { 2 } } \sum _ { i = 1 } ^ { n - 1 } { d x } _ { i } ^ { 2 } ,


================================================
FILE: data/test.txt
================================================
7944775fc9.png 1000
78228211ca.png 1001
15b9034ba8.png 1002
6968dfca15.png 1003
6cead0df53.png 1004
5381b22df4.png 1005
27f2b37ce9.png 1006
51a257cdf5.png 1007
5108925e21.png 1008
3882dd3d43.png 1009
566cf0c6f5.png 1010
7d1fe2cc05.png 1011
450b24df87.png 1012
667ff49bc5.png 1013
61928de22b.png 1014
4cd65285c9.png 1015
12697ce419.png 1016
a8ec0c091c.png 1017
72a80f57d9.png 1018
330f27c566.png 1019
58be3470dc.png 1020
3e82680317.png 1021
31068cb86d.png 1022
431dd6944e.png 1023
54a7b9d7f8.png 1024
632e971eb8.png 1025
399e18a85c.png 1026
707b5988e2.png 1027
25fe4d51bf.png 1028
3dc7799669.png 1029
76d30658bb.png 1030
6a366e1f12.png 1031
de8a312222.png 1032
2b891b21ac.png 1033
72e168fb21.png 1034
3d129cfe77.png 1035
6a85896075.png 1036
79edbca78a.png 1037
20032b2645.png 1038
3d15b5c484.png 1039
2608ceb605.png 1040
146a5fa39e.png 1041
159bf72783.png 1042
1e3aab9a4f.png 1043
62409f879c.png 1044
6beab42e50.png 1045
105ccc7946.png 1046
6df7276525.png 1047
65d07ed733.png 1048
34173474c4.png 1049
1a79f53af4.png 1050
57e32e5b33.png 1051
7e1098abc4.png 1052
5ada9733aa.png 1053
5b109d24dc.png 1054
5193ae2c89.png 1055
119b93a445.png 1056
4fa61dbf37.png 1057
25765b9391.png 1058
276c373567.png 1059
3fd05b449f.png 1060
6b2c7f0c1a.png 1061
7c2f256525.png 1062
3beaade5a5.png 1063
3cf56a8338.png 1064
4aea73b2b8.png 1065
2e96a960b1.png 1066
5b10a20227.png 1067
18049a05a9.png 1068
17ad05c612.png 1069
4b69ad5dc8.png 1070
4acf2a0344.png 1071
272667a2d1.png 1072
5a5e2b80dd.png 1073
6596750444.png 1074
4370181032.png 1075
11ff25534a.png 1076
22f7232e98.png 1077
3eaf444392.png 1078
25c3276f55.png 1079
32ebd66b47.png 1080
3b014d22b2.png 1081
6a88fd17f0.png 1082
4b49a4f210.png 1083
4b4263156b.png 1084
48f89a8fc4.png 1085
8d78ecda53.png 1086
4015ba8922.png 1087
acedffb147.png 1088
4f055acd1f.png 1089
5ecb739ec1.png 1090
2beadd086b.png 1091
2450656988.png 1092
147dde7fd3.png 1093
17d9b6a683.png 1094
55358c150e.png 1095
13dbb0dd7c.png 1096
1d796ff39a.png 1097
6a39a91654.png 1098
6661b12767.png 1099


================================================
FILE: data/train.txt
================================================
60ee748793.png 0
66667cee5b.png 1
1cbb05a562.png 2
ed164cc822.png 3
e265f9dc6b.png 4
242a58bc3a.png 6
a4d25113b2.png 7
72f6bc494a.png 8
3cf9d0b234.png 9
5be376c443.png 10
4dd5a0e4ad.png 11
6e7448ca84.png 12
5adf6fe332.png 13
4e9e19a3cf.png 14
3a950d09c0.png 15
5d1a1ed037.png 16
698111df57.png 17
73fdf824d0.png 18
35a6b52146.png 19
6fefdec123.png 20
2a7a69318b.png 21
2ce5749395.png 22
9016b4fca0.png 23
2cb87ed9c8.png 24
4765b43e98.png 25
33978e1330.png 26
3945cf2343.png 27
4c6c104eb5.png 28
5d58861c3f.png 29
2ae8eccc13.png 30
56f7827473.png 31
6176f74d0f.png 32
662ccce98f.png 33
2440895f67.png 34
45b9b7323d.png 35
672a31c2cc.png 36
10c37c445e.png 37
427968501c.png 38
65d334ea47.png 39
7ab1fc083e.png 40
5dab1ec4de.png 41
7d0867620d.png 42
67fcffa9a3.png 43
4b069edf09.png 44
2b8e14887f.png 45
1ba2cffb3c.png 47
7dfe32d42b.png 48
11421b7af6.png 49
22a003507e.png 50
17226e3e67.png 51
43f707d9b6.png 52
2df6c7abd6.png 53
7d791b3d50.png 54
1a87610486.png 56
50919ea0ce.png 57
7891985624.png 58
ae65a915db.png 59
4375e58a64.png 60
2c7002c337.png 61
12521128dc.png 62
25ad6cf3b0.png 63
489da689b3.png 64
224e79c77d.png 65
4f86c93855.png 66
5afd606d6c.png 67
64f51e3fbc.png 68
1fa9bb5655.png 69
50c17d7ef4.png 70
778496aed1.png 71
5aa85bc04d.png 72
6c2c59f99d.png 73
39baa93854.png 74
4849c785c7.png 75
46b3873fbc.png 76
712073fc17.png 77
6ffa4086e2.png 78
72e69b3f75.png 79
2d0a8b7347.png 80
616706dd5a.png 81
1dd5d34448.png 82
3cdc9f09c6.png 83
c49990f9c9.png 84
5f83ae277c.png 85
22692da57d.png 86
e2da14260e.png 87
396ff4def5.png 88
6bc6b638ef.png 89
7fb58b3ce2.png 90
1c5f0abd11.png 91
4552f04e75.png 92
4031e921fa.png 93
78a41122f4.png 94
5a8ba83188.png 95
795aa0d8b2.png 96
5fc2dd90f4.png 97
61ff1fa86d.png 98
58dfa3dbcf.png 99
7b701d6aba.png 100
5f96184332.png 101
742b30d224.png 102
67efc34ea9.png 103
47a37debff.png 104
2319dc19dd.png 105
3f4ac552d5.png 107
5a86fa9b1d.png 108
7e82073cd5.png 110
45e5afd956.png 111
24d85808a1.png 112
55d99738e5.png 113
526038ea4c.png 114
3c0c7b450c.png 115
490a32bf20.png 116
2763f1bb43.png 117
2fd32ca817.png 119
148efc192d.png 120
1a9a0575e7.png 121
7e3b3622f9.png 122
ff5d66560d.png 123
1cd201bfa8.png 124
57bb233b98.png 125
3a13510507.png 127
2358b58f46.png 128
24b499dc70.png 129
67f3e9e236.png 130
52c55aa311.png 131
31872ec89d.png 132
787a1f5717.png 133
5cdfccedbe.png 134
55783f169a.png 135
a20e473646.png 136
58e6560c6c.png 137
2fdcb5efee.png 138
121b29a34e.png 140
4b9c24d197.png 141
4a4f93cbe5.png 142
7149ecb438.png 143
7665dff25e.png 144
1eb0468393.png 147
12cc91e6d0.png 148
3ab85e1c73.png 149
57d8b6dd4a.png 150
2350414e35.png 151
c17e9094b3.png 152
70eb7647b5.png 153
6b36374d9d.png 154
4b79ffccd9.png 156
1481a60f40.png 157
44a2ebab25.png 159
40be12af5f.png 160
62fd4fd186.png 161
5b397b1cd0.png 162
cbd7757104.png 163
2983f1a42e.png 164
6da72deb03.png 165
5a0621bac5.png 167
68641b1f35.png 168
1de2776366.png 170
cb1a57840a.png 171
707fec2eb3.png 172
4fe7cc11b0.png 173
37b3dd5bd8.png 174
1f1d28bc77.png 176
5999b32439.png 177
13ef0a5f7d.png 178
7330bec29b.png 179
1ddfa77a15.png 180
532b6d7730.png 181
3dddd02569.png 183
4bb7d69576.png 184
2350b6d78e.png 185
df04b95eb0.png 186
3f0119b21c.png 187
3ae7d4c8aa.png 188
5a6fb6e214.png 189
20a17f575f.png 190
294619977e.png 191
1d883568bc.png 192
47cf711405.png 193
4695858a93.png 194
4a0cc80e74.png 195
542bfd32ab.png 196
6dbd31790b.png 197
77fb781660.png 199
607ee9e5c2.png 202
3362c78722.png 203
4fa16f1d3f.png 205
7a54e991b0.png 206
50a1f82ae6.png 207
256f1e5b2a.png 208
1c7a4617a5.png 209
1de6b57b0c.png 210
315201f078.png 211
2d2e6105f4.png 212
4e721cfe49.png 213
42c766c6dc.png 214
799f625cf9.png 215
323c465216.png 216
226979ee71.png 217
4d603c88f4.png 218
16a0874240.png 219
386bf9a37b.png 220
7352f31651.png 221
5d3a4f9d53.png 223
252f1b9cea.png 224
4c475e1afd.png 225
131a79148f.png 226
33fcdc8633.png 227
71feac6326.png 228
1e8595e5e1.png 229
3bf4f3ec2d.png 230
2581f66470.png 231
3b1cebad19.png 233
395413c90f.png 234
3b82cc7aee.png 235
7b00f012b2.png 237
9bcc0bb538.png 238
1be344a743.png 239
562c231867.png 240
78ef714591.png 241
173111e861.png 242
3f0ef5c2d1.png 243
3ab0d7b424.png 244
11a8eea207.png 245
738c6bc090.png 246
478234cbe5.png 247
1a882641b7.png 248
81ba9ce4a9.png 249
28a38729aa.png 250
1c2883bfac.png 251
3fdecf80a1.png 252
5f9a3eb396.png 253
580036eb68.png 254
6157449337.png 255
505cc30969.png 256
a43fa630b7.png 257
36fbd5706b.png 258
70a6e5b4b9.png 259
1420ac42ed.png 260
1e1bbaef2b.png 261
347196fad5.png 262
689d84bd57.png 263
7199fd1e3c.png 264
4f6c0a6083.png 265
5c01b8b718.png 266
7893ca4e1b.png 267
54682978af.png 268
3d6cbb761f.png 269
47b0771bd8.png 271
396bcb28c0.png 272
629c42b571.png 273
3bd3e38c6c.png 274
149b313ecf.png 275
e94fbdb3c3.png 276
234d3db18a.png 277
742d89ddeb.png 278
6e8fa596b4.png 279
49ca0559c8.png 280
534c7843ec.png 281
19c9f9219d.png 282
6f49be4f68.png 283
30a0e5b27c.png 285
3b4dd4f61d.png 286
7f22efe073.png 287
277fac3048.png 288
f1c1b75b1b.png 289
5b7bdc01b4.png 290
42b805b1d8.png 291
3ca343759c.png 292
1e216001c4.png 294
5f71c12252.png 295
4c32f7004f.png 296
4b836d0a2c.png 297
e6c3904a0c.png 298
7dd35fa179.png 300
20ceea5819.png 301
51a6e5dd63.png 302
177fda0427.png 303
e6abb29b53.png 304
33274d5123.png 305
4e0c819000.png 306
5c0583ceab.png 308
1c7d2271ab.png 309
e566e35a76.png 310
54e5851527.png 311
3faaa820d3.png 313
6b35aaf215.png 314
394f0f3fe5.png 315
3d455f1d1c.png 316
7467513585.png 317
18b4418f08.png 318
6f9af303a5.png 319
71e9f2ef59.png 321
4623987c43.png 322
21184c98a6.png 323
25346c4649.png 324
5ac8553f89.png 325
1a6ad5d0f5.png 327
61f7ecf1bf.png 328
40dfccc791.png 329
153407229b.png 330
2be537b9e3.png 331
56043a7864.png 332
531fb2a3ca.png 333
37e02be5cf.png 334
77f009c19a.png 335
5de0c822cf.png 336
5813db7bf7.png 337
4ff4ccee85.png 338
438ce70099.png 339
3152e5b5ad.png 340
73fc2e985e.png 341
6c2ba34b57.png 342
3d7bf0c732.png 343
4fe2b3c53c.png 344
545bdc02ca.png 346
4e6f914a8d.png 347
718c18ce99.png 348
20a6d707a0.png 349
fbffbcd491.png 350
4e4c16ef40.png 351
1015942522.png 352
11e04d81c5.png 353
336de40751.png 354
315c7b3eed.png 355
119cec3a3d.png 356
1be1fb7939.png 357
3e19ce7b87.png 358
7bdd525b27.png 359
74e13dac33.png 360
5d297628db.png 362
6d498aaaf2.png 363
37a00adccd.png 364
61c32b5afd.png 365
4e6d45eb04.png 366
cd5afedf4f.png 367
3670b56b83.png 368
7c7dde3340.png 370
412e5c3dcf.png 371
3dcc7624c2.png 372
19d67be751.png 373
38b878380b.png 374
52861fae05.png 375
60ee89c612.png 377
2177654260.png 378
4a4d202235.png 379
11b8f564e0.png 380
772e39cdf2.png 381
34f9117856.png 382
2a381e5b6c.png 383
152f92a6b0.png 384
5919bec0f8.png 385
5f994804a7.png 387
5c9852d327.png 388
56c9b6c684.png 389
618271f3cb.png 390
56faad495b.png 391
729ab0ab1b.png 392
18b0d1e4ff.png 393
b9edb0bb87.png 394
16a2ab5d25.png 395
73dd08e58e.png 396
69c07928bb.png 397
490dcc5dd6.png 398
5e732200a9.png 399
71069928e1.png 400
edccba9380.png 401
4e7d3cee1d.png 402
6c685710b7.png 403
74bc680700.png 404
63eb8cad92.png 405
34f0b42880.png 406
1b9f888640.png 407
227187aa68.png 408
bd4589a12f.png 409
5953fb43fa.png 410
3d9a125931.png 411
47517e0005.png 412
af4fb44184.png 413
67e91357d9.png 415
bdadb47ded.png 416
3fba654036.png 418
662cbd36e7.png 419
6722c940a2.png 420
72632adbf9.png 422
5bccec610e.png 423
24332daee4.png 424
2a54d2ba3b.png 425
658e2e67bb.png 426
4ec63b04f0.png 427
7655c206d2.png 428
2c06a9690c.png 430
63d175a5e0.png 431
19aa0666cc.png 432
2e6122dbfc.png 433
1d95e13fad.png 434
4c9f09e110.png 435
74190c896d.png 436
550610d5d3.png 437
d5c0896dd6.png 439
41dbc59f9c.png 440
7cf89ec296.png 441
412e20f23f.png 442
178ed65519.png 443
40e8f91d3e.png 445
216b9616b6.png 447
54a0a20a42.png 448
56113331d4.png 450
3982559db9.png 451
1e1505c92f.png 452
24d3cd46d2.png 453
3b7a1a6e70.png 454
1915502af4.png 456
6cd24e259e.png 457
6e33251164.png 458
795c17720d.png 460
62a80b376c.png 461
31546c8b41.png 463
46bc294f37.png 464
23d23c19eb.png 466
7cb32fac57.png 467
4f1986e78c.png 468
4c7fe5c962.png 469
7389651287.png 470
2d8ebc9df7.png 471
1955b98ce0.png 472
56077c7b8d.png 473
8adfa46d9e.png 474
14bb617dee.png 475
76653bc864.png 476
7620040742.png 477
20ecceacba.png 478
190a4d7839.png 479
2187611cd5.png 480
2817a56674.png 481
2dfcef9eb0.png 482
4e67fd1568.png 484
47bfa0eec5.png 485
6fa146daf6.png 486
7a6498e465.png 487
4c0346a484.png 488
667ed1d308.png 489
18dd5bebfe.png 490
e7dbdb276c.png 491
26a03f5f3e.png 492
1eba968689.png 493
28bee01ebd.png 494
1ccca004c3.png 495
8c0b3f6ff6.png 496
23ff5d43fa.png 497
75735bfa6b.png 498
68844fd309.png 499
2e35cd83e8.png 500
7fc9603050.png 501
7c6531f57f.png 502
43e44bc1e6.png 503
134ca7f589.png 504
124a510572.png 505
217c11c290.png 506
6a7d838ceb.png 507
2af5ce9da2.png 508
18d380f64a.png 509
4d1149ae54.png 510
2812d8bfc3.png 511
1d680e7a35.png 512
21af95c2df.png 513
20d488de0e.png 514
7d9ab20550.png 515
2d7d315ab7.png 516
7c95382414.png 517
de8534f55a.png 518
62756b2706.png 520
a6077cbcac.png 521
17f1c3c228.png 522
2b8c88ee64.png 523
3b48db8330.png 524
931e144505.png 525
7b549cf22c.png 526
7222c26c18.png 527
2eecb90b72.png 528
688df0df75.png 529
4a0e64d10d.png 530
488bd438ba.png 531
6516a2853c.png 532
f1959f2ba0.png 533
3d3af5cf5a.png 534
684b823b9f.png 535
6026138ebe.png 536
58b0af9a15.png 537
699cbf3379.png 539
45af0dfcca.png 540
36a8d9dc06.png 541
9eac02a9da.png 542
42b20bc6d2.png 543
759f4fc7c4.png 545
7a4a0fbe98.png 546
f95f5b6ca0.png 547
77a207b3bd.png 548
7927cb4cd3.png 549
50b736b0b3.png 551
3e30c4b1fe.png 552
47ab4b9eac.png 554
7611483793.png 555
5049616b13.png 556
2457925124.png 557
5319a5cd5b.png 558
47ece0f8c0.png 560
30a0b199f2.png 562
59f60efb0b.png 563
b58eb32942.png 564
6e3430c217.png 565
27a2603118.png 566
28bd09a362.png 567
783f45a74a.png 568
40438319b3.png 569
3bc0cd120b.png 570
7953333f74.png 571
12973ae6bf.png 572
51b9b3d233.png 573
3d592d51f4.png 574
2168ed75c2.png 575
62ebfd5a50.png 576
5e5e08a0ff.png 577
20386b7947.png 578
69f62b882a.png 579
4c9ed272d9.png 581
511af7f448.png 582
4fade9d45b.png 583
5ced0b6e06.png 584
2002a31c69.png 585
65504a130e.png 586
1de15504a5.png 587
1e244f5909.png 588
2feda5cee2.png 589
34a988525a.png 590
7cd6b7823a.png 591
10fd6c9d3b.png 592
6a1355bfe2.png 593
6c70a91560.png 594
77d2fb4eb6.png 595
4099955659.png 596
7c77facebb.png 597
3870f36b0a.png 598
1321dfa636.png 599
49ddb5edd3.png 600
20e5eb8023.png 601
58522752a3.png 602
2e80f95614.png 603
3daffde15e.png 604
a960f0d483.png 605
78388c122c.png 606
1e3d03189c.png 607
7840a3cb63.png 608
3c69b7b0e2.png 611
1c30697876.png 612
3d5173adba.png 613
66fbeaadc3.png 614
1532c71612.png 615
135b7e68ed.png 617
3d28f96cce.png 618
acf4358da0.png 619
74b9a16ada.png 620
2233f4b015.png 621
72586ce90c.png 622
5019c62bf3.png 623
59d59dd2aa.png 624
69f7e9ccf2.png 626
7139e6b6c4.png 627
33561ebcc0.png 628
7cf6a5cd8c.png 630
6fbe5abccb.png 631
77d8b27e5e.png 632
55fb69ef75.png 635
6cfb18298c.png 636
364367d4ca.png 637
54ecd49027.png 639
6bf1f71956.png 640
1c275ce7be.png 641
5ea177bd59.png 642
c5a5e86be4.png 643
31eb5311ab.png 644
5554b8bafe.png 645
ae2dfab323.png 646
57ee5200eb.png 647
c2815087ca.png 648
c93811c0a1.png 649
6b8fd26512.png 650
3347106f6c.png 651
380771147a.png 652
7e29bd9b2b.png 653
462d961136.png 654
20ecb2fa17.png 655
3077d2b5c5.png 656
26c18afc01.png 657
3b31ac97eb.png 659
34ff47642d.png 660
2bdee6dc8c.png 662
5aac83a092.png 663
49619f5e38.png 664
5ef7f48e2f.png 665
24d694e788.png 666
4ebb99e904.png 667
76d8be8e91.png 668
6a91cc4050.png 669
61db95d2d5.png 671
1b430de29b.png 672
4c50fe07a5.png 673
21983b6f45.png 674
2ee1cccd98.png 675
68d27e4c6b.png 676
5c7818e9bb.png 677
36fabb1629.png 678
2cb4925807.png 680
320edfe252.png 681
18c66a0966.png 682
432b66ec70.png 684
34a146d5cb.png 685
78087b54d2.png 686
6ca4ebfe75.png 688
45a0218f89.png 689
111bd8befd.png 692
3d0907a003.png 693
3077534c40.png 694
a7379b2793.png 695
1c7a5e88b7.png 696
45deabb336.png 697
3db78b4714.png 698
179ee2322b.png 699
5b6a5cc6a5.png 700
4b71e59fcd.png 701
42458ec06a.png 703
2480e734be.png 704
688672b558.png 705
5318f845f6.png 706
b18a1fb447.png 707
4b2b463726.png 708
4ef383bf1c.png 709
7b2a0c1fb6.png 710
c58200df88.png 712
6c24ee49a0.png 713
497b674d2d.png 714
1ba3ebbf0c.png 715
1213b8125f.png 717
43868b882e.png 718
2769dd9b8b.png 719
5619b13ab4.png 720
6a1b59a078.png 722
3549abd8d2.png 723
4fe7a39b99.png 724
1ccd797c55.png 725
77d5eedce4.png 726
78b2f220f2.png 728
66a04072d7.png 729
273e688d7e.png 730
2e86c71271.png 731
247cffb096.png 732
7ce85e3026.png 733
5e9540cb88.png 734
40caface69.png 735
4fd43678d5.png 736
7356a3b589.png 737
51f7423302.png 738
615be49b21.png 739
5c3a48f6b2.png 740
4b9e2e8659.png 741
62a8a1936a.png 742
dfef241e6c.png 743
1493d744aa.png 744
c5fb7b4822.png 745
7cc24f59f3.png 746
37041d1f7f.png 747
e3464d818a.png 748
49b1da67d5.png 749
1dace34d4a.png 750
6c1689ec47.png 751
16e7e8f3f2.png 752
108453837c.png 753
54ce26fd6d.png 754
16cb3262bd.png 755
7adad81603.png 756
da56fb536b.png 758
1da914be75.png 759
1f2c64922b.png 760
6597c474ab.png 762
2a20b5b1cf.png 763
4736919312.png 764
2aa5a25f06.png 765
7e38694661.png 767
51f0d37078.png 768
6bfb8ddf8c.png 769
33249f40f8.png 770
fb47c8400a.png 771
38ecfdce7e.png 772
6c0df8dbd1.png 773
93b145eb5e.png 774
374d5dcf51.png 776
4e40aa7ddc.png 777
706f13055c.png 778
355d2bb1b0.png 779
27b57bcd1f.png 781
1dcde3138e.png 782
62ca41ae62.png 784
3fef6d42a2.png 785
3b13d3cf38.png 786
2670b0a52c.png 787
5a6022ef11.png 788
76687e1577.png 789
62263f898d.png 790
3733924e20.png 791
5e6f762bde.png 793
44bbca7814.png 794
6031ff1d01.png 795
4bbcdcf152.png 796
1ce187f40a.png 797
4ed2183bb6.png 798
184acbfaf7.png 799
410a357212.png 800
377e3a8561.png 801
40a7157e7d.png 802
784bed712c.png 803
2b50c97ed0.png 804
27f2d26053.png 805
24c44bb286.png 806
349d518d66.png 807
3cfa82266e.png 809
5eba7d8a44.png 810
5193b149c1.png 811
17b91d8c32.png 812
deb12268ee.png 813
74540b5448.png 814
4972f72854.png 815
5e0187c848.png 816
2f18211d63.png 817
46d0ff38bf.png 818
2a8ab3e43c.png 819
ca912eb355.png 820
77eaeae3e4.png 821
f1a69e859d.png 822
43ed632fae.png 823
6bb7790b92.png 824
58cf29598d.png 825
efa360c71f.png 827
7a906e1e29.png 828
43b53998fc.png 829
263efbddad.png 830
6d34f3687e.png 831
2fc06d5ffa.png 832
150630c544.png 833
387f9d88dd.png 834
7e3fc2cd8a.png 835
3e4b91167c.png 836
76d665efaa.png 837
bc6584aadf.png 838
566c5e1ba4.png 839
16e90a4a4c.png 840
489f58d4e6.png 841
6922a6c959.png 842
6d751e80a1.png 843
7d7f34a745.png 844
47d61cf5f0.png 846
41d1835b9d.png 847
62657b5e68.png 848
319dd0463f.png 849
2bceca2af5.png 850
1fa7f3fa13.png 851
28a7805724.png 852
4f62e74af6.png 853
d41e91c436.png 854
7f9cb5466e.png 855
49e0f63e9d.png 856
18fcea5309.png 857
4ccb9dd306.png 859
4ad52b90e0.png 860
3d7079863e.png 861
756fcfed82.png 862
1466d6bf21.png 863
31da825201.png 864
164c43af41.png 865
49e504bde5.png 866
3fdfd3011e.png 867
1434096391.png 868
1671b4d29f.png 869
6f85e24401.png 870
71e97fae64.png 871
1a5e4f1ea5.png 872
5c5c0536ca.png 873
7a8a88a457.png 875
70f7d8cb2f.png 876
2efe089406.png 877
536023b45b.png 878
424dd560d5.png 879
d72dcd4146.png 880
163bb3b656.png 881
4652f0eee3.png 882
79230f17fe.png 883
3e4e1d1c9c.png 885
37122e6049.png 886
5a32b3ae99.png 887
4ae0099f82.png 889
132e4fe98e.png 890
53848ae56b.png 893
1a09f0e488.png 894
22ebca097d.png 895
5fe9341510.png 896
8fc62d8cee.png 898
415254995a.png 900
2bbcda22c8.png 901
2a8be07a0d.png 902
393fe5be69.png 903
3d674765e3.png 904
76a11a58c5.png 905
6c29c29288.png 906
4a9defb409.png 907
394b9c6f7a.png 908
55928482f0.png 909
66d373e188.png 910
511e378fdc.png 911
5a97c95d42.png 912
6fd193ddff.png 913
7f66b07a62.png 914
686db389cb.png 915
558cb70d57.png 916
72d002f812.png 917
23ec2a9866.png 918
4ebe29dcdc.png 919
ecddd11060.png 920
3185c48802.png 921
4f6a6bf201.png 922
3dbf43363a.png 923
4f8503ee9d.png 924
538fef06b2.png 925
7d471c4099.png 926
f79d80b12c.png 927
23a0256e05.png 928
4e01ccc31a.png 929
2bc4a215e1.png 930
286b161118.png 931
267a44a011.png 932
786c9a4c35.png 933
64df465ef2.png 934
4539737602.png 935
4bdf6fd523.png 936
8f3748e911.png 937
795b27411f.png 938
a219448585.png 939
2336fcc2f1.png 940
607728aa68.png 941
30dbabc472.png 942
285feda2c2.png 943
329e5a0233.png 944
795e058d5e.png 945
6d2a1c8fd2.png 946
2bcf312d4d.png 948
2e26b03a14.png 949
465798ba74.png 950
10b43f9ed8.png 951
50f03ab5e7.png 952
4c3ade589c.png 953
6535c971df.png 955
51973bf662.png 957
31d37292e5.png 958
21f28c027e.png 959
471d4175f7.png 960
42413b33eb.png 961
34375303b9.png 962
e8fac80be0.png 963
3948c68dfe.png 964
aa9c1f0792.png 965
800beaf2d4.png 966
6b309bd9e0.png 967
333eac3e9f.png 968
2296f849c2.png 969
30f320a4ae.png 970
bc13232098.png 971
40e9bc17aa.png 972
60577e99a1.png 973
55141725a8.png 974
633712725b.png 975
56d58715d5.png 976
6d6e4f3920.png 977
571314b80e.png 978
14fffb7ef9.png 979
180d06c7af.png 980
32a64a6e80.png 981
3c1d34583b.png 982
1a2262a3e2.png 983
200f79bf14.png 984
ef10941593.png 986
4ba7bc9664.png 987
5dcd648dea.png 988
29a11d16e3.png 989
104c5cec5a.png 990
66843f7a8a.png 991
2a2d9f39aa.png 992
25a3d6b46b.png 993
60e35372bc.png 994
20470c3d5f.png 995
19ac621c4b.png 996
61767f5294.png 997
144b2d2e28.png 999


================================================
FILE: data/validate.txt
================================================
5abbb9b19f.png 1100
329a44c373.png 1101
73b51f198b.png 1102
6331d9e7fd.png 1104
91a55d2cb9.png 1106
408fe63a30.png 1107
232d6fea7c.png 1108
702a6a1e50.png 1110
7a7df95199.png 1111
d39642fa83.png 1112
2c65d65f05.png 1113
3bea2f425a.png 1114
3e3f5d7c1f.png 1115
12e9e7dd1f.png 1116
7c99ded238.png 1119
652ec3e732.png 1120
72689463cf.png 1121
523ad74d44.png 1122
b7f8cd1f98.png 1123
3264c226dd.png 1124
c578283712.png 1125
672726b185.png 1126
18457514f2.png 1127
3603a44b2b.png 1128
288ea38816.png 1129
179b37b3d3.png 1130
64daea677f.png 1131
7495d151e1.png 1133
aacbc9b7e5.png 1135
7aed42a53c.png 1136
2ffc852060.png 1137
4efc22f0f5.png 1138
12603c7701.png 1140
105de61fa3.png 1141
47696e11d7.png 1142
53ded8fccd.png 1143
6b5b65e371.png 1144
73778ad253.png 1145
1c6c15eec1.png 1146
48eb8b68bd.png 1147
6c4b620ad6.png 1148
55d0a0fc56.png 1150
4d1592b40a.png 1151
2724dd250b.png 1152
38dbc7d096.png 1153
15c5d88a1a.png 1154
4d31b13054.png 1155
5cb4b5bd0e.png 1156
7376be8a16.png 1157
5caa163344.png 1158
2673247422.png 1159
6552f4db6d.png 1160
63a2608987.png 1162
7a672b3ac0.png 1163
4c986ebb34.png 1164
15329206b1.png 1165
36b2d3fb20.png 1166
71361ce21f.png 1168
23d91c1744.png 1169
31c875f0d6.png 1170
5a559fe73b.png 1171
5e1d3cec87.png 1172
53a1695e86.png 1174
e6b89266e6.png 1175
599063d58b.png 1176
2fdd8e53dc.png 1177
36f3afe9f3.png 1178
326031c68d.png 1179
4cf2740914.png 1180
486d8b9515.png 1181
b80ef03527.png 1182
4d55385c75.png 1183
438635c3bf.png 1184
27fa6d2160.png 1185
7010249b8a.png 1186
1d320e1c45.png 1187
5a51edfa94.png 1188
9cd4f5f61f.png 1189
746f9895f3.png 1190
164f87aa84.png 1192
28e6358ab6.png 1193
4104802d8a.png 1194
367fda4d81.png 1195
5cb46bae1b.png 1197
4785815348.png 1198
337e111a56.png 1199


================================================
FILE: data/vocab.txt
================================================
!
&
'
(
)
*
+
,
-
.
/
0
1
2
3
4
5
6
7
8
9
:
;
<
=
>
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z
[
\
\!
\,
\:
\;
\Big
\Bigg
\Bigl
\Bigr
\Delta
\Gamma
\Lambda
\Omega
\Phi
\Pi
\Psi
\Rightarrow
\Sigma
\Theta
\Xi
\\
\alpha
\approx
\ast
\bar
\begin{array}
\beta
\bf
\big
\bigg
\biggl
\biggr
\bigl
\bigr
\binom
\boldmath
\cal
\cdot
\cdots
\chi
\circ
\cong
\dag
\dagger
\ddot
\delta
\displaystyle
\dot
\dots
\ell
\end{array}
\epsilon
\equiv
\eta
\forall
\frac
\gamma
\geq
\hat
\hbar
\hspace
\imath
\in
\infty
\int
\jmath
\kappa
\lambda
\langle
\lbrack
\ldots
\le
\left(
\left.
\left[
\left\langle
\left\{
\leftrightarrow
\left|
\leq
\longrightarrow
\mapsto
\mathbf
\mathcal
\mathrm
\mid
\mp
\mu
\nabla
\neq
\nonumber
\nu
\oint
\omega
\operatorname
\operatorname*
\oplus
\otimes
\overleftarrow
\overline
\overrightarrow
\partial
\perp
\phi
\pi
\pm
\prime
\prod
\propto
\psi
\qquad
\quad
\rangle
\rbrack
\rho
\right)
\right.
\right\rangle
\right\}
\right]
\rightarrow
\rightleftharpoons
\right|
\scriptsize
\scriptstyle
\sigma
\sim
\simeq
\slash
\small
\smallskip
\sp
\sqrt
\stackrel
\star
\sum
\tau
\textrm
\textstyle
\theta
\thinspace
\tilde
\times
\tiny
\to
\triangle
\underline
\varepsilon
\varphi
\varpi
\varrho
\vartheta
\vec
\vert
\vspace
\wedge
\widehat
\widetilde
\xi
\zeta
\{
\}
]
^
_
a
b
c
d
e
f
g
h
i
j
k
l
m
n
o
p
q
r
s
t
u
v
w
x
y
z
{
|
}
~

================================================
FILE: opennmt/Constants.lua
================================================
return {
  PAD = 1,
  UNK = 2,
  BOS = 3,
  EOS = 4,

  PAD_WORD = '<blank>',
  UNK_WORD = '<unk>',
  BOS_WORD = '<s>',
  EOS_WORD = '</s>'
}


================================================
FILE: opennmt/Factory.lua
================================================
local Factory = torch.class('Factory')

local options = {
  {
    '-brnn', false,
    [[Use a bidirectional encoder.]],
    {
      structural = 0
    }
  },
  {
    '-dbrnn', false,
    [[Use a deep bidirectional encoder.]],
    {
      structural = 0
    }
  },
  {
    '-pdbrnn', false,
    [[Use a pyramidal deep bidirectional encoder.]],
    {
      structural = 0
    }
  },
  {
    '-attention', 'global',
    [[Attention model.]],
    {
      enum = {'none', 'global'}
    }
  }
}

function Factory.declareOpts(cmd)
  cmd:setCmdLineOptions(options)
  onmt.BiEncoder.declareOpts(cmd)
  onmt.DBiEncoder.declareOpts(cmd)
  onmt.PDBiEncoder.declareOpts(cmd)
  onmt.GlobalAttention.declareOpts(cmd)
end

-- Return effective embeddings size based on user options.
local function resolveEmbSizes(opt, dicts, wordSizes)
  local wordEmbSize
  local featEmbSizes = {}

  wordSizes = onmt.utils.String.split(tostring(wordSizes), ',')

  if type(opt.word_vec_size) == 'number' and opt.word_vec_size > 0 then
    wordEmbSize = opt.word_vec_size
  else
    wordEmbSize = tonumber(wordSizes[1])
  end

  for i = 1, #dicts.features do
    local size

    if i + 1 <= #wordSizes then
      size = tonumber(wordSizes[i + 1])
    elseif opt.feat_merge == 'sum' then
      size = opt.feat_vec_size
    else
      size = math.floor(dicts.features[i]:size() ^ opt.feat_vec_exponent)
    end

    table.insert(featEmbSizes, size)
  end

  return wordEmbSize, featEmbSizes
end

local function buildInputNetwork(opt, dicts, wordSizes, pretrainedWords, fixWords, verbose)
  local wordEmbSize, featEmbSizes = resolveEmbSizes(opt, dicts, wordSizes)

  local wordEmbedding = onmt.WordEmbedding.new(dicts.words:size(), -- vocab size
                                               wordEmbSize,
                                               pretrainedWords,
                                               fixWords)

  local inputs
  local inputSize = wordEmbSize

  local multiInputs = #dicts.features > 0

  if multiInputs then
    inputs = nn.ParallelTable()
      :add(wordEmbedding)
  else
    inputs = wordEmbedding
  end

  if verbose then
    _G.logger:info('   - word embeddings size: ' .. wordEmbSize)
  end

  -- Sequence with features.
  if #dicts.features > 0 then
    if verbose then
      _G.logger:info('   - features embeddings sizes: ' .. table.concat(featEmbSizes, ', '))
    end

    local vocabSizes = {}
    for i = 1, #dicts.features do
      table.insert(vocabSizes, dicts.features[i]:size())
    end

    local featEmbedding = onmt.FeaturesEmbedding.new(vocabSizes, featEmbSizes, opt.feat_merge)
    inputs:add(featEmbedding)
    inputSize = inputSize + featEmbedding.outputSize
  end

  local inputNetwork

  if multiInputs then
    inputNetwork = nn.Sequential()
      :add(inputs)
      :add(nn.JoinTable(2, 2))
  else
    inputNetwork = inputs
  end

  inputNetwork.inputSize = inputSize

  return inputNetwork
end

function Factory.getOutputSizes(dicts)
  local outputSizes = { dicts.words:size() }
  for i = 1, #dicts.features do
    table.insert(outputSizes, dicts.features[i]:size())
  end
  return outputSizes
end

function Factory.buildEncoder(opt, inputNetwork, verbose)

  local function describeEncoder(name)
    if verbose then
      _G.logger:info('   - type: %s', name)
      _G.logger:info('   - structure: cell = %s; layers = %d; rnn_size = %d; dropout = ' .. opt.dropout,
                     opt.rnn_type, opt.layers, opt.rnn_size)
    end
  end

  if opt.brnn then
    describeEncoder('bidirectional')
    return onmt.BiEncoder.new(opt, inputNetwork)
  elseif opt.dbrnn then
    describeEncoder('deep bidirectional')
    return onmt.DBiEncoder.new(opt, inputNetwork)
  elseif opt.pdbrnn then
    describeEncoder('pyramidal deep bidirectional')
    return onmt.PDBiEncoder.new(opt, inputNetwork)
  else
    describeEncoder('simple')
    return onmt.Encoder.new(opt, inputNetwork)
  end

end

function Factory.buildWordEncoder(opt, dicts, verbose)
  if verbose then
    _G.logger:info(' * Encoder:')
  end

  local inputNetwork = buildInputNetwork(opt, dicts,
                                         opt.src_word_vec_size or opt.word_vec_size,
                                         opt.pre_word_vecs_enc, opt.fix_word_vecs_enc == 1,
                                         verbose)

  return Factory.buildEncoder(opt, inputNetwork, verbose)
end

function Factory.loadEncoder(pretrained, clone)
  if clone then
    pretrained = onmt.utils.Tensor.deepClone(pretrained)
  end

  local encoder

  if pretrained.name == 'Encoder' then
    encoder = onmt.Encoder.load(pretrained)
  elseif pretrained.name == 'BiEncoder' then
    encoder = onmt.BiEncoder.load(pretrained)
  elseif pretrained.name == 'PDBiEncoder' then
    encoder = onmt.PDBiEncoder.load(pretrained)
  elseif pretrained.name == 'DBiEncoder' then
    encoder = onmt.DBiEncoder.load(pretrained)
  else
    -- Keep for backward compatibility.
    local brnn = #pretrained.modules == 2
    if brnn then
      encoder = onmt.BiEncoder.load(pretrained)
    else
      encoder = onmt.Encoder.load(pretrained)
    end
  end

  return encoder
end

function Factory.buildDecoder(opt, inputNetwork, generator, attnModel, verbose)
  if verbose then
    _G.logger:info('   - structure: cell = %s; layers = %d; rnn_size = %d; dropout = ' .. opt.dropout,
                   opt.rnn_type, opt.layers, opt.rnn_size)
  end

  return onmt.Decoder.new(opt, inputNetwork, generator, attnModel)
end

function Factory.buildWordDecoder(opt, dicts, verbose)
  if verbose then
    _G.logger:info(' * Decoder:')
  end

  local inputNetwork = buildInputNetwork(opt, dicts,
                                         opt.tgt_word_vec_size or opt.word_vec_size,
                                         opt.pre_word_vecs_dec, opt.fix_word_vecs_dec == 1,
                                         verbose)

  local generator = Factory.buildGenerator(opt.rnn_size, dicts)
  local attnModel = Factory.buildAttention(opt)

  return Factory.buildDecoder(opt, inputNetwork, generator, attnModel, verbose)
end

function Factory.loadDecoder(pretrained, clone)
  if clone then
    pretrained = onmt.utils.Tensor.deepClone(pretrained)
  end

  local decoder = onmt.Decoder.load(pretrained)

  return decoder
end

function Factory.buildGenerator(rnnSize, dicts)
  if #dicts.features > 0 then
    return onmt.FeaturesGenerator(rnnSize, Factory.getOutputSizes(dicts))
  else
    return onmt.Generator(rnnSize, dicts.words:size())
  end
end

function Factory.buildAttention(args)
  if args.attention == 'none' then
    _G.logger:info('   - attention: none')
    return onmt.NoAttention(args, args.rnn_size)
  else
    _G.logger:info('   - attention: global (%s)', args.global_attention)
    return onmt.GlobalAttention(args, args.rnn_size)
  end
end

function Factory.loadGenerator(pretrained, clone)
  if clone then
    pretrained = onmt.utils.Tensor.deepClone(pretrained)
  end

  return pretrained
end

return Factory


================================================
FILE: opennmt/LanguageModel.lua
================================================
--[[ Language Model. ]]
local LanguageModel, parent = torch.class('LanguageModel', 'Model')

local options = {
  {
    '-word_vec_size', '500',
    [[Comma-separated list of embedding sizes: `word[,feat1[,feat2[,...] ] ]`.]],
    {
      structural = 0
    }
  },
  {
    '-pre_word_vecs_enc', '',
    [[Path to pretrained word embeddings on the encoder side serialized as a Torch tensor.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists,
      init_only = true
    }
  },
  {
    '-fix_word_vecs_enc', 0,
    [[Fix word embeddings on the encoder side.]],
    {
      enum = {0, 1},
      structural = 1
    }
  },
  {
    '-feat_merge', 'concat',
    [[Merge action for the features embeddings.]],
    {
      enum = {'concat', 'sum'},
      structural = 0
    }
  },
  {
    '-feat_vec_exponent', 0.7,
    [[When features embedding sizes are not set and using `-feat_merge concat`, their dimension
      will be set to `N^feat_vec_exponent` where `N` is the number of values the feature takes.]],
    {
      structural = 0
    }
  },
  {
    '-feat_vec_size', 20,
    [[When features embedding sizes are not set and using `-feat_merge sum`,
      this is the common embedding size of the features]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt(),
      structural = 0
    }
  }
}

function LanguageModel.declareOpts(cmd)
  cmd:setCmdLineOptions(options, 'Language Model')
  onmt.Encoder.declareOpts(cmd)
  onmt.Factory.declareOpts(cmd)
end

function LanguageModel:__init(args, dicts, verbose)
  parent.__init(self, args)
  onmt.utils.Table.merge(self.args, onmt.utils.ExtendedCmdLine.getModuleOpts(args, options))

  self.models.encoder = onmt.Factory.buildWordEncoder(args, dicts.src, verbose)
  self.models.generator = onmt.Factory.buildGenerator(args.rnn_size, dicts.src)

  self.criterion = onmt.ParallelClassNLLCriterion(onmt.Factory.getOutputSizes(dicts.src))

  self.eosProto = {}
  for _ = 1, #dicts.src.features + 1 do
    table.insert(self.eosProto, torch.LongTensor())
  end
end

function LanguageModel.load(args, models, dicts, isReplica)
  local self = torch.factory('LanguageModel')()

  parent.__init(self, args)
  onmt.utils.Table.merge(self.args, onmt.utils.ExtendedCmdLine.getModuleOpts(args, options))

  self.models.encoder = onmt.Factory.loadEncoder(models.encoder, isReplica)
  self.models.generator = onmt.Factory.loadGenerator(models.generator, isReplica)
  self.criterion = onmt.ParallelClassNLLCriterion(onmt.Factory.getOutputSizes(dicts.src))

  return self
end

-- Returns model name.
function LanguageModel.modelName()
  return 'Language'
end

-- Returns expected dataMode.
function LanguageModel.dataType()
  return 'monotext'
end

function LanguageModel:enableProfiling()
  _G.profiler.addHook(self.models.encoder, 'encoder')
  _G.profiler.addHook(self.models.generator, 'generator')
  _G.profiler.addHook(self.criterion, 'criterion')
end

function LanguageModel:getOutput(batch)
  return batch.sourceInput
end

function LanguageModel:forwardComputeLoss(batch)
  local _, context = self.models.encoder:forward(batch)
  local eos = onmt.utils.Tensor.reuseTensorTable(self.eosProto, { batch.size })
  for i = 1, #eos do
    eos[i]:fill(onmt.Constants.EOS)
  end

  local loss = 0

  for t = 1, batch.sourceLength do
    local genOutputs = self.models.generator:forward(context:select(2, t))

    -- LanguageModel is supposed to predict the following word.
    local output
    if t ~= batch.sourceLength then
      output = batch:getSourceInput(t + 1)
    else
      output = eos
    end

    -- Same format with and without features.
    if torch.type(output) ~= 'table' then output = { output } end

    loss = loss + self.criterion:forward(genOutputs, output)
  end

  return loss
end

function LanguageModel:trainNetwork(batch)
  local loss = 0

  local _, context = self.models.encoder:forward(batch)

  local gradContexts = context:clone():zero()
  local eos = onmt.utils.Tensor.reuseTensorTable(self.eosProto, { batch.size })
  for i = 1, #eos do
    eos[i]:fill(onmt.Constants.EOS)
  end

  -- For each word of the sentence, generate target.
  for t = 1, batch.sourceLength do
    local genOutputs = self.models.generator:forward(context:select(2, t))

    -- LanguageModel is supposed to predict following word.
    local output
    if t ~= batch.sourceLength then
      output = batch:getSourceInput(t + 1)
    else
      output = eos
    end

    -- Same format with and without features.
    if torch.type(output) ~= 'table' then output = { output } end

    loss = loss + self.criterion:forward(genOutputs, output)

    local genGradOutput = self.criterion:backward(genOutputs, output)
    for j = 1, #genGradOutput do
      genGradOutput[j]:div(batch.totalSize)
    end

    gradContexts[{{}, t}]:copy(self.models.generator:backward(context:select(2, t), genGradOutput))
  end

  self.models.encoder:backward(batch, nil, gradContexts)

  return loss
end

return LanguageModel


================================================
FILE: opennmt/Model.lua
================================================
--[[ Generic Model class. ]]
local Model = torch.class('Model')

local options = {
  {
    '-model_type', 'seq2seq',
    [[Type of model to train. This option impacts all options choices.]],
    {
      enum = {'lm', 'seq2seq', 'seqtagger'},
      structural = 0
    }
  },
  {
    '-param_init', 0.1,
    [[Parameters are initialized over uniform distribution with support (-`param_init`, `param_init`).]],
    {
      valid = function(v) return v >= 0 and v <= 1 end,
      init_only = true
    }
  }
}

function Model.declareOpts(cmd)
  cmd:setCmdLineOptions(options, 'Model')
end

function Model:__init(args)
  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)
  self.args.train_from = args.train_from
  self.models = {}
end

-- Dynamically change parameters in the graph.
function Model:changeParameters(changes)
  _G.logger:info('Applying new parameters:')

  for k, v in pairs(changes) do
    _G.logger:info(' * %s = ' .. v, k)

    for _, model in pairs(self.models) do
      model:apply(function(m)
        if k == 'dropout' and torch.typename(m) == 'nn.Dropout' then
          m:setp(v)
        elseif k:find('fix_word_vecs') and torch.typename(m) == 'onmt.WordEmbedding' then
          local enc = k == 'fix_word_vecs_enc' and torch.typename(model):find('Encoder')
          local dec = k == 'fix_word_vecs_dec' and torch.typename(model):find('Decoder')
          if enc or dec then
            m:fixEmbeddings(v == 1)
          end
        end
      end)
    end
  end

end

function Model:getInputLabelsCount(batch)
  return batch.sourceInput:ne(onmt.Constants.PAD):sum()
end

function Model:getOutputLabelsCount(batch)
  return self:getOutput(batch):ne(onmt.Constants.PAD):sum()
end

function Model:evaluate()
  for _, m in pairs(self.models) do
    m:evaluate()
  end
end

function Model:training()
  for _, m in pairs(self.models) do
    m:training()
  end
end

function Model:initParams(verbose)
  local numParams = 0
  local params = {}
  local gradParams = {}

  if verbose then
    _G.logger:info('Initializing parameters...')
  end

  -- Order the model table because we need all replicas to have the same order.
  local orderedIndex = {}
  for key in pairs(self.models) do
    table.insert(orderedIndex, key)
  end
  table.sort(orderedIndex)

  for _, key in ipairs(orderedIndex) do
    local mod = self.models[key]
    local p, gp = mod:getParameters()

    if self.args.train_from:len() == 0 then
      p:uniform(-self.args.param_init, self.args.param_init)

      mod:apply(function (m)
        if m.postParametersInitialization then
          m:postParametersInitialization()
        end
      end)
    end

    numParams = numParams + p:size(1)
    table.insert(params, p)
    table.insert(gradParams, gp)
  end

  if verbose then
    _G.logger:info(' * number of parameters: ' .. numParams)
  end

  return params, gradParams
end

return Model


================================================
FILE: opennmt/ModelSelector.lua
================================================
return function(modelType)
  if modelType == 'seq2seq' then
    return onmt.Seq2Seq
  elseif modelType == 'lm' then
    return onmt.LanguageModel
  elseif modelType == 'seqtagger' then
    return onmt.SeqTagger
  else
    error('invalid model type ' .. modelType)
  end
end


================================================
FILE: opennmt/Seq2Seq.lua
================================================
--[[ Sequence to sequence model with attention. ]]
local Seq2Seq, parent = torch.class('Seq2Seq', 'Model')

local options = {
  {
    '-word_vec_size', 0,
    [[Shared word embedding size. If set, this overrides `-src_word_vec_size` and `-tgt_word_vec_size`.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt(),
      structural = 0
    }
  },
  {
    '-src_word_vec_size', '500',
    [[Comma-separated list of source embedding sizes: `word[,feat1[,feat2[,...] ] ]`.]],
    {
      structural = 0
    }
  },
  {
    '-tgt_word_vec_size', '500',
    [[Comma-separated list of target embedding sizes: `word[,feat1[,feat2[,...] ] ]`.]],
    {
      structural = 0
    }
  },
  {
    '-pre_word_vecs_enc', '',
    [[Path to pretrained word embeddings on the encoder side serialized as a Torch tensor.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists,
      init_only = true
    }
  },
  {
    '-pre_word_vecs_dec', '',
    [[Path to pretrained word embeddings on the decoder side serialized as a Torch tensor.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists,
      init_only = true
    }
  },
  {
    '-fix_word_vecs_enc', 0,
    [[Fix word embeddings on the encoder side.]],
    {
      enum = {0, 1},
      structural = 1
    }
  },
  {
    '-fix_word_vecs_dec', 0,
    [[Fix word embeddings on the decoder side.]],
    {
      enum = {0, 1},
      structural = 1
    }
  },
  {
    '-feat_merge', 'concat',
    [[Merge action for the features embeddings.]],
    {
      enum = {'concat', 'sum'},
      structural = 0
    }
  },
  {
    '-feat_vec_exponent', 0.7,
    [[When features embedding sizes are not set and using `-feat_merge concat`, their dimension
      will be set to `N^feat_vec_exponent` where `N` is the number of values the feature takes.]],
    {
      structural = 0
    }
  },
  {
    '-feat_vec_size', 20,
    [[When features embedding sizes are not set and using `-feat_merge sum`,
      this is the common embedding size of the features]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt(),
      structural = 0
    }
  }
}

function Seq2Seq.declareOpts(cmd)
  cmd:setCmdLineOptions(options, Seq2Seq.modelName())
  onmt.Encoder.declareOpts(cmd)
  onmt.Decoder.declareOpts(cmd)
  onmt.Factory.declareOpts(cmd)
end

function Seq2Seq:__init(args, dicts, verbose)
  parent.__init(self, args)
  onmt.utils.Table.merge(self.args, onmt.utils.ExtendedCmdLine.getModuleOpts(args, options))
  self.args.uneven_batches = args.uneven_batches

  self.models.encoder = onmt.Factory.buildWordEncoder(args, dicts.src, verbose)
  self.models.decoder = onmt.Factory.buildWordDecoder(args, dicts.tgt, verbose)
  self.criterion = onmt.ParallelClassNLLCriterion(onmt.Factory.getOutputSizes(dicts.tgt))
end

function Seq2Seq.load(args, models, dicts, isReplica)
  local self = torch.factory('Seq2Seq')()

  parent.__init(self, args)
  onmt.utils.Table.merge(self.args, onmt.utils.ExtendedCmdLine.getModuleOpts(args, options))
  self.args.uneven_batches = args.uneven_batches

  self.models.encoder = onmt.Factory.loadEncoder(models.encoder, isReplica)
  self.models.decoder = onmt.Factory.loadDecoder(models.decoder, isReplica)
  self.criterion = onmt.ParallelClassNLLCriterion(onmt.Factory.getOutputSizes(dicts.tgt))

  return self
end

-- Returns model name.
function Seq2Seq.modelName()
  return 'Sequence to Sequence with Attention'
end

-- Returns expected dataMode.
function Seq2Seq.dataType()
  return 'bitext'
end

function Seq2Seq:returnIndividualLosses(enable)
  if not self.models.decoder.returnIndividualLosses then
    _G.logger:info('Current Seq2Seq model does not support training with sample_w_ppl option')
    return false
  else
    self.models.decoder:returnIndividualLosses(enable)
  end
  return true
end

function Seq2Seq:enableProfiling()
  _G.profiler.addHook(self.models.encoder, 'encoder')
  _G.profiler.addHook(self.models.decoder, 'decoder')
  _G.profiler.addHook(self.models.decoder.modules[2], 'generator')
  _G.profiler.addHook(self.criterion, 'criterion')
end

function Seq2Seq:getOutput(batch)
  return batch.targetOutput
end

function Seq2Seq:maskPadding(batch)
  if self.args.uneven_batches then
    self.models.encoder:maskPadding()
    if batch.uneven then
      self.models.decoder:maskPadding(batch.sourceSize, batch.sourceLength)
    else
      self.models.decoder:maskPadding()
    end
  end
end

function Seq2Seq:forwardComputeLoss(batch)
  self:maskPadding(batch)
  local encoderStates, context = self.models.encoder:forward(batch)
  return self.models.decoder:computeLoss(batch, encoderStates, context, self.criterion)
end

function Seq2Seq:trainNetwork(batch, dryRun)
  self:maskPadding(batch)

  local encStates, context = self.models.encoder:forward(batch)

  local decOutputs = self.models.decoder:forward(batch, encStates, context)

  if dryRun then
    decOutputs = onmt.utils.Tensor.recursiveClone(decOutputs)
  end

  local encGradStatesOut, gradContext, loss, indvLoss = self.models.decoder:backward(batch, decOutputs, self.criterion)
  self.models.encoder:backward(batch, encGradStatesOut, gradContext)

  return loss, indvLoss
end

return Seq2Seq


================================================
FILE: opennmt/SeqTagger.lua
================================================
--[[ Sequence to sequence model with attention. ]]
local SeqTagger, parent = torch.class('SeqTagger', 'Model')

local options = {
  {
    '-word_vec_size', '500',
    [[Comma-separated list of embedding sizes: `word[,feat1[,feat2[,...] ] ]`.]],
    {
      structural = 0
    }
  },
  {
    '-pre_word_vecs_enc', '',
    [[Path to pretrained word embeddings on the encoder side serialized as a Torch tensor.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists,
      init_only = true
    }
  },
  {
    '-fix_word_vecs_enc', 0,
    [[Fix word embeddings on the encoder side.]],
    {
      enum = {0, 1},
      structural = 1
    }
  },
  {
    '-feat_merge', 'concat',
    [[Merge action for the features embeddings.]],
    {
      enum = {'concat', 'sum'},
      structural = 0
    }
  },
  {
    '-feat_vec_exponent', 0.7,
    [[When features embedding sizes are not set and using `-feat_merge concat`, their dimension
      will be set to `N^feat_vec_exponent` where `N` is the number of values the feature takes.]],
    {
      structural = 0
    }
  },
  {
    '-feat_vec_size', 20,
    [[When features embedding sizes are not set and using `-feat_merge sum`,
      this is the common embedding size of the features]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt(),
      structural = 0
    }
  }
}


function SeqTagger.declareOpts(cmd)
  cmd:setCmdLineOptions(options, SeqTagger.modelName())
  onmt.Encoder.declareOpts(cmd)
  onmt.Factory.declareOpts(cmd)
end

function SeqTagger:__init(args, dicts, verbose)
  parent.__init(self, args)
  onmt.utils.Table.merge(self.args, onmt.utils.ExtendedCmdLine.getModuleOpts(args, options))

  self.models.encoder = onmt.Factory.buildWordEncoder(self.args, dicts.src, verbose)
  self.models.generator = onmt.Factory.buildGenerator(self.args.rnn_size, dicts.tgt)
  self.criterion = onmt.ParallelClassNLLCriterion(onmt.Factory.getOutputSizes(dicts.tgt))
end

function SeqTagger.load(args, models, dicts, isReplica)
  local self = torch.factory('SeqTagger')()

  parent.__init(self, args)
  onmt.utils.Table.merge(self.args, onmt.utils.ExtendedCmdLine.getModuleOpts(args, options))

  self.models.encoder = onmt.Factory.loadEncoder(models.encoder, isReplica)
  self.models.generator = onmt.Factory.loadGenerator(models.generator, isReplica)
  self.criterion = onmt.ParallelClassNLLCriterion(onmt.Factory.getOutputSizes(dicts.tgt))

  return self
end

-- Returns model name.
function SeqTagger.modelName()
  return 'Sequence Tagger'
end

-- Returns expected dataMode
function SeqTagger.dataType()
  return 'bitext'
end

function SeqTagger:enableProfiling()
  _G.profiler.addHook(self.models.encoder, 'encoder')
  _G.profiler.addHook(self.models.generator, 'generator')
  _G.profiler.addHook(self.criterion, 'criterion')
end

function SeqTagger:getOutput(batch)
  return batch.targetOutput
end

function SeqTagger:forwardComputeLoss(batch)
  local _, context = self.models.encoder:forward(batch)

  local loss = 0

  for t = 1, batch.sourceLength do
    local genOutputs = self.models.generator:forward(context:select(2, t))

    local output = batch:getTargetOutput(t)

    -- Same format with and without features.
    if torch.type(output) ~= 'table' then output = { output } end

    loss = loss + self.criterion:forward(genOutputs, output)
  end

  return loss
end

function SeqTagger:trainNetwork(batch)
  local loss = 0

  local _, context = self.models.encoder:forward(batch)

  local gradContexts = context:clone():zero()

  -- For each word of the sentence, generate target.
  for t = 1, batch.sourceLength do
    local genOutputs = self.models.generator:forward(context:select(2, t))

    local output = batch:getTargetOutput(t)

    -- Same format with and without features.
    if torch.type(output) ~= 'table' then output = { output } end

    loss = loss + self.criterion:forward(genOutputs, output)

    local genGradOutput = self.criterion:backward(genOutputs, output)
    for j = 1, #genGradOutput do
      genGradOutput[j]:div(batch.totalSize)
    end

    gradContexts[{{}, t}]:copy(self.models.generator:backward(context:select(2, t), genGradOutput))
  end

  self.models.encoder:backward(batch, nil, gradContexts)

  return loss
end

return SeqTagger


================================================
FILE: opennmt/data/AliasMultinomial.lua
================================================

--[[
 Copied with small adjustments from:
    https://github.com/nicholas-leonard/torchx/blob/master/AliasMultinomial.lua (7eeb6ae)
]]

--[[
Copyright (c) 2014, Nikopia (Nicholas Léonard)
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notices, this
  list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
  this list of conditions and the following disclaimer in the documentation
  and/or other materials provided with the distribution.

* Neither the name of the Nikopia nor the names of its
  contributors may be used to endorse or promote products derived from
  this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
]]

-- ref.: https://hips.seas.harvard.edu/blog/2013/03/03/the-alias-method-efficient-sampling-with-many-discrete-outcomes/
local AM = torch.class("AliasMultinomial")

function AM:__init(probs)
   self.J, self.q = self:setup(probs)
end

function AM:setup(probs)
   assert(probs:dim() == 1)
   local K = probs:nElement()
   local q = probs.new(K):zero()
   local J = torch.LongTensor(K):zero()

   -- Sort the data into the outcomes with probabilities
   -- that are larger and smaller than 1/K.
   local smaller, larger = {}, {}
--   local maxk, maxp = 0, -1
   for kk = 1,K do
      local prob = probs[kk]
      q[kk] = K*prob
      if q[kk] < 1 then
         table.insert(smaller, kk)
      else
         table.insert(larger, kk)
      end
--      if maxk > maxp then
--
--      end
   end

   -- Loop through and create little binary mixtures that
   -- appropriately allocate the larger outcomes over the
   -- overall uniform mixture.
   while #smaller > 0 and #larger > 0 do
      local small = table.remove(smaller)
      local large = table.remove(larger)

      J[small] = large
      q[large] = q[large] - (1.0 - q[small])

      if q[large] < 1.0 then
         table.insert(smaller,large)
      else
         table.insert(larger,large)
      end
   end
   assert(q:min() >= 0)
   if q:max() > 1 then
      q:div(q:max())
   end
   assert(q:max() <= 1)
   if J:min() <= 0 then
      -- sometimes an large index isn't added to J.
      -- fix it by making the probability 1 so that J isn't indexed.
      local i = 0
      J:apply(function(x)
         i = i + 1
         if x <= 0 then
            q[i] = 1
         end
      end)
   end
   return J, q
end

function AM:draw()
   local J = self.J
   local q = self.q
   local K  = J:nElement()

   -- Draw from the overall uniform mixture.
   local kk = math.random(1,K)

   -- Draw from the binary mixture, either keeping the
   -- small one, or choosing the associated larger one.
   if math.random() < q[kk] then
      return kk
   else
      return J[kk]
   end
end

function AM:batchdraw(output)
   assert(torch.type(output) == 'torch.LongTensor')
   assert(output:nElement() > 0)
   local J = self.J
   local K  = J:nElement()

   self._kk = self._kk or output.new()
   self._kk:resizeAs(output):random(1,K)

   self._q = self._q or self.q.new()
   self._q:index(self.q, 1, self._kk:view(-1))

   self._mask = self._b or torch.LongTensor()
   self._mask:resize(self._q:size()):bernoulli(self._q)

   self.__kk = self.__kk or output.new()
   self.__kk:resize(self._kk:size()):copy(self._kk)
   self.__kk:cmul(self._mask)

   -- if mask == 0 then output[i] = J[kk[i]] else output[i] = 0

   self._mask:add(-1):mul(-1) -- (1,0) - > (0,1)
   output:view(-1):index(J, 1, self._kk:view(-1))
   output:cmul(self._mask)

   -- elseif mask == 1 then output[i] = kk[i]

   output:add(self.__kk)

   return output
end

return AM


================================================
FILE: opennmt/data/Batch.lua
================================================
--[[ Return the maxLength, sizes, and non-zero count
  of a batch of `seq`s ignoring `ignore` words.
--]]
local function getLength(seq, ignore)
  local sizes = torch.IntTensor(#seq):zero()
  local max = 0
  local uneven = false

  for i = 1, #seq do
    local len = seq[i]:size(1)
    if ignore ~= nil then
      len = len - ignore
    end
    if max == 0 or len > max then
      max = len
    end
    if i > 1 and sizes[i - 1] ~= len then
      uneven = true
    end
    sizes[i] = len
  end
  return max, sizes, uneven
end

--[[ Data management and batch creation.

Batch interface reference [size]:

  * size: number of sentences in the batch [1]
  * sourceLength: max length in source batch [1]
  * sourceSize:  lengths of each source [batch x 1]
  * sourceInput:  left-padded idx's of source (PPPPPPABCDE) [batch x max]
  * sourceInputFeatures: table of source features sequences
  * sourceInputRev: right-padded  idx's of source rev (EDCBAPPPPPP) [batch x max]
  * sourceInputRevFeatures: table of reversed source features sequences
  * targetLength: max length in source batch [1]
  * targetSize: lengths of each source [batch x 1]
  * targetInput: input idx's of target (SABCDEPPPPPP) [batch x max]
  * targetInputFeatures: table of target input features sequences
  * targetOutput: expected output idx's of target (ABCDESPPPPPP) [batch x max]
  * targetOutputFeatures: table of target output features sequences

 TODO: change name of size => maxlen
--]]


--[[ A batch of sentences to translate and targets. Manages padding,
  features, and batch alignment (for efficiency).

  Used by the decoder and encoder objects.
--]]
local Batch = torch.class('Batch')

--[[ Create a batch object.

Parameters:

  * `src` - 2D table of source batch indices
  * `srcFeatures` - 2D table of source batch features (opt)
  * `tgt` - 2D table of target batch indices
  * `tgtFeatures` - 2D table of target batch features (opt)
--]]
function Batch:__init(src, srcFeatures, tgt, tgtFeatures)
  src = src or {}
  srcFeatures = srcFeatures or {}
  tgtFeatures = tgtFeatures or {}

  if tgt ~= nil then
    assert(#src == #tgt, "source and target must have the same batch size")
  end

  self.size = #src
  self.totalSize = self.size -- updated when this batch is part of a larger one (data parallelism).

  self.sourceLength, self.sourceSize, self.uneven = getLength(src)

  local sourceSeq = torch.LongTensor(self.sourceLength, self.size):fill(onmt.Constants.PAD)
  self.sourceInput = sourceSeq:clone()
  self.sourceInputRev = sourceSeq:clone()

  self.sourceInputFeatures = {}
  self.sourceInputRevFeatures = {}
  -- will be used to return extra padded value
  self.padTensor = torch.LongTensor(self.size):fill(onmt.Constants.PAD)

  if #srcFeatures > 0 then
    for _ = 1, #srcFeatures[1] do
      table.insert(self.sourceInputFeatures, sourceSeq:clone())
      table.insert(self.sourceInputRevFeatures, sourceSeq:clone())
    end
  end

  if tgt ~= nil then
    self.targetLength, self.targetSize = getLength(tgt, 1)

    local targetSeq = torch.LongTensor(self.targetLength, self.size):fill(onmt.Constants.PAD)
    self.targetInput = targetSeq:clone()
    self.targetOutput = targetSeq:clone()

    self.targetInputFeatures = {}
    self.targetOutputFeatures = {}

    if #tgtFeatures > 0 then
      for _ = 1, #tgtFeatures[1] do
        table.insert(self.targetInputFeatures, targetSeq:clone())
        table.insert(self.targetOutputFeatures, targetSeq:clone())
      end
    end
  end

  for b = 1, self.size do
    local sourceOffset = self.sourceLength - self.sourceSize[b] + 1
    local sourceInput = src[b]
    local sourceInputRev = src[b]:index(1, torch.linspace(self.sourceSize[b], 1, self.sourceSize[b]):long())

    -- Source input is left padded [PPPPPPABCDE] .
    self.sourceInput[{{sourceOffset, self.sourceLength}, b}]:copy(sourceInput)
    self.sourceInputPadLeft = true

    -- Rev source input is right padded [EDCBAPPPPPP] .
    self.sourceInputRev[{{1, self.sourceSize[b]}, b}]:copy(sourceInputRev)
    self.sourceInputRevPadLeft = false

    for i = 1, #self.sourceInputFeatures do
      local sourceInputFeatures = srcFeatures[b][i]
      local sourceInputRevFeatures = srcFeatures[b][i]:index(1, torch.linspace(self.sourceSize[b], 1, self.sourceSize[b]):long())

      self.sourceInputFeatures[i][{{sourceOffset, self.sourceLength}, b}]:copy(sourceInputFeatures)
      self.sourceInputRevFeatures[i][{{1, self.sourceSize[b]}, b}]:copy(sourceInputRevFeatures)
    end

    if tgt ~= nil then
      -- Input: [<s>ABCDE]
      -- Ouput: [ABCDE</s>]
      local targetLength = tgt[b]:size(1) - 1
      local targetInput = tgt[b]:narrow(1, 1, targetLength)
      local targetOutput = tgt[b]:narrow(1, 2, targetLength)

      -- Target is right padded [<S>ABCDEPPPPPP] .
      self.targetInput[{{1, targetLength}, b}]:copy(targetInput)
      self.targetOutput[{{1, targetLength}, b}]:copy(targetOutput)

      for i = 1, #self.targetInputFeatures do
        local targetInputFeatures = tgtFeatures[b][i]:narrow(1, 1, targetLength)
        local targetOutputFeatures = tgtFeatures[b][i]:narrow(1, 2, targetLength)

        self.targetInputFeatures[i][{{1, targetLength}, b}]:copy(targetInputFeatures)
        self.targetOutputFeatures[i][{{1, targetLength}, b}]:copy(targetOutputFeatures)
      end
    end
  end
end

--[[ Set source input directly,

Parameters:

  * `sourceInput` - a Tensor of size (sequence_length, batch_size, feature_dim)
  ,or a sequence of size (sequence_length, batch_size). Be aware that sourceInput is not cloned here.

--]]
function Batch:setSourceInput(sourceInput)
  assert (sourceInput:dim() >= 2, 'The sourceInput tensor should be of size (seq_len, batch_size, ...)')
  self.size = sourceInput:size(2)
  self.sourceLength = sourceInput:size(1)
  self.sourceInputFeatures = {}
  self.sourceInputRevReatures = {}
  self.sourceInput = sourceInput
  self.sourceInputRev = self.sourceInput:index(1, torch.linspace(self.sourceLength, 1, self.sourceLength):long())
  return self
end

--[[ Set target input directly.

Parameters:

  * `targetInput` - a tensor of size (sequence_length, batch_size). Padded with onmt.Constants.PAD. Be aware that targetInput is not cloned here.
--]]
function Batch:setTargetInput(targetInput)
  assert (targetInput:dim() == 2, 'The targetInput tensor should be of size (seq_len, batch_size)')
  self.targetInput = targetInput
  self.size = targetInput:size(2)
  self.totalSize = self.size
  self.targetLength = targetInput:size(1)
  self.targetInputFeatures = {}
  self.targetSize = torch.sum(targetInput:transpose(1,2):ne(onmt.Constants.PAD), 2):view(-1):double()
  return self
end

--[[ Set target output directly.

Parameters:

  * `targetOutput` - a tensor of size (sequence_length, batch_size). Padded with onmt.Constants.PAD.  Be aware that targetOutput is not cloned here.
--]]
function Batch:setTargetOutput(targetOutput)
  assert (targetOutput:dim() == 2, 'The targetOutput tensor should be of size (seq_len, batch_size)')
  self.targetOutput = targetOutput
  self.targetOutputFeatures = {}
  return self
end

local function addInputFeatures(inputs, featuresSeq, t)
  local features = {}
  for j = 1, #featuresSeq do
    local feat
    if t > featuresSeq[j]:size(1) then
      feat = onmt.Constants.PAD
    else
      feat = featuresSeq[j][t]
    end
    table.insert(features, feat)
  end
  if #features > 1 then
    table.insert(inputs, features)
  else
    onmt.utils.Table.append(inputs, features)
  end
end

--[[ Get source input batch at timestep `t`. --]]
function Batch:getSourceInput(t)
  local inputs

  -- If a regular input, return word id, otherwise a table with features.
  if t > self.sourceInput:size(1) then
    inputs = self.padTensor
  else
    inputs = self.sourceInput[t]
  end

  if #self.sourceInputFeatures > 0 then
    inputs = { inputs }
    addInputFeatures(inputs, self.sourceInputFeatures, t)
  end

  return inputs
end

--[[ Get target input batch at timestep `t`. --]]
function Batch:getTargetInput(t)
  -- If a regular input, return word id, otherwise a table with features.
  local inputs = self.targetInput[t]

  if #self.targetInputFeatures > 0 then
    inputs = { inputs }
    addInputFeatures(inputs, self.targetInputFeatures, t)
  end

  return inputs
end

--[[ Get target output batch at timestep `t` (values t+1). --]]
function Batch:getTargetOutput(t)
  -- If a regular input, return word id, otherwise a table with features.
  local outputs = { self.targetOutput[t] }

  for j = 1, #self.targetOutputFeatures do
    table.insert(outputs, self.targetOutputFeatures[j][t])
  end

  return outputs
end

return Batch


================================================
FILE: opennmt/data/BatchTensor.lua
================================================
local BatchTensor = torch.class('BatchTensor')

--[[
  Take Batch x TimeStep x layer size tensors
]]
function BatchTensor:__init(T, sizes)
  self.size = T:size()[1]
  self.sourceLength = T:size()[2]

  self.sourceSize = sizes or torch.LongTensor(self.size):fill(self.sourceLength)

  self.sourceInput = T
  self.sourceInputPadLeft = true

  self.sourceInputRev = self.sourceInput
    :index(2, torch.linspace(self.sourceLength, 1, self.sourceLength):long())
  self.sourceInputRevPadLeft = false
end

function BatchTensor:getSourceInput(t)
  return self.sourceInput:select(2, t)
end

return BatchTensor


================================================
FILE: opennmt/data/Dataset.lua
================================================
--[[ Data management and batch creation. Handles data created by `preprocess.lua`. ]]
local Dataset = torch.class("Dataset")

--[[ Initialize a data object given aligned tables of IntTensors `srcData`
  and `tgtData`.
--]]
function Dataset:__init(srcData, tgtData)

  self.src = srcData.words
  self.srcFeatures = srcData.features

  if tgtData ~= nil then
    self.tgt = tgtData.words
    self.tgtFeatures = tgtData.features
  end
end

--[[ Setup up the training data to respect `maxBatchSize`.
     If uneven_batches - then build up batches with different lengths ]]
function Dataset:setBatchSize(maxBatchSize, uneven_batches)

  self.batchRange = {}
  self.maxSourceLength = 0
  self.maxTargetLength = 0

  local batchesCapacity = 0
  local batchesOccupation = 0

  -- Prepares batches in terms of range within self.src and self.tgt.
  local offset = 0
  local batchSize = 1
  local maxSourceLength = 0
  local targetLength = 0

  for i = 1, #self.src do
    -- Set up the offsets to make same source size batches of the
    -- correct size.
    local sourceLength = self.src[i]:size(1)
    if batchSize == maxBatchSize or i == 1 or
        (not(uneven_batches) and self.src[i]:size(1) ~= maxSourceLength) then
      if i > 1 then
        batchesCapacity = batchesCapacity + batchSize * maxSourceLength
        table.insert(self.batchRange, { ["begin"] = offset, ["end"] = i - 1 })
      end

      offset = i
      batchSize = 1
      targetLength = 0
      maxSourceLength = 0
    else
      batchSize = batchSize + 1
    end
    batchesOccupation = batchesOccupation + sourceLength
    maxSourceLength = math.max(maxSourceLength, sourceLength)

    self.maxSourceLength = math.max(self.maxSourceLength, sourceLength)

    if self.tgt ~= nil then
      -- Target contains <s> and </s>.
      local targetSeqLength = self.tgt[i]:size(1) - 1
      targetLength = math.max(targetLength, targetSeqLength)
      self.maxTargetLength = math.max(self.maxTargetLength, targetSeqLength)
    end
  end

  -- Catch last batch.
  batchesCapacity = batchesCapacity + batchSize * maxSourceLength
  table.insert(self.batchRange, { ["begin"] = offset, ["end"] = #self.src })
  return #self.batchRange, batchesOccupation/batchesCapacity
end

--[[ Return number of batches. ]]
function Dataset:batchCount()
  if self.batchRange == nil then
    if #self.src > 0 then
      return 1
    else
      return 0
    end
  end
  return #self.batchRange
end

--[[ Get `Batch` number `idx`. If nil make a batch of all the data. ]]
function Dataset:getBatch(idx)
  if #self.src == 0 then
    return nil
  end

  if idx == nil or self.batchRange == nil then
    return onmt.data.Batch.new(self.src, self.srcFeatures, self.tgt, self.tgtFeatures)
  end

  local rangeStart = self.batchRange[idx]["begin"]
  local rangeEnd = self.batchRange[idx]["end"]

  local src = {}
  local tgt

  if self.tgt ~= nil then
    tgt = {}
  end

  local srcFeatures = {}
  local tgtFeatures = {}

  for i = rangeStart, rangeEnd do
    table.insert(src, self.src[i])

    if self.srcFeatures[i] then
      table.insert(srcFeatures, self.srcFeatures[i])
    end

    if self.tgt ~= nil then
      table.insert(tgt, self.tgt[i])

      if self.tgtFeatures[i] then
        table.insert(tgtFeatures, self.tgtFeatures[i])
      end
    end
  end

  return onmt.data.Batch.new(src, srcFeatures, tgt, tgtFeatures)
end

return Dataset


================================================
FILE: opennmt/data/Preprocessor.lua
================================================
--[[ Data Preparation functions. ]]

local function vecToTensor(vec)
  local t = torch.Tensor(#vec)
  for i = 1, #vec do
    t[i] = vec[i]
  end
  return t
end

local Preprocessor = torch.class('Preprocessor')
local tds

local bitextOptions = {
  {
    '-train_src', '',
    [[Path to the training source data.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileExists
    }
  },
  {
    '-train_tgt', '',
    [[Path to the training target data.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileExists
    }
  },
  {
    '-valid_src', '',
    [[Path to the validation source data.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileExists
    }
  },
  {
    '-valid_tgt', '',
    [[Path to the validation target data.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileExists
    }
  },
  {
    '-src_vocab', '',
    [[Path to an existing source vocabulary.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists
    }
  },
  {
    '-tgt_vocab', '',
    [[Path to an existing target vocabulary.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists
    }
  },
  {
    '-src_vocab_size', '50000',
    [[Comma-separated list of source vocabularies size: `word[,feat1[,feat2[,...] ] ]`.
      If = 0, vocabularies are not pruned.]],
    {
      valid = onmt.utils.ExtendedCmdLine.listUInt
    }
  },
  {
    '-tgt_vocab_size', '50000',
    [[Comma-separated list of target vocabularies size: `word[,feat1[,feat2[,...] ] ]`.
      If = 0, vocabularies are not pruned.]],
    {
      valid = onmt.utils.ExtendedCmdLine.listUInt
    }
  },
  {
    '-src_words_min_frequency', '0',
    [[Comma-separated list of source words min frequency: `word[,feat1[,feat2[,...] ] ]`.
      If = 0, vocabularies are pruned by size.]],
    {
      valid=onmt.utils.ExtendedCmdLine.listUInt
    }
  },
  {
    '-tgt_words_min_frequency', '0',
    [[Comma-separated list of target words min frequency: `word[,feat1[,feat2[,...] ] ]`.
      If = 0, vocabularies are pruned by size.]],
    {
      valid=onmt.utils.ExtendedCmdLine.listUInt
    }
  },
  {
    '-src_seq_length', 50,
    [[Maximum source sequence length.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt
    }
  },
  {
    '-tgt_seq_length', 50,
    [[Maximum target sequence length.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt
    }
  }
}

local monotextOptions = {
  {
    '-train', '',
    [[Path to the training source data.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileExists
    }
  },
  {
    '-valid', '',
    [[Path to the validation source data.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileExists
    }
  },
  {
    '-vocab', '',
    [[Path to an existing source vocabulary.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists
    }
  },
  {
    '-vocab_size', '50000',
    [[Comma-separated list of source vocabularies size: `word[,feat1[,feat2[,...] ] ]`.
      If = 0, vocabularies are not pruned.]],
    {
      valid=onmt.utils.ExtendedCmdLine.listUInt
    }
  },
  {
    '-words_min_frequency', '0',
    [[Comma-separated list of source words min frequency: `word[,feat1[,feat2[,...] ] ]`.
      If = 0, vocabularies are pruned by size.]],
    {
      valid = onmt.utils.ExtendedCmdLine.listUInt
    }
  },
  {
    '-seq_length', 50,
    [[Maximum source sequence length.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt()
    }
  }
}

local commonOptions = {
  {
    '-features_vocabs_prefix', '',
    [[Path prefix to existing features vocabularies.]]
  },
  {
    '-time_shift_feature', 1,
    [[Time shift features on the decoder side.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isInt(0, 1)
    }
  },
  {
    '-sort', 1,
    [[If = 1, sort the sentences by size to build batches without source padding.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isInt(0, 1)
    }
  },
  {
    '-shuffle', 1,
    [[If = 1, shuffle data (prior sorting).]],
    {
      valid = onmt.utils.ExtendedCmdLine.isInt(0,1)
    }
  },
  {
    '-report_every', 100000,
    [[Report status every this many sentences.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt()
    }
  }
}

function Preprocessor.declareOpts(cmd, mode)
  mode = mode or 'bitext'
  local options
  if mode == 'bitext' then
    options = bitextOptions
  else
    options = monotextOptions
  end
  for _, v in ipairs(commonOptions) do
    table.insert(options, v)
  end
  cmd:setCmdLineOptions(options, 'Data')
end

function Preprocessor:__init(args, mode)
  tds = require('tds')

  mode = mode or 'bitext'
  local options
  if mode == 'bitext' then
    options = bitextOptions
  else
    options = monotextOptions
  end
  for _, v in ipairs(commonOptions) do
    table.insert(options, v)
  end
  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)
end

function Preprocessor:makeBilingualData(srcFile, tgtFile, srcDicts, tgtDicts, isValid)
    -- sentence length distribution
  local srcSentenceDist = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}
  local tgtSentenceDist = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}

  local src = tds.Vec()
  local srcFeatures = tds.Vec()

  local tgt = tds.Vec()
  local tgtFeatures = tds.Vec()

  local sizes = tds.Vec()

  local count = 0
  local ignored = 0
  local avgSrcLength = 0
  local avgTgtLength = 0
  local prunedRatioSrc = 0
  local prunedRatioTgt = 0

  local srcReader = onmt.utils.FileReader.new(srcFile)
  local tgtReader = onmt.utils.FileReader.new(tgtFile)

  while true do
    local srcTokens = srcReader:next()
    local tgtTokens = tgtReader:next()

    if srcTokens == nil or tgtTokens == nil then
      if srcTokens == nil and tgtTokens ~= nil or srcTokens ~= nil and tgtTokens == nil then
        _G.logger:warning('source and target do not have the same number of sentences')
      end
      break
    end

    local idxRange = math.floor(#srcTokens/10)+1
    if idxRange > #srcSentenceDist then
      idxRange = #srcSentenceDist
    end
    srcSentenceDist[idxRange] = srcSentenceDist[idxRange]+1
    idxRange = math.floor(#tgtTokens/10)+1
    if idxRange > #tgtSentenceDist then
      idxRange = #tgtSentenceDist
    end
    tgtSentenceDist[idxRange] = tgtSentenceDist[idxRange]+1

    if isValid(srcTokens, self.args.src_seq_length) and isValid(tgtTokens, self.args.tgt_seq_length) then
      avgSrcLength = avgSrcLength * (#src / (#src + 1)) + #srcTokens / (#src + 1)
      avgTgtLength = avgTgtLength * (#tgt / (#tgt + 1)) + #tgtTokens / (#tgt + 1)

      local srcWords, srcFeats = onmt.utils.Features.extract(srcTokens)
      local tgtWords, tgtFeats = onmt.utils.Features.extract(tgtTokens)

      local srcVec = srcDicts.words:convertToIdx(srcWords, onmt.Constants.UNK_WORD)
      local tgtVec = tgtDicts.words:convertToIdx(tgtWords,
                                                 onmt.Constants.UNK_WORD,
                                                 onmt.Constants.BOS_WORD,
                                                 onmt.Constants.EOS_WORD)

      local srcPruned = srcVec:eq(onmt.Constants.UNK):sum() / srcVec:size(1)
      local tgtPruned = tgtVec:eq(onmt.Constants.UNK):sum() / tgtVec:size(1)

      prunedRatioSrc = prunedRatioSrc * (#src / (#src + 1)) + srcPruned / (#src + 1)
      prunedRatioTgt = prunedRatioTgt * (#tgt / (#tgt + 1)) + tgtPruned / (#tgt + 1)

      src:insert(srcVec)
      tgt:insert(tgtVec)

      if #srcDicts.features > 0 then
        srcFeatures:insert(onmt.utils.Features.generateSource(srcDicts.features, srcFeats, true))
      end
      if #tgtDicts.features > 0 then
        tgtFeatures:insert(onmt.utils.Features.generateTarget(tgtDicts.features, tgtFeats, true, self.args.time_shift_feature))
      end

      sizes:insert(#srcWords)
    else
      ignored = ignored + 1
    end

    count = count + 1

    if count % self.args.report_every == 0 then
      _G.logger:info('... ' .. count .. ' sentences prepared')
    end
  end

  for i=1, #srcSentenceDist do
    srcSentenceDist[i] = srcSentenceDist[i]/count
    tgtSentenceDist[i] = tgtSentenceDist[i]/count
  end


  srcReader:close()
  tgtReader:close()

  local function reorderData(perm)
    src = onmt.utils.Table.reorder(src, perm, true)
    tgt = onmt.utils.Table.reorder(tgt, perm, true)

    if #srcDicts.features > 0 then
      srcFeatures = onmt.utils.Table.reorder(srcFeatures, perm, true)
    end
    if #tgtDicts.features > 0 then
      tgtFeatures = onmt.utils.Table.reorder(tgtFeatures, perm, true)
    end
  end

  if self.args.shuffle == 1 then
    _G.logger:info('... shuffling sentences')
    local perm = torch.randperm(#src)
    sizes = onmt.utils.Table.reorder(sizes, perm, true)
    reorderData(perm)
  end

  if self.args.sort == 1 then
    _G.logger:info('... sorting sentences by size')
    local _, perm = torch.sort(vecToTensor(sizes))
    reorderData(perm)
  end

  _G.logger:info('Prepared %d sentences:', #src)
  _G.logger:info(' * %d sequences ignored due to source length > %d or target length > %d',
                 ignored,
                 self.args.src_seq_length,
                 self.args.tgt_seq_length)
  _G.logger:info(' * average sequence length: source = %.1f, target = %.1f',
                 avgSrcLength,
                 avgTgtLength)
  _G.logger:info(' * %% of unkown words: source = %.1f%%, target = %.1f%%',
                 prunedRatioSrc * 100,
                 prunedRatioTgt * 100)

  local dist='[ '
  for i=1,#srcSentenceDist do
    if i>1 then
      dist = dist..' ; '
    end
    dist = dist..math.floor(srcSentenceDist[i]*100)..'%'
  end
  dist = dist..' ]'
  _G.logger:info(' * Source Sentence Length (range of 10): '..dist)
  dist='[ '
  for i=1,#tgtSentenceDist do
    if i>1 then
      dist = dist..' ; '
    end
    dist = dist..math.floor(tgtSentenceDist[i]*100)..'%'
  end
  dist = dist..' ]'
  _G.logger:info(' * Target Sentence Length (range of 10): '..dist)

  local srcData = {
    words = src,
    features = srcFeatures
  }

  local tgtData = {
    words = tgt,
    features = tgtFeatures
  }

  return srcData, tgtData
end

function Preprocessor:makeMonolingualData(file, dicts, isValid)
  local dataset = tds.Vec()
  local features = tds.Vec()

  local sizes = tds.Vec()

  local count = 0
  local ignored = 0
  local avgLength = 0
  local prunedRatio = 0

  local reader = onmt.utils.FileReader.new(file)

  while true do
    local tokens = reader:next()

    if tokens == nil then
      break
    end

    if isValid(tokens, self.args.seq_length) then
      avgLength = avgLength * (#dataset / (#dataset + 1)) + #tokens / (#dataset + 1)

      local words, feats = onmt.utils.Features.extract(tokens)
      local vec = dicts.words:convertToIdx(words, onmt.Constants.UNK_WORD)
      local pruned = vec:eq(onmt.Constants.UNK):sum() / vec:size(1)

      prunedRatio = prunedRatio * (#dataset / (#dataset + 1)) + pruned / (#dataset + 1)

      dataset:insert(vec)

      if #dicts.features > 0 then
        features:insert(onmt.utils.Features.generateSource(dicts.features, feats, true))
      end

      sizes:insert(#words)
    else
      ignored = ignored + 1
    end

    count = count + 1

  end

  reader:close()

  local function reorderData(perm)
    dataset = onmt.utils.Table.reorder(dataset, perm, true)

    if #dicts.features > 0 then
      features = onmt.utils.Table.reorder(features, perm, true)
    end
  end

  if self.args.shuffle == 1 then
    _G.logger:info('... shuffling sentences')
    local perm = torch.randperm(#dataset)
    sizes = onmt.utils.Table.reorder(sizes, perm, true)
    reorderData(perm)
  end

  if self.args.sort == 1 then
    _G.logger:info('... sorting sentences by size')
    local _, perm = torch.sort(vecToTensor(sizes))
    reorderData(perm)
  end

  _G.logger:info('Prepared %d sentences:', #dataset)
  _G.logger:info(' * %d sequences ignored due to length > %d', ignored, self.args.seq_length)
  _G.logger:info(' * average sequence length = %.1f', avgLength)
  _G.logger:info(' * %% of unkown words = %.1f%%', prunedRatio * 100)

  local data = {
    words = dataset,
    features = features
  }

  return data
end

return Preprocessor


================================================
FILE: opennmt/data/SampledDataset.lua
================================================
--[[ Data management and batch creation. Handles data created by `preprocess.lua`. ]]

local SampledDataset = torch.class("SampledDataset")

local options = {
  {
    '-sample', 0,
    [[Number of instances to sample from train data in each epoch.]]
  },
  {
    '-sample_w_ppl', false,
    [[If set, ese perplexity as a probability distribution when sampling.]]
  },
  {
    '-sample_w_ppl_init', 15,
    [[Start perplexity-based sampling when average train perplexity per batch
      falls below this value.]]
  },
  {
    '-sample_w_ppl_max', -1.5,
    [[When greater than 0, instances with perplexity above this value will be
      considered as noise and ignored; when less than 0, mode + `-sample_w_ppl_max` * stdev
      will be used as threshold.]]
  }
}

function SampledDataset.declareOpts(cmd)
  cmd:setCmdLineOptions(options, 'Sampled dataset')
end

--[[ Initialize a data object given aligned tables of IntTensors `srcData`
  and `tgtData`.
--]]
function SampledDataset:__init(srcData, tgtData, opt)

  self.src = srcData.words or srcData.vectors
  self.srcFeatures = srcData.features

  if tgtData ~= nil then
    self.tgt = tgtData.words  or tgtData.vectors
    self.tgtFeatures = tgtData.features
  end

  self.samplingSize = opt.sample
  self.sample_w_ppl = opt.sample_w_ppl
  self.sample_w_ppl_init = opt.sample_w_ppl_init
  self.sample_w_ppl_max = opt.sample_w_ppl_max
  self.startedPplSampling = false

  _G.logger:info(' * sampling ' .. opt.sample .. ' instances from ' .. #self.src .. ' at each epoch')
  if opt.sample_w_ppl then
    _G.logger:info(' * using train data perplexity as probability distribution when sampling')
    _G.logger:info(' * sample_w_ppl_init: ' .. opt.sample_w_ppl_init)
    _G.logger:info(' * sample_w_ppl_max: ' .. opt.sample_w_ppl_max)
  end

  if self.sample_w_ppl then
    self.samplingProb = torch.Tensor(#self.src)
    self.samplingProb:fill(self.sample_w_ppl_init)
    self.ppl = torch.Tensor(#self.src)
    self.ppl:fill(self.sample_w_ppl_init)
  else
    self.samplingProb = torch.ones(#self.src)
  end

  self.sampled = nil
  self.sampledCnt = torch.zeros(#self.src)
end

function SampledDataset:checkModel(model)
  if self:needIndividualLosses() and (not model.returnIndividualLosses or model:returnIndividualLosses(true) == false) then
    _G.logger:info('Current model does not support training with invididual losses; Sampling with individual loss will be disabled.')
    self.sample_w_ppl = false
    self.samplingProb = torch.ones(#self.src)
    self.ppl = nil
  else
    _G.logger:info('Current sampling does not require individual loss')
    if model.returnIndividualLosses then
      model:returnIndividualLosses(false)
    end
  end
end

function SampledDataset:needIndividualLosses()
  return self.sample_w_ppl
end

--[[ Initiate sampling. ]]
function SampledDataset:sample(logLevel)

  logLevel = logLevel or 'INFO'

  _G.logger:log('Sampling...', logLevel)

  -- Populate self.samplingProb with self.ppl if average ppl is below self.sample_w_ppl_init.
  if self.sample_w_ppl and not self.startedPplSampling then
    local avgPpl = torch.sum(self.ppl)
    avgPpl = avgPpl / self.ppl:size(1)
    if avgPpl < self.sample_w_ppl_init then
      _G.logger:log('Beginning to sample with ppl as probability distribution...', logLevel)
      self.startedPplSampling = true
    end
  end

  if self.startedPplSampling then
    local threshold = self.sample_w_ppl_max

    if self.sample_w_ppl_max < 0 then
      -- Use mode (instead of mean) and stdev of samples with ppl >= mode to
      -- find max ppl to consider (mode + x * stdev). when x is:
      --      x: 1 ~ 100% - 31.7%/2 of train data are not included (divide by 2 because we cut only one-tail)
      --      x: 2 ~ 100% - 4.55%/2
      --      x: 3 ~ 100% - 0.270%/2
      --      x: 4 ~ 100% - 0.00633%/2
      --      x: 5 ~ 100% - 0.0000573%/2
      --      x: 6 ~ 100% - 0.000000197%/2
      --  (https://en.wikipedia.org/wiki/Standard_deviation)
      -- We are using mode instead of average, and only samples above mode to calculate stdev, so
      -- this is not really theoretically valid numbers, but more for emperical uses.

      local x = math.abs(self.sample_w_ppl_max)

      -- Find mode.
      local pplRounded = torch.round(self.ppl * 100) / 100 -- keep up to the second decimal point
      local bin = {}
      for i = 1, pplRounded:size(1) do
        if self.ppl[i] ~=  self.sample_w_ppl_init then
          local idx = pplRounded[i]
          if bin[idx] == nil then
            bin[idx] = 0
          end
          bin[idx] = bin[idx] + 1
        end
      end
      local mode = nil
      for key, value in pairs(bin) do
        if mode == nil or bin[mode] < value then
          mode = key
        end
      end

      -- stdev with mode only using samples with ppl >= mode
      local sum = 0
      local cnt = 0
      for i = 1, self.ppl:size(1) do
        if self.ppl[i] > mode and self.ppl[i] ~= self.sample_w_ppl_init then
          sum = math.pow(self.ppl[i] - mode, 2)
          cnt = cnt + 1
        end
      end
      local stdev = math.sqrt(sum / (cnt - 1))

      threshold = mode + x * stdev

      _G.logger:log('Sampler count: ' .. cnt, logLevel)
      _G.logger:log('Sampler mode: ' .. mode, logLevel)
      _G.logger:log('Sampler stdev: ' .. stdev, logLevel)
      _G.logger:log('Sampler threshold: ' .. threshold, logLevel)
    end

    for i = 1, self.ppl:size(1) do
      if self.ppl[i] ~= self.sample_w_ppl_init and self.ppl[i] > threshold then
        -- Assign low value to instances with ppl above threshold (outliers).
        self.samplingProb[i] = 1
      else
        self.samplingProb[i] = self.ppl[i]
      end
    end
  end

  local sampler = onmt.data.AliasMultinomial.new(self.samplingProb)
  _G.logger:log('Created sampler...', logLevel)
  self.sampled = torch.LongTensor(self.samplingSize)
  self.sampled = sampler:batchdraw(self.sampled)

  self.sampledCnt:zero()
  for i = 1, self.sampled:size(1) do
    self.sampledCnt[self.sampled[i]] = self.sampledCnt[self.sampled[i]] + 1
  end

  _G.logger:log('Sampled ' .. self.sampled:size(1) .. ' instances', logLevel)

  -- Prepares batches in terms of range within self.src and self.tgt.
  local batchesCapacity = 0
  local batchesOccupation = 0
  self.batchRange = {}
  local offset = 0
  local sampleCntBegin = 1
  local batchSize = 1
  local maxSourceLength = -1
  for i = 1, #self.src do
    for j = 1, self.sampledCnt[i] do
      local sourceLength = self.src[i]:size(1)
      if batchSize == self.maxBatchSize or offset == 1 or
         (not(self.uneven_batches) and self.src[i]:size(1) ~= maxSourceLength) then
        if offset > 0 then
          batchesCapacity = batchesCapacity + batchSize * maxSourceLength
          local batchEnd = (j == 1) and i - 1 or i
          local sampleCntEnd = (j == 1) and self.sampledCnt[i - 1] or j - 1
          table.insert(self.batchRange, {
            ["begin"] = offset,
            ["end"] = batchEnd,
            ["sampleCntBegin"] = sampleCntBegin,
            ["sampleCntEnd"] = sampleCntEnd
          })
          sampleCntBegin = (j == 1) and 1 or j
        end
        offset = i
        batchSize = 1
        maxSourceLength = -1
      else
        batchSize = batchSize + 1
      end
      batchesOccupation = batchesOccupation + sourceLength
      maxSourceLength = math.max(maxSourceLength, sourceLength)
    end
  end
  -- Catch last batch.
  if offset < #self.src then
    batchesCapacity = batchesCapacity + batchSize * maxSourceLength
    table.insert(self.batchRange, {
      ["begin"] = offset,
      ["end"] = #self.src,
      ["sampleCntBegin"] = sampleCntBegin,
      ["sampleCntEnd"] = self.sampledCnt[#self.src]
    })
  end

  _G.logger:log('Prepared ' .. #self.batchRange .. ' batches', logLevel)
  return #self.batchRange, batchesOccupation / batchesCapacity
end

--[[ Get perplexity. ]]
function SampledDataset:getPpl()
  return self.ppl
end

--[[ Set perplexity. ]]
function SampledDataset:setLoss(batchIdx, loss)
  assert(self:batchCount() >= batchIdx, "Batch idx out of range: " .. batchIdx .. "/" .. self:batchCount())
  local rangeStart = self.batchRange[batchIdx]["begin"]
  local rangeEnd = self.batchRange[batchIdx]["end"]
  local sampleCntBegin = self.batchRange[batchIdx]["sampleCntBegin"]
  local sampleCntEnd =  self.batchRange[batchIdx]["sampleCntEnd"]
  loss = loss:exp()
  local pplIdx = 1
  for i = rangeStart, rangeEnd do
    local jBegin = (i == rangeStart) and sampleCntBegin or 1
    local jEnd = (i == rangeEnd) and math.min(self.sampledCnt[i], sampleCntEnd) or self.sampledCnt[i]
    for _ = jBegin, jEnd do
      self.ppl[i] = loss[pplIdx]
      pplIdx = pplIdx + 1
    end
  end
end

--[[ Setup up the training data to respect `maxBatchSize`. ]]
function SampledDataset:setBatchSize(maxBatchSize, uneven_batches)
  self.maxBatchSize = maxBatchSize
  self.uneven_batches = uneven_batches
  self.maxSourceLength = 0
  self.maxTargetLength = 0

  for i = 1, #self.src do
    self.maxSourceLength = math.max(self.maxSourceLength, self.src[i]:size(1))
    if self.tgt ~= nil then
      -- Target contains <s> and </s>.
      local targetSeqLength = self.tgt[i]:size(1) - 1
      self.maxTargetLength = math.max(self.maxTargetLength, targetSeqLength)
    end
  end

  return self:sample('DEBUG')
end

--[[ Return number of sampled instances. ]]
function SampledDataset:getNumSampled()
  return self.sampled:size(1)
end

--[[ Return number of batches. ]]
function SampledDataset:batchCount()
  if self.batchRange == nil then
    if #self.src > 0 then
      return 1
    else
      return 0
    end
  end
  return #self.batchRange
end

--[[ Get `Batch` number `idx`. If nil make a batch of all the data. ]]
function SampledDataset:getBatch(batchIdx)
  if #self.src == 0 then
    return nil
  end
  if batchIdx == nil or self.batchRange == nil then
    return onmt.data.Batch.new(self.src, self.srcFeatures, self.tgt, self.tgtFeatures)
  end

  assert(self:batchCount() >= batchIdx, "Batch idx out of range: " .. batchIdx .. "/" .. self:batchCount())

  local rangeStart = self.batchRange[batchIdx]["begin"]
  local rangeEnd = self.batchRange[batchIdx]["end"]
  local sampleCntBegin =  self.batchRange[batchIdx]["sampleCntBegin"]
  local sampleCntEnd =  self.batchRange[batchIdx]["sampleCntEnd"]

  local src = {}
  local tgt
  if self.tgt ~= nil then
    tgt = {}
  end

  local srcFeatures = {}
  local tgtFeatures = {}

  for i = rangeStart, rangeEnd do
    local jBegin = (i == rangeStart) and sampleCntBegin or 1
    local jEnd = (i == rangeEnd) and math.min(self.sampledCnt[i], sampleCntEnd) or self.sampledCnt[i]
    for _ = jBegin, jEnd do
      table.insert(src, self.src[i])
      if self.srcFeatures[i] then
        table.insert(srcFeatures, self.srcFeatures[i])
      end
      if self.tgt ~= nil then
        table.insert(tgt, self.tgt[i])
        if self.tgtFeatures[i] then
          table.insert(tgtFeatures, self.tgtFeatures[i])
        end
      end
    end
  end

  return onmt.data.Batch.new(src, srcFeatures, tgt, tgtFeatures)
end

return SampledDataset


================================================
FILE: opennmt/data/Vocabulary.lua
================================================
local path = require('pl.path')

--[[ Vocabulary management utility functions. ]]
local Vocabulary = torch.class("Vocabulary")

local function countFeatures(filename)
  local reader = onmt.utils.FileReader.new(filename)
  local _, _, numFeatures = onmt.utils.Features.extract(reader:next())
  reader:close()
  return numFeatures
end

function Vocabulary.make(filename, validFunc)
  local wordVocab = onmt.utils.Dict.new({onmt.Constants.PAD_WORD, onmt.Constants.UNK_WORD,
                                         onmt.Constants.BOS_WORD, onmt.Constants.EOS_WORD})
  local featuresVocabs = {}

  local reader = onmt.utils.FileReader.new(filename)
  local lineId = 0

  while true do
    local sent = reader:next()
    if sent == nil then
      break
    end

    lineId = lineId + 1

    if validFunc(sent) then
      local words, features, numFeatures
      local _, err = pcall(function ()
        words, features, numFeatures = onmt.utils.Features.extract(sent)
      end)

      if err then
        error(err .. ' (' .. filename .. ':' .. lineId .. ')')
      end

      if #featuresVocabs == 0 and numFeatures > 0 then
        for j = 1, numFeatures do
          featuresVocabs[j] = onmt.utils.Dict.new({onmt.Constants.PAD_WORD, onmt.Constants.UNK_WORD,
                                                   onmt.Constants.BOS_WORD, onmt.Constants.EOS_WORD})
        end
      else
        assert(#featuresVocabs == numFeatures,
               'all sentences must have the same numbers of additional features (' .. filename .. ':' .. lineId .. ')')
      end

      for i = 1, #words do
        wordVocab:add(words[i])

        for j = 1, numFeatures do
          featuresVocabs[j]:add(features[j][i])
        end
      end
    end

  end

  reader:close()

  return wordVocab, featuresVocabs
end

function Vocabulary.init(name, dataFile, vocabFile, vocabSize, wordsMinFrequency, featuresVocabsFiles, validFunc)
  local wordVocab
  local featuresVocabs = {}
  local numFeatures = countFeatures(dataFile)

  if vocabFile:len() > 0 then
    -- If given, load existing word dictionary.
    _G.logger:info('Reading ' .. name .. ' vocabulary from \'' .. vocabFile .. '\'...')
    wordVocab = onmt.utils.Dict.new()
    wordVocab:loadFile(vocabFile)
    _G.logger:info('Loaded ' .. wordVocab:size() .. ' ' .. name .. ' words')
  end

  if featuresVocabsFiles:len() > 0 and numFeatures > 0 then
    -- If given, discover existing features dictionaries.
    local j = 1

    while true do
      local file = featuresVocabsFiles .. '.' .. name .. '_feature_' .. j .. '.dict'

      if not path.exists(file) then
        break
      end

      _G.logger:info('Reading ' .. name .. ' feature ' .. j .. ' vocabulary from \'' .. file .. '\'...')
      featuresVocabs[j] = onmt.utils.Dict.new()
      featuresVocabs[j]:loadFile(file)
      _G.logger:info('Loaded ' .. featuresVocabs[j]:size() .. ' labels')

      j = j + 1
    end

    assert(#featuresVocabs > 0,
           'dictionary \'' .. featuresVocabsFiles .. '.' .. name .. '_feature_1.dict\' not found')
    assert(#featuresVocabs == numFeatures,
           'the data contains ' .. numFeatures .. ' ' .. name
             .. ' features but only ' .. #featuresVocabs .. ' dictionaries were found')
  end

  if wordVocab == nil or (#featuresVocabs == 0 and numFeatures > 0) then
    -- If a dictionary is still missing, generate it.
    _G.logger:info('Building ' .. name  .. ' vocabularies...')
    local genWordVocab, genFeaturesVocabs = Vocabulary.make(dataFile, validFunc)

    local originalSizes = { genWordVocab:size() }
    for i = 1, #genFeaturesVocabs do
      table.insert(originalSizes, genFeaturesVocabs[i]:size())
    end

    local newSizes = onmt.utils.String.split(vocabSize, ',')
    local minFrequency = onmt.utils.String.split(wordsMinFrequency, ',')

    for i = 1, 1 + #genFeaturesVocabs do
      newSizes[i] = (newSizes[i] and tonumber(newSizes[i])) or 0
      minFrequency[i] = (minFrequency[i] and tonumber(minFrequency[i])) or 0
    end

    if wordVocab == nil then
      if minFrequency[1] > 0 then
        wordVocab = genWordVocab:pruneByMinFrequency(minFrequency[1])
      elseif newSizes[1] > 0 then
        wordVocab = genWordVocab:prune(newSizes[1])
      else
        wordVocab = genWordVocab
      end

      _G.logger:info('Created word dictionary of size '
                       .. wordVocab:size() .. ' (pruned from ' .. originalSizes[1] .. ')')
    end

    if #featuresVocabs == 0 then
      for i = 1, #genFeaturesVocabs do
        if minFrequency[i + 1] > 0 then
          featuresVocabs[i] = genFeaturesVocabs[i]:pruneByMinFrequency(minFrequency[i + 1])
        elseif newSizes[i + 1] > 0 then
          featuresVocabs[i] = genFeaturesVocabs[i]:prune(newSizes[i + 1])
        else
          featuresVocabs[i] = genFeaturesVocabs[i]
        end

        _G.logger:info('Created feature ' .. i .. ' dictionary of size '
                         .. featuresVocabs[i]:size() .. ' (pruned from ' .. originalSizes[i + 1] .. ')')

      end
    end
  end

  _G.logger:info('')

  return {
    words = wordVocab,
    features = featuresVocabs
  }
end

function Vocabulary.save(name, vocab, file)
  _G.logger:info('Saving ' .. name .. ' vocabulary to \'' .. file .. '\'...')
  vocab:writeFile(file)
end

function Vocabulary.saveFeatures(name, vocabs, prefix)
  for j = 1, #vocabs do
    local file = prefix .. '.' .. name .. '_feature_' .. j .. '.dict'
    _G.logger:info('Saving ' .. name .. ' feature ' .. j .. ' vocabulary to \'' .. file .. '\'...')
    vocabs[j]:writeFile(file)
  end
end

return Vocabulary


================================================
FILE: opennmt/data/init.lua
================================================
local data = {}

data.Dataset = require('opennmt.data.Dataset')
data.AliasMultinomial = require('opennmt.data.AliasMultinomial')
data.SampledDataset = require('opennmt.data.SampledDataset')
data.Batch = require('opennmt.data.Batch')
data.BatchTensor = require('opennmt.data.BatchTensor')
data.Vocabulary = require('opennmt.data.Vocabulary')
data.Preprocessor = require('opennmt.data.Preprocessor')

return data


================================================
FILE: opennmt/init.lua
================================================
require('torch')

onmt = {}

onmt.utils = require('opennmt.utils.init')

require('opennmt.modules.init')

onmt.data = require('opennmt.data.init')
onmt.train = require('opennmt.train.init')
onmt.translate = require('opennmt.translate.init')
onmt.tagger = require('opennmt.tagger.init')

onmt.Constants = require('opennmt.Constants')
onmt.Factory = require('opennmt.Factory')
onmt.Model = require('opennmt.Model')
onmt.Seq2Seq = require('opennmt.Seq2Seq')
onmt.LanguageModel = require('opennmt.LanguageModel')
onmt.SeqTagger = require('opennmt.SeqTagger')
onmt.ModelSelector = require('opennmt.ModelSelector')

return onmt


================================================
FILE: opennmt/modules/BiEncoder.lua
================================================
---------------------------------------------------------------------------------
-- Local utility functions
---------------------------------------------------------------------------------

local function reverseInput(batch)
  batch.sourceInput, batch.sourceInputRev = batch.sourceInputRev, batch.sourceInput
  batch.sourceInputFeatures, batch.sourceInputRevFeatures = batch.sourceInputRevFeatures, batch.sourceInputFeatures
  batch.sourceInputPadLeft, batch.sourceInputRevPadLeft = batch.sourceInputRevPadLeft, batch.sourceInputPadLeft
end

---------------------------------------------------------------------------------

--[[ BiEncoder is a bidirectional Sequencer used for the source language.


 `netFwd`

    h_1 => h_2 => h_3 => ... => h_n
     |      |      |             |
     .      .      .             .
     |      |      |             |
    h_1 => h_2 => h_3 => ... => h_n
     |      |      |             |
     |      |      |             |
    x_1    x_2    x_3           x_n

 `netBwd`

    h_1 <= h_2 <= h_3 <= ... <= h_n
     |      |      |             |
     .      .      .             .
     |      |      |             |
    h_1 <= h_2 <= h_3 <= ... <= h_n
     |      |      |             |
     |      |      |             |
    x_1    x_2    x_3           x_n

Inherits from [onmt.Sequencer](onmt+modules+Sequencer).

--]]
local BiEncoder, parent = torch.class('onmt.BiEncoder', 'nn.Container')

local options = {
  {
    '-brnn_merge', 'sum',
    [[Merge action for the bidirectional states.]],
    {
      enum = {'concat', 'sum'},
      structural = 0
    }
  }
}

function BiEncoder.declareOpts(cmd)
  onmt.Encoder.declareOpts(cmd)
  cmd:setCmdLineOptions(options)
end


--[[ Create a bi-encoder.

Parameters:

  * `input` - input neural network.
  * `rnn` - recurrent template module.
  * `merge` - fwd/bwd merge operation {"concat", "sum"}
]]
function BiEncoder:__init(args, input)
  parent.__init(self)

  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)

  local orgRNNSize = args.rnn_size

  -- Compute rnn hidden size depending on hidden states merge action.
  if self.args.brnn_merge == 'concat' then
    if args.rnn_size % 2 ~= 0 then
      error('in concat mode, rnn_size must be divisible by 2')
    end
    args.rnn_size = args.rnn_size / 2
  end

  self.args.rnn_size = args.rnn_size

  self.fwd = onmt.Encoder.new(args, input)
  self.bwd = onmt.Encoder.new(args, input:clone('weight', 'bias', 'gradWeight', 'gradBias'))

  self.args.numEffectiveLayers = self.fwd.args.numEffectiveLayers

  if self.args.brnn_merge == 'concat' then
    self.args.hiddenSize = self.args.rnn_size * 2
  else
    self.args.hiddenSize = self.args.rnn_size
  end

  self:add(self.fwd)
  self:add(self.bwd)

  args.rnn_size = orgRNNSize

  self:resetPreallocation()
end

--[[ Return a new BiEncoder using the serialized data `pretrained`. ]]
function BiEncoder.load(pretrained)
  local self = torch.factory('onmt.BiEncoder')()

  parent.__init(self)

  self.fwd = onmt.Encoder.load(pretrained.modules[1])
  self.bwd = onmt.Encoder.load(pretrained.modules[2])
  self.args = pretrained.args

  -- backward compatibility
  self.args.rnn_size = self.args.rnn_size or self.args.rnnSize
  self.args.brnn_merge = self.args.brnn_merge or self.args.merge

  self:add(self.fwd)
  self:add(self.bwd)

  self:resetPreallocation()

  return self
end

--[[ Return data to serialize. ]]
function BiEncoder:serialize()
  local modulesData = {}
  for i = 1, #self.modules do
    table.insert(modulesData, self.modules[i]:serialize())
  end

  return {
    name = 'BiEncoder',
    modules = modulesData,
    args = self.args
  }
end

function BiEncoder:resetPreallocation()
  -- Prototype for preallocated full context vector.
  self.contextProto = torch.Tensor()

  -- Prototype for preallocated full hidden states tensors.
  self.stateProto = torch.Tensor()

  -- Prototype for preallocated gradient of the backward context
  self.gradContextBwdProto = torch.Tensor()
end

function BiEncoder:maskPadding()
  self.fwd:maskPadding()
  self.bwd:maskPadding()
end

function BiEncoder:forward(batch, statesFw, statesBw)
  if self.statesProto == nil then
    self.statesProto = onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,
                                                         self.stateProto,
                                                         { batch.size, self.args.hiddenSize })
  end

  local states = onmt.utils.Tensor.reuseTensorTable(self.statesProto, { batch.size, self.args.hiddenSize })
  local context = onmt.utils.Tensor.reuseTensor(self.contextProto,
                                                { batch.size, batch.sourceLength, self.args.hiddenSize })

  local fwdStates, fwdContext = self.fwd:forward(batch, statesFw)
  reverseInput(batch)
  local bwdStates, bwdContext = self.bwd:forward(batch, statesBw)
  reverseInput(batch)

  if self.args.brnn_merge == 'concat' then
    for i = 1, #fwdStates do
      states[i]:narrow(2, 1, self.args.rnn_size):copy(fwdStates[i])
      states[i]:narrow(2, self.args.rnn_size + 1, self.args.rnn_size):copy(bwdStates[i])
    end
    for t = 1, batch.sourceLength do
      context[{{}, t}]:narrow(2, 1, self.args.rnn_size)
        :copy(fwdContext[{{}, t}])
      context[{{}, t}]:narrow(2, self.args.rnn_size + 1, self.args.rnn_size)
        :copy(bwdContext[{{}, batch.sourceLength - t + 1}])
    end
  elseif self.args.brnn_merge == 'sum' then
    for i = 1, #states do
      states[i]:copy(fwdStates[i])
      states[i]:add(bwdStates[i])
    end
    for t = 1, batch.sourceLength do
      context[{{}, t}]:copy(fwdContext[{{}, t}])
      context[{{}, t}]:add(bwdContext[{{}, batch.sourceLength - t + 1}])
    end
  end

  return states, context
end

function BiEncoder:backward(batch, gradStatesOutput, gradContextOutput)
  gradStatesOutput = gradStatesOutput
    or onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,
                                         onmt.utils.Cuda.convert(torch.Tensor()),
                                         { batch.size, self.args.hiddenSize })

  local gradContextOutputFwd
  local gradContextOutputBwd

  local gradStatesOutputFwd = {}
  local gradStatesOutputBwd = {}

  if self.args.brnn_merge == 'concat' then
    local gradContextOutputSplit = gradContextOutput:chunk(2, 3)
    gradContextOutputFwd = gradContextOutputSplit[1]
    gradContextOutputBwd = gradContextOutputSplit[2]

    for i = 1, #gradStatesOutput do
      local statesSplit = gradStatesOutput[i]:chunk(2, 2)
      table.insert(gradStatesOutputFwd, statesSplit[1])
      table.insert(gradStatesOutputBwd, statesSplit[2])
    end
  elseif self.args.brnn_merge == 'sum' then
    gradContextOutputFwd = gradContextOutput
    gradContextOutputBwd = gradContextOutput

    gradStatesOutputFwd = gradStatesOutput
    gradStatesOutputBwd = gradStatesOutput
  end

  local gradInputFwd, gradStateFwd = self.fwd:backward(batch, gradStatesOutputFwd, gradContextOutputFwd)

  -- reverse gradients of the backward context
  local gradContextBwd = onmt.utils.Tensor.reuseTensor(self.gradContextBwdProto,
                                                       { batch.size, batch.sourceLength, self.args.rnn_size })

  for t = 1, batch.sourceLength do
    gradContextBwd[{{}, t}]:copy(gradContextOutputBwd[{{}, batch.sourceLength - t + 1}])
  end

  local gradInputBwd, gradStateBwd = self.bwd:backward(batch, gradStatesOutputBwd, gradContextBwd)

  for t = 1, batch.sourceLength do
    onmt.utils.Tensor.recursiveAdd(gradInputFwd[t], gradInputBwd[batch.sourceLength - t + 1])
  end

  onmt.utils.Table.append(gradStateFwd, gradStateBwd)

  return gradInputFwd, gradStateFwd
end


================================================
FILE: opennmt/modules/DBiEncoder.lua
================================================
--[[ DBiEncoder is a deep bidirectional Sequencer used for the source language.


--]]
local DBiEncoder, parent = torch.class('onmt.DBiEncoder', 'nn.Container')

local options = {}

function DBiEncoder.declareOpts(cmd)
  onmt.BiEncoder.declareOpts(cmd)
  cmd:setCmdLineOptions(options)
end


--[[ Create a deep bidirectional encoder - each layers reconnect before starting another bidirectional layer

Parameters:

  * `args` - global arguments
  * `input` - input neural network.
]]
function DBiEncoder:__init(args, input)
  parent.__init(self)

  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)
  self.args.layers = args.layers
  self.args.dropout = args.dropout
  local dropout_input = args.dropout_input

  self.layers = {}

  args.layers = 1
  args.brnn_merge = 'sum'

  for _= 1, self.args.layers do
    table.insert(self.layers, onmt.BiEncoder(args, input))
    local identity = nn.Identity()
    identity.inputSize = args.rnn_size
    input = identity
    self:add(self.layers[#self.layers])
    -- trick to force a dropout on each layer L > 1
    if #self.layers == 1 and args.dropout > 0 then
      args.dropout_input = true
    end
  end
  args.layers = self.args.layers
  self.args.numEffectiveLayers = self.layers[1].args.numEffectiveLayers * self.args.layers
  self.args.hiddenSize = args.rnn_size
  args.dropout_input = dropout_input

  self:resetPreallocation()
end

--[[ Return a new DBiEncoder using the serialized data `pretrained`. ]]
function DBiEncoder.load(pretrained)
  local self = torch.factory('onmt.DBiEncoder')()
  parent.__init(self)

  self.layers = {}

  for i=1, #pretrained.layers do
    self.layers[i] = onmt.BiEncoder.load(pretrained.layers[i])
    self:add(self.layers[i])
  end

  self.args = pretrained.args

  self:resetPreallocation()
  return self
end

--[[ Return data to serialize. ]]
function DBiEncoder:serialize()
  local layersData = {}

  for i = 1, #self.layers do
    table.insert(layersData, self.layers[i]:serialize())
  end

  return {
    name = 'DBiEncoder',
    layers = layersData,
    args = self.args
  }
end

function DBiEncoder:resetPreallocation()
  -- Prototype for preallocated full context vector.
  self.contextProto = torch.Tensor()

  -- Prototype for preallocated full hidden states tensors.
  self.stateProto = torch.Tensor()

  -- Prototype for preallocated gradient context
  self.gradContextProto = torch.Tensor()
end

function DBiEncoder:maskPadding()
  for _, layer in ipairs(self.layers) do
    layer:maskPadding()
  end
end

function DBiEncoder:forward(batch)
  if self.statesProto == nil then
    self.statesProto = onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,
                                                         self.stateProto,
                                                         { batch.size, self.args.hiddenSize })
  end

  local states = onmt.utils.Tensor.reuseTensorTable(self.statesProto, { batch.size, self.args.hiddenSize })
  local context = onmt.utils.Tensor.reuseTensor(self.contextProto,
                                                { batch.size, batch.sourceLength, self.args.hiddenSize })

  local stateIdx = 1
  self.inputs = { batch }
  self.lranges = {}
  for i = 1,#self.layers do
    local layerStates, layerContext = self.layers[i]:forward(self.inputs[i])
    if i ~= #self.layers then
      table.insert(self.inputs, onmt.data.BatchTensor.new(layerContext, batch.sourceSize))
    else
      context:copy(layerContext)
    end
    table.insert(self.lranges, {stateIdx, #layerStates})
    for j = 1,#layerStates do
      states[stateIdx]:copy(layerStates[j])
      stateIdx = stateIdx + 1
    end
  end
  return states, context
end

function DBiEncoder:backward(batch, gradStatesOutput, gradContextOutput)
  local gradInputs

  for i = #self.layers, 1, -1 do
    local lrange_gradStatesOutput
    if gradStatesOutput then
      lrange_gradStatesOutput = gradStatesOutput[{}]
    end
    gradInputs = self.layers[i]:backward(self.inputs[i], lrange_gradStatesOutput, gradContextOutput)
    if i ~= 1 then
      gradContextOutput = onmt.utils.Tensor.reuseTensor(self.gradContextProto,
                                              { batch.size, #gradInputs, self.args.hiddenSize })
      for t = 1, #gradInputs do
        gradContextOutput[{{},t,{}}]:copy(gradInputs[t])
      end
    end
  end

  return gradInputs
end


================================================
FILE: opennmt/modules/Decoder.lua
================================================
--[[ Unit to decode a sequence of output tokens.

     .      .      .             .
     |      |      |             |
    h_1 => h_2 => h_3 => ... => h_n
     |      |      |             |
     .      .      .             .
     |      |      |             |
    h_1 => h_2 => h_3 => ... => h_n
     |      |      |             |
     |      |      |             |
    x_1    x_2    x_3           x_n

Inherits from [onmt.Sequencer](onmt+modules+Sequencer).

--]]
local Decoder, parent = torch.class('onmt.Decoder', 'onmt.Sequencer')

local options = {
  {
    '-input_feed', 1,
    [[Feed the context vector at each time step as additional input
      (via concatenation with the word embeddings) to the decoder.]],
    {
      enum = {0, 1},
      structural = 0
    }
  }
}

function Decoder.declareOpts(cmd)
  cmd:setCmdLineOptions(options)
  onmt.Encoder.declareOpts(cmd)
end

--[[ Construct a decoder layer.

Parameters:

  * `args` - module arguments
  * `inputNetwork` - input nn module.
  * `generator` - an output generator.
  * `attentionModel` - attention model to apply on source.
--]]
function Decoder:__init(args, inputNetwork, generator, attentionModel)
  local RNN = onmt.LSTM
  if args.rnn_type == 'GRU' then
    RNN = onmt.GRU
  end

  -- Input feeding means the decoder takes an extra
  -- vector each time representing the attention at the
  -- previous step.
  local inputSize = inputNetwork.inputSize + (args.input_feed * args.rnn_size)

  local rnn = RNN.new(args.layers, inputSize, args.rnn_size,
                      args.dropout, args.residual, args.dropout_input)

  self.rnn = rnn
  self.inputNet = inputNetwork

  self.args = args
  self.args.rnnSize = self.rnn.outputSize
  self.args.numEffectiveLayers = self.rnn.numEffectiveLayers

  self.args.inputIndex = {}
  self.args.outputIndex = {}

  self.args.inputFeed = (args.input_feed == 1)

  parent.__init(self, self:_buildModel(attentionModel))

  -- The generator use the output of the decoder sequencer to generate the
  -- likelihoods over the target vocabulary.
  self.generator = generator
  self:add(self.generator)

  self:resetPreallocation()
end

function Decoder:returnIndividualLosses(enable)
  self.indvLoss = enable
end

--[[ Return a new Decoder using the serialized data `pretrained`. ]]
function Decoder.load(pretrained)
  local self = torch.factory('onmt.Decoder')()

  self.args = pretrained.args

  parent.__init(self, pretrained.modules[1])
  self.generator = pretrained.modules[2]
  self:add(self.generator)

  self:resetPreallocation()

  return self
end

--[[ Return data to serialize. ]]
function Decoder:serialize()
  return {
    modules = self.modules,
    args = self.args
  }
end

function Decoder:resetPreallocation()
  if self.args.inputFeed then
    self.inputFeedProto = torch.Tensor()
  end

  -- Prototype for preallocated hidden and cell states.
  self.stateProto = torch.Tensor()

  -- Prototype for preallocated output gradients.
  self.gradOutputProto = torch.Tensor()

  -- Prototype for preallocated context gradient.
  self.gradContextProto = torch.Tensor()
end

--[[ Build a default one time-step of the decoder

Returns: An nn-graph mapping

  $${(c^1_{t-1}, h^1_{t-1}, .., c^L_{t-1}, h^L_{t-1}, x_t, con/H, if) =>
  (c^1_{t}, h^1_{t}, .., c^L_{t}, h^L_{t}, a)}$$

  Where ${c^l}$ and ${h^l}$ are the hidden and cell states at each layer,
  ${x_t}$ is a sparse word to lookup,
  ${con/H}$ is the context/source hidden states for attention,
  ${if}$ is the input feeding, and
  ${a}$ is the context vector computed at this timestep.
--]]
function Decoder:_buildModel(attentionModel)
  local inputs = {}
  local states = {}

  -- Inputs are previous layers first.
  for _ = 1, self.args.numEffectiveLayers do
    local h0 = nn.Identity()() -- batchSize x rnnSize
    table.insert(inputs, h0)
    table.insert(states, h0)
  end

  local x = nn.Identity()() -- batchSize
  table.insert(inputs, x)
  self.args.inputIndex.x = #inputs

  local context = nn.Identity()() -- batchSize x sourceLength x rnnSize
  table.insert(inputs, context)
  self.args.inputIndex.context = #inputs

  local inputFeed
  if self.args.inputFeed then
    inputFeed = nn.Identity()() -- batchSize x rnnSize
    table.insert(inputs, inputFeed)
    self.args.inputIndex.inputFeed = #inputs
  end

  -- Compute the input network.
  local input = self.inputNet(x)

  -- If set, concatenate previous decoder output.
  if self.args.inputFeed then
    input = nn.JoinTable(2)({input, inputFeed})
  end
  table.insert(states, input)

  -- Forward states and input into the RNN.
  local outputs = self.rnn(states)

  if self.args.numEffectiveLayers > 1 then
    -- The output of a subgraph is a node: split it to access the last RNN output.
    outputs = { outputs:split(self.args.numEffectiveLayers) }
  else
    outputs = { outputs }
  end

  -- Compute the attention here using h^L as query.
  local attnLayer = attentionModel
  attnLayer.name = 'decoderAttn'
  local attnInput = {outputs[#outputs], context}
  local attnOutput = attnLayer(attnInput)
  if self.rnn.dropout > 0 then
    attnOutput = nn.Dropout(self.rnn.dropout)(attnOutput)
  end
  table.insert(outputs, attnOutput)
  return nn.gModule(inputs, outputs)
end

function Decoder:findAttentionModel()
  if not self.decoderAttn then
    self.network:apply(function (layer)
      if layer.name == 'decoderAttn' then
        self.decoderAttn = layer
      elseif layer.name == 'softmaxAttn' then
        self.softmaxAttn = layer
      end
    end)
    self.decoderAttnClones = {}
  end
  for t = #self.decoderAttnClones+1, #self.networkClones do
    self:net(t):apply(function (layer)
      if layer.name == 'decoderAttn' then
        self.decoderAttnClones[t] = layer
      elseif layer.name == 'softmaxAttn' then
        self.decoderAttnClones[t].softmaxAttn = layer
      end
    end)
  end
end

--[[ Mask padding means that the attention-layer is constrained to
  give zero-weight to padding. This is done by storing a reference
  to the softmax attention-layer.

  Parameters:

  * See  [onmt.MaskedSoftmax](onmt+modules+MaskedSoftmax).
--]]
function Decoder:maskPadding(sourceSizes, sourceLength)
  self:findAttentionModel()

  local function substituteSoftmax(module)
    if module.name == 'softmaxAttn' then
      local mod
      if sourceSizes ~= nil then
        mod = onmt.MaskedSoftmax(sourceSizes, sourceLength)
      else
        mod = nn.SoftMax()
      end

      mod.name = 'softmaxAttn'
      mod:type(module._type)
      self.softmaxAttn = mod
      return mod
    else
      return module
    end
  end

  self.decoderAttn:replace(substituteSoftmax)

  for t = 1, #self.networkClones do
    self.decoderAttnClones[t]:replace(substituteSoftmax)
  end
end

--[[ Run one step of the decoder.

Parameters:

  * `input` - input to be passed to inputNetwork.
  * `prevStates` - stack of hidden states (batch x layers*model x rnnSize)
  * `context` - encoder output (batch x n x rnnSize)
  * `prevOut` - previous distribution (batch x #words)
  * `t` - current timestep

Returns:

 1. `out` - Top-layer hidden state.
 2. `states` - All states.
--]]
function Decoder:forwardOne(input, prevStates, context, prevOut, t)
  local inputs = {}

  -- Create RNN input (see sequencer.lua `buildNetwork('dec')`).
  onmt.utils.Table.append(inputs, prevStates)
  table.insert(inputs, input)
  table.insert(inputs, context)
  local inputSize
  if torch.type(input) == 'table' then
    inputSize = input[1]:size(1)
  else
    inputSize = input:size(1)
  end

  if self.args.inputFeed then
    if prevOut == nil then
      table.insert(inputs, onmt.utils.Tensor.reuseTensor(self.inputFeedProto,
                                                         { inputSize, self.args.rnnSize }))
    else
      table.insert(inputs, prevOut)
    end
  end

  -- Remember inputs for the backward pass.
  if self.train then
    self.inputs[t] = inputs
  end

  local outputs = self:net(t):forward(inputs)

  -- Make sure decoder always returns table.
  if type(outputs) ~= "table" then outputs = { outputs } end

  local out = outputs[#outputs]
  local states = {}
  for i = 1, #outputs - 1 do
    table.insert(states, outputs[i])
  end

  return out, states
end

--[[Compute all forward steps.

  Parameters:

  * `batch` - `Batch` object
  * `encoderStates` -
  * `context` -
  * `func` - Calls `func(out, t)` each timestep.
--]]

function Decoder:forwardAndApply(batch, encoderStates, context, func)
  -- TODO: Make this a private method.

  if self.statesProto == nil then
    self.statesProto = onmt.utils.Tensor.initTensorTable(#encoderStates,
                                                         self.stateProto,
                                                         { batch.size, self.args.rnnSize })
  end

  local states = onmt.utils.Tensor.copyTensorTable(self.statesProto, encoderStates)

  local prevOut

  for t = 1, batch.targetLength do
    prevOut, states = self:forwardOne(batch:getTargetInput(t), states, context, prevOut, t)
    func(prevOut, t)
  end
end

--[[Compute all forward steps.

  Parameters:

  * `batch` - a `Batch` object.
  * `encoderStates` - a batch of initial decoder states (optional) [0]
  * `context` - the context to apply attention to.

  Returns: Table of top hidden state for each timestep.
--]]
function Decoder:forward(batch, encoderStates, context)
  encoderStates = encoderStates
    or onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,
                                         onmt.utils.Cuda.convert(torch.Tensor()),
                                         { batch.size, self.args.rnnSize })
  if self.train then
    self.inputs = {}
  end

  local outputs = {}

  self:forwardAndApply(batch, encoderStates, context, function (out)
    table.insert(outputs, out)
  end)

  return outputs
end

--[[ Compute the backward update.

Parameters:

  * `batch` - a `Batch` object
  * `outputs` - expected outputs
  * `criterion` - a single target criterion object

  Note: This code runs both the standard backward and criterion forward/backward.
  It returns both the gradInputs and the loss.
  -- ]]
function Decoder:backward(batch, outputs, criterion)
  if self.gradOutputsProto == nil then
    self.gradOutputsProto = onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers + 1,
                                                              self.gradOutputProto,
                                                              { batch.size, self.args.rnnSize })
  end

  local gradStatesInput = onmt.utils.Tensor.reuseTensorTable(self.gradOutputsProto,
                                                             { batch.size, self.args.rnnSize })
  local gradContextInput = onmt.utils.Tensor.reuseTensor(self.gradContextProto,
                                                         { batch.size, batch.encoderOutputLength or batch.sourceLength, self.args.rnnSize })

  local loss = 0
  local indvAvgLoss = torch.zeros(outputs[1]:size(1))

  for t = batch.targetLength, 1, -1 do
    -- Compute decoder output gradients.
    -- Note: This would typically be in the forward pass.
    local pred = self.generator:forward(outputs[t])
    local output = batch:getTargetOutput(t)

    if self.indvLoss then
      for i = 1, pred[1]:size(1) do
        if t <= batch.targetSize[i] then
          local tmpPred = {}
          local tmpOutput = {}
          for j = 1, #pred do
            table.insert(tmpPred, pred[j][{{i}, {}}])
            table.insert(tmpOutput, output[j][{{i}}])
          end
          local tmpLoss = criterion:forward(tmpPred, tmpOutput)
          indvAvgLoss[i] = indvAvgLoss[i] + tmpLoss
          loss = loss + tmpLoss
        end
      end
    else
      loss = loss + criterion:forward(pred, output)
    end

    -- Compute the criterion gradient.
    local genGradOut = criterion:backward(pred, output)
    for j = 1, #genGradOut do
      genGradOut[j]:div(batch.totalSize)
    end

    -- Compute the final layer gradient.
    local decGradOut = self.generator:backward(outputs[t], genGradOut)
    gradStatesInput[#gradStatesInput]:add(decGradOut)

    -- Compute the standard backward.
    local gradInput = self:net(t):backward(self.inputs[t], gradStatesInput)

    -- Accumulate encoder output gradients.
    gradContextInput:add(gradInput[self.args.inputIndex.context])
    gradStatesInput[#gradStatesInput]:zero()

    -- Accumulate previous output gradients with input feeding gradients.
    if self.args.inputFeed and t > 1 then
      gradStatesInput[#gradStatesInput]:add(gradInput[self.args.inputIndex.inputFeed])
    end

    -- Prepare next decoder output gradients.
    for i = 1, #self.statesProto do
      gradStatesInput[i]:copy(gradInput[i])
    end
  end

  if self.indvLoss then
    indvAvgLoss = torch.cdiv(indvAvgLoss, batch.targetSize:double())
  end

  return gradStatesInput, gradContextInput, loss, indvAvgLoss
end

--[[ Compute the loss on a batch.

Parameters:

  * `batch` - a `Batch` to score.
  * `encoderStates` - initialization of decoder.
  * `context` - the attention context.
  * `criterion` - a pointwise criterion.

--]]
function Decoder:computeLoss(batch, encoderStates, context, criterion)
  encoderStates = encoderStates
    or onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,
                                         onmt.utils.Cuda.convert(torch.Tensor()),
                                         { batch.size, self.args.rnnSize })

  local loss = 0
  self:forwardAndApply(batch, encoderStates, context, function (out, t)
    local pred = self.generator:forward(out)
    local output = batch:getTargetOutput(t)
    loss = loss + criterion:forward(pred, output)
  end)

  return loss
end


--[[ Compute the score of a batch.

Parameters:

  * `batch` - a `Batch` to score.
  * `encoderStates` - initialization of decoder.
  * `context` - the attention context.

--]]
function Decoder:computeScore(batch, encoderStates, context)
  encoderStates = encoderStates
    or onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,
                                         onmt.utils.Cuda.convert(torch.Tensor()),
                                         { batch.size, self.args.rnnSize })

  local score = {}

  self:forwardAndApply(batch, encoderStates, context, function (out, t)
    local pred = self.generator:forward(out)
    for b = 1, batch.size do
      if t <= batch.targetSize[b] then
        score[b] = (score[b] or 0) + pred[1][b][batch.targetOutput[t][b]]
      end
    end
  end)

  return score
end


================================================
FILE: opennmt/modules/Encoder.lua
================================================
--[[ Encoder is a unidirectional Sequencer used for the source language.

    h_1 => h_2 => h_3 => ... => h_n
     |      |      |             |
     .      .      .             .
     |      |      |             |
    h_1 => h_2 => h_3 => ... => h_n
     |      |      |             |
     |      |      |             |
    x_1    x_2    x_3           x_n


Inherits from [onmt.Sequencer](onmt+modules+Sequencer).
--]]
local Encoder, parent = torch.class('onmt.Encoder', 'onmt.Sequencer')

local options = {
  {
    '-layers', 2,
    [[Number of recurrent layers of the encoder and decoder.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt(),
      structural = 0
    }
  },
  {
    '-rnn_size', 500,
    [[Hidden size of the recurrent unit.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt(),
      structural = 0
    }
  },
  {
    '-rnn_type', 'LSTM',
    [[Type of recurrent cell.]],
    {
      enum = {'LSTM', 'GRU'},
      structural = 0
    }
  },
  {
    '-dropout', 0.3,
    [[Dropout probability applied between recurrent layers.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isFloat(0, 1),
      structural = 1
    }
  },
  {
    '-dropout_input', false,
    [[Also apply dropout to the input of the recurrent module.]],
    {
      structural = 0
    }
  },
  {
    '-residual', false,
    [[Add residual connections between recurrent layers.]],
    {
      structural = 0
    }
  }
}

function Encoder.declareOpts(cmd)
  cmd:setCmdLineOptions(options)
end

--[[ Construct an Encoder layer.

Parameters:

  * `inputNetwork` - input module.
  * `rnn` - recurrent module.
]]
function Encoder:__init(args, inputNetwork)
  local RNN = onmt.LSTM
  if args.rnn_type == 'GRU' then
    RNN = onmt.GRU
  end

  local rnn = RNN.new(args.layers, inputNetwork.inputSize, args.rnn_size, args.dropout, args.residual, args.dropout_input)

  self.rnn = rnn
  self.inputNet = inputNetwork

  self.args = {}
  self.args.rnnSize = self.rnn.outputSize
  self.args.numEffectiveLayers = self.rnn.numEffectiveLayers

  parent.__init(self, self:_buildModel())

  self:resetPreallocation()
end

--[[ Return a new Encoder using the serialized data `pretrained`. ]]
function Encoder.load(pretrained)
  local self = torch.factory('onmt.Encoder')()

  self.args = pretrained.args
  parent.__init(self, pretrained.modules[1])

  self:resetPreallocation()

  return self
end

--[[ Return data to serialize. ]]
function Encoder:serialize()
  return {
    name = 'Encoder',
    modules = self.modules,
    args = self.args
  }
end

function Encoder:resetPreallocation()
  -- Prototype for preallocated hidden and cell states.
  self.stateProto = torch.Tensor()

  -- Prototype for preallocated output gradients.
  self.gradOutputProto = torch.Tensor()

  -- Prototype for preallocated context vector.
  self.contextProto = torch.Tensor()
end

function Encoder:maskPadding()
  self.maskPad = true
end

--[[ Build one time-step of an Encoder

Returns: An nn-graph mapping

  $${(c^1_{t-1}, h^1_{t-1}, .., c^L_{t-1}, h^L_{t-1}, x_t) =>
  (c^1_{t}, h^1_{t}, .., c^L_{t}, h^L_{t})}$$

  Where $$c^l$$ and $$h^l$$ are the hidden and cell states at each layer,
  $$x_t$$ is a sparse word to lookup.
--]]
function Encoder:_buildModel()
  local inputs = {}
  local states = {}

  -- Inputs are previous layers first.
  for _ = 1, self.args.numEffectiveLayers do
    local h0 = nn.Identity()() -- batchSize x rnnSize
    table.insert(inputs, h0)
    table.insert(states, h0)
  end

  -- Input word.
  local x = nn.Identity()() -- batchSize
  table.insert(inputs, x)

  -- Compute input network.
  local input = self.inputNet(x)
  table.insert(states, input)

  -- Forward states and input into the RNN.
  local outputs = self.rnn(states)
  return nn.gModule(inputs, { outputs })
end

--[[Compute the context representation of an input.

Parameters:

  * `batch` - as defined in batch.lua.

Returns:

  1. - final hidden states
  2. - context matrix H
--]]
function Encoder:forward(batch, states)

  -- TODO: Change `batch` to `input`.

  local finalStates
  local outputSize = self.args.rnnSize

  if self.statesProto == nil then
    self.statesProto = onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,
                                                         self.stateProto,
                                                         { batch.size, outputSize })
  end

  -- Make initial states h_0.
  states = states or onmt.utils.Tensor.reuseTensorTable(self.statesProto, { batch.size, outputSize })

  -- Preallocated output matrix.
  local context = onmt.utils.Tensor.reuseTensor(self.contextProto,
                                                { batch.size, batch.sourceLength, outputSize })

  if self.maskPad and not batch.sourceInputPadLeft then
    finalStates = onmt.utils.Tensor.recursiveClone(states)
  end
  if self.train then
    self.inputs = {}
  end

  -- Act like nn.Sequential and call each clone in a feed-forward
  -- fashion.
  for t = 1, batch.sourceLength do

    -- Construct "inputs". Prev states come first then source.
    local inputs = {}
    onmt.utils.Table.append(inputs, states)
    table.insert(inputs, batch:getSourceInput(t))

    if self.train then
      -- Remember inputs for the backward pass.
      self.inputs[t] = inputs
    end
    states = self:net(t):forward(inputs)

    -- Make sure it always returns table.
    if type(states) ~= "table" then states = { states } end

    -- Special case padding.
    if self.maskPad then
      for b = 1, batch.size do
        if (batch.sourceInputPadLeft and t <= batch.sourceLength - batch.sourceSize[b])
        or (not batch.sourceInputPadLeft and t > batch.sourceSize[b]) then
          for j = 1, #states do
            states[j][b]:zero()
          end
        elseif not batch.sourceInputPadLeft and t == batch.sourceSize[b] then
          for j = 1, #states do
            finalStates[j][b]:copy(states[j][b])
          end
        end
      end
    end

    -- Copy output (h^L_t = states[#states]) to context.
    context[{{}, t}]:copy(states[#states])
  end

  if finalStates == nil then
    finalStates = states
  end

  return finalStates, context
end

--[[ Backward pass (only called during training)

  Parameters:

  * `batch` - must be same as for forward
  * `gradStatesOutput` gradient of loss wrt last state - this can be null if states are not used
  * `gradContextOutput` - gradient of loss wrt full context.

  Returns: `gradInputs` of input network.
--]]
function Encoder:backward(batch, gradStatesOutput, gradContextOutput)
  -- TODO: change this to (input, gradOutput) as in nngraph.
  local outputSize = self.args.rnnSize
  if self.gradOutputsProto == nil then
    self.gradOutputsProto = onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,
                                                              self.gradOutputProto,
                                                              { batch.size, outputSize })
  end

  local gradStatesInput
  if gradStatesOutput then
    gradStatesInput = onmt.utils.Tensor.copyTensorTable(self.gradOutputsProto, gradStatesOutput)
  else
    -- if gradStatesOutput is not defined - start with empty tensor
    gradStatesInput = onmt.utils.Tensor.reuseTensorTable(self.gradOutputsProto, { batch.size, outputSize })
  end

  local gradInputs = {}
  local gradStates = {}

  for t = batch.sourceLength, 1, -1 do
    -- Add context gradients to last hidden states gradients.
    gradStatesInput[#gradStatesInput]:add(gradContextOutput[{{}, t}])

    -- nngraph does not accept table of size 1.
    local timestepGradOutput = #gradStatesInput > 1 and gradStatesInput or gradStatesInput[1]

    local gradInput = self:net(t):backward(self.inputs[t], timestepGradOutput)

    -- Prepare next Encoder output gradients.
    for i = 1, #gradStatesInput do
      gradStatesInput[i]:copy(gradInput[i])
    end

    if t == 1 then
      gradStates = gradStatesInput
    end

    -- Gather gradients of all user inputs.
    gradInputs[t] = {}
    for i = #gradStatesInput + 1, #gradInput do
      table.insert(gradInputs[t], gradInput[i])
    end

    if #gradInputs[t] == 1 then
      gradInputs[t] = gradInputs[t][1]
    end
  end
  -- TODO: make these names clearer.
  -- Useful if input came from another network.
  return gradInputs, gradStates

end


================================================
FILE: opennmt/modules/FeaturesEmbedding.lua
================================================
--[[
  A nngraph unit that maps features ids to embeddings. When using multiple
  features this can be the concatenation or the sum of each individual embedding.
]]
local FeaturesEmbedding, parent = torch.class('onmt.FeaturesEmbedding', 'onmt.Network')

function FeaturesEmbedding:__init(vocabSizes, vecSizes, merge)
  assert(#vocabSizes == #vecSizes)

  if merge == 'sum' then
    for i = 2, #vecSizes do
      assert(vecSizes[i] == vecSizes[1], 'embeddings must have the same size when merging with a sum')
    end
    self.outputSize = vecSizes[1]
  else
    self.outputSize = 0
    for i = 1, #vecSizes do
      self.outputSize = self.outputSize + vecSizes[i]
    end
  end

  parent.__init(self, self:_buildModel(vocabSizes, vecSizes, merge))
end

function FeaturesEmbedding:_buildModel(vocabSizes, vecSizes, merge)
  local inputs = {}
  local outputs = {}

  for i = 1, #vocabSizes do
    table.insert(inputs, nn.Identity()())
    table.insert(outputs, nn.LookupTable(vocabSizes[i], vecSizes[i])(inputs[#inputs]))
  end

  local output

  if #outputs == 1 then
    output = outputs[1]
  else
    if merge == 'sum' then
      output = nn.CAddTable()(outputs)
    else
      output = nn.JoinTable(2, 2)(outputs)
    end
  end

  return nn.gModule(inputs, {output})
end


================================================
FILE: opennmt/modules/FeaturesGenerator.lua
================================================
--[[ Feature decoder generator. Given RNN state, produce categorical distribution over
tokens and features.

  Implements $$[softmax(W^1 h + b^1), softmax(W^2 h + b^2), ..., softmax(W^n h + b^n)] $$.
--]]
local FeaturesGenerator, parent = torch.class('onmt.FeaturesGenerator', 'onmt.Network')

--[[
Parameters:

  * `rnnSize` - Input rnn size.
  * `outputSizes` - Table of each output size.
--]]
function FeaturesGenerator:__init(rnnSize, outputSizes)
  parent.__init(self, self:_buildGenerator(rnnSize, outputSizes))
end

function FeaturesGenerator:_buildGenerator(rnnSize, outputSizes)
  local generator = nn.ConcatTable()

  for i = 1, #outputSizes do
    generator:add(nn.Sequential()
                    :add(nn.Linear(rnnSize, outputSizes[i]))
                    :add(nn.LogSoftMax()))
  end

  return generator
end


================================================
FILE: opennmt/modules/GRU.lua
================================================
require('nngraph')

--[[
Implementation of a single stacked-GRU step as
an nn unit.

      h^L_{t-1} --- h^L_t
                 |


                 .
                 |
             [dropout]
                 |
      h^1_{t-1} --- h^1_t
                 |
                 |
                x_t

Computes $$(h_{t-1}, x_t) => (h_{t})$$.

--]]
local GRU, parent = torch.class('onmt.GRU', 'onmt.Network')

--[[
Parameters:

  * `layers` - Number of layers
  * `inputSize` - Size of input layer
  * `hiddenSize` - Size of the hidden layers
  * `dropout` - Dropout rate to use (in $$[0,1]$$ range).
  * `residual` - Residual connections between layers (boolean)
  * `dropout_input` - if true, add a dropout layer on the first layer (useful for instance in complex encoders)
--]]
function GRU:__init(layers, inputSize, hiddenSize, dropout, residual, dropout_input)
  dropout = dropout or 0

  self.dropout = dropout
  self.numEffectiveLayers = layers
  self.outputSize = hiddenSize

  parent.__init(self, self:_buildModel(layers, inputSize, hiddenSize, dropout, residual, dropout_input))
end

--[[ Stack the GRU units. ]]
function GRU:_buildModel(layers, inputSize, hiddenSize, dropout, residual, dropout_input)
  -- inputs: { prevOutput L1, ..., prevOutput Ln, input }
  -- outputs: { output L1, ..., output Ln }

  local inputs = {}
  local outputs = {}

  for _ = 1, layers do
    table.insert(inputs, nn.Identity()()) -- h0: batchSize x hiddenSize
  end

  table.insert(inputs, nn.Identity()()) -- x: batchSize x inputSize
  local x = inputs[#inputs]

  local prevInput
  local nextH

  for L = 1, layers do
    local input
    local inputDim

    if L == 1 then
      -- First layer input is x.
      input = x
      inputDim = inputSize
    else
      inputDim = hiddenSize
      input = nextH
      if residual and (L > 2 or inputSize == hiddenSize) then
        input = nn.CAddTable()({input, prevInput})
      end
    end
    if dropout > 0 and (dropout_input or L > 1) then
      input = nn.Dropout(dropout)(input)
    end

    local prevH = inputs[L]

    nextH = self:_buildLayer(inputDim, hiddenSize)({prevH, input})
    prevInput = input

    table.insert(outputs, nextH)
  end

  return nn.gModule(inputs, outputs)
end

--[[ Build a single GRU unit layer.
    .. math::

            \begin{array}{ll}
            r_t = sigmoid(W_{xr} x_t + b_{xr} + W_{hr} h_{(t-1)} + b_{hr}) \\
            i_t = sigmoid(W_{xi} x_t + b_{xi} + W_hi h_{(t-1)} + b_{hi}) \\
            n_t = \tanh(W_{xn} x_t + b_{xn} + r_t * (W_{hn} h_{(t-1)} + b_{hn}) \\
            h_t = (1 - i_t) * n_t + i_t * h_{(t-1)} = n_t + i_t * (h_{(t-1) - n}) \\
            \end{array}

    where $$h_t` is the hidden state at time `t`, $$x_t$$ is the hidden
    state of the previous layer at time `t` or $$input_t$$ for the first layer,
    and $$r_t$$, $$i_t$$, $$n_t$$ are the reset, input, and new gates, respectively.

    In the function: `prevH`=$$h_{(t-1}}$$, `nextH`=$$h_t$$, `r`=$$r_t$$, `i`=$$i_t$$,
    `n`=$$n_t$$.

    `x2h_r,x2h_i,x2h_n` are $$W_{xr},W_{xi},W_{xn}$$ and the biases $$b_{xr},b_{xi},b_{xn}$$.

    And `h2h_r,h2h_i,h2h_n` are $$W_{hr},W_{hi},W_{hn}$$ and the biases $$b_{hr},b_{hi},b_{hn}$$.


]]
function GRU:_buildLayer(inputSize, hiddenSize)
  local inputs = {}
  table.insert(inputs, nn.Identity()())
  table.insert(inputs, nn.Identity()())

  -- Recurrent input.
  local prevH = inputs[1]
  -- Previous layer input.
  local x = inputs[2]

  -- Evaluate the input sums at once for efficiency.
  local x2h = nn.Linear(inputSize, 3 * hiddenSize)(x)
  local h2h = nn.Linear(hiddenSize, 3 * hiddenSize)(prevH)

  -- Extract Wxr.x+bir, Wxi.x+bxi, Wxn.x+bin.
  local x2h_reshaped = nn.Reshape(3, hiddenSize)(x2h)
  local x2h_r, x2h_i, x2h_n = nn.SplitTable(2)(x2h_reshaped):split(3)

  -- Extract Whr.x+bhr, Whi.x+bhi, Whn.x+bhn
  local h2h_reshaped = nn.Reshape(3, hiddenSize)(h2h)
  local h2h_r, h2h_i, h2h_n = nn.SplitTable(2)(h2h_reshaped):split(3)

  -- Decode the gates.
  local r = nn.Sigmoid()(nn.CAddTable()({x2h_r, h2h_r}))
  local i = nn.Sigmoid()(nn.CAddTable()({x2h_i, h2h_i}))
  local n = nn.Tanh()(nn.CAddTable()({
    x2h_n, nn.CMulTable()({r, h2h_n})
  }))

  -- Perform the GRU update.
  local nextH = nn.CAddTable()({
    n,
    nn.CMulTable()({i, nn.CAddTable()({prevH, nn.MulConstant(-1)(n)})})
  })

  return nn.gModule(inputs, {nextH})
end


================================================
FILE: opennmt/modules/Generator.lua
================================================
--[[ Default decoder generator. Given RNN state, produce categorical distribution.

Simply implements $$softmax(W h + b)$$.
--]]
local Generator, parent = torch.class('onmt.Generator', 'onmt.Network')


function Generator:__init(rnnSize, outputSize)
  parent.__init(self, self:_buildGenerator(rnnSize, outputSize))
end

function Generator:_buildGenerator(rnnSize, outputSize)
  return nn.Sequential()
    :add(nn.Linear(rnnSize, outputSize))
    :add(nn:LogSoftMax())
end

function Generator:updateOutput(input)
  self.output = {self.net:updateOutput(input)}
  return self.output
end

function Generator:updateGradInput(input, gradOutput)
  self.gradInput = self.net:updateGradInput(input, gradOutput[1])
  return self.gradInput
end

function Generator:accGradParameters(input, gradOutput, scale)
  self.net:accGradParameters(input, gradOutput[1], scale)
end


================================================
FILE: opennmt/modules/GlobalAttention.lua
================================================
require('nngraph')

--[[ Global attention takes a matrix and a query vector. It
then computes a parameterized convex combination of the matrix
based on the input query.


    H_1 H_2 H_3 ... H_n
     q   q   q       q
      |  |   |       |
       \ |   |      /
           .....
         \   |  /
             a

Constructs a unit mapping:
  $$(H_1 .. H_n, q) => (a)$$
  Where H is of `batch x n x dim` and q is of `batch x dim`.

  The full function is  $$\tanh(W_2 [(softmax((W_1 q + b_1) H) H), q] + b_2)$$.

  * dot: $$score(h_t,{\overline{h}}_s) = h_t^T{\overline{h}}_s$$
  * general: $$score(h_t,{\overline{h}}_s) = h_t^T W_a {\overline{h}}_s$$
  * concat: $$score(h_t,{\overline{h}}_s) = \nu_a^T tanh(W_a[h_t;{\overline{h}}_s])$$

--]]
local GlobalAttention, parent = torch.class('onmt.GlobalAttention', 'onmt.Network')

local options = {
  {'-global_attention', 'general',         [[Global attention model type.]],
        {enum={'general', 'dot', 'concat'}}}
}

function GlobalAttention.declareOpts(cmd)
  cmd:setCmdLineOptions(options, 'Global Attention Model')
end

--[[A nn-style module computing attention.

  Parameters:

  * `dim` - dimension of the context vectors.
--]]
function GlobalAttention:__init(opt, dim)
  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(opt, options)
  parent.__init(self, self:_buildModel(dim, self.args.global_attention))
end

function GlobalAttention:_buildModel(dim, global_attention)
  local inputs = {}
  table.insert(inputs, nn.Identity()())
  table.insert(inputs, nn.Identity()())

  local ht = inputs[1]
  local context = inputs[2] -- batchL x sourceTimesteps x dim

  -- Get attention.
  local score_ht_hs
  if global_attention ~= 'concat' then
    if global_attention == 'general' then
      ht = nn.Linear(dim, dim, false)(ht) -- batchL x dim
    end
    score_ht_hs = nn.MM()({context, nn.Replicate(1,3)(ht)}) -- batchL x sourceL x 1
  else
    local ht2 = nn.Replicate(1,2)(ht) -- batchL x 1 x dim
    local ht_hs = onmt.JoinReplicateTable(2,3)({ht2, context})
    local Wa_ht_hs = nn.Bottle(nn.Linear(dim*2, dim, false),2)(ht_hs)
    local tanh_Wa_ht_hs = nn.Tanh()(Wa_ht_hs)
    score_ht_hs = nn.Bottle(nn.Linear(dim,1),2)(tanh_Wa_ht_hs)
  end
  local attn = nn.Sum(3)(score_ht_hs) -- batchL x sourceL
  local softmaxAttn = nn.SoftMax()
  softmaxAttn.name = 'softmaxAttn'
  attn = softmaxAttn(attn)
  attn = nn.Replicate(1,2)(attn) -- batchL x 1 x sourceL

  -- Apply attention to context.
  local contextCombined = nn.MM()({attn, context}) -- batchL x 1 x dim
  contextCombined = nn.Sum(2)(contextCombined) -- batchL x dim
  contextCombined = nn.JoinTable(2)({contextCombined, inputs[1]}) -- batchL x dim*2
  local contextOutput = nn.Tanh()(nn.Linear(dim*2, dim, false)(contextCombined))

  return nn.gModule(inputs, {contextOutput})
end


================================================
FILE: opennmt/modules/JoinReplicateTable.lua
================================================
local JoinReplicateTable, parent = torch.class('onmt.JoinReplicateTable', 'nn.Module')

function JoinReplicateTable:__init(dimensionReplicate, dimensionJoin, nInputDims)
  parent.__init(self)
  self.size = torch.LongStorage()
  self.dimensionReplicate = dimensionReplicate
  self.dimensionJoin = dimensionJoin
  self.gradInput = {}
  self.nInputDims = nInputDims
end

function JoinReplicateTable:_getPositiveDimensions(input)
  local dimensionReplicate = self.dimensionReplicate
  local dimensionJoin = self.dimensionJoin
  if dimensionReplicate < 0 then
    dimensionReplicate = input[1]:dim() + dimensionReplicate + 1
  elseif self.nInputDims and input[1]:dim()==(self.nInputDims+1) then
    dimensionReplicate = dimensionReplicate + 1
  end
  if dimensionJoin < 0 then
    dimensionJoin = input[1]:dim() + dimensionJoin + 1
  elseif self.nInputDims and input[1]:dim()==(self.nInputDims+1) then
    dimensionJoin = dimensionJoin + 1
  end
  return dimensionReplicate, dimensionJoin
end

function JoinReplicateTable:_replicate(input, dim, dryRun)
  -- first replicate along dimensionReplicate, we assert all dimensions are the same but some can be one
  local maxSize = 0
  local hasOne = {}

  for i = 1, #input do
    local size = input[i]:size(dim)
    assert(maxSize == 0 or size == maxSize or size == 1 or maxSize == 1, "incorrect tensor size for replicate dimension")
    if size > maxSize then
      maxSize = size
    end
    if size == 1 then
      table.insert(hasOne, i)
    end
  end

  -- remember strides to restore after joining operation
  local memStrides = {}
  if maxSize > 1 and #hasOne > 0 then
    for i = 1, #hasOne do
      local idx = hasOne[i]
      local sz = input[idx]:size()
      sz[dim] = maxSize
      local st = input[idx]:stride()
      memStrides[idx] = st[dim]
      if not dryRun then
        st[dim] = 0
        input[idx]:set(input[idx]:storage(), input[idx]:storageOffset(), sz, st)
      end
    end
  end

  return memStrides
end

function JoinReplicateTable:_dereplicate(input, dim, memStrides)
  for idx, stval in ipairs(memStrides) do
    local sz = input[idx]:size()
    sz[dim] = 1
    local st = input[idx]:stride()
    st[dim] = stval
    input[idx]:set(input[idx]:storage(), input[idx]:storageOffset(), sz, st)
  end
end

function JoinReplicateTable:updateOutput(input)
  local dimensionReplicate, dimensionJoin = self:_getPositiveDimensions(input)

  local memStrides = self:_replicate(input, dimensionReplicate)

  for i=1,#input do
    local currentOutput = input[i]
    if i == 1 then
      self.size:resize(currentOutput:dim()):copy(currentOutput:size())
    else
      self.size[dimensionJoin] = self.size[dimensionJoin]
        + currentOutput:size(dimensionJoin)
    end
  end
  self.output:resize(self.size)

  local offset = 1
  for i=1,#input do
    local currentOutput = input[i]
    self.output:narrow(dimensionJoin, offset,
      currentOutput:size(dimensionJoin)):copy(currentOutput)
    offset = offset + currentOutput:size(dimensionJoin)
  end

  self:_dereplicate(input, dimensionReplicate, memStrides)

  return self.output
end

function JoinReplicateTable:updateGradInput(input, gradOutput)
  local dimensionReplicate, dimensionJoin = self:_getPositiveDimensions(input)

  local memStrides = self:_replicate(input, dimensionReplicate, true)

  for i=1,#input do
    if self.gradInput[i] == nil then
      self.gradInput[i] = input[i].new()
    end
    self.gradInput[i]:resizeAs(input[i])
  end

  -- clear out invalid gradInputs
  for i=#input+1, #self.gradInput do
    self.gradInput[i] = nil
  end

  local offset = 1
  for i=1,#input do
    local currentOutput = input[i]
    local currentGradInput = gradOutput:narrow(dimensionJoin, offset,
               currentOutput:size(dimensionJoin))
    if memStrides[i] then
      -- sum along the replicated dimension
      self.gradInput[i]:copy(currentGradInput:sum(dimensionReplicate))
    else
      self.gradInput[i]:copy(currentGradInput)
    end

    offset = offset + currentOutput:size(dimensionJoin)
  end

  return self.gradInput
end

function JoinReplicateTable:type(type, tensorCache)
  self.gradInput = {}
  return parent.type(self, type, tensorCache)
end


================================================
FILE: opennmt/modules/LSTM.lua
================================================
require('nngraph')

--[[
Implementation of a single stacked-LSTM step as
an nn unit.

      h^L_{t-1} --- h^L_t
      c^L_{t-1} --- c^L_t
                 |


                 .
                 |
             [dropout]
                 |
      h^1_{t-1} --- h^1_t
      c^1_{t-1} --- c^1_t
                 |
                 |
                x_t

Computes $$(c_{t-1}, h_{t-1}, x_t) => (c_{t}, h_{t})$$.

--]]
local LSTM, parent = torch.class('onmt.LSTM', 'onmt.Network')

--[[
Parameters:

  * `layers` - Number of LSTM layers, L.
  * `inputSize` - Size of input layer
  * `hiddenSize` - Size of the hidden layers.
  * `dropout` - Dropout rate to use (in $$[0,1]$$ range).
  * `residual` - Residual connections between layers.
  * `dropout_input` - if true, add a dropout layer on the first layer (useful for instance in complex encoders)
--]]
function LSTM:__init(layers, inputSize, hiddenSize, dropout, residual, dropout_input)
  dropout = dropout or 0

  self.dropout = dropout
  self.numEffectiveLayers = 2 * layers
  self.outputSize = hiddenSize

  parent.__init(self, self:_buildModel(layers, inputSize, hiddenSize, dropout, residual, dropout_input))
end

--[[ Stack the LSTM units. ]]
function LSTM:_buildModel(layers, inputSize, hiddenSize, dropout, residual, dropout_input)
  local inputs = {}
  local outputs = {}

  for _ = 1, layers do
    table.insert(inputs, nn.Identity()()) -- c0: batchSize x hiddenSize
    table.insert(inputs, nn.Identity()()) -- h0: batchSize x hiddenSize
  end

  table.insert(inputs, nn.Identity()()) -- x: batchSize x inputSize
  local x = inputs[#inputs]

  local prevInput
  local nextC
  local nextH

  for L = 1, layers do
    local input
    local inputDim

    if L == 1 then
      -- First layer input is x.
      input = x
      inputDim = inputSize
    else
      inputDim = hiddenSize
      input = nextH
      if residual and (L > 2 or inputSize == hiddenSize) then
        input = nn.CAddTable()({input, prevInput})
      end
    end
    if dropout > 0 and (dropout_input or L > 1) then
      input = nn.Dropout(dropout)(input)
    end

    local prevC = inputs[L*2 - 1]
    local prevH = inputs[L*2]

    nextC, nextH = self:_buildLayer(inputDim, hiddenSize)({prevC, prevH, input}):split(2)
    prevInput = input

    table.insert(outputs, nextC)
    table.insert(outputs, nextH)
  end

  return nn.gModule(inputs, outputs)
end

--[[ Build a single LSTM unit layer. ]]
function LSTM:_buildLayer(inputSize, hiddenSize)
  local inputs = {}
  table.insert(inputs, nn.Identity()())
  table.insert(inputs, nn.Identity()())
  table.insert(inputs, nn.Identity()())

  local prevC = inputs[1]
  local prevH = inputs[2]
  local x = inputs[3]

  -- Evaluate the input sums at once for efficiency.
  local i2h = nn.Linear(inputSize, 4 * hiddenSize)(x)
  local h2h = nn.Linear(hiddenSize, 4 * hiddenSize)(prevH)
  local allInputSums = nn.CAddTable()({i2h, h2h})

  local reshaped = nn.Reshape(4, hiddenSize)(allInputSums)
  local n1, n2, n3, n4 = nn.SplitTable(2)(reshaped):split(4)

  -- Decode the gates.
  local inGate = nn.Sigmoid()(n1)
  local forgetGate = nn.Sigmoid()(n2)
  local outGate = nn.Sigmoid()(n3)

  -- Decode the write inputs.
  local inTransform = nn.Tanh()(n4)

  -- Perform the LSTM update.
  local nextC = nn.CAddTable()({
    nn.CMulTable()({forgetGate, prevC}),
    nn.CMulTable()({inGate, inTransform})
  })

  -- Gated cells form the output.
  local nextH = nn.CMulTable()({outGate, nn.Tanh()(nextC)})

  return nn.gModule(inputs, {nextC, nextH})
end


================================================
FILE: opennmt/modules/MaskedSoftmax.lua
================================================
require('nngraph')

--[[ A batched-softmax wrapper to mask the probabilities of padding.

  For instance there may be a batch of instances where A is padding.

    XXXXAA
    XXAAAA
    XXXXXX

  MaskedSoftmax ensures that no probability is given to the A's.

  For this example, `sourceSizes` is {4, 2, 6} and `sourceLength` is 6.
--]]
local MaskedSoftmax, parent = torch.class('onmt.MaskedSoftmax', 'onmt.Network')


--[[ A nn-style module that applies a softmax on input that gives no weight to the left padding.

Parameters:

  * `sourceSizes` -  the true lengths (with left padding).
  * `sourceLength` - the length of the batch.
--]]
function MaskedSoftmax:__init(sourceSizes, sourceLength)
  parent.__init(self, self:_buildModel(sourceSizes, sourceLength))
end

function MaskedSoftmax:_buildModel(sourceSizes, sourceLength)
  local numSents = sourceSizes:size(1)
  local input = nn.Identity()()
  local softmax = nn.SoftMax()(input) -- numSents x State.sourceLength

  -- Now we are masking the part of the output we don't need
  local tab = nn.SplitTable(1)(softmax) -- numSents x { State.sourceLength }
  local par = nn.ParallelTable()

  for b = 1, numSents do
    local padLength = sourceLength - sourceSizes[b]

    local seq = nn.Sequential()
    seq:add(nn.Narrow(1, padLength + 1, sourceSizes[b]))
    seq:add(nn.Padding(1, -padLength, 1, 0))
    par:add(seq)
  end

  local outTab = par(tab) -- numSents x { State.sourceLength }
  local output = nn.JoinTable(1)(outTab) -- numSents x State.sourceLength
  output = nn.View(numSents, sourceLength)(output)

  -- Make sure the vector sums to 1 (softmax output)
  output = nn.Normalize(1)(output)

  return nn.gModule({input}, {output})
end


================================================
FILE: opennmt/modules/Network.lua
================================================
--[[ Wrapper around a single network. ]]
local Network, parent = torch.class('onmt.Network', 'nn.Container')

function Network:__init(net)
  parent.__init(self)
  self.net = net
  self:add(net)
end

function Network:updateOutput(input)
  self.output = self.net:updateOutput(input)
  return self.output
end

function Network:updateGradInput(input, gradOutput)
  self.gradInput = self.net:updateGradInput(input, gradOutput)
  return self.gradInput
end

function Network:accGradParameters(input, gradOutput, scale)
  self.net:accGradParameters(input, gradOutput, scale)
end


================================================
FILE: opennmt/modules/NoAttention.lua
================================================
require('nngraph')

--[[ No attention module

--]]
local NoAttention, parent = torch.class('onmt.NoAttention', 'onmt.Network')

--[[A nn-style module computing attention.

  Parameters:

  * `dim` - dimension of the context vectors.
--]]
function NoAttention:__init(_, dim)
  parent.__init(self, self:_buildModel(dim))
end

function NoAttention:_buildModel(dim)
  local inputs = {}
  table.insert(inputs, nn.Identity()())
  table.insert(inputs, nn.Identity()())

  local lastContext = nn.Select(2,-1)(inputs[2])
  local contextCombined = nn.JoinTable(2)({lastContext, inputs[1]}) -- batchL x dim*2
  local contextOutput = nn.Tanh()(nn.Linear(dim*2, dim, false)(contextCombined))

  return nn.gModule(inputs, {contextOutput})
end


================================================
FILE: opennmt/modules/PDBiEncoder.lua
================================================
--[[ PDBiEncoder is a pyramidal deep bidirectional Sequencer used for the source language.


--]]
local PDBiEncoder, parent = torch.class('onmt.PDBiEncoder', 'nn.Container')

local options = {
  {
    '-pdbrnn_reduction', 2,
    [[Time-reduction factor at each layer.]],
    {
      structural = 0
    }
  }
}

function PDBiEncoder.declareOpts(cmd)
  onmt.BiEncoder.declareOpts(cmd)
  cmd:setCmdLineOptions(options)
end


--[[ Create a pyrimal deep bidirectional encoder - each layers reconnect before starting another bidirectional layer

Parameters:

  * `args` - global arguments
  * `input` - input neural network.
]]
function PDBiEncoder:__init(args, input)
  parent.__init(self)

  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)
  self.args.layers = args.layers
  self.args.dropout = args.dropout
  local dropout_input = args.dropout_input

  self.layers = {}

  args.layers = 1
  args.brnn_merge = 'sum'
  self.args.multiplier = 1
  for _ = 1,self.args.layers do
    table.insert(self.layers, onmt.BiEncoder(args, input))
    local identity = nn.Identity()
    identity.inputSize = args.rnn_size
    input = identity
    self:add(self.layers[#self.layers])
    if #self.layers ~= 1 then
      self.args.multiplier = self.args.multiplier * self.args.pdbrnn_reduction
    else
      -- trick to force a dropout on each layer L > 1
      if args.dropout > 0 then
        args.dropout_input = true
      end
    end
  end
  args.layers = self.args.layers
  args.dropout_input = dropout_input
  self.args.numEffectiveLayers = self.layers[1].args.numEffectiveLayers * self.args.layers
  self.args.hiddenSize = args.rnn_size

  self:resetPreallocation()
end

--[[ Return a new PDBiEncoder using the serialized data `pretrained`. ]]
function PDBiEncoder.load(pretrained)
  local self = torch.factory('onmt.PDBiEncoder')()
  parent.__init(self)

  self.layers = {}

  for i=1, #pretrained.layers do
    self.layers[i] = onmt.BiEncoder.load(pretrained.layers[i])
    self:add(self.layers[i])
  end

  self.args = pretrained.args

  self:resetPreallocation()
  return self
end

--[[ Return data to serialize. ]]
function PDBiEncoder:serialize()
  local layersData = {}
  for i = 1, #self.layers do
    table.insert(layersData, self.layers[i]:serialize())
  end

  return {
    name = 'PDBiEncoder',
    layers = layersData,
    args = self.args
  }
end

function PDBiEncoder:resetPreallocation()
  -- Prototype for preallocated full context vector.
  self.contextProto = torch.Tensor()

  -- Prototype for preallocated full hidden states tensors.
  self.stateProto = torch.Tensor()

  -- Prototype for preallocated gradient context
  self.gradContextProto = torch.Tensor()
end

function PDBiEncoder:maskPadding()
  self.layers[1]:maskPadding()
end

function PDBiEncoder:forward(batch)
  -- adjust batch length so that it can be divided
  local batch_length = batch.sourceLength
  batch_length = math.ceil(batch_length/self.args.multiplier)*self.args.multiplier
  batch.sourceLength = batch_length

  if self.statesProto == nil then
    self.statesProto = onmt.utils.Tensor.initTensorTable(self.args.numEffectiveLayers,
                                                         self.stateProto,
                                                         { batch.size, self.args.hiddenSize })
  end

  local states = onmt.utils.Tensor.reuseTensorTable(self.statesProto, { batch.size, self.args.hiddenSize })
  local context

  local stateIdx = 1
  self.inputs = { batch }
  self.lranges = {}
  for i = 1,#self.layers do
    local layerStates, layerContext = self.layers[i]:forward(self.inputs[i])
    if i ~= #self.layers then
      -- compress the layer Context along time dimension
      local storageOffset = layerContext:storageOffset()
      local strideReduced = layerContext:stride()
      strideReduced[2] = strideReduced[2] * self.args.pdbrnn_reduction
      local sizeReduced = layerContext:size()
      sizeReduced[2] = math.floor(sizeReduced[2] / self.args.pdbrnn_reduction)
      local reducedContext = layerContext
      reducedContext:set(layerContext:storage(), storageOffset, sizeReduced, strideReduced)
      for j = 1, self.args.pdbrnn_reduction-1 do
        local to_add = layerContext
        to_add:set(layerContext:storage(), storageOffset+j, sizeReduced, strideReduced)
        reducedContext:add(to_add)
      end
      table.insert(self.inputs, onmt.data.BatchTensor.new(reducedContext))
      -- record what is the size of the last reduction
      batch.encoderOutputLength = sizeReduced[2]
    else
      context = onmt.utils.Tensor.reuseTensor(self.contextProto, layerContext:size())
      context:copy(layerContext)
    end
    table.insert(self.lranges, {stateIdx, #layerStates})
    for j = 1,#layerStates do
      states[stateIdx]:copy(layerStates[j])
      stateIdx = stateIdx + 1
    end
  end
  return states, context
end

function PDBiEncoder:backward(batch, gradStatesOutput, gradContextOutput)
  local gradInputs

  for i = #self.layers, 1, -1 do
    local lrange_gradStatesOutput
    if gradStatesOutput then
      lrange_gradStatesOutput = onmt.utils.Table.subrange(gradStatesOutput, self.lranges[i][1], self.lranges[i][2])
    end
    gradInputs = self.layers[i]:backward(self.inputs[i], lrange_gradStatesOutput, gradContextOutput)
    if i ~= 1 then
      gradContextOutput = onmt.utils.Tensor.reuseTensor(self.gradContextProto,
                                              { batch.size, #gradInputs*self.args.pdbrnn_reduction, self.args.hiddenSize })
      for t = 1, #gradInputs do
        for j = 1, self.args.pdbrnn_reduction do
          gradContextOutput[{{},self.args.pdbrnn_reduction*(t-1)+j,{}}]:copy(gradInputs[t])
        end
      end
    end
  end

  return gradInputs
end


================================================
FILE: opennmt/modules/ParallelClassNLLCriterion.lua
================================================
--[[
  Define parallel ClassNLLCriterion.
--]]
local ParallelClassNLLCriterion, parent = torch.class('onmt.ParallelClassNLLCriterion', 'nn.ParallelCriterion')

function ParallelClassNLLCriterion:__init(outputSizes)
  parent.__init(self, false)

  for i = 1, #outputSizes do
    self:_addCriterion(outputSizes[i])
  end
end

function ParallelClassNLLCriterion:_addCriterion(size)
  -- Ignores padding value.
  local w = torch.ones(size)
  w[onmt.Constants.PAD] = 0

  local nll = nn.ClassNLLCriterion(w)

  -- Let the training code manage loss normalization.
  nll.sizeAverage = false
  self:add(nll)
end


================================================
FILE: opennmt/modules/Sequencer.lua
================================================
require('nngraph')

--[[ Sequencer is the base class for encoder and decoder models.
  Main task is to manage `self.net(t)`, the unrolled network
  used during training.

     :net(1) => :net(2) => ... => :net(n-1) => :net(n)

--]]
local Sequencer, parent = torch.class('onmt.Sequencer', 'nn.Container')

--[[
Parameters:

  * `network` - recurrent step template.
--]]
function Sequencer:__init(network)
  parent.__init(self)

  self.network = network
  self:add(self.network)

  self.networkClones = {}
end

function Sequencer:_sharedClone()
  local clone = self.network:clone('weight', 'gradWeight', 'bias', 'gradBias')

  -- Share intermediate tensors if defined.
  if self.networkClones[1] then
    local sharedTensors = {}

    self.networkClones[1]:apply(function(m)
      if m.gradInputSharedIdx then
        sharedTensors[m.gradInputSharedIdx] = m.gradInput
      end
      if m.outputSharedIdx then
        sharedTensors[m.outputSharedIdx] = m.output
      end
    end)

    clone:apply(function(m)
      if m.gradInputSharedIdx then
        m.gradInput = sharedTensors[m.gradInputSharedIdx]
      end
      if m.outputSharedIdx then
        m.output = sharedTensors[m.outputSharedIdx]
      end
    end)
  end

  collectgarbage()

  return clone
end

--[[Get access to the recurrent unit at a timestep.

Parameters:
  * `t` - timestep.

Returns: The raw network clone at timestep t.
  When `evaluate()` has been called, cheat and return t=1.
]]
function Sequencer:net(t)
  if self.train and t then
    -- In train mode, the network has to be cloned to remember intermediate
    -- outputs for each timestep and to allow backpropagation through time.
    while #self.networkClones < t do
      table.insert(self.networkClones, self:_sharedClone())
    end
    return self.networkClones[t]
  elseif #self.networkClones > 0 then
    return self.networkClones[1]
  else
    return self.network
  end
end

--[[Return the id of the clone to use for timestep t or 0 if not using clones]]
function Sequencer:cloneId(t)
  if self.train and t then
    return t
  elseif #self.networkClones > 0 then
    return 1
  else
    return 0
  end
end


--[[ Move the network to train mode. ]]
function Sequencer:training()
  parent.training(self)

  if #self.networkClones > 0 then
    -- Only first clone was used for evaluation.
    self.networkClones[1]:training()
  end
end

--[[ Move the network to evaluation mode. ]]
function Sequencer:evaluate()
  parent.evaluate(self)

  if #self.networkClones > 0 then
    -- We only use the first clone for evaluation.
    self.networkClones[1]:evaluate()
  end
end


================================================
FILE: opennmt/modules/WordEmbedding.lua
================================================
--[[ nn unit. Maps from word ids to embeddings. Slim wrapper around
nn.LookupTable to allow fixed and pretrained embeddings.
--]]
local WordEmbedding, parent = torch.class('onmt.WordEmbedding', 'onmt.Network')

--[[
Parameters:

  * `vocabSize` - size of the vocabulary
  * `vecSize` - size of the embedding
  * `preTrainined` - path to a pretrained vector file
  * `fix` - keep the weights of the embeddings fixed.
--]]
function WordEmbedding:__init(vocabSize, vecSize, preTrained, fix)
  parent.__init(self, nn.LookupTable(vocabSize, vecSize, onmt.Constants.PAD))

  self.preTrained = preTrained
  if fix then
    self.net.gradWeight = nil
  end
end

function WordEmbedding:postParametersInitialization()
  -- If embeddings are given. Initialize them.
  if self.preTrained and self.preTrained:len() > 0 then
    local vecs = torch.load(self.preTrained)
    self.net.weight:copy(vecs)
  end

  self.net.weight[onmt.Constants.PAD]:zero()
end

function WordEmbedding:fixEmbeddings(fix)
  if fix and self.net.gradWeight then
    self.net.gradWeight = nil
  elseif not fix and not self.net.gradWeight then
    self.net.gradWeight = self.net.weight.new(self.net.weight:size()):zero()
  end
end

function WordEmbedding:accGradParameters(input, gradOutput, scale)
  if self.net.gradWeight then
    self.net:accGradParameters(input, gradOutput, scale)
    self.net.gradWeight[onmt.Constants.PAD]:zero()
  end
end

function WordEmbedding:parameters()
  if self.net.gradWeight then
    return parent.parameters(self)
  end
end


================================================
FILE: opennmt/modules/init.lua
================================================
onmt = onmt or {}

require('opennmt.modules.Sequencer')
require('opennmt.modules.Encoder')
require('opennmt.modules.BiEncoder')
require('opennmt.modules.DBiEncoder')
require('opennmt.modules.PDBiEncoder')
require('opennmt.modules.Decoder')

require('opennmt.modules.Network')

require('opennmt.modules.GRU')
require('opennmt.modules.LSTM')

require('opennmt.modules.MaskedSoftmax')
require('opennmt.modules.WordEmbedding')
require('opennmt.modules.FeaturesEmbedding')

require('opennmt.modules.NoAttention')
require('opennmt.modules.GlobalAttention')

require('opennmt.modules.Generator')
require('opennmt.modules.FeaturesGenerator')

require('opennmt.modules.JoinReplicateTable')
require('opennmt.modules.ParallelClassNLLCriterion')

return onmt


================================================
FILE: opennmt/tagger/Tagger.lua
================================================
local Tagger = torch.class('Tagger')

local options = {
  {
    '-model', '',
    [[Path to the serialized model file.]],
    {
      valid = onmt.utils.ExtendedCmdLine.nonEmpty
    }
  },
  {
    '-batch_size', 30,
    [[Batch size.]]
  }
}

function Tagger.declareOpts(cmd)
  cmd:setCmdLineOptions(options, 'Tagger')
end

function Tagger:__init(args)
  self.opt = args

  _G.logger:info('Loading \'' .. self.opt.model .. '\'...')
  self.checkpoint = torch.load(self.opt.model)

  if not self.checkpoint.options.model_type or self.checkpoint.options.model_type ~= 'seqtagger' then
    _G.logger:error('Tagger can only process seqtagger models')
    os.exit(0)
  end

  self.model = onmt.SeqTagger.load(self.checkpoint.options, self.checkpoint.models, self.checkpoint.dicts)
  onmt.utils.Cuda.convert(self.model)

  self.dicts = self.checkpoint.dicts
end

function Tagger:buildInput(tokens)
  local words, features = onmt.utils.Features.extract(tokens)

  local data = {}
  data.words = words

  if #features > 0 then
    data.features = features
  end

  return data
end

function Tagger:buildOutput(data)
  return table.concat(onmt.utils.Features.annotate(data.words, data.features), ' ')
end

function Tagger:buildData(src)
  local srcData = {}
  srcData.words = {}
  srcData.features = {}

  local ignored = {}
  local indexMap = {}
  local index = 1

  for b = 1, #src do
    if #src[b].words == 0 then
      table.insert(ignored, b)
    else
      indexMap[index] = b
      index = index + 1

      table.insert(srcData.words,
                   self.dicts.src.words:convertToIdx(src[b].words, onmt.Constants.UNK_WORD))

      if #self.dicts.src.features > 0 then
        table.insert(srcData.features,
                     onmt.utils.Features.generateSource(self.dicts.src.features, src[b].features))
      end
    end
  end

  return onmt.data.Dataset.new(srcData), ignored, indexMap
end

function Tagger:buildTargetWords(pred)
  local tokens = self.dicts.tgt.words:convertToLabels(pred, onmt.Constants.EOS)

  return tokens
end

function Tagger:buildTargetFeatures(predFeats)
  local numFeatures = #predFeats[1]

  if numFeatures == 0 then
    return {}
  end

  local feats = {}
  for _ = 1, numFeatures do
    table.insert(feats, {})
  end

  for i = 1, #predFeats do
    for j = 1, numFeatures do
      table.insert(feats[j], self.dicts.tgt.features[j]:lookup(predFeats[i][j]))
    end
  end

  return feats
end

function Tagger:tagBatch(batch)
  self.model.models.encoder:maskPadding()

  local pred = {}
  local feats = {}
  for _ = 1, batch.size do
    table.insert(pred, {})
    table.insert(feats, {})
  end
  local _, context = self.model.models.encoder:forward(batch)

  for t = 1, batch.sourceLength do
    local out = self.model.models.generator:forward(context:select(2, t))
    if type(out[1]) == 'table' then
      out = out[1]
    end
    local _, best = out[1]:max(2)
    for b = 1, batch.size do
      if t > batch.sourceLength - batch.sourceSize[b] then
        pred[b][t - batch.sourceLength + batch.sourceSize[b]] = best[b][1]
        feats[b][t - batch.sourceLength + batch.sourceSize[b]] = {}
      end
    end
    for j = 2, #out do
      _, best = out[j]:max(2)
      for b = 1, batch.size do
        if t > batch.sourceLength - batch.sourceSize[b] then
          feats[b][t - batch.sourceLength + batch.sourceSize[b]][j - 1] = best[b][1]
        end
      end
    end
  end

  return pred, feats
end

--[[ Tag a batch of source sequences.

Parameters:

  * `src` - a batch of tables containing:
    - `words`: the table of source words
    - `features`: the table of feaures sequences (`src.features[i][j]` is the value of the ith feature of the jth token)

Returns:

  * `results` - a batch of tables containing:
      - `words`: the table of target words
      - `features`: the table of target features sequences
]]
function Tagger:tag(src)
  local data, ignored = self:buildData(src)

  local results = {}

  if data:batchCount() > 0 then
    local batch = data:getBatch()

    local pred, predFeats = self:tagBatch(batch)

    for b = 1, batch.size do
      results[b] = {}
      results[b].words = self:buildTargetWords(pred[b])
      results[b].features = self:buildTargetFeatures(predFeats[b])
    end
  end

  for i = 1, #ignored do
    table.insert(results, ignored[i], {})
  end

  return results
end

return Tagger


================================================
FILE: opennmt/tagger/init.lua
================================================
local tagger = {}

tagger.Tagger = require('opennmt.tagger.Tagger')

return tagger


================================================
FILE: opennmt/train/Checkpoint.lua
================================================
-- Class for saving and loading models during training.
local Checkpoint = torch.class('Checkpoint')

local options = {
  {
    '-train_from', '',
    [[Path to a checkpoint.]],
    {
      valid = onmt.utils.ExtendedCmdLine.fileNullOrExists
    }
  },
  {
    '-continue', false,
    [[If set, continue the training where it left off.]]
  }
}

function Checkpoint.declareOpts(cmd)
  cmd:setCmdLineOptions(options, 'Checkpoint')
end

function Checkpoint:__init(opt, model, optim, dicts)
  self.options = opt
  self.model = model
  self.optim = optim
  self.dicts = dicts

  self.savePath = self.options.save_model
end

function Checkpoint:save(filePath, info)
  info.learningRate = self.optim:getLearningRate()
  info.optimStates = self.optim:getStates()
  info.rngStates = onmt.utils.Cuda.getRNGStates()

  local data = {
    models = {},
    options = self.options,
    info = info,
    dicts = self.dicts
  }

  for k, v in pairs(self.model.models) do
    if v.serialize then
      data.models[k] = v:serialize()
    else
      data.models[k] = v
    end
  end

  torch.save(filePath, data)
end

--[[ Save the model and data in the middle of an epoch sorting the iteration. ]]
function Checkpoint:saveIteration(iteration, epochState, batchOrder, verbose)
  local info = {}
  info.iteration = iteration + 1
  info.epoch = epochState.epoch
  info.batchOrder = batchOrder

  local filePath = string.format('%s_checkpoint.t7', self.savePath)

  if verbose then
    _G.logger:info('Saving checkpoint to \'' .. filePath .. '\'...')
  end

  -- Succeed serialization before overriding existing file
  self:save(filePath .. '.tmp', info)
  os.rename(filePath .. '.tmp', filePath)
end

function Checkpoint:saveEpoch(validPpl, epochState, verbose)
  local info = {}
  info.validPpl = validPpl
  info.epoch = epochState.epoch + 1
  info.iteration = 1
  info.trainTimeInMinute = epochState:getTime() / 60

  local filePath = string.format('%s_epoch%d_%.2f.t7', self.savePath, epochState.epoch, validPpl)

  if verbose then
    _G.logger:info('Saving checkpoint to \'' .. filePath .. '\'...')
  end

  self:save(filePath, info)
end

function Checkpoint.loadFromCheckpoint(opt)
  local checkpoint = {}
  local paramChanges = {}

  if opt.train_from:len() > 0 then
    _G.logger:info('Loading checkpoint \'' .. opt.train_from .. '\'...')

    checkpoint = torch.load(opt.train_from)

    local function restoreOption(name)
      if checkpoint.options[name] ~= nil then
        opt[name] = checkpoint.options[name]
      end
    end

    if checkpoint.info.rngStates and not (opt._is_default['seed'] or opt.seed == checkpoint.options.seed) then
      _G.logger:warning('\'-seed\' value has changed - ignoring saved rng states')
      checkpoint.info.rngStates = nil
    end

    if opt.continue and (not (opt._is_default['learning_rate'] or opt.learning_rate == checkpoint.options.learning_rate) or
        not (opt._is_default['start_epoch'] or opt.start_epoch == checkpoint.options.start_epoch)) then
      _G.logger:warning('\'-continue\' option is used but learning_rate or start_epoch are set and different than previous epoch. Ignoring \'-continue\'')
      opt.continue = nil
    end

    -- Reload and check options.
    for k, v in pairs(opt) do
      if k:sub(1, 1) ~= '_' then

        if opt.continue and opt._train_state[k] then
          -- Training states should be retrieved when continuing a training.
          restoreOption(k)
        elseif opt._structural[k] or opt._init_only[k] then
          -- If an option was set by the user, check that we can actually change it.
          local valueChanged = not opt._is_default[k] and v ~= checkpoint.options[k]

          if valueChanged then
            if opt._init_only[k] then
              _G.logger:warning('Cannot change initialization option -%s. Ignoring.', k)
              restoreOption(k)
            elseif opt._structural[k] and opt._structural[k] == 0 then
              _G.logger:warning('Cannot change dynamically option -%s. Ignoring.', k)
              restoreOption(k)
            elseif opt._structural[k] and opt._structural[k] == 1 then
              paramChanges[k] = v
            end
          else
            restoreOption(k)
          end

        end

      end
    end

    if opt.continue then
      -- When continuing, some options are initialized with their last known value.
      opt.learning_rate = checkpoint.info.learningRate
      opt.start_epoch = checkpoint.info.epoch
      opt.start_iteration = checkpoint.info.iteration

      _G.logger:info('Resuming training from epoch ' .. opt.start_epoch
                         .. ' at iteration ' .. opt.start_iteration .. '...')
    else
      -- Otherwise, we can drop previous training information.
      checkpoint.info.learningRate = nil
      checkpoint.info.epoch = nil
      checkpoint.info.iteration = nil
    end
  end

  return checkpoint, opt, paramChanges
end

return Checkpoint


================================================
FILE: opennmt/train/EpochState.lua
================================================
--[[ Class for managing the training process by logging and storing
  the state of the current epoch.
]]
local EpochState = torch.class('EpochState')

--[[ Initialize for epoch `epoch`]]
function EpochState:__init(epoch, startIterations, numIterations, learningRate)
  self.epoch = epoch
  self.iterations = startIterations - 1
  self.numIterations = numIterations
  self.learningRate = learningRate

  self.globalTimer = torch.Timer()

  self:reset()
end

function EpochState:reset()
  self.trainLoss = 0
  self.sourceWords = 0
  self.targetWords = 0

  self.timer = torch.Timer()
end

--[[ Update training status. Takes `batch` (described in data.lua) and last loss.]]
function EpochState:update(model, batch, loss)
  self.iterations = self.iterations + 1
  self.trainLoss = self.trainLoss + loss
  self.sourceWords = self.sourceWords + model:getInputLabelsCount(batch)
  self.targetWords = self.targetWords + model:getOutputLabelsCount(batch)
end

--[[ Log to status stdout. ]]
function EpochState:log(iteration)
  local ppl = math.exp(self.trainLoss / self.targetWords)
  local tokpersec = self.sourceWords / self.timer:time().real
  _G.logger:info('Epoch %d ; Iteration %d/%d ; Learning rate %.4f ; Source tokens/s %d ; Perplexity %.2f',
                  self.epoch,
                  iteration or self.iterations, self.numIterations,
                  self.learningRate,
                  tokpersec,
                  ppl)
  if _G.crayon_logger.on == true then
     _G.crayon_logger.exp:add_scalar_value("learning_rate", self.learningRate)
     _G.crayon_logger.exp:add_scalar_value("perplexity", ppl)
     _G.crayon_logger.exp:add_scalar_value("token_per_sec", tokpersec)
  end
  self:reset()
end

function EpochState:getTime()
  return self.globalTimer:time().real
end

return EpochState


================================================
FILE: opennmt/train/Optim.lua
================================================
---------------------------------------------------------------------------------
-- Local utility functions
---------------------------------------------------------------------------------

local function adagradStep(dfdx, lr, state)
  if not state.var then
    state.var = torch.Tensor():typeAs(dfdx):resizeAs(dfdx):zero()
    state.std = torch.Tensor():typeAs(dfdx):resizeAs(dfdx)
  end

  state.var:addcmul(1, dfdx, dfdx)
  state.std:sqrt(state.var)
  dfdx:cdiv(state.std:add(1e-10)):mul(-lr)
end

local function adamStep(dfdx, lr, state)
  local beta1 = state.beta1 or 0.9
  local beta2 = state.beta2 or 0.999
  local eps = state.eps or 1e-8

  state.t = state.t or 0
  state.m = state.m or dfdx.new(dfdx:size()):zero()
  state.v = state.v or dfdx.new(dfdx:size()):zero()
  state.denom = state.denom or dfdx.new(dfdx:size()):zero()

  state.t = state.t + 1
  state.m:mul(beta1):add(1-beta1, dfdx)
  state.v:mul(beta2):addcmul(1-beta2, dfdx, dfdx)
  state.denom:copy(state.v):sqrt():add(eps)

  local bias1 = 1-beta1^state.t
  local bias2 = 1-beta2^state.t
  local stepSize = lr * math.sqrt(bias2)/bias1

  dfdx:copy(state.m):cdiv(state.denom):mul(-stepSize)
end

local function adadeltaStep(dfdx, state)
  local rho = state.rho or 0.9
  local eps = state.eps or 1e-6
  state.var = state.var or dfdx.new(dfdx:size()):zero()
  state.std = state.std or dfdx.new(dfdx:size()):zero()
  state.delta = state.delta or dfdx.new(dfdx:size()):zero()
  state.accDelta = state.accDelta or dfdx.new(dfdx:size()):zero()
  state.var:mul(rho):addcmul(1-rho, dfdx, dfdx)
  state.std:copy(state.var):add(eps):sqrt()
  state.delta:copy(state.accDelta):add(eps):sqrt():cdiv(state.std):cmul(dfdx)
  dfdx:copy(state.delta):mul(-1)
  state.accDelta:mul(rho):addcmul(1-rho, state.delta, state.delta)
end

---------------------------------------------------------------------------------

local Optim = torch.class('Optim')

local options = {
  {
    '-max_batch_size', 64,
    [[Maximum batch size.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt()
    }
  },
  {
    '-uneven_batches', false,
    [[If set, batches are filled up to max_batch_size even if source lengths are different.
      Slower but needed for some tasks.]]
  },
  {
    '-optim', 'sgd',
    [[Optimization method.]],
    {
      enum = {'sgd', 'adagrad', 'adadelta', 'adam'},
      train_state = true
    }
  },
  {
    '-learning_rate', 1,
    [[Starting learning rate. If adagrad or adam is used, then this is the global learning rate.
      Recommended settings are: sgd = 1, adagrad = 0.1, adam = 0.0002.]],
    {
      train_state = true
    }
  },
  {
    '-min_learning_rate', 0,
    [[Do not continue the training past this learning rate value.]],
    {
      train_state = true
    }
  },
  {
    '-max_grad_norm', 50,
    [[Clip the gradients norm to this value.]],
    {
      train_state = true
    }
  },
  {
    '-learning_rate_decay', 0.5,
    [[Learning rate decay factor: `learning_rate = learning_rate * learning_rate_decay`.]],
    {
      train_state = true
    }
  },
  {
    '-start_decay_at', 9,
    [[In "default" decay mode, start decay after this epoch.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt(),
      train_state = true
    }
  },
  {
    '-start_decay_ppl_delta', 0,
    [[Start decay when validation perplexity improvement is lower than this value.]],
    {
      train_state = true
    }
  },
  {
    '-decay', 'default',
    [[When to apply learning rate decay.
      `default`: decay after each epoch past `-start_decay_at` or as soon as the
      validation perplexity is not improving more than `-start_decay_ppl_delta`,
      `perplexity_only`: only decay when validation perplexity is not improving more than
      `-start_decay_ppl_delta`.]],
    {
      enum = {'default', 'perplexity_only'},
      train_state = true
    }
  }
}

function Optim.declareOpts(cmd)
  cmd:setCmdLineOptions(options, 'Optimization')
end

function Optim:__init(args, optimStates)
  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)
  self.valPerf = {}

  if self.args.optim == 'sgd' then
    self.args.start_decay_at = args.start_decay_at
  else
    if optimStates ~= nil then
      self.optimStates = optimStates
    else
      self.optimStates = {}
    end
  end
end

function Optim:setOptimStates(num)
  if self.args.optim ~= 'sgd' then
    for j = 1, num do
      self.optimStates[j] = {}
    end
  end
end

function Optim:zeroGrad(gradParams)
  for j = 1, #gradParams do
    gradParams[j]:zero()
  end
end

function Optim:prepareGrad(gradParams)
  -- Compute gradients norm.
  local gradNorm = 0
  for j = 1, #gradParams do
    gradNorm = gradNorm + gradParams[j]:norm()^2
  end
  gradNorm = math.sqrt(gradNorm)

  local shrinkage = self.args.max_grad_norm / gradNorm

  for j = 1, #gradParams do
    -- Shrink gradients if needed.
    if shrinkage < 1 then
      gradParams[j]:mul(shrinkage)
    end

    -- Prepare gradients params according to the optimization method.
    if self.args.optim == 'adagrad' then
      adagradStep(gradParams[j], self.args.learning_rate, self.optimStates[j])
    elseif self.args.optim == 'adadelta' then
      adadeltaStep(gradParams[j], self.optimStates[j])
    elseif self.args.optim == 'adam' then
      adamStep(gradParams[j], self.args.learning_rate, self.optimStates[j])
    else
      gradParams[j]:mul(-self.args.learning_rate)
    end
  end
end

function Optim:updateParams(params, gradParams)
  for j = 1, #params do
    params[j]:add(gradParams[j])
  end
end

-- decay learning rate if val perf does not improve or we hit the startDecayAt limit
function Optim:updateLearningRate(score, epoch)
  local function decayLr()
    self.args.learning_rate = self.args.learning_rate * self.args.learning_rate_decay
    self.args.learning_rate = math.max(self.args.learning_rate, self.args.min_learning_rate)
  end
  if self.args.optim == 'sgd' then
    self.valPerf[#self.valPerf + 1] = score

    if epoch >= self.args.start_decay_at then
      self.startDecay = true
    end

    local decayConditionMet = false

    if self.valPerf[#self.valPerf] ~= nil and self.valPerf[#self.valPerf-1] ~= nil then
      local currPpl = self.valPerf[#self.valPerf]
      local prevPpl = self.valPerf[#self.valPerf-1]
      if prevPpl - currPpl < self.args.start_decay_ppl_delta then
        self.startDecay = true
        decayConditionMet = true
      end
    end

    if self.args.decay == 'default' and self.startDecay then
      decayLr()
    elseif self.args.decay == 'perplexity_only' and decayConditionMet then
      decayLr()
    end

    return self.args.learning_rate >= self.args.min_learning_rate
  end

  return true
end

function Optim:getLearningRate()
  return self.args.learning_rate
end

function Optim:getStates()
  return self.optimStates
end

return Optim


================================================
FILE: opennmt/train/Trainer.lua
================================================
---------------------------------------------------------------------------------
-- Local utility functions
---------------------------------------------------------------------------------

local function eval(model, data)
  local loss = 0
  local totalWords = 0

  model:evaluate()

  for i = 1, data:batchCount() do
    local batch = onmt.utils.Cuda.convert(data:getBatch(i))
    loss = loss + model:forwardComputeLoss(batch)
    totalWords = totalWords + model:getOutputLabelsCount(batch)
  end

  model:training()

  return math.exp(loss / totalWords)
end

---------------------------------------------------------------------------------

local Trainer = torch.class('Trainer')

local options = {
  {
    '-save_every', 5000,
    [[Save intermediate models every this many iterations within an epoch.
      If = 0, will not save intermediate models.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt()
    }
  },
  {
    '-report_every', 50,
    [[Report progress every this many iterations within an epoch.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt()
    }
  },
  {
    '-async_parallel', false,
    [[When training on multiple GPUs, update parameters asynchronously.]]
  },
  {
    '-async_parallel_minbatch', 1000,
    [[In asynchronous training, minimal number of sequential batches before being parallel.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt()
    }
  },
  {
    '-start_iteration', 1,
    [[If loading from a checkpoint, the iteration from which to start.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isInt(1)
    }
  },
  {
    '-start_epoch', 1,
    [[If loading from a checkpoint, the epoch from which to start.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isInt(1)
    }
  },
  {
    '-end_epoch', 13,
    [[The final epoch of the training.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isInt(1)
    }
  },
  {
    '-curriculum', 0,
    [[For this many epochs, order the minibatches based on source length (from smaller to longer).
      Sometimes setting this to 1 will increase convergence speed.]],
    {
      valid = onmt.utils.ExtendedCmdLine.isUInt(),
      train_state = true
    }
  }
}

function Trainer.declareOpts(cmd)
  cmd:setCmdLineOptions(options, 'Trainer')
end

function Trainer:__init(args)
  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)
  self.args.profiler = args.profiler
  self.args.disable_mem_optimization = args.disable_mem_optimization

  -- Make a difference with options which is only used in Checkpoint.
  self.options = args
end

function Trainer:train(model, optim, trainData, validData, dataset, info)
  local params, gradParams = {}, {}

  onmt.utils.Parallel.launch(function(idx)
    -- Only logs information of the first thread.
    local verbose = idx == 1

    -- Initialize and get model parameters.
    _G.params, _G.gradParams = _G.model:initParams(verbose)

    -- Switch to training mode.
    _G.model:training()

    if self.args.profiler then
      _G.model:enableProfiling()
    end

    -- optimize memory of the first clone
    if not self.args.disable_mem_optimization then
      local batch = onmt.utils.Cuda.convert(trainData:getBatch(1))
      onmt.utils.Memory.optimize(_G.model, batch, verbose)
    end

    return idx, _G.params, _G.gradParams
  end, function(idx, theparams, thegradParams)
    params[idx] = theparams
    gradParams[idx] = thegradParams
  end)

  if info then
    -- if we had saved a previous rng state - restore it
    onmt.utils.Cuda.setRNGStates(info.rngStates, true)
  end

  local checkpoint = onmt.train.Checkpoint.new(self.options, model, optim, dataset.dicts)

  optim:setOptimStates(#params[1])

  local function trainEpoch(epoch, doProfile)
    local epochProfiler = onmt.utils.Profiler.new(doProfile)

    local startI = self.args.start_iteration

    if trainData.sample then
      trainData:sample()
    end

    local numIterations = trainData:batchCount()
    -- In parallel mode, number of iterations is reduced to reflect larger batch size.
    if onmt.utils.Parallel.count > 1 and not self.args.async_parallel then
      numIterations = math.ceil(numIterations / onmt.utils.Parallel.count)
    end

    local epochState = onmt.train.EpochState.new(epoch, startI, numIterations, optim:getLearningRate())
    local batchOrder

    if startI > 1 and info ~= nil then
      batchOrder = info.batchOrder
    else
      -- Shuffle mini batch order.
      batchOrder = torch.randperm(trainData:batchCount())
    end

    self.args.start_iteration = 1
    local needLog = false

    if not self.args.async_parallel then
      -- Synchronous training.
      local iter = startI
      for i = startI, trainData:batchCount(), onmt.utils.Parallel.count do
        local batches = {}
        local totalSize = 0
        needLog = true

        for j = 1, math.min(onmt.utils.Parallel.count, trainData:batchCount() - i + 1) do
          local batchIdx = batchOrder[i + j - 1]
          if epoch <= self.args.curriculum then
            batchIdx = i + j - 1
          end
          table.insert(batches, trainData:getBatch(batchIdx))
          totalSize = totalSize + batches[#batches].size
        end

        local losses = {}
        local indvAvgLosses = {}

        onmt.utils.Parallel.launch(function(idx)
          _G.profiler = onmt.utils.Profiler.new(doProfile)

          _G.batch = batches[idx]
          if _G.batch == nil then
            return idx, 0, nil, _G.profiler:dump()
          end

          -- Send batch data to the GPU.
          onmt.utils.Cuda.convert(_G.batch)
          _G.batch.totalSize = totalSize

          optim:zeroGrad(_G.gradParams)
          local loss, indvAvgLoss = _G.model:trainNetwork(_G.batch)

          return idx, loss, indvAvgLoss, _G.profiler:dump()
        end,
        function(idx, loss, indvAvgLoss, profile)
          losses[idx]=loss
          if trainData.needIndividualLosses and trainData:needIndividualLosses() then
            indvAvgLosses[idx] = indvAvgLoss
          end
          epochProfiler:add(profile)
        end)

        -- Accumulate the gradients from the different parallel threads.
        onmt.utils.Parallel.accGradParams(gradParams, batches)

        -- Update the parameters.
        optim:prepareGrad(gradParams[1])
        optim:updateParams(params[1], gradParams[1])

        -- Synchronize the parameters with the different parallel threads.
        onmt.utils.Parallel.syncParams(params)

        for bi = 1, #batches do
          epochState:update(model, batches[bi], losses[bi])
          if trainData.needIndividualLosses and trainData:needIndividualLosses() then
            trainData:setLoss(batchOrder[i + bi - 1], indvAvgLosses[bi])
          end
        end

        if iter % self.args.report_every == 0 then
          epochState:log(iter)
          needLog = false
        end
        if self.args.save_every > 0 and iter % self.args.save_every == 0 then
          checkpoint:saveIteration(iter, epochState, batchOrder, true)
        end
        iter = iter + 1
      end
    else
      -- Asynchronous training.
      local counter = onmt.utils.Parallel.getCounter()
      local masterGPU = onmt.utils.Cuda.gpuIds[1]
      local gradBuffer = onmt.utils.Parallel.gradBuffer
      local gmutexId = onmt.utils.Parallel.gmutexId()

      local maxConcurrentIter = self.args.report_every
      if self.args.save_every > 0 and self.args.save_every < maxConcurrentIter then
        maxConcurrentIter = self.args.save_every
      end
      local iter = 0

      counter:set(startI)

      while counter:get() <= trainData:batchCount() do
        needLog = true
        local startCounter = counter:get()

        onmt.utils.Parallel.launch(function(idx)
          _G.profiler = onmt.utils.Profiler.new(doProfile)
          -- First GPU is only used for master parameters.
          -- Use 1 GPU only for 1000 first batch.
          if idx == 1 or (idx > 2 and epoch == 1 and counter:get() < self.args.async_parallel_minbatch) then
            return
          end

          local batches = {}
          local losses = {}
          local indvAvgLosses = {}

          while true do
            local i = counter:inc()
            if i - startCounter >= maxConcurrentIter or i > trainData:batchCount() then
              return batches, losses, indvAvgLosses, _G.profiler:dump()
            end

            local batchIdx = batchOrder[i]
            if epoch <= self.args.curriculum then
              batchIdx = i
            end

            _G.batch = trainData:getBatch(batchIdx)
            table.insert(batches, onmt.utils.Tensor.deepClone(_G.batch))
            onmt.utils.Cuda.convert(_G.batch)

            optim:zeroGrad(_G.gradParams)
            local loss, indvAvgLoss = _G.model:trainNetwork(_G.batch)
            table.insert(losses, loss)
            if trainData.needIndividualLosses and trainData:needIndividualLosses() then
              indvAvgLosses[batchIdx] = indvAvgLoss
            end

            -- Update the parameters.
            optim:prepareGrad(_G.gradParams)

            -- Add up gradParams to params and synchronize back to this thread.
            onmt.utils.Parallel.updateAndSync(params[1], _G.gradParams, _G.params, gradBuffer, masterGPU, gmutexId)
          end
        end,
        function(batches, losses, indvAvgLosses, profile)
          if batches then
            iter = iter + #batches
            for i = 1, #batches do
              epochState:update(model, batches[i], losses[i])
              if trainData.needIndividualLosses and trainData:needIndividualLosses() then
                trainData:setLoss(batchOrder[i], indvAvgLosses[batchOrder[i]])
              end
            end
            epochProfiler:add(profile)
          end
        end)

        if iter % self.args.report_every == 0 then
          epochState:log()
          needLog = false
        end
        if iter % self.args.save_every == 0 then
          checkpoint:saveIteration(iter, epochState, batchOrder, true)
        end
      end
    end

    if needLog then
      epochState:log(numIterations)
    end

    return epochState, epochProfiler:dump()
  end

  _G.logger:info('Start training...')

  for epoch = self.args.start_epoch, self.args.end_epoch do
    _G.logger:info('')

    local globalProfiler = onmt.utils.Profiler.new(self.args.profiler)

    globalProfiler:start('train')
    local epochState, epochProfile = trainEpoch(epoch, self.args.profiler)
    globalProfiler:add(epochProfile)
    globalProfiler:stop('train')

    globalProfiler:start('valid')
    local validPpl = eval(model, validData)
    globalProfiler:stop('valid')

    if self.args.profiler then _G.logger:info('profile: %s', globalProfiler:log()) end
    _G.logger:info('Validation perplexity: %.2f', validPpl)

    local continue = optim:updateLearningRate(validPpl, epoch)

    checkpoint:saveEpoch(validPpl, epochState, true)

    if not continue then
      _G.logger:warning('Stopping training due to a too small learning rate value.')
      break
    end
  end
end

return Trainer


================================================
FILE: opennmt/train/init.lua
================================================
local train = {}

train.Trainer = require('opennmt.train.Trainer')
train.Checkpoint = require('opennmt.train.Checkpoint')
train.EpochState = require('opennmt.train.EpochState')
train.Optim = require('opennmt.train.Optim')

return train


================================================
FILE: opennmt/translate/Advancer.lua
================================================
--[[ Class for specifying how to advance one step. A beam mainly consists of
  a list of `tokens` and a `state`. `tokens[t]` stores a flat tensors of size
  `batchSize * beamSize` representing tokens at step `t`. `state` can be either
  a tensor with first dimension size `batchSize * beamSize`, or an iterable
  object containing several such tensors.

  Pseudocode:

      finished = []

      beams = {}

      -- Initialize the beam.

      [ beams[1] ] <-- initBeam()

      FOR t = 1, ... DO

        -- Update beam states based on new tokens.

        update([ beams[t] ])

        -- Expand beams by all possible tokens and return the scores.

        [ [scores] ] <-- expand([ beams[t] ])

        -- Find k best next beams (maintained by BeamSearcher).

        _findKBest([beams], [ [scores] ])

        completed <-- isComplete([ beams[t + 1] ])

        -- Remove completed hypotheses (maintained by BeamSearcher).

        finished += _completeHypotheses([beams], completed)

        IF all(completed) THEN

          BREAK

        END

      ENDWHILE

 ==================================================================
--]]
local Advancer = torch.class('Advancer')

--[[Returns an initial beam.

Returns:

  * `beam` - an `onmt.translate.Beam` object.

]]
function Advancer:initBeam()
end

--[[Updates beam states given new tokens.

Parameters:

  * `beam` - beam with updated token list.

]]
function Advancer:update(beam) -- luacheck: no unused args
end

--[[Expands beam by all possible tokens and returns the scores.

Parameters:

  * `beam` - an `onmt.translate.Beam` object.

Returns:

  * `scores` - a 2D tensor of size `(batchSize * beamSize, numTokens)`.

]]
function Advancer:expand(beam) -- luacheck: no unused args
end

--[[Checks which hypotheses in the beam are already finished.

Parameters:

  * `beam` - an `onmt.translate.Beam` object.

Returns: a binary flat tensor of size `(batchSize * beamSize)`, indicating
  which hypotheses are finished.

]]
function Advancer:isComplete(beam) -- luacheck: no unused args
end

--[[Specifies which states to keep track of. After beam search, those states
  can be retrieved during all steps along with the tokens. This is used
  for memory efficiency.

Parameters:

  * `indexes` - a table of iterators, specifying the indexes in the `states` to track.

]]
function Advancer:setKeptStateIndexes(indexes)
  self.keptStateIndexes = indexes
end

--[[Checks which hypotheses in the beam shall be pruned.

Parameters:

  * `beam` - an `onmt.translate.Beam` object.

Returns: a binary flat tensor of size `(batchSize * beamSize)`, indicating
  which beams shall be pruned.

]]
function Advancer:filter()
end

return Advancer


================================================
FILE: opennmt/translate/Beam.lua
================================================
--[[ Class for maintaining statistics of each step. A beam mainly consists of
  a list of tokens `tokens` and a state `state`. `tokens[t]` stores a flat tensor
  of size `batchSize * beamSize` representing the tokens at step `t`, while
  `state` can be either a tensor with first dimension size `batchSize * beamSize`,
  or an iterable object containing several such tensors.
--]]
local Beam = torch.class('Beam')

--[[Helper function. Recursively convert flat `batchSize * beamSize` tensors
 to 2D `(batchSize, beamSize)` tensors.

Parameters:

  * `v` - flat tensor of size `batchSize * beamSize` or a table containing such
  tensors.
  * `beamSize` - beam size

Returns: `(batchSize, beamSize)` tensor or a table containing such tensors.

--]]
local function flatToRc(v, beamSize)
  return onmt.utils.Tensor.recursiveApply(v, function (h)
    local batchSize = math.floor(h:size(1) / beamSize)
    local sizes = {}
    for j = 2, #h:size() do
      sizes[j - 1] = h:size(j)
    end
    return h:view(batchSize, beamSize, table.unpack(sizes))
  end)
end

--[[Helper function. Recursively convert 2D `(batchSize, beamSize)` tensors to
 flat `batchSize * beamSize` tensors.

Parameters:

  * `v` - flat tensor of size `(batchSize, beamSize)` or a table containing such
  tensors.
  * `beamSize` - beam size

Returns: `batchSize * beamSize` tensor or a table containing such tensors.

--]]
local function rcToFlat(v)
  return onmt.utils.Tensor.recursiveApply(v, function (h)
    local sizes = {}
    sizes[1] = h:size(1) * h:size(2)
    for j = 3, #h:size() do
      sizes[j - 1] = h:size(j)
    end
    return h:view(table.unpack(sizes))
  end)
end

--[[Helper function. Recursively select `(batchSize, ...)` tensors by
  specified batch indexes.

Parameters:

  * `v` - tensor of size `(batchSize, ...)` or a table containing such tensors
  * `indexes` - a table of the desired batch indexes

Returns: Indexed `(newBatchSize, ...)` tensor or a table containing such tensors

--]]
local function selectBatch(v, remaining)
  return onmt.utils.Tensor.recursiveApply(v, function (h)
    if not torch.isTensor(remaining) then
      remaining = torch.LongTensor(remaining)
    end
    return h:index(1, remaining)
  end)
end

--[[Helper function. Recursively select `(batchSize * beamSize, ...)` tensors by
  specified batch index and beam index.

Parameters:

  * `v` - tensor of size `(batchSize * beamSize, ...)` or a table containing
  such tensors.
  * `beamSize` - beam size
  * `batch` - the desired batch index
  * `beam` - the desired beam index

Returns: Indexed `(...)` tensor or a table containing such tensors

--]]
local function selectBatchBeam(v, beamSize, batch, beam)
  return onmt.utils.Tensor.recursiveApply(v, function (h)
    local batchSize = math.floor(h:size(1) / beamSize)
    local sizes = {}
    for j = 2, #h:size() do
      sizes[j - 1] = h:size(j)
    end
    local hOut = h:view(batchSize, beamSize, table.unpack(sizes))
    return hOut[{batch, beam}]
  end)
end

--[[Helper function. Recursively index `(batchSize * beamSize, ...)`
  tensors by specified indexes.

Parameters:

  * `v` - tensor of size `(batchSize * beamSize, ...)` or a table containing
  such tensors.
  * `indexes` - a tensor of size `(batchSize, k)` specifying the desired indexes
  * `beamSize` - beam size

Returns: Indexed `(batchSize * k, ...)` tensor or a table containing such tensors

--]]
local function selectBeam(v, indexes, beamSize)
  return onmt.utils.Tensor.recursiveApply(v, function (h)
    local batchSize = indexes:size(1)
    local k = indexes:size(2)
    beamSize = beamSize or k
    local sizes = {}
    local ones = {}
    for j = 2, #h:size() do
      sizes[j - 1] = h:size(j)
      ones[j - 1] = 1
    end
    return h
      :contiguous()
      :view(batchSize, beamSize, table.unpack(sizes))
      :gather(2, indexes
                :view(batchSize, k, table.unpack(ones))
                :expand(batchSize, k, table.unpack(sizes)))
      :view(batchSize * k, table.unpack(sizes))
  end)
end

--[[Helper function. Recursively expand `(batchSize, ...)` tensors
  to `(batchSize * beamSize, ...)` tensors.

Parameters:

  * `v` - tensor of size `(batchSize, ...)` or a table containing such tensors
  * `beamSize` - beam size

Returns: Expanded `(batchSize * beamSize, ...)` tensor or a table containing
  such tensors

--]]
local function replicateBeam(v, beamSize)
  return onmt.utils.Tensor.recursiveApply(v, function (h)
    local batchSize = h:size(1)
    local sizes = {}
    for j = 2, #h:size() do
      sizes[j - 1] = h:size(j)
    end
    return h
      :contiguous()
      :view(batchSize, 1, table.unpack(sizes))
      :expand(batchSize, beamSize, table.unpack(sizes))
      :contiguous()
      :view(batchSize * beamSize, table.unpack(sizes))
  end)
end


--[[Constructor. We allow users to either specify all initial hypotheses by
  passing in `token` and `state` with first dimension `batchSize * beamSize`
  such that there are `beamSize` initial hypotheses for every sequence in the
  batch and pass in the number of sequences `batchSize`, or only specify one
  hypothesis per sequence by passing `token` and `state` with first dimension
  `batchSize`, and then `onmt.translate.BeamSearcher` will pad with auxiliary
  hypotheses with scores `-inf` such that each sequence starts with `beamSize`
  hypotheses as in the former case.

Parameters:

  * `token` - tensor of size `(batchSize, vocabSize)` (if start with one initial
  hypothesis per sequence) or `(batchSize * beamSize, vocabSize)` (if start with
  `beamSize` initial hypotheses per sequence), or a list of such tensors.
  * `state` - an iterable object, where the contained tensors should have the
  same first dimension as `token`.
  * `batchSize` - optional, number of sentences. Only necessary if
  start with `beamSize` hypotheses per sequence. [`token:size(1)`]

--]]
function Beam:__init(token, state, params, batchSize)
  self._remaining = batchSize or token:size(1)

  if torch.type(token) == 'table' then
    self._tokens = token
  else
    self._tokens = { token }
  end
  self._state = state

  self._params = {}
  if params then
    self._params = params
  else
    self._params.length_norm = 0.0
    self._params.coverage_norm = 0.0
    self._params.eos_norm = 0.0
  end

  self._scores = torch.zeros(self._remaining)
  self._backPointer = nil
  self._prevBeam = nil
  self._orig2Remaining = {}
  self._remaining2Orig = {}
  self._step = 1
end

--[[

Returns:

  * `tokens` - a list of tokens. Note that the start-of-sequence symbols are
  also included. `tokens[t]` stores the tokens at step `t`, which is a tensor
  of size `batchSize * beamSize`.

--]]
function Beam:getTokens()
  return self._tokens
end

--[[

Returns:

  * `state` - an abstract iterable object as passed by constructor. Every tensor
  inside the `state` has first dimension `batchSize * beamSize`

--]]
function Beam:getState()
  return self._state
end

--[[

Returns:

  * `scores` - a flat tensor storing the total scores for each batch. It is of
  size `batchSize * beamSize`.

--]]
function Beam:getScores()
  return self._scores
end

--[[

Returns:

  * `backPointer` - a flat tensor storing the backpointers for each batch. It is
  of size `batchSize * beamSize`

--]]
function Beam:getBackPointer()
  return self._backPointer
end

--[[ Returns the number of unfinished sequences. The finished sequences will be
  removed from batch.

Returns:

  * `remaining` - the number of unfinished sequences.

--]]
function Beam:getRemaining()
  return self._remaining
end

--[[ Since finished sequences are being removed from the batch, this function
  provides a way to convert the remaining batch id to the original batch id.

--]]
function Beam:remaining2Orig(remainingId)
  if remainingId then
    return self._remaining2Orig[remainingId]
  else
    return self._remaining2Orig
  end
end

--[[ Since finished sequences are being removed from the batch, this function
  provides a way to convert the original batch id to the remaining batch id.

--]]
function Beam:orig2Remaining(origId)
  if origId then
    return self._orig2Remaining[origId]
  else
    return self._orig2Remaining
  end
end

--[[ Set state.
--]]
function Beam:setState(state)
  self._state = state
end

--[[ Set scores.
--]]
function Beam:setScores(scores)
  self._scores = scores:view(-1)
end

--[[ Set backPointer.
--]]
function Beam:setBackPointer(backPointer)
  self._backPointer = backPointer:view(-1)
end

-- In the first step, if there is only 1 hypothesis per batch, then each
-- hypothesis is replicated beamSize times to keep consistency with the
-- following beam search steps, while the scores of the auxiliary hypotheses
-- are set to -inf.
function Beam:_replicate(beamSize)
  assert (#self._tokens == 1, 'only the first beam may need replicating!')
  local token = self._tokens[1]
  local batchSize = token:size(1)
  self._tokens[1] = replicateBeam(token, beamSize)
  self._state = replicateBeam(self._state, beamSize)
  self._scores = replicateBeam(self._scores, beamSize)
  local maskScores = self._scores.new():resize(batchSize, beamSize)
  maskScores:fill(-math.huge)
  maskScores:select(2,1):fill(0)
  self._scores:add(maskScores:view(-1))
end

-- Normalize scores by length and coverage
function Beam:_normalizeScores(scores)

  if #self._state ~= 8 then
    return scores
  end

  local function normalizeLength(t)
    local alpha = self._params.length_norm
    local norm_term = math.pow((5.0 + t)/6.0, alpha)
    return norm_term
  end

  local function normalizeCoverage(ap)
    local beta = self._params.coverage_norm
    local result = torch.cmin(ap, 1.0):log1p():sum(3):mul(beta)
    return result
  end

  local step = self._step
  local lengthPenalty = normalizeLength(step)

  local attnProba = self._state[8]:view(self._remaining, scores:size(2), -1)
  local coveragePenalty = normalizeCoverage(attnProba)

  if (scores:nDimension() > 2) then
    coveragePenalty =  coveragePenalty:expand(scores:size())
  else
    coveragePenalty = coveragePenalty:viewAs(scores)
  end

  local normScores = torch.add(torch.div(scores, lengthPenalty), coveragePenalty)

  return normScores

end


-- Given new scores, combine that with the previous total scores and find the
-- top K hypotheses to form the next beam.
function Beam:_expandScores(scores, beamSize)
  local remaining = math.floor(scores:size(1) / beamSize)
  local vocabSize = scores:size(2)

  if #self._state == 8 and self._params.eos_norm > 0 then
    local EOS_penalty = torch.div(self._state[6]:view(remaining, beamSize), self._step/self._params.eos_norm)
    scores:view(remaining, beamSize, -1)[{{},{},onmt.Constants.EOS}]:cmul(EOS_penalty)
  end

  self._scores = self._scores:typeAs(scores)
  local expandedScores
    = (scores:typeAs(self._scores):view(remaining, beamSize, -1)
         + self._scores:view(remaining, beamSize, 1):expand(remaining, beamSize, vocabSize)
      )

  local normExpandedScores = self:_normalizeScores(expandedScores)
  return expandedScores:view(remaining, -1), normExpandedScores:view(remaining, -1)
end

-- Create a new beam given new token, scores and backpointer.
function Beam:_nextBeam(token, scores, backPointer, beamSize)
  local remaining = math.floor(token:size(1) / beamSize)
  local params = self._params
  local newBeam = Beam.new(self:_nextTokens(token, backPointer, beamSize),
                           self:_nextState(backPointer, beamSize),
                           params,
                           remaining)
  newBeam:setScores(scores)
  newBeam:setBackPointer(backPointer)
  newBeam._prevBeam = self
  newBeam._step = self._step + 1
  return newBeam
end

-- Select the on-beam states using the pointers
function Beam:_nextState(backPointer, beamSize)
  local nextState = selectBeam(self._state, backPointer, beamSize)
  return nextState
end

-- Given backpointers, index the tokens in the history to form the next beam's
-- token list.
function Beam:_nextTokens(token, backPointer, beamSize)
  local nextTokens = selectBeam(self._tokens, backPointer, beamSize)
  nextTokens[#nextTokens + 1] = token
  return nextTokens
end

-- Remove finished sequences to save computation.
function Beam:_removeFinishedBatches(remainingIds, beamSize)
  self._remaining = #remainingIds
  if #remainingIds > 0 then
    self._state = rcToFlat(selectBatch(flatToRc(self._state, beamSize), remainingIds))
    self._tokens = rcToFlat(selectBatch(flatToRc(self._tokens, beamSize), remainingIds))
    self._scores = rcToFlat(selectBatch(flatToRc(self._scores, beamSize), remainingIds))
    self._backPointer = rcToFlat(selectBatch(flatToRc(self._backPointer, beamSize), remainingIds))
  end
end

-- Index the iterable state object by given batch id and beam id.
function Beam:_indexState(beamSize, batchId, beamId, keptIndexes)
  keptIndexes = keptIndexes or {}
  local keptState = {}
  for _, val in pairs(keptIndexes) do
    keptState[val] = self._state[val]
  end
  return selectBatchBeam(keptState, beamSize, batchId, beamId)
end

-- Index the last step token by given batch id and beam id.
function Beam:_indexToken(beamSize, batchId, beamId)
  local token = self._tokens[#self._tokens]
  return selectBatchBeam(token, beamSize, batchId, beamId)
end

-- Index backpointer by given batch id and beam id.
function Beam:_indexBackPointer(beamSize, batchId, beamId)
  if self._backPointer then
    return selectBatchBeam(self._backPointer, beamSize, batchId, beamId)
  end
end

-- To save memory, only states at keptIndexes will be kept, while the rest
-- are discarded.
function Beam:_cleanUp(keptIndexes)
  keptIndexes = keptIndexes or {}
  local keptState = {}
  for _, val in pairs(keptIndexes) do
    keptState[val] = self._state[val]
  end
  self._state = keptState
end

-- Add completed hypotheses to buffer.
function Beam:_addCompletedHypotheses(batchId, completed)
  local origId = self:_getOrigId(batchId)
  self._remainingId = self._remainingId or 0
  self._remainingId = self._remainingId + 1
  self._orig2Remaining[origId] = self._remainingId
  self._remaining2Orig[self._remainingId] = origId
  completed = completed:view(self._remaining, -1)
  local scores = self._scores:view(self._remaining, -1)
  local normScores = self:_normalizeScores(scores)
  local tokens = self._tokens[#self._tokens]:view(self._remaining, -1)
  local backPointers = self._backPointer:view(self._remaining, -1)

  Beam._completed = Beam._completed or {}
  Beam._completed[origId] = Beam._completed[origId] or {}
  for k = 1, completed:size(2) do
    if completed[batchId][k] == 1 then
      local token = tokens[batchId][k]
      local backPointer = backPointers[batchId][k]
      local normScore = normScores[batchId][k]
      local hypothesis = {origId, normScore, token, backPointer, self._step}

      -- Maintain a sorted list.
      local id = #Beam._completed[origId] + 1
      Beam._completed[origId][id] = hypothesis
      while id > 1 do
        if Beam._completed[origId][id - 1][2] < normScore then
          Beam._completed[origId][id - 1], Beam._completed[origId][id]
            = Beam._completed[origId][id], Beam._completed[origId][id - 1]
          id = id - 1
        else
          break
        end
      end
    end
  end
end

-- Free buffer when a sequence is finished.
function Beam._removeCompleted(batchId)
  if Beam._completed then
    Beam._completed[batchId] = nil
  end
end

-- Get the original if of a sequence given its current position in the batch.
function Beam:_getOrigId(remainingId)
  local origId
  if self._step <= 2 then
    origId = remainingId
  else
    origId = self._prevBeam:remaining2Orig(remainingId)
  end
  return origId
end

-- Get nBest hypotheses for a particular sequence in the batch.
function Beam:_getTopHypotheses(remainingId, nBest, completed)
  local origId = self:_getOrigId(remainingId)

  -- Get previously completed hypotheses of the sequence.
  local prevCompleted
  if Beam._completed then
    prevCompleted = Beam._completed[origId] or {}
  else
    prevCompleted = {}
  end

  local hypotheses = {}
  local prevId = 1
  local currId = 1
  completed = completed:view(self._remaining, -1)
  local scores = self._scores:view(self._remaining, -1)
  local normScores = self:_normalizeScores(scores)
  local tokens = self._tokens[#self._tokens]:view(self._remaining, -1)
  local backPointers = self._backPointer:view(self._remaining, -1)
  for _ = 1, nBest do
    local hypothesis, finished
    if prevId <= #prevCompleted and prevCompleted[prevId][2] > normScores[remainingId][currId] then
      hypothesis = prevCompleted[prevId]
      finished = true
      prevId = prevId + 1
    else
      finished = (completed[remainingId][currId] == 1)
      if finished then
        local normScore = normScores[remainingId][currId]
        local token = tokens[remainingId][currId]
        local backPointer = backPointers[remainingId][currId]
        hypothesis = {origId, normScore, token, backPointer, self._step}
      end
      currId = currId + 1
    end
    table.insert(hypotheses, {hypothesis = hypothesis, finished = finished})
  end
  return hypotheses
end
return Beam


================================================
FILE: opennmt/translate/BeamSearcher.lua
================================================
--[[ Class for managing the internals of the beam search process.


      hyp1---hyp1---hyp1 -hyp1
          \             /
      hyp2 \-hyp2 /-hyp2--hyp2
                 /      \
      hyp3---hyp3---hyp3 -hyp3
      ========================

Takes care of beams.
--]]
local BeamSearcher = torch.class('BeamSearcher')

--[[Constructor

Parameters:

  * `advancer` - an `onmt.translate.Advancer` object.

]]
function BeamSearcher:__init(advancer)
  self.advancer = advancer
end

--[[ Performs beam search.

Parameters:

  * `beamSize` - beam size. [1]
  * `nBest` - the `nBest` top hypotheses will be returned after beam search. [1]
  * `preFilterFactor` - optional, set this only if filter is being used. Before
  applying filters, hypotheses with top `beamSize * preFilterFactor` scores will
  be considered. If the returned hypotheses voilate filters, then set this to a
  larger value to consider more. [1]
  * `keepInitial` - optional, whether return the initial token or not. [false]

Returns: a table `finished`. `finished[b][n].score`, `finished[b][n].tokens`
and `finished[b][n].states` describe the n-th best hypothesis for b-th sample
in the batch.

]]
function BeamSearcher:search(beamSize, nBest, preFilterFactor, keepInitial)
  self.nBest = nBest or 1
  self.beamSize = beamSize or 1
  assert (self.nBest <= self.beamSize)
  self.preFilterFactor = preFilterFactor or 1
  self.keepInitial = keepInitial or false

  local beams = {}
  local finished = {}

  -- Initialize the beam.
  beams[1] = self.advancer:initBeam()
  local remaining = beams[1]:getRemaining()
  if beams[1]:getTokens()[1]:size(1) ~= remaining * beamSize then
    beams[1]:_replicate(self.beamSize)
  end
  local t = 1
  while remaining > 0 do
    -- Update beam states based on new tokens.
    self.advancer:update(beams[t])

    -- Expand beams by all possible tokens and return the scores.
    local scores = self.advancer:expand(beams[t])

    -- Find k best next beams (maintained by BeamSearcher).
    self:_findKBest(beams, scores)

    -- Determine which hypotheses are complete.
    local completed = self.advancer:isComplete(beams[t + 1])

    -- Remove completed hypotheses (maintained by BeamSearcher).
    local finishedBatches, finishedHypotheses = self:_completeHypotheses(beams, completed)

    for b = 1, #finishedBatches do
      finished[finishedBatches[b]] = finishedHypotheses[b]
    end
    t = t + 1
    remaining = beams[t]:getRemaining()
  end
  return finished
end

-- Find the top beamSize hypotheses (satisfying filters).
function BeamSearcher:_findKBest(beams, scores)

  local function topk(tensor, ...)
    if torch.typename(tensor) == 'torch.CudaHalfTensor' then
      tensor = tensor:cuda()
    end
    return tensor:topk(...)
  end

  local t = #beams
  local vocabSize = scores:size(2)
  local expandedScores, expandedNormScores = beams[t]:_expandScores(scores, self.beamSize)

  -- Find top beamSize * preFilterFactor hypotheses.
  local considered = self.beamSize * self.preFilterFactor
  local consideredNormScores, consideredIds = topk(expandedNormScores, considered, 2, true, true)
  local consideredScores = expandedScores:gather(2, consideredIds)

  consideredIds:add(-1)

  local consideredBackPointer = (consideredIds:clone():div(vocabSize)):add(1)
  local consideredToken = consideredIds:view(-1)
  if consideredToken.fmod then
    consideredToken = consideredToken:fmod(vocabSize):add(1)
  else
    for i = 1, consideredToken:size(1) do
      consideredToken[i] = math.fmod(consideredToken[i], vocabSize) + 1
    end
  end

  local newBeam = beams[t]:_nextBeam(consideredToken, consideredScores,
                                    consideredBackPointer, self.beamSize)

  -- Prune hypotheses if necessary.
  local pruned = self.advancer:filter(newBeam)
  if pruned and pruned:any() then
    consideredScores:view(-1):maskedFill(pruned, -math.huge)
    consideredNormScores:view(-1):maskedFill(pruned, -math.huge)
  end

  -- Find top beamSize hypotheses.
  if ((not pruned) or (not pruned:any())) and (self.preFilterFactor == 1) then
    beams[t + 1] = newBeam
  else
    local _, kBestIds = topk(consideredNormScores, self.beamSize, 2, true, true)
    local kBestScores = consideredScores:gather(2, kBestIds)
    local backPointer = consideredBackPointer:gather(2, kBestIds)
    local token = consideredToken
      :viewAs(consideredIds)
      :gather(2, kBestIds)
      :view(-1)
    newBeam = beams[t]:_nextBeam(token, kBestScores, backPointer, self.beamSize)
    beams[t + 1] = newBeam
  end

  -- Cleanup unused memory.
  beams[t]:_cleanUp(self.advancer.keptStateIndexes)
end

-- Do a backward pass to get the tokens and states throughout the history.
function BeamSearcher:_retrieveHypothesis(beams, batchId, score, tok, bp, t)
  local states = {}
  local tokens = {}

  tokens[t - 1] = tok
  t = t - 1
  local remainingId
  while t > 0 do
    if t == 1 then
      remainingId = batchId
    else
      remainingId = beams[t]:orig2Remaining(batchId)
    end
    assert (remainingId)
    states[t] = beams[t]:_indexState(self.beamSize, remainingId, bp, self.advancer.keptStateIndexes)
    tokens[t - 1] = beams[t]:_indexToken(self.beamSize, remainingId, bp)
    bp = beams[t]:_indexBackPointer(self.beamSize, remainingId, bp)
    t = t - 1
  end
  if not self.keepInitial then
    tokens[0] = nil
  end

  -- Transpose states
  local statesTemp = {}
    for r = 1, #states do
      for j, _ in pairs(states[r]) do
        statesTemp[j] = statesTemp[j] or {}
        statesTemp[j][r] = states[r][j]
      end
    end
  states = statesTemp
  return {tokens = tokens, states = states, score = score}
end

-- Checks which sequences are finished and moves finished hypothese to a buffer.
function BeamSearcher:_completeHypotheses(beams, completed)
  local t = #beams
  local batchSize = beams[t]:getRemaining()
  completed = completed:view(batchSize, -1)

  local finishedBatches = {}
  local finishedHypotheses = {}

  -- Keep track of unfinished batch ids.
  local remainingIds = {}

  -- For each sequence in the batch, check whether it is finished or not.
  for b = 1, batchSize do
    local batchFinished = true
    local hypotheses = beams[t]:_getTopHypotheses(b, self.nBest, completed)

    -- Checks whether the top nBest hypotheses are all finished.
    for k = 1, self.nBest do
      local hypothesis = hypotheses[k]
      if not hypothesis.finished then
        batchFinished = false
        break
      end
    end

    if not batchFinished then
      -- For incomplete sequences, the complete hypotheses will be removed
      -- from beam and saved to buffer.
      table.insert(remainingIds, b)
      beams[t]:_addCompletedHypotheses(b, completed)
    else
      -- For complete sequences, we do a backward pass to retrieve the state
      -- values and tokens throughout the history.
      local origId = beams[t]:_getOrigId(b)
      table.insert(finishedBatches, origId)
      local hypothesis = {}
      for k = 1, self.nBest do
        table.insert(hypothesis, self:_retrieveHypothesis(beams,
                                                          table.unpack(hypotheses[k].hypothesis)))
      end
      table.insert(finishedHypotheses, hypothesis)
      onmt.translate.Beam._removeCompleted(origId)
    end
  end

  beams[t]:getScores():maskedFill(completed:view(-1), -math.huge)

  -- Remove finished sequences from batch.
  if #remainingIds < batchSize then
    beams[t]:_removeFinishedBatches(remainingIds, self.beamSize)
  end
  return finishedBatches, finishedHypotheses
end

return BeamSearcher


================================================
FILE: opennmt/translate/DecoderAdvancer.lua
================================================
--[[ DecoderAdvancer is an implementation of the interface Advancer for
  specifyinghow to advance one step in decoder.
--]]
local DecoderAdvancer = torch.class('DecoderAdvancer', 'Advancer')

--[[ Constructor.

Parameters:

  * `decoder` - an `onmt.Decoder` object.
  * `batch` - an `onmt.data.Batch` object.
  * `context` - encoder output (batch x n x rnnSize).
  * `max_sent_length` - optional, maximum output sentence length.
  * `max_num_unks` - optional, maximum number of UNKs.
  * `decStates` - optional, initial decoder states.
  * `dicts` - optional, dictionary for additional features.

--]]
function DecoderAdvancer:__init(decoder, batch, context, max_sent_length, max_num_unks, decStates, dicts, length_norm, coverage_norm, eos_norm)
  self.decoder = decoder
  self.batch = batch
  self.context = context
  self.max_sent_length = max_sent_length or math.huge
  self.max_num_unks = max_num_unks or math.huge
  self.length_norm = length_norm or 0.0
  self.coverage_norm = coverage_norm or 0.0
  self.eos_norm = eos_norm or 0.0
  self.decStates = decStates or onmt.utils.Tensor.initTensorTable(
    decoder.args.numEffectiveLayers,
    onmt.utils.Cuda.convert(torch.Tensor()),
    { self.batch.size, decoder.args.rnnSize })
  self.dicts = dicts
end

--[[Returns an initial beam.

Returns:

  * `beam` - an `onmt.translate.Beam` object.

--]]
function DecoderAdvancer:initBeam()
  local tokens = onmt.utils.Cuda.convert(torch.IntTensor(self.batch.size)):fill(onmt.Constants.BOS)
  local features = {}
  if self.dicts then
    for j = 1, #self.dicts.tgt.features do
      features[j] = torch.IntTensor(self.batch.size):fill(onmt.Constants.EOS)
    end
  end
  local sourceSizes = onmt.utils.Cuda.convert(self.batch.sourceSize)
  local attnProba = torch.FloatTensor(self.batch.size, self.context:size(2)):fill(0.0001)
  -- Mask padding
  for i = 1,self.batch.size do
    local pad_size = self.context:size(2) - sourceSizes[i]
    if (pad_size ~= 0) then
      attnProba[{ i, {1,pad_size} }] = 1.0
    end
  end

  -- Define state to be { decoder states, decoder output, context,
  -- attentions, features, sourceSizes, step, cumulated attention probablities }.
  local state = { self.decStates, nil, self.context, nil, features, sourceSizes, 1, attnProba }
  local params = {}
  params.length_norm = self.length_norm
  params.coverage_norm = self.coverage_norm
  params.eos_norm = self.eos_norm
  return onmt.translate.Beam.new(tokens, state, params)
end

--[[Updates beam states given new tokens.

Parameters:

  * `beam` - beam with updated token list.

]]
function DecoderAdvancer:update(beam)
  local state = beam:getState()
  local decStates, decOut, context, _, features, sourceSizes, t, cumAttnProba
    = table.unpack(state, 1, 8)
  local tokens = beam:getTokens()
  local token = tokens[#tokens]
  local inputs
  if #features == 0 then
    inputs = token
  elseif #features == 1 then
    inputs = { token, features[1] }
  else
    inputs = { token }
    table.insert(inputs, features)
  end
  self.decoder:maskPadding(sourceSizes, self.batch.sourceLength)
  decOut, decStates = self.decoder:forwardOne(inputs, decStates, context, decOut)
  t = t + 1
  local softmaxOut = self.decoder.softmaxAttn.output

  cumAttnProba = cumAttnProba:typeAs(softmaxOut):add(softmaxOut)

  local nextState = {decStates, decOut, context, softmaxOut, nil, sourceSizes, t, cumAttnProba}
  beam:setState(nextState)
end

--[[Expand function. Expands beam by all possible tokens and returns the
  scores.

Parameters:

  * `beam` - an `onmt.translate.Beam` object.

Returns:

  * `scores` - a 2D tensor of size `(batchSize * beamSize, numTokens)`.

]]
function DecoderAdvancer:expand(beam)
  local state = beam:getState()
  local decOut = state[2]
  local out = self.decoder.generator:forward(decOut)
  local features = {}
  for j = 2, #out do
    local _, best = out[j]:max(2)
    features[j - 1] = best:view(-1)
  end
  state[5] = features
  local scores = out[1]
  return scores
end

--[[Checks which hypotheses in the beam are already finished. A hypothesis is
  complete if i) an onmt.Constants.EOS is encountered, or ii) the length of the
  sequence is greater than or equal to `max_sent_length`.

Parameters:

  * `beam` - an `onmt.translate.Beam` object.

Returns: a binary flat tensor of size `(batchSize * beamSize)`, indicating
  which hypotheses are finished.

]]
function DecoderAdvancer:isComplete(beam)
  local tokens = beam:getTokens()
  local seqLength = #tokens - 1
  local complete = tokens[#tokens]:eq(onmt.Constants.EOS)
  if seqLength > self.max_sent_length then
    complete:fill(1)
  end
  return complete
end

--[[Checks which hypotheses in the beam shall be pruned. We disallow empty
 predictions, as well as predictions with more UNKs than `max_num_unks`.

Parameters:

  * `beam` - an `onmt.translate.Beam` object.

Returns: a binary flat tensor of size `(batchSize * beamSize)`, indicating
  which beams shall be pruned.

]]
function DecoderAdvancer:filter(beam)
  local tokens = beam:getTokens()
  local numUnks = onmt.utils.Cuda.convert(torch.zeros(tokens[1]:size(1)))
  for t = 1, #tokens do
    local token = tokens[t]
    numUnks:add(onmt.utils.Cuda.convert(token:eq(onmt.Constants.UNK):double()))
  end

  -- Disallow too many UNKs
  local pruned = numUnks:gt(self.max_num_unks)

  -- Disallow empty hypotheses
  if #tokens == 2 then
    pruned:add(tokens[2]:eq(onmt.Constants.EOS))
  end
  return pruned:ge(1)
end

return DecoderAdvancer


================================================
FILE: opennmt/translate/PhraseTable.lua
================================================
--[[Parse and lookup a words from a phrase table.
--]]
local PhraseTable = torch.class('PhraseTable')

function PhraseTable:__init(filePath)
  local f = assert(io.open(filePath, 'r'))

  self.table = {}

  for line in f:lines() do
    local c = onmt.utils.String.split(line, '|||')
    assert(#c == 2, 'badly formatted phrase table: ' .. line)
    self.table[onmt.utils.String.strip(c[1])] = onmt.utils.String.strip(c[2])
  end

  f:close()
end

--[[ Return the phrase table match for `word`. ]]
function PhraseTable:lookup(word)
  return self.table[word]
end

--[[ Return true if the phrase table contains the source word `word`. ]]
function PhraseTable:contains(word)
  return self:lookup(word) ~= nil
end

return PhraseTable


================================================
FILE: opennmt/translate/Translator.lua
================================================
local Translator = torch.class('Translator')

local options = {
  {
    '-model', '',
    [[Path to the serialized model file.]],
    {
      valid = onmt.utils.ExtendedCmdLine.nonEmpty
    }
  },
  {
    '-beam_size', 5,
    [[Beam size.]]
  },
  {
    '-batch_size', 30,
    [[Batch size.]]
  },
  {
    '-max_sent_length', 250,
    [[Maximum output sentence length.]]
  },
  {
    '-replace_unk', false,
    [[Replace the generated <unk> tokens with the source token that
      has the highest attention weight. If `-phrase_table` is provided,
      it will lookup the identified source token and give the corresponding
      target token. If it is not provided (or the identified source token
      does not exist in the table) then it will copy the source token]]},
  {
    '-phrase_table', '',
    [[Path to source-target dictionary to replace `<unk>` tokens.]]
  },
  {
    '-n_best', 1,
    [[If > 1, it will also output an n-best list of decoded sentences.]]
  },
  {
    '-max_num_unks', math.huge,
    [[All sequences with more `<unk>`s than this will be ignored during beam search.]]
  },
  {
    '-pre_filter_factor', 1,
    [[Optional, set this only if filter is being used. Before
      applying filters, hypotheses with top `beam_size * pre_filter_factor`
      scores will be considered. If the returned hypotheses voilate filters,
      then set this to a larger value to consider more.]]},
  {
    '-length_norm', 0.0,
    [[Length normalization coefficient (alpha). If set to 0, no length normalization.]]
  },
  {
    '-coverage_norm', 0.0,
    [[Coverage normalization coefficient (beta).
      An extra coverage term multiplied by beta is added to hypotheses scores.
      If is set to 0, no coverage normalization.]]
  },
  {
    '-eos_norm', 0.0,
    [[End of sentence normalization coefficient (gamma). If set to 0, no EOS normalization.]]
  },
  {
    '-dump_input_encoding', false,
    [[Instead of generating target tokens conditional on
    the source tokens, we print the representation
    (encoding/embedding) of the input.]]
  }
}

function Translator.declareOpts(cmd)
  cmd:setCmdLineOptions(options, 'Translator')
end


function Translator:__init(args)
  self.opt = args

  _G.logger:info('Loading \'' .. self.opt.model .. '\'...')
  self.checkpoint = torch.load(self.opt.model)

  if self.checkpoint.options.model_type and self.checkpoint.options.model_type ~= 'seq2seq' then
    _G.logger:error('Translator can only process seq2seq models')
    os.exit(0)
  end

  self.models = {}
  self.models.encoder = onmt.Factory.loadEncoder(self.checkpoint.models.encoder)
  self.models.decoder = onmt.Factory.loadDecoder(self.checkpoint.models.decoder)

  self.models.encoder:evaluate()
  self.models.decoder:evaluate()

  onmt.utils.Cuda.convert(self.models.encoder)
  onmt.utils.Cuda.convert(self.models.decoder)

  self.dicts = self.checkpoint.dicts

  if self.opt.phrase_table:len() > 0 then
    self.phraseTable = onmt.translate.PhraseTable.new(self.opt.phrase_table)
  end
end

function Translator:buildInput(tokens)
  local words, features = onmt.utils.Features.extract(tokens)

  local data = {}
  data.words = words

  if #features > 0 then
    data.features = features
  end

  return data
end

function Translator:buildOutput(data)
  return table.concat(onmt.utils.Features.annotate(data.words, data.features), ' ')
end

function Translator:buildData(src, gold)
  local srcData = {}
  srcData.words = {}
  srcData.features = {}

  local goldData
  if gold then
    goldData = {}
    goldData.words = {}
    goldData.features = {}
  end

  local ignored = {}
  local indexMap = {}
  local index = 1

  for b = 1, #src do
    if #src[b].words == 0 then
      table.insert(ignored, b)
    else
      indexMap[index] = b
      index = index + 1

      table.insert(srcData.words,
                   self.dicts.src.words:convertToIdx(src[b].words, onmt.Constants.UNK_WORD))

      if #self.dicts.src.features > 0 then
        table.insert(srcData.features,
                     onmt.utils.Features.generateSource(self.dicts.src.features, src[b].features))
      end

      if gold then
        table.insert(goldData.words,
                     self.dicts.tgt.words:convertToIdx(gold[b].words,
                                                       onmt.Constants.UNK_WORD,
                                                       onmt.Constants.BOS_WORD,
                                                       onmt.Constants.EOS_WORD))

        if #self.dicts.tgt.features > 0 then
          table.insert(goldData.features,
                       onmt.utils.Features.generateTarget(self.dicts.tgt.features, gold[b].features))
        end
      end
    end
  end

  return onmt.data.Dataset.new(srcData, goldData), ignored, indexMap
end

function Translator:buildTargetWords(pred, src, attn)
  local tokens = self.dicts.tgt.words:convertToLabels(pred, onmt.Constants.EOS)

  if self.opt.replace_unk then
    for i = 1, #tokens do
      if tokens[i] == onmt.Constants.UNK_WORD then
        local _, maxIndex = attn[i]:max(1)
        local source = src[maxIndex[1]]

        if self.phraseTable and self.phraseTable:contains(source) then
          tokens[i] = self.phraseTable:lookup(source)
        else
          tokens[i] = source
        end
      end
    end
  end

  return tokens
end

function Translator:buildTargetFeatures(predFeats)
  local numFeatures = #predFeats[1]

  if numFeatures == 0 then
    return {}
  end

  local feats = {}
  for _ = 1, numFeatures do
    table.insert(feats, {})
  end

  for i = 2, #predFeats do
    for j = 1, numFeatures do
      table.insert(feats[j], self.dicts.tgt.features[j]:lookup(predFeats[i][j]))
    end
  end

  return feats
end

function Translator:translateBatch(batch)
  self.models.encoder:maskPadding()
  self.models.decoder:maskPadding()

  local encStates, context = self.models.encoder:forward(batch)
  if self.opt.dump_input_encoding then
    return encStates[#encStates]
  end
  -- Compute gold score.
  local goldScore
  if batch.targetInput ~= nil then
    if batch.uneven then
      self.models.decoder:maskPadding(batch.sourceSize, batch.sourceLength)
    end
    goldScore = self.models.decoder:computeScore(batch, encStates, context)
  end

  -- Specify how to go one step forward.
  local advancer = onmt.translate.DecoderAdvancer.new(self.models.decoder,
                                                      batch,
                                                      context,
                                                      self.opt.max_sent_length,
                                                      self.opt.max_num_unks,
                                                      encStates,
                                                      self.dicts,
                                                      self.opt.length_norm,
                                                      self.opt.coverage_norm,
                                                      self.opt.eos_norm)

  -- Save memory by only keeping track of necessary elements in the states.
  -- Attentions are at index 4 in the states defined in onmt.translate.DecoderAdvancer.
  local attnIndex = 4

  -- Features are at index 5 in the states defined in onmt.translate.DecoderAdvancer.
  local featsIndex = 5

  advancer:setKeptStateIndexes({attnIndex, featsIndex})

  -- Conduct beam search.
  local beamSearcher = onmt.translate.BeamSearcher.new(advancer)
  local results = beamSearcher:search(self.opt.beam_size, self.opt.n_best, self.opt.pre_filter_factor)

  local allHyp = {}
  local allFeats = {}
  local allAttn = {}
  local allScores = {}

  for b = 1, batch.size do
    local hypBatch = {}
    local featsBatch = {}
    local attnBatch = {}
    local scoresBatch = {}

    for n = 1, self.opt.n_best do
      local result = results[b][n]
      local tokens = result.tokens
      local score = result.score
      local states = result.states
      local attn = states[attnIndex] or {}
      local feats = states[featsIndex] or {}

      -- Ignore generated </s>.
      table.remove(tokens)
      if #attn > 0 then
        table.remove(attn)
      end

      -- Remove unnecessary values from the attention vectors.
      local size = batch.sourceSize[b]
      for j = 1, #attn do
        attn[j] = attn[j]:narrow(1, batch.sourceLength - size + 1, size)
      end

      table.insert(hypBatch, tokens)
      if #feats > 0 then
        table.insert(featsBatch, feats)
      end
      table.insert(attnBatch, attn)
      table.insert(scoresBatch, score)
    end

    table.insert(allHyp, hypBatch)
    table.insert(allFeats, featsBatch)
    table.insert(allAttn, attnBatch)
    table.insert(allScores, scoresBatch)
  end

  return allHyp, allFeats, allScores, allAttn, goldScore
end

--[[ Translate a batch of source sequences.

Parameters:

  * `src` - a batch of tables containing:
    - `words`: the table of source words
    - `features`: the table of feaures sequences (`src.features[i][j]` is the value of the ith feature of the jth token)
  * `gold` - gold data to compute confidence score (same format as `src`)

Returns:

  * `results` - a batch of tables containing:
    - `goldScore`: if `gold` was given, this is the confidence score
    - `preds`: an array of `opt.n_best` tables containing:
      - `words`: the table of target words
      - `features`: the table of target features sequences
      - `attention`: the attention vectors of each target word over the source words
      - `score`: the confidence score of the prediction
]]
function Translator:translate(src, gold)
  local data, ignored, indexMap = self:buildData(src, gold)

  local results = {}

  if data:batchCount() > 0 then
    local batch = data:getBatch()

    local encStates = {}
    local pred = {}
    local predFeats = {}
    local predScore = {}
    local attn = {}
    local goldScore = {}
    if self.opt.dump_input_encoding then
      encStates = self:translateBatch(batch)
    else
      pred, predFeats, predScore, attn, goldScore = self:translateBatch(batch)
    end

    for b = 1, batch.size do
      if self.opt.dump_input_encoding then
        results[b] = encStates[b]
      else
        results[b] = {}

        results[b].preds = {}
        for n = 1, self.opt.n_best do
          results[b].preds[n] = {}
          results[b].preds[n].words = self:buildTargetWords(pred[b][n], src[indexMap[b]].words, attn[b][n])
          results[b].preds[n].features = self:buildTargetFeatures(predFeats[b][n])
          results[b].preds[n].attention = attn[b][n]
          results[b].preds[n].score = predScore[b][n]
        end
      end

      if goldScore and next(goldScore) ~= nil then
        results[b].goldScore = goldScore[b]
      end
    end
  end

  for i = 1, #ignored do
    table.insert(results, ignored[i], {})
  end

  return results
end

return Translator


================================================
FILE: opennmt/translate/init.lua
================================================
local translate = {}

translate.Advancer = require('opennmt.translate.Advancer')
translate.Beam = require('opennmt.translate.Beam')
translate.BeamSearcher = require('opennmt.translate.BeamSearcher')
translate.DecoderAdvancer = require('opennmt.translate.DecoderAdvancer')
translate.PhraseTable = require('opennmt.translate.PhraseTable')
translate.Translator = require('opennmt.translate.Translator')

return translate


================================================
FILE: opennmt/utils/CrayonLogger.lua
================================================
local CrayonLogger = torch.class('CrayonLogger')

local options = {
  {
    '-exp_host', '127.0.0.1',
    [[Crayon server IP.]]
  },
  {
    '-exp_port', '8889',
    [[Crayon server port.]]
  },
  {
    '-exp', '',
    [[Crayon experiment name.]]
  }
}

function CrayonLogger.declareOpts(cmd)
  cmd:setCmdLineOptions(options, 'Crayon')
end

function CrayonLogger:__init(args)
  self.args = onmt.utils.ExtendedCmdLine.getModuleOpts(args, options)
  if args.exp ~= '' then
    self.host = self.args.exp_host
    self.port = self.args.exp_port

    local crayon = require("crayon")
    self.cc = crayon.CrayonClient(self.host, self.port)
    self.exp = self.cc:create_experiment(args.exp)
    self.on = true


  else
    self.on = false
  end
end

return CrayonLogger


================================================
FILE: opennmt/utils/Cuda.lua
================================================
local ExtendedCmdLine = require('opennmt.utils.ExtendedCmdLine')

local Cuda = {
  fp16 = false,
  gpuIds = {},
  activated = false
}

local options = {
  {
    '-gpuid', '0',
    [[List of comma-separated GPU identifiers (1-indexed). CPU is used when set to 0.]],
    {
      valid = ExtendedCmdLine.listUInt
    }
  },
  {
    '-fallback_to_cpu', false,
    [[If GPU can't be used, rollback on the CPU.]]
  },
  {
    '-fp16', false,
    [[Use half-precision float on GPU.]]
  },
  {
    '-no_nccl', false,
    [[Disable usage of nccl in parallel mode.]]
  }
}

function Cuda.declareOpts(cmd)
  cmd:setCmdLineOptions(options, 'Cuda')
end

function Cuda.init(opt, masterGPU)
  for _, val in ipairs(onmt.utils.String.split(opt.gpuid, ',')) do
    local id = tonumber(val)
    assert(id ~= nil and id >= 0, 'invalid GPU identifier: ' .. val)
    if id > 0 then
      table.insert(Cuda.gpuIds, id)
    end
  end

  Cuda.activated = #Cuda.gpuIds > 0

  if Cuda.activated then
    local _, err = pcall(function()
      require('cutorch')
      require('cunn')
      Cuda.fp16 = opt.fp16

      if masterGPU == nil then
        masterGPU = 1

        -- Validate GPU identifiers.
        for i = 1, #Cuda.gpuIds do
          assert(Cuda.gpuIds[i] <= cutorch.getDeviceCount(),
                 'GPU ' .. Cuda.gpuIds[i] .. ' is requested but only '
                   .. cutorch.getDeviceCount() .. ' GPUs are available')
        end

        _G.logger:info('Using GPU(s): ' .. table.concat(Cuda.gpuIds, ', '))

        if cutorch.isCachingAllocatorEnabled and cutorch.isCachingAllocatorEnabled() then
          _G.logger:warning('The caching CUDA memory allocator is enabled. This allocator improves performance at the cost of a higher GPU memory usage. To optimize for memory, consider disabling it by setting the environment variable: THC_CACHING_ALLOCATOR=0')
        end

      end

      cutorch.setDevice(Cuda.gpuIds[masterGPU])

      if opt.seed then
        cutorch.manualSeed(opt.seed)
      end
    end)

    if err then
      if opt.fallback_to_cpu then
        _G.logger:warning('Falling back to CPU')
        Cuda.activated = false
      else
        error(err)
      end
    end
    if Cuda.fp16 and not cutorch.hasHalf then
      error("installed cutorch does not support half-tensor")
    end
  end
end

-- returns RNGState for CPU and enabled GPUs
function Cuda.getRNGStates()
  local rngStates = { torch.getRNGState() }
  for _,idx in ipairs(Cuda.gpuIds) do
    table.insert(rngStates, cutorch.getRNGState(idx))
  end
  return rngStates
end

-- set RNGState from saved state
function Cuda.setRNGStates(rngStates, verbose)
  if not rngStates then
    return
  end
  if verbose then
    _G.logger:info("Restoring Random Number Generator states")
  end
  torch.setRNGState(rngStates[1])
  if #rngStates-1 ~= #Cuda.gpuIds then
    _G.logger:warning('GPU count does not match for resetting Random Number Generator - skipping')
  else
    for idx = 2, #rngStates do
      cutorch.setRNGState(rngStates[idx], idx-1)
    end
  end
end

--[[
  Recursively move all supported objects in `obj` on the GPU.
  When using CPU only, converts to float instead of the default double.
]]
function Cuda.convert(obj)
  local objtype = torch.typename(obj)
  if objtype then
    if Cuda.activated and obj.cuda ~= nil then
      if objtype:find('torch%..*LongTensor') then
        return obj:type('torch.CudaLongTensor')
      elseif Cuda.fp16 then
        return obj:type('torch.CudaHalfTensor')
      else
        return obj:type('torch.CudaTensor')
      end
    elseif not Cuda.activated and obj.float ~= nil then
      -- Defaults to float instead of double.
      if objtype:find('torch%..*LongTensor') then
        return obj:type('torch.LongTensor')
      else
        return obj:type('torch.FloatTensor')
      end
    end
  end

  if objtype or type(obj) == 'table' then
    for k, v in pairs(obj) do
      obj[k] = Cuda.convert(v)
    end
  end

  return obj
end

--[[
  Synchronize operations on current device if working on GPU.
  Do nothing otherwise.
]]
function Cuda.synchronize()
  if Cuda.activated then cutorch.synchronize() end
end

--[[
  Number of available GPU.
]]
function Cuda.gpuCount()
  return #Cuda.gpuIds
end

--[[
  Free memory on the current GPU device.
]]
function Cuda.freeMemory()
  if Cuda.activated then
    local freeMemory = cutorch.getMemoryUsage(cutorch.getDevice())
    return freeMemory
  end
  return 0
end

return Cuda


================================================
FILE: opennmt/utils/Dict.lua
================================================
local Dict = torch.class("Dict")

function Dict:__init(data)
  self.idxToLabel = {}
  self.labelToIdx = {}
  self.frequencies = {}

  -- Special entries will not be pruned.
  self.special = {}

  if data ~= nil then
    if type(data) == "string" then -- File to load.
      self:loadFile(data)
    else
      self:addSpecials(data)
    end
  end
end

--[[ Return the number of entries in the dictionary. ]]
function Dict:size()
  return #self.idxToLabel
end

--[[ Load entries from a file. ]]
function Dict:loadFile(filename)
  local reader = onmt.utils.FileReader.new(filename)

  while true do
    local fields = reader:next()

    if not fields then
      break
    end

    local label = fields[1]
    local idx = tonumber(fields[2])

    self:add(label, idx)
  end

  reader:close()
end

--[[ Write entries to a file. ]]
function Dict:writeFile(filename)
  local file = assert(io.open(filename, 'w'))

  for i = 1, self:size() do
    local label = self.idxToLabel[i]
    file:write(label .. ' ' .. i .. '\n')
  end

  file:close()
end

--[[ Lookup `key` in the dictionary: it can be an index or a string. ]]
function Dict:lookup(key)
  if type(key) == "string" then
    return self.labelToIdx[key]
  else
    return self.idxToLabel[key]
  end
end

--[[ Mark this `label` and `idx` as special (i.e. will not be pruned). ]]
function Dict:addSpecial(label, idx)
  idx = self:add(label, idx)
  table.insert(self.special, idx)
end

--[[ Mark all labels in `labels` as specials (i.e. will not be pruned). ]]
function Dict:addSpecials(labels)
  for i = 1, #labels do
    self:addSpecial(labels[i])
  end
end

--[[ Add `label` in the dictionary. Use `idx` as its index if given. ]]
function Dict:add(label, idx)
  if idx ~= nil then
    self.idxToLabel[idx] = label
    self.labelToIdx[label] = idx
  else
    idx = self.labelToIdx[label]
    if idx == nil then
      idx = #self.idxToLabel + 1
      self.idxToLabel[idx] = label
      self.labelToIdx[label] = idx
    end
  end

  if self.frequencies[idx] == nil then
    self.frequencies[idx] = 1
  else
    self.frequencies[idx] = self.frequencies[idx] + 1
  end

  return idx
end

--[[ Return a new dictionary with the `size` most frequent entries. ]]
function Dict:prune(size)
  if size >= self:size() then
    return self
  end

  -- Only keep the `size` most frequent entries.
  local freq = torch.Tensor(self.frequencies)
  local _, idx = torch.sort(freq, 1, true)

  local newDict = Dict.new()

  -- Add special entries in all cases.
  for i = 1, #self.special do
    newDict:addSpecial(self.idxToLabel[self.special[i]])
  end

  for i = 1, size do
    newDict:add(self.idxToLabel[idx[i]])
  end

  return newDict
end

--[[ Return a new dictionary with entries appearing at least `minFrequency` times. ]]
function Dict:pruneByMinFrequency(minFrequency)
  if minFrequency < 2 then
    return self
  end

  local freq = torch.Tensor(self.frequencies)
  local sortedFreq, idx = torch.sort(freq, 1, true)

  local newDict = Dict.new()

  -- Add special entries in all cases.
  for i = 1, #self.special do
    newDict:addSpecial(self.idxToLabel[self.special[i]])
  end

  for i = 1, self:size() do
    if sortedFreq[i] < minFrequency then
      break
    end
    newDict:add(self.idxToLabel[idx[i]])
  end

  return newDict
end

--[[
  Convert `labels` to indices. Use `unkWord` if not found.
  Optionally insert `bosWord` at the beginning and `eosWord` at the end.
]]
function Dict:convertToIdx(labels, unkWord, bosWord, eosWord)
  local vec = {}

  if bosWord ~= nil then
    table.insert(vec, self:lookup(bosWord))
  end

  for i = 1, #labels do
    local idx = self:lookup(labels[i])
    if idx == nil then
      idx = self:lookup(unkWord)
    end
    table.insert(vec, idx)
  end

  if eosWord ~= nil then
    table.insert(vec, self:lookup(eosWord))
  end

  return torch.IntTensor(vec)
end

--[[ Convert `idx` to labels. If index `stop` is reached, convert it and return. ]]
function Dict:convertToLabels(idx, stop)
  local labels = {}

  for i = 1, #idx do
    table.insert(labels, self:lookup(idx[i]))
    if idx[i] == stop then
      break
    end
  end

  return labels
end

return Dict


================================================
FILE: opennmt/utils/ExtendedCmdLine.lua
================================================
---------------------------------------------------------------------------------
-- Local utility functions
---------------------------------------------------------------------------------

--[[ Convert `val` string to its actual type (boolean, number or string). ]]
local function convert(key, val, ref)
  local new

  if type(val) == type(ref) then
    new = val
  elseif type(ref) == 'boolean' then
    if val == 'true' then
      new = true
    elseif val == 'false' then
      new = false
    else
      error('option ' .. key .. ' expects a boolean value [true, false]')
    end
  elseif type(ref) == 'number' then
    new = tonumber(val)
    assert(new ~= nil, 'option ' .. key .. ' expects a number value')
  end

  return new
end

local function wrapIndent(text, size, pad)
  local p = 0
  while true do
    local q = text:find(" ", size+p)
    if not q then
      return text
    end
    text = text:sub(1, q) .. "\n" ..pad .. text:sub(q+1, #text)
    p = q+2+#pad
  end
end

---------------------------------------------------------------------------------

local ExtendedCmdLine, parent, path
-- the utils function can be run without torch
if torch then
  ExtendedCmdLine, parent = torch.class('ExtendedCmdLine', 'torch.CmdLine')
  path = require('pl.path')
else
  ExtendedCmdLine = {}
end

--[[
  Extended handling of command line options - provide validation methods, and utilities for handling options
  at module level.

  For end-user - provides '-h' for help, with possible '-md' variant for preparing MD ready help file.

  Provides also possibility to read options from config file with '-config file' or save current options to
  config file with '-save_config file'

  Example:

    cmd = onmt.utils.ExtendedCmdLine.new()
    local options = {
      {'-max_batch_size',     64   , 'Maximum batch size', {valid=onmt.utils.ExtendedCmdLine.isUInt()}}
    }

    cmd:setCmdLineOptions(options, 'Optimization')
    local opt = cmd:parse(arg)

    local optimArgs = cmd.getModuleOpts(opt, options)

  Additional meta-fields:

  * `valid`: validation method
  * `enum`: enumeration list
  * `structural`: if defined, mark a structural parameter - 0 means cannot change value, 1 means that it can change dynamically
  * `init_only`: if true, mark a parameter that can only be set at init time
  * `train_state`: if true, the option will be automatically reused when continuing a training

]]

function ExtendedCmdLine:__init(script)
  self.script = script
  parent.__init(self)

  self:text('')
  self:option('-h', false, 'This help.')
  self:option('-md', false, 'Dump help in Markdown format.')
  self:option('-config', '', 'Load options from this file.', {valid=ExtendedCmdLine.fileNullOrExists})
  self:option('-save_config', '', 'Save options to this file.')

end

function ExtendedCmdLine:help(arg, doMd)
  if doMd then
    io.write('`' .. self.script .. '` options:\n')
    for _, option in ipairs(self.helplines) do
      if type(option) == 'table' then
        io.write('* ')
        if option.default ~= nil then -- It is an option.
          local args = type(option.default) == 'boolean' and '' or ' <' .. type(option.default) .. '>'
          io.write('`' .. option.key .. args ..'`')

          local valInfo = {}
          if option.meta and option.meta.enum then
            for k, v in pairs(option.meta.enum) do
              option.meta.enum[k] = '`' .. v .. '`'
            end
            table.insert(valInfo, 'accepted: ' .. table.concat(option.meta.enum, ', '))
          end
          if type(option.default) ~= "boolean" and option.default ~= '' then
            table.insert(valInfo, 'default: `' .. tostring(option.default) .. '`')
          end
          if #valInfo > 0 then
            io.write(' (' .. table.concat(valInfo, '; ') .. ')')
          end

          io.write('<br/>')

          option.help = option.help:gsub(' *\n   *', ' ')
          if option.help then
            io.write(option.help)
          end
        else -- It is an argument.
          io.write('<' .. onmt.utils.String.stripHyphens(option.key) .. '>')
          if option.help then
            io.write(' ' .. option.help)
          end
        end
      else
        local display = option:gsub('%*', '')
        if display:len() > 0 then
          io.write('## ')
        end
        io.write(display) -- Just some additional help.
      end
      io.write('\n')
    end
    io.write('\n')
  else
    if arg then
      io.write('Usage: ')
      io.write(arg[0] .. ' ' .. self.script .. ' ')
      io.write('[options] ')
      for i = 1, #self.arguments do
        io.write('<' .. onmt.utils.String.stripHyphens(self.arguments[i].key) .. '>')
      end
      io.write('\n')
    end

    -- First pass to compute max length.
    local optsz = 0
    for _, option in ipairs(self.helplines) do
      if type(option) == 'table' then
        if option.default ~= nil then -- It is an option.
          if #option.key > optsz then
            optsz = #option.key
          end
        else -- It is an argument.
          local stripOptionKey = onmt.utils.String.stripHyphens(option.key)
          if #stripOptionKey + 2 > optsz then
            optsz = #stripOptionKey + 2
          end
        end
      end
    end

    local padMultiLine = onmt.utils.String.pad('', optsz)
    -- Second pass to print.
    for _, option in ipairs(self.helplines) do
      if type(option) == 'table' then
        io.write('  ')
        if option.default ~= nil then -- It is an option.
          io.write(onmt.utils.String.pad(option.key, optsz))
          local msg = ''
          msg = msg .. option.help:gsub('\n', ' ')
          local valInfo = {}
          if option.meta and option.meta.enum then
            table.insert(valInfo, 'accepted: ' .. table.concat(option.meta.enum, ', '))
          end
          if type(option.default) ~= "boolean" and option.default ~= '' then
            table.insert(valInfo, 'default: ' .. tostring(option.default))
          end
          if #valInfo > 0 then
            msg = msg .. ' (' .. table.concat(valInfo, '; ') .. ')'
          end
          io.write(' ' .. wrapIndent(msg:gsub('  *', ' '),60,padMultiLine..'     '))
        else -- It is an argument.
          io.write(onmt.utils.String.pad('<' .. onmt.utils.String.stripHyphens(option.key) .. '>', optsz))
          if option.help then
            io.write(' ' .. option.help)
          end
        end
      else
        io.write(option) -- Just some additional help.
      end
      io.write('\n')
    end
    io.write('\n')
  end
end

function ExtendedCmdLine:error(msg)
   print('ERROR: '..msg)
   print('> Use -h for help\n')
   os.exit(0)
end

function ExtendedCmdLine:option(key, default, help, _meta_)
  for _,v in ipairs(self.helplines) do
    if v.key == key then
      return
    end
  end
  parent.option(self, key, default, help)
  self.options[key].meta = _meta_
end

--[[ Override options with option values set in file `filename`. ]]
function ExtendedCmdLine:loadConfig(filename, opt)
  local file = assert(io.open(filename, 'r'))

  for line in file:lines() do
    -- Ignore empty or commented out lines.
    if line:len() > 0 and string.sub(line, 1, 1) ~= '#' then
      local field = onmt.utils.String.split(line, '=')
      assert(#field == 2, 'badly formatted config file')

      local key = onmt.utils.String.strip(field[1])
      local val = onmt.utils.String.strip(field[2])

      assert(opt[key] ~= nil, 'unkown option ' .. key)

      opt[key] = convert(key, val, opt[key])
      opt._is_default[key] = nil

    end
  end

  file:close()
  return opt
end

function ExtendedCmdLine:logConfig(opt)
  local keys = {}
  for key in pairs(opt) do
    table.insert(keys, key)
  end

  table.sort(keys)
  _G.logger:debug('Options:')

  for _, key in ipairs(keys) do
    if key:sub(1, 1) ~= '_' then
      local val = opt[key]
      if type(val) == 'string' then
        val = '\'' .. val .. '\''
      end
      _G.logger:debug(' * ' .. key .. ' = ' .. tostring(val))
    end
  end
end

function ExtendedCmdLine:dumpConfig(opt, filename)
  local file = assert(io.open(filename, 'w'))

  for key, val in pairs(opt) do
    if key:sub(1, 1) ~= '_' then
      file:write(key .. ' = ' .. tostring(val) .. '\n')
    end
  end

  file:close()
end

function ExtendedCmdLine:parse(arg)
  local i = 1

  -- set default value
  local params = { _is_default={}, _structural={}, _init_only={}, _train_state={} }
  for option,v in pairs(self.options) do
    local soption = onmt.utils.String.stripHyphens(option)
    params[soption] = v.default
    params._is_default[soption] = true
  end

  local nArgument = 0

  local doHelp = false
  local doMd = false
  local readConfig
  local saveConfig

  local cmdlineOptions = {}

  while i <= #arg do
    if arg[i] == '-help' or arg[i] == '-h' or arg[i] == '--help' then
      doHelp = true
      i = i + 1
    elseif arg[i] == '-md' then
      doMd = true
      i = i + 1
    elseif arg[i] == '-config' then
      readConfig = arg[i + 1]
      i = i + 2
    elseif arg[i] == '-save_config' then
      saveConfig = arg[i + 1]
      i = i + 2
    else
      local sopt = onmt.utils.String.stripHyphens(arg[i])
      params._is_default[sopt] = nil
      if self.options[arg[i]] then
        if cmdlineOptions[arg[i]] then
          self:error('duplicate cmdline option: '..arg[i])
        end
        cmdlineOptions[arg[i]] = true
        i = i + self:__readOption__(params, arg, i)
      else
        nArgument = nArgument + 1
        i = i + self:__readArgument__(params, arg, i, nArgument)
      end
    end
  end

  if doHelp then
    self:help(arg, doMd)
    os.exit(0)
  end

  if nArgument ~= #self.arguments then
    self:error('not enough arguments')
  end

  if readConfig then
    params = self:loadConfig(readConfig, params)
  end

  if saveConfig then
    self:dumpConfig(params, saveConfig)
  end

  for k, v in pairs(params) do
    if k:sub(1, 1) ~= '_' then
      local K = '-' .. k
      if not self.options[K] and self.options[k] then
        K = k
      end
      local meta = self.options[K].meta
      if meta then
        -- check option validity
        local isValid = true
        local reason = nil

        if meta.valid then
          isValid, reason = meta.valid(v)
        end

        if not isValid then
          local msg = 'invalid option -' .. k
          if reason then
            msg = msg .. ': ' .. reason
          end
          self:error(msg)
        end

        if meta.enum and not onmt.utils.Table.hasValue(meta.enum, v) then
          self:error('option -' .. k.. ' is not in accepted values: ' .. table.concat(meta.enum, ', '))
        end
        if meta.structural then
          params._structural[k] = meta.structural
        end
        if meta.init_only then
          params._init_only[k] = meta.init_only
        end
        if meta.train_state then
          params._train_state[k] = meta.train_state
        end
      end
    end
  end

  return params
end

function ExtendedCmdLine:setCmdLineOptions(moduleOptions, group)
  if group then
    self:text('')
    self:text('**' .. group .. ' options**')
    self:text('')
  end

  for i = 1, #moduleOptions do
    if type(moduleOptions[i]) == 'table' then
      self:option(table.unpack(moduleOptions[i]))
    else
      self:argument(moduleOptions[i])
    end
  end
end

function ExtendedCmdLine.getModuleOpts(args, moduleOptions)
  local moduleArgs = {}
  for i = 1, #moduleOptions do
    local optname = moduleOptions[i][1]
    if optname:sub(1, 1) == '-' then
      optname = optname:sub(2)
    end
    moduleArgs[optname] = args[optname]
  end
  return moduleArgs
end

function ExtendedCmdLine.getArgument(args, optName)
  for i = 1, #args do
    if args[i] == optName and i < #args then
      return args[i + 1]
    end
  end
  return nil
end

---------------------------------------------------------------------------------
-- Validators
---------------------------------------------------------------------------------

local function buildRangeError(prefix, minValue, maxValue)
  local err = 'the ' .. prefix .. ' should be'
  if minValue then
    err = err .. ' greater than ' .. minValue
  end
  if maxValue then
    if minValue then
      err = err .. ' and'
    end
    err = err .. ' lower than ' .. maxValue
  end
  return err
end

-- Check if is integer between minValue and maxValue.
function ExtendedCmdLine.isInt(minValue, maxValue)
  return function(v)
    return (math.floor(v) == v and
      (not minValue or v >= minValue) and
      (not maxValue or v <= maxValue)),
      buildRangeError('integer', minValue, maxValue)
    end
end

-- Check if is positive integer.
function ExtendedCmdLine.isUInt(maxValue)
  return ExtendedCmdLine.isInt(0, maxValue)
end

-- Check if list of positive integers.
function ExtendedCmdLine.listUInt(v)
  local sv = tostring(v)
  local p = 1

  while true do
    local q
    p, q = sv:find('%d+',p)
    if q == #sv then
      return true
    end
    if not p or sv:sub(q+1,q+1) ~= ',' then
      return false
    end
    p = q+2
  end
end

-- Check if value between minValue and maxValue.
function ExtendedCmdLine.isFloat(minValue, maxValue)
  return function(v)
    return (type(v) == 'number' and
      (not minValue or v >= minValue) and
      (not maxValue or v <= maxValue)),
      buildRangeError('number', minValue, maxValue)
    end
end

-- Check if non empty.
function ExtendedCmdLine.nonEmpty(v)
  return v and v ~= '', 'the argument should not be empty'
end

-- Check if the corresponding file exists.
function ExtendedCmdLine.fileExists(v)
  return path.exists(v), 'the file should exist'
end

-- Check non set or if the corresponding file exists.
function ExtendedCmdLine.fileNullOrExists(v)
  return v == '' or ExtendedCmdLine.fileExists(v), 'if set, the file should exist'
end

-- Check it is a directory and some file exists
function ExtendedCmdLine.dirStructure(files)
  return function(v)
    for _,f in ipairs(files) do
      if not path.exists(v.."/"..f) then
        return false, 'the directory should exist'
      end
    end
    return true
  end
end

return ExtendedCmdLine


================================================
FILE: opennmt/utils/Features.lua
================================================
-- tds is lazy loaded.
local tds

--[[ Separate words and features (if any). ]]
local function extract(tokens)
  local words = {}
  local features = {}
  local numFeatures = nil

  for t = 1, #tokens do
    local field = onmt.utils.String.split(tokens[t], '￨')
    local word = field[1]

    if word:len() > 0 then
      table.insert(words, word)

      if numFeatures == nil then
        numFeatures = #field - 1
      else
        assert(#field - 1 == numFeatures,
               'all words must have the same number of features')
      end

      if #field > 1 then
        for i = 2, #field do
          if features[i - 1] == nil then
            features[i - 1] = {}
          end
          table.insert(features[i - 1], field[i])
        end
      end
    end
  end
  return words, features, numFeatures or 0
end

--[[ Reverse operation: attach features to tokens. ]]
local function annotate(tokens, features)
  if not features or #features == 0 then
    return tokens
  end

  for i = 1, #tokens do
    for j = 1, #features do
      tokens[i] = tokens[i] .. '￨' .. features[j][i]
    end
  end

  return tokens
end

--[[ Check that data contains the expected number of features. ]]
local function check(label, dicts, data)
  local expected = #dicts
  local got = 0
  if data ~= nil then
    got = #data
  end

  assert(expected == got, "expected " .. expected .. " " .. label .. " features, got " .. got)
end

--[[ Generate source sequences from labels. ]]
local function generateSource(dicts, src, cdata)
  check('source', dicts, src)

  local srcId
  if cdata then
    if not tds then
      tds = require('tds')
    end
    srcId = tds.Vec()
  else
    srcId = {}
  end

  for j = 1, #dicts do
    srcId[j] = dicts[j]:convertToIdx(src[j], onmt.Constants.UNK_WORD)
  end

  return srcId
end

--[[ Generate target sequences from labels. ]]
local function generateTarget(dicts, tgt, cdata, shift_feature)
  check('target', dicts, tgt)

  -- back compatibility
  shift_feature = shift_feature or 1
  local tgtId
  if cdata then
    if not tds then
      tds = require('tds')
    end
    tgtId = tds.Vec()
  else
    tgtId = {}
  end

  for j = 1, #dicts do
    -- if shift_feature then target features are shifted relative to the target words.
    -- Use EOS tokens as a placeholder.
    table.insert(tgt[j], 1, onmt.Constants.BOS_WORD)
    if shift_feature == 1 then
      table.insert(tgt[j], 1, onmt.Constants.EOS_WORD)
    else
      table.insert(tgt[j], onmt.Constants.EOS_WORD)
    end
    tgtId[j] = dicts[j]:convertToIdx(tgt[j], onmt.Constants.UNK_WORD)
    table.remove(tgt[j], 1)
    if shift_feature == 1 then
      table.remove(tgt[j], 1)
    else
      table.remove(tgt[j])
    end
  end

  return tgtId
end

return {
  extract = extract,
  annotate = annotate,
  generateSource = generateSource,
  generateTarget = generateTarget
}


================================================
FILE: opennmt/utils/FileReader.lua
================================================
local FileReader = torch.class("FileReader")

function FileReader:__init(filename)
  self.file = assert(io.open(filename, "r"))
end

--[[ Read next line in the file and split it on spaces. If EOF is reached, returns nil. ]]
function FileReader:next()
  local line = self.file:read()

  if line == nil then
    return nil
  end

  local sent = {}
  for word in line:gmatch'([^%s]+)' do
    table.insert(sent, word)
  end

  return sent
end

function FileReader:close()
  self.file:close()
end

return FileReader


================================================
FILE: opennmt/utils/Logger.lua
================================================
--[[ Logger is a class used for maintaining logs in a log file.
--]]
local Logger = torch.class('Logger')

local options = {
  {
    '-log_file', '',
    [[Output logs to a file under this path instead of stdout.]]
  },
  {
    '-disable_logs', false,
    [[If set, output nothing.]]
  },
  {
    '-log_level', 'INFO',
    [[Output logs at this level and above.]],
    {
      enum = {'DEBUG', 'INFO', 'WARNING', 'ERROR'}
    }
  }
}

function Logger.declareOpts(cmd)
  cmd:setCmdLineOptions(options, 'Logger')
end

--[[ Construct a Logger object.

Parameters:
  * `logFile` - Outputs logs to a file under this path instead of stdout. ['']
  * `disableLogs` - If = true, output nothing. [false]
  * `logLevel` - Outputs logs at this level and above. Possible options are: DEBUG, INFO, WARNING and ERROR. ['INFO']

Example:

    logger = onmt.utils.Logger.new('log.txt')
    logger:info('%s is an extension of OpenNMT.', 'Im2Text')
    logger:shutDown()

]]
function Logger:__init(logFile, disableLogs, logLevel)
  logFile = logFile or ''
  disableLogs = disableLogs or false
  logLevel = logLevel or 'INFO'

  self.mute = (logFile:len() > 0)
  if disableLogs then
    self:setVisibleLevel('ERROR')
  else
    self:setVisibleLevel(logLevel)
  end
  local openMode = 'w'
  local f = io.open(logFile, 'r')
  if f then
    f:close()
    local input = nil
    while not input do
      print('Logging file exits. Overwrite(o)? Append(a)? Abort(q)?')
      input = io.read()
      if input == 'o' or input == 'O' then
        openMode = 'w'
      elseif input == 'a' or input == 'A' then
        openMode = 'a'
      elseif input == 'q' or input == 'Q' then
        os.exit()
      else
        openMode = 'a'
      end
    end
  end
  if string.len(logFile) > 0 then
    self.logFile = io.open(logFile, openMode)
  else
    self.logFile = nil
  end
  self.LEVELS = { DEBUG = 0, INFO = 1, WARNING = 2, ERROR = 3 }
end

--[[ Log a message at a specified level.

Parameters:
  * `message` - the message to log.
  * `level` - the desired message level. ['INFO']

]]
function Logger:log(message, level)
  level = level or 'INFO'
  local timeStamp = os.date('%x %X')
  local msgFormatted = string.format('[%s %s] %s', timeStamp, level, message)
  if (not self.mute) and self:_isVisible(level) then
    print (msgFormatted)
  end
  if self.logFile and self:_isVisible(level) then
    self.logFile:write(msgFormatted .. '\n')
    self.logFile:flush()
  end
end

--[[ Log a message at 'INFO' level.

Parameters:
  * `message` - the message to log. Supports formatting string.

]]
function Logger:info(...)
  self:log(self:_format(...), 'INFO')
end

--[[ Log a message at 'WARNING' level.

Parameters:
  * `message` - the message to log. Supports formatting string.

]]
function Logger:warning(...)
  self:log(self:_format(...), 'WARNING')
end

--[[ Log a message at 'ERROR' level.

Parameters:
  * `message` - the message to log. Supports formatting string.

]]
function Logger:error(...)
  self:log(self:_format(...), 'ERROR')
end

--[[ Log a message at 'DEBUG' level.

Parameters:
  * `message` - the message to log. Supports formatting string.

]]
function Logger:debug(...)
  self:log(self:_format(...), 'DEBUG')
end

--[[ Log a message as exactly it is.

Parameters:
  * `message` - the message to log. Supports formatting string.

]]
function Logger:writeMsg(...)
  local msg = self:_format(...)
  if (not self.mute) and self:_isVisible('WARNING') then
    io.write(msg)
  end
  if self.logFile and self:_isVisible('WARNING') then
    self.logFile:write(msg)
    self.logFile:flush()
  end
end

--[[ Set the visible message level. Lower level messages will be muted.

Parameters:
  * `level` - 'DEBUG', 'INFO', 'WARNING' or 'ERROR'.

]]
function Logger:setVisibleLevel(level)
  assert (level == 'DEBUG' or level == 'INFO' or
          level == 'WARNING' or level == 'ERROR')
  self.level = level
end

-- Private function for comparing level against visible level.
-- `level` - 'DEBUG', 'INFO', 'WARNING' or 'ERROR'.
function Logger:_isVisible(level)
  self.level = self.level or 'INFO'
  return self.LEVELS[level] >= self.LEVELS[self.level]
end

function Logger:_format(...)
  if #table.pack(...) == 1 then
    return ...
  else
    return string.format(...)
  end
end

--[[ Deconstructor. Close the log file.
]]
function Logger:shutDown()
  if self.logFile then
    self.logFile:close()
  end
end

return Logger


================================================
FILE: opennmt/utils/Memory.lua
================================================
local Memory = {}

local options = {
  {
    '-disable_mem_optimization', false,
    [[Disable sharing of internal buffers between clones for visualization or development.]]
  }
}

function Memory.declareOpts(cmd)
  cmd:setCmdLineOptions(options)
end

--[[ Optimize memory usage of Neural Machine Translation.

Parameters:
  * `model` - a table containing encoder and decoder
  * `batch` - a Batch object
  * `verbose` - produce output or not

Example:

  local model = {}
  model.encoder = onmt.Models.buildEncoder(...)
  model.decoder = onmt.Models.buildDecoder(...)
  Memory.optimize(model, batch, verbose)

]]
function Memory.optimize(model, batch, verbose)

  if verbose then
    _G.logger:info('Preparing memory optimization...')
  end

  -- Prepare memory optimization
  local memoryOptimizer = onmt.utils.MemoryOptimizer.new(model.models)

  -- Batch of one single word since we optimize the first clone.
  local realSizes = { sourceLength = batch.sourceLength, targetLength = batch.targetLength, uneven = batch.uneven }

  batch.sourceLength = 1
  batch.targetLength = 1
  batch.uneven = false

  model:trainNetwork(batch, true)

  -- mark shared tensors
  local sharedSize, totSize = memoryOptimizer:optimize()

  if verbose then
    _G.logger:info(' * sharing %d%% of output/gradInput tensors memory between clones', (sharedSize / totSize)*100)
  end

  -- Restore batch to be transparent for the calling code.
  batch.sourceLength = realSizes.sourceLength
  batch.targetLength = realSizes.targetLength
  batch.uneven = realSizes.uneven
end

return Memory


================================================
FILE: opennmt/utils/MemoryOptimizer.lua
================================================
--[[ MemoryOptimizer is a class used for optimizing memory usage
--]]
local MemoryOptimizer = torch.class('MemoryOptimizer')

-- We cannot share the output of these modules as they use it in their backward pass.
local protectOutput = {
  'nn.Sigmoid',
  'nn.SoftMax',
  'nn.Tanh'
}

-- We cannot share the input of these modules as they use it in their backward pass.
local protectInput = {
  'nn.Linear',
  'nn.JoinTable',
  'onmt.JoinReplicateTable',
  'nn.CMulTable',
  'nn.MM'
}

local function contains(list, m)
  for i = 1, #list do
    if torch.typename(m) == list[i] then
      return true
    end
  end
  return false
end

local function tensorIncluded(t, l)
  if torch.isTensor(l) then
    return torch.pointer(t:storage()) == torch.pointer(l:storage())
  elseif torch.type(l) == 'table' then
    for _, m in ipairs(l) do
      if tensorIncluded(t, m) then
        return true
      end
    end
  end
  return false
end

-- We cannot share a tensor if it is exposed or coming from outside of the net
-- otherwise we could generate side-effects.
local function canShare(t, net, protected)
  if torch.isTensor(t) and t:storage() then
    if not tensorIncluded(t, net.gradInput) and not tensorIncluded(t, net.output) and not tensorIncluded(t, protected) then
      return true
    end
  elseif torch.type(t) == 'table' then
    for _, m in ipairs(t) do
      if not canShare(m, net, protected) then
        return false
      end
    end
    return true
  end
  return false
end

local function getSize(t, mempool)
  local size = 0
  if torch.isTensor(t) then
    if t:storage() then
      if not mempool[torch.pointer(t:storage())] then
        mempool[torch.pointer(t:storage())] = t:storage():size()*t:elementSize()
        return mempool[torch.pointer(t:storage())]
      end
    end
  elseif torch.type(t) == 'table' then
    for _, m in ipairs(t) do
      size = size + getSize(m, mempool)
    end
  end
  return size
end

-- Convenience function to register a network to optimize.
local function registerNet(store, net, base)
  store.net = net
  store.base = base
  store.forward = net.forward
  net.forward = function(network, input)
    store.input = input
    return store.forward(network, input)
  end
  store.backward = net.backward
  net.backward = function(network, input, gradOutput)
    store.gradOutput = gradOutput
    return store.backward(network, input, gradOutput)
  end

  -- Add a wrapper around updateOutput to catch the module input.
  net:apply(function (m)
    local updateOutput = m.updateOutput
    m.updateOutput = function (mod, input)
      mod.input = input
      return updateOutput(mod, input)
    end
  end)
end

--[[ Construct a MemoryOptimizer object. In this function, forward and backward function will
--  be overwrited to record input and gradOutput in order to determine which tensors can be shared.

Parameters:
  * `modules` - a list of modules to optimize.

Example:

  local memoryOptimizer = onmt.utils.MemoryOptimizer.new(model) -- prepare memory optimization.
  model:forward(...) -- initialize output tensors
  model:backward(...) -- intialize gradInput tensors
  memoryOptimizer.optimize(model) -- actual optimization by marking shared tensors

]]
function MemoryOptimizer:__init(modules)
  self.modelDesc = {}

  for name, mod in pairs(modules) do
    self.modelDesc[name] = {}

    if torch.isTypeOf(mod, 'onmt.Sequencer') then
      -- If the module directly contains a network, take the first clone.
      self.modelDesc[name][1] = {}
      registerNet(self.modelDesc[name][1], mod:net(1), mod.network)
    elseif mod.modules then
      -- Otherwise, look in submodules instead.
      local i = 1
      mod:apply(function(m)
        if m.network then
          self.modelDesc[name][i] = {}
          registerNet(self.modelDesc[name][i], m:net(1), m.network)
          i = i + 1
        end
      end)
    end
  end
end

--[[ Enable memory optimization by marking tensors to share. Note that the modules must have been initialized
-- by calling forward() and backward() before calling this function and after calling the MemoryOptimizer constructor.

Returns:
  1. `sharedSize` - shared tensor size
  2. `totSize` - total tensor size
]]
function MemoryOptimizer:optimize()
  local totSize = 0
  local sharedSize = 0
  for _, desc in pairs(self.modelDesc) do
    for i = 1, #desc do
      local net = desc[i].net
      local base = desc[i].base
      local mempool = {}

      -- Some modules are using output when performing updateGradInput so we cannot share these.
      local protectedOutput = { desc[i].input }
      net:apply(function(m)
        if contains(protectOutput, m) then
          table.insert(protectedOutput, m.output)
        end
        if contains(protectInput, m) then
          table.insert(protectedOutput, m.input)
        end
      end)

      local globalIdx = 1
      local idx = 1

      local gradInputMap = {}
      local outputMap = {}

      -- Go over the network to determine which tensors can be shared.
      net:apply(function(m)
        local giSize = getSize(m.gradInput, mempool)
        local oSize = getSize(m.output, mempool)
        totSize = totSize + giSize
        totSize = totSize + oSize
        if canShare(m.gradInput, net, desc[i].gradOutput) then
          sharedSize = sharedSize + giSize
          m.gradInputSharedIdx = idx
          gradInputMap[globalIdx] = idx
          idx = idx + 1
        end
        if canShare(m.output, net, protectedOutput) then
          sharedSize = sharedSize + oSize
          m.outputSharedIdx = idx
          outputMap[globalIdx] = idx
          idx = idx + 1
        end

        -- Remove the wrapper around updateOutput to catch the module input.
        m.updateOutput = nil
        m.input = nil

        globalIdx = globalIdx + 1
      end)

      globalIdx = 1

      -- Mark shareable tensors in the base network.
      base:apply(function (m)
        if gradInputMap[globalIdx] then
          m.gradInputSharedIdx = gradInputMap[globalIdx]
        end
        if outputMap[globalIdx] then
          m.outputSharedIdx = outputMap[globalIdx]
        end
        globalIdx = globalIdx + 1
      end)

      -- Restore function on network backward/forward interception input.
      net.backward = nil
      net.forward = nil
    end
  end
  return sharedSize, totSize
end

return MemoryOptimizer


================================================
FILE: opennmt/utils/Parallel.lua
================================================
--[[
  This file provides generic parallel class - allowing to run functions
  in different threads and on different GPU
]]--

local Parallel = {
  _pool = nil,
  count = 1,
  gradBuffer = torch.Tensor()
}

-- Synchronizes the current stream on dst device with src device. This is only
-- necessary if we are not on the default stream
local function waitForDevice(dst, src)
   local stream = cutorch.getStream()
   if stream ~= 0 then
      cutorch.streamWaitForMultiDevice(dst, stream, { [src] = {stream} })
   end
end

function Parallel.getCounter()
  return Parallel._tds.AtomicCounter()
end

function Parallel.gmutexId()
  return Parallel._gmutex:id()
end

function Parallel.init(opt)
  if onmt.utils.Cuda.activated then
    Parallel.count = onmt.utils.Cuda.gpuCount()
    Parallel.gradBuffer = onmt.utils.Cuda.convert(Parallel.gradBuffer)
    Parallel._tds = require('tds')

    if Parallel.count > 1 then
      local globalLogger = _G.logger
      local globalProfiler = _G.profiler
      local threads = require('threads')
      threads.Threads.serialization('threads.sharedserialize')
      Parallel._gmutex = threads.Mutex()
      Parallel._pool = threads.Threads(
        Parallel.count,
        function()
          require('cunn')
          require('nngraph')
          require('opennmt.init')
          _G.threads = require('threads')
        end,
        function(threadid)
          _G.logger = globalLogger
          _G.profiler = globalProfiler
          onmt.utils.Cuda.init(opt, threadid)
        end
      ) -- dedicate threads to GPUs
      Parallel._pool:specific(true)
    end

    if Parallel.count > 1 and not opt.no_nccl and not opt.async_parallel then
      -- check if we have nccl installed
      local ret
      ret, Parallel.usenccl = pcall(require, 'nccl')
      if not ret then
        _G.logger:warning("For improved efficiency with multiple GPUs, consider installing nccl")
        Parallel.usenccl = nil
      elseif os.getenv('CUDA_LAUNCH_BLOCKING') == '1' then
        _G.logger:warning("CUDA_LAUNCH_BLOCKING set - cannot use nccl")
        Parallel.usenccl = nil
      end
    end

  end
end

--[[ Launch function in parallel on different threads. ]]
function Parallel.launch(closure, endCallback)
  endCallback = endCallback or function() end
  for j = 1, Parallel.count do
    if Parallel._pool == nil then
      endCallback(closure(j))
    else
      Parallel._pool:addjob(j, function() return closure(j) end, endCallback)
    end
  end
  if Parallel._pool then
    Parallel._pool:synchronize()
  end
end

--[[ Accumulate the gradient parameters from the different parallel threads. ]]
function Parallel.accGradParams(gradParams, batches)
  if Parallel.count > 1 then
    for h = 1, #gradParams[1] do
      local inputs = { gradParams[1][h] }
      for j = 2, #batches do
        if not Parallel.usenccl then
          -- TODO - this is memory costly since we need to clone full parameters from one GPU to another
          -- to avoid out-of-memory, we can copy/add by batch

         -- Synchronize before and after copy to ensure that it doesn't overlap
         -- with this add or previous adds
          waitForDevice(onmt.utils.Cuda.gpuIds[j], onmt.utils.Cuda.gpuIds[1])
          local remoteGrads = onmt.utils.Tensor.reuseTensor(Parallel.gradBuffer, gradParams[j][h]:size())
          remoteGrads:copy(gradParams[j][h])
          waitForDevice(onmt.utils.Cuda.gpuIds[1], onmt.utils.Cuda.gpuIds[j])
          gradParams[1][h]:add(remoteGrads)
        else
          table.insert(inputs, gradParams[j][h])
        end
      end
      if Parallel.usenccl then
        Parallel.usenccl.reduce(inputs, nil, true)
      end
    end
  end
end

-- [[ In async mode, sync the parameters from all replica to master replica. ]]
function Parallel.updateAndSync(masterParams, replicaGradParams, replicaParams, gradBuffer, masterGPU, gmutexId)
  -- Add a mutex to avoid competition while accessing shared buffer and while updating parameters.
  local mutex = _G.threads.Mutex(gmutexId)
  mutex:lock()
  local device = cutorch.getDevice()
  cutorch.setDevice(masterGPU)
  for h = 1, #replicaGradParams do
    waitForDevice(device, masterGPU)
    local remoteGrads = onmt.utils.Tensor.reuseTensor(gradBuffer, replicaGradParams[h]:size())
    remoteGrads:copy(replicaGradParams[h])
    waitForDevice(masterGPU, device)
    masterParams[h]:add(remoteGrads)
  end
  cutorch.setDevice(device)
  for h = 1, #replicaGradParams do
    replicaParams[h]:copy(masterParams[h])
    waitForDevice(device, masterGPU)
  end
  mutex:unlock()
end

--[[ Sync parameters from main model to different parallel threads. ]]
function Parallel.syncParams(params)
  if Parallel.count > 1 then
    if not Parallel.usenccl then
      for j = 2, Parallel.count do
        for h = 1, #params[1] do
          params[j][h]:copy(params[1][h])
        end
        waitForDevice(onmt.utils.Cuda.gpuIds[j], onmt.utils.Cuda.gpuIds[1])
      end
    else
      for h = 1, #params[1] do
        local inputs = { params[1][h] }
        for j = 2, Parallel.count do
          table.insert(inputs, params[j][h])
        end
        Parallel.usenccl.bcast(inputs, true, 1)
      end
    end
  end
end

return Parallel


================================================
FILE: opennmt/utils/Profiler.lua
================================================
--[[ Profile is a class used for generating profiling of a training
--]]
local Profiler = torch.class('Profiler')

local options = {
  {
    '-profiler', false,
    [[Generate profiling logs.]]
  }
}

function Profiler.declareOpts(cmd)
  cmd:setCmdLineOptions(options)
end

--[[ Profiler object

To avoid concurrency problem for parallel processing, each thread should have its own Profiler.
Profiles can be embedded.

Parameters:
  * `doProfile` - enable profiling

Documentation:
  Profile is recording/aggregating time spent in sections. Sections have hierarchical structure.
  A section is opened with `P:start("name")` and closed with `P:close("name")`.
  Start and Stop command can be stacked: `P:stop("b"):start("a")` or combined: `P:start("a.b")`

Example:
    -- global profiler initialization
    globalProfiler = onmt.utils.Profiler.new(opt)

    -- thread-specific profiler
      _G.profiler = onmt.utils.Profiler.new(opt)

      _G.profiler:reset()

      _G.profiler:start("encoder")
      [...]
      _G.profiler:stop("encoder"):start("decoder")
      [...]
      _G.profiler:stop("decoder")

      local profile = _G.profiler.dump()

    -- adds up thread profile
    globalProfiler:add(profile)

    Logger:info(globalProfiler:log())

]]
function Profiler:__init(opt)
  if type(opt) == 'table' then opt=opt.profiler end
  if not opt then
    self.disable = true
  end
  self:reset()
end

-- Reset Profiler.
function Profiler:reset()
  self.profiles = {}
  self.timers = {}
  self.stack = {}
end

-- Start recording a section.
function Profiler:start(name)
  if self.disable then return self end
  -- Synchronize current operations on cuda.
  onmt.utils.Cuda.synchronize()
  -- if section is multiple, decompose
  local pos = name:find("%.")
  if pos then
    self:start(name:sub(1, pos-1))
    self:start(name:sub(pos+1))
  else
    if not self.timers[name] then self.timers[name] = {} end
    table.insert(self.timers[name], torch.Timer())
    table.insert(self.stack, name)
  end
  return self
end

-- Stop recording a section.
function Profiler:stop(name)
  if self.disable then return self end
  -- Synchronize current operations on cuda.
  onmt.utils.Cuda.synchronize()
  -- if section is multiple, decompose
  local pos = name:find("%.")
  if pos then
    self:stop(name:sub(pos+1))
    self:stop(name:sub(1, pos-1))
  else
    assert(self.stack[#self.stack] == name, 'Invalid profiler stop action: '..name)
    local path = table.concat(self.stack, ".")
    if not self.profiles[path] then self.profiles[path] = 0 end
    local timer = table.remove(self.timers[name])
    self.profiles[path] = self.profiles[path] + timer:time().real
    table.remove(self.stack)
  end
  return self
end

-- Dump profile.
function Profiler:dump()
  if self.disable then return end
  return self.profiles
end

-- Aggregage profiles with a previous dump in the current namespace
function Profiler:add(profile)
  if self.disable then return end
  local prefix = table.concat(self.stack, '.')
  if #prefix > 0 then prefix = prefix .. '.' end
  for name,v in pairs(profile) do
    if not self.profiles[prefix..name] then self.profiles[prefix..name] = 0 end
    self.profiles[prefix..name] = self.profiles[prefix..name] + v
  end
end

-- Returns text string with log structured by sub level.
-- e.g. train:{total:23, encoder_fwd:10, encoder_bwd:14}
function Profiler:log(prefix)
  prefix = prefix or ''
  local t = {}
  for name,v in pairs(self.profiles) do
    v = string.format("%g", v)
    if name:sub(1,#prefix) == prefix then
      local pos = #prefix + 1
      if not name:sub(pos):find("%.") then
        local subtree = self:log(name..'.')
        if #subtree > 0 then
          v='{total:'..v..','..subtree..'}'
        end
        table.insert(t, name:sub(pos)..':'..v)
      end
    end
  end
  return table.concat(t, ",")
end

function Profiler.addHook(module, name)
  module.fwdFunc = module.forward
  module.bwdFunc = module.backward
  function module:forward(...)
    _G.profiler:start(name..".fwd")
    local res, context = self:fwdFunc(...)
    _G.profiler:stop(name..".fwd")
    return res, context
  end
  function module:backward(...)
    _G.profiler:start(name..".bwd")
    local res, context, loss = self:bwdFunc(...)
    _G.profiler:stop(name..".bwd")
    return res, context, loss
  end
end

return Profiler


================================================
FILE: opennmt/utils/String.lua
================================================
--[[
  Split `str` on string or pattern separator `sep`.
]]
local function split(str, sep)
  local res = {}
  local index = 1

  while index <= str:len() do
    local sepStart, sepEnd = str:find(sep, index)

    local sub
    if not sepStart then
      sub = str:sub(index)
      table.insert(res, sub)
      index = str:len() + 1
    else
      sub = str:sub(index, sepStart - 1)
      table.insert(res, sub)
      index = sepEnd + 1
      if index > str:len() then
        table.insert(res, '')
      end
    end
  end

  return res
end

--[[ Remove whitespaces at the start and end of the string `s`. ]]
local function strip(s)
  return s:gsub('^%s+', ''):gsub('%s+$', '')
end

--[[ Remove initial hyphen(s). ]]
local function stripHyphens(str)
   return string.match(str, '%-*(.*)')
end

--[[ Right pad a strip with spaces. ]]
local function pad(str, sz)
   return str .. string.rep(' ', sz - #str)
end

--[[ Convenience function to test `s` for emptiness. ]]
local function isEmpty(s)
  return s == nil or s == ''
end

return {
  split = split,
  strip = strip,
  isEmpty = isEmpty,
  pad = pad,
  stripHyphens = stripHyphens
}


================================================
FILE: opennmt/utils/Table.lua
================================================
-- tds is lazy loaded.
local tds

--[[ Return subset of table ]]
local function subrange(t, first, count)
  local sub = {}
  for i=first,first+count-1 do
    sub[#sub + 1] = t[i]
  end
  return sub
end

--[[ Append table `src` to `dst`. ]]
local function append(dst, src)
  for i = 1, #src do
    table.insert(dst, src[i])
  end
end

--[[ Merge dict `src` to `dst`. ]]
local function merge(dst, src)
  for k, v in pairs(src) do
    dst[k] = v
  end
end

local function empty (self)
  if next(self) == nil then
    return true
  else
    return false
  end
end

--[[ Reorder table `tab` based on the `index` array. ]]
local function reorder(tab, index, cdata)
  local newTab
  if cdata then
    if not tds then
      tds = require('tds')
    end
    newTab = tds.Vec()
    newTab:resize(#tab)
  else
    newTab = {}
  end

  for i = 1, #tab do
    newTab[i] = tab[index[i]]
  end

  return newTab
end

--[[ Check if value is part of list/table. ]]
local function hasValue(tab, value)
  for _, v in ipairs(tab) do
    if v == value then
      return true
    end
  end
  return false
end


return {
  subrange = subrange,
  reorder = reorder,
  append = append,
  merge = merge,
  hasValue = hasValue,
  empty = empty
}


================================================
FILE: opennmt/utils/Tensor.lua
================================================
--[[ Recursively call `func()` on all tensors within `out`. ]]
local function recursiveApply(out, func, ...)
  local res
  if torch.type(out) == 'table' then
    res = {}
    for k, v in pairs(out) do
      res[k] = recursiveApply(v, func, ...)
    end
    return res
  end
  if torch.isTensor(out) then
    res = func(out, ...)
  else
    res = out
  end
  return res
end

--[[ Recursively call `clone()` on all tensors within `out`. ]]
local function recursiveClone(out)
  return recursiveApply(out, function (h) return h:clone() end)
end

--[[ Recursively add `b` tensors into `a`'s. ]]
local function recursiveAdd(a, b)
  if torch.isTensor(a) then
    a:add(b)
  else
    for i = 1, #a do
      recursiveAdd(a[i], b[i])
    end
  end
  return a
end

local function recursiveSet(dst, src)
  if torch.isTensor(dst) then
    dst:set(src)
  else
    for k, _ in ipairs(dst) do
      recursiveSet(dst[k], src[k])
    end
  end
end

--[[ Clone any serializable Torch object. ]]
local function deepClone(obj)
  local mem = torch.MemoryFile("rw"):binary()
  mem:writeObject(obj)
  mem:seek(1)
  local clone = mem:readObject()
  mem:close()
  return clone
end

--[[
Reuse Tensor storage.

Parameters:

  * `t` - the tensor to be reused
  * `sizes` - a table or tensor of new sizes

Returns: a view on zero-tensor `t`.

--]]
local function reuseTensor(t, sizes)
  assert(t ~= nil, 'tensor must not be nil for it to be reused')

  if torch.type(sizes) == 'table' then
    sizes = torch.LongStorage(sizes)
  end

  return t:resize(sizes):zero()
end

--[[
Reuse all Tensors within the table with new sizes.

Parameters:

  * `tab` - the table of tensors
  * `sizes` - a table of new sizes

Returns: a table of tensors using the same storage as `tab`.

--]]
local function reuseTensorTable(tab, sizes)
  local newTab = {}

  for i = 1, #tab do
    table.insert(newTab, reuseTensor(tab[i], sizes))
  end

  return newTab
end

--[[
Initialize a table of tensors with the given sizes.

Parameters:

  * `size` - the number of clones to create
  * `proto` - tensor to be clone for each index
  * `sizes` - a table of new sizes

Returns: an initialized table of tensors.

--]]
local function initTensorTable(size, proto, sizes)
  local tab = {}

  local base = reuseTensor(proto, sizes)

  for _ = 1, size do
    table.insert(tab, base:clone())
  end

  return tab
end

--[[
Copy tensors from `src` reusing all tensors from `proto`.

Parameters:

  * `proto` - the table of tensors to be reused
  * `src` - the source table of tensors

Returns: a copy of `src`.

--]]
local function copyTensorTable(proto, src)
  local tab = reuseTensorTable(proto, src[1]:size())

  for i = 1, #tab do
    tab[i]:copy(src[i])
  end

  return tab
end

return {
  recursiveApply = recursiveApply,
  recursiveClone = recursiveClone,
  recursiveAdd = recursiveAdd,
  recursiveSet = recursiveSet,
  deepClone = deepClone,
  reuseTensor = reuseTensor,
  reuseTensorTable = reuseTensorTable,
  initTensorTable = initTensorTable,
  copyTensorTable = copyTensorTable
}


================================================
FILE: opennmt/utils/init.lua
================================================
local utils = {}

utils.Cuda = require('opennmt.utils.Cuda')
utils.Dict = require('opennmt.utils.Dict')
utils.FileReader = require('opennmt.utils.FileReader')
utils.Tensor = require('opennmt.utils.Tensor')
utils.Table = require('opennmt.utils.Table')
utils.String = require('opennmt.utils.String')
utils.Memory = require('opennmt.utils.Memory')
utils.MemoryOptimizer = require('opennmt.utils.MemoryOptimizer')
utils.Parallel = require('opennmt.utils.Parallel')
utils.Features = require('opennmt.utils.Features')
utils.Logger = require('opennmt.utils.Logger')
utils.Profiler = require('opennmt.utils.Profiler')
utils.ExtendedCmdLine = require('opennmt.utils.ExtendedCmdLine')
utils.CrayonLogger = require('opennmt.utils.CrayonLogger')

return utils


================================================
FILE: src/cnn.lua
================================================
function createCNNModel()
  local model = nn.Sequential()

  -- input shape: (batch_size, 1, imgH, imgW)
  model:add(nn.AddConstant(-128.0))
  model:add(nn.MulConstant(1.0 / 128))

  model:add(cudnn.SpatialConvolution(1, 64, 3, 3, 1, 1, 1, 1)) -- (batch_size, 64, imgH, imgW)
  model:add(cudnn.ReLU(true))

  model:add(cudnn.SpatialMaxPooling(2, 2, 2, 2)) -- (batch_size, 64, imgH/2, imgW/2)

  model:add(cudnn.SpatialConvolution(64, 128, 3, 3, 1, 1, 1, 1)) -- (batch_size, 128, imgH/2, imgW/2)
  model:add(cudnn.ReLU(true))

  model:add(cudnn.SpatialMaxPooling(2, 2, 2, 2)) -- (batch_size, 128, imgH/2/2, imgW/2/2)

  model:add(cudnn.SpatialConvolution(128, 256, 3, 3, 1, 1, 1, 1)) -- (batch_size, 256, imgH/2/2, imgW/2/2)
  model:add(nn.SpatialBatchNormalization(256))
  model:add(cudnn.ReLU(true))

  model:add(cudnn.SpatialConvolution(256, 256, 3, 3, 1, 1, 1, 1)) -- (batch_size, 256, imgH/2/2, imgW/2/2)
  model:add(cudnn.ReLU(true))

  model:add(cudnn.SpatialMaxPooling(1, 2, 1, 2, 0, 0)) -- (batch_size, 256, imgH/2/2/2, imgW/2/2)

  model:add(cudnn.SpatialConvolution(256, 512, 3, 3, 1, 1, 1, 1)) -- (batch_size, 512, imgH/2/2/2, imgW/2/2)
  model:add(nn.SpatialBatchNormalization(512))
  model:add(cudnn.ReLU(true))

  model:add(cudnn.SpatialMaxPooling(2, 1, 2, 1, 0, 0)) -- (batch_size, 512, imgH/2/2/2, imgW/2/2/2)
  model:add(cudnn.SpatialConvolution(512, 512, 3, 3, 1, 1, 1, 1)) -- (batch_size, 512, imgH/2/2/2, imgW/2/2/2)
  model:add(nn.SpatialBatchNormalization(512))
  model:add(cudnn.ReLU(true))

  -- (batch_size, 512, H, W)
  model:add(nn.Transpose({2, 3}, {3,4})) -- (batch_size, H, W, 512)
  model:add(nn.SplitTable(1, 3)) -- #H list of (batch_size, W, 512)

  return model
end


================================================
FILE: src/data.lua
================================================
 --[[ Load data. Adapted from https://github.com/da03/Attention-OCR/blob/master/src/data_util/data_gen.py.
 --  ARGS:
     * `imageDir`       : string. The base directory of the image path in dataPath.
     * `dataPath`       : string. The file containing data file names and label indexes. Format per line: imagePath[Space]labelIndex. Note that the imagePath is the relative path to imageDir. LabelIndex counts from 0.
     * `labelPath`       : string. The file containing labels. Each line corresponds to a label.
     * `maxImageHeight` : int. Maximum image height. Default: unlimited.
     * `maxImageWidth`  : int. Maximum image width. Default: unlimited.
     * `maxNumTokens`   : int. Maximum number of output tokens. Default: unlimited.
 --]]
require 'image'
require 'paths'
require 'class'
local tds = require('tds')

local DataLoader = torch.class('DataLoader')

function DataLoader:__init(imageDir, dataPath, labelPath, maxImageHeight, maxImageWidth, maxNumTokens)
  self.imageDir = imageDir
  self.labelPath = labelPath
  self.maxImageHeight = maxImageHeight or math.huge
  self.maxImageWidth = maxImageWidth or math.huge
  self.maxNumTokens = maxNumTokens or math.huge

  local file, err = io.open(dataPath, "r")
  if err then
    file, err = io.open(paths.concat(imageDir, dataPath), "r")
    if err then
      _G.logger:error('Data file %s not found', dataPath)
      os.exit()
    end
  end
  self.lines = tds.Hash()
  local idx = 0
  for line in file:lines() do
    idx = idx + 1
    if idx % 1000000 == 0 then
      _G.logger:info ('%d lines read', idx)
    end
    local imagePath, label = unpack(line:split('[%s]+'))
    self.lines[idx] = tds.Vec({imagePath, label})
  end
  self.perm = torch.range(1, #self.lines)
  self.cursor = 1
  self.buffer = {}
  collectgarbage()
end

function DataLoader:shuffle()
  local perm = torch.randperm(#self.lines)
  self.lines = onmt.utils.Table.reorder(self.lines, perm)
  self.perm = self.perm:index(1, perm:type('torch.LongTensor'))
end

function DataLoader:load(dataPath)
  assert(paths.filep(dataPath), string.format('Data file %s does not exist!', dataPath))
  local s
  self.perm, self.cursor, self.buffer, self.epoch, s = table.unpack(torch.load(dataPath))
  self.lines = onmt.utils.Table.reorder(self.lines, self.perm)
  torch.setRNGState(s)
end

function DataLoader:save(dataPath)
  local s = torch.getRNGState()
  torch.save(dataPath, {self.perm, self.cursor, self.buffer, self.epoch, s})
end

function DataLoader:size()
  return #self.lines
end

function DataLoader:nextBatch(batchSize)
  while true do -- accumulate samples of the same size in self.buffer until batchSize or the last data example is reached
    if self.cursor > #self.lines then
      break
    end
    local imagePath = self.lines[self.cursor][1]
    local status, imageData = pcall(image.load, paths.concat(self.imageDir, imagePath))
    if not status then
      self.cursor = self.cursor + 1
      _G.logger:warning('Fails to read image file %s', imagePath)
    else
      local labelIndex = self.lines[self.cursor][2]
      local tokenIds = labelIndexToTokenIds(labelIndex, self.labelPath)
      self.cursor = self.cursor + 1
      imageData = 255.0*image.rgb2y(imageData) -- convert to greyscale
      local imageHeight, imageWidth = imageData:size(2), imageData:size(3)
      if #tokenIds > self.maxNumTokens + 2 then -- truncate target sequence
        _G.logger:warning('Image %s\'s target sequence is too long, will be truncated. Consider using a larger maxNumTokens', imagePath)
        local tokenIdsTemp = {}
        for i = 1, self.maxNumTokens + 2 do
          tokenIdsTemp[i] = tokenIds[i]
        end
        tokenIds = tokenIdsTemp
      end
      if imageHeight <= self.maxImageHeight and imageWidth <= self.maxImageWidth then
        if self.buffer[imageWidth] == nil then
          self.buffer[imageWidth] = {}
        end
        if self.buffer[imageWidth][imageHeight] == nil then
          self.buffer[imageWidth][imageHeight] = {}
        end
        table.insert(self.buffer[imageWidth][imageHeight], {imageData, tokenIds, imagePath})
        if #self.buffer[imageWidth][imageHeight] >= batchSize then
          local images = torch.Tensor(batchSize, 1, imageHeight, imageWidth)
          local maxTargetLength = -math.huge
          local imagePaths = {}
          local offset = #self.buffer[imageWidth][imageHeight] - batchSize
          for i = 1, batchSize do
            imagePaths[i] = self.buffer[imageWidth][imageHeight][i+offset][3]
            images[i]:copy(self.buffer[imageWidth][imageHeight][i+offset][1])
            maxTargetLength = math.max(maxTargetLength, #self.buffer[imageWidth][imageHeight][i+offset][2])
          end
          -- targetInput: used as input to decoder. SOS, tokenId1, tokenId2, ..., tokenIdN
          local targetInput = torch.IntTensor(batchSize, maxTargetLength-1):fill(onmt.Constants.PAD)
          -- targetOutput: used for comparing against decoder's output. tokenId1, tokenId2, ..., tokenIdN, EOS
          local targetOutput = torch.IntTensor(batchSize, maxTargetLength-1):fill(onmt.Constants.PAD)
          local numNonzeros = 0
          for i = 1, batchSize do
            numNonzeros = numNonzeros + #self.buffer[imageWidth][imageHeight][i+offset][2] - 1
            for j = 1, #self.buffer[imageWidth][imageHeight][i+offset][2]-1 do
              targetInput[i][j] = self.buffer[imageWidth][imageHeight][i+offset][2][j]
              targetOutput[i][j] = self.buffer[imageWidth][imageHeight][i+offset][2][j+1]
            end
          end
          if offset == 0 then
            self.buffer[imageWidth][imageHeight] = nil
          else
            for i = 1, batchSize do
              self.buffer[imageWidth][imageHeight][i+offset] = nil
            end
          end

          do return {images, targetInput, targetOutput, numNonzeros, imagePaths} end
        end
      else --  not (imageHeight <= self.maxImageHeight and imageWidth <= self.maxImageWidth)
        _G.logger:warning('Image %s is too large, will be ignored. Consider using a larger maxImageWidth or maxImageHeight'%imagePath)
      end
    end
  end -- cannot accumulate batchSize samples

  -- find if there are any samples left in order to finish the current epoch
  if next(self.buffer) == nil then
    self.cursor = 1
    collectgarbage()
    return nil
  end
  local imageWidth = next(self.buffer)
  while next(self.buffer[imageWidth]) == nil do
    if next(self.buffer, imageWidth) == nil then
      self.cursor = 1
      collectgarbage()
      return nil
    end
    imageWidth = next(self.buffer, imageWidth)
  end
  local imageHeight = next(self.buffer[imageWidth], nil)
  local actualBatchSize = math.min(batchSize, #self.buffer[imageWidth][imageHeight])
  local offset = math.max(0, #self.buffer[imageWidth][imageHeight]-batchSize)
  local images = torch.Tensor(actualBatchSize, 1, imageHeight, imageWidth)
  local maxTargetLength = -math.huge
  local imagePaths = {}
  for i = 1, actualBatchSize do
    imagePaths[i] = self.buffer[imageWidth][imageHeight][i+offset][3]
    images[i]:copy(self.buffer[imageWidth][imageHeight][i+offset][1])
    maxTargetLength = math.max(maxTargetLength, #self.buffer[imageWidth][imageHeight][i+offset][2])
  end
  local targetInput = torch.IntTensor(actualBatchSize, maxTargetLength-1):fill(onmt.Constants.PAD)
  local targetOutput = torch.IntTensor(actualBatchSize, maxTargetLength-1):fill(onmt.Constants.PAD)
  local numNonzeros = 0
  for i = 1, actualBatchSize do
    numNonzeros = numNonzeros + #self.buffer[imageWidth][imageHeight][i+offset][2] - 1
    for j = 1, #self.buffer[imageWidth][imageHeight][i+offset][2]-1 do
      targetInput[i][j] = self.buffer[imageWidth][imageHeight][i+offset][2][j]
      targetOutput[i][j] = self.buffer[imageWidth][imageHeight][i+offset][2][j+1]
    end
  end
  if offset == 0 then
    self.buffer[imageWidth][imageHeight] = nil
  else
    for i = 1, actualBatchSize do
      self.buffer[imageWidth][imageHeight][i+offset] = nil
    end
  end
  return {images, targetInput, targetOutput, numNonzeros, imagePaths}
end

-- convert labelIndex to a list of token ids
function labelIndexToTokenIds(labelIndex, labelPath)
  assert (_G.idToVocab, '_G.idToVocab must be ready before calling labelIndexToTokenIds')
  if _G.vocabToId == nil then
    _G.vocabToId = tds.Hash()
    for i = 1, #_G.idToVocab do
      _G.vocabToId[_G.idToVocab[i]] = i+4
    end
  end
  if labelLines == nil then
    labelLines = tds.Hash()
    local labelFile, err = io.open(labelPath, "r")
    if not err then
      for line in labelFile:lines() do
        local tokenList = (onmt.utils.String.strip(line)):split('[%s]+')
        labelLines[#labelLines+1] = tds.Vec(tokenList)
      end
    end
  end
  local tokenIds = tds.Hash()
  tokenIds[1] = onmt.Constants.BOS
  if labelIndex == nil then
    tokenIds[#tokenIds+1] = onmt.Constants.EOS
    do return tokenIds end
  end
  local tokens = labelLines[tonumber(labelIndex)+1]
  if tokens == nil then
    tokenIds[#tokenIds+1] = onmt.Constants.EOS
    do return tokenIds end
  end
  for i = 1, #tokens do
    local token = tokens[i]
    if _G.vocabToId[token] then
      tokenIds[#tokenIds+1] = _G.vocabToId[token]
    else
      tokenIds[#tokenIds+1] = onmt.Constants.UNK -- unknown token
    end
  end
  tokenIds[#tokenIds+1] = onmt.Constants.EOS
  return tokenIds
end

-- convert targets tensor to a list of label strings
function targetsTensorToLabelStrings(targets)
  assert (targets:dim() == 2)
  local batchSize = targets:size(1)
  local targetLength = targets:size()[2]

  local labelStrings = {}
  for b = 1, batchSize do
    local tokenIds = {}
    for t = 1, targetLength do
      local tokenId = targets[b][t]
      if tokenId == onmt.Constants.EOS then -- ignore tokens after EOS
        break
      end
      table.insert(tokenIds, tokenId)
    end
    local labelString = tokenIdsToLabelString(tokenIds)
    table.insert(labelStrings, labelString)
  end
  return labelStrings
end

-- evaluate the edit distance error rate of the predictions
function evalEditDistanceRate(goldLabelStrings, predLabelStrings)
  assert(#goldLabelStrings == #predLabelStrings)

  local totalEditDistanceRate = 0.0
  for b = 1, #goldLabelStrings do
    local editDistance = string.levenshtein(goldLabelStrings[b], predLabelStrings[b])
    totalEditDistanceRate = totalEditDistanceRate + editDistance / (string.len(goldLabelStrings[b]) + string.len(predLabelStrings[b]))
  end
  return totalEditDistanceRate
end

-- convert a list of token ids to label string
function tokenIdsToLabelString(tokenIds)
  local labelString = tds.Vec()
  for i = 1, #tokenIds do
    local tokenId = tokenIds[i]
    if tokenId == onmt.Constants.PAD or tokenId == onmt.Constants.BOS or tokenId == onmt.Constants.EOS then
      break
    end
    local token = _G.idToVocab[tokenId-4]
    if tokenId == onmt.Constants.UNK then
      token = 'UNK'
    end
    assert (token, 'Make sure your target vocab size is correct!')
    for c in token:gmatch"." do
      labelString:insert(c)
    end
    labelString:insert(' ')
  end
  labelString = labelString:concat()
  return labelString
end


-- https://gist.github.com/Badgerati/3261142
-- Returns the Levenshtein distance between the two given strings
function string.levenshtein(str1, str2)
  local len1 = string.len(str1)
  local len2 = string.len(str2)
  local matrix = {}
  local cost

  -- quick cut-offs to save time
  if (len1 == 0) then
    return len2
  elseif (len2 == 0) then
    return len1
  elseif (str1 == str2) then
    return 0
  end

  -- initialise the base matrix values
  for i = 0, len1, 1 do
    matrix[i] = {}
    matrix[i][0] = i
  end
  for j = 0, len2, 1 do
    matrix[0][j] = j
  end

  -- actual Levenshtein algorithm
  for i = 1, len1, 1 do
    for j = 1, len2, 1 do
      if (str1:byte(i) == str2:byte(j)) then
        cost = 0
      else
        cost = 1
      end
      matrix[i][j] = math.min(matrix[i-1][j] + 1, matrix[i][j-1] + 1, matrix[i-1][j-1] + cost)
    end
  end

  -- return the last value - this is the Levenshtein distance
  return matrix[len1][len2]
end


================================================
FILE: src/model.lua
================================================
 --[[ Model, adapted from https://github.com/harvardnlp/seq2seq-attn/blob/master/train.lua
--]]
require 'nn'
require 'cudnn'
require 'optim'
require 'paths'
require 'src.cnn'

local model = torch.class('WYGIWYS')

-- constructor
function model:__init(optim)
  self.optim = optim
end

-- in test phase, open a file for predictions
function model:setOutputDirectory(outputDir)
  if not paths.dirp(outputDir) then
    paths.mkdir(outputDir)
  end
  local outputPath = paths.concat(outputDir, 'results.txt')
  local outputFile, err = io.open(outputPath, "w")
  if err then
    _G.logger:error('Output file %s cannot be created', outputPath)
    os.exit(1)
  end
  self.outputFile = outputFile
  self.outputPath = outputPath
end

-- load model from model_path
function model:load(modelPath, config)
  assert(paths.filep(modelPath), string.format('Model file %s does not exist!', modelPath))

  local checkpoint = torch.load(modelPath)
  local loadedModel, modelConfig = checkpoint[1], checkpoint[2]
  self.models = {}
  self.models.cnn = loadedModel[1]:double()
  self.models.encoder = onmt.BiEncoder.load(loadedModel[2])
  self.models.decoder = onmt.Decoder.load(loadedModel[3])
  self.models.posEmbeddingFw, self.models.posEmbeddingBw = loadedModel[4]:double(), loadedModel[5]:double()
  self.numSteps = checkpoint[3]
  self.numSamples = checkpoint[4]
  self.optim = checkpoint[5]
  _G.idToVocab = checkpoint[6] -- _G.idToVocab is global

  -- Load model structure parameters
  self.config = {}
  self.config.phase = config.phase
  self.config.cnnFeatureSize = modelConfig.cnnFeatureSize
  self.config.encoderNumHidden = modelConfig.encoderNumHidden
  self.config.encoderNumLayers = modelConfig.encoderNumLayers
  self.config.decoderNumHidden = self.config.encoderNumHidden * 2 -- the decoder rnn size is the same as the output size of biEncoder
  self.config.decoderNumLayers = modelConfig.decoderNumLayers
  self.config.targetVocabSize = #_G.idToVocab + 4
  self.config.targetEmbeddingSize = modelConfig.targetEmbeddingSize
  self.config.inputFeed = modelConfig.inputFeed

  self.config.maxEncoderLengthWidth = config.maxEncoderLengthWidth or modelConfig.maxEncoderLengthWidth
  self.config.maxEncoderLengthHeight = config.maxEncoderLengthHeight or modelConfig.maxEncoderLengthHeight
  self.config.maxDecoderLength = config.maxDecoderLength or modelConfig.maxDecoderLength
  self.config.batchSize = config.batch_size or modelConfig.batchSize
  self.config.valBatchSize = config.val_batch_size or modelConfig.val_batchSize or self.config.batchSize

  self.config.maxImageWidth = config.max_image_width or modelConfig.max_image_width
  self.config.maxImageHeight = config.max_image_height or modelConfig.max_image_height

  -- If we want to allow higher images, since the trained positional embeddings are valid only up to modelConfig.maxEncoderLengthHeight, we use the largest available one to initialize the invalid embdddings
  if self.config.maxEncoderLengthHeight > modelConfig.maxEncoderLengthHeight then
  local posEmbeddingFw = nn.LookupTable(self.config.maxEncoderLengthHeight, self.config.encoderNumLayers * self.config.encoderNumHidden * 2)
  local posEmbeddingBw = nn.LookupTable(self.config.maxEncoderLengthHeight, self.config.encoderNumLayers * self.config.encoderNumHidden * 2)
  for i = 1, self.config.maxEncoderLengthHeight do
    local j = math.min(i, modelConfig.maxEncoderLengthHeight)
    posEmbeddingFw.weight[i] = self.models.posEmbeddingFw.weight[j]
    posEmbeddingBw.weight[i] = self.models.posEmbeddingBw.weight[j]
  end
  self.models.posEmbeddingFw, self.models.posEmbeddingBw = posEmbeddingFw, posEmbeddingBw
  end

  self.config.no_stress_test = config.no_stress_test

  -- build model
  self:_build()
end

-- create model with fresh parameters
function model:create(config)
  self.optim = onmt.train.Optim.new(config)
  -- set parameters
  self.config = {}
  self.config.phase = config.phase
  self.config.cnnFeatureSize = 512
  self.config.batchSize = config.batch_size
  self.config.valBatchSize = config.val_batch_size
  self.config.inputFeed = config.input_feed
  self.config.encoderNumHidden = config.encoder_num_hidden
  self.config.encoderNumLayers = config.encoder_num_layers
  self.config.decoderNumHidden = config.encoder_num_hidden * 2
  self.config.decoderNumLayers = config.decoder_num_layers
  self.config.targetEmbeddingSize = config.target_embedding_size
  self.config.targetVocabSize = config.targetVocabSize
  self.config.maxEncoderLengthWidth = config.maxEncoderLengthWidth
  self.config.maxEncoderLengthHeight = config.maxEncoderLengthHeight
  self.config.maxDecoderLength = config.maxDecoderLength
  self.config.maxImageWidth = config.max_image_width
  self.config.maxImageHeight = config.max_image_height

  -- Create model modules
  self.models = {}
  -- positional embeddings
  self.models.posEmbeddingFw = nn.LookupTable(self.config.maxEncoderLengthHeight, self.config.encoderNumLayers * self.config.encoderNumHidden * 2)
  self.models.posEmbeddingBw = nn.LookupTable(self.config.maxEncoderLengthHeight, self.config.encoderNumLayers * self.config.encoderNumHidden * 2)
  -- CNN model, input size: (batchSize, 1, height, width), output size: (batchSize, sequenceLength, cnnFeatureSize)
  self.models.cnn = createCNNModel()
  -- biLSTM encoder
  --local encoderRnn = onmt.LSTM.new(self.config.encoderNumLayers, self.config.cnnFeatureSize, self.config.encoderNumHidden, 0.0)
  --self.models.encoder = onmt.BiEncoder.new(config, nn.Identity())
  local encoderConfig = {layers = self.config.encoderNumLayers, rnn_size = self.config.encoderNumHidden*2,
    brnn=true, brnn_merge = 'concat', dropout = 0, rnn_type = 'LSTM'}
  local encoderInputNetwork = nn.Identity()
  encoderInputNetwork.inputSize = self.config.cnnFeatureSize
  self.models.encoder = onmt.Factory.buildEncoder(encoderConfig, encoderInputNetwork, false)

  -- decoder
  local inputSize = self.config.targetEmbeddingSize
  if self.config.inputFeed then
    inputSize = inputSize + self.config.decoderNumHidden
  end
  local generator = onmt.Generator.new(self.config.decoderNumHidden, self.config.targetVocabSize)
  generator:cuda()
  local inputNetwork = onmt.WordEmbedding.new(self.config.targetVocabSize, self.config.targetEmbeddingSize)
  inputNetwork.inputSize = self.config.targetEmbeddingSize
  local attentionModel = onmt.GlobalAttention({global_attention='general'}, self.config.decoderNumHidden)
  local inputFeed = 0
  if self.config.inputFeed then
    inputFeed = 1
  end
  local decoderConfig = {input_feed = inputFeed, layers = self.config.decoderNumLayers, rnn_size = self.config.decoderNumHidden,
    residual = false, dropout = 0, dropout_input = false}
  self.models.decoder = onmt.Decoder.new(decoderConfig, inputNetwork, generator, attentionModel)

  self.numSteps = 0
  self.numSamples = 0
  self.config.no_stress_test = config.no_stress_test
  self._init = true

  self:_build()
end

-- build
function model:_build()

  -- log options
  for k, v in pairs(self.config) do
    _G.logger:info('%s: %s', k, v)
  end

  -- create criterion
  self.criterion = onmt.ParallelClassNLLCriterion({self.config.targetVocabSize})
  onmt.utils.Cuda.convert(self.criterion)

  -- convert to cuda
  self.layers = {self.models.cnn, self.models.encoder, self.models.decoder, self.models.posEmbeddingFw, self.models.posEmbeddingBw}
  self.models.cnn:cuda()
  for i = 2, #self.layers do
    onmt.utils.Cuda.convert(self.layers[i])
  end
  onmt.utils.Cuda.convert(self.criterion)

  self.contextProto = onmt.utils.Cuda.convert(torch.zeros(math.max(self.config.batchSize,self.config.valBatchSize or 0), self.config.maxEncoderLengthWidth * self.config.maxEncoderLengthHeight, 2 * self.config.encoderNumHidden))
  self.cnnGradProto = onmt.utils.Cuda.convert(torch.zeros(self.config.maxEncoderLengthHeight, self.config.batchSize, self.config.maxEncoderLengthWidth, self.config.cnnFeatureSize))

  local numParams = 0
  self.params, self.gradParams = {}, {}
  for i = 1, #self.layers do
    local p, gp = self.layers[i]:getParameters()
    if self._init then
      p:uniform(-0.05,0.05)
    end
    numParams = numParams + p:size(1)
    self.params[i] = p
    self.gradParams[i] = gp
  end
  _G.logger:info('Number of parameters: %d', numParams)

  if self.config.phase == 'train' then
    self:_optimizeMemory()
    if not self.config.no_stress_test then
      self:_stressTest()
    end
  end
  collectgarbage()
end

function model:_stressTest()
  _G.logger:info('Stress Test starts')
  local s = torch.getRNGState()
  local images = torch.rand(self.config.batchSize, 1, self.config.maxImageHeight, self.config.maxImageWidth)
  local targetInput = torch.IntTensor(self.config.batchSize, self.config.maxDecoderLength):fill(onmt.Constants.PAD)
  local targetOutput = torch.IntTensor(self.config.batchSize, self.config.maxDecoderLength):fill(onmt.Constants.PAD)
  local numNonzeros = 1
  local inputBatch = {images, targetInput, targetOutput, numNonzeros, {}}
  self:step(inputBatch, false, 1)
  images = torch.rand(self.config.valBatchSize, 1, self.config.maxImageHeight, self.config.maxImageWidth)
  targetInput = torch.IntTensor(self.config.valBatchSize, self.config.maxDecoderLength):fill(onmt.Constants.PAD)
  targetOutput = torch.IntTensor(self.config.valBatchSize, self.config.maxDecoderLength):fill(onmt.Constants.PAD)
  inputBatch = {images, targetInput, targetOutput, numNonzeros, {}}
  self:step(inputBatch, true, 1, true)
  torch.setRNGState(s)
  _G.logger:info('Stress Test ends')
end

-- one step forward (and optionally backward)
function model:step(inputBatch, isForwardOnly, beamSize, mute)
  mute = mute or false
  beamSize = beamSize or 1 -- default greedy decoding
  assert (beamSize <= self.config.targetVocabSize)
  --local images = onmt.utils.Cuda.convert(inputBatch[1])
  local images = inputBatch[1]:cuda()
  local targetInput = onmt.utils.Cuda.convert(inputBatch[2])
  local targetOutput = onmt.utils.Cuda.convert(inputBatch[3])
  local numNonzeros = inputBatch[4]
  local imagePaths = inputBatch[5]

  local batchSize = images:size(1)
  local targetLength = targetInput:size(2)

  assert(targetLength <= self.config.maxDecoderLength, string.format('maxDecoderLength (%d) < targetLength (%d)!', self.config.maxDecoderLength, targetLength))
  -- if isForwardOnly, then re-generate the targetInput with maxDecoderLength for fair evaluation
  if isForwardOnly then
    local targetInputTemp = onmt.utils.Cuda.convert(torch.IntTensor(batchSize, self.config.maxDecoderLength)):fill(onmt.Constants.PAD)
    targetInputTemp[{{}, {1,targetLength}}]:copy(targetInput)
    targetInput = targetInputTemp
    local targetOutputTemp = onmt.utils.Cuda.convert(torch.IntTensor(batchSize, self.config.maxDecoderLength)):fill(onmt.Constants.PAD)
    targetOutputTemp[{{}, {1,targetLength}}]:copy(targetOutput)
    targetOutput = targetOutputTemp
    targetLength = self.config.maxDecoderLength
  end

  -- set phase
  if not isForwardOnly then
    self.models.cnn:training()
    self.models.encoder:training()
    self.models.decoder:training()
    self.models.posEmbeddingFw:training()
    self.models.posEmbeddingBw:training()
  else
    self.models.cnn:evaluate()
    self.models.encoder:evaluate()
    self.models.decoder:evaluate()
    self.models.posEmbeddingFw:evaluate()
    self.models.posEmbeddingBw:evaluate()
  end

  -- given parameters, evaluate loss (and optionally calculate gradients)
  local feval = function()
    local targetIn = targetInput:transpose(1,2)
    local targetOut = targetOutput:transpose(1,2)
    local cnnOutputs = self.models.cnn:forward(images) -- list of (batchSize, featureMapWidth, cnnFeatureSize)
    for i = 1, #cnnOutputs do
      cnnOutputs[i] = onmt.utils.Cuda.convert(cnnOutputs[i])
    end
    local featureMapHeight = #cnnOutputs
    local featureMapWidth = cnnOutputs[1]:size(2)
    local context = self.contextProto[{{1, batchSize}, {1, featureMapHeight * featureMapWidth}}]
    local decoderBatch = Batch():setTargetInput(targetIn):setTargetOutput(targetOut)
    decoderBatch.sourceLength = context:size(2)
    decoderBatch.sourceSize = onmt.utils.Cuda.convert(torch.IntTensor(batchSize)):fill(context:size(2))
    for i = 1, featureMapHeight do
      local pos = onmt.utils.Cuda.convert(torch.zeros(batchSize)):fill(i)
      local posEmbeddingFw  = self.models.posEmbeddingFw:forward(pos):view(batchSize, -1) -- (1, batchSize, encoderNumLayers*2*encoderNumHidden)
      local posEmbeddingBw  = self.models.posEmbeddingBw:forward(pos):view(batchSize, -1)  -- (1, batchSize, encoderNumLayers*2*encoderNumHidden)
      local cnnOutput = cnnOutputs[i] -- (batchSize, featureMapWidth, cnnFeatureSize)
      local source = cnnOutput:transpose(1, 2) -- (featureMapWidth, batchSize, cnnFeatureSize)
      local encoderBatch = Batch():setSourceInput(source)
      local encoderStatesFw = onmt.utils.Tensor.initTensorTable(self.config.encoderNumLayers*2,
                                                         onmt.utils.Cuda.convert(torch.Tensor()),
                                                         { batchSize,  self.config.encoderNumHidden})
      for k = 1, #encoderStatesFw do
        encoderStatesFw[k]:copy(posEmbeddingFw[{{}, {(k-1)*self.config.encoderNumHidden+1, k*self.config.encoderNumHidden}}])
      end
      local encoderStatesBw = onmt.utils.Tensor.initTensorTable(self.config.encoderNumLayers*2,
                                                         onmt.utils.Cuda.convert(torch.Tensor()),
                                                         { batchSize,  self.config.encoderNumHidden})
      for k = 1, #encoderStatesBw do
        encoderStatesBw[k]:copy(posEmbeddingBw[{{}, {(k-1)*self.config.encoderNumHidden+1, k*self.config.encoderNumHidden}}])
      end
      local _, rowContext = self.models.encoder:forward(encoderBatch, encoderStatesFw, encoderStatesBw)
      for t = 1, featureMapWidth do
        local index = (i - 1) * featureMapWidth + t
        context[{{}, index, {}}]:copy(rowContext[{{}, t, {}}])
      end
    end

    -- evaluate loss (and optionally do backward)
    local loss, numCorrect
    numCorrect = 0
    if isForwardOnly then
      if self.outputFile then
        -- Specify how to go one step forward.
        local advancer = onmt.translate.DecoderAdvancer.new(self.models.decoder, decoderBatch, context, self.config.maxDecoderLength)
        
        -- Conduct beam search.
        local beamSearcher = onmt.translate.BeamSearcher.new(advancer)
        local results = beamSearcher:search(beamSize, 1)
        local predTarget = onmt.utils.Cuda.convert(torch.zeros(batchSize, targetLength)):fill(onmt.Constants.PAD)
        for b = 1, batchSize do
          local tokens = results[b][1].tokens
          for t = 1, math.min(#tokens, targetLength) do
            predTarget[b][t] = tokens[t]
          end
        end
        local predLabels = targetsTensorToLabelStrings(predTarget)
        local goldLabels = targetsTensorToLabelStrings(targetOutput)
        local editDistanceRate = evalEditDistanceRate(goldLabels, predLabels)
        numCorrect = batchSize - editDistanceRate
        if not mute then
          for i = 1, #imagePaths do
            _G.logger:info('%s\t%s\n', imagePaths[i], predLabels[i])
            self.outputFile:write(string.format('%s\t%s\n', imagePaths[i], predLabels[i]))
          end
          self.outputFile:flush()
        end
      end
      -- get loss
      self.models.decoder:maskPadding()
      loss = self.models.decoder:computeLoss(decoderBatch, nil, context, self.criterion) / batchSize
    else -- isForwardOnly == false
      local decoderOutputs = self.models.decoder:forward(decoderBatch, nil, context)
      local _, gradContext, totalLoss = self.models.decoder:backward(decoderBatch, decoderOutputs, self.criterion)
      loss = totalLoss / batchSize
      gradContext = gradContext:contiguous():view(batchSize, featureMapHeight, featureMapWidth, -1) -- (batchSize, featureMapHeight, featureMapWidth, cnnFeatureSize)
      local cnnGrad = self.cnnGradProto[{ {1, featureMapHeight}, {1, batchSize}, {1, featureMapWidth}, {} }]
      for i = 1, featureMapHeight do
        local cnnOutput = cnnOutputs[i]
        local source = cnnOutput:transpose(1,2)
        local pos = onmt.utils.Cuda.convert(torch.zeros(batchSize)):fill(i)
        local posEmbeddingFw = self.models.posEmbeddingFw:forward(pos):view(batchSize, -1)
        local posEmbeddingBw = self.models.posEmbeddingBw:forward(pos):view(batchSize, -1)
        local encoderStatesFw = onmt.utils.Tensor.initTensorTable(self.config.encoderNumLayers*2,
                                                           onmt.utils.Cuda.convert(torch.Tensor()),
                                                           { batchSize,  self.config.encoderNumHidden})
        for k = 1, #encoderStatesFw do
          encoderStatesFw[k]:copy(posEmbeddingFw[{{}, {(k-1)*self.config.encoderNumHidden+1, k*self.config.encoderNumHidden}}])
        end
        local encoderStatesBw = onmt.utils.Tensor.initTensorTable(self.config.encoderNumLayers*2,
                                                           onmt.utils.Cuda.convert(torch.Tensor()),
                                                           { batchSize,  self.config.encoderNumHidden})
        for k = 1, #encoderStatesBw do
          encoderStatesBw[k]:copy(posEmbeddingBw[{{}, {(k-1)*self.config.encoderNumHidden+1, k*self.config.encoderNumHidden}}])
        end
        local encoderBatch = Batch():setSourceInput(source)
        self.models.encoder:forward(encoderBatch, encoderStatesFw, encoderStatesBw)
        local rowContextGrad, posEmbeddingGrad = self.models.encoder:backward(encoderBatch, nil, gradContext:select(2,i))
        for t = 1, featureMapWidth do
          cnnGrad[{i, {}, t, {}}]:copy(rowContextGrad[t])
        end
        local posEmbeddingGradFw = onmt.utils.Cuda.convert(torch.zeros(batchSize, self.config.encoderNumLayers*2*self.config.encoderNumHidden))
        for k = 1, 2*self.config.encoderNumLayers do
          posEmbeddingGradFw[{{}, {(k-1)*self.config.encoderNumHidden+1, k*self.config.encoderNumHidden}}]:copy(posEmbeddingGrad[k])
        end
        self.models.posEmbeddingFw:backward(pos, posEmbeddingGradFw)
        for k = 1, 2*self.config.encoderNumLayers do
          posEmbeddingGradFw[{{}, {(k-1)*self.config.encoderNumHidden+1, k*self.config.encoderNumHidden}}]:copy(posEmbeddingGrad[k+2*self.config.encoderNumLayers])
        end
        
        self.models.posEmbeddingBw:backward(pos, posEmbeddingGradFw)
      end
      -- cnn
      cnnGrad = cnnGrad:split(1, 1)
      for i = 1, #cnnGrad do
        cnnGrad[i] = cnnGrad[i]:contiguous():view(batchSize, featureMapWidth, -1):type('torch.CudaTensor')
      end
      self.models.cnn:backward(images, cnnGrad)
      collectgarbage()
    end
    return loss, self.gradParams, {numNonzeros, numCorrect}
  end
  if not isForwardOnly then
    -- optimizer
    self.optim:zeroGrad(self.gradParams)
    local loss, _, stats = feval(self.params)
    local flagNan = false
    for i = 1, #self.gradParams do
      if self.gradParams[i]:ne(self.gradParams[i]):any() then
        flagNan = true
        _G.logger:warning('nans detected in gradients!')
      end
    end
    if not flagNan then
      self.optim:prepareGrad(self.gradParams)
      self.optim:updateParams(self.params, self.gradParams)
    end

    return loss * batchSize, stats
  else
    local loss, _, stats = feval(self.params)
    return loss * batchSize, stats
  end
end

-- Optimize Memory Usage by sharing output and gradInput among clones
function model:_optimizeMemory()
  self.models.encoder:training()
  self.models.decoder:training()
  _G.logger:info('Preparing memory optimization...')
  local memoryOptimizer = onmt.utils.MemoryOptimizer.new({self.models.encoder, self.models.decoder})

  -- Initialize all intermediate tensors with a first batch.
  local source = onmt.utils.Cuda.convert(torch.zeros(1, 1, self.config.cnnFeatureSize))
  local targetIn = onmt.utils.Cuda.convert(torch.ones(1, 1))
  local targetOut = targetIn:clone()
  local batch = Batch():setSourceInput(source):setTargetInput(targetIn):setTargetOutput(targetOut)
  self.models.encoder:forward(batch)
  local context = onmt.utils.Cuda.convert(torch.zeros(1, 1, 2 * self.config.encoderNumHidden))
  local decOutputs = self.models.decoder:forward(batch, nil, context)
  decOutputs = onmt.utils.Tensor.recursiveClone(decOutputs)
  local _, gradContext = self.models.decoder:backward(batch, decOutputs, self.criterion)
  self.models.encoder:backward(batch, nil, gradContext)

  local sharedSize, totSize = memoryOptimizer:optimize()
  _G.logger:info(' * sharing %d%% of output/gradInput tensors memory between clones', (sharedSize / totSize)*100)
end

-- Save model to model_path
function model:save(modelPath)
  for i = 1, #self.layers do
    self.layers[i]:clearState()
  end
  torch.save(modelPath, {{self.models.cnn, self.models.encoder:serialize(), self.models.decoder:serialize(), self.models.posEmbeddingFw, self.models.posEmbeddingBw}, self.config, self.numSteps, self.numSamples, self.optim, _G.idToVocab})
end

-- destructor
function model:shutDown()
  if self.outputFile then
    self.outputFile:close()
    _G.logger:info('Results saved to %s.', self.outputPath)
  end
end


================================================
FILE: src/train.lua
================================================
 --[[ Training, adapted from https://github.com/harvardnlp/seq2seq-attn/blob/master/train.lua
--]]
require 'nn'
require 'nngraph'
require 'cunn'
require 'cutorch'
require 'cudnn'
require 'paths'
local status = pcall(function() require('opennmt.init') end)
if not status then
  print('OpenNMT not found. Please enter the path to OpenNMT: ')
  local onmtPath = io.read()
  package.path = package.path .. ';' .. paths.concat(onmtPath, '?.lua')
  status = pcall(function() require('opennmt.init') end)
  if not status then
    print ('Error: opennmt not found in the specified path!')
    os.exit(1)
  end
end
tds = require 'tds'

require 'src.model'
require 'src.data'

local cmd = onmt.utils.ExtendedCmdLine.new('src/train.lua')

-- Input and Output
cmd:text('')
cmd:text('**Control**')
cmd:text('')
cmd:option('-phase', 'test', [[train or test]])
cmd:option('-load_model', false, [[Load model from model_dir or not]])
cmd:option('-no_stress_test', false, [[]])
cmd:option('-gpu_id', 1, [[Which gpu to use]])

cmd:text('')
cmd:text('**Input and Output**')
cmd:text('')
cmd:option('-image_dir', '', [[The base directory of the image path in data-path.]])
cmd:option('-data_path', '', [[The file containing data file names and label indexes. Format per line: image_path label_index. Note that label_index count from 0.]])
cmd:option('-label_path', '', [[The file containing tokenized labels. Each line corresponds to a label.]])
cmd:option('-val_data_path', '', [[The path containing validate data file names and labels. Format per line: image_path characters]])
cmd:option('-vocab_file', '', [[Vocabulary file. A token per line.]])
cmd:option('-model_dir', 'model', [[The directory for saving and loading model parameters (structure is not stored)]])
cmd:option('-output_dir', 'results', [[The path to put results]])

-- Logging
cmd:text('')
cmd:text('**Display**')
cmd:text('')
cmd:option('-steps_per_checkpoint', 100, [[Checkpointing (print perplexity, save model) per how many steps]])
cmd:option('-log_path', 'log.txt', [[The path to put log]])

-- Optimization
cmd:text('')
cmd:text('**Optimization**')
cmd:text('')
cmd:option('-num_epochs', 15, [[The number of whole data passes]])
cmd:option('-batch_size', 1, [[Batch size]])
cmd:option('-val_batch_size', 10, [[Batch size]])

-- Network
cmd:text('')
cmd:text('**Network**')
cmd:text('')
cmd:option('-input_feed', false, [[Whether or not use LSTM attention decoder cell]])
cmd:option('-encoder_num_hidden', 256, [[Number of hidden units in encoder cell]])
cmd:option('-encoder_num_layers', 1, [[Number of hidden layers in encoder cell]])
cmd:option('-decoder_num_layers', 1, [[Number of hidden units in decoder cell]])
cmd:option('-target_embedding_size', 80, [[Embedding dimension for each target]])

-- Other
cmd:text('')
cmd:text('**Other**')
cmd:text('')
cmd:option('-beam_size', 1, [[Beam size for decoding]])
cmd:option('-max_num_tokens', 150, [[Maximum number of output tokens]]) -- when evaluate, this is the cut-off length.
cmd:option('-max_image_width', 500, [[Maximum image width]]) --800/2/2/2
cmd:option('-max_image_height', 160, [[Maximum image height]]) --80 / (2*2*2)
cmd:option('-seed', 920110, [[Load model from model_dir or not]])
cmd:option('-fp16', false, [[Use half-precision float on GPU]])

--onmt.BiEncoder.declareOpts(cmd)
onmt.utils.Profiler.declareOpts(cmd)
onmt.train.Optim.declareOpts(cmd)

local opt = cmd:parse(arg)
torch.manualSeed(opt.seed)
math.randomseed(opt.seed)
cutorch.manualSeed(opt.seed)

local function evaluateModel(model, valData, valBatchSize, beamSize, modelDir, epoch, epochEnd)
  local valLoss, valNumSamples, valNumNonzeros = 0, 0, 0
  -- Run 1 epoch on validation data
  while true do
    local valBatch = valData:nextBatch(valBatchSize)
    if valBatch == nil then
      break
    end
    local actualBatchSize = valBatch[1]:size(1)
    local stepLoss, stats = model:step(valBatch, true, beamSize)
    valLoss = valLoss + stepLoss
    valNumSamples = valNumSamples + actualBatchSize
    valNumNonzeros = valNumNonzeros + stats[1]
  end -- Run 1 epoch
  local ppl = math.exp(valLoss/valNumNonzeros)
  if not epochEnd then
    _G.logger:info('Epoch: %d. Step: %d. #Sample: %d. Val Perplexity: %f', epoch, model.numSteps, model.numSamples, ppl)
  else
    _G.logger:info('Epoch (square): %d. Step: %d. #Sample: %d. Val Perplexity: %f', epoch, model.numSteps, model.numSamples, ppl)
  end
  return ppl
end

local function saveModel(modelDir, model, trainData)
  _G.logger:info('Saving Model')
  local modelPath = paths.concat(modelDir, string.format('model_%d', model.numSteps))
  local modelPathTemp = paths.concat(modelDir, '.model.tmp')
  local modelPathLatest = paths.concat(modelDir, 'model_latest')
  model:save(modelPath)
  local dataPath = paths.concat(modelDir, string.format('model_%d-data', model.numSteps or 0))
  trainData:save(dataPath)
  _G.logger:info('Model saved to %s', modelPath)
  os.execute(string.format('cp %s %s', modelPath, modelPathTemp))
  os.execute(string.format('mv %s %s', modelPathTemp, modelPathLatest))
end

local function run(model, phase, batchSize, valBatchSize, numEpochs, trainData, valData, modelDir, stepsPerCheckpoint, beamSize, outputDir, learningRateInit, learningRateDecay)
  local loss = 0
  local numSamples = 0
  local numNonzeros = 0

  assert(phase == 'train' or phase == 'test', 'phase must be either train or test')
  local isForwardOnly
  if phase == 'train' then
    isForwardOnly = false
  else
    isForwardOnly = true
    numEpochs = 1
    model.numSteps = 0
    model.numSamples = 0
    model:setOutputDirectory(outputDir)
  end

  _G.logger:info('Running...')
  --local valLosses = {}
  -- Run numEpochs epochs
  if not trainData.epoch then
    trainData.epoch = 1
    if not isForwardOnly then
      trainData:shuffle()
    end
    _G.logger:info('Learning rate: %f', model.optim.args.learning_rate)
  end

  local epoch = trainData.epoch
  while epoch <= numEpochs do
    -- Run 1 epoch
    while true do
      local trainBatch = trainData:nextBatch(batchSize)
      if trainBatch == nil then
        break
      end
      local actualBatchSize = trainBatch[1]:size(1)
      local stepLoss, stats = model:step(trainBatch, isForwardOnly, beamSize) -- do one step
      collectgarbage()
      if not isForwardOnly then
        _G.logger:info('step perplexity: %f', math.exp(stepLoss/stats[1]))
      end
      numNonzeros = numNonzeros + stats[1]
      loss = loss + stepLoss
      model.numSteps = model.numSteps + 1
      model.numSamples = model.numSamples + actualBatchSize
      if model.numSteps % stepsPerCheckpoint == 0 then
        if isForwardOnly then
          _G.logger:info('Step: %d. Number of samples: %d.', model.numSteps, model.numSamples)
        else
          _G.logger:info('Step: %d. #Sample: %d. Training Perplexity: %f', model.numSteps, model.numSamples, math.exp(loss/numNonzeros))
          --_G.logger:info('Evaluating model on validation data')
          --local ppl = evaluateModel(model, valData, valBatchSize, beamSize, modelDir, epoch)
          saveModel(modelDir, model, trainData)
          loss, numNonzeros = 0, 0
          collectgarbage()
        end
      end
    end -- Run 1 epoch
    if not isForwardOnly then
      trainData:shuffle()
    end
    -- After each epoch, evaluate on validation if phase is train
    if not isForwardOnly then
      _G.logger:info('Evaluating model on validation data')
      local ppl = evaluateModel(model, valData, valBatchSize, beamSize, modelDir, epoch, true)
      local learningRateOld = model.optim.args.learning_rate
      model.optim:updateLearningRate(ppl, -math.huge)
      local learningRate = model.optim.args.learning_rate
      if learningRate < learningRateOld then
        _G.logger:info('Decay learning rate to %f', learningRate)
      end
      saveModel(modelDir, model, trainData)
    else -- isForwardOnly == true
      _G.logger:info('Epoch ends. Number of samples: %d.', numSamples)
    end
    epoch = epoch + 1
    trainData.epoch = epoch
  end -- for epoch
end -- run function

local function main()
  assert (opt.gpu_id > 0, 'Only support using GPU! Please specify a valid gpu_id.')

  _G.profiler = onmt.utils.Profiler.new(opt.profiler)
  _G.logger = onmt.utils.Logger.new(opt.log_path)
  _G.logger.mute = false
  _G.logger:info('Command Line Arguments: %s', table.concat(arg, ' ') or '')

  local gpuId = opt.gpu_id
  _G.logger:info('Using CUDA on GPU %d', gpuId)
  cutorch.setDevice(gpuId)
  onmt.utils.Cuda.init({gpuid=string.format('%d', gpuId), fp16 = opt.fp16})

  -- Convert Options
  opt.maxDecoderLength = opt.max_num_tokens + 1 -- since <StartOfSequence> is prepended to the sequence
  opt.maxEncoderLengthWidth = math.floor(opt.max_image_width / 8.0) -- feature maps after CNN become 8 times smaller
  opt.maxEncoderLengthHeight = math.floor(opt.max_image_height / 8.0) -- feature maps after CNN become 8 times smaller

  -- Build Model
  _G.logger:info('Building model')
  local model = WYGIWYS(optim)
  local modelPath = paths.concat(opt.model_dir, 'model_latest')
  if opt.load_model and paths.filep(modelPath) then
    _G.logger:info('Loading model from %s', modelPath)
    model:load(modelPath, opt)
  else
    -- Load Vocabulary
    _G.logger:info('Loading vocabulary from %s', opt.vocab_file)
    _G.idToVocab = tds.Hash() -- vocabulary file is global
    local file, err = io.open(opt.vocab_file, "r")
    if err then
      _G.logger:error('Vocabulary file %s does not exist!', opt.vocab_file)
      os.exit()
    end
    for line in file:lines() do
      local token = onmt.utils.String.strip(line)
      if onmt.utils.String.isEmpty(token) then
        token = ' '
      end
      _G.idToVocab[#_G.idToVocab+1] = token
    end
    opt.targetVocabSize = #_G.idToVocab + 4
    _G.logger:info('Creating model with fresh parameters')
    model:create(opt)
  end

  if not paths.dirp(opt.model_dir) then
    paths.mkdir(opt.model_dir)
  end

  -- Load Data
  _G.logger:info('Image directory: %s', opt.image_dir)
  _G.logger:info('Loading %s data from %s', opt.phase, opt.data_path)
  if opt.phase == 'train' and (not paths.filep(opt.label_path)) then
      _G.logger:error('Label file %s does not exist!', opt.label_path)
      os.exit(1)
  end
  local trainData = DataLoader(opt.image_dir, opt.data_path, opt.label_path, opt.max_image_height, opt.max_image_width, opt.max_num_tokens)
  local dataPath = paths.concat(opt.model_dir, string.format('model_%d-data', model.numSteps or 0))
  if opt.load_model and paths.filep(dataPath) and opt.phase == 'train' then
    _G.logger:info('Loading data state from %s', dataPath)
    trainData:load(dataPath)
  end
  _G.logger:info('Loaded')
  local valData
  if opt.phase == 'train' then
    _G.logger:info('Loading validation data from %s', opt.val_data_path)
    valData = DataLoader(opt.image_dir, opt.val_data_path, opt.label_path, opt.max_image_height, opt.max_image_width, opt.max_num_tokens)
    _G.logger:info('Loaded')
  end

  -- Run Model
  run(model, opt.phase, opt.batch_size, opt.val_batch_size or opt.batch_size, opt.num_epochs, trainData, valData, opt.model_dir, opt.steps_per_checkpoint, opt.beam_size, opt.output_dir, opt.learning_rate, opt.lr_decay)

  model:shutDown()
  _G.logger:shutDown()
end -- function main

main()


================================================
FILE: tools/README.md
================================================
# Tools

This directory contains additional tools.

## Generate Vocabulary

To generate the vocabulary:

```
python tools/generate_vocab.py -data_path data/train.txt -label_path data/labels.txt -vocab_file data/vocab.txt
```

where the options are:

* `-data_path`: Input file containing <image_path> <label_index> per line. This should be the file used for training.
    
* `-label_path`: Input file containing a tokenized formula per line.

* `-vocab_file`: Output file for the generated vocabulary. One token per line.
    
* `-unk_threshold`: If a token appears less than (including) the threshold, then it will be excluded from the vocabulary.


================================================
FILE: tools/generate_vocab.py
================================================
import sys, logging, argparse, os

def process_args(args):
    parser = argparse.ArgumentParser(description='Generate vocabulary file.')

    parser.add_argument('-data_path', dest='data_path',
                        type=str, required=True,
                        help=('The file containing <image_path> <label_index> per line. This should be the file used for training.'
                        ))
    parser.add_argument('-label_path', dest='label_path',
                        type=str, required=True,
                        help=('The file containing a tokenized label per line.'
                        ))
    parser.add_argument('-vocab_file', dest='vocab_file',
                        type=str, required=True,
                        help=('Output file for the generated vocabulary. One token per line.'
                        ))
    parser.add_argument('--unk_threshold', dest='unk_threshold',
                        type=int, default=1,
                        help=('If the number of occurences of a token is less than (including) the threshold, then it will be excluded from the generated vocabulary.'
                        ))
    parser.add_argument('--log-path', dest="log_path",
                        type=str, default='log.txt',
                        help=('Log file path, default=log.txt' 
                        ))
    parameters = parser.parse_args(args)
    return parameters

def main(args):
    parameters = process_args(args)
    logging.basicConfig(
        level=logging.INFO,
        format='%(asctime)-15s %(name)-5s %(levelname)-8s %(message)s',
        filename=parameters.log_path)

    console = logging.StreamHandler()
    console.setLevel(logging.INFO)
    formatter = logging.Formatter('%(asctime)-15s %(name)-5s %(levelname)-8s %(message)s')
    console.setFormatter(formatter)
    logging.getLogger('').addHandler(console)

    logging.info('Script being executed: %s'%__file__)

    label_path = parameters.label_path
    assert os.path.exists(label_path), label_path
    data_path = parameters.data_path
    assert os.path.exists(data_path), data_path

    formulas = open(label_path).readlines()
    vocab = {}
    max_len = 0
    with open(data_path) as fin:
        for line in fin:
            _, line_idx = line.strip().split()
            line_strip = formulas[int(line_idx)].strip()
            tokens = line_strip.split()
            tokens_out = []
            for token in tokens:
                tokens_out.append(token)
                if token not in vocab:
                    vocab[token] = 0
                vocab[token] += 1

    vocab_sort = sorted(list(vocab.keys()))
    vocab_out = []
    num_unknown = 0
    for word in vocab_sort:
        if vocab[word] > parameters.unk_threshold:
            vocab_out.append(word)
        else:
            num_unknown += 1
    vocab = [word for word in vocab_out]

    with open(parameters.vocab_file, 'w') as fout:
        fout.write('\n'.join(vocab))
    logging.info('#UNK\'s: %d'%num_unknown)

if __name__ == '__main__':
    main(sys.argv[1:])
    logging.info('Jobs finished')