\n```\n\n#### Args:\n\n\n* `method`: The method to wrap.\n\n\n#### Returns:\n\nThe original method wrapped such that it enters the module's name scope.\n\n\n\n\n"
},
{
"path": "docs/tf/haste_tf/GRUCell.md",
"content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n\n# haste_tf.GRUCell\n\n\n\n\n## Class `GRUCell`\n\nA GRU cell that's compatible with the Haste GRU layer.\n\n\n\n\n\nThis cell can be used on hardware other than GPUs and with other TensorFlow\nclasses that operate on RNN cells (e.g. `dynamic_rnn`, `BasicDecoder`, cell\nwrappers, etc.).\n\n\n\n``` python\n__init__(\n num_units,\n name=None,\n **kwargs\n)\n```\n\n\n\n\n\n\n## Properties\n\nactivity_regularizer
\n\nOptional regularizer function for the output of this layer.\n\n\ndtype
\n\n\n\n\ndynamic
\n\n\n\n\ngraph
\n\nDEPRECATED FUNCTION\n\nWarning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version.\nInstructions for updating:\nStop using this property because tf.layers layers no longer track their graph.\n\n\n\nRetrieves the input tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one input,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nInput tensor or list of input tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to\nmore than one incoming layers.\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n* `AttributeError`: If no inbound nodes are found.\n\n\n\nRetrieves the input mask tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one inbound node,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nInput mask tensor (potentially None) or list of input\nmask tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to\nmore than one incoming layers.\n\n\n\nRetrieves the input shape(s) of a layer.\n\nOnly applicable if the layer has exactly one input,\ni.e. if it is connected to one incoming layer, or if all inputs\nhave the same shape.\n\n#### Returns:\n\nInput shape, as an integer shape tuple\n(or list of shape tuples, one tuple per input tensor).\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer has no defined input_shape.\n* `RuntimeError`: if called in Eager mode.\n\nlosses
\n\nLosses which are associated with this `Layer`.\n\nVariable regularization tensors are created when this property is accessed,\nso it is eager safe: accessing `losses` under a `tf.GradientTape` will\npropagate gradients back to the corresponding variables.\n\n#### Returns:\n\nA list of tensors.\n\n\nmetrics
\n\n\n\n\nname
\n\nReturns the name of this module as passed or determined in the ctor.\n\nNOTE: This is not the same as the `self.name_scope.name` which includes\nparent module names.\n\nname_scope
\n\nReturns a `tf.name_scope` instance for this class.\n\n\nnon_trainable_variables
\n\n\n\n\nnon_trainable_weights
\n\n\n\n\noutput
\n\nRetrieves the output tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one output,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nOutput tensor or list of output tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to more than one incoming\n layers.\n* `RuntimeError`: if called in Eager mode.\n\noutput_mask
\n\nRetrieves the output mask tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one inbound node,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nOutput mask tensor (potentially None) or list of output\nmask tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to\nmore than one incoming layers.\n\noutput_shape
\n\nRetrieves the output shape(s) of a layer.\n\nOnly applicable if the layer has one output,\nor if all outputs have the same shape.\n\n#### Returns:\n\nOutput shape, as an integer shape tuple\n(or list of shape tuples, one tuple per output tensor).\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer has no defined output shape.\n* `RuntimeError`: if called in Eager mode.\n\noutput_size
\n\nInteger or TensorShape: size of outputs produced by this cell.\n\n\nscope_name
\n\n\n\n\nstate_size
\n\nsize(s) of state(s) used by this cell.\n\nIt can be represented by an Integer, a TensorShape or a tuple of Integers\nor TensorShapes.\n\nsubmodules
\n\nSequence of all sub-modules.\n\nSubmodules are modules which are properties of this module, or found as\nproperties of modules which are properties of this module (and so on).\n\n```\na = tf.Module()\nb = tf.Module()\nc = tf.Module()\na.b = b\nb.c = c\nassert list(a.submodules) == [b, c]\nassert list(b.submodules) == [c]\nassert list(c.submodules) == []\n```\n\n#### Returns:\n\nA sequence of all submodules.\n\n\ntrainable
\n\n\n\n\ntrainable_variables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\ntrainable_weights
\n\n\n\n\nupdates
\n\n\n\n\nvariables
\n\nReturns the list of all layer variables/weights.\n\nAlias of `self.weights`.\n\n#### Returns:\n\nA list of variables.\n\n\nweights
\n\nReturns the list of all layer variables/weights.\n\n\n#### Returns:\n\nA list of variables.\n\n\n\n\n## Methods\n\n\n\n``` python\n__call__(\n inputs,\n state,\n scope=None\n)\n```\n\nRun this RNN cell on inputs, starting from the given state.\n\n\n#### Args:\n\n\n* `inputs`: `2-D` tensor with shape `[batch_size, input_size]`.\n* `state`: if `self.state_size` is an integer, this should be a `2-D Tensor`\n with shape `[batch_size, self.state_size]`. Otherwise, if\n `self.state_size` is a tuple of integers, this should be a tuple with\n shapes `[batch_size, s] for s in self.state_size`.\n* `scope`: VariableScope for the created subgraph; defaults to class name.\n\n\n#### Returns:\n\n\n* `A pair containing`: \n- Output: A `2-D` tensor with shape `[batch_size, self.output_size]`.\n- New state: Either a single `2-D` tensor, or a tuple of tensors matching\n the arity and shapes of `state`.\n\n\n\n``` python\napply(\n inputs,\n *args,\n **kwargs\n)\n```\n\nApply the layer on a input.\n\nThis is an alias of `self.__call__`.\n\n#### Arguments:\n\n\n* `inputs`: Input tensor(s).\n* `*args`: additional positional arguments to be passed to `self.call`.\n* `**kwargs`: additional keyword arguments to be passed to `self.call`.\n\n\n#### Returns:\n\nOutput tensor(s).\n\n\n\n\n``` python\nbuild(shape)\n```\n\nCreates the variables of the layer (optional, for subclass implementers).\n\nThis is a method that implementers of subclasses of `Layer` or `Model`\ncan override if they need a state-creation step in-between\nlayer instantiation and layer call.\n\nThis is typically used to create the weights of `Layer` subclasses.\n\n#### Arguments:\n\n\n* `input_shape`: Instance of `TensorShape`, or list of instances of\n `TensorShape` if the layer expects a list of inputs\n (one instance per input).\n\n\n\n``` python\ncompute_mask(\n inputs,\n mask=None\n)\n```\n\nComputes an output mask tensor.\n\n\n#### Arguments:\n\n\n* `inputs`: Tensor or list of tensors.\n* `mask`: Tensor or list of tensors.\n\n\n#### Returns:\n\nNone or a tensor (or list of tensors,\n one per output tensor of the layer).\n\n\n\n\n``` python\ncompute_output_shape(input_shape)\n```\n\nComputes the output shape of the layer.\n\nAssumes that the layer will be built\nto match that input shape provided.\n\n#### Arguments:\n\n\n* `input_shape`: Shape tuple (tuple of integers)\n or list of shape tuples (one per output tensor of the layer).\n Shape tuples can include None for free dimensions,\n instead of an integer.\n\n\n#### Returns:\n\nAn input shape tuple.\n\n\n\n\n``` python\ncount_params()\n```\n\nCount the total number of scalars composing the weights.\n\n\n#### Returns:\n\nAn integer count.\n\n\n\n#### Raises:\n\n\n* `ValueError`: if the layer isn't yet built\n (in which case its weights aren't yet defined).\n\n\n\n``` python\n@classmethod\nfrom_config(\n cls,\n config\n)\n```\n\nCreates a layer from its config.\n\nThis method is the reverse of `get_config`,\ncapable of instantiating the same layer from the config\ndictionary. It does not handle layer connectivity\n(handled by Network), nor weights (handled by `set_weights`).\n\n#### Arguments:\n\n\n* `config`: A Python dictionary, typically the\n output of get_config.\n\n\n#### Returns:\n\nA layer instance.\n\n\n\n\n``` python\nget_config()\n```\n\nReturns the config of the layer.\n\nA layer config is a Python dictionary (serializable)\ncontaining the configuration of a layer.\nThe same layer can be reinstantiated later\n(without its trained weights) from this configuration.\n\nThe config of a layer does not include connectivity\ninformation, nor the layer class name. These are handled\nby `Network` (one layer of abstraction above).\n\n#### Returns:\n\nPython dictionary.\n\n\n\n\n``` python\nget_initial_state(\n inputs=None,\n batch_size=None,\n dtype=None\n)\n```\n\n\n\n\n\n\n``` python\nget_input_at(node_index)\n```\n\nRetrieves the input tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA tensor (or list of tensors if the layer has multiple inputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_input_mask_at(node_index)\n```\n\nRetrieves the input mask tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA mask tensor\n(or list of tensors if the layer has multiple inputs).\n\n\n\n\n``` python\nget_input_shape_at(node_index)\n```\n\nRetrieves the input shape(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA shape tuple\n(or list of shape tuples if the layer has multiple inputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_losses_for(inputs)\n```\n\nRetrieves losses relevant to a specific set of inputs.\n\n\n#### Arguments:\n\n\n* `inputs`: Input tensor or list/tuple of input tensors.\n\n\n#### Returns:\n\nList of loss tensors of the layer that depend on `inputs`.\n\n\n\n\n``` python\nget_output_at(node_index)\n```\n\nRetrieves the output tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA tensor (or list of tensors if the layer has multiple outputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_output_mask_at(node_index)\n```\n\nRetrieves the output mask tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA mask tensor\n(or list of tensors if the layer has multiple outputs).\n\n\n\n\n``` python\nget_output_shape_at(node_index)\n```\n\nRetrieves the output shape(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA shape tuple\n(or list of shape tuples if the layer has multiple outputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_updates_for(inputs)\n```\n\nRetrieves updates relevant to a specific set of inputs.\n\n\n#### Arguments:\n\n\n* `inputs`: Input tensor or list/tuple of input tensors.\n\n\n#### Returns:\n\nList of update ops of the layer that depend on `inputs`.\n\n\n\n\n``` python\nget_weights()\n```\n\nReturns the current weights of the layer.\n\n\n#### Returns:\n\nWeights values as a list of numpy arrays.\n\n\n\n\n``` python\nset_weights(weights)\n```\n\nSets the weights of the layer, from Numpy arrays.\n\n\n#### Arguments:\n\n\n* `weights`: a list of Numpy arrays. The number\n of arrays and their shape must match\n number of the dimensions of the weights\n of the layer (i.e. it should match the\n output of `get_weights`).\n\n\n#### Raises:\n\n\n* `ValueError`: If the provided weights list does not match the\n layer's specifications.\n\n\n\n``` python\n@classmethod\nwith_name_scope(\n cls,\n method\n)\n```\n\nDecorator to automatically enter the module name scope.\n\n```\nclass MyModule(tf.Module):\n @tf.Module.with_name_scope\n def __call__(self, x):\n if not hasattr(self, 'w'):\n self.w = tf.Variable(tf.random.normal([x.shape[1], 64]))\n return tf.matmul(x, self.w)\n```\n\nUsing the above module would produce `tf.Variable`s and `tf.Tensor`s whose\nnames included the module name:\n\n```\nmod = MyModule()\nmod(tf.ones([8, 32]))\n# ==> \nmod.w\n# ==> \n```\n\n#### Args:\n\n\n* `method`: The method to wrap.\n\n\n#### Returns:\n\nThe original method wrapped such that it enters the module's name scope.\n\n\n\n\n``` python\nzero_state(\n batch_size,\n dtype\n)\n```\n\nReturn zero-filled state tensor(s).\n\n\n#### Args:\n\n\n* `batch_size`: int, float, or unit Tensor representing the batch size.\n* `dtype`: the data type to use for the state.\n\n\n#### Returns:\n\nIf `state_size` is an int or TensorShape, then the return value is a\n`N-D` tensor of shape `[batch_size, state_size]` filled with zeros.\n\nIf `state_size` is a nested list or tuple, then the return value is\na nested list or tuple (of the same structure) of `2-D` tensors with\nthe shapes `[batch_size, s]` for each s in `state_size`.\n\n\n\n\n"
},
{
"path": "docs/tf/haste_tf/IndRNN.md",
"content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n\n# haste_tf.IndRNN\n\n\n\n\n## Class `IndRNN`\n\nIndependently Recurrent Neural Network layer.\n\n\n\n\n\nThis layer offers a fused, GPU-accelerated TensorFlow op for inference and\ntraining. It also supports Zoneout regularization.\n\n\n\n``` python\n__init__(\n num_units,\n direction='unidirectional',\n **kwargs\n)\n```\n\nInitialize the parameters of the IndRNN layer.\n\n\n#### Arguments:\n\n\n* `num_units`: int, the number of units in the IndRNN cell.\n* `direction`: string, 'unidirectional' or 'bidirectional'.\n* `**kwargs`: Dict, keyword arguments (see below).\n\n\n#### Keyword Arguments:\n\n\n* `kernel_initializer`: (optional) the initializer to use for the input\n matrix weights. Defaults to `glorot_uniform`.\n* `recurrent_initializer`: (optional) the initializer to use for the\n recurrent scale weights. Defaults to uniform random in [-0.5, 0.5].\n Note that this initialization scheme is different than in the original\n authors' implementation. See https://github.com/lmnt-com/haste/issues/7\n for details.\n* `bias_initializer`: (optional) the initializer to use for the bias vector.\n Defaults to `zeros`.\n* `kernel_transform`: (optional) a function with signature\n `(kernel: Tensor) -> Tensor` that transforms the kernel before it is\n used. Defaults to the identity function.\n* `recurrent_transform`: (optional) a function with signature\n `(recurrent_scale: Tensor) -> Tensor` that transforms the recurrent\n scale vector before it is used. Defaults to the identity function.\n* `bias_transform`: (optional) a function with signature\n `(bias: Tensor) -> Tensor` that transforms the bias before it is used.\n Defaults to the identity function.\n* `zoneout`: (optional) float, sets the zoneout rate for Zoneout\n regularization. Defaults to 0.\n* `dtype`: (optional) the data type for this layer. Defaults to `tf.float32`.\n* `name`: (optional) string, the name for this layer.\n\n\n\n## Properties\n\nbidirectional
\n\n`True` if this is a bidirectional RNN, `False` otherwise.\n\n\nname
\n\nReturns the name of this module as passed or determined in the ctor.\n\nNOTE: This is not the same as the `self.name_scope.name` which includes\nparent module names.\n\nname_scope
\n\nReturns a `tf.name_scope` instance for this class.\n\n\noutput_size
\n\n\n\n\nstate_size
\n\n\n\n\nsubmodules
\n\nSequence of all sub-modules.\n\nSubmodules are modules which are properties of this module, or found as\nproperties of modules which are properties of this module (and so on).\n\n```\na = tf.Module()\nb = tf.Module()\nc = tf.Module()\na.b = b\nb.c = c\nassert list(a.submodules) == [b, c]\nassert list(b.submodules) == [c]\nassert list(c.submodules) == []\n```\n\n#### Returns:\n\nA sequence of all submodules.\n\n\ntrainable_variables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\nvariables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\n\n\n## Methods\n\n\n\n``` python\n__call__(\n inputs,\n training,\n sequence_length=None,\n time_major=False\n)\n```\n\nRuns the RNN layer.\n\n\n#### Arguments:\n\n\n* `inputs`: Tensor, a rank 3 input tensor with shape [N,T,C] if `time_major`\n is `False`, or with shape [T,N,C] if `time_major` is `True`.\n* `training`: bool, `True` if running in training mode, `False` if running\n in inference mode.\n* `sequence_length`: (optional) Tensor, a rank 1 tensor with shape [N] and\n dtype of `tf.int32` or `tf.int64`. This tensor specifies the unpadded\n length of each example in the input minibatch.\n* `time_major`: (optional) bool, specifies whether `input` has shape [N,T,C]\n (`time_major=False`) or shape [T,N,C] (`time_major=True`).\n\n\n#### Returns:\n\nA pair, `(output, state)` for unidirectional layers, or a pair\n`([output_fw, output_bw], [state_fw, state_bw])` for bidirectional\nlayers.\n\n\n\n\n``` python\nbuild(shape)\n```\n\nCreates the variables of the layer.\n\nCalling this method is optional for users of the RNN class. It is called\ninternally with the correct shape when `__call__` is invoked.\n\n#### Arguments:\n\n\n* `shape`: instance of `TensorShape`.\n\n\n\n``` python\n@classmethod\nwith_name_scope(\n cls,\n method\n)\n```\n\nDecorator to automatically enter the module name scope.\n\n```\nclass MyModule(tf.Module):\n @tf.Module.with_name_scope\n def __call__(self, x):\n if not hasattr(self, 'w'):\n self.w = tf.Variable(tf.random.normal([x.shape[1], 64]))\n return tf.matmul(x, self.w)\n```\n\nUsing the above module would produce `tf.Variable`s and `tf.Tensor`s whose\nnames included the module name:\n\n```\nmod = MyModule()\nmod(tf.ones([8, 32]))\n# ==> \nmod.w\n# ==> \n```\n\n#### Args:\n\n\n* `method`: The method to wrap.\n\n\n#### Returns:\n\nThe original method wrapped such that it enters the module's name scope.\n\n\n\n\n"
},
{
"path": "docs/tf/haste_tf/LSTM.md",
"content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n\n# haste_tf.LSTM\n\n\n\n\n## Class `LSTM`\n\nLong Short-Term Memory layer.\n\n\n\n\n\nThis LSTM layer offers a fused, GPU-accelerated TensorFlow op for inference\nand training. Its weights and variables are compatible with `BasicLSTMCell`,\n`LSTMCell`, and `LSTMBlockCell` by default, and is able to load weights\nfrom `tf.contrib.cudnn_rnn.CudnnLSTM` when `cudnn_compat=True` is specified.\n\nAlthough this implementation is comparable in performance to cuDNN's LSTM,\nit offers additional options not typically found in other high-performance\nimplementations. DropConnect and Zoneout regularization are built-in, and\nthis layer allows setting a non-zero initial forget gate bias.\n\n\n\n``` python\n__init__(\n num_units,\n direction='unidirectional',\n **kwargs\n)\n```\n\nInitialize the parameters of the LSTM layer.\n\n\n#### Arguments:\n\n\n* `num_units`: int, the number of units in the LSTM cell.\n* `direction`: string, 'unidirectional' or 'bidirectional'.\n* `**kwargs`: Dict, keyword arguments (see below).\n\n\n#### Keyword Arguments:\n\n\n* `kernel_initializer`: (optional) the initializer to use for the input\n matrix weights. Defaults to `glorot_uniform`.\n* `recurrent_initializer`: (optional) the initializer to use for the\n recurrent matrix weights. Defaults to `orthogonal`.\n* `bias_initializer`: (optional) the initializer to use for both input and\n recurrent bias vectors. Defaults to `zeros` unless `forget_bias` is\n non-zero (see below).\n* `kernel_transform`: (optional) a function with signature\n `(kernel: Tensor) -> Tensor` that transforms the kernel before it is\n used. Defaults to the identity function.\n* `recurrent_transform`: (optional) a function with signature\n `(recurrent_kernel: Tensor) -> Tensor` that transforms the recurrent\n kernel before it is used. Defaults to the identity function.\n* `bias_transform`: (optional) a function with signature\n `(bias: Tensor) -> Tensor` that transforms the bias before it is used.\n Defaults to the identity function.\n* `forget_bias`: (optional) float, sets the initial weights for the forget\n gates. Defaults to 1 and overrides the `bias_initializer` unless this\n argument is set to 0.\n* `dropout`: (optional) float, sets the dropout rate for DropConnect\n regularization on the recurrent matrix. Defaults to 0.\n* `zoneout`: (optional) float, sets the zoneout rate for Zoneout\n regularization. Defaults to 0.\n* `dtype`: (optional) the data type for this layer. Defaults to `tf.float32`.\n* `name`: (optional) string, the name for this layer.\n* `cudnn_compat`: (optional) bool, if `True`, the variables created by this\n layer are compatible with `tf.contrib.cudnn_rnn.CudnnLSTM`. Note that\n this should only be set if you're restoring variables from a cuDNN\n model. It's currently not possible to train a model with\n `cudnn_compat=True` and restore it with CudnnLSTM. Defaults to `False`.\n\n\n\n## Properties\n\nbidirectional
\n\n`True` if this is a bidirectional RNN, `False` otherwise.\n\n\nname
\n\nReturns the name of this module as passed or determined in the ctor.\n\nNOTE: This is not the same as the `self.name_scope.name` which includes\nparent module names.\n\nname_scope
\n\nReturns a `tf.name_scope` instance for this class.\n\n\noutput_size
\n\n\n\n\nstate_size
\n\n\n\n\nsubmodules
\n\nSequence of all sub-modules.\n\nSubmodules are modules which are properties of this module, or found as\nproperties of modules which are properties of this module (and so on).\n\n```\na = tf.Module()\nb = tf.Module()\nc = tf.Module()\na.b = b\nb.c = c\nassert list(a.submodules) == [b, c]\nassert list(b.submodules) == [c]\nassert list(c.submodules) == []\n```\n\n#### Returns:\n\nA sequence of all submodules.\n\n\ntrainable_variables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\nvariables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\n\n\n## Methods\n\n\n\n``` python\n__call__(\n inputs,\n training,\n sequence_length=None,\n time_major=False\n)\n```\n\nRuns the RNN layer.\n\n\n#### Arguments:\n\n\n* `inputs`: Tensor, a rank 3 input tensor with shape [N,T,C] if `time_major`\n is `False`, or with shape [T,N,C] if `time_major` is `True`.\n* `training`: bool, `True` if running in training mode, `False` if running\n in inference mode.\n* `sequence_length`: (optional) Tensor, a rank 1 tensor with shape [N] and\n dtype of `tf.int32` or `tf.int64`. This tensor specifies the unpadded\n length of each example in the input minibatch.\n* `time_major`: (optional) bool, specifies whether `input` has shape [N,T,C]\n (`time_major=False`) or shape [T,N,C] (`time_major=True`).\n\n\n#### Returns:\n\nA pair, `(output, state)` for unidirectional layers, or a pair\n`([output_fw, output_bw], [state_fw, state_bw])` for bidirectional\nlayers.\n\n\n\n\n``` python\nbuild(shape)\n```\n\nCreates the variables of the layer.\n\nCalling this method is optional for users of the RNN class. It is called\ninternally with the correct shape when `__call__` is invoked.\n\n#### Arguments:\n\n\n* `shape`: instance of `TensorShape`.\n\n\n\n``` python\n@classmethod\nwith_name_scope(\n cls,\n method\n)\n```\n\nDecorator to automatically enter the module name scope.\n\n```\nclass MyModule(tf.Module):\n @tf.Module.with_name_scope\n def __call__(self, x):\n if not hasattr(self, 'w'):\n self.w = tf.Variable(tf.random.normal([x.shape[1], 64]))\n return tf.matmul(x, self.w)\n```\n\nUsing the above module would produce `tf.Variable`s and `tf.Tensor`s whose\nnames included the module name:\n\n```\nmod = MyModule()\nmod(tf.ones([8, 32]))\n# ==> \nmod.w\n# ==> \n```\n\n#### Args:\n\n\n* `method`: The method to wrap.\n\n\n#### Returns:\n\nThe original method wrapped such that it enters the module's name scope.\n\n\n\n\n"
},
{
"path": "docs/tf/haste_tf/LayerNorm.md",
"content": "\n\n\n\n\n\n\n\n\n\n\n\n
\n\n# haste_tf.LayerNorm\n\n\n\n\n## Class `LayerNorm`\n\nLayer normalization layer.\n\n\n\n\n\nThis class exposes a fused and GPU-accelerated implementation of layer\nnormalization as described by [Ba et al.](https://arxiv.org/abs/1607.06450)\n\n\n\n``` python\n__init__(name=None)\n```\n\nInitialize the parameters of the layer normalization layer.\n\n\n#### Arguments:\n\n\n* `name`: (optional) string, the name for this layer.\n\n\n\n## Properties\n\nname
\n\nReturns the name of this module as passed or determined in the ctor.\n\nNOTE: This is not the same as the `self.name_scope.name` which includes\nparent module names.\n\nname_scope
\n\nReturns a `tf.name_scope` instance for this class.\n\n\nsubmodules
\n\nSequence of all sub-modules.\n\nSubmodules are modules which are properties of this module, or found as\nproperties of modules which are properties of this module (and so on).\n\n```\na = tf.Module()\nb = tf.Module()\nc = tf.Module()\na.b = b\nb.c = c\nassert list(a.submodules) == [b, c]\nassert list(b.submodules) == [c]\nassert list(c.submodules) == []\n```\n\n#### Returns:\n\nA sequence of all submodules.\n\n\ntrainable_variables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\nvariables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\n\n\n## Methods\n\n\n\n``` python\n__call__(x)\n```\n\nRuns the layer.\n\n\n#### Arguments:\n\n\n* `x`: Tensor, a rank R tensor.\n\n\n#### Returns:\n\n\n* `y`: Tensor, a rank R tensor with the last dimension normalized.\n\n\n\n``` python\nbuild(shape)\n```\n\nCreates the variables of the layer.\n\nCalling this method is optional for users of the LayerNorm class. It is\ncalled internally with the correct shape when `__call__` is invoked.\n\n#### Arguments:\n\n\n* `shape`: instance of `TensorShape`.\n\n\n\n``` python\n@classmethod\nwith_name_scope(\n cls,\n method\n)\n```\n\nDecorator to automatically enter the module name scope.\n\n```\nclass MyModule(tf.Module):\n @tf.Module.with_name_scope\n def __call__(self, x):\n if not hasattr(self, 'w'):\n self.w = tf.Variable(tf.random.normal([x.shape[1], 64]))\n return tf.matmul(x, self.w)\n```\n\nUsing the above module would produce `tf.Variable`s and `tf.Tensor`s whose\nnames included the module name:\n\n```\nmod = MyModule()\nmod(tf.ones([8, 32]))\n# ==> \nmod.w\n# ==> \n```\n\n#### Args:\n\n\n* `method`: The method to wrap.\n\n\n#### Returns:\n\nThe original method wrapped such that it enters the module's name scope.\n\n\n\n\n"
},
{
"path": "docs/tf/haste_tf/LayerNormGRU.md",
"content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n\n# haste_tf.LayerNormGRU\n\n\n\n\n## Class `LayerNormGRU`\n\nLayer Normalized Gated Recurrent Unit layer.\n\n\n\n\n\nThis GRU layer applies layer normalization to the input and recurrent output\nactivations of a standard GRU. The implementation is fused and\nGPU-accelerated. There are two commonly-used variants of GRU cells. This one\nimplements 1406.1078v1 which applies the reset gate to the hidden state\nafter matrix multiplication. The other variant, 1406.1078v3, applies the\nreset gate before matrix multiplication and is currently unsupported.\n\nThis layer has built-in support for DropConnect and Zoneout, which are\nboth techniques used to regularize RNNs.\n\n\n\n``` python\n__init__(\n num_units,\n direction='unidirectional',\n **kwargs\n)\n```\n\nInitialize the parameters of the GRU layer.\n\n\n#### Arguments:\n\n\n* `num_units`: int, the number of units in the LSTM cell.\n* `direction`: string, 'unidirectional' or 'bidirectional'.\n* `**kwargs`: Dict, keyword arguments (see below).\n\n\n#### Keyword Arguments:\n\n\n* `kernel_initializer`: (optional) the initializer to use for the input\n matrix weights. Defaults to `glorot_uniform`.\n* `recurrent_initializer`: (optional) the initializer to use for the\n recurrent matrix weights. Defaults to `orthogonal`.\n* `bias_initializer`: (optional) the initializer to use for input bias\n vectors. Defaults to `zeros`.\n* `recurrent_bias_initializer`: (optional) the initializer to use for\n recurrent bias vectors. Defaults to `zeros`.\n* `kernel_transform`: (optional) a function with signature\n `(kernel: Tensor) -> Tensor` that transforms the kernel before it is\n used. Defaults to the identity function.\n* `recurrent_transform`: (optional) a function with signature\n `(recurrent_kernel: Tensor) -> Tensor` that transforms the recurrent\n kernel before it is used. Defaults to the identity function.\n* `bias_transform`: (optional) a function with signature\n `(bias: Tensor) -> Tensor` that transforms the bias before it is used.\n Defaults to the identity function.\n* `recurrent_bias_transform`: (optional) a function with signature\n `(recurrent_bias: Tensor) -> Tensor` that transforms the recurrent bias\n before it is used. Defaults to the identity function.\n* `dropout`: (optional) float, sets the dropout rate for DropConnect\n regularization on the recurrent matrix. Defaults to 0.\n* `zoneout`: (optional) float, sets the zoneout rate for Zoneout\n regularization. Defaults to 0.\n* `dtype`: (optional) the data type for this layer. Defaults to `tf.float32`.\n* `name`: (optional) string, the name for this layer.\n\n\n\n## Properties\n\nbidirectional
\n\n`True` if this is a bidirectional RNN, `False` otherwise.\n\n\nname
\n\nReturns the name of this module as passed or determined in the ctor.\n\nNOTE: This is not the same as the `self.name_scope.name` which includes\nparent module names.\n\nname_scope
\n\nReturns a `tf.name_scope` instance for this class.\n\n\noutput_size
\n\n\n\n\nstate_size
\n\n\n\n\nsubmodules
\n\nSequence of all sub-modules.\n\nSubmodules are modules which are properties of this module, or found as\nproperties of modules which are properties of this module (and so on).\n\n```\na = tf.Module()\nb = tf.Module()\nc = tf.Module()\na.b = b\nb.c = c\nassert list(a.submodules) == [b, c]\nassert list(b.submodules) == [c]\nassert list(c.submodules) == []\n```\n\n#### Returns:\n\nA sequence of all submodules.\n\n\ntrainable_variables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\nvariables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\n\n\n## Methods\n\n\n\n``` python\n__call__(\n inputs,\n training,\n sequence_length=None,\n time_major=False\n)\n```\n\nRuns the RNN layer.\n\n\n#### Arguments:\n\n\n* `inputs`: Tensor, a rank 3 input tensor with shape [N,T,C] if `time_major`\n is `False`, or with shape [T,N,C] if `time_major` is `True`.\n* `training`: bool, `True` if running in training mode, `False` if running\n in inference mode.\n* `sequence_length`: (optional) Tensor, a rank 1 tensor with shape [N] and\n dtype of `tf.int32` or `tf.int64`. This tensor specifies the unpadded\n length of each example in the input minibatch.\n* `time_major`: (optional) bool, specifies whether `input` has shape [N,T,C]\n (`time_major=False`) or shape [T,N,C] (`time_major=True`).\n\n\n#### Returns:\n\nA pair, `(output, state)` for unidirectional layers, or a pair\n`([output_fw, output_bw], [state_fw, state_bw])` for bidirectional\nlayers.\n\n\n\n\n``` python\nbuild(shape)\n```\n\nCreates the variables of the layer.\n\nCalling this method is optional for users of the RNN class. It is called\ninternally with the correct shape when `__call__` is invoked.\n\n#### Arguments:\n\n\n* `shape`: instance of `TensorShape`.\n\n\n\n``` python\n@classmethod\nwith_name_scope(\n cls,\n method\n)\n```\n\nDecorator to automatically enter the module name scope.\n\n```\nclass MyModule(tf.Module):\n @tf.Module.with_name_scope\n def __call__(self, x):\n if not hasattr(self, 'w'):\n self.w = tf.Variable(tf.random.normal([x.shape[1], 64]))\n return tf.matmul(x, self.w)\n```\n\nUsing the above module would produce `tf.Variable`s and `tf.Tensor`s whose\nnames included the module name:\n\n```\nmod = MyModule()\nmod(tf.ones([8, 32]))\n# ==> \nmod.w\n# ==> \n```\n\n#### Args:\n\n\n* `method`: The method to wrap.\n\n\n#### Returns:\n\nThe original method wrapped such that it enters the module's name scope.\n\n\n\n\n"
},
{
"path": "docs/tf/haste_tf/LayerNormGRUCell.md",
"content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n\n# haste_tf.LayerNormGRUCell\n\n\n\n\n## Class `LayerNormGRUCell`\n\nA GRU cell that's compatible with the Haste LayerNormGRU layer.\n\n\n\n\n\nThis cell can be used on hardware other than GPUs and with other TensorFlow\nclasses that operate on RNN cells (e.g. `dynamic_rnn`, `BasicDecoder`, cell\nwrappers, etc.).\n\n\n\n``` python\n__init__(\n num_units,\n forget_bias=1.0,\n dropout=0.0,\n dtype=None,\n name=None,\n **kwargs\n)\n```\n\n\n\n\n\n\n## Properties\n\nactivity_regularizer
\n\nOptional regularizer function for the output of this layer.\n\n\ndtype
\n\n\n\n\ndynamic
\n\n\n\n\ngraph
\n\nDEPRECATED FUNCTION\n\nWarning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version.\nInstructions for updating:\nStop using this property because tf.layers layers no longer track their graph.\n\n\n\nRetrieves the input tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one input,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nInput tensor or list of input tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to\nmore than one incoming layers.\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n* `AttributeError`: If no inbound nodes are found.\n\n\n\nRetrieves the input mask tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one inbound node,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nInput mask tensor (potentially None) or list of input\nmask tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to\nmore than one incoming layers.\n\n\n\nRetrieves the input shape(s) of a layer.\n\nOnly applicable if the layer has exactly one input,\ni.e. if it is connected to one incoming layer, or if all inputs\nhave the same shape.\n\n#### Returns:\n\nInput shape, as an integer shape tuple\n(or list of shape tuples, one tuple per input tensor).\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer has no defined input_shape.\n* `RuntimeError`: if called in Eager mode.\n\nlosses
\n\nLosses which are associated with this `Layer`.\n\nVariable regularization tensors are created when this property is accessed,\nso it is eager safe: accessing `losses` under a `tf.GradientTape` will\npropagate gradients back to the corresponding variables.\n\n#### Returns:\n\nA list of tensors.\n\n\nmetrics
\n\n\n\n\nname
\n\nReturns the name of this module as passed or determined in the ctor.\n\nNOTE: This is not the same as the `self.name_scope.name` which includes\nparent module names.\n\nname_scope
\n\nReturns a `tf.name_scope` instance for this class.\n\n\nnon_trainable_variables
\n\n\n\n\nnon_trainable_weights
\n\n\n\n\noutput
\n\nRetrieves the output tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one output,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nOutput tensor or list of output tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to more than one incoming\n layers.\n* `RuntimeError`: if called in Eager mode.\n\noutput_mask
\n\nRetrieves the output mask tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one inbound node,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nOutput mask tensor (potentially None) or list of output\nmask tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to\nmore than one incoming layers.\n\noutput_shape
\n\nRetrieves the output shape(s) of a layer.\n\nOnly applicable if the layer has one output,\nor if all outputs have the same shape.\n\n#### Returns:\n\nOutput shape, as an integer shape tuple\n(or list of shape tuples, one tuple per output tensor).\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer has no defined output shape.\n* `RuntimeError`: if called in Eager mode.\n\noutput_size
\n\nInteger or TensorShape: size of outputs produced by this cell.\n\n\nscope_name
\n\n\n\n\nstate_size
\n\nsize(s) of state(s) used by this cell.\n\nIt can be represented by an Integer, a TensorShape or a tuple of Integers\nor TensorShapes.\n\nsubmodules
\n\nSequence of all sub-modules.\n\nSubmodules are modules which are properties of this module, or found as\nproperties of modules which are properties of this module (and so on).\n\n```\na = tf.Module()\nb = tf.Module()\nc = tf.Module()\na.b = b\nb.c = c\nassert list(a.submodules) == [b, c]\nassert list(b.submodules) == [c]\nassert list(c.submodules) == []\n```\n\n#### Returns:\n\nA sequence of all submodules.\n\n\ntrainable
\n\n\n\n\ntrainable_variables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\ntrainable_weights
\n\n\n\n\nupdates
\n\n\n\n\nvariables
\n\nReturns the list of all layer variables/weights.\n\nAlias of `self.weights`.\n\n#### Returns:\n\nA list of variables.\n\n\nweights
\n\nReturns the list of all layer variables/weights.\n\n\n#### Returns:\n\nA list of variables.\n\n\n\n\n## Methods\n\n\n\n``` python\n__call__(\n inputs,\n state,\n scope=None\n)\n```\n\nRun this RNN cell on inputs, starting from the given state.\n\n\n#### Args:\n\n\n* `inputs`: `2-D` tensor with shape `[batch_size, input_size]`.\n* `state`: if `self.state_size` is an integer, this should be a `2-D Tensor`\n with shape `[batch_size, self.state_size]`. Otherwise, if\n `self.state_size` is a tuple of integers, this should be a tuple with\n shapes `[batch_size, s] for s in self.state_size`.\n* `scope`: VariableScope for the created subgraph; defaults to class name.\n\n\n#### Returns:\n\n\n* `A pair containing`: \n- Output: A `2-D` tensor with shape `[batch_size, self.output_size]`.\n- New state: Either a single `2-D` tensor, or a tuple of tensors matching\n the arity and shapes of `state`.\n\n\n\n``` python\napply(\n inputs,\n *args,\n **kwargs\n)\n```\n\nApply the layer on a input.\n\nThis is an alias of `self.__call__`.\n\n#### Arguments:\n\n\n* `inputs`: Input tensor(s).\n* `*args`: additional positional arguments to be passed to `self.call`.\n* `**kwargs`: additional keyword arguments to be passed to `self.call`.\n\n\n#### Returns:\n\nOutput tensor(s).\n\n\n\n\n``` python\nbuild(shape)\n```\n\nCreates the variables of the layer (optional, for subclass implementers).\n\nThis is a method that implementers of subclasses of `Layer` or `Model`\ncan override if they need a state-creation step in-between\nlayer instantiation and layer call.\n\nThis is typically used to create the weights of `Layer` subclasses.\n\n#### Arguments:\n\n\n* `input_shape`: Instance of `TensorShape`, or list of instances of\n `TensorShape` if the layer expects a list of inputs\n (one instance per input).\n\n\n\n``` python\ncompute_mask(\n inputs,\n mask=None\n)\n```\n\nComputes an output mask tensor.\n\n\n#### Arguments:\n\n\n* `inputs`: Tensor or list of tensors.\n* `mask`: Tensor or list of tensors.\n\n\n#### Returns:\n\nNone or a tensor (or list of tensors,\n one per output tensor of the layer).\n\n\n\n\n``` python\ncompute_output_shape(input_shape)\n```\n\nComputes the output shape of the layer.\n\nAssumes that the layer will be built\nto match that input shape provided.\n\n#### Arguments:\n\n\n* `input_shape`: Shape tuple (tuple of integers)\n or list of shape tuples (one per output tensor of the layer).\n Shape tuples can include None for free dimensions,\n instead of an integer.\n\n\n#### Returns:\n\nAn input shape tuple.\n\n\n\n\n``` python\ncount_params()\n```\n\nCount the total number of scalars composing the weights.\n\n\n#### Returns:\n\nAn integer count.\n\n\n\n#### Raises:\n\n\n* `ValueError`: if the layer isn't yet built\n (in which case its weights aren't yet defined).\n\n\n\n``` python\n@classmethod\nfrom_config(\n cls,\n config\n)\n```\n\nCreates a layer from its config.\n\nThis method is the reverse of `get_config`,\ncapable of instantiating the same layer from the config\ndictionary. It does not handle layer connectivity\n(handled by Network), nor weights (handled by `set_weights`).\n\n#### Arguments:\n\n\n* `config`: A Python dictionary, typically the\n output of get_config.\n\n\n#### Returns:\n\nA layer instance.\n\n\n\n\n``` python\nget_config()\n```\n\nReturns the config of the layer.\n\nA layer config is a Python dictionary (serializable)\ncontaining the configuration of a layer.\nThe same layer can be reinstantiated later\n(without its trained weights) from this configuration.\n\nThe config of a layer does not include connectivity\ninformation, nor the layer class name. These are handled\nby `Network` (one layer of abstraction above).\n\n#### Returns:\n\nPython dictionary.\n\n\n\n\n``` python\nget_initial_state(\n inputs=None,\n batch_size=None,\n dtype=None\n)\n```\n\n\n\n\n\n\n``` python\nget_input_at(node_index)\n```\n\nRetrieves the input tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA tensor (or list of tensors if the layer has multiple inputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_input_mask_at(node_index)\n```\n\nRetrieves the input mask tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA mask tensor\n(or list of tensors if the layer has multiple inputs).\n\n\n\n\n``` python\nget_input_shape_at(node_index)\n```\n\nRetrieves the input shape(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA shape tuple\n(or list of shape tuples if the layer has multiple inputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_losses_for(inputs)\n```\n\nRetrieves losses relevant to a specific set of inputs.\n\n\n#### Arguments:\n\n\n* `inputs`: Input tensor or list/tuple of input tensors.\n\n\n#### Returns:\n\nList of loss tensors of the layer that depend on `inputs`.\n\n\n\n\n``` python\nget_output_at(node_index)\n```\n\nRetrieves the output tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA tensor (or list of tensors if the layer has multiple outputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_output_mask_at(node_index)\n```\n\nRetrieves the output mask tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA mask tensor\n(or list of tensors if the layer has multiple outputs).\n\n\n\n\n``` python\nget_output_shape_at(node_index)\n```\n\nRetrieves the output shape(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA shape tuple\n(or list of shape tuples if the layer has multiple outputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_updates_for(inputs)\n```\n\nRetrieves updates relevant to a specific set of inputs.\n\n\n#### Arguments:\n\n\n* `inputs`: Input tensor or list/tuple of input tensors.\n\n\n#### Returns:\n\nList of update ops of the layer that depend on `inputs`.\n\n\n\n\n``` python\nget_weights()\n```\n\nReturns the current weights of the layer.\n\n\n#### Returns:\n\nWeights values as a list of numpy arrays.\n\n\n\n\n``` python\nset_weights(weights)\n```\n\nSets the weights of the layer, from Numpy arrays.\n\n\n#### Arguments:\n\n\n* `weights`: a list of Numpy arrays. The number\n of arrays and their shape must match\n number of the dimensions of the weights\n of the layer (i.e. it should match the\n output of `get_weights`).\n\n\n#### Raises:\n\n\n* `ValueError`: If the provided weights list does not match the\n layer's specifications.\n\n\n\n``` python\n@classmethod\nwith_name_scope(\n cls,\n method\n)\n```\n\nDecorator to automatically enter the module name scope.\n\n```\nclass MyModule(tf.Module):\n @tf.Module.with_name_scope\n def __call__(self, x):\n if not hasattr(self, 'w'):\n self.w = tf.Variable(tf.random.normal([x.shape[1], 64]))\n return tf.matmul(x, self.w)\n```\n\nUsing the above module would produce `tf.Variable`s and `tf.Tensor`s whose\nnames included the module name:\n\n```\nmod = MyModule()\nmod(tf.ones([8, 32]))\n# ==> \nmod.w\n# ==> \n```\n\n#### Args:\n\n\n* `method`: The method to wrap.\n\n\n#### Returns:\n\nThe original method wrapped such that it enters the module's name scope.\n\n\n\n\n``` python\nzero_state(\n batch_size,\n dtype\n)\n```\n\nReturn zero-filled state tensor(s).\n\n\n#### Args:\n\n\n* `batch_size`: int, float, or unit Tensor representing the batch size.\n* `dtype`: the data type to use for the state.\n\n\n#### Returns:\n\nIf `state_size` is an int or TensorShape, then the return value is a\n`N-D` tensor of shape `[batch_size, state_size]` filled with zeros.\n\nIf `state_size` is a nested list or tuple, then the return value is\na nested list or tuple (of the same structure) of `2-D` tensors with\nthe shapes `[batch_size, s]` for each s in `state_size`.\n\n\n\n\n"
},
{
"path": "docs/tf/haste_tf/LayerNormLSTM.md",
"content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n\n# haste_tf.LayerNormLSTM\n\n\n\n\n## Class `LayerNormLSTM`\n\nLayer Normalized Long Short-Term Memory layer.\n\n\n\n\n\nThis LSTM layer applies layer normalization to the input, recurrent, and\noutput activations of a standard LSTM. The implementation is fused and\nGPU-accelerated. DropConnect and Zoneout regularization are built-in, and\nthis layer allows setting a non-zero initial forget gate bias.\n\nDetails about the exact function this layer implements can be found at\nhttps://github.com/lmnt-com/haste/issues/1.\n\n\n\n``` python\n__init__(\n num_units,\n direction='unidirectional',\n **kwargs\n)\n```\n\nInitialize the parameters of the LSTM layer.\n\n\n#### Arguments:\n\n\n* `num_units`: int, the number of units in the LSTM cell.\n* `direction`: string, 'unidirectional' or 'bidirectional'.\n* `**kwargs`: Dict, keyword arguments (see below).\n\n\n#### Keyword Arguments:\n\n\n* `kernel_initializer`: (optional) the initializer to use for the input\n matrix weights. Defaults to `glorot_uniform`.\n* `recurrent_initializer`: (optional) the initializer to use for the\n recurrent matrix weights. Defaults to `orthogonal`.\n* `bias_initializer`: (optional) the initializer to use for both input and\n recurrent bias vectors. Defaults to `zeros` unless `forget_bias` is\n non-zero (see below).\n* `kernel_transform`: (optional) a function with signature\n `(kernel: Tensor) -> Tensor` that transforms the kernel before it is\n used. Defaults to the identity function.\n* `recurrent_transform`: (optional) a function with signature\n `(recurrent_kernel: Tensor) -> Tensor` that transforms the recurrent\n kernel before it is used. Defaults to the identity function.\n* `bias_transform`: (optional) a function with signature\n `(bias: Tensor) -> Tensor` that transforms the bias before it is used.\n Defaults to the identity function.\n* `forget_bias`: (optional) float, sets the initial weights for the forget\n gates. Defaults to 1 and overrides the `bias_initializer` unless this\n argument is set to 0.\n* `dropout`: (optional) float, sets the dropout rate for DropConnect\n regularization on the recurrent matrix. Defaults to 0.\n* `zoneout`: (optional) float, sets the zoneout rate for Zoneout\n regularization. Defaults to 0.\n* `dtype`: (optional) the data type for this layer. Defaults to `tf.float32`.\n* `name`: (optional) string, the name for this layer.\n\n\n\n## Properties\n\nbidirectional
\n\n`True` if this is a bidirectional RNN, `False` otherwise.\n\n\nname
\n\nReturns the name of this module as passed or determined in the ctor.\n\nNOTE: This is not the same as the `self.name_scope.name` which includes\nparent module names.\n\nname_scope
\n\nReturns a `tf.name_scope` instance for this class.\n\n\noutput_size
\n\n\n\n\nstate_size
\n\n\n\n\nsubmodules
\n\nSequence of all sub-modules.\n\nSubmodules are modules which are properties of this module, or found as\nproperties of modules which are properties of this module (and so on).\n\n```\na = tf.Module()\nb = tf.Module()\nc = tf.Module()\na.b = b\nb.c = c\nassert list(a.submodules) == [b, c]\nassert list(b.submodules) == [c]\nassert list(c.submodules) == []\n```\n\n#### Returns:\n\nA sequence of all submodules.\n\n\ntrainable_variables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\nvariables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\n\n\n## Methods\n\n\n\n``` python\n__call__(\n inputs,\n training,\n sequence_length=None,\n time_major=False\n)\n```\n\nRuns the RNN layer.\n\n\n#### Arguments:\n\n\n* `inputs`: Tensor, a rank 3 input tensor with shape [N,T,C] if `time_major`\n is `False`, or with shape [T,N,C] if `time_major` is `True`.\n* `training`: bool, `True` if running in training mode, `False` if running\n in inference mode.\n* `sequence_length`: (optional) Tensor, a rank 1 tensor with shape [N] and\n dtype of `tf.int32` or `tf.int64`. This tensor specifies the unpadded\n length of each example in the input minibatch.\n* `time_major`: (optional) bool, specifies whether `input` has shape [N,T,C]\n (`time_major=False`) or shape [T,N,C] (`time_major=True`).\n\n\n#### Returns:\n\nA pair, `(output, state)` for unidirectional layers, or a pair\n`([output_fw, output_bw], [state_fw, state_bw])` for bidirectional\nlayers.\n\n\n\n\n``` python\nbuild(shape)\n```\n\nCreates the variables of the layer.\n\nCalling this method is optional for users of the RNN class. It is called\ninternally with the correct shape when `__call__` is invoked.\n\n#### Arguments:\n\n\n* `shape`: instance of `TensorShape`.\n\n\n\n``` python\n@classmethod\nwith_name_scope(\n cls,\n method\n)\n```\n\nDecorator to automatically enter the module name scope.\n\n```\nclass MyModule(tf.Module):\n @tf.Module.with_name_scope\n def __call__(self, x):\n if not hasattr(self, 'w'):\n self.w = tf.Variable(tf.random.normal([x.shape[1], 64]))\n return tf.matmul(x, self.w)\n```\n\nUsing the above module would produce `tf.Variable`s and `tf.Tensor`s whose\nnames included the module name:\n\n```\nmod = MyModule()\nmod(tf.ones([8, 32]))\n# ==> \nmod.w\n# ==> \n```\n\n#### Args:\n\n\n* `method`: The method to wrap.\n\n\n#### Returns:\n\nThe original method wrapped such that it enters the module's name scope.\n\n\n\n\n"
},
{
"path": "docs/tf/haste_tf/LayerNormLSTMCell.md",
"content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n\n# haste_tf.LayerNormLSTMCell\n\n\n\n\n## Class `LayerNormLSTMCell`\n\nAn LSTM cell that's compatible with the Haste LayerNormLSTM layer.\n\n\n\n\n\nThis cell can be used on hardware other than GPUs and with other TensorFlow\nclasses that operate on RNN cells (e.g. `dynamic_rnn`, `BasicDecoder`, cell\nwrappers, etc.).\n\n\n\n``` python\n__init__(\n num_units,\n forget_bias=1.0,\n dropout=0.0,\n dtype=None,\n name=None,\n **kwargs\n)\n```\n\n\n\n\n\n\n## Properties\n\nactivity_regularizer
\n\nOptional regularizer function for the output of this layer.\n\n\ndtype
\n\n\n\n\ndynamic
\n\n\n\n\ngraph
\n\nDEPRECATED FUNCTION\n\nWarning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version.\nInstructions for updating:\nStop using this property because tf.layers layers no longer track their graph.\n\n\n\nRetrieves the input tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one input,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nInput tensor or list of input tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to\nmore than one incoming layers.\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n* `AttributeError`: If no inbound nodes are found.\n\n\n\nRetrieves the input mask tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one inbound node,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nInput mask tensor (potentially None) or list of input\nmask tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to\nmore than one incoming layers.\n\n\n\nRetrieves the input shape(s) of a layer.\n\nOnly applicable if the layer has exactly one input,\ni.e. if it is connected to one incoming layer, or if all inputs\nhave the same shape.\n\n#### Returns:\n\nInput shape, as an integer shape tuple\n(or list of shape tuples, one tuple per input tensor).\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer has no defined input_shape.\n* `RuntimeError`: if called in Eager mode.\n\nlosses
\n\nLosses which are associated with this `Layer`.\n\nVariable regularization tensors are created when this property is accessed,\nso it is eager safe: accessing `losses` under a `tf.GradientTape` will\npropagate gradients back to the corresponding variables.\n\n#### Returns:\n\nA list of tensors.\n\n\nmetrics
\n\n\n\n\nname
\n\nReturns the name of this module as passed or determined in the ctor.\n\nNOTE: This is not the same as the `self.name_scope.name` which includes\nparent module names.\n\nname_scope
\n\nReturns a `tf.name_scope` instance for this class.\n\n\nnon_trainable_variables
\n\n\n\n\nnon_trainable_weights
\n\n\n\n\noutput
\n\nRetrieves the output tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one output,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nOutput tensor or list of output tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to more than one incoming\n layers.\n* `RuntimeError`: if called in Eager mode.\n\noutput_mask
\n\nRetrieves the output mask tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one inbound node,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nOutput mask tensor (potentially None) or list of output\nmask tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to\nmore than one incoming layers.\n\noutput_shape
\n\nRetrieves the output shape(s) of a layer.\n\nOnly applicable if the layer has one output,\nor if all outputs have the same shape.\n\n#### Returns:\n\nOutput shape, as an integer shape tuple\n(or list of shape tuples, one tuple per output tensor).\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer has no defined output shape.\n* `RuntimeError`: if called in Eager mode.\n\noutput_size
\n\nInteger or TensorShape: size of outputs produced by this cell.\n\n\nscope_name
\n\n\n\n\nstate_size
\n\nsize(s) of state(s) used by this cell.\n\nIt can be represented by an Integer, a TensorShape or a tuple of Integers\nor TensorShapes.\n\nsubmodules
\n\nSequence of all sub-modules.\n\nSubmodules are modules which are properties of this module, or found as\nproperties of modules which are properties of this module (and so on).\n\n```\na = tf.Module()\nb = tf.Module()\nc = tf.Module()\na.b = b\nb.c = c\nassert list(a.submodules) == [b, c]\nassert list(b.submodules) == [c]\nassert list(c.submodules) == []\n```\n\n#### Returns:\n\nA sequence of all submodules.\n\n\ntrainable
\n\n\n\n\ntrainable_variables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\ntrainable_weights
\n\n\n\n\nupdates
\n\n\n\n\nvariables
\n\nReturns the list of all layer variables/weights.\n\nAlias of `self.weights`.\n\n#### Returns:\n\nA list of variables.\n\n\nweights
\n\nReturns the list of all layer variables/weights.\n\n\n#### Returns:\n\nA list of variables.\n\n\n\n\n## Methods\n\n\n\n``` python\n__call__(\n inputs,\n state,\n scope=None\n)\n```\n\nRun this RNN cell on inputs, starting from the given state.\n\n\n#### Args:\n\n\n* `inputs`: `2-D` tensor with shape `[batch_size, input_size]`.\n* `state`: if `self.state_size` is an integer, this should be a `2-D Tensor`\n with shape `[batch_size, self.state_size]`. Otherwise, if\n `self.state_size` is a tuple of integers, this should be a tuple with\n shapes `[batch_size, s] for s in self.state_size`.\n* `scope`: VariableScope for the created subgraph; defaults to class name.\n\n\n#### Returns:\n\n\n* `A pair containing`: \n- Output: A `2-D` tensor with shape `[batch_size, self.output_size]`.\n- New state: Either a single `2-D` tensor, or a tuple of tensors matching\n the arity and shapes of `state`.\n\n\n\n``` python\napply(\n inputs,\n *args,\n **kwargs\n)\n```\n\nApply the layer on a input.\n\nThis is an alias of `self.__call__`.\n\n#### Arguments:\n\n\n* `inputs`: Input tensor(s).\n* `*args`: additional positional arguments to be passed to `self.call`.\n* `**kwargs`: additional keyword arguments to be passed to `self.call`.\n\n\n#### Returns:\n\nOutput tensor(s).\n\n\n\n\n``` python\nbuild(shape)\n```\n\nCreates the variables of the layer (optional, for subclass implementers).\n\nThis is a method that implementers of subclasses of `Layer` or `Model`\ncan override if they need a state-creation step in-between\nlayer instantiation and layer call.\n\nThis is typically used to create the weights of `Layer` subclasses.\n\n#### Arguments:\n\n\n* `input_shape`: Instance of `TensorShape`, or list of instances of\n `TensorShape` if the layer expects a list of inputs\n (one instance per input).\n\n\n\n``` python\ncompute_mask(\n inputs,\n mask=None\n)\n```\n\nComputes an output mask tensor.\n\n\n#### Arguments:\n\n\n* `inputs`: Tensor or list of tensors.\n* `mask`: Tensor or list of tensors.\n\n\n#### Returns:\n\nNone or a tensor (or list of tensors,\n one per output tensor of the layer).\n\n\n\n\n``` python\ncompute_output_shape(input_shape)\n```\n\nComputes the output shape of the layer.\n\nAssumes that the layer will be built\nto match that input shape provided.\n\n#### Arguments:\n\n\n* `input_shape`: Shape tuple (tuple of integers)\n or list of shape tuples (one per output tensor of the layer).\n Shape tuples can include None for free dimensions,\n instead of an integer.\n\n\n#### Returns:\n\nAn input shape tuple.\n\n\n\n\n``` python\ncount_params()\n```\n\nCount the total number of scalars composing the weights.\n\n\n#### Returns:\n\nAn integer count.\n\n\n\n#### Raises:\n\n\n* `ValueError`: if the layer isn't yet built\n (in which case its weights aren't yet defined).\n\n\n\n``` python\n@classmethod\nfrom_config(\n cls,\n config\n)\n```\n\nCreates a layer from its config.\n\nThis method is the reverse of `get_config`,\ncapable of instantiating the same layer from the config\ndictionary. It does not handle layer connectivity\n(handled by Network), nor weights (handled by `set_weights`).\n\n#### Arguments:\n\n\n* `config`: A Python dictionary, typically the\n output of get_config.\n\n\n#### Returns:\n\nA layer instance.\n\n\n\n\n``` python\nget_config()\n```\n\nReturns the config of the layer.\n\nA layer config is a Python dictionary (serializable)\ncontaining the configuration of a layer.\nThe same layer can be reinstantiated later\n(without its trained weights) from this configuration.\n\nThe config of a layer does not include connectivity\ninformation, nor the layer class name. These are handled\nby `Network` (one layer of abstraction above).\n\n#### Returns:\n\nPython dictionary.\n\n\n\n\n``` python\nget_initial_state(\n inputs=None,\n batch_size=None,\n dtype=None\n)\n```\n\n\n\n\n\n\n``` python\nget_input_at(node_index)\n```\n\nRetrieves the input tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA tensor (or list of tensors if the layer has multiple inputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_input_mask_at(node_index)\n```\n\nRetrieves the input mask tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA mask tensor\n(or list of tensors if the layer has multiple inputs).\n\n\n\n\n``` python\nget_input_shape_at(node_index)\n```\n\nRetrieves the input shape(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA shape tuple\n(or list of shape tuples if the layer has multiple inputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_losses_for(inputs)\n```\n\nRetrieves losses relevant to a specific set of inputs.\n\n\n#### Arguments:\n\n\n* `inputs`: Input tensor or list/tuple of input tensors.\n\n\n#### Returns:\n\nList of loss tensors of the layer that depend on `inputs`.\n\n\n\n\n``` python\nget_output_at(node_index)\n```\n\nRetrieves the output tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA tensor (or list of tensors if the layer has multiple outputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_output_mask_at(node_index)\n```\n\nRetrieves the output mask tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA mask tensor\n(or list of tensors if the layer has multiple outputs).\n\n\n\n\n``` python\nget_output_shape_at(node_index)\n```\n\nRetrieves the output shape(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA shape tuple\n(or list of shape tuples if the layer has multiple outputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_updates_for(inputs)\n```\n\nRetrieves updates relevant to a specific set of inputs.\n\n\n#### Arguments:\n\n\n* `inputs`: Input tensor or list/tuple of input tensors.\n\n\n#### Returns:\n\nList of update ops of the layer that depend on `inputs`.\n\n\n\n\n``` python\nget_weights()\n```\n\nReturns the current weights of the layer.\n\n\n#### Returns:\n\nWeights values as a list of numpy arrays.\n\n\n\n\n``` python\nset_weights(weights)\n```\n\nSets the weights of the layer, from Numpy arrays.\n\n\n#### Arguments:\n\n\n* `weights`: a list of Numpy arrays. The number\n of arrays and their shape must match\n number of the dimensions of the weights\n of the layer (i.e. it should match the\n output of `get_weights`).\n\n\n#### Raises:\n\n\n* `ValueError`: If the provided weights list does not match the\n layer's specifications.\n\n\n\n``` python\n@classmethod\nwith_name_scope(\n cls,\n method\n)\n```\n\nDecorator to automatically enter the module name scope.\n\n```\nclass MyModule(tf.Module):\n @tf.Module.with_name_scope\n def __call__(self, x):\n if not hasattr(self, 'w'):\n self.w = tf.Variable(tf.random.normal([x.shape[1], 64]))\n return tf.matmul(x, self.w)\n```\n\nUsing the above module would produce `tf.Variable`s and `tf.Tensor`s whose\nnames included the module name:\n\n```\nmod = MyModule()\nmod(tf.ones([8, 32]))\n# ==> \nmod.w\n# ==> \n```\n\n#### Args:\n\n\n* `method`: The method to wrap.\n\n\n#### Returns:\n\nThe original method wrapped such that it enters the module's name scope.\n\n\n\n\n``` python\nzero_state(\n batch_size,\n dtype\n)\n```\n\nReturn zero-filled state tensor(s).\n\n\n#### Args:\n\n\n* `batch_size`: int, float, or unit Tensor representing the batch size.\n* `dtype`: the data type to use for the state.\n\n\n#### Returns:\n\nIf `state_size` is an int or TensorShape, then the return value is a\n`N-D` tensor of shape `[batch_size, state_size]` filled with zeros.\n\nIf `state_size` is a nested list or tuple, then the return value is\na nested list or tuple (of the same structure) of `2-D` tensors with\nthe shapes `[batch_size, s]` for each s in `state_size`.\n\n\n\n\n"
},
{
"path": "docs/tf/haste_tf/ZoneoutWrapper.md",
"content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n\n# haste_tf.ZoneoutWrapper\n\n\n\n\n## Class `ZoneoutWrapper`\n\nAn LSTM/GRU cell wrapper that applies zoneout to the inner cell's hidden state.\n\n\n\n\n\nThe zoneout paper applies zoneout to both the cell state and hidden state,\neach with its own zoneout rate. This class (and the `LSTM` implementation in Haste)\napplies zoneout to the hidden state and not the cell state.\n\n\n\n``` python\n__init__(\n cell,\n rate,\n training\n)\n```\n\nInitialize the parameters of the zoneout wrapper.\n\n\n#### Arguments:\n\n\n* `cell`: RNNCell, an instance of {`BasicLSTMCell`, `LSTMCell`,\n `LSTMBlockCell`, haste_tf.GRUCell} on which to apply zoneout.\n* `rate`: float, 0 <= rate <= 1, the percent of hidden units to zone out per\n time step.\n* `training`: bool, `True` if used during training, `False` if used during\n inference.\n\n\n\n## Properties\n\nactivity_regularizer
\n\nOptional regularizer function for the output of this layer.\n\n\ndtype
\n\n\n\n\ndynamic
\n\n\n\n\ngraph
\n\nDEPRECATED FUNCTION\n\nWarning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version.\nInstructions for updating:\nStop using this property because tf.layers layers no longer track their graph.\n\n\n\nRetrieves the input tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one input,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nInput tensor or list of input tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to\nmore than one incoming layers.\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n* `AttributeError`: If no inbound nodes are found.\n\n\n\nRetrieves the input mask tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one inbound node,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nInput mask tensor (potentially None) or list of input\nmask tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to\nmore than one incoming layers.\n\n\n\nRetrieves the input shape(s) of a layer.\n\nOnly applicable if the layer has exactly one input,\ni.e. if it is connected to one incoming layer, or if all inputs\nhave the same shape.\n\n#### Returns:\n\nInput shape, as an integer shape tuple\n(or list of shape tuples, one tuple per input tensor).\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer has no defined input_shape.\n* `RuntimeError`: if called in Eager mode.\n\nlosses
\n\nLosses which are associated with this `Layer`.\n\nVariable regularization tensors are created when this property is accessed,\nso it is eager safe: accessing `losses` under a `tf.GradientTape` will\npropagate gradients back to the corresponding variables.\n\n#### Returns:\n\nA list of tensors.\n\n\nmetrics
\n\n\n\n\nname
\n\nReturns the name of this module as passed or determined in the ctor.\n\nNOTE: This is not the same as the `self.name_scope.name` which includes\nparent module names.\n\nname_scope
\n\nReturns a `tf.name_scope` instance for this class.\n\n\nnon_trainable_variables
\n\n\n\n\nnon_trainable_weights
\n\n\n\n\noutput
\n\nRetrieves the output tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one output,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nOutput tensor or list of output tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to more than one incoming\n layers.\n* `RuntimeError`: if called in Eager mode.\n\noutput_mask
\n\nRetrieves the output mask tensor(s) of a layer.\n\nOnly applicable if the layer has exactly one inbound node,\ni.e. if it is connected to one incoming layer.\n\n#### Returns:\n\nOutput mask tensor (potentially None) or list of output\nmask tensors.\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer is connected to\nmore than one incoming layers.\n\noutput_shape
\n\nRetrieves the output shape(s) of a layer.\n\nOnly applicable if the layer has one output,\nor if all outputs have the same shape.\n\n#### Returns:\n\nOutput shape, as an integer shape tuple\n(or list of shape tuples, one tuple per output tensor).\n\n\n\n#### Raises:\n\n\n* `AttributeError`: if the layer has no defined output shape.\n* `RuntimeError`: if called in Eager mode.\n\noutput_size
\n\nInteger or TensorShape: size of outputs produced by this cell.\n\n\nscope_name
\n\n\n\n\nstate_size
\n\nsize(s) of state(s) used by this cell.\n\nIt can be represented by an Integer, a TensorShape or a tuple of Integers\nor TensorShapes.\n\nsubmodules
\n\nSequence of all sub-modules.\n\nSubmodules are modules which are properties of this module, or found as\nproperties of modules which are properties of this module (and so on).\n\n```\na = tf.Module()\nb = tf.Module()\nc = tf.Module()\na.b = b\nb.c = c\nassert list(a.submodules) == [b, c]\nassert list(b.submodules) == [c]\nassert list(c.submodules) == []\n```\n\n#### Returns:\n\nA sequence of all submodules.\n\n\ntrainable
\n\n\n\n\ntrainable_variables
\n\nSequence of variables owned by this module and it's submodules.\n\nNote: this method uses reflection to find variables on the current instance\nand submodules. For performance reasons you may wish to cache the result\nof calling this method if you don't expect the return value to change.\n\n#### Returns:\n\nA sequence of variables for the current module (sorted by attribute\nname) followed by variables from all submodules recursively (breadth\nfirst).\n\n\ntrainable_weights
\n\n\n\n\nupdates
\n\n\n\n\nvariables
\n\nReturns the list of all layer variables/weights.\n\nAlias of `self.weights`.\n\n#### Returns:\n\nA list of variables.\n\n\nweights
\n\nReturns the list of all layer variables/weights.\n\n\n#### Returns:\n\nA list of variables.\n\n\n\n\n## Methods\n\n\n\n``` python\n__call__(\n inputs,\n state,\n scope=None\n)\n```\n\nRuns one step of the RNN cell with zoneout applied.\n\n\n#### Arguments:\n\nsee documentation for the inner cell.\n\n\n\n\n``` python\napply(\n inputs,\n *args,\n **kwargs\n)\n```\n\nApply the layer on a input.\n\nThis is an alias of `self.__call__`.\n\n#### Arguments:\n\n\n* `inputs`: Input tensor(s).\n* `*args`: additional positional arguments to be passed to `self.call`.\n* `**kwargs`: additional keyword arguments to be passed to `self.call`.\n\n\n#### Returns:\n\nOutput tensor(s).\n\n\n\n\n``` python\nbuild(_)\n```\n\nCreates the variables of the layer (optional, for subclass implementers).\n\nThis is a method that implementers of subclasses of `Layer` or `Model`\ncan override if they need a state-creation step in-between\nlayer instantiation and layer call.\n\nThis is typically used to create the weights of `Layer` subclasses.\n\n#### Arguments:\n\n\n* `input_shape`: Instance of `TensorShape`, or list of instances of\n `TensorShape` if the layer expects a list of inputs\n (one instance per input).\n\n\n\n``` python\ncompute_mask(\n inputs,\n mask=None\n)\n```\n\nComputes an output mask tensor.\n\n\n#### Arguments:\n\n\n* `inputs`: Tensor or list of tensors.\n* `mask`: Tensor or list of tensors.\n\n\n#### Returns:\n\nNone or a tensor (or list of tensors,\n one per output tensor of the layer).\n\n\n\n\n``` python\ncompute_output_shape(input_shape)\n```\n\nComputes the output shape of the layer.\n\nAssumes that the layer will be built\nto match that input shape provided.\n\n#### Arguments:\n\n\n* `input_shape`: Shape tuple (tuple of integers)\n or list of shape tuples (one per output tensor of the layer).\n Shape tuples can include None for free dimensions,\n instead of an integer.\n\n\n#### Returns:\n\nAn input shape tuple.\n\n\n\n\n``` python\ncount_params()\n```\n\nCount the total number of scalars composing the weights.\n\n\n#### Returns:\n\nAn integer count.\n\n\n\n#### Raises:\n\n\n* `ValueError`: if the layer isn't yet built\n (in which case its weights aren't yet defined).\n\n\n\n``` python\n@classmethod\nfrom_config(\n cls,\n config\n)\n```\n\nCreates a layer from its config.\n\nThis method is the reverse of `get_config`,\ncapable of instantiating the same layer from the config\ndictionary. It does not handle layer connectivity\n(handled by Network), nor weights (handled by `set_weights`).\n\n#### Arguments:\n\n\n* `config`: A Python dictionary, typically the\n output of get_config.\n\n\n#### Returns:\n\nA layer instance.\n\n\n\n\n``` python\nget_config()\n```\n\nReturns the config of the layer.\n\nA layer config is a Python dictionary (serializable)\ncontaining the configuration of a layer.\nThe same layer can be reinstantiated later\n(without its trained weights) from this configuration.\n\nThe config of a layer does not include connectivity\ninformation, nor the layer class name. These are handled\nby `Network` (one layer of abstraction above).\n\n#### Returns:\n\nPython dictionary.\n\n\n\n\n``` python\nget_initial_state(\n inputs=None,\n batch_size=None,\n dtype=None\n)\n```\n\n\n\n\n\n\n``` python\nget_input_at(node_index)\n```\n\nRetrieves the input tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA tensor (or list of tensors if the layer has multiple inputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_input_mask_at(node_index)\n```\n\nRetrieves the input mask tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA mask tensor\n(or list of tensors if the layer has multiple inputs).\n\n\n\n\n``` python\nget_input_shape_at(node_index)\n```\n\nRetrieves the input shape(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA shape tuple\n(or list of shape tuples if the layer has multiple inputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_losses_for(inputs)\n```\n\nRetrieves losses relevant to a specific set of inputs.\n\n\n#### Arguments:\n\n\n* `inputs`: Input tensor or list/tuple of input tensors.\n\n\n#### Returns:\n\nList of loss tensors of the layer that depend on `inputs`.\n\n\n\n\n``` python\nget_output_at(node_index)\n```\n\nRetrieves the output tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA tensor (or list of tensors if the layer has multiple outputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_output_mask_at(node_index)\n```\n\nRetrieves the output mask tensor(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA mask tensor\n(or list of tensors if the layer has multiple outputs).\n\n\n\n\n``` python\nget_output_shape_at(node_index)\n```\n\nRetrieves the output shape(s) of a layer at a given node.\n\n\n#### Arguments:\n\n\n* `node_index`: Integer, index of the node\n from which to retrieve the attribute.\n E.g. `node_index=0` will correspond to the\n first time the layer was called.\n\n\n#### Returns:\n\nA shape tuple\n(or list of shape tuples if the layer has multiple outputs).\n\n\n\n#### Raises:\n\n\n* `RuntimeError`: If called in Eager mode.\n\n\n\n``` python\nget_updates_for(inputs)\n```\n\nRetrieves updates relevant to a specific set of inputs.\n\n\n#### Arguments:\n\n\n* `inputs`: Input tensor or list/tuple of input tensors.\n\n\n#### Returns:\n\nList of update ops of the layer that depend on `inputs`.\n\n\n\n\n``` python\nget_weights()\n```\n\nReturns the current weights of the layer.\n\n\n#### Returns:\n\nWeights values as a list of numpy arrays.\n\n\n\n\n``` python\nset_weights(weights)\n```\n\nSets the weights of the layer, from Numpy arrays.\n\n\n#### Arguments:\n\n\n* `weights`: a list of Numpy arrays. The number\n of arrays and their shape must match\n number of the dimensions of the weights\n of the layer (i.e. it should match the\n output of `get_weights`).\n\n\n#### Raises:\n\n\n* `ValueError`: If the provided weights list does not match the\n layer's specifications.\n\n\n\n``` python\n@classmethod\nwith_name_scope(\n cls,\n method\n)\n```\n\nDecorator to automatically enter the module name scope.\n\n```\nclass MyModule(tf.Module):\n @tf.Module.with_name_scope\n def __call__(self, x):\n if not hasattr(self, 'w'):\n self.w = tf.Variable(tf.random.normal([x.shape[1], 64]))\n return tf.matmul(x, self.w)\n```\n\nUsing the above module would produce `tf.Variable`s and `tf.Tensor`s whose\nnames included the module name:\n\n```\nmod = MyModule()\nmod(tf.ones([8, 32]))\n# ==> \nmod.w\n# ==> \n```\n\n#### Args:\n\n\n* `method`: The method to wrap.\n\n\n#### Returns:\n\nThe original method wrapped such that it enters the module's name scope.\n\n\n\n\n``` python\nzero_state(\n batch_size,\n dtype\n)\n```\n\nReturn zero-filled state tensor(s).\n\n\n#### Args:\n\n\n* `batch_size`: int, float, or unit Tensor representing the batch size.\n* `dtype`: the data type to use for the state.\n\n\n#### Returns:\n\nIf `state_size` is an int or TensorShape, then the return value is a\n`N-D` tensor of shape `[batch_size, state_size]` filled with zeros.\n\nIf `state_size` is a nested list or tuple, then the return value is\na nested list or tuple (of the same structure) of `2-D` tensors with\nthe shapes `[batch_size, s]` for each s in `state_size`.\n\n\n\n\n"
},
{
"path": "docs/tf/haste_tf.md",
"content": "\n\n\n
\n\n# Module: haste_tf\n\n\n\nHaste: a fast, simple, and open RNN library.\n\n\n\n## Classes\n\n[`class GRU`](./haste_tf/GRU.md): Gated Recurrent Unit layer.\n\n[`class GRUCell`](./haste_tf/GRUCell.md): A GRU cell that's compatible with the Haste GRU layer.\n\n[`class IndRNN`](./haste_tf/IndRNN.md): Independently Recurrent Neural Network layer.\n\n[`class LSTM`](./haste_tf/LSTM.md): Long Short-Term Memory layer.\n\n[`class LayerNorm`](./haste_tf/LayerNorm.md): Layer normalization layer.\n\n[`class LayerNormGRU`](./haste_tf/LayerNormGRU.md): Layer Normalized Gated Recurrent Unit layer.\n\n[`class LayerNormGRUCell`](./haste_tf/LayerNormGRUCell.md): A GRU cell that's compatible with the Haste LayerNormGRU layer.\n\n[`class LayerNormLSTM`](./haste_tf/LayerNormLSTM.md): Layer Normalized Long Short-Term Memory layer.\n\n[`class LayerNormLSTMCell`](./haste_tf/LayerNormLSTMCell.md): An LSTM cell that's compatible with the Haste LayerNormLSTM layer.\n\n[`class ZoneoutWrapper`](./haste_tf/ZoneoutWrapper.md): An LSTM/GRU cell wrapper that applies zoneout to the inner cell's hidden state.\n\n"
},
{
"path": "examples/device_ptr.h",
"content": "// Copyright 2020 LMNT, Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n// ==============================================================================\n\n#pragma once\n\n#include \n#include \n#include \n\ntemplate\nstruct device_ptr {\n static constexpr size_t ElemSize = sizeof(typename T::Scalar);\n\n static device_ptr NewByteSized(size_t bytes) {\n return device_ptr((bytes + ElemSize - 1) / ElemSize);\n }\n\n explicit device_ptr(size_t size_)\n : data(nullptr), size(size_) {\n void* tmp;\n cudaMalloc(&tmp, size * ElemSize);\n data = static_cast(tmp);\n }\n\n explicit device_ptr(const T& elem)\n : data(nullptr), size(elem.size()) {\n void* tmp;\n cudaMalloc(&tmp, size * ElemSize);\n data = static_cast(tmp);\n ToDevice(elem);\n }\n\n device_ptr(device_ptr![](\"https://lmnt.com/assets/haste-logo_social_media.png\"){kind=logo}
\n![](\"https://lmnt.com/assets/haste/benchmark/report_n=16_c=128.png\")
![](\"https://lmnt.com/assets/haste/benchmark/report_n=32_c=256.png\")
![](\"https://lmnt.com/assets/haste/benchmark/report_n=64_c=128.png\")
![](\"https://lmnt.com/assets/haste/benchmark/report_n=128_c=256.png\")