\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n-->\r\n
\r\n"
},
{
"path": "SOURCE/api_docs/index.md",
"content": "# Overview \r\n\r\nTensorFlow has APIs available in several languages both for constructing and\r\nexecuting a TensorFlow graph. The Python API is at present the most complete\r\nand the easiest to use, but the C++ API may offer some performance advantages\r\nin graph execution, and supports deployment to small devices such as Android.\r\n\r\nOver time, we hope that the TensorFlow community will develop front ends for\r\nlanguages like Go, Java, JavaScript, Lua R, and perhaps others. With\r\n[SWIG](http://swig.org), it's relatively easy to develop a TensorFlow interface\r\nfor your favorite language.\r\n\r\nNote: Many practical aspects of usage are covered in the Mechanics tab, and\r\nsome additional documentation not specific to any particular language API is\r\navailable in the Resources tab.\r\n\r\n"
},
{
"path": "SOURCE/api_docs/python/array_ops.md",
"content": "\r\n\r\n# Tensor Transformations \r\n\r\nNote: Functions taking `Tensor` arguments can also take anything accepted by\r\n[`tf.convert_to_tensor`](../../api_docs/python/framework.md#convert_to_tensor).\r\n\r\n\r\n## Contents\r\n### [Tensor Transformations](#AUTOGENERATED-tensor-transformations)\r\n* [Casting](#AUTOGENERATED-casting)\r\n * [`tf.string_to_number(string_tensor, out_type=None, name=None)`](#string_to_number)\r\n * [`tf.to_double(x, name='ToDouble')`](#to_double)\r\n * [`tf.to_float(x, name='ToFloat')`](#to_float)\r\n * [`tf.to_bfloat16(x, name='ToBFloat16')`](#to_bfloat16)\r\n * [`tf.to_int32(x, name='ToInt32')`](#to_int32)\r\n * [`tf.to_int64(x, name='ToInt64')`](#to_int64)\r\n * [`tf.cast(x, dtype, name=None)`](#cast)\r\n* [Shapes and Shaping](#AUTOGENERATED-shapes-and-shaping)\r\n * [`tf.shape(input, name=None)`](#shape)\r\n * [`tf.size(input, name=None)`](#size)\r\n * [`tf.rank(input, name=None)`](#rank)\r\n * [`tf.reshape(tensor, shape, name=None)`](#reshape)\r\n * [`tf.squeeze(input, squeeze_dims=None, name=None)`](#squeeze)\r\n * [`tf.expand_dims(input, dim, name=None)`](#expand_dims)\r\n* [Slicing and Joining](#AUTOGENERATED-slicing-and-joining)\r\n * [`tf.slice(input_, begin, size, name=None)`](#slice)\r\n * [`tf.split(split_dim, num_split, value, name='split')`](#split)\r\n * [`tf.tile(input, multiples, name=None)`](#tile)\r\n * [`tf.pad(input, paddings, name=None)`](#pad)\r\n * [`tf.concat(concat_dim, values, name='concat')`](#concat)\r\n * [`tf.pack(values, name='pack')`](#pack)\r\n * [`tf.unpack(value, num=None, name='unpack')`](#unpack)\r\n * [`tf.reverse_sequence(input, seq_lengths, seq_dim, name=None)`](#reverse_sequence)\r\n * [`tf.reverse(tensor, dims, name=None)`](#reverse)\r\n * [`tf.transpose(a, perm=None, name='transpose')`](#transpose)\r\n * [`tf.gather(params, indices, name=None)`](#gather)\r\n * [`tf.dynamic_partition(data, partitions, num_partitions, name=None)`](#dynamic_partition)\r\n * [`tf.dynamic_stitch(indices, data, name=None)`](#dynamic_stitch)\r\n\r\n\r\n\r\n\r\n## Casting \r\n\r\nTensorFlow provides several operations that you can use to cast tensor data\r\ntypes in your graph.\r\n\r\n- - -\r\n\r\n### `tf.string_to_number(string_tensor, out_type=None, name=None)` \r\n\r\nConverts each string in the input Tensor to the specified numeric type.\r\n\r\n(Note that int32 overflow results in an error while float overflow\r\nresults in a rounded value.)\r\n\r\n##### Args: \r\n\r\n\r\n* `string_tensor`: A `Tensor` of type `string`.\r\n* `out_type`: An optional `tf.DType` from: `tf.float32, tf.int32`. Defaults to `tf.float32`.\r\n The numeric type to interpret each string in string_tensor as.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `out_type`.\r\n A Tensor of the same shape as the input string_tensor.\r\n\r\n\r\n- - -\r\n\r\n### `tf.to_double(x, name='ToDouble')` \r\n\r\nCasts a tensor to type `float64`.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` or `SparseTensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` or `SparseTensor` with same shape as `x` with type `float64`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `x` cannot be cast to the `float64`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.to_float(x, name='ToFloat')` \r\n\r\nCasts a tensor to type `float32`.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` or `SparseTensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` or `SparseTensor` with same shape as `x` with type `float32`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `x` cannot be cast to the `float32`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.to_bfloat16(x, name='ToBFloat16')` \r\n\r\nCasts a tensor to type `bfloat16`.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` or `SparseTensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` or `SparseTensor` with same shape as `x` with type `bfloat16`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `x` cannot be cast to the `bfloat16`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.to_int32(x, name='ToInt32')` \r\n\r\nCasts a tensor to type `int32`.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` or `SparseTensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` or `SparseTensor` with same shape as `x` with type `int32`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `x` cannot be cast to the `int32`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.to_int64(x, name='ToInt64')` \r\n\r\nCasts a tensor to type `int64`.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` or `SparseTensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` or `SparseTensor` with same shape as `x` with type `int64`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `x` cannot be cast to the `int64`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.cast(x, dtype, name=None)` \r\n\r\nCasts a tensor to a new type.\r\n\r\nThe operation casts `x` (in case of `Tensor`) or `x.values`\r\n(in case of `SparseTensor`) to `dtype`.\r\n\r\nFor example:\r\n\r\n```python\r\n# tensor `a` is [1.8, 2.2], dtype=tf.float\r\ntf.cast(a, tf.int32) ==> [1, 2] # dtype=tf.int32\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` or `SparseTensor`.\r\n* `dtype`: The destination type.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` or `SparseTensor` with same shape as `x`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `x` cannot be cast to the `dtype`.\r\n\r\n\r\n\r\n## Shapes and Shaping \r\n\r\nTensorFlow provides several operations that you can use to determine the shape\r\nof a tensor and change the shape of a tensor.\r\n\r\n- - -\r\n\r\n### `tf.shape(input, name=None)` \r\n\r\nReturns the shape of a tensor.\r\n\r\nThis operation returns a 1-D integer tensor representing the shape of `input`.\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# 't' is [[[1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]]]\r\nshape(t) ==> [2, 2, 3]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `int32`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.size(input, name=None)` \r\n\r\nReturns the size of a tensor.\r\n\r\nThis operation returns an integer representing the number of elements in\r\n`input`.\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# 't' is [[[1, 1,, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]]]]\r\nsize(t) ==> 12\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `int32`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.rank(input, name=None)` \r\n\r\nReturns the rank of a tensor.\r\n\r\nThis operation returns an integer representing the rank of `input`.\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# 't' is [[[1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]]]\r\n# shape of tensor 't' is [2, 2, 3]\r\nrank(t) ==> 3\r\n```\r\n\r\n**Note**: The rank of a tensor is not the same as the rank of a matrix. The rank\r\nof a tensor is the number of indices required to uniquely select each element\r\nof the tensor. Rank is also known as \"order\", \"degree\", or \"ndims.\"\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `int32`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.reshape(tensor, shape, name=None)` \r\n\r\nReshapes a tensor.\r\n\r\nGiven `tensor`, this operation returns a tensor that has the same values\r\nas `tensor` with shape `shape`.\r\n\r\nIf `shape` is the special value `[-1]`, then `tensor` is flattened and the\r\noperation outputs a 1-D tensor with all elements of `tensor`.\r\n\r\nIf `shape` is 1-D or higher, then the operation returns a tensor with shape\r\n`shape` filled with the values of `tensor`. In this case, the number of elements\r\nimplied by `shape` must be the same as the number of elements in `tensor`.\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# tensor 't' is [1, 2, 3, 4, 5, 6, 7, 8, 9]\r\n# tensor 't' has shape [9]\r\nreshape(t, [3, 3]) ==> [[1, 2, 3]\r\n [4, 5, 6]\r\n [7, 8, 9]]\r\n\r\n# tensor 't' is [[[1, 1], [2, 2]]\r\n# [[3, 3], [4, 4]]]\r\n# tensor 't' has shape [2, 2]\r\nreshape(t, [2, 4]) ==> [[1, 1, 2, 2]\r\n [3, 3, 4, 4]]\r\n\r\n# tensor 't' is [[[1, 1, 1],\r\n# [2, 2, 2]],\r\n# [[3, 3, 3],\r\n# [4, 4, 4]],\r\n# [[5, 5, 5],\r\n# [6, 6, 6]]]\r\n# tensor 't' has shape [3, 2, 3]\r\n# pass '[-1]' to flatten 't'\r\nreshape(t, [-1]) ==> [1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5, 6, 6, 6]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `tensor`: A `Tensor`.\r\n* `shape`: A `Tensor` of type `int32`. Defines the shape of the output tensor.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `tensor`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.squeeze(input, squeeze_dims=None, name=None)` \r\n\r\nRemoves dimensions of size 1 from the shape of a tensor.\r\n\r\nGiven a tensor `input`, this operation returns a tensor of the same type with\r\nall dimensions of size 1 removed. If you don't want to remove all size 1\r\ndimensions, you can remove specific size 1 dimensions by specifying\r\n`squeeze_dims`.\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# 't' is a tensor of shape [1, 2, 1, 3, 1, 1]\r\nshape(squeeze(t)) ==> [2, 3]\r\n```\r\n\r\nOr, to remove specific size 1 dimensions:\r\n\r\n```prettyprint\r\n# 't' is a tensor of shape [1, 2, 1, 3, 1, 1]\r\nshape(squeeze(t, [2, 4])) ==> [1, 2, 3, 1]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. The `input` to squeeze.\r\n* `squeeze_dims`: An optional list of `ints`. Defaults to `[]`.\r\n If specified, only squeezes the dimensions listed. The dimension\r\n index starts at 0. It is an error to squeeze a dimension that is not 1.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`.\r\n Contains the same data as `input`, but has one or more dimensions of\r\n size 1 removed.\r\n\r\n\r\n- - -\r\n\r\n### `tf.expand_dims(input, dim, name=None)` \r\n\r\nInserts a dimension of 1 into a tensor's shape.\r\n\r\nGiven a tensor `input`, this operation inserts a dimension of 1 at the\r\ndimension index `dim` of `input`'s shape. The dimension index `dim` starts at\r\nzero; if you specify a negative number for `dim` it is counted backward from\r\nthe end.\r\n\r\nThis operation is useful if you want to add a batch dimension to a single\r\nelement. For example, if you have a single image of shape `[height, width,\r\nchannels]`, you can make it a batch of 1 image with `expand_dims(image, 0)`,\r\nwhich will make the shape `[1, height, width, channels]`.\r\n\r\nOther examples:\r\n\r\n```prettyprint\r\n# 't' is a tensor of shape [2]\r\nshape(expand_dims(t, 0)) ==> [1, 2]\r\nshape(expand_dims(t, 1)) ==> [2, 1]\r\nshape(expand_dims(t, -1)) ==> [2, 1]\r\n\r\n# 't2' is a tensor of shape [2, 3, 5]\r\nshape(expand_dims(t2, 0)) ==> [1, 2, 3, 5]\r\nshape(expand_dims(t2, 2)) ==> [2, 3, 1, 5]\r\nshape(expand_dims(t2, 3)) ==> [2, 3, 5, 1]\r\n```\r\n\r\nThis operation requires that:\r\n\r\n`-1-input.dims() <= dim <= input.dims()`\r\n\r\nThis operation is related to `squeeze()`, which removes dimensions of\r\nsize 1.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`.\r\n* `dim`: A `Tensor` of type `int32`.\r\n 0-D (scalar). Specifies the dimension index at which to\r\n expand the shape of `input`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`.\r\n Contains the same data as `input`, but its shape has an additional\r\n dimension of size 1 added.\r\n\r\n\r\n\r\n## Slicing and Joining \r\n\r\nTensorFlow provides several operations to slice or extract parts of a tensor,\r\nor join multiple tensors together.\r\n\r\n- - -\r\n\r\n### `tf.slice(input_, begin, size, name=None)` \r\n\r\nExtracts a slice from a tensor.\r\n\r\nThis operation extracts a slice of size `size` from a tensor `input` starting\r\nat the location specified by `begin`. The slice `size` is represented as a\r\ntensor shape, where `size[i]` is the number of elements of the 'i'th dimension\r\nof `input` that you want to slice. The starting location (`begin`) for the\r\nslice is represented as an offset in each dimension of `input`. In other\r\nwords, `begin[i]` is the offset into the 'i'th dimension of `input` that you\r\nwant to slice from.\r\n\r\n`begin` is zero-based; `size` is one-based. If `size[i]` is -1,\r\nall remaining elements in dimension i are included in the\r\nslice. In other words, this is equivalent to setting:\r\n\r\n`size[i] = input.dim_size(i) - begin[i]`\r\n\r\nThis operation requires that:\r\n\r\n`0 <= begin[i] <= begin[i] + size[i] <= Di for i in [0, n]`\r\n\r\nFor example:\r\n\r\n```\r\n# 'input' is [[[1, 1, 1], [2, 2, 2]],\r\n# [[3, 3, 3], [4, 4, 4]],\r\n# [[5, 5, 5], [6, 6, 6]]]\r\ntf.slice(input, [1, 0, 0], [1, 1, 3]) ==> [[[3, 3, 3]]]\r\ntf.slice(input, [1, 0, 0], [1, 2, 3]) ==> [[[3, 3, 3],\r\n [4, 4, 4]]]\r\ntf.slice(input, [1, 0, 0], [2, 1, 3]) ==> [[[3, 3, 3]],\r\n [[5, 5, 5]]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `input_`: A `Tensor`.\r\n* `begin`: An `int32` or `int64` `Tensor`.\r\n* `size`: An `int32` or `int64` `Tensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` the same type as `input`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.split(split_dim, num_split, value, name='split')` \r\n\r\nSplits a tensor into `num_split` tensors along one dimension.\r\n\r\nSplits `value` along dimension `split_dim` into `num_split` smaller tensors.\r\nRequires that `num_split` evenly divide `value.shape[split_dim]`.\r\n\r\nFor example:\r\n\r\n```python\r\n# 'value' is a tensor with shape [5, 30]\r\n# Split 'value' into 3 tensors along dimension 1\r\nsplit0, split1, split2 = tf.split(1, 3, value)\r\ntf.shape(split0) ==> [5, 10]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `split_dim`: A 0-D `int32` `Tensor`. The dimension along which to split.\r\n Must be in the range `[0, rank(value))`.\r\n* `num_split`: A 0-D `int32` `Tensor`. The number of ways to split.\r\n* `value`: The `Tensor` to split.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n `num_split` `Tensor` objects resulting from splitting `value`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.tile(input, multiples, name=None)` \r\n\r\nConstructs a tensor by tiling a given tensor.\r\n\r\nThis operation creates a new tensor by replicating `input` `multiples` times.\r\nThe output tensor's i'th dimension has `input.dims(i) * multiples[i]` elements,\r\nand the values of `input` are replicated `multiples[i]` times along the 'i'th\r\ndimension. For example, tiling `[a b c d]` by `[2]` produces\r\n`[a b c d a b c d]`.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. 1-D or higher.\r\n* `multiples`: A `Tensor` of type `int32`.\r\n 1-D. Length must be the same as the number of dimensions in `input`\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.pad(input, paddings, name=None)` \r\n\r\nPads a tensor with zeros.\r\n\r\nThis operation pads a `input` with zeros according to the `paddings` you\r\nspecify. `paddings` is an integer tensor with shape `[Dn, 2]`, where n is the\r\nrank of `input`. For each dimension D of `input`, `paddings[D, 0]` indicates\r\nhow many zeros to add before the contents of `input` in that dimension, and\r\n`paddings[D, 1]` indicates how many zeros to add after the contents of `input`\r\nin that dimension.\r\n\r\nThe padded size of each dimension D of the output is:\r\n\r\n`paddings(D, 0) + input.dim_size(D) + paddings(D, 1)`\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# 't' is [[1, 1], [2, 2]]\r\n# 'paddings' is [[1, 1]], [2, 2]]\r\n# rank of 't' is 2\r\npad(t, paddings) ==> [[0, 0, 0, 0, 0]\r\n [0, 0, 0, 0, 0]\r\n [0, 1, 1, 0, 0]\r\n [[0, 2, 2, 0, 0]\r\n [0, 0, 0, 0, 0]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`.\r\n* `paddings`: A `Tensor` of type `int32`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.concat(concat_dim, values, name='concat')` \r\n\r\nConcatenates tensors along one dimension.\r\n\r\nConcatenates the list of tensors `values` along dimension `concat_dim`. If\r\n`values[i].shape = [D0, D1, ... Dconcat_dim(i), ...Dn]`, the concatenated\r\nresult has shape\r\n\r\n [D0, D1, ... Rconcat_dim, ...Dn]\r\n\r\nwhere\r\n\r\n Rconcat_dim = sum(Dconcat_dim(i))\r\n\r\nThat is, the data from the input tensors is joined along the `concat_dim`\r\ndimension.\r\n\r\nThe number of dimensions of the input tensors must match, and all dimensions\r\nexcept `concat_dim` must be equal.\r\n\r\nFor example:\r\n\r\n```python\r\nt1 = [[1, 2, 3], [4, 5, 6]]\r\nt2 = [[7, 8, 9], [10, 11, 12]]\r\ntf.concat(0, [t1, t2]) ==> [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]\r\ntf.concat(1, [t1, t2]) ==> [[1, 2, 3, 7, 8, 9], [4, 5, 6, 10, 11, 12]]\r\n\r\n# tensor t3 with shape [2, 3]\r\n# tensor t4 with shape [2, 3]\r\ntf.shape(tf.concat(0, [t3, t4])) ==> [4, 3]\r\ntf.shape(tf.concat(1, [t3, t4])) ==> [2, 6]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `concat_dim`: 0-D `int32` `Tensor`. Dimension along which to concatenate.\r\n* `values`: A list of `Tensor` objects or a single `Tensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` resulting from concatenation of the input tensors.\r\n\r\n\r\n- - -\r\n\r\n### `tf.pack(values, name='pack')` \r\n\r\nPacks a list of rank-`R` tensors into one rank-`(R+1)` tensor.\r\n\r\nPacks tensors in `values` into a tensor with rank one higher than each tensor\r\nin `values` and shape `[len(values)] + values[0].shape`. The output satisfies\r\n`output[i, ...] = values[i][...]`.\r\n\r\nThis is the opposite of unpack. The numpy equivalent is\r\n\r\n tf.pack([x, y, z]) = np.asarray([x, y, z])\r\n\r\n##### Args: \r\n\r\n\r\n* `values`: A list of `Tensor` objects with the same shape and type.\r\n* `name`: A name for this operation (optional).\r\n\r\n##### Returns: \r\n\r\n\r\n* `output`: A packed `Tensor` with the same type as `values`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.unpack(value, num=None, name='unpack')` \r\n\r\nUnpacks the outer dimension of a rank-`R` tensor into rank-`(R-1)` tensors.\r\n\r\nUnpacks `num` tensors from `value` along the first dimension.\r\nIf `num` is not specified (the default), it is inferred from `value`'s shape.\r\nIf `value.shape[0]` is not known, `ValueError` is raised.\r\n\r\nThe ith tensor in `output` is the slice `value[i, ...]`. Each tensor in\r\n`output` has shape `value.shape[1:]`.\r\n\r\nThis is the opposite of pack. The numpy equivalent is\r\n\r\n tf.unpack(x, n) = list(x)\r\n\r\n##### Args: \r\n\r\n\r\n* `value`: A rank `R > 0` `Tensor` to be unpacked.\r\n* `num`: An `int`. The first dimension of value. Automatically inferred if\r\n `None` (the default).\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The list of `Tensor` objects unpacked from `value`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `num` is unspecified and cannot be inferred.\r\n\r\n\r\n- - -\r\n\r\n### `tf.reverse_sequence(input, seq_lengths, seq_dim, name=None)` \r\n\r\nReverses variable length slices in dimension `seq_dim`.\r\n\r\nThis op first slices `input` along the first dimension, and for each slice `i`,\r\nreverses the first `seq_lengths[i]` elements along the dimension `seq_dim`.\r\n\r\nThe elements of `seq_lengths` must obey `seq_lengths[i] < input.dims[seq_dim]`,\r\nand `seq_lengths` must be a vector of length `input.dims(0)`.\r\n\r\nThe output slice `i` along dimension 0 is then given by input slice `i`, with\r\nthe first `seq_lengths[i]` slices along dimension `seq_dim` reversed.\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# Given this:\r\nseq_dim = 1\r\ninput.dims = (4, ...)\r\nseq_lengths = [7, 2, 3, 5]\r\n\r\n# then slices of input are reversed on seq_dim, but only up to seq_lengths:\r\noutput[0, 0:7, :, ...] = input[0, 7:0:-1, :, ...]\r\noutput[1, 0:2, :, ...] = input[1, 2:0:-1, :, ...]\r\noutput[2, 0:3, :, ...] = input[2, 3:0:-1, :, ...]\r\noutput[3, 0:5, :, ...] = input[3, 5:0:-1, :, ...]\r\n\r\n# while entries past seq_lens are copied through:\r\noutput[0, 7:, :, ...] = input[0, 7:, :, ...]\r\noutput[1, 2:, :, ...] = input[1, 2:, :, ...]\r\noutput[2, 3:, :, ...] = input[2, 3:, :, ...]\r\noutput[3, 2:, :, ...] = input[3, 2:, :, ...]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. The input to reverse.\r\n* `seq_lengths`: A `Tensor` of type `int64`.\r\n 1-D with length `input.dims(0)` and\r\n `max(seq_lengths) < input.dims(seq_dim)`\r\n* `seq_dim`: An `int`. The dimension which is partially reversed.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`.\r\n The partially reversed input. It has the same shape as `input`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.reverse(tensor, dims, name=None)` \r\n\r\nReverses specific dimensions of a tensor.\r\n\r\nGiven a `tensor`, and a `bool` tensor `dims` representing the dimensions\r\nof `tensor`, this operation reverses each dimension i of `tensor` where\r\n`dims[i]` is `True`.\r\n\r\n`tensor` can have up to 8 dimensions. The number of dimensions\r\nof `tensor` must equal the number of elements in `dims`. In other words:\r\n\r\n`rank(tensor) = size(dims)`\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# tensor 't' is [[[[ 0, 1, 2, 3],\r\n# [ 4, 5, 6, 7],\r\n# [ 8, 9, 10, 11]],\r\n# [[12, 13, 14, 15],\r\n# [16, 17, 18, 19],\r\n# [20, 21, 22, 23]]]]\r\n# tensor 't' shape is [1, 2, 3, 4]\r\n\r\n# 'dims' is [False, False, False, True]\r\nreverse(t, dims) ==> [[[[ 3, 2, 1, 0],\r\n [ 7, 6, 5, 4],\r\n [ 11, 10, 9, 8]],\r\n [[15, 14, 13, 12],\r\n [19, 18, 17, 16],\r\n [23, 22, 21, 20]]]]\r\n\r\n# 'dims' is [False, True, False, False]\r\nreverse(t, dims) ==> [[[[12, 13, 14, 15],\r\n [16, 17, 18, 19],\r\n [20, 21, 22, 23]\r\n [[ 0, 1, 2, 3],\r\n [ 4, 5, 6, 7],\r\n [ 8, 9, 10, 11]]]]\r\n\r\n# 'dims' is [False, False, True, False]\r\nreverse(t, dims) ==> [[[[8, 9, 10, 11],\r\n [4, 5, 6, 7],\r\n [0, 1, 2, 3]]\r\n [[20, 21, 22, 23],\r\n [16, 17, 18, 19],\r\n [12, 13, 14, 15]]]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `tensor`: A `Tensor`. Must be one of the following types: `uint8`, `int8`, `int32`, `bool`, `float32`, `float64`.\r\n Up to 8-D.\r\n* `dims`: A `Tensor` of type `bool`. 1-D. The dimensions to reverse.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `tensor`. The same shape as `tensor`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.transpose(a, perm=None, name='transpose')` \r\n\r\nTransposes `a`. Permutes the dimensions according to `perm`.\r\n\r\nThe returned tensor's dimension i will correspond to the input dimension\r\n`perm[i]`. If `perm` is not given, it is set to (n-1...0), where n is\r\nthe rank of the input tensor. Hence by default, this operation performs a\r\nregular matrix transpose on 2-D input Tensors.\r\n\r\nFor example:\r\n\r\n```python\r\n# 'x' is [[1 2 3]\r\n# [4 5 6]]\r\ntf.transpose(x) ==> [[1 4]\r\n [2 5]\r\n [3 6]]\r\n\r\n# Equivalently\r\ntf.transpose(x perm=[0, 1]) ==> [[1 4]\r\n [2 5]\r\n [3 6]]\r\n\r\n# 'perm' is more useful for n-dimensional tensors, for n > 2\r\n# 'x' is [[[1 2 3]\r\n# [4 5 6]]\r\n# [[7 8 9]\r\n# [10 11 12]]]\r\n# Take the transpose of the matrices in dimension-0\r\ntf.transpose(b, perm=[0, 2, 1]) ==> [[[1 4]\r\n [2 5]\r\n [3 6]]\r\n\r\n [[7 10]\r\n [8 11]\r\n [9 12]]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `a`: A `Tensor`.\r\n* `perm`: A permutation of the dimensions of `a`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A transposed `Tensor`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.gather(params, indices, name=None)` \r\n\r\nGather slices from `params` according to `indices`.\r\n\r\n`indices` must be an integer tensor of any dimension (usually 0-D or 1-D).\r\nProduces an output tensor with shape `indices.shape + params.shape[1:]` where:\r\n\r\n # Scalar indices\r\n output[:, ..., :] = params[indices, :, ... :]\r\n\r\n # Vector indices\r\n output[i, :, ..., :] = params[indices[i], :, ... :]\r\n\r\n # Higher rank indices\r\n output[i, ..., j, :, ... :] = params[indices[i, ..., j], :, ..., :]\r\n\r\nIf `indices` is a permutation and `len(indices) == params.shape[0]` then\r\nthis operation will permute `params` accordingly.\r\n\r\n\r\n![](\"../images/Gather.png\")
\r\n
\r\n\r\n##### Args: \r\n\r\n\r\n* `params`: A `Tensor`.\r\n* `indices`: A `Tensor`. Must be one of the following types: `int32`, `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `params`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.dynamic_partition(data, partitions, num_partitions, name=None)` \r\n\r\nPartitions `data` into `num_partitions` tensors using indices from `partitions`.\r\n\r\nFor each index tuple `js` of size `partitions.ndim`, the slice `data[js, ...]`\r\nbecomes part of `outputs[partitions[js]]`. The slices with `partitions[js] = i`\r\nare placed in `outputs[i]` in lexicographic order of `js`, and the first\r\ndimension of `outputs[i]` is the number of entries in `partitions` equal to `i`.\r\nIn detail,\r\n\r\n outputs[i].shape = [sum(partitions == i)] + data.shape[partitions.ndim:]\r\n\r\n outputs[i] = pack([data[js, ...] for js if partitions[js] == i])\r\n\r\n`data.shape` must start with `partitions.shape`.\r\n\r\nFor example:\r\n\r\n # Scalar partitions\r\n partitions = 1\r\n num_partitions = 2\r\n data = [10, 20]\r\n outputs[0] = [] # Empty with shape [0, 2]\r\n outputs[1] = [[10, 20]]\r\n\r\n # Vector partitions\r\n partitions = [0, 0, 1, 1, 0]\r\n num_partitions = 2\r\n data = [10, 20, 30, 40, 50]\r\n outputs[0] = [10, 20, 50]\r\n outputs[1] = [30, 40]\r\n\r\n\r\n\r\n![](\"../images/DynamicPartition.png\")
\r\n
\r\n\r\n##### Args: \r\n\r\n\r\n* `data`: A `Tensor`.\r\n* `partitions`: A `Tensor` of type `int32`.\r\n Any shape. Indices in the range `[0, num_partitions)`.\r\n* `num_partitions`: An `int` that is `>= 1`.\r\n The number of partitions to output.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A list of `num_partitions` `Tensor` objects of the same type as data.\r\n\r\n\r\n- - -\r\n\r\n### `tf.dynamic_stitch(indices, data, name=None)` \r\n\r\nInterleave the values from the `data` tensors into a single tensor.\r\n\r\nBuilds a merged tensor such that\r\n\r\n merged[indices[m][i, ..., j], ...] = data[m][i, ..., j, ...]\r\n\r\nFor example, if each `indices[m]` is scalar or vector, we have\r\n\r\n # Scalar indices\r\n merged[indices[m], ...] = data[m][...]\r\n\r\n # Vector indices\r\n merged[indices[m][i], ...] = data[m][i, ...]\r\n\r\nEach `data[i].shape` must start with the corresponding `indices[i].shape`,\r\nand the rest of `data[i].shape` must be constant w.r.t. `i`. That is, we\r\nmust have `data[i].shape = indices[i].shape + constant`. In terms of this\r\n`constant`, the output shape is\r\n\r\n merged.shape = [max(indices)] + constant\r\n\r\nValues are merged in order, so if an index appears in both `indices[m][i]` and\r\n`indices[n][j]` for `(m,i) < (n,j)` the slice `data[n][j]` will appear in the\r\nmerged result.\r\n\r\nFor example:\r\n\r\n indices[0] = 6\r\n indices[1] = [4, 1]\r\n indices[2] = [[5, 2], [0, 3]]\r\n data[0] = [61, 62]\r\n data[1] = [[41, 42], [11, 12]]\r\n data[2] = [[[51, 52], [21, 22]], [[1, 2], [31, 32]]]\r\n merged = [[1, 2], [11, 12], [21, 22], [31, 32], [41, 42],\r\n [51, 52], [61, 62]]\r\n\r\n\r\n\r\n![](\"../images/DynamicStitch.png\")
\r\n
\r\n\r\n##### Args: \r\n\r\n\r\n* `indices`: A list of at least 2 `Tensor` objects of type `int32`.\r\n* `data`: A list with the same number of `Tensor` objects as `indices` of `Tensor` objects of the same type.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `data`.\r\n\r\n\r\n"
},
{
"path": "SOURCE/api_docs/python/client.md",
"content": "\r\n\r\n# Running Graphs \r\n\r\n## Contents\r\n### [Running Graphs](#AUTOGENERATED-running-graphs)\r\n* [Session management](#AUTOGENERATED-session-management)\r\n * [`class tf.Session`](#Session)\r\n * [`class tf.InteractiveSession`](#InteractiveSession)\r\n * [`tf.get_default_session()`](#get_default_session)\r\n* [Error classes](#AUTOGENERATED-error-classes)\r\n * [`class tf.OpError`](#OpError)\r\n * [`class tf.errors.CancelledError`](#CancelledError)\r\n * [`class tf.errors.UnknownError`](#UnknownError)\r\n * [`class tf.errors.InvalidArgumentError`](#InvalidArgumentError)\r\n * [`class tf.errors.DeadlineExceededError`](#DeadlineExceededError)\r\n * [`class tf.errors.NotFoundError`](#NotFoundError)\r\n * [`class tf.errors.AlreadyExistsError`](#AlreadyExistsError)\r\n * [`class tf.errors.PermissionDeniedError`](#PermissionDeniedError)\r\n * [`class tf.errors.UnauthenticatedError`](#UnauthenticatedError)\r\n * [`class tf.errors.ResourceExhaustedError`](#ResourceExhaustedError)\r\n * [`class tf.errors.FailedPreconditionError`](#FailedPreconditionError)\r\n * [`class tf.errors.AbortedError`](#AbortedError)\r\n * [`class tf.errors.OutOfRangeError`](#OutOfRangeError)\r\n * [`class tf.errors.UnimplementedError`](#UnimplementedError)\r\n * [`class tf.errors.InternalError`](#InternalError)\r\n * [`class tf.errors.UnavailableError`](#UnavailableError)\r\n * [`class tf.errors.DataLossError`](#DataLossError)\r\n\r\n\r\n\r\n\r\nThis library contains classes for launching graphs and executing operations.\r\n\r\nThe [basic usage](../../get_started/introduction.md#basic-usage) guide has\r\nexamples of how a graph is launched in a [`tf.Session`](#Session).\r\n\r\n## Session management \r\n\r\n- - -\r\n\r\n### `class tf.Session` \r\n\r\nA class for running TensorFlow operations.\r\n\r\nA `Session` object encapsulates the environment in which `Operation`\r\nobjects are executed, and `Tensor` objects are evaluated. For\r\nexample:\r\n\r\n```python\r\n# Build a graph.\r\na = tf.constant(5.0)\r\nb = tf.constant(6.0)\r\nc = a * b\r\n\r\n# Launch the graph in a session.\r\nsess = tf.Session()\r\n\r\n# Evaluate the tensor `c`.\r\nprint sess.run(c)\r\n```\r\n\r\nA session may own resources, such as\r\n[variables](../../api_docs/python/state_ops.md#Variable), [queues](../../api_docs/python/io_ops.md#QueueBase),\r\nand [readers](../../api_docs/python/io_ops.md#ReaderBase). It is important to release\r\nthese resources when they are no longer required. To do this, either\r\ninvoke the [`close()`](#Session.close) method on the session, or use\r\nthe session as a context manager. The following two examples are\r\nequivalent:\r\n\r\n```python\r\n# Using the `close()` method.\r\nsess = tf.Session()\r\nsess.run(...)\r\nsess.close()\r\n\r\n# Using the context manager.\r\nwith tf.Session() as sess:\r\n sess.run(...)\r\n```\r\n\r\nThe [`ConfigProto`]\r\n(https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/framework/config.proto)\r\nprotocol buffer exposes various configuration options for a\r\nsession. For example, to create a session that uses soft constraints\r\nfor device placement, and log the resulting placement decisions,\r\ncreate a session as follows:\r\n\r\n```python\r\n# Launch the graph in a session that allows soft device placement and\r\n# logs the placement decisions.\r\nsess = tf.Session(config=tf.ConfigProto(allow_soft_placement=True,\r\n log_device_placement=True))\r\n```\r\n\r\n- - -\r\n\r\n#### `tf.Session.__init__(target='', graph=None, config=None)` \r\n\r\nCreates a new TensorFlow session.\r\n\r\nIf no `graph` argument is specified when constructing the session,\r\nthe default graph will be launched in the session. If you are\r\nusing more than one graph (created with `tf.Graph()` in the same\r\nprocess, you will have to use different sessions for each graph,\r\nbut each graph can be used in multiple sessions. In this case, it\r\nis often clearer to pass the graph to be launched explicitly to\r\nthe session constructor.\r\n\r\n##### Args: \r\n\r\n\r\n* `target`: (Optional.) The execution engine to connect to.\r\n Defaults to using an in-process engine. At present, no value\r\n other than the empty string is supported.\r\n* `graph`: (Optional.) The `Graph` to be launched (described above).\r\n* `config`: (Optional.) A [`ConfigProto`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/framework/config.proto)\r\n protocol buffer with configuration options for the session.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Session.run(fetches, feed_dict=None)` \r\n\r\nRuns the operations and evaluates the tensors in `fetches`.\r\n\r\nThis method runs one \"step\" of TensorFlow computation, by\r\nrunning the necessary graph fragment to execute every `Operation`\r\nand evaluate every `Tensor` in `fetches`, substituting the values in\r\n`feed_dict` for the corresponding input values.\r\n\r\nThe `fetches` argument may be a list of graph elements or a single\r\ngraph element, and these determine the return value of this\r\nmethod. A graph element can be one of the following types:\r\n\r\n* If the *i*th element of `fetches` is an\r\n [`Operation`](../../api_docs/python/framework.md#Operation), the *i*th\r\n return value will be `None`.\r\n* If the *i*th element of `fetches` is a\r\n [`Tensor`](../../api_docs/python/framework.md#Tensor), the *i*th return\r\n value will be a numpy ndarray containing the value of that tensor.\r\n* If the *i*th element of `fetches` is a\r\n [`SparseTensor`](../../api_docs/python/sparse_ops.md#SparseTensor),\r\n the *i*th return value will be a\r\n [`SparseTensorValue`](../../api_docs/python/sparse_ops.md#SparseTensorValue)\r\n containing the value of that sparse tensor.\r\n\r\nThe optional `feed_dict` argument allows the caller to override\r\nthe value of tensors in the graph. Each key in `feed_dict` can be\r\none of the following types:\r\n\r\n* If the key is a [`Tensor`](../../api_docs/python/framework.md#Tensor), the\r\n value may be a Python scalar, string, list, or numpy ndarray\r\n that can be converted to the same `dtype` as that\r\n tensor. Additionally, if the key is a\r\n [placeholder](../../api_docs/python/io_ops.md#placeholder), the shape of\r\n the value will be checked for compatibility with the placeholder.\r\n* If the key is a\r\n [`SparseTensor`](../../api_docs/python/sparse_ops.md#SparseTensor),\r\n the value should be a\r\n [`SparseTensorValue`](../../api_docs/python/sparse_ops.md#SparseTensorValue).\r\n\r\n##### Args: \r\n\r\n\r\n* `fetches`: A single graph element, or a list of graph elements\r\n (described above).\r\n* `feed_dict`: A dictionary that maps graph elements to values\r\n (described above).\r\n\r\n##### Returns: \r\n\r\n Either a single value if `fetches` is a single graph element, or\r\n a list of values if `fetches` is a list (described above).\r\n\r\n##### Raises: \r\n\r\n\r\n* `RuntimeError`: If this `Session` is in an invalid state (e.g. has been\r\n closed).\r\n* `TypeError`: If `fetches` or `feed_dict` keys are of an inappropriate type.\r\n* `ValueError`: If `fetches` or `feed_dict` keys are invalid or refer to a\r\n `Tensor` that doesn't exist.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Session.close()` \r\n\r\nCloses this session.\r\n\r\nCalling this method frees all resources associated with the session.\r\n\r\n##### Raises: \r\n\r\n\r\n* `RuntimeError`: If an error occurs while closing the session.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Session.graph` \r\n\r\nThe graph that was launched in this session.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Session.as_default()` \r\n\r\nReturns a context manager that makes this object the default session.\r\n\r\nUse with the `with` keyword to specify that calls to\r\n[`Operation.run()`](../../api_docs/python/framework.md#Operation.run) or\r\n[`Tensor.run()`](../../api_docs/python/framework.md#Tensor.run) should be\r\nexecuted in this session.\r\n\r\n```python\r\nc = tf.constant(..)\r\nsess = tf.Session()\r\n\r\nwith sess.as_default():\r\n assert tf.get_default_session() is sess\r\n print c.eval()\r\n```\r\n\r\nTo get the current default session, use\r\n[`tf.get_default_session()`](#get_default_session).\r\n\r\n\r\n*N.B.* The `as_default` context manager *does not* close the\r\nsession when you exit the context, and you must close the session\r\nexplicitly.\r\n\r\n```python\r\nc = tf.constant(...)\r\nsess = tf.Session()\r\nwith sess.as_default():\r\n print c.eval()\r\n# ...\r\nwith sess.as_default():\r\n print c.eval()\r\n\r\nsess.close()\r\n```\r\n\r\nAlternatively, you can use `with tf.Session():` to create a\r\nsession that is automatically closed on exiting the context,\r\nincluding when an uncaught exception is raised.\r\n\r\n*N.B.* The default graph is a property of the current thread. If you\r\ncreate a new thread, and wish to use the default session in that\r\nthread, you must explicitly add a `with sess.as_default():` in that\r\nthread's function.\r\n\r\n##### Returns: \r\n\r\n A context manager using this session as the default session.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.InteractiveSession` \r\n\r\nA TensorFlow `Session` for use in interactive contexts, such as a shell.\r\n\r\nThe only difference with a regular `Session` is that an `InteractiveSession`\r\ninstalls itself as the default session on construction.\r\nThe methods [`Tensor.eval()`](../../api_docs/python/framework.md#Tensor.eval)\r\nand [`Operation.run()`](../../api_docs/python/framework.md#Operation.run)\r\nwill use that session to run ops.\r\n\r\nThis is convenient in interactive shells and [IPython\r\nnotebooks](http://ipython.org), as it avoids having to pass an explicit\r\n`Session` object to run ops.\r\n\r\nFor example:\r\n\r\n```python\r\nsess = tf.InteractiveSession()\r\na = tf.constant(5.0)\r\nb = tf.constant(6.0)\r\nc = a * b\r\n# We can just use 'c.eval()' without passing 'sess'\r\nprint c.eval()\r\nsess.close()\r\n```\r\n\r\nNote that a regular session installs itself as the default session when it\r\nis created in a `with` statement. The common usage in non-interactive\r\nprograms is to follow that pattern:\r\n\r\n```python\r\na = tf.constant(5.0)\r\nb = tf.constant(6.0)\r\nc = a * b\r\nwith tf.Session():\r\n # We can also use 'c.eval()' here.\r\n print c.eval()\r\n```\r\n\r\n- - -\r\n\r\n#### `tf.InteractiveSession.__init__(target='', graph=None)` \r\n\r\nCreates a new interactive TensorFlow session.\r\n\r\nIf no `graph` argument is specified when constructing the session,\r\nthe default graph will be launched in the session. If you are\r\nusing more than one graph (created with `tf.Graph()` in the same\r\nprocess, you will have to use different sessions for each graph,\r\nbut each graph can be used in multiple sessions. In this case, it\r\nis often clearer to pass the graph to be launched explicitly to\r\nthe session constructor.\r\n\r\n##### Args: \r\n\r\n\r\n* `target`: (Optional.) The execution engine to connect to.\r\n Defaults to using an in-process engine. At present, no value\r\n other than the empty string is supported.\r\n* `graph`: (Optional.) The `Graph` to be launched (described above).\r\n\r\n\r\n- - -\r\n\r\n#### `tf.InteractiveSession.close()` \r\n\r\nCloses an `InteractiveSession`.\r\n\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.get_default_session()` \r\n\r\nReturns the default session for the current thread.\r\n\r\nThe returned `Session` will be the innermost session on which a\r\n`Session` or `Session.as_default()` context has been entered.\r\n\r\n*N.B.* The default session is a property of the current thread. If you\r\ncreate a new thread, and wish to use the default session in that\r\nthread, you must explicitly add a `with sess.as_default():` in that\r\nthread's function.\r\n\r\n##### Returns: \r\n\r\n The default `Session` being used in the current thread.\r\n\r\n\r\n\r\n## Error classes \r\n\r\n- - -\r\n\r\n### `class tf.OpError` \r\n\r\nA generic error that is raised when TensorFlow execution fails.\r\n\r\nWhenever possible, the session will raise a more specific subclass\r\nof `OpError` from the `tf.errors` module.\r\n\r\n- - -\r\n\r\n#### `tf.OpError.op` \r\n\r\nThe operation that failed, if known.\r\n\r\n*N.B.* If the failed op was synthesized at runtime, e.g. a `Send`\r\nor `Recv` op, there will be no corresponding\r\n[`Operation`](../../api_docs/python/framework.md#Operation) object. In that case, this\r\nwill return `None`, and you should instead use the\r\n[`OpError.node_def`](#OpError.node_def) to discover information about the\r\nop.\r\n\r\n##### Returns: \r\n\r\n The `Operation` that failed, or None.\r\n\r\n- - -\r\n\r\n#### `tf.OpError.node_def` \r\n\r\nThe `NodeDef` proto representing the op that failed.\r\n\r\n\r\n#### Other Methods \r\n- - -\r\n\r\n#### `tf.OpError.__init__(node_def, op, message, error_code)` \r\n\r\nCreates a new OpError indicating that a particular op failed.\r\n\r\n##### Args: \r\n\r\n\r\n* `node_def`: The graph_pb2.NodeDef proto representing the op that failed.\r\n* `op`: The ops.Operation that failed, if known; otherwise None.\r\n* `message`: The message string describing the failure.\r\n* `error_code`: The error_codes_pb2.Code describing the error.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.OpError.error_code` \r\n\r\nThe integer error code that describes the error.\r\n\r\n- - -\r\n\r\n#### `tf.OpError.message` \r\n\r\nThe error message that describes the error.\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.CancelledError` \r\n\r\nRaised when an operation or step is cancelled.\r\n\r\nFor example, a long-running operation (e.g.\r\n[`queue.enqueue()`](../../api_docs/python/io_ops.md#QueueBase.enqueue) may be\r\ncancelled by running another operation (e.g.\r\n[`queue.close(cancel_pending_enqueues=True)`](../../api_docs/python/io_ops.md#QueueBase.close),\r\nor by [closing the session](../../api_docs/python/client.md#Session.close).\r\nA step that is running such a long-running operation will fail by raising\r\n`CancelledError`.\r\n\r\n- - -\r\n\r\n#### `tf.errors.CancelledError.__init__(node_def, op, message)` \r\n\r\nCreates a `CancelledError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.UnknownError` \r\n\r\nUnknown error.\r\n\r\nAn example of where this error may be returned is if a Status value\r\nreceived from another address space belongs to an error-space that\r\nis not known to this address space. Also errors raised by APIs that\r\ndo not return enough error information may be converted to this\r\nerror.\r\n\r\n- - -\r\n\r\n#### `tf.errors.UnknownError.__init__(node_def, op, message, error_code=2)` \r\n\r\nCreates an `UnknownError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.InvalidArgumentError` \r\n\r\nRaised when an operation receives an invalid argument.\r\n\r\nThis may occur, for example, if an operation is receives an input\r\ntensor that has an invalid value or shape. For example, the\r\n[`tf.matmul()`](../../api_docs/python/math_ops.md#matmul) op will raise this\r\nerror if it receives an input that is not a matrix, and the\r\n[`tf.reshape()`](../../api_docs/python/array_ops.md#reshape) op will raise\r\nthis error if the new shape does not match the number of elements in the input\r\ntensor.\r\n\r\n- - -\r\n\r\n#### `tf.errors.InvalidArgumentError.__init__(node_def, op, message)` \r\n\r\nCreates an `InvalidArgumentError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.DeadlineExceededError` \r\n\r\nRaised when a deadline expires before an operation could complete.\r\n\r\nThis exception is not currently used.\r\n\r\n- - -\r\n\r\n#### `tf.errors.DeadlineExceededError.__init__(node_def, op, message)` \r\n\r\nCreates a `DeadlineExceededError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.NotFoundError` \r\n\r\nRaised when a requested entity (e.g., a file or directory) was not found.\r\n\r\nFor example, running the\r\n[`tf.WholeFileReader.read()`](../../api_docs/python/io_ops.md#WholeFileReader)\r\noperation could raise `NotFoundError` if it receives the name of a file that\r\ndoes not exist.\r\n\r\n- - -\r\n\r\n#### `tf.errors.NotFoundError.__init__(node_def, op, message)` \r\n\r\nCreates a `NotFoundError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.AlreadyExistsError` \r\n\r\nRaised when an entity that we attempted to create already exists.\r\n\r\nFor example, running an operation that saves a file\r\n(e.g. [`tf.train.Saver.save()`](../../api_docs/python/train.md#Saver.save))\r\ncould potentially raise this exception if an explicit filename for an\r\nexisting file was passed.\r\n\r\n- - -\r\n\r\n#### `tf.errors.AlreadyExistsError.__init__(node_def, op, message)` \r\n\r\nCreates an `AlreadyExistsError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.PermissionDeniedError` \r\n\r\nRaised when the caller does not have permission to run an operation.\r\n\r\nFor example, running the\r\n[`tf.WholeFileReader.read()`](../../api_docs/python/io_ops.md#WholeFileReader)\r\noperation could raise `PermissionDeniedError` if it receives the name of a\r\nfile for which the user does not have the read file permission.\r\n\r\n- - -\r\n\r\n#### `tf.errors.PermissionDeniedError.__init__(node_def, op, message)` \r\n\r\nCreates a `PermissionDeniedError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.UnauthenticatedError` \r\n\r\nThe request does not have valid authentication credentials.\r\n\r\nThis exception is not currently used.\r\n\r\n- - -\r\n\r\n#### `tf.errors.UnauthenticatedError.__init__(node_def, op, message)` \r\n\r\nCreates an `UnauthenticatedError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.ResourceExhaustedError` \r\n\r\nSome resource has been exhausted.\r\n\r\nFor example, this error might be raised if a per-user quota is\r\nexhausted, or perhaps the entire file system is out of space.\r\n\r\n- - -\r\n\r\n#### `tf.errors.ResourceExhaustedError.__init__(node_def, op, message)` \r\n\r\nCreates a `ResourceExhaustedError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.FailedPreconditionError` \r\n\r\nOperation was rejected because the system is not in a state to execute it.\r\n\r\nThis exception is most commonly raised when running an operation\r\nthat reads a [`tf.Variable`](../../api_docs/python/state_ops.md#Variable)\r\nbefore it has been initialized.\r\n\r\n- - -\r\n\r\n#### `tf.errors.FailedPreconditionError.__init__(node_def, op, message)` \r\n\r\nCreates a `FailedPreconditionError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.AbortedError` \r\n\r\nThe operation was aborted, typically due to a concurrent action.\r\n\r\nFor example, running a\r\n[`queue.enqueue()`](../../api_docs/python/io_ops.md#QueueBase.enqueue)\r\noperation may raise `AbortedError` if a\r\n[`queue.close()`](../../api_docs/python/io_ops.md#QueueBase.close) operation\r\npreviously ran.\r\n\r\n- - -\r\n\r\n#### `tf.errors.AbortedError.__init__(node_def, op, message)` \r\n\r\nCreates an `AbortedError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.OutOfRangeError` \r\n\r\nRaised when an operation executed past the valid range.\r\n\r\nThis exception is raised in \"end-of-file\" conditions, such as when a\r\n[`queue.dequeue()`](../../api_docs/python/io_ops.md#QueueBase.dequeue)\r\noperation is blocked on an empty queue, and a\r\n[`queue.close()`](../../api_docs/python/io_ops.md#QueueBase.close)\r\noperation executes.\r\n\r\n- - -\r\n\r\n#### `tf.errors.OutOfRangeError.__init__(node_def, op, message)` \r\n\r\nCreates an `OutOfRangeError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.UnimplementedError` \r\n\r\nRaised when an operation has not been implemented.\r\n\r\nSome operations may raise this error when passed otherwise-valid\r\narguments that it does not currently support. For example, running\r\nthe [`tf.nn.max_pool()`](../../api_docs/python/nn.md#max_pool) operation\r\nwould raise this error if pooling was requested on the batch dimension,\r\nbecause this is not yet supported.\r\n\r\n- - -\r\n\r\n#### `tf.errors.UnimplementedError.__init__(node_def, op, message)` \r\n\r\nCreates an `UnimplementedError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.InternalError` \r\n\r\nRaised when the system experiences an internal error.\r\n\r\nThis exception is raised when some invariant expected by the runtime\r\nhas been broken. Catching this exception is not recommended.\r\n\r\n- - -\r\n\r\n#### `tf.errors.InternalError.__init__(node_def, op, message)` \r\n\r\nCreates an `InternalError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.UnavailableError` \r\n\r\nRaised when the runtime is currently unavailable.\r\n\r\nThis exception is not currently used.\r\n\r\n- - -\r\n\r\n#### `tf.errors.UnavailableError.__init__(node_def, op, message)` \r\n\r\nCreates an `UnavailableError`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.errors.DataLossError` \r\n\r\nRaised when unrecoverable data loss or corruption is encountered.\r\n\r\nFor example, this may be raised by running a\r\n[`tf.WholeFileReader.read()`](../../api_docs/python/io_ops.md#WholeFileReader)\r\noperation, if the file is truncated while it is being read.\r\n\r\n- - -\r\n\r\n#### `tf.errors.DataLossError.__init__(node_def, op, message)` \r\n\r\nCreates a `DataLossError`.\r\n\r\n\r\n\r\n"
},
{
"path": "SOURCE/api_docs/python/constant_op.md",
"content": "\r\n\r\n# Constants, Sequences, and Random Values \r\n\r\nNote: Functions taking `Tensor` arguments can also take anything accepted by\r\n[`tf.convert_to_tensor`](../../api_docs/python/framework.md#convert_to_tensor).\r\n\r\n\r\n## Contents\r\n### [Constants, Sequences, and Random Values](#AUTOGENERATED-constants--sequences--and-random-values)\r\n* [Constant Value Tensors](#AUTOGENERATED-constant-value-tensors)\r\n * [`tf.zeros(shape, dtype=tf.float32, name=None)`](#zeros)\r\n * [`tf.zeros_like(tensor, dtype=None, name=None)`](#zeros_like)\r\n * [`tf.ones(shape, dtype=tf.float32, name=None)`](#ones)\r\n * [`tf.ones_like(tensor, dtype=None, name=None)`](#ones_like)\r\n * [`tf.fill(dims, value, name=None)`](#fill)\r\n * [`tf.constant(value, dtype=None, shape=None, name='Const')`](#constant)\r\n* [Sequences](#AUTOGENERATED-sequences)\r\n * [`tf.linspace(start, stop, num, name=None)`](#linspace)\r\n * [`tf.range(start, limit, delta=1, name='range')`](#range)\r\n* [Random Tensors](#AUTOGENERATED-random-tensors)\r\n * [Examples:](#AUTOGENERATED-examples-)\r\n * [`tf.random_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)`](#random_normal)\r\n * [`tf.truncated_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)`](#truncated_normal)\r\n * [`tf.random_uniform(shape, minval=0.0, maxval=1.0, dtype=tf.float32, seed=None, name=None)`](#random_uniform)\r\n * [`tf.random_shuffle(value, seed=None, name=None)`](#random_shuffle)\r\n * [`tf.set_random_seed(seed)`](#set_random_seed)\r\n\r\n\r\n\r\n\r\n## Constant Value Tensors \r\n\r\nTensorFlow provides several operations that you can use to generate constants.\r\n\r\n- - -\r\n\r\n### `tf.zeros(shape, dtype=tf.float32, name=None)` \r\n\r\nCreates a tensor with all elements set to zero.\r\n\r\nThis operation returns a tensor of type `dtype` with shape `shape` and\r\nall elements set to zero.\r\n\r\nFor example:\r\n\r\n```python\r\ntf.zeros([3, 4], int32) ==> [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `shape`: Either a list of integers, or a 1-D `Tensor` of type `int32`.\r\n* `dtype`: The type of an element in the resulting `Tensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` with all elements set to zero.\r\n\r\n\r\n- - -\r\n\r\n### `tf.zeros_like(tensor, dtype=None, name=None)` \r\n\r\nCreates a tensor with all elements set to zero.\r\n\r\nGiven a single tensor (`tensor`), this operation returns a tensor of the\r\nsame type and shape as `tensor` with all elements set to zero. Optionally,\r\nyou can use `dtype` to specify a new type for the returned tensor.\r\n\r\nFor example:\r\n\r\n```python\r\n# 'tensor' is [[1, 2, 3], [4, 5, 6]]\r\ntf.zeros_like(tensor) ==> [[0, 0, 0], [0, 0, 0]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `tensor`: A `Tensor`.\r\n* `dtype`: A type for the returned `Tensor`. Must be `float32`, `float64`,\r\n `int8`, `int16`, `int32`, `int64`, `uint8`, or `complex64`.\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` with all elements set to zero.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.ones(shape, dtype=tf.float32, name=None)` \r\n\r\nCreates a tensor with all elements set to 1.\r\n\r\nThis operation returns a tensor of type `dtype` with shape `shape` and all\r\nelements set to 1.\r\n\r\nFor example:\r\n\r\n```python\r\ntf.ones([2, 3], int32) ==> [[1, 1, 1], [1, 1, 1]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `shape`: Either a list of integers, or a 1-D `Tensor` of type `int32`.\r\n* `dtype`: The type of an element in the resulting `Tensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` with all elements set to 1.\r\n\r\n\r\n- - -\r\n\r\n### `tf.ones_like(tensor, dtype=None, name=None)` \r\n\r\nCreates a tensor with all elements set to 1.\r\n\r\nGiven a single tensor (`tensor`), this operation returns a tensor of the same\r\ntype and shape as `tensor` with all elements set to 1. Optionally, you can\r\nspecify a new type (`dtype`) for the returned tensor.\r\n\r\nFor example:\r\n\r\n```python\r\n# 'tensor' is [[1, 2, 3], [4, 5, 6]]\r\ntf.ones_like(tensor) ==> [[1, 1, 1], [1, 1, 1]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `tensor`: A `Tensor`.\r\n* `dtype`: A type for the returned `Tensor`. Must be `float32`, `float64`,\r\n `int8`, `int16`, `int32`, `int64`, `uint8`, or `complex64`.\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` with all elements set to 1.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.fill(dims, value, name=None)` \r\n\r\nCreates a tensor filled with a scalar value.\r\n\r\nThis operation creates a tensor of shape `dims` and fills it with `value`.\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# output tensor shape needs to be [2, 3]\r\n# so 'dims' is [2, 3]\r\nfill(dims, 9) ==> [[9, 9, 9]\r\n [9, 9, 9]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `dims`: A `Tensor` of type `int32`.\r\n 1-D. Represents the shape of the output tensor.\r\n* `value`: A `Tensor`. 0-D (scalar). Value to fill the returned tensor.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `value`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.constant(value, dtype=None, shape=None, name='Const')` \r\n\r\nCreates a constant tensor.\r\n\r\n The resulting tensor is populated with values of type `dtype`, as\r\n specified by arguments `value` and (optionally) `shape` (see examples\r\n below).\r\n\r\n The argument `value` can be a constant value, or a list of values of type\r\n `dtype`. If `value` is a list, then the length of the list must be less\r\n than or equal to the number of elements implied by the `shape` argument (if\r\n specified). In the case where the list length is less than the number of\r\n elements specified by `shape`, the last element in the list will be used\r\n to fill the remaining entries.\r\n\r\n The argument `shape` is optional. If present, it specifies the dimensions\r\n of the resulting tensor. If not present, then the tensor is a scalar (0-D)\r\n if `value` is a scalar, or 1-D otherwise.\r\n\r\n If the argument `dtype` is not specified, then the type is inferred from\r\n the type of `value`.\r\n\r\n For example:\r\n\r\n ```python\r\n # Constant 1-D Tensor populated with value list.\r\n tensor = tf.constant([1, 2, 3, 4, 5, 6, 7]) => [1 2 3 4 5 6 7]\r\n\r\n # Constant 2-D tensor populated with scalar value -1.\r\n tensor = tf.constant(-1.0, shape=[2, 3]) => [[-1. -1. -1.]\r\n [-1. -1. -1.]]\r\n ```\r\n\r\n##### Args: \r\n\r\n\r\n* `value`: A constant value (or list) of output type `dtype`.\r\n\r\n\r\n* `dtype`: The type of the elements of the resulting tensor.\r\n\r\n\r\n* `shape`: Optional dimensions of resulting tensor.\r\n\r\n\r\n* `name`: Optional name for the tensor.\r\n\r\n##### Returns: \r\n\r\n A Constant Tensor.\r\n\r\n\r\n\r\n## Sequences \r\n\r\n- - -\r\n\r\n### `tf.linspace(start, stop, num, name=None)` \r\n\r\nGenerates values in an interval.\r\n\r\nA sequence of `num` evenly-spaced values are generated beginning at `start`.\r\nIf `num > 1`, the values in the sequence increase by `stop - start / num - 1`,\r\nso that the last one is exactly `stop`.\r\n\r\nFor example:\r\n\r\n```\r\ntf.linspace(10.0, 12.0, 3, name=\"linspace\") => [ 10.0 11.0 12.0]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `start`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n First entry in the range.\r\n* `stop`: A `Tensor`. Must have the same type as `start`.\r\n Last entry in the range.\r\n* `num`: A `Tensor` of type `int32`. Number of values to generate.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `start`. 1-D. The generated values.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.range(start, limit, delta=1, name='range')` \r\n\r\nCreates a sequence of integers.\r\n\r\nThis operation creates a sequence of integers that begins at `start` and\r\nextends by increments of `delta` up to but not including `limit`.\r\n\r\nFor example:\r\n\r\n```\r\n# 'start' is 3\r\n# 'limit' is 18\r\n# 'delta' is 3\r\ntf.range(start, limit, delta) ==> [3, 6, 9, 12, 15]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `start`: A 0-D (scalar) of type `int32`. First entry in sequence.\r\n* `limit`: A 0-D (scalar) of type `int32`. Upper limit of sequence,\r\n exclusive.\r\n* `delta`: A 0-D `Tensor` (scalar) of type `int32`. Optional. Default is 1.\r\n Number that increments `start`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An 1-D `int32` `Tensor`.\r\n\r\n\r\n\r\n## Random Tensors \r\n\r\nTensorFlow has several ops that create random tensors with different\r\ndistributions. The random ops are stateful, and create new random values each\r\ntime they are evaluated.\r\n\r\nThe `seed` keyword argument in these functions acts in conjunction with\r\nthe graph-level random seed. Changing either the graph-level seed using\r\n[`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed) or the\r\nop-level seed will change the underlying seed of these operations. Setting\r\nneither graph-level nor op-level seed, results in a random seed for all\r\noperations.\r\nSee [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\nfor details on the interaction between operation-level and graph-level random\r\nseeds.\r\n\r\n### Examples: \r\n\r\n```python\r\n# Create a tensor of shape [2, 3] consisting of random normal values, with mean\r\n# -1 and standard deviation 4.\r\nnorm = tf.random_normal([2, 3], mean=-1, stddev=4)\r\n\r\n# Shuffle the first dimension of a tensor\r\nc = tf.constant([[1, 2], [3, 4], [5, 6]])\r\nshuff = tf.random_shuffle(c)\r\n\r\n# Each time we run these ops, different results are generated\r\nsess = tf.Session()\r\nprint sess.run(norm)\r\nprint sess.run(norm)\r\n\r\n# Set an op-level seed to generate repeatable sequences across sessions.\r\nc = tf.constant([[1, 2], [3, 4], [5, 6]])\r\nsess = tf.Session()\r\nnorm = tf.random_normal(c, seed=1234)\r\nprint sess.run(norm)\r\nprint sess.run(norm)\r\n```\r\n\r\nAnother common use of random values is the intialization of variables. Also see\r\nthe [Variables How To](../../how_tos/variables/index.md).\r\n\r\n```python\r\n# Use random uniform values in [0, 1) as the initializer for a variable of shape\r\n# [2, 3]. The default type is float32.\r\nvar = tf.Variable(tf.random_uniform([2, 3]), name=\"var\")\r\ninit = tf.initialize_all_variables()\r\n\r\nsess = tf.Session()\r\nsess.run(init)\r\nprint sess.run(var)\r\n```\r\n\r\n- - -\r\n\r\n### `tf.random_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)` \r\n\r\nOutputs random values from a normal distribution.\r\n\r\n##### Args: \r\n\r\n\r\n* `shape`: A 1-D integer Tensor or Python array. The shape of the output tensor.\r\n* `mean`: A 0-D Tensor or Python value of type `dtype`. The mean of the normal\r\n distribution.\r\n* `stddev`: A 0-D Tensor or Python value of type `dtype`. The standard deviation\r\n of the normal distribution.\r\n* `dtype`: The type of the output.\r\n* `seed`: A Python integer. Used to create a random seed for the distribution.\r\n See\r\n [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\n for behavior.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tensor of the specified shape filled with random normal values.\r\n\r\n\r\n- - -\r\n\r\n### `tf.truncated_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)` \r\n\r\nOutputs random values from a truncated normal distribution.\r\n\r\nThe generated values follow a normal distribution with specified mean and\r\nstandard deviation, except that values whose magnitude is more than 2 standard\r\ndeviations from the mean are dropped and re-picked.\r\n\r\n##### Args: \r\n\r\n\r\n* `shape`: A 1-D integer Tensor or Python array. The shape of the output tensor.\r\n* `mean`: A 0-D Tensor or Python value of type `dtype`. The mean of the\r\n truncated normal distribution.\r\n* `stddev`: A 0-D Tensor or Python value of type `dtype`. The standard deviation\r\n of the truncated normal distribution.\r\n* `dtype`: The type of the output.\r\n* `seed`: A Python integer. Used to create a random seed for the distribution.\r\n See\r\n [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\n for behavior.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tensor of the specified shape filled with random truncated normal values.\r\n\r\n\r\n- - -\r\n\r\n### `tf.random_uniform(shape, minval=0.0, maxval=1.0, dtype=tf.float32, seed=None, name=None)` \r\n\r\nOutputs random values from a uniform distribution.\r\n\r\nThe generated values follow a uniform distribution in the range\r\n`[minval, maxval)`. The lower bound `minval` is included in the range, while\r\nthe upper bound `maxval` is excluded.\r\n\r\n##### Args: \r\n\r\n\r\n* `shape`: A 1-D integer Tensor or Python array. The shape of the output tensor.\r\n* `minval`: A 0-D Tensor or Python value of type `dtype`. The lower bound on the\r\n range of random values to generate.\r\n* `maxval`: A 0-D Tensor or Python value of type `dtype`. The upper bound on\r\n the range of random values to generate.\r\n* `dtype`: The type of the output.\r\n* `seed`: A Python integer. Used to create a random seed for the distribution.\r\n See\r\n [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\n for behavior.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tensor of the specified shape filled with random uniform values.\r\n\r\n\r\n- - -\r\n\r\n### `tf.random_shuffle(value, seed=None, name=None)` \r\n\r\nRandomly shuffles a tensor along its first dimension.\r\n\r\nThe tensor is shuffled along dimension 0, such that each `value[j]` is mapped\r\nto one and only one `output[i]`. For example, a mapping that might occur for a\r\n3x2 tensor is:\r\n\r\n```python\r\n[[1, 2], [[5, 6],\r\n [3, 4], ==> [1, 2],\r\n [5, 6]] [3, 4]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `value`: A Tensor to be shuffled.\r\n* `seed`: A Python integer. Used to create a random seed for the distribution.\r\n See\r\n [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\n for behavior.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tensor of same shape and type as `value`, shuffled along its first\r\n dimension.\r\n\r\n\r\n- - -\r\n\r\n### `tf.set_random_seed(seed)` \r\n\r\nSets the graph-level random seed.\r\n\r\nOperations that rely on a random seed actually derive it from two seeds:\r\nthe graph-level and operation-level seeds. This sets the graph-level seed.\r\n\r\nIts interactions with operation-level seeds is as follows:\r\n\r\n 1. If neither the graph-level nor the operation seed is set:\r\n A random seed is used for this op.\r\n 2. If the graph-level seed is set, but the operation seed is not:\r\n The system deterministically picks an operation seed in conjunction\r\n with the graph-level seed so that it gets a unique random sequence.\r\n 3. If the graph-level seed is not set, but the operation seed is set:\r\n A default graph-level seed and the specified operation seed are used to\r\n determine the random sequence.\r\n 4. If both the graph-level and the operation seed are set:\r\n Both seeds are used in conjunction to determine the random sequence.\r\n\r\nTo illustrate the user-visible effects, consider these examples:\r\n\r\nTo generate different sequences across sessions, set neither\r\ngraph-level nor op-level seeds:\r\n\r\n```python\r\na = tf.random_uniform([1])\r\nb = tf.random_normal([1])\r\n\r\nprint \"Session 1\"\r\nwith tf.Session() as sess1:\r\n print sess1.run(a) # generates 'A1'\r\n print sess1.run(a) # generates 'A2'\r\n print sess1.run(b) # generates 'B1'\r\n print sess1.run(b) # generates 'B2'\r\n\r\nprint \"Session 2\"\r\nwith tf.Session() as sess2:\r\n print sess2.run(a) # generates 'A3'\r\n print sess2.run(a) # generates 'A4'\r\n print sess2.run(b) # generates 'B3'\r\n print sess2.run(b) # generates 'B4'\r\n```\r\n\r\nTo generate the same repeatable sequence for an op across sessions, set the\r\nseed for the op:\r\n\r\n```python\r\na = tf.random_uniform([1], seed=1)\r\nb = tf.random_normal([1])\r\n\r\n# Repeatedly running this block with the same graph will generate the same\r\n# sequence of values for 'a', but different sequences of values for 'b'.\r\nprint \"Session 1\"\r\nwith tf.Session() as sess1:\r\n print sess1.run(a) # generates 'A1'\r\n print sess1.run(a) # generates 'A2'\r\n print sess1.run(b) # generates 'B1'\r\n print sess1.run(b) # generates 'B2'\r\n\r\nprint \"Session 2\"\r\nwith tf.Session() as sess2:\r\n print sess2.run(a) # generates 'A1'\r\n print sess2.run(a) # generates 'A2'\r\n print sess2.run(b) # generates 'B3'\r\n print sess2.run(b) # generates 'B4'\r\n```\r\n\r\nTo make the random sequences generated by all ops be repeatable across\r\nsessions, set a graph-level seed:\r\n\r\n```python\r\ntf.set_random_seed(1234)\r\na = tf.random_uniform([1])\r\nb = tf.random_normal([1])\r\n\r\n# Repeatedly running this block with the same graph will generate different\r\n# sequences of 'a' and 'b'.\r\nprint \"Session 1\"\r\nwith tf.Session() as sess1:\r\n print sess1.run(a) # generates 'A1'\r\n print sess1.run(a) # generates 'A2'\r\n print sess1.run(b) # generates 'B1'\r\n print sess1.run(b) # generates 'B2'\r\n\r\nprint \"Session 2\"\r\nwith tf.Session() as sess2:\r\n print sess2.run(a) # generates 'A1'\r\n print sess2.run(a) # generates 'A2'\r\n print sess2.run(b) # generates 'B1'\r\n print sess2.run(b) # generates 'B2'\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `seed`: integer.\r\n\r\n\r\n"
},
{
"path": "SOURCE/api_docs/python/control_flow_ops.md",
"content": "\r\n\r\n# Control Flow \r\n\r\nNote: Functions taking `Tensor` arguments can also take anything accepted by\r\n[`tf.convert_to_tensor`](../../api_docs/python/framework.md#convert_to_tensor).\r\n\r\n\r\n## Contents\r\n### [Control Flow](#AUTOGENERATED-control-flow)\r\n* [Control Flow Operations](#AUTOGENERATED-control-flow-operations)\r\n * [`tf.identity(input, name=None)`](#identity)\r\n * [`tf.tuple(tensors, name=None, control_inputs=None)`](#tuple)\r\n * [`tf.group(*inputs, **kwargs)`](#group)\r\n * [`tf.no_op(name=None)`](#no_op)\r\n * [`tf.count_up_to(ref, limit, name=None)`](#count_up_to)\r\n* [Logical Operators](#AUTOGENERATED-logical-operators)\r\n * [`tf.logical_and(x, y, name=None)`](#logical_and)\r\n * [`tf.logical_not(x, name=None)`](#logical_not)\r\n * [`tf.logical_or(x, y, name=None)`](#logical_or)\r\n * [`tf.logical_xor(x, y, name='LogicalXor')`](#logical_xor)\r\n* [Comparison Operators](#AUTOGENERATED-comparison-operators)\r\n * [`tf.equal(x, y, name=None)`](#equal)\r\n * [`tf.not_equal(x, y, name=None)`](#not_equal)\r\n * [`tf.less(x, y, name=None)`](#less)\r\n * [`tf.less_equal(x, y, name=None)`](#less_equal)\r\n * [`tf.greater(x, y, name=None)`](#greater)\r\n * [`tf.greater_equal(x, y, name=None)`](#greater_equal)\r\n * [`tf.select(condition, t, e, name=None)`](#select)\r\n * [`tf.where(input, name=None)`](#where)\r\n* [Debugging Operations](#AUTOGENERATED-debugging-operations)\r\n * [`tf.is_finite(x, name=None)`](#is_finite)\r\n * [`tf.is_inf(x, name=None)`](#is_inf)\r\n * [`tf.is_nan(x, name=None)`](#is_nan)\r\n * [`tf.verify_tensor_all_finite(t, msg, name=None)`](#verify_tensor_all_finite)\r\n * [`tf.check_numerics(tensor, message, name=None)`](#check_numerics)\r\n * [`tf.add_check_numerics_ops()`](#add_check_numerics_ops)\r\n * [`tf.Assert(condition, data, summarize=None, name=None)`](#Assert)\r\n * [`tf.Print(input_, data, message=None, first_n=None, summarize=None, name=None)`](#Print)\r\n\r\n\r\n\r\n\r\n## Control Flow Operations \r\n\r\nTensorFlow provides several operations and classes that you can use to control\r\nthe execution of operations and add conditional dependencies to your graph.\r\n\r\n- - -\r\n\r\n### `tf.identity(input, name=None)` \r\n\r\nReturn a tensor with the same shape and contents as the input tensor or value.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.tuple(tensors, name=None, control_inputs=None)` \r\n\r\nGroup tensors together.\r\n\r\nThis creates a tuple of tensors with the same values as the `tensors`\r\nargument, except that the value of each tensor is only returned after the\r\nvalues of all tensors have been computed.\r\n\r\n`control_inputs` contains additional ops that have to finish before this op\r\nfinishes, but whose outputs are not returned.\r\n\r\nThis can be used as a \"join\" mechanism for parallel computations: all the\r\nargument tensors can be computed in parallel, but the values of any tensor\r\nreturned by `tuple` are only available after all the parallel computations\r\nare done.\r\n\r\nSee also `group` and `with_dependencies`.\r\n\r\n##### Args: \r\n\r\n\r\n* `tensors`: A list of `Tensor`s or `IndexedSlices`, some entries can be `None`.\r\n* `name`: (optional) A name to use as a `name_scope` for the operation.\r\n* `control_inputs`: List of additional ops to finish before returning.\r\n\r\n##### Returns: \r\n\r\n Same as `tensors`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `tensors` does not contain any `Tensor` or `IndexedSlices`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.group(*inputs, **kwargs)` \r\n\r\nCreate an op that groups multiple operations.\r\n\r\nWhen this op finishes, all ops in `input` have finished. This op has no\r\noutput.\r\n\r\nSee also `tuple` and `with_dependencies`.\r\n\r\n##### Args: \r\n\r\n\r\n* `*inputs`: One or more tensors to group.\r\n* `**kwargs`: Optional parameters to pass when constructing the NodeDef.\r\n* `name`: A name for this operation (optional).\r\n\r\n##### Returns: \r\n\r\n An Operation that executes all its inputs.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If an unknown keyword argument is provided, or if there are\r\n no inputs.\r\n\r\n\r\n- - -\r\n\r\n### `tf.no_op(name=None)` \r\n\r\nDoes nothing. Only useful as a placeholder for control edges.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n### `tf.count_up_to(ref, limit, name=None)` \r\n\r\nIncrements 'ref' until it reaches 'limit'.\r\n\r\nThis operation outputs \"ref\" after the update is done. This makes it\r\neasier to chain operations that need to use the updated value.\r\n\r\n##### Args: \r\n\r\n\r\n* `ref`: A mutable `Tensor`. Must be one of the following types: `int32`, `int64`.\r\n Should be from a scalar `Variable` node.\r\n* `limit`: An `int`.\r\n If incrementing ref would bring it above limit, instead generates an\r\n 'OutOfRange' error.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `ref`.\r\n A copy of the input before increment. If nothing else modifies the\r\n input, the values produced will all be distinct.\r\n\r\n\r\n\r\n## Logical Operators \r\n\r\nTensorFlow provides several operations that you can use to add logical operators\r\nto your graph.\r\n\r\n- - -\r\n\r\n### `tf.logical_and(x, y, name=None)` \r\n\r\nReturns the truth value of x AND y element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` of type `bool`.\r\n* `y`: A `Tensor` of type `bool`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.logical_not(x, name=None)` \r\n\r\nReturns the truth value of NOT x element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` of type `bool`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.logical_or(x, y, name=None)` \r\n\r\nReturns the truth value of x OR y element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` of type `bool`.\r\n* `y`: A `Tensor` of type `bool`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.logical_xor(x, y, name='LogicalXor')` \r\n\r\nx ^ y = (x | y) & ~(x & y).\r\n\r\n\r\n\r\n## Comparison Operators \r\n\r\nTensorFlow provides several operations that you can use to add comparison\r\noperators to your graph.\r\n\r\n- - -\r\n\r\n### `tf.equal(x, y, name=None)` \r\n\r\nReturns the truth value of (x == y) element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`, `complex64`, `quint8`, `qint8`, `qint32`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.not_equal(x, y, name=None)` \r\n\r\nReturns the truth value of (x != y) element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`, `complex64`, `quint8`, `qint8`, `qint32`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.less(x, y, name=None)` \r\n\r\nReturns the truth value of (x < y) element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.less_equal(x, y, name=None)` \r\n\r\nReturns the truth value of (x <= y) element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.greater(x, y, name=None)` \r\n\r\nReturns the truth value of (x > y) element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.greater_equal(x, y, name=None)` \r\n\r\nReturns the truth value of (x >= y) element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.select(condition, t, e, name=None)` \r\n\r\nSelects elements from `t` or `e`, depending on `condition`.\r\n\r\nThe `condition`, `t`, and `e` tensors must all have the same shape,\r\nand the output will also have that shape. The `condition` tensor acts\r\nas an element-wise mask that chooses, based on the value at each\r\nelement, whether the corresponding element in the output should be\r\ntaken from `t` (if true) or `e` (if false). For example:\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# 'condition' tensor is [[True, False]\r\n# [True, False]]\r\n# 't' is [[1, 1],\r\n# [1, 1]]\r\n# 'e' is [[2, 2],\r\n# [2, 2]]\r\nselect(condition, t, e) ==> [[1, 2],\r\n [1, 2]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `condition`: A `Tensor` of type `bool`.\r\n* `t`: A `Tensor` with the same shape as `condition`.\r\n* `e`: A `Tensor` with the same type and shape as `t`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` with the same type and shape as `t` and `e`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.where(input, name=None)` \r\n\r\nReturns locations of true values in a boolean tensor.\r\n\r\nThis operation returns the coordinates of true elements in `input`. The\r\ncoordinates are returned in a 2-D tensor where the first dimension (rows)\r\nrepresents the number of true elements, and the second dimension (columns)\r\nrepresents the coordinates of the true elements. Keep in mind, the shape of\r\nthe output tensor can vary depending on how many true values there are in\r\n`input`. Indices are output in row-major order.\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# 'input' tensor is [[True, False]\r\n# [True, False]]\r\n# 'input' has two true values, so output has two coordinates.\r\n# 'input' has rank of 2, so coordinates have two indices.\r\nwhere(input) ==> [[0, 0],\r\n [1, 0]]\r\n\r\n# `input` tensor is [[[True, False]\r\n# [True, False]]\r\n# [[False, True]\r\n# [False, True]]\r\n# [[False, False]\r\n# [False, True]]]\r\n# 'input' has 5 true values, so output has 5 coordinates.\r\n# 'input' has rank of 3, so coordinates have three indices.\r\nwhere(input) ==> [[0, 0, 0],\r\n [0, 1, 0],\r\n [1, 0, 1],\r\n [1, 1, 1],\r\n [2, 1, 1]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor` of type `bool`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `int64`.\r\n\r\n\r\n\r\n## Debugging Operations \r\n\r\nTensorFlow provides several operations that you can use to validate values and\r\ndebug your graph.\r\n\r\n- - -\r\n\r\n### `tf.is_finite(x, name=None)` \r\n\r\nReturns which elements of x are finite.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.is_inf(x, name=None)` \r\n\r\nReturns which elements of x are Inf.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.is_nan(x, name=None)` \r\n\r\nReturns which elements of x are NaN.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.verify_tensor_all_finite(t, msg, name=None)` \r\n\r\nAssert that the tensor does not contain any NaN's or Inf's.\r\n\r\n##### Args: \r\n\r\n\r\n* `t`: Tensor to check.\r\n* `msg`: Message to log on failure.\r\n* `name`: A name for this operation (optional).\r\n\r\n##### Returns: \r\n\r\n Same tensor as `t`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.check_numerics(tensor, message, name=None)` \r\n\r\nChecks a tensor for NaN and Inf values.\r\n\r\nWhen run, reports an `InvalidArgument` error if `tensor` has any values\r\nthat are not a number (NaN) or infinity (Inf). Otherwise, passes `tensor` as-is.\r\n\r\n##### Args: \r\n\r\n\r\n* `tensor`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n* `message`: A `string`. Prefix of the error message.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `tensor`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.add_check_numerics_ops()` \r\n\r\nConnect a check_numerics to every floating point tensor.\r\n\r\n`check_numerics` operations themselves are added for each `float` or `double`\r\ntensor in the graph. For all ops in the graph, the `check_numerics` op for\r\nall of its (`float` or `double`) inputs is guaranteed to run before the\r\n`check_numerics` op on any of its outputs.\r\n\r\n##### Returns: \r\n\r\n A `group` op depending on all `check_numerics` ops added.\r\n\r\n\r\n- - -\r\n\r\n### `tf.Assert(condition, data, summarize=None, name=None)` \r\n\r\nAsserts that the given condition is true.\r\n\r\nIf `condition` evaluates to false, print the list of tensors in `data`.\r\n`summarize` determines how many entries of the tensors to print.\r\n\r\n##### Args: \r\n\r\n\r\n* `condition`: The condition to evaluate.\r\n* `data`: The tensors to print out when condition is false.\r\n* `summarize`: Print this many entries of each tensor.\r\n* `name`: A name for this operation (optional).\r\n\r\n\r\n- - -\r\n\r\n### `tf.Print(input_, data, message=None, first_n=None, summarize=None, name=None)` \r\n\r\nPrints a list of tensors.\r\n\r\nThis is an identity op with the side effect of printing `data` when\r\nevaluating.\r\n\r\n##### Args: \r\n\r\n\r\n* `input_`: A tensor passed through this op.\r\n* `data`: A list of tensors to print out when op is evaluated.\r\n* `message`: A string, prefix of the error message.\r\n* `first_n`: Only log `first_n` number of times. Negative numbers log always;\r\n this is the default.\r\n* `summarize`: Only print this many entries of each tensor.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n Same tensor as `input_`.\r\n\r\n\r\n"
},
{
"path": "SOURCE/api_docs/python/framework.md",
"content": "\r\n\r\n# Building Graphs \r\n\r\n## Contents\r\n### [Building Graphs](#AUTOGENERATED-building-graphs)\r\n* [Core graph data structures](#AUTOGENERATED-core-graph-data-structures)\r\n * [`class tf.Graph`](#Graph)\r\n * [`class tf.Operation`](#Operation)\r\n * [`class tf.Tensor`](#Tensor)\r\n* [Tensor types](#AUTOGENERATED-tensor-types)\r\n * [`class tf.DType`](#DType)\r\n * [`tf.as_dtype(type_value)`](#as_dtype)\r\n* [Utility functions](#AUTOGENERATED-utility-functions)\r\n * [`tf.device(dev)`](#device)\r\n * [`tf.name_scope(name)`](#name_scope)\r\n * [`tf.control_dependencies(control_inputs)`](#control_dependencies)\r\n * [`tf.convert_to_tensor(value, dtype=None, name=None)`](#convert_to_tensor)\r\n * [`tf.get_default_graph()`](#get_default_graph)\r\n * [`tf.import_graph_def(graph_def, input_map=None, return_elements=None, name=None, op_dict=None)`](#import_graph_def)\r\n* [Graph collections](#AUTOGENERATED-graph-collections)\r\n * [`tf.add_to_collection(name, value)`](#add_to_collection)\r\n * [`tf.get_collection(key, scope=None)`](#get_collection)\r\n * [`class tf.GraphKeys`](#GraphKeys)\r\n* [Defining new operations](#AUTOGENERATED-defining-new-operations)\r\n * [`class tf.RegisterGradient`](#RegisterGradient)\r\n * [`tf.NoGradient(op_type)`](#NoGradient)\r\n * [`class tf.RegisterShape`](#RegisterShape)\r\n * [`class tf.TensorShape`](#TensorShape)\r\n * [`class tf.Dimension`](#Dimension)\r\n * [`tf.op_scope(values, name, default_name)`](#op_scope)\r\n * [`tf.get_seed(op_seed)`](#get_seed)\r\n\r\n\r\n\r\n\r\nClasses and functions for building TensorFlow graphs.\r\n\r\n## Core graph data structures \r\n\r\n- - -\r\n\r\n### `class tf.Graph` \r\n\r\nA TensorFlow computation, represented as a dataflow graph.\r\n\r\nA `Graph` contains a set of\r\n[`Operation`](../../api_docs/python/framework.md#Operation) objects,\r\nwhich represent units of computation; and\r\n[`Tensor`](../../api_docs/python/framework.md#Tensor) objects, which represent\r\nthe units of data that flow between operations.\r\n\r\nA default `Graph` is always registered, and accessible by calling\r\n[`tf.get_default_graph()`](../../api_docs/python/framework.md#get_default_graph).\r\nTo add an operation to the default graph, simply call one of the functions\r\nthat defines a new `Operation`:\r\n\r\n```\r\nc = tf.constant(4.0)\r\nassert c.graph is tf.get_default_graph()\r\n```\r\n\r\nAnother typical usage involves the\r\n[`Graph.as_default()`](../../api_docs/python/framework.md#Graph.as_default)\r\ncontext manager, which overrides the current default graph for the\r\nlifetime of the context:\r\n\r\n```python\r\ng = tf.Graph()\r\nwith g.as_default():\r\n # Define operations and tensors in `g`.\r\n c = tf.constant(30.0)\r\n assert c.graph is g\r\n```\r\n\r\nImportant note: This class *is not* thread-safe for graph construction. All\r\noperations should be created from a single thread, or external\r\nsynchronization must be provided. Unless otherwise specified, all methods\r\nare not thread-safe.\r\n\r\n- - -\r\n\r\n#### `tf.Graph.__init__()` \r\n\r\nCreates a new, empty Graph.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.as_default()` \r\n\r\nReturns a context manager that makes this `Graph` the default graph.\r\n\r\nThis method should be used if you want to create multiple graphs\r\nin the same process. For convenience, a global default graph is\r\nprovided, and all ops will be added to this graph if you do not\r\ncreate a new graph explicitly. Use this method the `with` keyword\r\nto specify that ops created within the scope of a block should be\r\nadded to this graph.\r\n\r\nThe default graph is a property of the current thread. If you\r\ncreate a new thread, and wish to use the default graph in that\r\nthread, you must explicitly add a `with g.as_default():` in that\r\nthread's function.\r\n\r\nThe following code examples are equivalent:\r\n\r\n```python\r\n# 1. Using Graph.as_default():\r\ng = tf.Graph()\r\nwith g.as_default():\r\n c = tf.constant(5.0)\r\n assert c.graph is g\r\n\r\n# 2. Constructing and making default:\r\nwith tf.Graph().as_default() as g:\r\n c = tf.constant(5.0)\r\n assert c.graph is g\r\n```\r\n\r\n##### Returns: \r\n\r\n A context manager for using this graph as the default graph.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.as_graph_def(from_version=None)` \r\n\r\nReturns a serialized `GraphDef` representation of this graph.\r\n\r\nThe serialized `GraphDef` can be imported into another `Graph`\r\n(using [`import_graph_def()`](#import_graph_def)) or used with the\r\n[C++ Session API](../../api_docs/cc/index.md).\r\n\r\nThis method is thread-safe.\r\n\r\n##### Args: \r\n\r\n\r\n* `from_version`: Optional. If this is set, returns a `GraphDef`\r\n containing only the nodes that were added to this graph since\r\n its `version` property had the given value.\r\n\r\n##### Returns: \r\n\r\n A [`GraphDef`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/framework/graph.proto)\r\n protocol buffer.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.finalize()` \r\n\r\nFinalizes this graph, making it read-only.\r\n\r\nAfter calling `g.finalize()`, no new operations can be added to\r\n`g`. This method is used to ensure that no operations are added\r\nto a graph when it is shared between multiple threads, for example\r\nwhen using a [`QueueRunner`](../../api_docs/python/train.md#QueueRunner).\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.finalized` \r\n\r\nTrue if this graph has been finalized.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.control_dependencies(control_inputs)` \r\n\r\nReturns a context manager that specifies control dependencies.\r\n\r\nUse with the `with` keyword to specify that all operations constructed\r\nwithin the context should have control dependencies on\r\n`control_inputs`. For example:\r\n\r\n```python\r\nwith g.control_dependencies([a, b, c]):\r\n # `d` and `e` will only run after `a`, `b`, and `c` have executed.\r\n d = ...\r\n e = ...\r\n```\r\n\r\nMultiple calls to `control_dependencies()` can be nested, and in\r\nthat case a new `Operation` will have control dependencies on the union\r\nof `control_inputs` from all active contexts.\r\n\r\n```python\r\nwith g.control_dependencies([a, b]):\r\n # Ops declared here run after `a` and `b`.\r\n with g.control_dependencies([c, d]):\r\n # Ops declared here run after `a`, `b`, `c`, and `d`.\r\n```\r\n\r\n*N.B.* The control dependencies context applies *only* to ops that\r\nare constructed within the context. Merely using an op or tensor\r\nin the context does not add a control dependency. The following\r\nexample illustrates this point:\r\n\r\n```python\r\n# WRONG\r\ndef my_func(pred, tensor):\r\n t = tf.matmul(tensor, tensor)\r\n with tf.control_dependencies([pred]):\r\n # The matmul op is created outside the context, so no control\r\n # dependency will be added.\r\n return t\r\n\r\n# RIGHT\r\ndef my_func(pred, tensor):\r\n with tf.control_dependencies([pred]):\r\n # The matmul op is created in the context, so a control dependency\r\n # will be added.\r\n return tf.matmul(tensor, tensor)\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `control_inputs`: A list of `Operation` or `Tensor` objects, which\r\n must be executed or computed before running the operations\r\n defined in the context.\r\n\r\n##### Returns: \r\n\r\n A context manager that specifies control dependencies for all\r\n operations constructed within the context.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `control_inputs` is not a list of `Operation` or\r\n `Tensor` objects.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.device(device_name_or_function)` \r\n\r\nReturns a context manager that specifies the default device to use.\r\n\r\nThe `device_name_or_function` argument may either be a device name\r\nstring, a device function, or None:\r\n\r\n* If it is a device name string, all operations constructed in\r\n this context will be assigned to the device with that name.\r\n* If it is a function, it will be treated as function from\r\n Operation objects to device name strings, and invoked each time\r\n a new Operation is created. The Operation will be assigned to\r\n the device with the returned name.\r\n* If it is None, the default device will be cleared.\r\n\r\nFor example:\r\n\r\n```python\r\nwith g.device('/gpu:0'):\r\n # All operations constructed in this context will be placed\r\n # on GPU 0.\r\n with g.device(None):\r\n # All operations constructed in this context will have no\r\n # assigned device.\r\n\r\n# Defines a function from `Operation` to device string.\r\ndef matmul_on_gpu(n):\r\n if n.type == \"MatMul\":\r\n return \"/gpu:0\"\r\n else:\r\n return \"/cpu:0\"\r\n\r\nwith g.device(matmul_on_gpu):\r\n # All operations of type \"MatMul\" constructed in this context\r\n # will be placed on GPU 0; all other operations will be placed\r\n # on CPU 0.\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `device_name_or_function`: The device name or function to use in\r\n the context.\r\n\r\n##### Returns: \r\n\r\n A context manager that specifies the default device to use for newly\r\n created ops.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.name_scope(name)` \r\n\r\nReturns a context manager that creates hierarchical names for operations.\r\n\r\nA graph maintains a stack of name scopes. A `with name_scope(...):`\r\nstatement pushes a new name onto the stack for the lifetime of the context.\r\n\r\nThe `name` argument will be interpreted as follows:\r\n\r\n* A string (not ending with '/') will create a new name scope, in which\r\n `name` is appended to the prefix of all operations created in the\r\n context. If `name` has been used before, it will be made unique by\r\n calling `self.unique_name(name)`.\r\n* A scope previously captured from a `with g.name_scope(...) as\r\n scope:` statement will be treated as an \"absolute\" name scope, which\r\n makes it possible to re-enter existing scopes.\r\n* A value of `None` or the empty string will reset the current name scope\r\n to the top-level (empty) name scope.\r\n\r\nFor example:\r\n\r\n```python\r\nwith tf.Graph().as_default() as g:\r\n c = tf.constant(5.0, name=\"c\")\r\n assert c_1.name == \"c\"\r\n c_1 = tf.constant(6.0, name=\"c\")\r\n assert c_1.name == \"c_1\"\r\n\r\n # Creates a scope called \"nested\"\r\n with g.name_scope(\"nested\") as scope:\r\n nested_c = tf.constant(10.0, name=\"c\")\r\n assert nested_c.name == \"nested/c\"\r\n\r\n # Creates a nested scope called \"inner\".\r\n with g.name_scope(\"inner\"):\r\n nested_inner_c = tf.constant(20.0, name=\"c\")\r\n assert nested_inner_c.name == \"nested/inner/c\"\r\n\r\n # Create a nested scope called \"inner_1\".\r\n with g.name_scope(\"inner\"):\r\n nested_inner_1_c = tf.constant(30.0, name=\"c\")\r\n assert nested_inner_1_c.name == \"nested/inner_1/c\"\r\n\r\n # Treats `scope` as an absolute name scope, and\r\n # switches to the \"nested/\" scope.\r\n with g.name_scope(scope):\r\n nested_d = tf.constant(40.0, name=\"d\")\r\n assert nested_d.name == \"nested/d\"\r\n\r\n with g.name_scope(\"\"):\r\n e = tf.constant(50.0, name=\"e\")\r\n assert e.name == \"e\"\r\n```\r\n\r\nThe name of the scope itself can be captured by `with\r\ng.name_scope(...) as scope:`, which stores the name of the scope\r\nin the variable `scope`. This value can be used to name an\r\noperation that represents the overall result of executing the ops\r\nin a scope. For example:\r\n\r\n```python\r\ninputs = tf.constant(...)\r\nwith g.name_scope('my_layer') as scope:\r\n weights = tf.Variable(..., name=\"weights\")\r\n biases = tf.Variable(..., name=\"biases\")\r\n affine = tf.matmul(inputs, weights) + biases\r\n output = tf.nn.relu(affine, name=scope)\r\n```\r\n\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the scope.\r\n\r\n##### Returns: \r\n\r\n A context manager that installs `name` as a new name scope.\r\n\r\n\r\n\r\nA `Graph` instance supports an arbitrary number of \"collections\"\r\nthat are identified by name. For convenience when building a large\r\ngraph, collections can store groups of related objects: for\r\nexample, the `tf.Variable` uses a collection (named\r\n[`tf.GraphKeys.VARIABLES`](../../api_docs/python/framework.md#GraphKeys)) for\r\nall variables that are created during the construction of a graph. The caller\r\nmay define additional collections by specifying a new name.\r\n\r\n- - -\r\n\r\n#### `tf.Graph.add_to_collection(name, value)` \r\n\r\nStores `value` in the collection with the given `name`.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: The key for the collection. For example, the `GraphKeys` class\r\n contains many standard names for collections.\r\n* `value`: The value to add to the collection.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.get_collection(name, scope=None)` \r\n\r\nReturns a list of values in the collection with the given `name`.\r\n\r\n##### Args: \r\n\r\n\r\n* `key`: The key for the collection. For example, the `GraphKeys` class\r\n contains many standard names for collections.\r\n* `scope`: (Optional.) If supplied, the resulting list is filtered to include\r\n only items whose name begins with this string.\r\n\r\n##### Returns: \r\n\r\n The list of values in the collection with the given `name`, or\r\n an empty list if no value has been added to that collection. The\r\n list contains the values in the order under which they were\r\n collected.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.as_graph_element(obj, allow_tensor=True, allow_operation=True)` \r\n\r\nReturns the object referred to by `obj`, as an `Operation` or `Tensor`.\r\n\r\nThis function validates that `obj` represents an element of this\r\ngraph, and gives an informative error message if it is not.\r\n\r\nThis function is the canonical way to get/validate an object of\r\none of the allowed types from an external argument reference in the\r\nSession API.\r\n\r\nThis method may be called concurrently from multiple threads.\r\n\r\n##### Args: \r\n\r\n\r\n* `obj`: A `Tensor`, an `Operation`, or the name of a tensor or operation.\r\n Can also be any object with an `_as_graph_element()` method that returns\r\n a value of one of these types.\r\n* `allow_tensor`: If true, `obj` may refer to a `Tensor`.\r\n* `allow_operation`: If true, `obj` may refer to an `Operation`.\r\n\r\n##### Returns: \r\n\r\n The `Tensor` or `Operation` in the Graph corresponding to `obj`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `obj` is not a type we support attempting to convert\r\n to types.\r\n* `ValueError`: If `obj` is of an appropriate type but invalid. For\r\n example, an invalid string.\r\n* `KeyError`: If `obj` is not an object in the graph.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.get_operation_by_name(name)` \r\n\r\nReturns the `Operation` with the given `name`.\r\n\r\nThis method may be called concurrently from multiple threads.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: The name of the `Operation` to return.\r\n\r\n##### Returns: \r\n\r\n The `Operation` with the given `name`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `name` is not a string.\r\n* `KeyError`: If `name` does not correspond to an operation in this graph.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.get_tensor_by_name(name)` \r\n\r\nReturns the `Tensor` with the given `name`.\r\n\r\nThis method may be called concurrently from multiple threads.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: The name of the `Tensor` to return.\r\n\r\n##### Returns: \r\n\r\n The `Tensor` with the given `name`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `name` is not a string.\r\n* `KeyError`: If `name` does not correspond to a tensor in this graph.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.get_operations()` \r\n\r\nReturn the list of operations in the graph.\r\n\r\nYou can modify the operations in place, but modifications\r\nto the list such as inserts/delete have no effect on the\r\nlist of operations known to the graph.\r\n\r\nThis method may be called concurrently from multiple threads.\r\n\r\n##### Returns: \r\n\r\n A list of Operations.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.get_default_device()` \r\n\r\nReturns the default device.\r\n\r\n##### Returns: \r\n\r\n A string.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.seed` \r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.unique_name(name)` \r\n\r\nReturn a unique Operation name for \"name\".\r\n\r\nNote: You rarely need to call unique_name() directly. Most of the time you\r\njust need to create \"with g.name_scope()\" blocks to generate structured\r\nnames.\r\n\r\n`unique_name` is used to generate structured names, separated by \"/\",\r\nto help identify Operations when debugging a Graph. Operation names\r\nare displayed in error messages reported by the TensorFlow runtime,\r\nand in various visualization tools such as TensorBoard.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: The name for an `Operation`.\r\n\r\n##### Returns: \r\n\r\n A string to be passed to `create_op()` that will be used\r\n to name the operation being created.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.version` \r\n\r\nReturns a version number that increases as ops are added to the graph.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.create_op(op_type, inputs, dtypes, input_types=None, name=None, attrs=None, op_def=None, compute_shapes=True)` \r\n\r\nCreates an `Operation` in this graph.\r\n\r\nThis is a low-level interface for creating an `Operation`. Most\r\nprograms will not call this method directly, and instead use the\r\nPython op constructors, such as `tf.constant()`, which add ops to\r\nthe default graph.\r\n\r\n##### Args: \r\n\r\n\r\n* `op_type`: The `Operation` type to create. This corresponds to the\r\n `OpDef.name` field for the proto that defines the operation.\r\n* `inputs`: A list of `Tensor` objects that will be inputs to the `Operation`.\r\n* `dtypes`: A list of `DType` objects that will be the types of the tensors\r\n that the operation produces.\r\n* `input_types`: (Optional.) A list of `DType`s that will be the types of\r\n the tensors that the operation consumes. By default, uses the base\r\n `DType` of each input in `inputs`. Operations that expect\r\n reference-typed inputs must specify `input_types` explicitly.\r\n* `name`: (Optional.) A string name for the operation. If not specified, a\r\n name is generated based on `op_type`.\r\n* `attrs`: (Optional.) A list of `AttrValue` protos for the `attr` field of\r\n the `NodeDef` proto that will represent the operation.\r\n* `op_def`: (Optional.) The `OpDef` proto that describes the `op_type` that\r\n the operation will have.\r\n* `compute_shapes`: (Optional.) If True, shape inference will be performed\r\n to compute the shapes of the outputs.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: if any of the inputs is not a `Tensor`.\r\n\r\n##### Returns: \r\n\r\n An `Operation` object.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Graph.gradient_override_map(op_type_map)` \r\n\r\nEXPERIMENTAL: A context manager for overriding gradient functions.\r\n\r\nThis context manager can be used to override the gradient function\r\nthat will be used for ops within the scope of the context.\r\n\r\nFor example:\r\n\r\n```python\r\n@tf.RegisterGradient(\"CustomSquare\")\r\ndef _custom_square_grad(op, inputs):\r\n # ...\r\n\r\nwith tf.Graph().as_default() as g:\r\n c = tf.constant(5.0)\r\n s_1 = tf.square(c) # Uses the default gradient for tf.square.\r\n with g.gradient_override_map({\"Square\": \"CustomSquare\"}):\r\n s_2 = tf.square(s_2) # Uses _custom_square_grad to compute the\r\n # gradient of s_2.\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `op_type_map`: A dictionary mapping op type strings to alternative op\r\n type strings.\r\n\r\n##### Returns: \r\n\r\n A context manager that sets the alternative op type to be used for one\r\n or more ops created in that context.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `op_type_map` is not a dictionary mapping strings to\r\n strings.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.Operation` \r\n\r\nRepresents a graph node that performs computation on tensors.\r\n\r\nAn `Operation` is a node in a TensorFlow `Graph` that takes zero or\r\nmore `Tensor` objects as input, and produces zero or more `Tensor`\r\nobjects as output. Objects of type `Operation` are created by\r\ncalling a Python op constructor (such as\r\n[`tf.matmul()`](../../api_docs/python/math_ops.md#matmul))\r\nor [`Graph.create_op()`](../../api_docs/python/framework.md#Graph.create_op).\r\n\r\nFor example `c = tf.matmul(a, b)` creates an `Operation` of type\r\n\"MatMul\" that takes tensors `a` and `b` as input, and produces `c`\r\nas output.\r\n\r\nAfter the graph has been launched in a session, an `Operation` can\r\nbe executed by passing it to\r\n[`Session.run()`](../../api_docs/python/client.md#Session.run).\r\n`op.run()` is a shortcut for calling `tf.get_default_session().run(op)`.\r\n\r\n- - -\r\n\r\n#### `tf.Operation.name` \r\n\r\nThe full name of this operation.\r\n\r\n- - -\r\n\r\n#### `tf.Operation.type` \r\n\r\nThe type of the op (e.g. `\"MatMul\"`).\r\n\r\n- - -\r\n\r\n#### `tf.Operation.inputs` \r\n\r\nThe list of `Tensor` objects representing the data inputs of this op.\r\n\r\n- - -\r\n\r\n#### `tf.Operation.control_inputs` \r\n\r\nThe `Operation` objects on which this op has a control dependency.\r\n\r\nBefore this op is executed, TensorFlow will ensure that the\r\noperations in `self.control_inputs` have finished executing. This\r\nmechanism can be used to run ops sequentially for performance\r\nreasons, or to ensure that the side effects of an op are observed\r\nin the correct order.\r\n\r\n##### Returns: \r\n\r\n A list of `Operation` objects.\r\n\r\n- - -\r\n\r\n#### `tf.Operation.outputs` \r\n\r\nThe list of `Tensor` objects representing the outputs of this op.\r\n\r\n- - -\r\n\r\n#### `tf.Operation.device` \r\n\r\nThe name of the device to which this op has been assigned, if any.\r\n\r\n##### Returns: \r\n\r\n The string name of the device to which this op has been\r\n assigned, or None if it has not been assigned to a device.\r\n\r\n- - -\r\n\r\n#### `tf.Operation.graph` \r\n\r\nThe `Graph` that contains this operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Operation.run(feed_dict=None, session=None)` \r\n\r\nRuns this operation in a `Session`.\r\n\r\nCalling this method will execute all preceding operations that\r\nproduce the inputs needed for this operation.\r\n\r\n*N.B.* Before invoking `Operation.run()`, its graph must have been\r\nlaunched in a session, and either a default session must be\r\navailable, or `session` must be specified explicitly.\r\n\r\n##### Args: \r\n\r\n\r\n* `feed_dict`: A dictionary that maps `Tensor` objects to feed values.\r\n See [`Session.run()`](../../api_docs/python/client.md#Session.run)\r\n for a description of the valid feed values.\r\n* `session`: (Optional.) The `Session` to be used to run to this operation. If\r\n none, the default session will be used.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Operation.get_attr(name)` \r\n\r\nReturns the value of the attr of this op with the given `name`.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: The name of the attr to fetch.\r\n\r\n##### Returns: \r\n\r\n The value of the attr, as a Python object.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If this op does not have an attr with the given `name`.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Operation.traceback` \r\n\r\nReturns the call stack from when this operation was constructed.\r\n\r\n\r\n#### Other Methods \r\n- - -\r\n\r\n#### `tf.Operation.__init__(node_def, g, inputs=None, output_types=None, control_inputs=None, input_types=None, original_op=None, op_def=None)` \r\n\r\nCreates an `Operation`.\r\n\r\nNOTE: This constructor validates the name of the Operation (passed\r\nas \"node_def.name\"). Valid Operation names match the following\r\nregular expression:\r\n\r\n [A-Za-z0-9.][A-Za-z0-9_.\\-/]*\r\n\r\n##### Args: \r\n\r\n\r\n* `node_def`: graph_pb2.NodeDef. NodeDef for the Operation.\r\n Used for attributes of graph_pb2.NodeDef, typically \"name\",\r\n \"op\", and \"device\". The \"input\" attribute is irrelevant here\r\n as it will be computed when generating the model.\r\n* `g`: Graph. The parent graph.\r\n* `inputs`: list of Tensor objects. The inputs to this Operation.\r\n* `output_types`: list of types_pb2.DataType. List of the types of the\r\n Tensors computed by this operation. The length of this list indicates\r\n the number of output endpoints of the Operation.\r\n* `control_inputs`: list of operations or tensors from which to have a\r\n control dependency.\r\n* `input_types`: List of types_pb2.DataType representing the\r\n types of the Tensors accepted by the Operation. By default\r\n uses [x.dtype.base_dtype for x in inputs]. Operations that expect\r\n reference-typed inputs must specify these explicitly.\r\n* `original_op`: Optional. Used to associate the new Operation with an\r\n existing Operation (for example, a replica with the op that was\r\n replicated).\r\n* `op_def`: Optional. The op_def_pb2.OpDef proto that describes the\r\n op type that this Operation represents.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: if control inputs are not Operations or Tensors,\r\n or if node_def is not a NodeDef,\r\n or if g is not a Graph,\r\n or if inputs are not Tensors,\r\n or if inputs and input_types are incompatible.\r\n* `ValueError`: if the node_def name is not valid.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Operation.node_def` \r\n\r\nReturns a serialized `NodeDef` representation of this operation.\r\n\r\n##### Returns: \r\n\r\n A\r\n [`NodeDef`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/framework/graph.proto)\r\n protocol buffer.\r\n\r\n- - -\r\n\r\n#### `tf.Operation.op_def` \r\n\r\nReturns the `OpDef` proto that represents the type of this op.\r\n\r\n##### Returns: \r\n\r\n An\r\n [`OpDef`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/framework/op_def.proto)\r\n protocol buffer.\r\n\r\n- - -\r\n\r\n#### `tf.Operation.values()` \r\n\r\nDEPRECATED: Use outputs.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.Tensor` \r\n\r\nRepresents a value produced by an `Operation`.\r\n\r\nA `Tensor` is a symbolic handle to one of the outputs of an\r\n`Operation`. It does not hold the values of that operation's output,\r\nbut instead provides a means of computing those values in a\r\nTensorFlow [`Session`](../../api_docs/python/client.md#Session).\r\n\r\nThis class has two primary purposes:\r\n\r\n1. A `Tensor` can be passed as an input to another `Operation`.\r\n This builds a dataflow connection between operations, which\r\n enables TensorFlow to execute an entire `Graph` that represents a\r\n large, multi-step computation.\r\n\r\n2. After the graph has been launched in a session, the value of the\r\n `Tensor` can be computed by passing it to\r\n [`Session.run()`](../../api_docs/python/client.md#Session.run).\r\n `t.eval()` is a shortcut for calling\r\n `tf.get_default_session().run(t)`.\r\n\r\nIn the following example, `c`, `d`, and `e` are symbolic `Tensor`\r\nobjects, whereas `result` is a numpy array that stores a concrete\r\nvalue:\r\n\r\n```python\r\n# Build a dataflow graph.\r\nc = tf.constant([[1.0, 2.0], [3.0, 4.0]])\r\nd = tf.constant([[1.0, 1.0], [0.0, 1.0]])\r\ne = tf.matmul(c, d)\r\n\r\n# Construct a `Session` to execut the graph.\r\nsess = tf.Session()\r\n\r\n# Execute the graph and store the value that `e` represents in `result`.\r\nresult = sess.run(e)\r\n```\r\n\r\n- - -\r\n\r\n#### `tf.Tensor.dtype` \r\n\r\nThe `DType` of elements in this tensor.\r\n\r\n- - -\r\n\r\n#### `tf.Tensor.name` \r\n\r\nThe string name of this tensor.\r\n\r\n- - -\r\n\r\n#### `tf.Tensor.value_index` \r\n\r\nThe index of this tensor in the outputs of its `Operation`.\r\n\r\n- - -\r\n\r\n#### `tf.Tensor.graph` \r\n\r\nThe `Graph` that contains this tensor.\r\n\r\n- - -\r\n\r\n#### `tf.Tensor.op` \r\n\r\nThe `Operation` that produces this tensor as an output.\r\n\r\n- - -\r\n\r\n#### `tf.Tensor.consumers()` \r\n\r\nReturns a list of `Operation`s that consume this tensor.\r\n\r\n##### Returns: \r\n\r\n A list of `Operation`s.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Tensor.eval(feed_dict=None, session=None)` \r\n\r\nEvaluates this tensor in a `Session`.\r\n\r\nCalling this method will execute all preceding operations that\r\nproduce the inputs needed for the operation that produces this\r\ntensor.\r\n\r\n*N.B.* Before invoking `Tensor.eval()`, its graph must have been\r\nlaunched in a session, and either a default session must be\r\navailable, or `session` must be specified explicitly.\r\n\r\n##### Args: \r\n\r\n\r\n* `feed_dict`: A dictionary that maps `Tensor` objects to feed values.\r\n See [`Session.run()`](../../api_docs/python/client.md#Session.run) for a\r\n description of the valid feed values.\r\n* `session`: (Optional.) The `Session` to be used to evaluate this tensor. If\r\n none, the default session will be used.\r\n\r\n##### Returns: \r\n\r\n A numpy array corresponding to the value of this tensor.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Tensor.get_shape()` \r\n\r\nReturns the `TensorShape` that represents the shape of this tensor.\r\n\r\nThe shape is computed using shape inference functions that are\r\nregistered for each `Operation` type using `tf.RegisterShape`.\r\nSee [`TensorShape`](../../api_docs/python/framework.md#TensorShape) for more\r\ndetails of what a shape represents.\r\n\r\nThe inferred shape of a tensor is used to provide shape\r\ninformation without having to launch the graph in a session. This\r\ncan be used for debugging, and providing early error messages. For\r\nexample:\r\n\r\n```python\r\nc = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])\r\n\r\nprint c.get_shape()\r\n==> TensorShape([Dimension(2), Dimension(3)])\r\n\r\nd = tf.constant([[1.0, 0.0], [0.0, 1.0], [1.0, 0.0], [0.0, 1.0]])\r\n\r\nprint d.get_shape()\r\n==> TensorShape([Dimension(4), Dimension(2)])\r\n\r\n# Raises a ValueError, because `c` and `d` do not have compatible\r\n# inner dimensions.\r\ne = tf.matmul(c, d)\r\n\r\nf = tf.matmul(c, d, transpose_a=True, transpose_b=True)\r\n\r\nprint f.get_shape()\r\n==> TensorShape([Dimension(3), Dimension(4)])\r\n```\r\n\r\nIn some cases, the inferred shape may have unknown dimensions. If\r\nthe caller has additional information about the values of these\r\ndimensions, `Tensor.set_shape()` can be used to augment the\r\ninferred shape.\r\n\r\n##### Returns: \r\n\r\n A `TensorShape` representing the shape of this tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Tensor.set_shape(shape)` \r\n\r\nUpdates the shape of this tensor.\r\n\r\nThis method can be called multiple times, and will merge the given\r\n`shape` with the current shape of this tensor. It can be used to\r\nprovide additional information about the shape of this tensor that\r\ncannot be inferred from the graph alone. For example, this can be used\r\nto provide additional information about the shapes of images:\r\n\r\n```python\r\n_, image_data = tf.TFRecordReader(...).read(...)\r\nimage = tf.image.decode_png(image_data, channels=3)\r\n\r\n# The height and width dimensions of `image` are data dependent, and\r\n# cannot be computed without executing the op.\r\nprint image.get_shape()\r\n==> TensorShape([Dimension(None), Dimension(None), Dimension(3)])\r\n\r\n# We know that each image in this dataset is 28 x 28 pixels.\r\nimage.set_shape([28, 28, 3])\r\nprint image.get_shape()\r\n==> TensorShape([Dimension(28), Dimension(28), Dimension(3)])\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `shape`: A `TensorShape` representing the shape of this tensor.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `shape` is not compatible with the current shape of\r\n this tensor.\r\n\r\n\r\n\r\n#### Other Methods \r\n- - -\r\n\r\n#### `tf.Tensor.__init__(op, value_index, dtype)` \r\n\r\nCreates a new `Tensor`.\r\n\r\n##### Args: \r\n\r\n\r\n* `op`: An `Operation`. `Operation` that computes this tensor.\r\n* `value_index`: An `int`. Index of the operation's endpoint that produces\r\n this tensor.\r\n* `dtype`: A `types.DType`. Type of data stored in this tensor.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If the op is not an `Operation`.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Tensor.device` \r\n\r\nThe name of the device on which this tensor will be produced, or None.\r\n\r\n\r\n\r\n## Tensor types \r\n\r\n- - -\r\n\r\n### `class tf.DType` \r\n\r\nRepresents the type of the elements in a `Tensor`.\r\n\r\nThe following `DType` objects are defined:\r\n\r\n* `tf.float32`: 32-bit single-precision floating-point.\r\n* `tf.float64`: 64-bit double-precision floating-point.\r\n* `tf.bfloat16`: 16-bit truncated floating-point.\r\n* `tf.complex64`: 64-bit single-precision complex.\r\n\r\n* `tf.int8`: 8-bit signed integer.\r\n* `tf.uint8`: 8-bit unsigned integer.\r\n* `tf.int32`: 32-bit signed integer.\r\n* `tf.int64`: 64-bit signed integer.\r\n\r\n* `tf.bool`: Boolean.\r\n\r\n* `tf.string`: String.\r\n\r\n* `tf.qint8`: Quantized 8-bit signed integer.\r\n* `tf.quint8`: Quantized 8-bit unsigned integer.\r\n* `tf.qint32`: Quantized 32-bit signed integer.\r\n\r\nIn addition, variants of these types with the `_ref` suffix are\r\ndefined for reference-typed tensors.\r\n\r\nThe `tf.as_dtype()` function converts numpy types and string type\r\nnames to a `DType` object.\r\n\r\n- - -\r\n\r\n#### `tf.DType.is_compatible_with(other)` \r\n\r\nReturns True if the `other` DType will be converted to this DType.\r\n\r\nThe conversion rules are as follows:\r\n\r\n```\r\nDType(T) .is_compatible_with(DType(T)) == True\r\nDType(T) .is_compatible_with(DType(T).as_ref) == True\r\nDType(T).as_ref.is_compatible_with(DType(T)) == False\r\nDType(T).as_ref.is_compatible_with(DType(T).as_ref) == True\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `other`: A `DType` (or object that may be converted to a `DType`).\r\n\r\n##### Returns: \r\n\r\n True if a Tensor of the `other` `DType` will be implicitly converted to\r\n this `DType`.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.DType.name` \r\n\r\nReturns the string name for this `DType`.\r\n\r\n- - -\r\n\r\n#### `tf.DType.base_dtype` \r\n\r\nReturns a non-reference `DType` based on this `DType`.\r\n\r\n- - -\r\n\r\n#### `tf.DType.is_ref_dtype` \r\n\r\nReturns `True` if this `DType` represents a reference type.\r\n\r\n- - -\r\n\r\n#### `tf.DType.as_ref` \r\n\r\nReturns a reference `DType` based on this `DType`.\r\n\r\n- - -\r\n\r\n#### `tf.DType.is_integer` \r\n\r\nReturns whether this is a (non-quantized) integer type.\r\n\r\n- - -\r\n\r\n#### `tf.DType.is_quantized` \r\n\r\nReturns whether this is a quantized data type.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.DType.as_numpy_dtype` \r\n\r\nReturns a `numpy.dtype` based on this `DType`.\r\n\r\n- - -\r\n\r\n#### `tf.DType.as_datatype_enum` \r\n\r\nReturns a `types_pb2.DataType` enum value based on this `DType`.\r\n\r\n\r\n#### Other Methods \r\n- - -\r\n\r\n#### `tf.DType.__init__(type_enum)` \r\n\r\nCreates a new `DataType`.\r\n\r\nNOTE(mrry): In normal circumstances, you should not need to\r\nconstruct a DataType object directly. Instead, use the\r\ntypes.as_dtype() function.\r\n\r\n##### Args: \r\n\r\n\r\n* `type_enum`: A `types_pb2.DataType` enum value.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `type_enum` is not a value `types_pb2.DataType`.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.DType.max` \r\n\r\nReturns the maximum representable value in this data type.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: if this is a non-numeric, unordered, or quantized type.\r\n\r\n- - -\r\n\r\n#### `tf.DType.min` \r\n\r\nReturns the minimum representable value in this data type.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: if this is a non-numeric, unordered, or quantized type.\r\n\r\n\r\n- - -\r\n\r\n### `tf.as_dtype(type_value)` \r\n\r\nConverts the given `type_value` to a `DType`.\r\n\r\n##### Args: \r\n\r\n\r\n* `type_value`: A value that can be converted to a `tf.DType`\r\n object. This may currently be a `tf.DType` object, a\r\n [`DataType` enum](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/framework/types.proto),\r\n a string type name, or a `numpy.dtype`.\r\n\r\n##### Returns: \r\n\r\n A `DType` corresponding to `type_value`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `type_value` cannot be converted to a `DType`.\r\n\r\n\r\n\r\n## Utility functions \r\n\r\n- - -\r\n\r\n### `tf.device(dev)` \r\n\r\nWrapper for `Graph.device()` using the default graph.\r\n\r\nSee\r\n[`Graph.name_scope()`](../../api_docs/python/framework.md#Graph.name_scope)\r\nfor more details.\r\n\r\n##### Args: \r\n\r\n\r\n* `device_name_or_function`: The device name or function to use in\r\n the context.\r\n\r\n##### Returns: \r\n\r\n A context manager that specifies the default device to use for newly\r\n created ops.\r\n\r\n\r\n- - -\r\n\r\n### `tf.name_scope(name)` \r\n\r\nWrapper for `Graph.name_scope()` using the default graph.\r\n\r\nSee\r\n[`Graph.name_scope()`](../../api_docs/python/framework.md#Graph.name_scope)\r\nfor more details.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the scope.\r\n\r\n##### Returns: \r\n\r\n A context manager that installs `name` as a new name scope in the\r\n default graph.\r\n\r\n\r\n- - -\r\n\r\n### `tf.control_dependencies(control_inputs)` \r\n\r\nWrapper for `Graph.control_dependencies()` using the default graph.\r\n\r\nSee [`Graph.control_dependencies()`](../../api_docs/python/framework.md#Graph.control_dependencies)\r\nfor more details.\r\n\r\n##### Args: \r\n\r\n\r\n* `control_inputs`: A list of `Operation` or `Tensor` objects, which\r\n must be executed or computed before running the operations\r\n defined in the context.\r\n\r\n##### Returns: \r\n\r\n A context manager that specifies control dependencies for all\r\n operations constructed within the context.\r\n\r\n\r\n- - -\r\n\r\n### `tf.convert_to_tensor(value, dtype=None, name=None)` \r\n\r\nConverts the given `value` to a `Tensor`.\r\n\r\nThis function converts Python objects of various types to `Tensor`\r\nobjects. It accepts `Tensor` objects, numpy arrays, Python lists,\r\nand Python scalars. For example:\r\n\r\n```python\r\nimport numpy as np\r\narray = np.random.rand((32, 100, 100))\r\n\r\ndef my_func(arg):\r\n arg = tf.convert_to_tensor(arg, dtype=tf.float32)\r\n return tf.matmul(arg, arg) + arg\r\n\r\n# The following calls are equivalent.\r\nvalue_1 = my_func(tf.constant([[1.0, 2.0], [3.0, 4.0]]))\r\nvalue_2 = my_func([[1.0, 2.0], [3.0, 4.0]])\r\nvalue_3 = my_func(np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float32))\r\n```\r\n\r\nThis function can be useful when composing a new operation in Python\r\n(such as `my_func` in the example above). All standard Python op\r\nconstructors apply this function to each of their Tensor-valued\r\ninputs, which allows those ops to accept numpy arrays, Python lists,\r\nand scalars in addition to `Tensor` objects.\r\n\r\n##### Args: \r\n\r\n\r\n* `value`: An object whose type has a registered `Tensor` conversion function.\r\n* `dtype`: Optional element type for the returned tensor. If missing, the\r\n type is inferred from the type of `value`.\r\n* `name`: Optional name to use if a new `Tensor` is created.\r\n\r\n##### Returns: \r\n\r\n A `Tensor` based on `value`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If no conversion function is registered for `value`.\r\n* `RuntimeError`: If a registered conversion function returns an invalid value.\r\n\r\n\r\n- - -\r\n\r\n### `tf.get_default_graph()` \r\n\r\nReturns the default graph for the current thread.\r\n\r\nThe returned graph will be the innermost graph on which a\r\n`Graph.as_default()` context has been entered, or a global default\r\ngraph if none has been explicitly created.\r\n\r\n*N.B.* The default graph is a property of the current thread. If you\r\ncreate a new thread, and wish to use the default graph in that\r\nthread, you must explicitly add a `with g.as_default():` in that\r\nthread's function.\r\n\r\n##### Returns: \r\n\r\n The default `Graph` being used in the current thread.\r\n\r\n\r\n- - -\r\n\r\n### `tf.import_graph_def(graph_def, input_map=None, return_elements=None, name=None, op_dict=None)` \r\n\r\nImports the TensorFlow graph in `graph_def` into the Python `Graph`.\r\n\r\nThis function provides a way to import a serialized TensorFlow\r\n[`GraphDef`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/framework/graph.proto)\r\nprotocol buffer, and extract individual objects in the `GraphDef` as\r\n[`Tensor`](#Tensor) and [`Operation`](#Operation) objects. See\r\n[`Graph.as_graph_def()`](#Graph.as_graph_def) for a way to create a\r\n`GraphDef` proto.\r\n\r\n##### Args: \r\n\r\n\r\n* `graph_def`: A `GraphDef` proto containing operations to be imported into\r\n the default graph.\r\n* `input_map`: A dictionary mapping input names (as strings) in `graph_def`\r\n to `Tensor` objects. The values of the named input tensors in the\r\n imported graph will be re-mapped to the respective `Tensor` values.\r\n* `return_elements`: A list of strings containing operation names in\r\n `graph_def` that will be returned as `Operation` objects; and/or\r\n tensor names in `graph_def` that will be returned as `Tensor` objects.\r\n* `name`: (Optional.) A prefix that will be prepended to the names in\r\n `graph_def`. Defaults to `\"import\"`.\r\n* `op_dict`: (Optional.) A dictionary mapping op type names to `OpDef` protos.\r\n Must contain an `OpDef` proto for each op type named in `graph_def`.\r\n If omitted, uses the `OpDef` protos registered in the global registry.\r\n\r\n##### Returns: \r\n\r\n A list of `Operation` and/or `Tensor` objects from the imported graph,\r\n corresponding to the names in `return_elements'.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `graph_def` is not a `GraphDef` proto,\r\n `input_map' is not a dictionary mapping strings to `Tensor` objects,\r\n or `return_elements` is not a list of strings.\r\n* `ValueError`: If `input_map`, or `return_elements` contains names that\r\n do not appear in `graph_def`, or `graph_def` is not well-formed (e.g.\r\n it refers to an unknown tensor).\r\n\r\n\r\n\r\n## Graph collections \r\n\r\n- - -\r\n\r\n### `tf.add_to_collection(name, value)` \r\n\r\nWrapper for `Graph.add_to_collection()` using the default graph.\r\n\r\nSee [`Graph.add_to_collection()`](../../api_docs/python/framework.md#Graph.add_to_collection)\r\nfor more details.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: The key for the collection. For example, the `GraphKeys` class\r\n contains many standard names for collections.\r\n* `value`: The value to add to the collection.\r\n\r\n\r\n- - -\r\n\r\n### `tf.get_collection(key, scope=None)` \r\n\r\nWrapper for `Graph.get_collection()` using the default graph.\r\n\r\nSee [`Graph.get_collection()`](../../api_docs/python/framework.md#Graph.get_collection)\r\nfor more details.\r\n\r\n##### Args: \r\n\r\n\r\n* `key`: The key for the collection. For example, the `GraphKeys` class\r\n contains many standard names for collections.\r\n* `scope`: (Optional.) If supplied, the resulting list is filtered to include\r\n only items whose name begins with this string.\r\n\r\n##### Returns: \r\n\r\n The list of values in the collection with the given `name`, or\r\n an empty list if no value has been added to that collection. The\r\n list contains the values in the order under which they were\r\n collected.\r\n\r\n\r\n- - -\r\n\r\n### `class tf.GraphKeys` \r\n\r\nStandard names to use for graph collections.\r\n\r\nThe standard library uses various well-known names to collect and\r\nretrieve values associated with a graph. For example, the\r\n`tf.Optimizer` subclasses default to optimizing the variables\r\ncollected under `tf.GraphKeys.TRAINABLE_VARIABLES` if none is\r\nspecified, but it is also possible to pass an explicit list of\r\nvariables.\r\n\r\nThe following standard keys are defined:\r\n\r\n* `VARIABLES`: the `Variable` objects that comprise a model, and\r\n must be saved and restored together. See\r\n [`tf.all_variables()`](../../api_docs/python/state_ops.md#all_variables)\r\n for more details.\r\n* `TRAINABLE_VARIABLES`: the subset of `Variable` objects that will\r\n be trained by an optimizer. See\r\n [`tf.trainable_variables()`](../../api_docs/python/state_ops.md#trainable_variables)\r\n for more details.\r\n* `SUMMARIES`: the summary `Tensor` objects that have been created in the\r\n graph. See\r\n [`tf.merge_all_summaries()`](../../api_docs/python/train.md#merge_all_summaries)\r\n for more details.\r\n* `QUEUE_RUNNERS`: the `QueueRunner` objects that are used to\r\n produce input for a computation. See\r\n [`tf.start_queue_runners()`](../../api_docs/python/train.md#start_queue_runners)\r\n for more details.\r\n\r\n\r\n## Defining new operations \r\n\r\n- - -\r\n\r\n### `class tf.RegisterGradient` \r\n\r\nA decorator for registering the gradient function for an op type.\r\n\r\nThis decorator is only used when defining a new op type. For an op\r\nwith `m` inputs and `n` inputs, the gradient function is a function\r\nthat takes the original `Operation` and `n` `Tensor` objects\r\n(representing the gradients with respect to each output of the op),\r\nand returns `m` `Tensor` objects (representing the partial gradients\r\nwith respect to each input of the op).\r\n\r\nFor example, assuming that operations of type `\"Sub\"` take two\r\ninputs `x` and `y`, and return a single output `x - y`, the\r\nfollowing gradient function would be registered:\r\n\r\n```python\r\n@tf.RegisterGradient(\"Sub\")\r\ndef _sub_grad(unused_op, grad):\r\n return grad, tf.Neg(grad)\r\n```\r\n\r\nThe decorator argument `op_type` is the string type of an\r\noperation. This corresponds to the `OpDef.name` field for the proto\r\nthat defines the operation.\r\n\r\n- - -\r\n\r\n#### `tf.RegisterGradient.__init__(op_type)` \r\n\r\nCreates a new decorator with `op_type` as the Operation type.\r\n\r\n##### Args: \r\n\r\n\r\n* `op_type`: The string type of an operation. This corresponds to the\r\n `OpDef.name` field for the proto that defines the operation.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.NoGradient(op_type)` \r\n\r\nSpecifies that ops of type `op_type` do not have a defined gradient.\r\n\r\nThis function is only used when defining a new op type. It may be\r\nused for ops such as `tf.size()` that are not differentiable. For\r\nexample:\r\n\r\n```python\r\ntf.NoGradient(\"Size\")\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `op_type`: The string type of an operation. This corresponds to the\r\n `OpDef.name` field for the proto that defines the operation.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `op_type` is not a string.\r\n\r\n\r\n- - -\r\n\r\n### `class tf.RegisterShape` \r\n\r\nA decorator for registering the shape function for an op type.\r\n\r\nThis decorator is only used when defining a new op type. A shape\r\nfunction is a function from an `Operation` object to a list of\r\n`TensorShape` objects, with one `TensorShape` for each output of the\r\noperation.\r\n\r\nFor example, assuming that operations of type `\"Sub\"` take two\r\ninputs `x` and `y`, and return a single output `x - y`, all with the\r\nsame shape, the following shape function would be registered:\r\n\r\n```python\r\n@tf.RegisterShape(\"Sub\")\r\ndef _sub_shape(op):\r\n return [op.inputs[0].get_shape().merge_with(op.inputs[1].get_shape())]\r\n```\r\n\r\nThe decorator argument `op_type` is the string type of an\r\noperation. This corresponds to the `OpDef.name` field for the proto\r\nthat defines the operation.\r\n- - -\r\n\r\n#### `tf.RegisterShape.__init__(op_type)` \r\n\r\nSaves the \"op_type\" as the Operation type.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.TensorShape` \r\n\r\nRepresents the shape of a `Tensor`.\r\n\r\nA `TensorShape` represents a possibly-partial shape specification for a\r\n`Tensor`. It may be one of the following:\r\n\r\n* *Fully-known shape:* has a known number of dimensions and a known size\r\n for each dimension.\r\n* *Partially-known shape:* has a known number of dimensions, and an unknown\r\n size for one or more dimension.\r\n* *Unknown shape:* has an unknown number of dimensions, and an unknown\r\n size in all dimensions.\r\n\r\nIf a tensor is produced by an operation of type `\"Foo\"`, its shape\r\nmay be inferred if there is a registered shape function for\r\n`\"Foo\"`. See [`tf.RegisterShape()`](../../api_docs/python/framework.md#RegisterShape)\r\nfor details of shape\r\nfunctions and how to register them. Alternatively, the shape may be set\r\nexplicitly using [`Tensor.set_shape()`](../../api_docs/python/framework.md#Tensor.set_shape).\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.merge_with(other)` \r\n\r\nReturns a `TensorShape` combining the information in `self` and `other`.\r\n\r\nThe dimensions in `self` and `other` are merged elementwise,\r\naccording to the rules defined for `Dimension.merge_with()`.\r\n\r\n##### Args: \r\n\r\n\r\n* `other`: Another `TensorShape`.\r\n\r\n##### Returns: \r\n\r\n A `TensorShape` containing the combined information of `self` and\r\n `other`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `self` and `other` are not compatible.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.concatenate(other)` \r\n\r\nReturns the concatenation of the dimension in `self` and `other`.\r\n\r\n*N.B.* If either `self` or `other` is completely unknown,\r\nconcatenation will discard information about the other shape. In\r\nfuture, we might support concatenation that preserves this\r\ninformation for use with slicing.\r\n\r\n##### Args: \r\n\r\n\r\n* `other`: Another `TensorShape`.\r\n\r\n##### Returns: \r\n\r\n A `TensorShape` whose dimensions are the concatenation of the\r\n dimensions in `self` and `other`.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.ndims` \r\n\r\nReturns the rank of this shape, or None if it is unspecified.\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.dims` \r\n\r\nReturns a list of Dimensions, or None if the shape is unspecified.\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.as_list()` \r\n\r\nReturns a list of integers or None for each dimension.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.is_compatible_with(other)` \r\n\r\nReturns True iff `self` is compatible with `other`.\r\n\r\nTwo possibly-partially-defined shapes are compatible if there\r\nexists a fully-defined shape that both shapes can represent. Thus,\r\ncompatibility allows the shape inference code to reason about\r\npartially-defined shapes. For example:\r\n\r\n* TensorShape(None) is compatible with all shapes.\r\n\r\n* TensorShape([None, None]) is compatible with all two-dimensional\r\n shapes, such as TensorShape([32, 784]), and also TensorShape(None). It is\r\n not compatible with, for example, TensorShape([None]) or\r\n TensorShape([None, None, None]).\r\n\r\n* TensorShape([32, None]) is compatible with all two-dimensional shapes\r\n with size 32 in the 0th dimension, and also TensorShape([None, None])\r\n and TensorShape(None). It is not compatible with, for example,\r\n TensorShape([32]), TensorShape([32, None, 1]) or TensorShape([64, None]).\r\n\r\n* TensorShape([32, 784]) is compatible with itself, and also\r\n TensorShape([32, None]), TensorShape([None, 784]), TensorShape([None,\r\n None]) and TensorShape(None). It is not compatible with, for example,\r\n TensorShape([32, 1, 784]) or TensorShape([None]).\r\n\r\nThe compatibility relation is reflexive and symmetric, but not\r\ntransitive. For example, TensorShape([32, 784]) is compatible with\r\nTensorShape(None), and TensorShape(None) is compatible with\r\nTensorShape([4, 4]), but TensorShape([32, 784]) is not compatible with\r\nTensorShape([4, 4]).\r\n\r\n##### Args: \r\n\r\n\r\n* `other`: Another TensorShape.\r\n\r\n##### Returns: \r\n\r\n True iff `self` is compatible with `other`.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.is_fully_defined()` \r\n\r\nReturns True iff `self` is fully defined in every dimension.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.with_rank(rank)` \r\n\r\nReturns a shape based on `self` with the given rank.\r\n\r\nThis method promotes a completely unknown shape to one with a\r\nknown rank.\r\n\r\n##### Args: \r\n\r\n\r\n* `rank`: An integer.\r\n\r\n##### Returns: \r\n\r\n A shape that is at least as specific as `self` with the given rank.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `self` does not represent a shape with the given `rank`.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.with_rank_at_least(rank)` \r\n\r\nReturns a shape based on `self` with at least the given rank.\r\n\r\n##### Args: \r\n\r\n\r\n* `rank`: An integer.\r\n\r\n##### Returns: \r\n\r\n A shape that is at least as specific as `self` with at least the given\r\n rank.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `self` does not represent a shape with at least the given\r\n `rank`.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.with_rank_at_most(rank)` \r\n\r\nReturns a shape based on `self` with at most the given rank.\r\n\r\n##### Args: \r\n\r\n\r\n* `rank`: An integer.\r\n\r\n##### Returns: \r\n\r\n A shape that is at least as specific as `self` with at most the given\r\n rank.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `self` does not represent a shape with at most the given\r\n `rank`.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.assert_has_rank(rank)` \r\n\r\nRaises an exception if `self` is not compatible with the given `rank`.\r\n\r\n##### Args: \r\n\r\n\r\n* `rank`: An integer.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `self` does not represent a shape with the given `rank`.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.assert_same_rank(other)` \r\n\r\nRaises an exception if `self` and `other` do not have compatible ranks.\r\n\r\n##### Args: \r\n\r\n\r\n* `other`: Another `TensorShape`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `self` and `other` do not represent shapes with the\r\n same rank.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.assert_is_compatible_with(other)` \r\n\r\nRaises exception if `self` and `other` do not represent the same shape.\r\n\r\nThis method can be used to assert that there exists a shape that both\r\n`self` and `other` represent.\r\n\r\n##### Args: \r\n\r\n\r\n* `other`: Another TensorShape.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `self` and `other` do not represent the same shape.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.assert_is_fully_defined()` \r\n\r\nRaises an exception if `self` is not fully defined in every dimension.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `self` does not have a known value for every dimension.\r\n\r\n\r\n\r\n#### Other Methods \r\n- - -\r\n\r\n#### `tf.TensorShape.__init__(dims)` \r\n\r\nCreates a new TensorShape with the given dimensions.\r\n\r\n##### Args: \r\n\r\n\r\n* `dims`: A list of Dimensions, or None if the shape is unspecified.\r\n* `DEPRECATED`: A single integer is treated as a singleton list.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.as_dimension_list()` \r\n\r\nDEPRECATED: use as_list().\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TensorShape.num_elements()` \r\n\r\nReturns the total number of elements, or none for incomplete shapes.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.Dimension` \r\n\r\nRepresents the value of one dimension in a TensorShape.\r\n- - -\r\n\r\n#### `tf.Dimension.__init__(value)` \r\n\r\nCreates a new Dimension with the given value.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Dimension.assert_is_compatible_with(other)` \r\n\r\nRaises an exception if `other` is not compatible with this Dimension.\r\n\r\n##### Args: \r\n\r\n\r\n* `other`: Another Dimension.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `self` and `other` are not compatible (see\r\n is_compatible_with).\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Dimension.is_compatible_with(other)` \r\n\r\nReturns true if `other` is compatible with this Dimension.\r\n\r\nTwo known Dimensions are compatible if they have the same value.\r\nAn unknown Dimension is compatible with all other Dimensions.\r\n\r\n##### Args: \r\n\r\n\r\n* `other`: Another Dimension.\r\n\r\n##### Returns: \r\n\r\n True if this Dimension and `other` are compatible.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Dimension.merge_with(other)` \r\n\r\nReturns a Dimension that combines the information in `self` and `other`.\r\n\r\nDimensions are combined as follows:\r\n\r\n Dimension(n) .merge_with(Dimension(n)) == Dimension(n)\r\n Dimension(n) .merge_with(Dimension(None)) == Dimension(n)\r\n Dimension(None).merge_with(Dimension(n)) == Dimension(n)\r\n Dimension(None).merge_with(Dimension(None)) == Dimension(None)\r\n Dimension(n) .merge_with(Dimension(m)) raises ValueError for n != m\r\n\r\n##### Args: \r\n\r\n\r\n* `other`: Another Dimension.\r\n\r\n##### Returns: \r\n\r\n A Dimension containing the combined information of `self` and\r\n `other`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `self` and `other` are not compatible (see\r\n is_compatible_with).\r\n\r\n\r\n- - -\r\n\r\n#### `tf.Dimension.value` \r\n\r\nThe value of this dimension, or None if it is unknown.\r\n\r\n\r\n- - -\r\n\r\n### `tf.op_scope(values, name, default_name)` \r\n\r\nReturns a context manager for use when defining a Python op.\r\n\r\nThis context manager validates that the given `values` are from the\r\nsame graph, ensures that that graph is the default graph, and pushes a\r\nname scope.\r\n\r\nFor example, to define a new Python op called `my_op`:\r\n\r\n```python\r\ndef my_op(a, b, c, name=None):\r\n with tf.op_scope([a, b, c], name, \"MyOp\") as scope:\r\n a = tf.convert_to_tensor(a, name=\"a\")\r\n b = tf.convert_to_tensor(b, name=\"b\")\r\n c = tf.convert_to_tensor(c, name=\"c\")\r\n # Define some computation that uses `a`, `b`, and `c`.\r\n return foo_op(..., name=scope)\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `values`: The list of `Tensor` arguments that are passed to the op function.\r\n* `name`: The name argument that is passed to the op function.\r\n* `default_name`: The default name to use if the `name` argument is `None`.\r\n\r\n##### Returns: \r\n\r\n A context manager for use in defining a Python op.\r\n\r\n\r\n- - -\r\n\r\n### `tf.get_seed(op_seed)` \r\n\r\nReturns the local seeds an operation should use given an op-specific seed.\r\n\r\nGiven operation-specific seed, `op_seed`, this helper function returns two\r\nseeds derived from graph-level and op-level seeds. Many random operations\r\ninternally use the two seeds to allow user to change the seed globally for a\r\ngraph, or for only specific operations.\r\n\r\nFor details on how the graph-level seed interacts with op seeds, see\r\n[`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed).\r\n\r\n##### Args: \r\n\r\n\r\n* `op_seed`: integer.\r\n\r\n##### Returns: \r\n\r\n A tuple of two integers that should be used for the local seed of this\r\n operation.\r\n\r\n\r\n"
},
{
"path": "SOURCE/api_docs/python/image.md",
"content": "\r\n\r\n# Images \r\n\r\nNote: Functions taking `Tensor` arguments can also take anything accepted by\r\n[`tf.convert_to_tensor`](../../api_docs/python/framework.md#convert_to_tensor).\r\n\r\n\r\n## Contents\r\n### [Images](#AUTOGENERATED-images)\r\n* [Encoding and Decoding](#AUTOGENERATED-encoding-and-decoding)\r\n * [`tf.image.decode_jpeg(contents, channels=None, ratio=None, fancy_upscaling=None, try_recover_truncated=None, acceptable_fraction=None, name=None)`](#decode_jpeg)\r\n * [`tf.image.encode_jpeg(image, format=None, quality=None, progressive=None, optimize_size=None, chroma_downsampling=None, density_unit=None, x_density=None, y_density=None, xmp_metadata=None, name=None)`](#encode_jpeg)\r\n * [`tf.image.decode_png(contents, channels=None, name=None)`](#decode_png)\r\n * [`tf.image.encode_png(image, compression=None, name=None)`](#encode_png)\r\n* [Resizing](#AUTOGENERATED-resizing)\r\n * [`tf.image.resize_images(images, new_height, new_width, method=0)`](#resize_images)\r\n * [`tf.image.resize_area(images, size, name=None)`](#resize_area)\r\n * [`tf.image.resize_bicubic(images, size, name=None)`](#resize_bicubic)\r\n * [`tf.image.resize_bilinear(images, size, name=None)`](#resize_bilinear)\r\n * [`tf.image.resize_nearest_neighbor(images, size, name=None)`](#resize_nearest_neighbor)\r\n* [Cropping](#AUTOGENERATED-cropping)\r\n * [`tf.image.resize_image_with_crop_or_pad(image, target_height, target_width)`](#resize_image_with_crop_or_pad)\r\n * [`tf.image.pad_to_bounding_box(image, offset_height, offset_width, target_height, target_width)`](#pad_to_bounding_box)\r\n * [`tf.image.crop_to_bounding_box(image, offset_height, offset_width, target_height, target_width)`](#crop_to_bounding_box)\r\n * [`tf.image.random_crop(image, size, seed=None, name=None)`](#random_crop)\r\n * [`tf.image.extract_glimpse(input, size, offsets, centered=None, normalized=None, uniform_noise=None, name=None)`](#extract_glimpse)\r\n* [Flipping and Transposing](#AUTOGENERATED-flipping-and-transposing)\r\n * [`tf.image.flip_up_down(image)`](#flip_up_down)\r\n * [`tf.image.random_flip_up_down(image, seed=None)`](#random_flip_up_down)\r\n * [`tf.image.flip_left_right(image)`](#flip_left_right)\r\n * [`tf.image.random_flip_left_right(image, seed=None)`](#random_flip_left_right)\r\n * [`tf.image.transpose_image(image)`](#transpose_image)\r\n* [Image Adjustments](#AUTOGENERATED-image-adjustments)\r\n * [`tf.image.adjust_brightness(image, delta, min_value=None, max_value=None)`](#adjust_brightness)\r\n * [`tf.image.random_brightness(image, max_delta, seed=None)`](#random_brightness)\r\n * [`tf.image.adjust_contrast(images, contrast_factor, min_value=None, max_value=None)`](#adjust_contrast)\r\n * [`tf.image.random_contrast(image, lower, upper, seed=None)`](#random_contrast)\r\n * [`tf.image.per_image_whitening(image)`](#per_image_whitening)\r\n\r\n\r\n\r\n\r\n## Encoding and Decoding \r\n\r\nTensorFlow provides Ops to decode and encode JPEG and PNG formats. Encoded\r\nimages are represented by scalar string Tensors, decoded images by 3-D uint8\r\ntensors of shape `[height, width, channels]`.\r\n\r\nThe encode and decode Ops apply to one image at a time. Their input and output\r\nare all of variable size. If you need fixed size images, pass the output of\r\nthe decode Ops to one of the cropping and resizing Ops.\r\n\r\nNote: The PNG encode and decode Ops support RGBA, but the conversions Ops\r\npresently only support RGB, HSV, and GrayScale.\r\n\r\n- - -\r\n\r\n### `tf.image.decode_jpeg(contents, channels=None, ratio=None, fancy_upscaling=None, try_recover_truncated=None, acceptable_fraction=None, name=None)` \r\n\r\nDecode a JPEG-encoded image to a uint8 tensor.\r\n\r\nThe attr `channels` indicates the desired number of color channels for the\r\ndecoded image.\r\n\r\nAccepted values are:\r\n\r\n* 0: Use the number of channels in the JPEG-encoded image.\r\n* 1: output a grayscale image.\r\n* 3: output an RGB image.\r\n\r\nIf needed, the JPEG-encoded image is transformed to match the requested number\r\nof color channels.\r\n\r\nThe attr `ratio` allows downscaling the image by an integer factor during\r\ndecoding. Allowed values are: 1, 2, 4, and 8. This is much faster than\r\ndownscaling the image later.\r\n\r\n##### Args: \r\n\r\n\r\n* `contents`: A `Tensor` of type `string`. 0-D. The JPEG-encoded image.\r\n* `channels`: An optional `int`. Defaults to `0`.\r\n Number of color channels for the decoded image.\r\n* `ratio`: An optional `int`. Defaults to `1`. Downscaling ratio.\r\n* `fancy_upscaling`: An optional `bool`. Defaults to `True`.\r\n If true use a slower but nicer upscaling of the\r\n chroma planes (yuv420/422 only).\r\n* `try_recover_truncated`: An optional `bool`. Defaults to `False`.\r\n If true try to recover an image from truncated input.\r\n* `acceptable_fraction`: An optional `float`. Defaults to `1`.\r\n The minimum required fraction of lines before a truncated\r\n input is accepted.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `uint8`. 3-D with shape `[height, width, channels]`..\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.encode_jpeg(image, format=None, quality=None, progressive=None, optimize_size=None, chroma_downsampling=None, density_unit=None, x_density=None, y_density=None, xmp_metadata=None, name=None)` \r\n\r\nJPEG-encode an image.\r\n\r\n`image` is a 3-D uint8 Tensor of shape `[height, width, channels]`.\r\n\r\nThe attr `format` can be used to override the color format of the encoded\r\noutput. Values can be:\r\n\r\n* `''`: Use a default format based on the number of channels in the image.\r\n* `grayscale`: Output a grayscale JPEG image. The `channels` dimension\r\n of `image` must be 1.\r\n* `rgb`: Output an RGB JPEG image. The `channels` dimension\r\n of `image` must be 3.\r\n\r\nIf `format` is not specified or is the empty string, a default format is picked\r\nin function of the number of channels in `image`:\r\n\r\n* 1: Output a grayscale image.\r\n* 3: Output an RGB image.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: A `Tensor` of type `uint8`.\r\n 3-D with shape `[height, width, channels]`.\r\n* `format`: An optional `string` from: `\"\", \"grayscale\", \"rgb\"`. Defaults to `\"\"`.\r\n Per pixel image format.\r\n* `quality`: An optional `int`. Defaults to `95`.\r\n Quality of the compression from 0 to 100 (higher is better and slower).\r\n* `progressive`: An optional `bool`. Defaults to `False`.\r\n If True, create a JPEG that loads progressively (coarse to fine).\r\n* `optimize_size`: An optional `bool`. Defaults to `False`.\r\n If True, spend CPU/RAM to reduce size with no quality change.\r\n* `chroma_downsampling`: An optional `bool`. Defaults to `True`.\r\n See http://en.wikipedia.org/wiki/Chroma_subsampling.\r\n* `density_unit`: An optional `string` from: `\"in\", \"cm\"`. Defaults to `\"in\"`.\r\n Unit used to specify `x_density` and `y_density`:\r\n pixels per inch (`'in'`) or centimeter (`'cm'`).\r\n* `x_density`: An optional `int`. Defaults to `300`.\r\n Horizontal pixels per density unit.\r\n* `y_density`: An optional `int`. Defaults to `300`.\r\n Vertical pixels per density unit.\r\n* `xmp_metadata`: An optional `string`. Defaults to `\"\"`.\r\n If not empty, embed this XMP metadata in the image header.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `string`. 0-D. JPEG-encoded image.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.decode_png(contents, channels=None, name=None)` \r\n\r\nDecode a PNG-encoded image to a uint8 tensor.\r\n\r\nThe attr `channels` indicates the desired number of color channels for the\r\ndecoded image.\r\n\r\nAccepted values are:\r\n\r\n* 0: Use the number of channels in the PNG-encoded image.\r\n* 1: output a grayscale image.\r\n* 3: output an RGB image.\r\n* 4: output an RGBA image.\r\n\r\nIf needed, the PNG-encoded image is transformed to match the requested number\r\nof color channels.\r\n\r\n##### Args: \r\n\r\n\r\n* `contents`: A `Tensor` of type `string`. 0-D. The PNG-encoded image.\r\n* `channels`: An optional `int`. Defaults to `0`.\r\n Number of color channels for the decoded image.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `uint8`. 3-D with shape `[height, width, channels]`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.encode_png(image, compression=None, name=None)` \r\n\r\nPNG-encode an image.\r\n\r\n`image` is a 3-D uint8 Tensor of shape `[height, width, channels]` where\r\n`channels` is:\r\n\r\n* 1: for grayscale.\r\n* 3: for RGB.\r\n* 4: for RGBA.\r\n\r\nThe ZLIB compression level, `compression`, can be -1 for the PNG-encoder\r\ndefault or a value from 0 to 9. 9 is the highest compression level, generating\r\nthe smallest output, but is slower.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: A `Tensor` of type `uint8`.\r\n 3-D with shape `[height, width, channels]`.\r\n* `compression`: An optional `int`. Defaults to `-1`. Compression level.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `string`. 0-D. PNG-encoded image.\r\n\r\n\r\n\r\n## Resizing \r\n\r\nThe resizing Ops accept input images as tensors of several types. They always\r\noutput resized images as float32 tensors.\r\n\r\nThe convenience function [resize_images()](#resize_images) supports both 4-D\r\nand 3-D tensors as input and output. 4-D tensors are for batches of images,\r\n3-D tensors for individual images.\r\n\r\nOther resizing Ops only support 3-D individual images as input:\r\n[resize_area](#resize_area), [resize_bicubic](#resize_bicubic),\r\n[resize_bilinear](#resize_bilinear),\r\n[resize_nearest_neighbor](#resize_nearest_neighbor).\r\n\r\nExample:\r\n\r\n```python\r\n# Decode a JPG image and resize it to 299 by 299.\r\nimage = tf.image.decode_jpeg(...)\r\nresized_image = tf.image.resize_bilinear(image, [299, 299])\r\n```\r\n\r\nMaybe refer to the Queue examples that show how to add images to a Queue\r\nafter resizing them to a fixed size, and how to dequeue batches of resized\r\nimages from the Queue.\r\n\r\n- - -\r\n\r\n### `tf.image.resize_images(images, new_height, new_width, method=0)` \r\n\r\nResize `images` to `new_width`, `new_height` using the specified `method`.\r\n\r\nResized images will be distorted if their original aspect ratio is not\r\nthe same as `new_width`, `new_height`. To avoid distortions see\r\n[resize_image_with_crop_or_pad](#resize_image_with_crop_or_pad).\r\n\r\n`method` can be one of:\r\n\r\n* ResizeMethod.BILINEAR: [Bilinear interpolation.]\r\n (https://en.wikipedia.org/wiki/Bilinear_interpolation)\r\n* ResizeMethod.NEAREST_NEIGHBOR: [Nearest neighbor interpolation.]\r\n (https://en.wikipedia.org/wiki/Nearest-neighbor_interpolation)\r\n* ResizeMethod.BICUBIC: [Bicubic interpolation.]\r\n (https://en.wikipedia.org/wiki/Bicubic_interpolation)\r\n* ResizeMethod.AREA: Area interpolation.\r\n\r\n##### Args: \r\n\r\n\r\n* `images`: 4-D Tensor of shape `[batch, height, width, channels]` or\r\n 3-D Tensor of shape `[height, width, channels]`.\r\n* `new_height`: integer.\r\n* `new_width`: integer.\r\n* `method`: ResizeMethod. Defaults to `ResizeMethod.BILINEAR`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if the shape of `images` is incompatible with the\r\n shape arguments to this function\r\n* `ValueError`: if an unsupported resize method is specified.\r\n\r\n##### Returns: \r\n\r\n If `images` was 4-D, a 4-D float Tensor of shape\r\n `[batch, new_height, new_width, channels]`.\r\n If `images` was 3-D, a 3-D float Tensor of shape\r\n `[new_height, new_width, channels]`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.resize_area(images, size, name=None)` \r\n\r\nResize `images` to `size` using area interpolation.\r\n\r\nInput images can be of different types but output images are always float.\r\n\r\n##### Args: \r\n\r\n\r\n* `images`: A `Tensor`. Must be one of the following types: `uint8`, `int8`, `int32`, `float32`, `float64`.\r\n 4-D with shape `[batch, height, width, channels]`.\r\n* `size`: A 1-D int32 Tensor of 2 elements: `new_height, new_width`. The\r\n new size for the images.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `float32`. 4-D with shape\r\n `[batch, new_height, new_width, channels]`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.resize_bicubic(images, size, name=None)` \r\n\r\nResize `images` to `size` using bicubic interpolation.\r\n\r\nInput images can be of different types but output images are always float.\r\n\r\n##### Args: \r\n\r\n\r\n* `images`: A `Tensor`. Must be one of the following types: `uint8`, `int8`, `int32`, `float32`, `float64`.\r\n 4-D with shape `[batch, height, width, channels]`.\r\n* `size`: A 1-D int32 Tensor of 2 elements: `new_height, new_width`. The\r\n new size for the images.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `float32`. 4-D with shape\r\n `[batch, new_height, new_width, channels]`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.resize_bilinear(images, size, name=None)` \r\n\r\nResize `images` to `size` using bilinear interpolation.\r\n\r\nInput images can be of different types but output images are always float.\r\n\r\n##### Args: \r\n\r\n\r\n* `images`: A `Tensor`. Must be one of the following types: `uint8`, `int8`, `int32`, `float32`, `float64`.\r\n 4-D with shape `[batch, height, width, channels]`.\r\n* `size`: A 1-D int32 Tensor of 2 elements: `new_height, new_width`. The\r\n new size for the images.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `float32`. 4-D with shape\r\n `[batch, new_height, new_width, channels]`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.resize_nearest_neighbor(images, size, name=None)` \r\n\r\nResize `images` to `size` using nearest neighbor interpolation.\r\n\r\nInput images can be of different types but output images are always float.\r\n\r\n##### Args: \r\n\r\n\r\n* `images`: A `Tensor`. Must be one of the following types: `uint8`, `int8`, `int32`, `float32`, `float64`.\r\n 4-D with shape `[batch, height, width, channels]`.\r\n* `size`: A 1-D int32 Tensor of 2 elements: `new_height, new_width`. The\r\n new size for the images.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `images`. 4-D with shape\r\n `[batch, new_height, new_width, channels]`.\r\n\r\n\r\n\r\n\r\n## Cropping \r\n\r\n- - -\r\n\r\n### `tf.image.resize_image_with_crop_or_pad(image, target_height, target_width)` \r\n\r\nCrops and/or pads an image to a target width and height.\r\n\r\nResizes an image to a target width and height by either centrally\r\ncropping the image or padding it evenly with zeros.\r\n\r\nIf `width` or `height` is greater than the specified `target_width` or\r\n`target_height` respectively, this op centrally crops along that dimension.\r\nIf `width` or `height` is smaller than the specified `target_width` or\r\n`target_height` respectively, this op centrally pads with 0 along that\r\ndimension.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: 3-D tensor of shape [height, width, channels]\r\n* `target_height`: Target height.\r\n* `target_width`: Target width.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if `target_height` or `target_width` are zero or negative.\r\n\r\n##### Returns: \r\n\r\n Cropped and/or padded image of shape\r\n `[target_height, target_width, channels]`\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.pad_to_bounding_box(image, offset_height, offset_width, target_height, target_width)` \r\n\r\nPad `image` with zeros to the specified `height` and `width`.\r\n\r\nAdds `offset_height` rows of zeros on top, `offset_width` columns of\r\nzeros on the left, and then pads the image on the bottom and right\r\nwith zeros until it has dimensions `target_height`, `target_width`.\r\n\r\nThis op does nothing if `offset_*` is zero and the image already has size\r\n`target_height` by `target_width`.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: 3-D tensor with shape `[height, width, channels]`\r\n* `offset_height`: Number of rows of zeros to add on top.\r\n* `offset_width`: Number of columns of zeros to add on the left.\r\n* `target_height`: Height of output image.\r\n* `target_width`: Width of output image.\r\n\r\n##### Returns: \r\n\r\n 3-D tensor of shape `[target_height, target_width, channels]`\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If the shape of `image` is incompatible with the `offset_*` or\r\n `target_*` arguments\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.crop_to_bounding_box(image, offset_height, offset_width, target_height, target_width)` \r\n\r\nCrops an image to a specified bounding box.\r\n\r\nThis op cuts a rectangular part out of `image`. The top-left corner of the\r\nreturned image is at `offset_height, offset_width` in `image`, and its\r\nlower-right corner is at\r\n`offset_height + target_height, offset_width + target_width'.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: 3-D tensor with shape `[height, width, channels]`\r\n* `offset_height`: Vertical coordinate of the top-left corner of the result in\r\n the input.\r\n* `offset_width`: Horizontal coordinate of the top-left corner of the result in\r\n the input.\r\n* `target_height`: Height of the result.\r\n* `target_width`: Width of the result.\r\n\r\n##### Returns: \r\n\r\n 3-D tensor of image with shape `[target_height, target_width, channels]`\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If the shape of `image` is incompatible with the `offset_*` or\r\n `target_*` arguments\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.random_crop(image, size, seed=None, name=None)` \r\n\r\nRandomly crops `image` to size `[target_height, target_width]`.\r\n\r\nThe offset of the output within `image` is uniformly random. `image` always\r\nfully contains the result.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: 3-D tensor of shape `[height, width, channels]`\r\n* `size`: 1-D tensor with two elements, specifying target `[height, width]`\r\n* `seed`: A Python integer. Used to create a random seed. See\r\n [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\n for behavior.\r\n* `name`: A name for this operation (optional).\r\n\r\n##### Returns: \r\n\r\n A cropped 3-D tensor of shape `[target_height, target_width, channels]`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.extract_glimpse(input, size, offsets, centered=None, normalized=None, uniform_noise=None, name=None)` \r\n\r\nExtracts a glimpse from the input tensor.\r\n\r\nReturns a set of windows called glimpses extracted at location `offsets`\r\nfrom the input tensor. If the windows only partially overlaps the inputs, the\r\nnon overlapping areas will be filled with random noise.\r\n\r\nThe result is a 4-D tensor of shape `[batch_size, glimpse_height,\r\nglimpse_width, channels]`. The channels and batch dimensions are the same as that\r\nof the input tensor. The height and width of the output windows are\r\nspecified in the `size` parameter.\r\n\r\nThe argument `normalized` and `centered` controls how the windows are built:\r\n* If the coordinates are normalized but not centered, 0.0 and 1.0\r\n correspond to the minimum and maximum of each height and width dimension.\r\n* If the coordinates are both normalized and centered, they range from -1.0 to\r\n 1.0. The coordinates (-1.0, -1.0) correspond to the upper left corner, the\r\n lower right corner is located at (1.0, 1.0) and the center is at (0, 0).\r\n* If the coordinates are not normalized they are interpreted as numbers of pixels.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor` of type `float32`.\r\n A 4-D float tensor of shape `[batch_size, height, width, channels]`.\r\n* `size`: A `Tensor` of type `int32`.\r\n A 1-D tensor of 2 elements containing the size of the glimpses to extract.\r\n The glimpse height must be specified first, following by the glimpse width.\r\n* `offsets`: A `Tensor` of type `float32`.\r\n A 2-D integer tensor of shape `[batch_size, 2]` containing the x, y\r\n locations of the center of each window.\r\n* `centered`: An optional `bool`. Defaults to `True`.\r\n indicates if the offset coordinates are centered relative to\r\n the image, in which case the (0, 0) offset is relative to the center of the\r\n input images. If false, the (0,0) offset corresponds to the upper left corner\r\n of the input images.\r\n* `normalized`: An optional `bool`. Defaults to `True`.\r\n indicates if the offset coordinates are normalized.\r\n* `uniform_noise`: An optional `bool`. Defaults to `True`.\r\n indicates if the noise should be generated using a\r\n uniform distribution or a gaussian distribution.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `float32`.\r\n A tensor representing the glimpses `[batch_size, glimpse_height,\r\n glimpse_width, channels]`.\r\n\r\n\r\n\r\n## Flipping and Transposing \r\n\r\n- - -\r\n\r\n### `tf.image.flip_up_down(image)` \r\n\r\nFlip an image horizontally (upside down).\r\n\r\nOutputs the contents of `image` flipped along the first dimension, which is\r\n`height`.\r\n\r\nSee also `reverse()`.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: A 3-D tensor of shape `[height, width, channels].`\r\n\r\n##### Returns: \r\n\r\n A 3-D tensor of the same type and shape as `image`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if the shape of `image` not supported.\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.random_flip_up_down(image, seed=None)` \r\n\r\nRandomly flips an image vertically (upside down).\r\n\r\nWith a 1 in 2 chance, outputs the contents of `image` flipped along the first\r\ndimension, which is `height`. Otherwise output the image as-is.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: A 3-D tensor of shape `[height, width, channels].`\r\n* `seed`: A Python integer. Used to create a random seed. See\r\n [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\n for behavior.\r\n\r\n##### Returns: \r\n\r\n A 3-D tensor of the same type and shape as `image`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if the shape of `image` not supported.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.flip_left_right(image)` \r\n\r\nFlip an image horizontally (left to right).\r\n\r\nOutputs the contents of `image` flipped along the second dimension, which is\r\n`width`.\r\n\r\nSee also `reverse()`.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: A 3-D tensor of shape `[height, width, channels].`\r\n\r\n##### Returns: \r\n\r\n A 3-D tensor of the same type and shape as `image`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if the shape of `image` not supported.\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.random_flip_left_right(image, seed=None)` \r\n\r\nRandomly flip an image horizontally (left to right).\r\n\r\nWith a 1 in 2 chance, outputs the contents of `image` flipped along the\r\nsecond dimension, which is `width`. Otherwise output the image as-is.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: A 3-D tensor of shape `[height, width, channels].`\r\n* `seed`: A Python integer. Used to create a random seed. See\r\n [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\n for behavior.\r\n\r\n##### Returns: \r\n\r\n A 3-D tensor of the same type and shape as `image`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if the shape of `image` not supported.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.transpose_image(image)` \r\n\r\nTranspose an image by swapping the first and second dimension.\r\n\r\nSee also `transpose()`.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: 3-D tensor of shape `[height, width, channels]`\r\n\r\n##### Returns: \r\n\r\n A 3-D tensor of shape `[width, height, channels]`\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if the shape of `image` not supported.\r\n\r\n\r\n\r\n## Image Adjustments \r\n\r\nTensorFlow provides functions to adjust images in various ways: brightness,\r\ncontrast, hue, and saturation. Each adjustment can be done with predefined\r\nparameters or with random parameters picked from predefined intervals. Random\r\nadjustments are often useful to expand a training set and reduce overfitting.\r\n\r\n- - -\r\n\r\n### `tf.image.adjust_brightness(image, delta, min_value=None, max_value=None)` \r\n\r\nAdjust the brightness of RGB or Grayscale images.\r\n\r\nThe value `delta` is added to all components of the tensor `image`. `image`\r\nand `delta` are cast to `float` before adding, and the resulting values are\r\nclamped to `[min_value, max_value]`. Finally, the result is cast back to\r\n`images.dtype`.\r\n\r\nIf `min_value` or `max_value` are not given, they are set to the minimum and\r\nmaximum allowed values for `image.dtype` respectively.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: A tensor.\r\n* `delta`: A scalar. Amount to add to the pixel values.\r\n* `min_value`: Minimum value for output.\r\n* `max_value`: Maximum value for output.\r\n\r\n##### Returns: \r\n\r\n A tensor of the same shape and type as `image`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.random_brightness(image, max_delta, seed=None)` \r\n\r\nAdjust the brightness of images by a random factor.\r\n\r\nEquivalent to `adjust_brightness()` using a `delta` randomly picked in the\r\ninterval `[-max_delta, max_delta)`.\r\n\r\nNote that `delta` is picked as a float. Because for integer type images,\r\nthe brightness adjusted result is rounded before casting, integer images may\r\nhave modifications in the range `[-max_delta,max_delta]`.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: 3-D tensor of shape `[height, width, channels]`.\r\n* `max_delta`: float, must be non-negative.\r\n* `seed`: A Python integer. Used to create a random seed. See\r\n [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\n for behavior.\r\n\r\n##### Returns: \r\n\r\n 3-D tensor of images of shape `[height, width, channels]`\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if max_delta is negative.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.adjust_contrast(images, contrast_factor, min_value=None, max_value=None)` \r\n\r\nAdjust contrast of RGB or grayscale images.\r\n\r\n`images` is a tensor of at least 3 dimensions. The last 3 dimensions are\r\ninterpreted as `[height, width, channels]`. The other dimensions only\r\nrepresent a collection of images, such as `[batch, height, width, channels].`\r\n\r\nContrast is adjusted independently for each channel of each image.\r\n\r\nFor each channel, this Op first computes the mean of the image pixels in the\r\nchannel and then adjusts each component `x` of each pixel to\r\n`(x - mean) * contrast_factor + mean`.\r\n\r\nThe adjusted values are then clipped to fit in the `[min_value, max_value]`\r\ninterval. If `min_value` or `max_value` is not given, it is replaced with the\r\nminimum and maximum values for the data type of `images` respectively.\r\n\r\nThe contrast-adjusted image is always computed as `float`, and it is\r\ncast back to its original type after clipping.\r\n\r\n##### Args: \r\n\r\n\r\n* `images`: Images to adjust. At least 3-D.\r\n* `contrast_factor`: A float multiplier for adjusting contrast.\r\n* `min_value`: Minimum value for clipping the adjusted pixels.\r\n* `max_value`: Maximum value for clipping the adjusted pixels.\r\n\r\n##### Returns: \r\n\r\n The constrast-adjusted image or images.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if the arguments are invalid.\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.random_contrast(image, lower, upper, seed=None)` \r\n\r\nAdjust the contrase of an image by a random factor.\r\n\r\nEquivalent to `adjust_constrast()` but uses a `contrast_factor` randomly\r\npicked in the interval `[lower, upper]`.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: 3-D tensor of shape `[height, width, channels]`.\r\n* `lower`: float. Lower bound for the random contrast factor.\r\n* `upper`: float. Upper bound for the random contrast factor.\r\n* `seed`: A Python integer. Used to create a random seed. See\r\n [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\n for behavior.\r\n\r\n##### Returns: \r\n\r\n 3-D tensor of shape `[height, width, channels]`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if `upper <= lower` or if `lower < 0`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.image.per_image_whitening(image)` \r\n\r\nLinearly scales `image` to have zero mean and unit norm.\r\n\r\nThis op computes `(x - mean) / adjusted_stddev`, where `mean` is the average\r\nof all values in image, and\r\n`adjusted_stddev = max(stddev, 1.0/srqt(image.NumElements()))`.\r\n\r\n`stddev` is the standard deviation of all values in `image`. It is capped\r\naway from zero to protect against division by 0 when handling uniform images.\r\n\r\nNote that this implementation is limited:\r\n* It only whitens based on the statistics of an individual image.\r\n* It does not take into account the covariance structure.\r\n\r\n##### Args: \r\n\r\n\r\n* `image`: 3-D tensor of shape `[height, width, channels]`.\r\n\r\n##### Returns: \r\n\r\n The whitened image with same shape as `image`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if the shape of 'image' is incompatible with this function.\r\n\r\n\r\n"
},
{
"path": "SOURCE/api_docs/python/index.md",
"content": "\r\n\r\n# TensorFlow Python reference documentation \r\n\r\n* **[Building Graphs](../../api_docs/python/framework.md)**:\r\n * [`add_to_collection`](../../api_docs/python/framework.md#add_to_collection)\r\n * [`as_dtype`](../../api_docs/python/framework.md#as_dtype)\r\n * [`control_dependencies`](../../api_docs/python/framework.md#control_dependencies)\r\n * [`convert_to_tensor`](../../api_docs/python/framework.md#convert_to_tensor)\r\n * [`device`](../../api_docs/python/framework.md#device)\r\n * [`Dimension`](../../api_docs/python/framework.md#Dimension)\r\n * [`DType`](../../api_docs/python/framework.md#DType)\r\n * [`get_collection`](../../api_docs/python/framework.md#get_collection)\r\n * [`get_default_graph`](../../api_docs/python/framework.md#get_default_graph)\r\n * [`get_seed`](../../api_docs/python/framework.md#get_seed)\r\n * [`Graph`](../../api_docs/python/framework.md#Graph)\r\n * [`GraphKeys`](../../api_docs/python/framework.md#GraphKeys)\r\n * [`import_graph_def`](../../api_docs/python/framework.md#import_graph_def)\r\n * [`name_scope`](../../api_docs/python/framework.md#name_scope)\r\n * [`NoGradient`](../../api_docs/python/framework.md#NoGradient)\r\n * [`op_scope`](../../api_docs/python/framework.md#op_scope)\r\n * [`Operation`](../../api_docs/python/framework.md#Operation)\r\n * [`RegisterGradient`](../../api_docs/python/framework.md#RegisterGradient)\r\n * [`RegisterShape`](../../api_docs/python/framework.md#RegisterShape)\r\n * [`Tensor`](../../api_docs/python/framework.md#Tensor)\r\n * [`TensorShape`](../../api_docs/python/framework.md#TensorShape)\r\n\r\n* **[Constants, Sequences, and Random Values](../../api_docs/python/constant_op.md)**:\r\n * [`constant`](../../api_docs/python/constant_op.md#constant)\r\n * [`fill`](../../api_docs/python/constant_op.md#fill)\r\n * [`linspace`](../../api_docs/python/constant_op.md#linspace)\r\n * [`ones`](../../api_docs/python/constant_op.md#ones)\r\n * [`ones_like`](../../api_docs/python/constant_op.md#ones_like)\r\n * [`random_normal`](../../api_docs/python/constant_op.md#random_normal)\r\n * [`random_shuffle`](../../api_docs/python/constant_op.md#random_shuffle)\r\n * [`random_uniform`](../../api_docs/python/constant_op.md#random_uniform)\r\n * [`range`](../../api_docs/python/constant_op.md#range)\r\n * [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\n * [`truncated_normal`](../../api_docs/python/constant_op.md#truncated_normal)\r\n * [`zeros`](../../api_docs/python/constant_op.md#zeros)\r\n * [`zeros_like`](../../api_docs/python/constant_op.md#zeros_like)\r\n\r\n* **[Variables](../../api_docs/python/state_ops.md)**:\r\n * [`all_variables`](../../api_docs/python/state_ops.md#all_variables)\r\n * [`assert_variables_initialized`](../../api_docs/python/state_ops.md#assert_variables_initialized)\r\n * [`assign`](../../api_docs/python/state_ops.md#assign)\r\n * [`assign_add`](../../api_docs/python/state_ops.md#assign_add)\r\n * [`assign_sub`](../../api_docs/python/state_ops.md#assign_sub)\r\n * [`constant_initializer`](../../api_docs/python/state_ops.md#constant_initializer)\r\n * [`count_up_to`](../../api_docs/python/state_ops.md#count_up_to)\r\n * [`device`](../../api_docs/python/state_ops.md#device)\r\n * [`get_checkpoint_state`](../../api_docs/python/state_ops.md#get_checkpoint_state)\r\n * [`get_variable`](../../api_docs/python/state_ops.md#get_variable)\r\n * [`get_variable_scope`](../../api_docs/python/state_ops.md#get_variable_scope)\r\n * [`IndexedSlices`](../../api_docs/python/state_ops.md#IndexedSlices)\r\n * [`initialize_all_variables`](../../api_docs/python/state_ops.md#initialize_all_variables)\r\n * [`initialize_variables`](../../api_docs/python/state_ops.md#initialize_variables)\r\n * [`latest_checkpoint`](../../api_docs/python/state_ops.md#latest_checkpoint)\r\n * [`random_normal_initializer`](../../api_docs/python/state_ops.md#random_normal_initializer)\r\n * [`random_uniform_initializer`](../../api_docs/python/state_ops.md#random_uniform_initializer)\r\n * [`Saver`](../../api_docs/python/state_ops.md#Saver)\r\n * [`scatter_add`](../../api_docs/python/state_ops.md#scatter_add)\r\n * [`scatter_sub`](../../api_docs/python/state_ops.md#scatter_sub)\r\n * [`scatter_update`](../../api_docs/python/state_ops.md#scatter_update)\r\n * [`sparse_mask`](../../api_docs/python/state_ops.md#sparse_mask)\r\n * [`trainable_variables`](../../api_docs/python/state_ops.md#trainable_variables)\r\n * [`truncated_normal_initializer`](../../api_docs/python/state_ops.md#truncated_normal_initializer)\r\n * [`uniform_unit_scaling_initializer`](../../api_docs/python/state_ops.md#uniform_unit_scaling_initializer)\r\n * [`update_checkpoint_state`](../../api_docs/python/state_ops.md#update_checkpoint_state)\r\n * [`Variable`](../../api_docs/python/state_ops.md#Variable)\r\n * [`variable_scope`](../../api_docs/python/state_ops.md#variable_scope)\r\n * [`zeros_initializer`](../../api_docs/python/state_ops.md#zeros_initializer)\r\n\r\n* **[Tensor Transformations](../../api_docs/python/array_ops.md)**:\r\n * [`cast`](../../api_docs/python/array_ops.md#cast)\r\n * [`concat`](../../api_docs/python/array_ops.md#concat)\r\n * [`dynamic_partition`](../../api_docs/python/array_ops.md#dynamic_partition)\r\n * [`dynamic_stitch`](../../api_docs/python/array_ops.md#dynamic_stitch)\r\n * [`expand_dims`](../../api_docs/python/array_ops.md#expand_dims)\r\n * [`gather`](../../api_docs/python/array_ops.md#gather)\r\n * [`pack`](../../api_docs/python/array_ops.md#pack)\r\n * [`pad`](../../api_docs/python/array_ops.md#pad)\r\n * [`rank`](../../api_docs/python/array_ops.md#rank)\r\n * [`reshape`](../../api_docs/python/array_ops.md#reshape)\r\n * [`reverse`](../../api_docs/python/array_ops.md#reverse)\r\n * [`reverse_sequence`](../../api_docs/python/array_ops.md#reverse_sequence)\r\n * [`shape`](../../api_docs/python/array_ops.md#shape)\r\n * [`size`](../../api_docs/python/array_ops.md#size)\r\n * [`slice`](../../api_docs/python/array_ops.md#slice)\r\n * [`split`](../../api_docs/python/array_ops.md#split)\r\n * [`squeeze`](../../api_docs/python/array_ops.md#squeeze)\r\n * [`string_to_number`](../../api_docs/python/array_ops.md#string_to_number)\r\n * [`tile`](../../api_docs/python/array_ops.md#tile)\r\n * [`to_bfloat16`](../../api_docs/python/array_ops.md#to_bfloat16)\r\n * [`to_double`](../../api_docs/python/array_ops.md#to_double)\r\n * [`to_float`](../../api_docs/python/array_ops.md#to_float)\r\n * [`to_int32`](../../api_docs/python/array_ops.md#to_int32)\r\n * [`to_int64`](../../api_docs/python/array_ops.md#to_int64)\r\n * [`transpose`](../../api_docs/python/array_ops.md#transpose)\r\n * [`unpack`](../../api_docs/python/array_ops.md#unpack)\r\n\r\n* **[Math](../../api_docs/python/math_ops.md)**:\r\n * [`abs`](../../api_docs/python/math_ops.md#abs)\r\n * [`accumulate_n`](../../api_docs/python/math_ops.md#accumulate_n)\r\n * [`add`](../../api_docs/python/math_ops.md#add)\r\n * [`add_n`](../../api_docs/python/math_ops.md#add_n)\r\n * [`argmax`](../../api_docs/python/math_ops.md#argmax)\r\n * [`argmin`](../../api_docs/python/math_ops.md#argmin)\r\n * [`batch_cholesky`](../../api_docs/python/math_ops.md#batch_cholesky)\r\n * [`batch_matmul`](../../api_docs/python/math_ops.md#batch_matmul)\r\n * [`batch_matrix_determinant`](../../api_docs/python/math_ops.md#batch_matrix_determinant)\r\n * [`batch_matrix_inverse`](../../api_docs/python/math_ops.md#batch_matrix_inverse)\r\n * [`ceil`](../../api_docs/python/math_ops.md#ceil)\r\n * [`cholesky`](../../api_docs/python/math_ops.md#cholesky)\r\n * [`complex`](../../api_docs/python/math_ops.md#complex)\r\n * [`complex_abs`](../../api_docs/python/math_ops.md#complex_abs)\r\n * [`conj`](../../api_docs/python/math_ops.md#conj)\r\n * [`cos`](../../api_docs/python/math_ops.md#cos)\r\n * [`diag`](../../api_docs/python/math_ops.md#diag)\r\n * [`div`](../../api_docs/python/math_ops.md#div)\r\n * [`edit_distance`](../../api_docs/python/math_ops.md#edit_distance)\r\n * [`exp`](../../api_docs/python/math_ops.md#exp)\r\n * [`floor`](../../api_docs/python/math_ops.md#floor)\r\n * [`imag`](../../api_docs/python/math_ops.md#imag)\r\n * [`inv`](../../api_docs/python/math_ops.md#inv)\r\n * [`invert_permutation`](../../api_docs/python/math_ops.md#invert_permutation)\r\n * [`listdiff`](../../api_docs/python/math_ops.md#listdiff)\r\n * [`log`](../../api_docs/python/math_ops.md#log)\r\n * [`matmul`](../../api_docs/python/math_ops.md#matmul)\r\n * [`matrix_determinant`](../../api_docs/python/math_ops.md#matrix_determinant)\r\n * [`matrix_inverse`](../../api_docs/python/math_ops.md#matrix_inverse)\r\n * [`maximum`](../../api_docs/python/math_ops.md#maximum)\r\n * [`minimum`](../../api_docs/python/math_ops.md#minimum)\r\n * [`mod`](../../api_docs/python/math_ops.md#mod)\r\n * [`mul`](../../api_docs/python/math_ops.md#mul)\r\n * [`neg`](../../api_docs/python/math_ops.md#neg)\r\n * [`pow`](../../api_docs/python/math_ops.md#pow)\r\n * [`real`](../../api_docs/python/math_ops.md#real)\r\n * [`reduce_all`](../../api_docs/python/math_ops.md#reduce_all)\r\n * [`reduce_any`](../../api_docs/python/math_ops.md#reduce_any)\r\n * [`reduce_max`](../../api_docs/python/math_ops.md#reduce_max)\r\n * [`reduce_mean`](../../api_docs/python/math_ops.md#reduce_mean)\r\n * [`reduce_min`](../../api_docs/python/math_ops.md#reduce_min)\r\n * [`reduce_prod`](../../api_docs/python/math_ops.md#reduce_prod)\r\n * [`reduce_sum`](../../api_docs/python/math_ops.md#reduce_sum)\r\n * [`round`](../../api_docs/python/math_ops.md#round)\r\n * [`rsqrt`](../../api_docs/python/math_ops.md#rsqrt)\r\n * [`segment_max`](../../api_docs/python/math_ops.md#segment_max)\r\n * [`segment_mean`](../../api_docs/python/math_ops.md#segment_mean)\r\n * [`segment_min`](../../api_docs/python/math_ops.md#segment_min)\r\n * [`segment_prod`](../../api_docs/python/math_ops.md#segment_prod)\r\n * [`segment_sum`](../../api_docs/python/math_ops.md#segment_sum)\r\n * [`sign`](../../api_docs/python/math_ops.md#sign)\r\n * [`sin`](../../api_docs/python/math_ops.md#sin)\r\n * [`sparse_segment_mean`](../../api_docs/python/math_ops.md#sparse_segment_mean)\r\n * [`sparse_segment_sum`](../../api_docs/python/math_ops.md#sparse_segment_sum)\r\n * [`sqrt`](../../api_docs/python/math_ops.md#sqrt)\r\n * [`square`](../../api_docs/python/math_ops.md#square)\r\n * [`sub`](../../api_docs/python/math_ops.md#sub)\r\n * [`transpose`](../../api_docs/python/math_ops.md#transpose)\r\n * [`unique`](../../api_docs/python/math_ops.md#unique)\r\n * [`unsorted_segment_sum`](../../api_docs/python/math_ops.md#unsorted_segment_sum)\r\n * [`where`](../../api_docs/python/math_ops.md#where)\r\n\r\n* **[Control Flow](../../api_docs/python/control_flow_ops.md)**:\r\n * [`add_check_numerics_ops`](../../api_docs/python/control_flow_ops.md#add_check_numerics_ops)\r\n * [`Assert`](../../api_docs/python/control_flow_ops.md#Assert)\r\n * [`check_numerics`](../../api_docs/python/control_flow_ops.md#check_numerics)\r\n * [`count_up_to`](../../api_docs/python/control_flow_ops.md#count_up_to)\r\n * [`equal`](../../api_docs/python/control_flow_ops.md#equal)\r\n * [`greater`](../../api_docs/python/control_flow_ops.md#greater)\r\n * [`greater_equal`](../../api_docs/python/control_flow_ops.md#greater_equal)\r\n * [`group`](../../api_docs/python/control_flow_ops.md#group)\r\n * [`identity`](../../api_docs/python/control_flow_ops.md#identity)\r\n * [`is_finite`](../../api_docs/python/control_flow_ops.md#is_finite)\r\n * [`is_inf`](../../api_docs/python/control_flow_ops.md#is_inf)\r\n * [`is_nan`](../../api_docs/python/control_flow_ops.md#is_nan)\r\n * [`less`](../../api_docs/python/control_flow_ops.md#less)\r\n * [`less_equal`](../../api_docs/python/control_flow_ops.md#less_equal)\r\n * [`logical_and`](../../api_docs/python/control_flow_ops.md#logical_and)\r\n * [`logical_not`](../../api_docs/python/control_flow_ops.md#logical_not)\r\n * [`logical_or`](../../api_docs/python/control_flow_ops.md#logical_or)\r\n * [`logical_xor`](../../api_docs/python/control_flow_ops.md#logical_xor)\r\n * [`no_op`](../../api_docs/python/control_flow_ops.md#no_op)\r\n * [`not_equal`](../../api_docs/python/control_flow_ops.md#not_equal)\r\n * [`Print`](../../api_docs/python/control_flow_ops.md#Print)\r\n * [`select`](../../api_docs/python/control_flow_ops.md#select)\r\n * [`tuple`](../../api_docs/python/control_flow_ops.md#tuple)\r\n * [`verify_tensor_all_finite`](../../api_docs/python/control_flow_ops.md#verify_tensor_all_finite)\r\n * [`where`](../../api_docs/python/control_flow_ops.md#where)\r\n\r\n* **[Images](../../api_docs/python/image.md)**:\r\n * [`adjust_brightness`](../../api_docs/python/image.md#adjust_brightness)\r\n * [`adjust_contrast`](../../api_docs/python/image.md#adjust_contrast)\r\n * [`crop_to_bounding_box`](../../api_docs/python/image.md#crop_to_bounding_box)\r\n * [`decode_jpeg`](../../api_docs/python/image.md#decode_jpeg)\r\n * [`decode_png`](../../api_docs/python/image.md#decode_png)\r\n * [`encode_jpeg`](../../api_docs/python/image.md#encode_jpeg)\r\n * [`encode_png`](../../api_docs/python/image.md#encode_png)\r\n * [`extract_glimpse`](../../api_docs/python/image.md#extract_glimpse)\r\n * [`flip_left_right`](../../api_docs/python/image.md#flip_left_right)\r\n * [`flip_up_down`](../../api_docs/python/image.md#flip_up_down)\r\n * [`pad_to_bounding_box`](../../api_docs/python/image.md#pad_to_bounding_box)\r\n * [`per_image_whitening`](../../api_docs/python/image.md#per_image_whitening)\r\n * [`random_brightness`](../../api_docs/python/image.md#random_brightness)\r\n * [`random_contrast`](../../api_docs/python/image.md#random_contrast)\r\n * [`random_crop`](../../api_docs/python/image.md#random_crop)\r\n * [`random_flip_left_right`](../../api_docs/python/image.md#random_flip_left_right)\r\n * [`random_flip_up_down`](../../api_docs/python/image.md#random_flip_up_down)\r\n * [`resize_area`](../../api_docs/python/image.md#resize_area)\r\n * [`resize_bicubic`](../../api_docs/python/image.md#resize_bicubic)\r\n * [`resize_bilinear`](../../api_docs/python/image.md#resize_bilinear)\r\n * [`resize_image_with_crop_or_pad`](../../api_docs/python/image.md#resize_image_with_crop_or_pad)\r\n * [`resize_images`](../../api_docs/python/image.md#resize_images)\r\n * [`resize_nearest_neighbor`](../../api_docs/python/image.md#resize_nearest_neighbor)\r\n * [`transpose_image`](../../api_docs/python/image.md#transpose_image)\r\n\r\n* **[Sparse Tensors](../../api_docs/python/sparse_ops.md)**:\r\n * [`shape`](../../api_docs/python/sparse_ops.md#shape)\r\n * [`sparse_concat`](../../api_docs/python/sparse_ops.md#sparse_concat)\r\n * [`sparse_fill_empty_rows`](../../api_docs/python/sparse_ops.md#sparse_fill_empty_rows)\r\n * [`sparse_reorder`](../../api_docs/python/sparse_ops.md#sparse_reorder)\r\n * [`sparse_retain`](../../api_docs/python/sparse_ops.md#sparse_retain)\r\n * [`sparse_tensor_to_dense`](../../api_docs/python/sparse_ops.md#sparse_tensor_to_dense)\r\n * [`sparse_to_dense`](../../api_docs/python/sparse_ops.md#sparse_to_dense)\r\n * [`sparse_to_indicator`](../../api_docs/python/sparse_ops.md#sparse_to_indicator)\r\n * [`SparseTensor`](../../api_docs/python/sparse_ops.md#SparseTensor)\r\n * [`SparseTensorValue`](../../api_docs/python/sparse_ops.md#SparseTensorValue)\r\n\r\n* **[Inputs and Readers](../../api_docs/python/io_ops.md)**:\r\n * [`batch`](../../api_docs/python/io_ops.md#batch)\r\n * [`batch_join`](../../api_docs/python/io_ops.md#batch_join)\r\n * [`decode_csv`](../../api_docs/python/io_ops.md#decode_csv)\r\n * [`decode_raw`](../../api_docs/python/io_ops.md#decode_raw)\r\n * [`FIFOQueue`](../../api_docs/python/io_ops.md#FIFOQueue)\r\n * [`FixedLengthRecordReader`](../../api_docs/python/io_ops.md#FixedLengthRecordReader)\r\n * [`IdentityReader`](../../api_docs/python/io_ops.md#IdentityReader)\r\n * [`limit_epochs`](../../api_docs/python/io_ops.md#limit_epochs)\r\n * [`match_filenames_once`](../../api_docs/python/io_ops.md#match_filenames_once)\r\n * [`matching_files`](../../api_docs/python/io_ops.md#matching_files)\r\n * [`parse_example`](../../api_docs/python/io_ops.md#parse_example)\r\n * [`parse_single_example`](../../api_docs/python/io_ops.md#parse_single_example)\r\n * [`placeholder`](../../api_docs/python/io_ops.md#placeholder)\r\n * [`QueueBase`](../../api_docs/python/io_ops.md#QueueBase)\r\n * [`RandomShuffleQueue`](../../api_docs/python/io_ops.md#RandomShuffleQueue)\r\n * [`range_input_producer`](../../api_docs/python/io_ops.md#range_input_producer)\r\n * [`read_file`](../../api_docs/python/io_ops.md#read_file)\r\n * [`ReaderBase`](../../api_docs/python/io_ops.md#ReaderBase)\r\n * [`shuffle_batch`](../../api_docs/python/io_ops.md#shuffle_batch)\r\n * [`shuffle_batch_join`](../../api_docs/python/io_ops.md#shuffle_batch_join)\r\n * [`size`](../../api_docs/python/io_ops.md#size)\r\n * [`slice_input_producer`](../../api_docs/python/io_ops.md#slice_input_producer)\r\n * [`string_input_producer`](../../api_docs/python/io_ops.md#string_input_producer)\r\n * [`TextLineReader`](../../api_docs/python/io_ops.md#TextLineReader)\r\n * [`TFRecordReader`](../../api_docs/python/io_ops.md#TFRecordReader)\r\n * [`WholeFileReader`](../../api_docs/python/io_ops.md#WholeFileReader)\r\n\r\n* **[Data IO (Python functions)](../../api_docs/python/python_io.md)**:\r\n * [`tf_record_iterator`](../../api_docs/python/python_io.md#tf_record_iterator)\r\n * [`TFRecordWriter`](../../api_docs/python/python_io.md#TFRecordWriter)\r\n\r\n* **[Neural Network](../../api_docs/python/nn.md)**:\r\n * [`avg_pool`](../../api_docs/python/nn.md#avg_pool)\r\n * [`bias_add`](../../api_docs/python/nn.md#bias_add)\r\n * [`compute_accidental_hits`](../../api_docs/python/nn.md#compute_accidental_hits)\r\n * [`conv2d`](../../api_docs/python/nn.md#conv2d)\r\n * [`depthwise_conv2d`](../../api_docs/python/nn.md#depthwise_conv2d)\r\n * [`dropout`](../../api_docs/python/nn.md#dropout)\r\n * [`embedding_lookup`](../../api_docs/python/nn.md#embedding_lookup)\r\n * [`fixed_unigram_candidate_sampler`](../../api_docs/python/nn.md#fixed_unigram_candidate_sampler)\r\n * [`in_top_k`](../../api_docs/python/nn.md#in_top_k)\r\n * [`l2_loss`](../../api_docs/python/nn.md#l2_loss)\r\n * [`l2_normalize`](../../api_docs/python/nn.md#l2_normalize)\r\n * [`learned_unigram_candidate_sampler`](../../api_docs/python/nn.md#learned_unigram_candidate_sampler)\r\n * [`local_response_normalization`](../../api_docs/python/nn.md#local_response_normalization)\r\n * [`log_uniform_candidate_sampler`](../../api_docs/python/nn.md#log_uniform_candidate_sampler)\r\n * [`max_pool`](../../api_docs/python/nn.md#max_pool)\r\n * [`max_pool_with_argmax`](../../api_docs/python/nn.md#max_pool_with_argmax)\r\n * [`moments`](../../api_docs/python/nn.md#moments)\r\n * [`nce_loss`](../../api_docs/python/nn.md#nce_loss)\r\n * [`relu`](../../api_docs/python/nn.md#relu)\r\n * [`relu6`](../../api_docs/python/nn.md#relu6)\r\n * [`sampled_softmax_loss`](../../api_docs/python/nn.md#sampled_softmax_loss)\r\n * [`separable_conv2d`](../../api_docs/python/nn.md#separable_conv2d)\r\n * [`sigmoid`](../../api_docs/python/nn.md#sigmoid)\r\n * [`sigmoid_cross_entropy_with_logits`](../../api_docs/python/nn.md#sigmoid_cross_entropy_with_logits)\r\n * [`softmax`](../../api_docs/python/nn.md#softmax)\r\n * [`softmax_cross_entropy_with_logits`](../../api_docs/python/nn.md#softmax_cross_entropy_with_logits)\r\n * [`softplus`](../../api_docs/python/nn.md#softplus)\r\n * [`tanh`](../../api_docs/python/nn.md#tanh)\r\n * [`top_k`](../../api_docs/python/nn.md#top_k)\r\n * [`uniform_candidate_sampler`](../../api_docs/python/nn.md#uniform_candidate_sampler)\r\n\r\n* **[Running Graphs](../../api_docs/python/client.md)**:\r\n * [`AbortedError`](../../api_docs/python/client.md#AbortedError)\r\n * [`AlreadyExistsError`](../../api_docs/python/client.md#AlreadyExistsError)\r\n * [`CancelledError`](../../api_docs/python/client.md#CancelledError)\r\n * [`DataLossError`](../../api_docs/python/client.md#DataLossError)\r\n * [`DeadlineExceededError`](../../api_docs/python/client.md#DeadlineExceededError)\r\n * [`FailedPreconditionError`](../../api_docs/python/client.md#FailedPreconditionError)\r\n * [`get_default_session`](../../api_docs/python/client.md#get_default_session)\r\n * [`InteractiveSession`](../../api_docs/python/client.md#InteractiveSession)\r\n * [`InternalError`](../../api_docs/python/client.md#InternalError)\r\n * [`InvalidArgumentError`](../../api_docs/python/client.md#InvalidArgumentError)\r\n * [`NotFoundError`](../../api_docs/python/client.md#NotFoundError)\r\n * [`OpError`](../../api_docs/python/client.md#OpError)\r\n * [`OutOfRangeError`](../../api_docs/python/client.md#OutOfRangeError)\r\n * [`PermissionDeniedError`](../../api_docs/python/client.md#PermissionDeniedError)\r\n * [`ResourceExhaustedError`](../../api_docs/python/client.md#ResourceExhaustedError)\r\n * [`Session`](../../api_docs/python/client.md#Session)\r\n * [`UnauthenticatedError`](../../api_docs/python/client.md#UnauthenticatedError)\r\n * [`UnavailableError`](../../api_docs/python/client.md#UnavailableError)\r\n * [`UnimplementedError`](../../api_docs/python/client.md#UnimplementedError)\r\n * [`UnknownError`](../../api_docs/python/client.md#UnknownError)\r\n\r\n* **[Training](../../api_docs/python/train.md)**:\r\n * [`AdagradOptimizer`](../../api_docs/python/train.md#AdagradOptimizer)\r\n * [`AdamOptimizer`](../../api_docs/python/train.md#AdamOptimizer)\r\n * [`add_queue_runner`](../../api_docs/python/train.md#add_queue_runner)\r\n * [`AggregationMethod`](../../api_docs/python/train.md#AggregationMethod)\r\n * [`clip_by_average_norm`](../../api_docs/python/train.md#clip_by_average_norm)\r\n * [`clip_by_global_norm`](../../api_docs/python/train.md#clip_by_global_norm)\r\n * [`clip_by_norm`](../../api_docs/python/train.md#clip_by_norm)\r\n * [`clip_by_value`](../../api_docs/python/train.md#clip_by_value)\r\n * [`Coordinator`](../../api_docs/python/train.md#Coordinator)\r\n * [`exponential_decay`](../../api_docs/python/train.md#exponential_decay)\r\n * [`ExponentialMovingAverage`](../../api_docs/python/train.md#ExponentialMovingAverage)\r\n * [`FtrlOptimizer`](../../api_docs/python/train.md#FtrlOptimizer)\r\n * [`global_norm`](../../api_docs/python/train.md#global_norm)\r\n * [`global_step`](../../api_docs/python/train.md#global_step)\r\n * [`GradientDescentOptimizer`](../../api_docs/python/train.md#GradientDescentOptimizer)\r\n * [`gradients`](../../api_docs/python/train.md#gradients)\r\n * [`histogram_summary`](../../api_docs/python/train.md#histogram_summary)\r\n * [`image_summary`](../../api_docs/python/train.md#image_summary)\r\n * [`merge_all_summaries`](../../api_docs/python/train.md#merge_all_summaries)\r\n * [`merge_summary`](../../api_docs/python/train.md#merge_summary)\r\n * [`MomentumOptimizer`](../../api_docs/python/train.md#MomentumOptimizer)\r\n * [`Optimizer`](../../api_docs/python/train.md#Optimizer)\r\n * [`QueueRunner`](../../api_docs/python/train.md#QueueRunner)\r\n * [`RMSPropOptimizer`](../../api_docs/python/train.md#RMSPropOptimizer)\r\n * [`scalar_summary`](../../api_docs/python/train.md#scalar_summary)\r\n * [`start_queue_runners`](../../api_docs/python/train.md#start_queue_runners)\r\n * [`stop_gradient`](../../api_docs/python/train.md#stop_gradient)\r\n * [`summary_iterator`](../../api_docs/python/train.md#summary_iterator)\r\n * [`SummaryWriter`](../../api_docs/python/train.md#SummaryWriter)\r\n * [`write_graph`](../../api_docs/python/train.md#write_graph)\r\n * [`zero_fraction`](../../api_docs/python/train.md#zero_fraction)\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n-->\r\n
\r\n"
},
{
"path": "SOURCE/api_docs/python/io_ops.md",
"content": "\r\n\r\n# Inputs and Readers \r\n\r\nNote: Functions taking `Tensor` arguments can also take anything accepted by\r\n[`tf.convert_to_tensor`](../../api_docs/python/framework.md#convert_to_tensor).\r\n\r\n\r\n## Contents\r\n### [Inputs and Readers](#AUTOGENERATED-inputs-and-readers)\r\n* [Placeholders](#AUTOGENERATED-placeholders)\r\n * [`tf.placeholder(dtype, shape=None, name=None)`](#placeholder)\r\n* [Readers](#AUTOGENERATED-readers)\r\n * [`class tf.ReaderBase`](#ReaderBase)\r\n * [`class tf.TextLineReader`](#TextLineReader)\r\n * [`class tf.WholeFileReader`](#WholeFileReader)\r\n * [`class tf.IdentityReader`](#IdentityReader)\r\n * [`class tf.TFRecordReader`](#TFRecordReader)\r\n * [`class tf.FixedLengthRecordReader`](#FixedLengthRecordReader)\r\n* [Converting](#AUTOGENERATED-converting)\r\n * [`tf.decode_csv(records, record_defaults, field_delim=None, name=None)`](#decode_csv)\r\n * [`tf.decode_raw(bytes, out_type, little_endian=None, name=None)`](#decode_raw)\r\n * [Example protocol buffer](#AUTOGENERATED-example-protocol-buffer)\r\n * [`tf.parse_example(serialized, names=None, sparse_keys=None, sparse_types=None, dense_keys=None, dense_types=None, dense_defaults=None, dense_shapes=None, name='ParseExample')`](#parse_example)\r\n * [`tf.parse_single_example(serialized, names=None, sparse_keys=None, sparse_types=None, dense_keys=None, dense_types=None, dense_defaults=None, dense_shapes=None, name='ParseSingleExample')`](#parse_single_example)\r\n* [Queues](#AUTOGENERATED-queues)\r\n * [`class tf.QueueBase`](#QueueBase)\r\n * [`class tf.FIFOQueue`](#FIFOQueue)\r\n * [`class tf.RandomShuffleQueue`](#RandomShuffleQueue)\r\n* [Dealing with the filesystem](#AUTOGENERATED-dealing-with-the-filesystem)\r\n * [`tf.matching_files(pattern, name=None)`](#matching_files)\r\n * [`tf.read_file(filename, name=None)`](#read_file)\r\n* [Input pipeline](#AUTOGENERATED-input-pipeline)\r\n * [Beginning of an input pipeline](#AUTOGENERATED-beginning-of-an-input-pipeline)\r\n * [`tf.train.match_filenames_once(pattern, name=None)`](#match_filenames_once)\r\n * [`tf.train.limit_epochs(tensor, num_epochs=None, name=None)`](#limit_epochs)\r\n * [`tf.train.range_input_producer(limit, num_epochs=None, shuffle=True, seed=None, capacity=32, name=None)`](#range_input_producer)\r\n * [`tf.train.slice_input_producer(tensor_list, num_epochs=None, shuffle=True, seed=None, capacity=32, name=None)`](#slice_input_producer)\r\n * [`tf.train.string_input_producer(string_tensor, num_epochs=None, shuffle=True, seed=None, capacity=32, name=None)`](#string_input_producer)\r\n * [Batching at the end of an input pipeline](#AUTOGENERATED-batching-at-the-end-of-an-input-pipeline)\r\n * [`tf.train.batch(tensor_list, batch_size, num_threads=1, capacity=32, enqueue_many=False, shapes=None, name=None)`](#batch)\r\n * [`tf.train.batch_join(tensor_list_list, batch_size, capacity=32, enqueue_many=False, shapes=None, name=None)`](#batch_join)\r\n * [`tf.train.shuffle_batch(tensor_list, batch_size, capacity, min_after_dequeue, num_threads=1, seed=None, enqueue_many=False, shapes=None, name=None)`](#shuffle_batch)\r\n * [`tf.train.shuffle_batch_join(tensor_list_list, batch_size, capacity, min_after_dequeue, seed=None, enqueue_many=False, shapes=None, name=None)`](#shuffle_batch_join)\r\n\r\n\r\n\r\n\r\n## Placeholders \r\n\r\nTensorFlow provides a placeholder operation that must be fed with data\r\non execution. For more info, see the section on [Feeding\r\ndata](../../how_tos/reading_data/index.md#feeding).\r\n\r\n- - -\r\n\r\n### `tf.placeholder(dtype, shape=None, name=None)` \r\n\r\nInserts a placeholder for a tensor that will be always fed.\r\n\r\n**Important**: This tensor will produce an error if evaluated. Its value must\r\nbe fed using the `feed_dict` optional argument to `Session.run()`,\r\n`Tensor.eval()`, or `Operation.run()`.\r\n\r\nFor example:\r\n\r\n```python\r\nx = tf.placeholder(float, shape=(1024, 1024))\r\ny = tf.matmul(x, x)\r\n\r\nwith tf.Session() as sess:\r\n print sess.run(y) # ERROR: will fail because x was not fed.\r\n\r\n rand_array = np.random.rand(1024, 1024)\r\n print sess.run(y, feed_dict={x: rand_array}) # Will succeed.\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `dtype`: The type of elements in the tensor to be fed.\r\n* `shape`: The shape of the tensor to be fed (optional). If the shape is not\r\n specified, you can feed a tensor of any shape.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` that may be used as a handle for feeding a value, but not\r\n evaluated directly.\r\n\r\n\r\n\r\n## Readers \r\n\r\nTensorFlow provides a set of Reader classes for reading data formats.\r\nFor more information on inputs and readers, see [Reading\r\ndata](../../how_tos/reading_data/index.md).\r\n\r\n- - -\r\n\r\n### `class tf.ReaderBase` \r\n\r\nBase class for different Reader types, that produce a record every step.\r\n\r\nConceptually, Readers convert string 'work units' into records (key,\r\nvalue pairs). Typically the 'work units' are filenames and the\r\nrecords are extracted from the contents of those files. We want a\r\nsingle record produced per step, but a work unit can correspond to\r\nmany records.\r\n\r\nTherefore we introduce some decoupling using a queue. The queue\r\ncontains the work units and the Reader dequeues from the queue when\r\nit is asked to produce a record (via Read()) but it has finished the\r\nlast work unit.\r\n- - -\r\n\r\n#### `tf.ReaderBase.__init__(reader_ref, supports_serialize=False)` \r\n\r\nCreates a new ReaderBase.\r\n\r\n##### Args: \r\n\r\n\r\n* `reader_ref`: The operation that implements the reader.\r\n* `supports_serialize`: True if the reader implementation can\r\n serialize its state.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.ReaderBase.num_records_produced(name=None)` \r\n\r\nReturns the number of records this reader has produced.\r\n\r\nThis is the same as the number of Read executions that have\r\nsucceeded.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An int64 Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.ReaderBase.num_work_units_completed(name=None)` \r\n\r\nReturns the number of work units this reader has finished processing.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An int64 Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.ReaderBase.read(queue, name=None)` \r\n\r\nReturns the next record (key, value pair) produced by a reader.\r\n\r\nWill dequeue a work unit from queue if necessary (e.g. when the\r\nReader needs to start reading from a new file since it has\r\nfinished with the previous file).\r\n\r\n##### Args: \r\n\r\n\r\n* `queue`: A Queue or a mutable string Tensor representing a handle\r\n to a Queue, with string work items.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tuple of Tensors (key, value).\r\n\r\n* `key`: A string scalar Tensor.\r\n* `value`: A string scalar Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.ReaderBase.reader_ref` \r\n\r\nOp that implements the reader.\r\n\r\n- - -\r\n\r\n#### `tf.ReaderBase.reset(name=None)` \r\n\r\nRestore a reader to its initial clean state.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.ReaderBase.restore_state(state, name=None)` \r\n\r\nRestore a reader to a previously saved state.\r\n\r\nNot all Readers support being restored, so this can produce an\r\nUnimplemented error.\r\n\r\n##### Args: \r\n\r\n\r\n* `state`: A string Tensor.\r\n Result of a SerializeState of a Reader with matching type.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.ReaderBase.serialize_state(name=None)` \r\n\r\nProduce a string tensor that encodes the state of a reader.\r\n\r\nNot all Readers support being serialized, so this can produce an\r\nUnimplemented error.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A string Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.ReaderBase.supports_serialize` \r\n\r\nWhether the Reader implementation can serialize its state.\r\n\r\n\r\n- - -\r\n\r\n### `class tf.TextLineReader` \r\n\r\nA Reader that outputs the lines of a file delimited by newlines.\r\n\r\nNewlines are stripped from the output.\r\nSee ReaderBase for supported methods.\r\n- - -\r\n\r\n#### `tf.TextLineReader.__init__(skip_header_lines=None, name=None)` \r\n\r\nCreate a TextLineReader.\r\n\r\n##### Args: \r\n\r\n\r\n* `skip_header_lines`: An optional int. Defaults to 0. Number of lines\r\n to skip from the beginning of every file.\r\n* `name`: A name for the operation (optional).\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TextLineReader.num_records_produced(name=None)` \r\n\r\nReturns the number of records this reader has produced.\r\n\r\nThis is the same as the number of Read executions that have\r\nsucceeded.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An int64 Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TextLineReader.num_work_units_completed(name=None)` \r\n\r\nReturns the number of work units this reader has finished processing.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An int64 Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TextLineReader.read(queue, name=None)` \r\n\r\nReturns the next record (key, value pair) produced by a reader.\r\n\r\nWill dequeue a work unit from queue if necessary (e.g. when the\r\nReader needs to start reading from a new file since it has\r\nfinished with the previous file).\r\n\r\n##### Args: \r\n\r\n\r\n* `queue`: A Queue or a mutable string Tensor representing a handle\r\n to a Queue, with string work items.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tuple of Tensors (key, value).\r\n\r\n* `key`: A string scalar Tensor.\r\n* `value`: A string scalar Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TextLineReader.reader_ref` \r\n\r\nOp that implements the reader.\r\n\r\n- - -\r\n\r\n#### `tf.TextLineReader.reset(name=None)` \r\n\r\nRestore a reader to its initial clean state.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TextLineReader.restore_state(state, name=None)` \r\n\r\nRestore a reader to a previously saved state.\r\n\r\nNot all Readers support being restored, so this can produce an\r\nUnimplemented error.\r\n\r\n##### Args: \r\n\r\n\r\n* `state`: A string Tensor.\r\n Result of a SerializeState of a Reader with matching type.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TextLineReader.serialize_state(name=None)` \r\n\r\nProduce a string tensor that encodes the state of a reader.\r\n\r\nNot all Readers support being serialized, so this can produce an\r\nUnimplemented error.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A string Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TextLineReader.supports_serialize` \r\n\r\nWhether the Reader implementation can serialize its state.\r\n\r\n\r\n- - -\r\n\r\n### `class tf.WholeFileReader` \r\n\r\nA Reader that outputs the entire contents of a file as a value.\r\n\r\nTo use, enqueue filenames in a Queue. The output of Read will\r\nbe a filename (key) and the contents of that file (value).\r\n\r\nSee ReaderBase for supported methods.\r\n- - -\r\n\r\n#### `tf.WholeFileReader.__init__(name=None)` \r\n\r\nCreate a WholeFileReader.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n\r\n- - -\r\n\r\n#### `tf.WholeFileReader.num_records_produced(name=None)` \r\n\r\nReturns the number of records this reader has produced.\r\n\r\nThis is the same as the number of Read executions that have\r\nsucceeded.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An int64 Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.WholeFileReader.num_work_units_completed(name=None)` \r\n\r\nReturns the number of work units this reader has finished processing.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An int64 Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.WholeFileReader.read(queue, name=None)` \r\n\r\nReturns the next record (key, value pair) produced by a reader.\r\n\r\nWill dequeue a work unit from queue if necessary (e.g. when the\r\nReader needs to start reading from a new file since it has\r\nfinished with the previous file).\r\n\r\n##### Args: \r\n\r\n\r\n* `queue`: A Queue or a mutable string Tensor representing a handle\r\n to a Queue, with string work items.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tuple of Tensors (key, value).\r\n\r\n* `key`: A string scalar Tensor.\r\n* `value`: A string scalar Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.WholeFileReader.reader_ref` \r\n\r\nOp that implements the reader.\r\n\r\n- - -\r\n\r\n#### `tf.WholeFileReader.reset(name=None)` \r\n\r\nRestore a reader to its initial clean state.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.WholeFileReader.restore_state(state, name=None)` \r\n\r\nRestore a reader to a previously saved state.\r\n\r\nNot all Readers support being restored, so this can produce an\r\nUnimplemented error.\r\n\r\n##### Args: \r\n\r\n\r\n* `state`: A string Tensor.\r\n Result of a SerializeState of a Reader with matching type.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.WholeFileReader.serialize_state(name=None)` \r\n\r\nProduce a string tensor that encodes the state of a reader.\r\n\r\nNot all Readers support being serialized, so this can produce an\r\nUnimplemented error.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A string Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.WholeFileReader.supports_serialize` \r\n\r\nWhether the Reader implementation can serialize its state.\r\n\r\n\r\n- - -\r\n\r\n### `class tf.IdentityReader` \r\n\r\nA Reader that outputs the queued work as both the key and value.\r\n\r\nTo use, enqueue strings in a Queue. Read will take the front\r\nwork string and output (work, work).\r\n\r\nSee ReaderBase for supported methods.\r\n- - -\r\n\r\n#### `tf.IdentityReader.__init__(name=None)` \r\n\r\nCreate a IdentityReader.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n\r\n- - -\r\n\r\n#### `tf.IdentityReader.num_records_produced(name=None)` \r\n\r\nReturns the number of records this reader has produced.\r\n\r\nThis is the same as the number of Read executions that have\r\nsucceeded.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An int64 Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.IdentityReader.num_work_units_completed(name=None)` \r\n\r\nReturns the number of work units this reader has finished processing.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An int64 Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.IdentityReader.read(queue, name=None)` \r\n\r\nReturns the next record (key, value pair) produced by a reader.\r\n\r\nWill dequeue a work unit from queue if necessary (e.g. when the\r\nReader needs to start reading from a new file since it has\r\nfinished with the previous file).\r\n\r\n##### Args: \r\n\r\n\r\n* `queue`: A Queue or a mutable string Tensor representing a handle\r\n to a Queue, with string work items.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tuple of Tensors (key, value).\r\n\r\n* `key`: A string scalar Tensor.\r\n* `value`: A string scalar Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.IdentityReader.reader_ref` \r\n\r\nOp that implements the reader.\r\n\r\n- - -\r\n\r\n#### `tf.IdentityReader.reset(name=None)` \r\n\r\nRestore a reader to its initial clean state.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.IdentityReader.restore_state(state, name=None)` \r\n\r\nRestore a reader to a previously saved state.\r\n\r\nNot all Readers support being restored, so this can produce an\r\nUnimplemented error.\r\n\r\n##### Args: \r\n\r\n\r\n* `state`: A string Tensor.\r\n Result of a SerializeState of a Reader with matching type.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.IdentityReader.serialize_state(name=None)` \r\n\r\nProduce a string tensor that encodes the state of a reader.\r\n\r\nNot all Readers support being serialized, so this can produce an\r\nUnimplemented error.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A string Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.IdentityReader.supports_serialize` \r\n\r\nWhether the Reader implementation can serialize its state.\r\n\r\n\r\n- - -\r\n\r\n### `class tf.TFRecordReader` \r\n\r\nA Reader that outputs the records from a TFRecords file.\r\n\r\nSee ReaderBase for supported methods.\r\n- - -\r\n\r\n#### `tf.TFRecordReader.__init__(name=None)` \r\n\r\nCreate a TFRecordReader.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TFRecordReader.num_records_produced(name=None)` \r\n\r\nReturns the number of records this reader has produced.\r\n\r\nThis is the same as the number of Read executions that have\r\nsucceeded.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An int64 Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TFRecordReader.num_work_units_completed(name=None)` \r\n\r\nReturns the number of work units this reader has finished processing.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An int64 Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TFRecordReader.read(queue, name=None)` \r\n\r\nReturns the next record (key, value pair) produced by a reader.\r\n\r\nWill dequeue a work unit from queue if necessary (e.g. when the\r\nReader needs to start reading from a new file since it has\r\nfinished with the previous file).\r\n\r\n##### Args: \r\n\r\n\r\n* `queue`: A Queue or a mutable string Tensor representing a handle\r\n to a Queue, with string work items.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tuple of Tensors (key, value).\r\n\r\n* `key`: A string scalar Tensor.\r\n* `value`: A string scalar Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TFRecordReader.reader_ref` \r\n\r\nOp that implements the reader.\r\n\r\n- - -\r\n\r\n#### `tf.TFRecordReader.reset(name=None)` \r\n\r\nRestore a reader to its initial clean state.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TFRecordReader.restore_state(state, name=None)` \r\n\r\nRestore a reader to a previously saved state.\r\n\r\nNot all Readers support being restored, so this can produce an\r\nUnimplemented error.\r\n\r\n##### Args: \r\n\r\n\r\n* `state`: A string Tensor.\r\n Result of a SerializeState of a Reader with matching type.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TFRecordReader.serialize_state(name=None)` \r\n\r\nProduce a string tensor that encodes the state of a reader.\r\n\r\nNot all Readers support being serialized, so this can produce an\r\nUnimplemented error.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A string Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.TFRecordReader.supports_serialize` \r\n\r\nWhether the Reader implementation can serialize its state.\r\n\r\n\r\n- - -\r\n\r\n### `class tf.FixedLengthRecordReader` \r\n\r\nA Reader that outputs fixed-length records from a file.\r\n\r\nSee ReaderBase for supported methods.\r\n- - -\r\n\r\n#### `tf.FixedLengthRecordReader.__init__(record_bytes, header_bytes=None, footer_bytes=None, name=None)` \r\n\r\nCreate a FixedLengthRecordReader.\r\n\r\n##### Args: \r\n\r\n\r\n* `record_bytes`: An int.\r\n* `header_bytes`: An optional int. Defaults to 0.\r\n* `footer_bytes`: An optional int. Defaults to 0.\r\n* `name`: A name for the operation (optional).\r\n\r\n\r\n- - -\r\n\r\n#### `tf.FixedLengthRecordReader.num_records_produced(name=None)` \r\n\r\nReturns the number of records this reader has produced.\r\n\r\nThis is the same as the number of Read executions that have\r\nsucceeded.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An int64 Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.FixedLengthRecordReader.num_work_units_completed(name=None)` \r\n\r\nReturns the number of work units this reader has finished processing.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n An int64 Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.FixedLengthRecordReader.read(queue, name=None)` \r\n\r\nReturns the next record (key, value pair) produced by a reader.\r\n\r\nWill dequeue a work unit from queue if necessary (e.g. when the\r\nReader needs to start reading from a new file since it has\r\nfinished with the previous file).\r\n\r\n##### Args: \r\n\r\n\r\n* `queue`: A Queue or a mutable string Tensor representing a handle\r\n to a Queue, with string work items.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tuple of Tensors (key, value).\r\n\r\n* `key`: A string scalar Tensor.\r\n* `value`: A string scalar Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.FixedLengthRecordReader.reader_ref` \r\n\r\nOp that implements the reader.\r\n\r\n- - -\r\n\r\n#### `tf.FixedLengthRecordReader.reset(name=None)` \r\n\r\nRestore a reader to its initial clean state.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.FixedLengthRecordReader.restore_state(state, name=None)` \r\n\r\nRestore a reader to a previously saved state.\r\n\r\nNot all Readers support being restored, so this can produce an\r\nUnimplemented error.\r\n\r\n##### Args: \r\n\r\n\r\n* `state`: A string Tensor.\r\n Result of a SerializeState of a Reader with matching type.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The created Operation.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.FixedLengthRecordReader.serialize_state(name=None)` \r\n\r\nProduce a string tensor that encodes the state of a reader.\r\n\r\nNot all Readers support being serialized, so this can produce an\r\nUnimplemented error.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A string Tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.FixedLengthRecordReader.supports_serialize` \r\n\r\nWhether the Reader implementation can serialize its state.\r\n\r\n\r\n\r\n## Converting \r\n\r\nTensorFlow provides several operations that you can use to convert various data\r\nformats into tensors.\r\n\r\n- - -\r\n\r\n### `tf.decode_csv(records, record_defaults, field_delim=None, name=None)` \r\n\r\nConvert CSV records to tensors. Each column maps to one tensor.\r\n\r\nRFC 4180 format is expected for the CSV records.\r\n(https://tools.ietf.org/html/rfc4180)\r\nNote that we allow leading and trailing spaces with int or float field.\r\n\r\n##### Args: \r\n\r\n\r\n* `records`: A `Tensor` of type `string`.\r\n Each string is a record/row in the csv and all records should have\r\n the same format.\r\n* `record_defaults`: A list of `Tensor` objects with types from: `float32`, `int32`, `int64`, `string`.\r\n One tensor per column of the input record, with either a\r\n scalar default value for that column or empty if the column is required.\r\n* `field_delim`: An optional `string`. Defaults to `\",\"`.\r\n delimiter to separate fields in a record.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A list of `Tensor` objects. Has the same type as `record_defaults`.\r\n Each tensor will have the same shape as records.\r\n\r\n\r\n- - -\r\n\r\n### `tf.decode_raw(bytes, out_type, little_endian=None, name=None)` \r\n\r\nReinterpret the bytes of a string as a vector of numbers.\r\n\r\n##### Args: \r\n\r\n\r\n* `bytes`: A `Tensor` of type `string`.\r\n All the elements must have the same length.\r\n* `out_type`: A `tf.DType` from: `tf.float32, tf.float64, tf.int32, tf.uint8, tf.int16, tf.int8, tf.int64`.\r\n* `little_endian`: An optional `bool`. Defaults to `True`.\r\n Whether the input bytes are in little-endian order.\r\n Ignored for out_types that are stored in a single byte like uint8.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `out_type`.\r\n A Tensor with one more dimension than the input bytes. The\r\n added dimension will have size equal to the length of the elements\r\n of bytes divided by the number of bytes to represent out_type.\r\n\r\n\r\n\r\n- - -\r\n\r\n### Example protocol buffer \r\n\r\nTensorFlow's [recommended format for training\r\nexamples](../../how_tos/reading_data/index.md#standard-tensorflow-format)\r\nis serialized `Example` protocol buffers, [described\r\nhere](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/example/example.proto).\r\nThey contain `Features`, [described\r\nhere](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/example/feature.proto).\r\n\r\n- - -\r\n\r\n### `tf.parse_example(serialized, names=None, sparse_keys=None, sparse_types=None, dense_keys=None, dense_types=None, dense_defaults=None, dense_shapes=None, name='ParseExample')` \r\n\r\nParses `Example` protos.\r\n\r\nParses a number of serialized [`Example`]\r\n(https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/example/example.proto)\r\nprotos given in `serialized`.\r\n\r\n`names` may contain descriptive names for the corresponding serialized protos.\r\nThese may be useful for debugging purposes, but they have no effect on the\r\noutput. If not `None`, `names` must be the same length as `serialized`.\r\n\r\nThis op parses serialized examples into a dictionary mapping keys to `Tensor`\r\nand `SparseTensor` objects respectively, depending on whether the keys appear\r\nin `sparse_keys` or `dense_keys`.\r\n\r\nThe key `dense_keys[j]` is mapped to a `Tensor` of type `dense_types[j]` and\r\nof shape `(serialized.size(),) + dense_shapes[j]`.\r\n\r\n`dense_defaults` provides defaults for values referenced using `dense_keys`.\r\nIf a key is not present in this dictionary, the corresponding dense `Feature`\r\nis required in all elements of `serialized`.\r\n\r\n`dense_shapes[j]` provides the shape of each `Feature` entry referenced by\r\n`dense_keys[j]`. The number of elements in the `Feature` corresponding to\r\n`dense_key[j]` must always have `np.prod(dense_shapes[j])` entries. The\r\nreturned `Tensor` for `dense_key[j]` has shape `[N] + dense_shape[j]`, where\r\n`N` is the number of `Example`s in `serialized`.\r\n\r\nThe key `sparse_keys[j]` is mapped to a `SparseTensor` of type\r\n`sparse_types[j]`. The `SparseTensor` represents a ragged matrix.\r\nIts indices are `[batch, index]` where `batch` is the batch entry the value\r\nis from, and `index` is the value's index in the list of values associated\r\nwith that feature and example.\r\n\r\nExamples:\r\n\r\nFor example, if one expects a `tf.float32` sparse feature `ft` and three\r\nserialized `Example`s are provided:\r\n\r\n```\r\nserialized = [\r\n features:\r\n { feature: [ key: { \"ft\" value: float_list: { value: [1.0, 2.0] } } ] },\r\n features:\r\n { feature: [] },\r\n features:\r\n { feature: [ key: { \"ft\" value: float_list: { value: [3.0] } } ] }\r\n]\r\n```\r\n\r\nthen the output will look like:\r\n\r\n```\r\n{\"ft\": SparseTensor(indices=[[0, 0], [0, 1], [2, 0]],\r\n values=[1.0, 2.0, 3.0],\r\n shape=(3, 2)) }\r\n```\r\n\r\nGiven two `Example` input protos in `serialized`:\r\n\r\n```\r\n[\r\n features: {\r\n feature: { key: \"kw\" value: { bytes_list: { value: [ \"knit\", \"big\" ] } } }\r\n feature: { key: \"gps\" value: { float_list: { value: [] } } }\r\n },\r\n features: {\r\n feature: { key: \"kw\" value: { bytes_list: { value: [ \"emmy\" ] } } }\r\n feature: { key: \"dank\" value: { int64_list: { value: [ 42 ] } } }\r\n feature: { key: \"gps\" value: { } }\r\n }\r\n]\r\n```\r\n\r\nAnd arguments\r\n\r\n```\r\n names: [\"input0\", \"input1\"],\r\n sparse_keys: [\"kw\", \"dank\", \"gps\"]\r\n sparse_types: [DT_STRING, DT_INT64, DT_FLOAT]\r\n```\r\n\r\nThen the output is a dictionary:\r\n\r\n```python\r\n{\r\n \"kw\": SparseTensor(\r\n indices=[[0, 0], [0, 1], [1, 0]],\r\n values=[\"knit\", \"big\", \"emmy\"]\r\n shape=[2, 2]),\r\n \"dank\": SparseTensor(\r\n indices=[[1, 0]],\r\n values=[42],\r\n shape=[2, 1]),\r\n \"gps\": SparseTensor(\r\n indices=[],\r\n values=[],\r\n shape=[2, 0]),\r\n}\r\n```\r\n\r\nFor dense results in two serialized `Example`s:\r\n\r\n```\r\n[\r\n features: {\r\n feature: { key: \"age\" value: { int64_list: { value: [ 0 ] } } }\r\n feature: { key: \"gender\" value: { bytes_list: { value: [ \"f\" ] } } }\r\n },\r\n features: {\r\n feature: { key: \"age\" value: { int64_list: { value: [] } } }\r\n feature: { key: \"gender\" value: { bytes_list: { value: [ \"f\" ] } } }\r\n }\r\n]\r\n```\r\n\r\nWe can use arguments:\r\n\r\n```\r\nnames: [\"input0\", \"input1\"],\r\ndense_keys: np.array([\"age\", \"gender\"]),\r\ndense_types: [tf.int64, tf.string],\r\ndense_defaults: {\r\n \"age\": -1 # \"age\" defaults to -1 if missing\r\n # \"gender\" has no specified default so it's required\r\n}\r\ndense_shapes: [(1,), (1,)], # age, gender, label, weight\r\n```\r\n\r\nAnd the expected output is:\r\n\r\n```python\r\n{\r\n \"age\": [[0], [-1]],\r\n \"gender\": [[\"f\"], [\"f\"]],\r\n}\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `serialized`: A list of strings, a batch of binary serialized `Example`\r\n protos.\r\n* `names`: A list of strings, the names of the serialized protos.\r\n* `sparse_keys`: A list of string keys in the examples' features.\r\n The results for these keys will be returned as `SparseTensor` objects.\r\n* `sparse_types`: A list of `DTypes` of the same length as `sparse_keys`.\r\n Only `tf.float32` (`FloatList`), `tf.int64` (`Int64List`),\r\n and `tf.string` (`BytesList`) are supported.\r\n* `dense_keys`: A list of string keys in the examples' features.\r\n The results for these keys will be returned as `Tensor`s\r\n* `dense_types`: A list of DTypes of the same length as `dense_keys`.\r\n Only `tf.float32` (`FloatList`), `tf.int64` (`Int64List`),\r\n and `tf.string` (`BytesList`) are supported.\r\n* `dense_defaults`: A dict mapping string keys to `Tensor`s.\r\n The keys of the dict must match the dense_keys of the feature.\r\n* `dense_shapes`: A list of tuples with the same length as `dense_keys`.\r\n The shape of the data for each dense feature referenced by `dense_keys`.\r\n* `name`: A name for this operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `dict` mapping keys to `Tensor`s and `SparseTensor`s.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If sparse and dense key sets intersect, or input lengths do not\r\n match up.\r\n\r\n\r\n- - -\r\n\r\n### `tf.parse_single_example(serialized, names=None, sparse_keys=None, sparse_types=None, dense_keys=None, dense_types=None, dense_defaults=None, dense_shapes=None, name='ParseSingleExample')` \r\n\r\nParses a single `Example` proto.\r\n\r\nSimilar to `parse_example`, except:\r\n\r\nFor dense tensors, the returned `Tensor` is identical to the output of\r\n`parse_example`, except there is no batch dimension, the output shape is the\r\nsame as the shape given in `dense_shape`.\r\n\r\nFor `SparseTensor`s, the first (batch) column of the indices matrix is removed\r\n(the indices matrix is a column vector), the values vector is unchanged, and\r\nthe first (batch_size) entry of the shape vector is removed (it is now a\r\nsingle element vector).\r\n\r\nSee also `parse_example`.\r\n\r\n##### Args: \r\n\r\n\r\n* `serialized`: A scalar string, a single serialized Example.\r\n See parse_example documentation for more details.\r\n* `names`: (Optional) A scalar string, the associated name.\r\n See parse_example documentation for more details.\r\n* `sparse_keys`: See parse_example documentation for more details.\r\n* `sparse_types`: See parse_example documentation for more details.\r\n* `dense_keys`: See parse_example documentation for more details.\r\n* `dense_types`: See parse_example documentation for more details.\r\n* `dense_defaults`: See parse_example documentation for more details.\r\n* `dense_shapes`: See parse_example documentation for more details.\r\n* `name`: A name for this operation (optional).\r\n\r\n##### Returns: \r\n\r\n A dictionary mapping keys to Tensors and SparseTensors.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if \"scalar\" or \"names\" have known shapes, and are not scalars.\r\n\r\n\r\n\r\n## Queues \r\n\r\nTensorFlow provides several implementations of 'Queues', which are\r\nstructures within the TensorFlow computation graph to stage pipelines\r\nof tensors together. The following describe the basic Queue interface\r\nand some implementations. To see an example use, see [Threading and\r\nQueues](../../how_tos/threading_and_queues/index.md).\r\n\r\n- - -\r\n\r\n### `class tf.QueueBase` \r\n\r\nBase class for queue implementations.\r\n\r\nA queue is a TensorFlow data structure that stores tensors across\r\nmultiple steps, and exposes operations that enqueue and dequeue\r\ntensors.\r\n\r\nEach queue element is a tuple of one or more tensors, where each\r\ntuple component has a static dtype, and may have a static shape. The\r\nqueue implementations support versions of enqueue and dequeue that\r\nhandle single elements, versions that support enqueuing and\r\ndequeuing a batch of elements at once.\r\n\r\nSee [`tf.FIFOQueue`](#FIFOQueue) and\r\n[`tf.RandomShuffleQueue`](#RandomShuffleQueue) for concrete\r\nimplementations of this class, and instructions on how to create\r\nthem.\r\n\r\n- - -\r\n\r\n#### `tf.QueueBase.enqueue(vals, name=None)` \r\n\r\nEnqueues one element to this queue.\r\n\r\nIf the queue is full when this operation executes, it will block\r\nuntil the element has been enqueued.\r\n\r\n##### Args: \r\n\r\n\r\n* `vals`: The tuple of `Tensor` objects to be enqueued.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The operation that enqueues a new tuple of tensors to the queue.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.QueueBase.enqueue_many(vals, name=None)` \r\n\r\nEnqueues zero or elements to this queue.\r\n\r\nThis operation slices each component tensor along the 0th dimension to\r\nmake multiple queue elements. All of the tensors in `vals` must have the\r\nsame size in the 0th dimension.\r\n\r\nIf the queue is full when this operation executes, it will block\r\nuntil all of the elements have been enqueued.\r\n\r\n##### Args: \r\n\r\n\r\n* `vals`: The tensor or tuple of tensors from which the queue elements\r\n are taken.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The operation that enqueues a batch of tuples of tensors to the queue.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.QueueBase.dequeue(name=None)` \r\n\r\nDequeues one element from this queue.\r\n\r\nIf the queue is empty when this operation executes, it will block\r\nuntil there is an element to dequeue.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The tuple of tensors that was dequeued.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.QueueBase.dequeue_many(n, name=None)` \r\n\r\nDequeues and concatenates `n` elements from this queue.\r\n\r\nThis operation concatenates queue-element component tensors along\r\nthe 0th dimension to make a single component tensor. All of the\r\ncomponents in the dequeued tuple will have size `n` in the 0th dimension.\r\n\r\nIf the queue contains fewer than `n` elements when this operation\r\nexecutes, it will block until `n` elements have been dequeued.\r\n\r\n##### Args: \r\n\r\n\r\n* `n`: A scalar `Tensor` containing the number of elements to dequeue.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The tuple of concatenated tensors that was dequeued.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.QueueBase.size(name=None)` \r\n\r\nCompute the number of elements in this queue.\r\n\r\n##### Args: \r\n\r\n\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A scalar tensor containing the number of elements in this queue.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.QueueBase.close(cancel_pending_enqueues=False, name=None)` \r\n\r\nCloses this queue.\r\n\r\nThis operation signals that no more elements will be enqueued in\r\nthe given queue. Subsequent `enqueue` and `enqueue_many`\r\noperations will fail. Subsequent `dequeue` and `dequeue_many`\r\noperations will continue to succeed if sufficient elements remain\r\nin the queue. Subsequent `dequeue` and `dequeue_many` operations\r\nthat would block will fail immediately.\r\n\r\nIf `cancel_pending_enqueues` is `True`, all pending requests will also\r\nbe cancelled.\r\n\r\n##### Args: \r\n\r\n\r\n* `cancel_pending_enqueues`: (Optional.) A boolean, defaulting to\r\n `False` (described above).\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The operation that closes the queue.\r\n\r\n\r\n\r\n#### Other Methods \r\n- - -\r\n\r\n#### `tf.QueueBase.__init__(dtypes, shapes, queue_ref)` \r\n\r\nConstructs a queue object from a queue reference.\r\n\r\n##### Args: \r\n\r\n\r\n* `dtypes`: A list of types. The length of dtypes must equal the number\r\n of tensors in each element.\r\n* `shapes`: Constraints on the shapes of tensors in an element:\r\n A list of shape tuples or None. This list is the same length\r\n as dtypes. If the shape of any tensors in the element are constrained,\r\n all must be; shapes can be None if the shapes should not be constrained.\r\n* `queue_ref`: The queue reference, i.e. the output of the queue op.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.QueueBase.dtypes` \r\n\r\nThe list of dtypes for each component of a queue element.\r\n\r\n- - -\r\n\r\n#### `tf.QueueBase.name` \r\n\r\nThe name of the underlying queue.\r\n\r\n- - -\r\n\r\n#### `tf.QueueBase.queue_ref` \r\n\r\nThe underlying queue reference.\r\n\r\n\r\n- - -\r\n\r\n### `class tf.FIFOQueue` \r\n\r\nA queue implementation that dequeues elements in first-in-first out order.\r\n\r\nSee [`tf.QueueBase`](#QueueBase) for a description of the methods on\r\nthis class.\r\n\r\n- - -\r\n\r\n#### `tf.FIFOQueue.__init__(capacity, dtypes, shapes=None, shared_name=None, name='fifo_queue')` \r\n\r\nCreates a queue that dequeues elements in a first-in first-out order.\r\n\r\nA `FIFOQueue` has bounded capacity; supports multiple concurrent\r\nproducers and consumers; and provides exactly-once delivery.\r\n\r\nA `FIFOQueue` holds a list of up to `capacity` elements. Each\r\nelement is a fixed-length tuple of tensors whose dtypes are\r\ndescribed by `dtypes`, and whose shapes are optionally described\r\nby the `shapes` argument.\r\n\r\nIf the `shapes` argument is specified, each component of a queue\r\nelement must have the respective fixed shape. If it is\r\nunspecified, different queue elements may have different shapes,\r\nbut the use of `dequeue_many` is disallowed.\r\n\r\n##### Args: \r\n\r\n\r\n* `capacity`: An integer. The upper bound on the number of elements\r\n that may be stored in this queue.\r\n* `dtypes`: A list of `DType` objects. The length of `dtypes` must equal\r\n the number of tensors in each queue element.\r\n* `shapes`: (Optional.) A list of fully-defined `TensorShape` objects,\r\n with the same length as `dtypes` or `None`.\r\n* `shared_name`: (Optional.) If non-empty, this queue will be shared under\r\n the given name across multiple sessions.\r\n* `name`: Optional name for the queue operation.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.RandomShuffleQueue` \r\n\r\nA queue implementation that dequeues elements in a random order.\r\n\r\nSee [`tf.QueueBase`](#QueueBase) for a description of the methods on\r\nthis class.\r\n\r\n- - -\r\n\r\n#### `tf.RandomShuffleQueue.__init__(capacity, min_after_dequeue, dtypes, shapes=None, seed=None, shared_name=None, name='random_shuffle_queue')` \r\n\r\nCreate a queue that dequeues elements in a random order.\r\n\r\nA `RandomShuffleQueue` has bounded capacity; supports multiple\r\nconcurrent producers and consumers; and provides exactly-once\r\ndelivery.\r\n\r\nA `RandomShuffleQueue` holds a list of up to `capacity`\r\nelements. Each element is a fixed-length tuple of tensors whose\r\ndtypes are described by `dtypes`, and whose shapes are optionally\r\ndescribed by the `shapes` argument.\r\n\r\nIf the `shapes` argument is specified, each component of a queue\r\nelement must have the respective fixed shape. If it is\r\nunspecified, different queue elements may have different shapes,\r\nbut the use of `dequeue_many` is disallowed.\r\n\r\nThe `min_after_dequeue` argument allows the caller to specify a\r\nminimum number of elements that will remain in the queue after a\r\n`dequeue` or `dequeue_many` operation completes, to ensure a\r\nminimum level of mixing of elements. This invariant is maintained\r\nby blocking those operations until sufficient elements have been\r\nenqueued. The `min_after_dequeue` argument is ignored after the\r\nqueue has been closed.\r\n\r\n##### Args: \r\n\r\n\r\n* `capacity`: An integer. The upper bound on the number of elements\r\n that may be stored in this queue.\r\n* `min_after_dequeue`: An integer (described above).\r\n* `dtypes`: A list of `DType` objects. The length of `dtypes` must equal\r\n the number of tensors in each queue element.\r\n* `shapes`: (Optional.) A list of fully-defined `TensorShape` objects,\r\n with the same length as `dtypes` or `None`.\r\n* `seed`: A Python integer. Used to create a random seed. See\r\n [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\n for behavior.\r\n* `shared_name`: (Optional.) If non-empty, this queue will be shared under\r\n the given name across multiple sessions.\r\n* `name`: Optional name for the queue operation.\r\n\r\n\r\n\r\n\r\n## Dealing with the filesystem \r\n\r\n- - -\r\n\r\n### `tf.matching_files(pattern, name=None)` \r\n\r\nReturns the set of files matching a pattern.\r\n\r\nNote that this routine only supports wildcard characters in the\r\nbasename portion of the pattern, not in the directory portion.\r\n\r\n##### Args: \r\n\r\n\r\n* `pattern`: A `Tensor` of type `string`. A (scalar) shell wildcard pattern.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `string`. A vector of matching filenames.\r\n\r\n\r\n- - -\r\n\r\n### `tf.read_file(filename, name=None)` \r\n\r\nReads and outputs the entire contents of the input filename.\r\n\r\n##### Args: \r\n\r\n\r\n* `filename`: A `Tensor` of type `string`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `string`.\r\n\r\n\r\n\r\n## Input pipeline \r\n\r\nTensorFlow functions for setting up an input-prefetching pipeline.\r\nPlease see the [reading data how-to](../../how_tos/reading_data/index.md)\r\nfor context.\r\n\r\n### Beginning of an input pipeline \r\n\r\nThe \"producer\" functions add a queue to the graph and a corresponding\r\n`QueueRunner` for running the subgraph that fills that queue.\r\n\r\n- - -\r\n\r\n### `tf.train.match_filenames_once(pattern, name=None)` \r\n\r\nSave the list of files matching pattern, so it is only computed once.\r\n\r\n##### Args: \r\n\r\n\r\n* `pattern`: A file pattern (glob).\r\n* `name`: A name for the operations (optional).\r\n\r\n##### Returns: \r\n\r\n A variable that is initialized to the list of files matching pattern.\r\n\r\n\r\n- - -\r\n\r\n### `tf.train.limit_epochs(tensor, num_epochs=None, name=None)` \r\n\r\nReturns tensor num_epochs times and then raises an OutOfRange error.\r\n\r\n##### Args: \r\n\r\n\r\n* `tensor`: Any Tensor.\r\n* `num_epochs`: An integer (optional). If specified, limits the number\r\n of steps the output tensor may be evaluated.\r\n* `name`: A name for the operations (optional).\r\n\r\n##### Returns: \r\n\r\n tensor or OutOfRange.\r\n\r\n\r\n- - -\r\n\r\n### `tf.train.range_input_producer(limit, num_epochs=None, shuffle=True, seed=None, capacity=32, name=None)` \r\n\r\nProduces the integers from 0 to limit-1 in a queue.\r\n\r\n##### Args: \r\n\r\n\r\n* `limit`: An int32 scalar tensor.\r\n* `num_epochs`: An integer (optional). If specified, `range_input_producer`\r\n produces each integer `num_epochs` times before generating an\r\n OutOfRange error. If not specified, `range_input_producer` can cycle\r\n through the integers an unlimited number of times.\r\n* `shuffle`: Boolean. If true, the integers are randomly shuffled within each\r\n epoch.\r\n* `seed`: An integer (optional). Seed used if shuffle == True.\r\n* `capacity`: An integer. Sets the queue capacity.\r\n* `name`: A name for the operations (optional).\r\n\r\n##### Returns: \r\n\r\n A Queue with the output integers. A QueueRunner for the Queue\r\n is added to the current Graph's QUEUE_RUNNER collection.\r\n\r\n\r\n- - -\r\n\r\n### `tf.train.slice_input_producer(tensor_list, num_epochs=None, shuffle=True, seed=None, capacity=32, name=None)` \r\n\r\nProduces a slice of each Tensor in tensor_list.\r\n\r\nImplemented using a Queue -- a QueueRunner for the Queue\r\nis added to the current Graph's QUEUE_RUNNER collection.\r\n\r\n##### Args: \r\n\r\n\r\n* `tensor_list`: A list of Tensors. Every Tensor in tensor_list must\r\n have the same size in the first dimension.\r\n* `num_epochs`: An integer (optional). If specified, `slice_input_producer`\r\n produces each slice `num_epochs` times before generating\r\n an OutOfRange error. If not specified, `slice_input_producer` can cycle\r\n through the slices an unlimited number of times.\r\n* `seed`: An integer (optional). Seed used if shuffle == True.\r\n* `capacity`: An integer. Sets the queue capacity.\r\n* `name`: A name for the operations (optional).\r\n\r\n##### Returns: \r\n\r\n A list of tensors, one for each element of tensor_list. If the tensor\r\n in tensor_list has shape [N, a, b, .., z], then the corresponding output\r\n tensor will have shape [a, b, ..., z].\r\n\r\n\r\n- - -\r\n\r\n### `tf.train.string_input_producer(string_tensor, num_epochs=None, shuffle=True, seed=None, capacity=32, name=None)` \r\n\r\nOutput strings (e.g. filenames) to a queue for an input pipeline.\r\n\r\n##### Args: \r\n\r\n\r\n* `string_tensor`: A 1-D string tensor with the strings to produce.\r\n* `num_epochs`: An integer (optional). If specified, `string_input_producer`\r\n produces each string from `string_tensor` `num_epochs` times before\r\n generating an OutOfRange error. If not specified, `string_input_producer`\r\n can cycle through the strings in `string_tensor` an unlimited number of\r\n times.\r\n* `shuffle`: Boolean. If true, the strings are randomly shuffled within each\r\n epoch.\r\n* `seed`: An integer (optional). Seed used if shuffle == True.\r\n* `capacity`: An integer. Sets the queue capacity.\r\n* `name`: A name for the operations (optional).\r\n\r\n##### Returns: \r\n\r\n A queue with the output strings. A QueueRunner for the Queue\r\n is added to the current Graph's QUEUE_RUNNER collection.\r\n\r\n\r\n\r\n### Batching at the end of an input pipeline \r\n\r\nThese functions add a queue to the graph to assemble a batch of examples, with\r\npossible shuffling. They also add a `QueueRunner` for running the subgraph\r\nthat fills that queue.\r\n\r\nUse [batch](#batch) or [batch_join](#batch_join) for batching examples that have\r\nalready been well shuffled. Use [shuffle_batch](#shuffle_batch) or\r\n[shuffle_batch_join](#shuffle_batch_join) for examples that\r\nwould benefit from additional shuffling.\r\n\r\nUse [batch](#batch) or [shuffle_batch](#shuffle_batch) if you want a\r\nsingle thread producing examples to batch, or if you have a\r\nsingle subgraph producing examples but you want to run it in N threads\r\n(where you increase N until it can keep the queue full). Use\r\n[batch_join](#batch_join) or [shuffle_batch_join](#shuffle_batch_join)\r\nif you have N different subgraphs producing examples to batch and you\r\nwant them run by N threads.\r\n\r\n- - -\r\n\r\n### `tf.train.batch(tensor_list, batch_size, num_threads=1, capacity=32, enqueue_many=False, shapes=None, name=None)` \r\n\r\nCreates batches of tensors in `tensor_list`.\r\n\r\nThis function is implemented using a queue. A `QueueRunner` for the\r\nqueue is added to the current `Graph`'s `QUEUE_RUNNER` collection.\r\n\r\nIf `enqueue_many` is `False`, `tensor_list` is assumed to represent a\r\nsingle example. An input tensor with shape `[x, y, z]` will be output\r\nas a tensor with shape `[batch_size, x, y, z]`.\r\n\r\nIf `enqueue_many` is `True`, `tensor_list` is assumed to represent a\r\nbatch of examples, where the first dimension is indexed by example,\r\nand all members of `tensor_list` should have the same size in the\r\nfirst dimension. If an input tensor has shape `[*, x, y, z]`, the\r\noutput will have shape `[batch_size, x, y, z]`. The `capacity` argument\r\ncontrols the how long the prefetching is allowed to grow the queues.\r\n\r\n##### Args: \r\n\r\n\r\n* `tensor_list`: The list of tensors to enqueue.\r\n* `batch_size`: The new batch size pulled from the queue.\r\n* `num_threads`: The number of threads enqueuing `tensor_list`.\r\n* `capacity`: An integer. The maximum number of elements in the queue.\r\n* `enqueue_many`: Whether each tensor in `tensor_list` is a single example.\r\n* `shapes`: (Optional) The shapes for each example. Defaults to the\r\n inferred shapes for `tensor_list`.\r\n* `name`: (Optional) A name for the operations.\r\n\r\n##### Returns: \r\n\r\n A list of tensors with the same number and types as `tensor_list`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.train.batch_join(tensor_list_list, batch_size, capacity=32, enqueue_many=False, shapes=None, name=None)` \r\n\r\nRuns a list of tensors to fill a queue to create batches of examples.\r\n\r\nEnqueues a different list of tensors in different threads.\r\nImplemented using a queue -- a `QueueRunner` for the queue\r\nis added to the current `Graph`'s `QUEUE_RUNNER` collection.\r\n\r\n`len(tensor_list_list)` threads will be started,\r\nwith thread `i` enqueuing the tensors from\r\n`tensor_list_list[i]`. `tensor_list_list[i1][j]` must match\r\n`tensor_list_list[i2][j]` in type and shape, except in the first\r\ndimension if `enqueue_many` is true.\r\n\r\nIf `enqueue_many` is `False`, each `tensor_list_list[i]` is assumed\r\nto represent a single example. An input tensor `x` will be output as a\r\ntensor with shape `[batch_size] + x.shape`.\r\n\r\nIf `enqueue_many` is `True`, `tensor_list_list[i]` is assumed to\r\nrepresent a batch of examples, where the first dimension is indexed\r\nby example, and all members of `tensor_list_list[i]` should have the\r\nsame size in the first dimension. The slices of any input tensor\r\n`x` are treated as examples, and the output tensors will have shape\r\n`[batch_size] + x.shape[1:]`.\r\n\r\nThe `capacity` argument controls the how long the prefetching is allowed to\r\ngrow the queues.\r\n\r\n##### Args: \r\n\r\n\r\n* `tensor_list_list`: A list of tuples of tensors to enqueue.\r\n* `batch_size`: An integer. The new batch size pulled from the queue.\r\n* `capacity`: An integer. The maximum number of elements in the queue.\r\n* `enqueue_many`: Whether each tensor in `tensor_list_list` is a single\r\n example.\r\n* `shapes`: (Optional) The shapes for each example. Defaults to the\r\n inferred shapes for `tensor_list_list[i]`.\r\n* `name`: (Optional) A name for the operations.\r\n\r\n##### Returns: \r\n\r\n A list of tensors with the same number and types as\r\n `tensor_list_list[i]`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.train.shuffle_batch(tensor_list, batch_size, capacity, min_after_dequeue, num_threads=1, seed=None, enqueue_many=False, shapes=None, name=None)` \r\n\r\nCreates batches by randomly shuffling tensors.\r\n\r\nThis function adds the following to the current `Graph`:\r\n\r\n* A shuffling queue into which tensors from `tensor_list` are enqueued.\r\n* A `dequeue_many` operation to create batches from the queue.\r\n* A `QueueRunner` to `QUEUE_RUNNER` collection, to enqueue the tensors\r\n from `tensor_list`.\r\n\r\nIf `enqueue_many` is `False`, `tensor_list` is assumed to represent a\r\nsingle example. An input tensor with shape `[x, y, z]` will be output\r\nas a tensor with shape `[batch_size, x, y, z]`.\r\n\r\nIf `enqueue_many` is `True`, `tensor_list` is assumed to represent a\r\nbatch of examples, where the first dimension is indexed by example,\r\nand all members of `tensor_list` should have the same size in the\r\nfirst dimension. If an input tensor has shape `[*, x, y, z]`, the\r\noutput will have shape `[batch_size, x, y, z]`.\r\n\r\nThe `capacity` argument controls the how long the prefetching is allowed to\r\ngrow the queues.\r\n\r\nFor example:\r\n\r\n```python\r\n# Creates batches of 32 images and 32 labels.\r\nimage_batch, label_batch = tf.train.shuffle_batch(\r\n [single_image, single_label],\r\n batch_size=32,\r\n num_threads=4,\r\n capacity=50000,\r\n min_after_dequeue=10000)\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `tensor_list`: The list of tensors to enqueue.\r\n* `batch_size`: The new batch size pulled from the queue.\r\n* `capacity`: An integer. The maximum number of elements in the queue.\r\n* `min_after_dequeue`: Minimum number elements in the queue after a\r\n dequeue, used to ensure a level of mixing of elements.\r\n* `num_threads`: The number of threads enqueuing `tensor_list`.\r\n* `seed`: Seed for the random shuffling within the queue.\r\n* `enqueue_many`: Whether each tensor in `tensor_list` is a single example.\r\n* `shapes`: (Optional) The shapes for each example. Defaults to the\r\n inferred shapes for `tensor_list`.\r\n* `name`: (Optional) A name for the operations.\r\n\r\n##### Returns: \r\n\r\n A list of tensors with the same number and types as `tensor_list`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.train.shuffle_batch_join(tensor_list_list, batch_size, capacity, min_after_dequeue, seed=None, enqueue_many=False, shapes=None, name=None)` \r\n\r\nCreate batches by randomly shuffling tensors.\r\n\r\nThis version enqueues a different list of tensors in different threads.\r\nIt adds the following to the current `Graph`:\r\n\r\n* A shuffling queue into which tensors from `tensor_list_list` are enqueued.\r\n* A `dequeue_many` operation to create batches from the queue.\r\n* A `QueueRunner` to `QUEUE_RUNNER` collection, to enqueue the tensors\r\n from `tensor_list_list`.\r\n\r\n`len(tensor_list_list)` threads will be started, with thread `i` enqueuing\r\nthe tensors from `tensor_list_list[i]`. `tensor_list_list[i1][j]` must match\r\n`tensor_list_list[i2][j]` in type and shape, except in the first dimension if\r\n`enqueue_many` is true.\r\n\r\nIf `enqueue_many` is `False`, each `tensor_list_list[i]` is assumed\r\nto represent a single example. An input tensor with shape `[x, y,\r\nz]` will be output as a tensor with shape `[batch_size, x, y, z]`.\r\n\r\nIf `enqueue_many` is `True`, `tensor_list_list[i]` is assumed to\r\nrepresent a batch of examples, where the first dimension is indexed\r\nby example, and all members of `tensor_list_list[i]` should have the\r\nsame size in the first dimension. If an input tensor has shape `[*, x,\r\ny, z]`, the output will have shape `[batch_size, x, y, z]`.\r\n\r\nThe `capacity` argument controls the how long the prefetching is allowed to\r\ngrow the queues.\r\n\r\n##### Args: \r\n\r\n\r\n* `tensor_list_list`: A list of tuples of tensors to enqueue.\r\n* `batch_size`: An integer. The new batch size pulled from the queue.\r\n* `capacity`: An integer. The maximum number of elements in the queue.\r\n* `min_after_dequeue`: Minimum number elements in the queue after a\r\n dequeue, used to ensure a level of mixing of elements.\r\n* `seed`: Seed for the random shuffling within the queue.\r\n* `enqueue_many`: Whether each tensor in `tensor_list_list` is a single\r\n example.\r\n* `shapes`: (Optional) The shapes for each example. Defaults to the\r\n inferred shapes for `tensor_list_list[i]`.\r\n* `name`: (Optional) A name for the operations.\r\n\r\n##### Returns: \r\n\r\n A list of tensors with the same number and types as `tensor_list_list[i]`.\r\n\r\n\r\n"
},
{
"path": "SOURCE/api_docs/python/math_ops.md",
"content": "\r\n\r\n# Math \r\n\r\nNote: Functions taking `Tensor` arguments can also take anything accepted by\r\n[`tf.convert_to_tensor`](../../api_docs/python/framework.md#convert_to_tensor).\r\n\r\n\r\n## Contents\r\n### [Math](#AUTOGENERATED-math)\r\n* [Arithmetic Operators](#AUTOGENERATED-arithmetic-operators)\r\n * [`tf.add(x, y, name=None)`](#add)\r\n * [`tf.sub(x, y, name=None)`](#sub)\r\n * [`tf.mul(x, y, name=None)`](#mul)\r\n * [`tf.div(x, y, name=None)`](#div)\r\n * [`tf.mod(x, y, name=None)`](#mod)\r\n* [Basic Math Functions](#AUTOGENERATED-basic-math-functions)\r\n * [`tf.add_n(inputs, name=None)`](#add_n)\r\n * [`tf.abs(x, name=None)`](#abs)\r\n * [`tf.neg(x, name=None)`](#neg)\r\n * [`tf.sign(x, name=None)`](#sign)\r\n * [`tf.inv(x, name=None)`](#inv)\r\n * [`tf.square(x, name=None)`](#square)\r\n * [`tf.round(x, name=None)`](#round)\r\n * [`tf.sqrt(x, name=None)`](#sqrt)\r\n * [`tf.rsqrt(x, name=None)`](#rsqrt)\r\n * [`tf.pow(x, y, name=None)`](#pow)\r\n * [`tf.exp(x, name=None)`](#exp)\r\n * [`tf.log(x, name=None)`](#log)\r\n * [`tf.ceil(x, name=None)`](#ceil)\r\n * [`tf.floor(x, name=None)`](#floor)\r\n * [`tf.maximum(x, y, name=None)`](#maximum)\r\n * [`tf.minimum(x, y, name=None)`](#minimum)\r\n * [`tf.cos(x, name=None)`](#cos)\r\n * [`tf.sin(x, name=None)`](#sin)\r\n* [Matrix Math Functions](#AUTOGENERATED-matrix-math-functions)\r\n * [`tf.diag(diagonal, name=None)`](#diag)\r\n * [`tf.transpose(a, perm=None, name='transpose')`](#transpose)\r\n * [`tf.matmul(a, b, transpose_a=False, transpose_b=False, a_is_sparse=False, b_is_sparse=False, name=None)`](#matmul)\r\n * [`tf.batch_matmul(x, y, adj_x=None, adj_y=None, name=None)`](#batch_matmul)\r\n * [`tf.matrix_determinant(input, name=None)`](#matrix_determinant)\r\n * [`tf.batch_matrix_determinant(input, name=None)`](#batch_matrix_determinant)\r\n * [`tf.matrix_inverse(input, name=None)`](#matrix_inverse)\r\n * [`tf.batch_matrix_inverse(input, name=None)`](#batch_matrix_inverse)\r\n * [`tf.cholesky(input, name=None)`](#cholesky)\r\n * [`tf.batch_cholesky(input, name=None)`](#batch_cholesky)\r\n* [Complex Number Functions](#AUTOGENERATED-complex-number-functions)\r\n * [`tf.complex(real, imag, name=None)`](#complex)\r\n * [`tf.complex_abs(x, name=None)`](#complex_abs)\r\n * [`tf.conj(in_, name=None)`](#conj)\r\n * [`tf.imag(in_, name=None)`](#imag)\r\n * [`tf.real(in_, name=None)`](#real)\r\n* [Reduction](#AUTOGENERATED-reduction)\r\n * [`tf.reduce_sum(input_tensor, reduction_indices=None, keep_dims=False, name=None)`](#reduce_sum)\r\n * [`tf.reduce_prod(input_tensor, reduction_indices=None, keep_dims=False, name=None)`](#reduce_prod)\r\n * [`tf.reduce_min(input_tensor, reduction_indices=None, keep_dims=False, name=None)`](#reduce_min)\r\n * [`tf.reduce_max(input_tensor, reduction_indices=None, keep_dims=False, name=None)`](#reduce_max)\r\n * [`tf.reduce_mean(input_tensor, reduction_indices=None, keep_dims=False, name=None)`](#reduce_mean)\r\n * [`tf.reduce_all(input_tensor, reduction_indices=None, keep_dims=False, name=None)`](#reduce_all)\r\n * [`tf.reduce_any(input_tensor, reduction_indices=None, keep_dims=False, name=None)`](#reduce_any)\r\n * [`tf.accumulate_n(inputs, shape=None, tensor_dtype=None, name=None)`](#accumulate_n)\r\n* [Segmentation](#AUTOGENERATED-segmentation)\r\n * [`tf.segment_sum(data, segment_ids, name=None)`](#segment_sum)\r\n * [`tf.segment_prod(data, segment_ids, name=None)`](#segment_prod)\r\n * [`tf.segment_min(data, segment_ids, name=None)`](#segment_min)\r\n * [`tf.segment_max(data, segment_ids, name=None)`](#segment_max)\r\n * [`tf.segment_mean(data, segment_ids, name=None)`](#segment_mean)\r\n * [`tf.unsorted_segment_sum(data, segment_ids, num_segments, name=None)`](#unsorted_segment_sum)\r\n * [`tf.sparse_segment_sum(data, indices, segment_ids, name=None)`](#sparse_segment_sum)\r\n * [`tf.sparse_segment_mean(data, indices, segment_ids, name=None)`](#sparse_segment_mean)\r\n* [Sequence Comparison and Indexing](#AUTOGENERATED-sequence-comparison-and-indexing)\r\n * [`tf.argmin(input, dimension, name=None)`](#argmin)\r\n * [`tf.argmax(input, dimension, name=None)`](#argmax)\r\n * [`tf.listdiff(x, y, name=None)`](#listdiff)\r\n * [`tf.where(input, name=None)`](#where)\r\n * [`tf.unique(x, name=None)`](#unique)\r\n * [`tf.edit_distance(hypothesis, truth, normalize=True, name='edit_distance')`](#edit_distance)\r\n * [`tf.invert_permutation(x, name=None)`](#invert_permutation)\r\n\r\n\r\n\r\n\r\n## Arithmetic Operators \r\n\r\nTensorFlow provides several operations that you can use to add basic arithmetic\r\noperators to your graph.\r\n\r\n- - -\r\n\r\n### `tf.add(x, y, name=None)` \r\n\r\nReturns x + y element-wise.\r\n\r\n*NOTE*: Add supports broadcasting. AddN does not.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int8`, `int16`, `int32`, `complex64`, `int64`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.sub(x, y, name=None)` \r\n\r\nReturns x - y element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `complex64`, `int64`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.mul(x, y, name=None)` \r\n\r\nReturns x * y element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int8`, `int16`, `int32`, `complex64`, `int64`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.div(x, y, name=None)` \r\n\r\nReturns x / y element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `complex64`, `int64`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.mod(x, y, name=None)` \r\n\r\nReturns element-wise remainder of division.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `int32`, `int64`, `float32`, `float64`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n\r\n## Basic Math Functions \r\n\r\nTensorFlow provides several operations that you can use to add basic\r\nmathematical functions to your graph.\r\n\r\n- - -\r\n\r\n### `tf.add_n(inputs, name=None)` \r\n\r\nAdd all input tensors element wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `inputs`: A list of at least 1 `Tensor` objects of the same type in: `float32`, `float64`, `int64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `qint8`, `quint8`, `qint32`.\r\n Must all be the same size and shape.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `inputs`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.abs(x, name=None)` \r\n\r\nComputes the absolute value of a tensor.\r\n\r\nGiven a tensor of real numbers `x`, this operation returns a tensor\r\ncontaining the absolute value of each element in `x`. For example, if x is\r\nan input element and y is an output element, this operation computes\r\n\\\\(y = |x|\\\\).\r\n\r\nSee [`tf.complex_abs()`](#tf_complex_abs) to compute the absolute value of a complex\r\nnumber.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` of type `float`, `double`, `int32`, or `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` the same size and type as `x` with absolute values.\r\n\r\n\r\n- - -\r\n\r\n### `tf.neg(x, name=None)` \r\n\r\nComputes numerical negative value element-wise.\r\n\r\nI.e., \\\\(y = -x\\\\).\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `complex64`, `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.sign(x, name=None)` \r\n\r\nReturns an element-wise indication of the sign of a number.\r\n\r\ny = sign(x) = -1 if x < 0; 0 if x == 0; 1 if x > 0.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.inv(x, name=None)` \r\n\r\nComputes the reciprocal of x element-wise.\r\n\r\nI.e., \\\\(y = 1 / x\\\\).\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `complex64`, `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.square(x, name=None)` \r\n\r\nComputes square of x element-wise.\r\n\r\nI.e., \\\\(y = x * x = x^2\\\\).\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `complex64`, `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.round(x, name=None)` \r\n\r\nRounds the values of a tensor to the nearest integer, element-wise.\r\n\r\nFor example:\r\n\r\n```python\r\n# 'a' is [0.9, 2.5, 2.3, -4.4]\r\ntf.round(a) ==> [ 1.0, 3.0, 2.0, -4.0 ]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` of type `float` or `double`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of same shape and type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.sqrt(x, name=None)` \r\n\r\nComputes square root of x element-wise.\r\n\r\nI.e., \\\\(y = \\sqrt{x} = x^{1/2}\\\\).\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `complex64`, `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.rsqrt(x, name=None)` \r\n\r\nComputes reciprocal of square root of x element-wise.\r\n\r\nI.e., \\\\(y = 1 / \\sqrt{x}\\\\).\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `complex64`, `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.pow(x, y, name=None)` \r\n\r\nComputes the power of one value to another.\r\n\r\nGiven a tensor `x` and a tensor `y`, this operation computes \\\\(x^y\\\\) for\r\ncorresponding elements in `x` and `y`. For example:\r\n\r\n```\r\n# tensor 'x' is [[2, 2]], [3, 3]]\r\n# tensor 'y' is [[8, 16], [2, 3]]\r\ntf.pow(x, y) ==> [[256, 65536], [9, 27]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` of type `float`, `double`, `int32`, `complex64`, or `int64`.\r\n* `y`: A `Tensor` of type `float`, `double`, `int32`, `complex64`, or `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.exp(x, name=None)` \r\n\r\nComputes exponential of x element-wise. \\\\(y = e^x\\\\).\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `complex64`, `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.log(x, name=None)` \r\n\r\nComputes natural logrithm of x element-wise.\r\n\r\nI.e., \\\\(y = \\log_e x\\\\).\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `complex64`, `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.ceil(x, name=None)` \r\n\r\nReturns element-wise smallest integer in not less than x.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.floor(x, name=None)` \r\n\r\nReturns element-wise largest integer not greater than x.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.maximum(x, y, name=None)` \r\n\r\nReturns the max of x and y (i.e. x > y ? x : y) element-wise, broadcasts.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.minimum(x, y, name=None)` \r\n\r\nReturns the min of x and y (i.e. x < y ? x : y) element-wise, broadcasts.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.cos(x, name=None)` \r\n\r\nComputes cos of x element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `complex64`, `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.sin(x, name=None)` \r\n\r\nComputes sin of x element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `complex64`, `int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n\r\n\r\n\r\n## Matrix Math Functions \r\n\r\nTensorFlow provides several operations that you can use to add basic\r\nmathematical functions for matrices to your graph.\r\n\r\n- - -\r\n\r\n### `tf.diag(diagonal, name=None)` \r\n\r\nReturns a diagonal tensor with a given diagonal values.\r\n\r\nGiven a `diagonal`, this operation returns a tensor with the `diagonal` and\r\neverything else padded with zeros. The diagonal is computed as follows:\r\n\r\nAssume `diagonal` has dimensions [D1,..., Dk], then the output is a tensor of\r\nrank 2k with dimensions [D1,..., Dk, D1,..., Dk] where:\r\n\r\n`output[i1,..., ik, i1,..., ik] = diagonal[i1, ..., ik]` and 0 everywhere else.\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# 'diagonal' is [1, 2, 3, 4]\r\ntf.diag(diagonal) ==> [[1, 0, 0, 0]\r\n [0, 2, 0, 0]\r\n [0, 0, 3, 0]\r\n [0, 0, 0, 4]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `diagonal`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`.\r\n Rank k tensor where k is at most 3.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `diagonal`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.transpose(a, perm=None, name='transpose')` \r\n\r\nTransposes `a`. Permutes the dimensions according to `perm`.\r\n\r\nThe returned tensor's dimension i will correspond to the input dimension\r\n`perm[i]`. If `perm` is not given, it is set to (n-1...0), where n is\r\nthe rank of the input tensor. Hence by default, this operation performs a\r\nregular matrix transpose on 2-D input Tensors.\r\n\r\nFor example:\r\n\r\n```python\r\n# 'x' is [[1 2 3]\r\n# [4 5 6]]\r\ntf.transpose(x) ==> [[1 4]\r\n [2 5]\r\n [3 6]]\r\n\r\n# Equivalently\r\ntf.transpose(x perm=[0, 1]) ==> [[1 4]\r\n [2 5]\r\n [3 6]]\r\n\r\n# 'perm' is more useful for n-dimensional tensors, for n > 2\r\n# 'x' is [[[1 2 3]\r\n# [4 5 6]]\r\n# [[7 8 9]\r\n# [10 11 12]]]\r\n# Take the transpose of the matrices in dimension-0\r\ntf.transpose(b, perm=[0, 2, 1]) ==> [[[1 4]\r\n [2 5]\r\n [3 6]]\r\n\r\n [[7 10]\r\n [8 11]\r\n [9 12]]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `a`: A `Tensor`.\r\n* `perm`: A permutation of the dimensions of `a`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A transposed `Tensor`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.matmul(a, b, transpose_a=False, transpose_b=False, a_is_sparse=False, b_is_sparse=False, name=None)` \r\n\r\nMultiplies matrix `a` by matrix `b`, producing `a` * `b`.\r\n\r\nThe inputs must be two-dimensional matrices, with matching inner dimensions,\r\npossibly after transposition.\r\n\r\nBoth matrices must be of the same type. The supported types are:\r\n`float`, `double`, `int32`, `complex64`.\r\n\r\nEither matrix can be transposed on the fly by setting the corresponding flag\r\nto `True`. This is `False` by default.\r\n\r\nIf one or both of the matrices contain a lot of zeros, a more efficient\r\nmultiplication algorithm can be used by setting the corresponding\r\n`a_is_sparse` or `b_is_sparse` flag to `True`. These are `False` by default.\r\n\r\nFor example:\r\n\r\n```python\r\n# 2-D tensor `a`\r\na = tf.constant([1, 2, 3, 4, 5, 6], shape=[2, 3]) => [[1. 2. 3.]\r\n [4. 5. 6.]]\r\n# 2-D tensor `b`\r\nb = tf.constant([7, 8, 9, 10, 11, 12], shape=[3, 2]) => [[7. 8.]\r\n [9. 10.]\r\n [11. 12.]]\r\nc = tf.matmul(a, b) => [[58 64]\r\n [139 154]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `a`: `Tensor` of type `float`, `double`, `int32` or `complex64`.\r\n* `b`: `Tensor` with same type as `a`.\r\n* `transpose_a`: If `True`, `a` is transposed before multiplication.\r\n* `transpose_b`: If `True`, `b` is transposed before multiplication.\r\n* `a_is_sparse`: If `True`, `a` is treated as a sparse matrix.\r\n* `b_is_sparse`: If `True`, `b` is treated as a sparse matrix.\r\n* `name`: Name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of the same type as `a`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.batch_matmul(x, y, adj_x=None, adj_y=None, name=None)` \r\n\r\nMultiplies slices of two tensors in batches.\r\n\r\nMultiplies all slices of `Tensor` `x` and `y` (each slice can be\r\nviewed as an element of a batch), and arranges the individual results\r\nin a single output tensor of the same batch size. Each of the\r\nindividual slices can optionally be adjointed (to adjoint a matrix\r\nmeans to transpose and conjugate it) before multiplication by setting\r\nthe `adj_x` or `adj_y` flag to `True`, which are by default `False`.\r\n\r\nThe input tensors `x` and `y` are 3-D or higher with shape `[..., r_x, c_x]`\r\nand `[..., r_y, c_y]`.\r\n\r\nThe output tensor is 3-D or higher with shape `[..., r_o, c_o]`, where:\r\n\r\n r_o = c_x if adj_x else r_x\r\n c_o = r_y if adj_y else c_y\r\n\r\nIt is computed as:\r\n\r\n out[..., :, :] = matrix(x[..., :, :]) * matrix(y[..., :, :])\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `complex64`.\r\n 3-D or higher with shape `[..., r_x, c_x]`.\r\n* `y`: A `Tensor`. Must have the same type as `x`.\r\n 3-D or higher with shape `[..., r_y, c_y]`.\r\n* `adj_x`: An optional `bool`. Defaults to `False`.\r\n If `True`, adjoint the slices of `x`. Defaults to `False`.\r\n* `adj_y`: An optional `bool`. Defaults to `False`.\r\n If `True`, adjoint the slices of `y`. Defaults to `False`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `x`.\r\n 3-D or higher with shape `[..., r_o, c_o]`\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.matrix_determinant(input, name=None)` \r\n\r\nCalculates the determinant of a square matrix.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n A tensor of shape `[M, M]`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`.\r\n A scalar, equal to the determinant of the input.\r\n\r\n\r\n- - -\r\n\r\n### `tf.batch_matrix_determinant(input, name=None)` \r\n\r\nCalculates the determinants for a batch of square matrices.\r\n\r\nThe input is a tensor of shape `[..., M, M]` whose inner-most 2 dimensions\r\nform square matrices. The output is a 1-D tensor containing the determinants\r\nfor all input submatrices `[..., :, :]`.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n Shape is `[..., M, M]`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`. Shape is `[...]`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.matrix_inverse(input, name=None)` \r\n\r\nCalculates the inverse of a square invertible matrix. Checks for invertibility.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n Shape is `[M, M]`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`.\r\n Shape is `[M, M]` containing the matrix inverse of the input.\r\n\r\n\r\n- - -\r\n\r\n### `tf.batch_matrix_inverse(input, name=None)` \r\n\r\nCalculates the inverse of square invertible matrices. Checks for invertibility.\r\n\r\nThe input is a tensor of shape `[..., M, M]` whose inner-most 2 dimensions\r\nform square matrices. The output is a tensor of the same shape as the input\r\ncontaining the inverse for all input submatrices `[..., :, :]`.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n Shape is `[..., M, M]`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`. Shape is `[..., M, M]`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.cholesky(input, name=None)` \r\n\r\nCalculates the Cholesky decomposition of a square matrix.\r\n\r\nThe input has to be symmetric and positive definite. Only the lower-triangular\r\npart of the input will be used for this operation. The upper-triangular part\r\nwill not be read.\r\n\r\nThe result is the lower-triangular matrix of the Cholesky decomposition of the\r\ninput.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. Must be one of the following types: `float64`, `float32`.\r\n Shape is `[M, M]`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`. Shape is `[M, M]`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.batch_cholesky(input, name=None)` \r\n\r\nCalculates the Cholesky decomposition of a batch of square matrices.\r\n\r\nThe input is a tensor of shape `[..., M, M]` whose inner-most 2 dimensions\r\nform square matrices, with the same constraints as the single matrix Cholesky\r\ndecomposition above. The output is a tensor of the same shape as the input\r\ncontaining the Cholesky decompositions for all input submatrices `[..., :, :]`.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. Must be one of the following types: `float64`, `float32`.\r\n Shape is `[..., M, M]`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`. Shape is `[..., M, M]`.\r\n\r\n\r\n\r\n## Complex Number Functions \r\n\r\nTensorFlow provides several operations that you can use to add complex number\r\nfunctions to your graph.\r\n\r\n- - -\r\n\r\n### `tf.complex(real, imag, name=None)` \r\n\r\nConverts two real numbers to a complex number.\r\n\r\nGiven a tensor `real` representing the real part of a complex number, and a\r\ntensor `imag` representing the imaginary part of a complex number, this\r\noperation computes complex numbers elementwise of the form \\\\(a + bj\\\\),\r\nwhere *a* represents the `real` part and *b* represents the `imag` part.\r\n\r\nThe input tensors `real` and `imag` must be the same shape.\r\n\r\nFor example:\r\n\r\n```\r\n# tensor 'real' is [2.25, 3.25]\r\n# tensor `imag` is [4.75, 5.75]\r\ntf.complex(real, imag) ==> [[2.25 + 4.74j], [3.25 + 5.75j]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `real`: A `Tensor` of type `float`.\r\n* `imag`: A `Tensor` of type `float`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `complex64`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.complex_abs(x, name=None)` \r\n\r\nComputes the complex absolute value of a tensor.\r\n\r\nGiven a tensor `x` of complex numbers, this operation returns a tensor of type\r\n`float` that is the absolute value of each element in `x`. All elements in `x`\r\nmust be complex numbers of the form \\\\(a + bj\\\\). The absolute value is\r\ncomputed as \\\\( \\sqrt{a^2 + b^2}\\\\).\r\n\r\nFor example:\r\n\r\n```\r\n# tensor 'x' is [[-2.25 + 4.75j], [-3.25 + 5.75j]]\r\ntf.complex_abs(x) ==> [5.25594902, 6.60492229]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` of type `complex64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `float32`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.conj(in_, name=None)` \r\n\r\nReturns the complex conjugate of a complex number.\r\n\r\nGiven a tensor `in` of complex numbers, this operation returns a tensor of\r\ncomplex numbers that are the complex conjugate of each element in `in`. The\r\ncomplex numbers in `in` must be of the form \\\\(a + bj\\\\), where *a* is the real\r\npart and *b* is the imaginary part.\r\n\r\nThe complex conjugate returned by this operation is of the form \\\\(a - bj\\\\).\r\n\r\nFor example:\r\n\r\n```\r\n# tensor 'in' is [-2.25 + 4.75j, 3.25 + 5.75j]\r\ntf.conj(in) ==> [-2.25 - 4.75j, 3.25 - 5.75j]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `in_`: A `Tensor` of type `complex64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `complex64`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.imag(in_, name=None)` \r\n\r\nReturns the imaginary part of a complex number.\r\n\r\nGiven a tensor `in` of complex numbers, this operation returns a tensor of type\r\n`float` that is the imaginary part of each element in `in`. All elements in `in`\r\nmust be complex numbers of the form \\\\(a + bj\\\\), where *a* is the real part\r\nand *b* is the imaginary part returned by this operation.\r\n\r\nFor example:\r\n\r\n```\r\n# tensor 'in' is [-2.25 + 4.75j, 3.25 + 5.75j]\r\ntf.imag(in) ==> [4.75, 5.75]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `in_`: A `Tensor` of type `complex64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `float32`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.real(in_, name=None)` \r\n\r\nReturns the real part of a complex number.\r\n\r\nGiven a tensor `in` of complex numbers, this operation returns a tensor of type\r\n`float` that is the real part of each element in `in`. All elements in `in`\r\nmust be complex numbers of the form \\\\(a + bj\\\\), where *a* is the real part\r\nreturned by this operation and *b* is the imaginary part.\r\n\r\nFor example:\r\n\r\n```\r\n# tensor 'in' is [-2.25 + 4.75j, 3.25 + 5.75j]\r\ntf.real(in) ==> [-2.25, 3.25]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `in_`: A `Tensor` of type `complex64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `float32`.\r\n\r\n\r\n\r\n## Reduction \r\n\r\nTensorFlow provides several operations that you can use to perform\r\ncommon math computations that reduce various dimensions of a tensor.\r\n\r\n- - -\r\n\r\n### `tf.reduce_sum(input_tensor, reduction_indices=None, keep_dims=False, name=None)` \r\n\r\nComputes the sum of elements across dimensions of a tensor.\r\n\r\nReduces `input_tensor` along the dimensions given in `reduction_indices`.\r\nUnless `keep_dims` is true, the rank of the tensor is reduced by 1 for each\r\nentry in `reduction_indices`. If `keep_dims` is true, the reduced dimensions\r\nare retained with length 1.\r\n\r\nIf `reduction_indices` has no entries, all dimensions are reduced, and a\r\ntensor with a single element is returned.\r\n\r\nFor example:\r\n\r\n```python\r\n# 'x' is [[1, 1, 1]]\r\n# [1, 1, 1]]\r\ntf.reduce_sum(x) ==> 6\r\ntf.reduce_sum(x, 0) ==> [2, 2, 2]\r\ntf.reduce_sum(x, 1) ==> [3, 3]\r\ntf.reduce_sum(x, 1, keep_dims=True) ==> [[3], [3]]\r\ntf.reduce_sum(x, [0, 1]) ==> 6\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `input_tensor`: The tensor to reduce. Should have numeric type.\r\n* `reduction_indices`: The dimensions to reduce. If `None` (the defaut),\r\n reduces all dimensions.\r\n* `keep_dims`: If true, retains reduced dimensions with length 1.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The reduced tensor.\r\n\r\n\r\n- - -\r\n\r\n### `tf.reduce_prod(input_tensor, reduction_indices=None, keep_dims=False, name=None)` \r\n\r\nComputes the product of elements across dimensions of a tensor.\r\n\r\nReduces `input_tensor` along the dimensions given in `reduction_indices`.\r\nUnless `keep_dims` is true, the rank of the tensor is reduced by 1 for each\r\nentry in `reduction_indices`. If `keep_dims` is true, the reduced dimensions\r\nare retained with length 1.\r\n\r\nIf `reduction_indices` has no entries, all dimensions are reduced, and a\r\ntensor with a single element is returned.\r\n\r\n##### Args: \r\n\r\n\r\n* `input_tensor`: The tensor to reduce. Should have numeric type.\r\n* `reduction_indices`: The dimensions to reduce. If `None` (the defaut),\r\n reduces all dimensions.\r\n* `keep_dims`: If true, retains reduced dimensions with length 1.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The reduced tensor.\r\n\r\n\r\n- - -\r\n\r\n### `tf.reduce_min(input_tensor, reduction_indices=None, keep_dims=False, name=None)` \r\n\r\nComputes the minimum of elements across dimensions of a tensor.\r\n\r\nReduces `input_tensor` along the dimensions given in `reduction_indices`.\r\nUnless `keep_dims` is true, the rank of the tensor is reduced by 1 for each\r\nentry in `reduction_indices`. If `keep_dims` is true, the reduced dimensions\r\nare retained with length 1.\r\n\r\nIf `reduction_indices` has no entries, all dimensions are reduced, and a\r\ntensor with a single element is returned.\r\n\r\n##### Args: \r\n\r\n\r\n* `input_tensor`: The tensor to reduce. Should have numeric type.\r\n* `reduction_indices`: The dimensions to reduce. If `None` (the defaut),\r\n reduces all dimensions.\r\n* `keep_dims`: If true, retains reduced dimensions with length 1.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The reduced tensor.\r\n\r\n\r\n- - -\r\n\r\n### `tf.reduce_max(input_tensor, reduction_indices=None, keep_dims=False, name=None)` \r\n\r\nComputes the maximum of elements across dimensions of a tensor.\r\n\r\nReduces `input_tensor` along the dimensions given in `reduction_indices`.\r\nUnless `keep_dims` is true, the rank of the tensor is reduced by 1 for each\r\nentry in `reduction_indices`. If `keep_dims` is true, the reduced dimensions\r\nare retained with length 1.\r\n\r\nIf `reduction_indices` has no entries, all dimensions are reduced, and a\r\ntensor with a single element is returned.\r\n\r\n##### Args: \r\n\r\n\r\n* `input_tensor`: The tensor to reduce. Should have numeric type.\r\n* `reduction_indices`: The dimensions to reduce. If `None` (the defaut),\r\n reduces all dimensions.\r\n* `keep_dims`: If true, retains reduced dimensions with length 1.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The reduced tensor.\r\n\r\n\r\n- - -\r\n\r\n### `tf.reduce_mean(input_tensor, reduction_indices=None, keep_dims=False, name=None)` \r\n\r\nComputes the mean of elements across dimensions of a tensor.\r\n\r\nReduces `input_tensor` along the dimensions given in `reduction_indices`.\r\nUnless `keep_dims` is true, the rank of the tensor is reduced by 1 for each\r\nentry in `reduction_indices`. If `keep_dims` is true, the reduced dimensions\r\nare retained with length 1.\r\n\r\nIf `reduction_indices` has no entries, all dimensions are reduced, and a\r\ntensor with a single element is returned.\r\n\r\nFor example:\r\n\r\n```python\r\n# 'x' is [[1., 1. ]]\r\n# [2., 2.]]\r\ntf.reduce_mean(x) ==> 1.5\r\ntf.reduce_mean(x, 0) ==> [1.5, 1.5]\r\ntf.reduce_mean(x, 1) ==> [1., 2.]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `input_tensor`: The tensor to reduce. Should have numeric type.\r\n* `reduction_indices`: The dimensions to reduce. If `None` (the defaut),\r\n reduces all dimensions.\r\n* `keep_dims`: If true, retains reduced dimensions with length 1.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The reduced tensor.\r\n\r\n\r\n- - -\r\n\r\n### `tf.reduce_all(input_tensor, reduction_indices=None, keep_dims=False, name=None)` \r\n\r\nComputes the \"logical and\" of elements across dimensions of a tensor.\r\n\r\nReduces `input_tensor` along the dimensions given in `reduction_indices`.\r\nUnless `keep_dims` is true, the rank of the tensor is reduced by 1 for each\r\nentry in `reduction_indices`. If `keep_dims` is true, the reduced dimensions\r\nare retained with length 1.\r\n\r\nIf `reduction_indices` has no entries, all dimensions are reduced, and a\r\ntensor with a single element is returned.\r\n\r\nFor example:\r\n\r\n```python\r\n# 'x' is [[True, True]]\r\n# [False, False]]\r\ntf.reduce_all(x) ==> False\r\ntf.reduce_all(x, 0) ==> [False, False]\r\ntf.reduce_all(x, 1) ==> [True, False]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `input_tensor`: The boolean tensor to reduce.\r\n* `reduction_indices`: The dimensions to reduce. If `None` (the defaut),\r\n reduces all dimensions.\r\n* `keep_dims`: If true, retains reduced dimensions with length 1.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The reduced tensor.\r\n\r\n\r\n- - -\r\n\r\n### `tf.reduce_any(input_tensor, reduction_indices=None, keep_dims=False, name=None)` \r\n\r\nComputes the \"logical or\" of elements across dimensions of a tensor.\r\n\r\nReduces `input_tensor` along the dimensions given in `reduction_indices`.\r\nUnless `keep_dims` is true, the rank of the tensor is reduced by 1 for each\r\nentry in `reduction_indices`. If `keep_dims` is true, the reduced dimensions\r\nare retained with length 1.\r\n\r\nIf `reduction_indices` has no entries, all dimensions are reduced, and a\r\ntensor with a single element is returned.\r\n\r\nFor example:\r\n\r\n```python\r\n# 'x' is [[True, True]]\r\n# [False, False]]\r\ntf.reduce_any(x) ==> True\r\ntf.reduce_any(x, 0) ==> [True, True]\r\ntf.reduce_any(x, 1) ==> [True, False]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `input_tensor`: The boolean tensor to reduce.\r\n* `reduction_indices`: The dimensions to reduce. If `None` (the defaut),\r\n reduces all dimensions.\r\n* `keep_dims`: If true, retains reduced dimensions with length 1.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The reduced tensor.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.accumulate_n(inputs, shape=None, tensor_dtype=None, name=None)` \r\n\r\nReturns the element-wise sum of a list of tensors.\r\n\r\nOptionally, pass `shape` and `tensor_dtype` for shape and type checking,\r\notherwise, these are inferred.\r\n\r\nFor example:\r\n\r\n```python\r\n# tensor 'a' is [[1, 2], [3, 4]\r\n# tensor `b` is [[5, 0], [0, 6]]\r\ntf.accumulate_n([a, b, a]) ==> [[7, 4], [6, 14]]\r\n\r\n# Explicitly pass shape and type\r\ntf.accumulate_n([a, b, a], shape=[2, 2], tensor_dtype=tf.int32)\r\n ==> [[7, 4], [6, 14]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `inputs`: A list of `Tensor` objects, each with same shape and type.\r\n* `shape`: Shape of elements of `inputs`.\r\n* `tensor_dtype`: The type of `inputs`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of same shape and type as the elements of `inputs`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `inputs` don't all have same shape and dtype or the shape\r\n cannot be inferred.\r\n\r\n\r\n\r\n## Segmentation \r\n\r\nTensorFlow provides several operations that you can use to perform common\r\nmath computations on tensor segments.\r\nHere a segmentation is a partitioning of a tensor along\r\nthe first dimension, i.e. it defines a mapping from the first dimension onto\r\n`segment_ids`. The `segment_ids` tensor should be the size of\r\nthe first dimension, `d0`, with consecutive IDs in the range `0` to `k`,\r\nwhere `k\r\n![](\"../images/SegmentSum.png\")
\r\n
\r\n\r\n##### Args: \r\n\r\n\r\n* `data`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`, `uint8`, `int16`, `int8`.\r\n* `segment_ids`: A `Tensor`. Must be one of the following types: `int32`, `int64`.\r\n A 1-D tensor whose rank is equal to the rank of `data`'s\r\n first dimension. Values should be sorted and can be repeated.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `data`.\r\n Has same shape as data, except for dimension_0 which\r\n has size `k`, the number of segments.\r\n\r\n\r\n- - -\r\n\r\n### `tf.segment_prod(data, segment_ids, name=None)` \r\n\r\nComputes the product along segments of a tensor.\r\n\r\nRead [the section on\r\nSegmentation](../../api_docs/python/math_ops.md#segmentation) for an explanation\r\nof segments.\r\n\r\nComputes a tensor such that\r\n\\\\(output_i = \\prod_j data_j\\\\) where the product is over `j` such\r\nthat `segment_ids[j] == i`.\r\n\r\n\r\n\r\n![](\"../images/SegmentProd.png\")
\r\n
\r\n\r\n##### Args: \r\n\r\n\r\n* `data`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`, `uint8`, `int16`, `int8`.\r\n* `segment_ids`: A `Tensor`. Must be one of the following types: `int32`, `int64`.\r\n A 1-D tensor whose rank is equal to the rank of `data`'s\r\n first dimension. Values should be sorted and can be repeated.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `data`.\r\n Has same shape as data, except for dimension_0 which\r\n has size `k`, the number of segments.\r\n\r\n\r\n- - -\r\n\r\n### `tf.segment_min(data, segment_ids, name=None)` \r\n\r\nComputes the minimum along segments of a tensor.\r\n\r\nRead [the section on\r\nSegmentation](../../api_docs/python/math_ops.md#segmentation) for an explanation\r\nof segments.\r\n\r\nComputes a tensor such that\r\n\\\\(output_i = \\min_j(data_j)\\\\) where `min` is over `j` such\r\nthat `segment_ids[j] == i`.\r\n\r\n\r\n\r\n![](\"../images/SegmentMin.png\")
\r\n
\r\n\r\n##### Args: \r\n\r\n\r\n* `data`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`, `uint8`, `int16`, `int8`.\r\n* `segment_ids`: A `Tensor`. Must be one of the following types: `int32`, `int64`.\r\n A 1-D tensor whose rank is equal to the rank of `data`'s\r\n first dimension. Values should be sorted and can be repeated.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `data`.\r\n Has same shape as data, except for dimension_0 which\r\n has size `k`, the number of segments.\r\n\r\n\r\n- - -\r\n\r\n### `tf.segment_max(data, segment_ids, name=None)` \r\n\r\nComputes the maximum along segments of a tensor.\r\n\r\nRead [the section on Segmentation](../../api_docs/python/math_ops.md#segmentation)\r\nfor an explanation of segments.\r\n\r\nComputes a tensor such that\r\n\\\\(output_i = \\max_j(data_j)\\\\) where `max` is over `j` such\r\nthat `segment_ids[j] == i`.\r\n\r\n\r\n\r\n![](\"../images/SegmentMax.png\")
\r\n
\r\n\r\n##### Args: \r\n\r\n\r\n* `data`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`, `uint8`, `int16`, `int8`.\r\n* `segment_ids`: A `Tensor`. Must be one of the following types: `int32`, `int64`.\r\n A 1-D tensor whose rank is equal to the rank of `data`'s\r\n first dimension. Values should be sorted and can be repeated.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `data`.\r\n Has same shape as data, except for dimension_0 which\r\n has size `k`, the number of segments.\r\n\r\n\r\n- - -\r\n\r\n### `tf.segment_mean(data, segment_ids, name=None)` \r\n\r\nComputes the mean along segments of a tensor.\r\n\r\nRead [the section on\r\nSegmentation](../../api_docs/python/math_ops.md#segmentation) for an explanation\r\nof segments.\r\n\r\nComputes a tensor such that\r\n\\\\(output_i = \\frac{\\sum_j data_j}{N}\\\\) where `mean` is\r\nover `j` such that `segment_ids[j] == i` and `N` is the total number of\r\nvalues summed.\r\n\r\n\r\n\r\n![](\"../images/SegmentMean.png\")
\r\n
\r\n\r\n##### Args: \r\n\r\n\r\n* `data`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`, `uint8`, `int16`, `int8`.\r\n* `segment_ids`: A `Tensor`. Must be one of the following types: `int32`, `int64`.\r\n A 1-D tensor whose rank is equal to the rank of `data`'s\r\n first dimension. Values should be sorted and can be repeated.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `data`.\r\n Has same shape as data, except for dimension_0 which\r\n has size `k`, the number of segments.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.unsorted_segment_sum(data, segment_ids, num_segments, name=None)` \r\n\r\nComputes the sum along segments of a tensor.\r\n\r\nRead [the section on\r\nSegmentation](../../api_docs/python/math_ops.md#segmentation) for an explanation\r\nof segments.\r\n\r\nComputes a tensor such that\r\n\\\\(output_i = \\sum_j data_j\\\\) where sum is over `j` such\r\nthat `segment_ids[j] == i`. Unlike `SegmentSum`, `segment_ids`\r\nneed not be sorted and need not cover all values in the full\r\n range of valid values.\r\n\r\nIf the sum is empty for a given segment ID `i`, `output[i] = 0`.\r\n\r\n`num_segments` should equal the number of distinct segment IDs.\r\n\r\n\r\n\r\n![](\"../images/UnsortedSegmentSum.png\")
\r\n
\r\n\r\n##### Args: \r\n\r\n\r\n* `data`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`, `uint8`, `int16`, `int8`.\r\n* `segment_ids`: A `Tensor`. Must be one of the following types: `int32`, `int64`.\r\n A 1-D tensor whose rank is equal to the rank of `data`'s\r\n first dimension.\r\n* `num_segments`: A `Tensor` of type `int32`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `data`.\r\n Has same shape as data, except for dimension_0 which\r\n has size `num_segments`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.sparse_segment_sum(data, indices, segment_ids, name=None)` \r\n\r\nComputes the sum along sparse segments of a tensor.\r\n\r\nRead [the section on\r\nSegmentation](../../api_docs/python/math_ops.md#segmentation) for an explanation\r\nof segments.\r\n\r\nLike `SegmentSum`, but `segment_ids` can have rank less than `data`'s first\r\ndimension, selecting a subset of dimension_0, specified by `indices`.\r\n\r\nFor example:\r\n\r\n```prettyprint\r\nc = tf.constant([[1,2,3,4], [-1,-2,-3,-4], [5,6,7,8]])\r\n\r\n# Select two rows, one segment.\r\ntf.sparse_segment_sum(c, tf.constant([0, 1]), tf.constant([0, 0]))\r\n ==> [[0 0 0 0]]\r\n\r\n# Select two rows, two segment.\r\ntf.sparse_segment_sum(c, tf.constant([0, 1]), tf.constant([0, 1]))\r\n ==> [[ 1 2 3 4]\r\n [-1 -2 -3 -4]]\r\n\r\n# Select all rows, two segments.\r\ntf.sparse_segment_sum(c, tf.constant([0, 1, 2]), tf.constant([0, 0, 1]))\r\n ==> [[0 0 0 0]\r\n [5 6 7 8]]\r\n\r\n# Which is equivalent to:\r\ntf.segment_sum(c, tf.constant([0, 0, 1]))\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `data`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`, `uint8`, `int16`, `int8`.\r\n* `indices`: A `Tensor` of type `int32`.\r\n A 1-D tensor. Has same rank as `segment_ids`.\r\n* `segment_ids`: A `Tensor` of type `int32`.\r\n A 1-D tensor. Values should be sorted and can be repeated.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `data`.\r\n Has same shape as data, except for dimension_0 which\r\n has size `k`, the number of segments.\r\n\r\n\r\n- - -\r\n\r\n### `tf.sparse_segment_mean(data, indices, segment_ids, name=None)` \r\n\r\nComputes the mean along sparse segments of a tensor.\r\n\r\nRead [the section on\r\nSegmentation](../../api_docs/python/math_ops.md#segmentation) for an explanation\r\nof segments.\r\n\r\nLike `SegmentMean`, but `segment_ids` can have rank less than `data`'s first\r\ndimension, selecting a subset of dimension_0, specified by `indices`.\r\n\r\n##### Args: \r\n\r\n\r\n* `data`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n* `indices`: A `Tensor` of type `int32`.\r\n A 1-D tensor. Has same rank as `segment_ids`.\r\n* `segment_ids`: A `Tensor` of type `int32`.\r\n A 1-D tensor. Values should be sorted and can be repeated.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `data`.\r\n Has same shape as data, except for dimension_0 which\r\n has size `k`, the number of segments.\r\n\r\n\r\n\r\n\r\n## Sequence Comparison and Indexing \r\n\r\nTensorFlow provides several operations that you can use to add sequence\r\ncomparison and index extraction to your graph. You can use these operations to\r\ndetermine sequence differences and determine the indexes of specific values in\r\na tensor.\r\n\r\n- - -\r\n\r\n### `tf.argmin(input, dimension, name=None)` \r\n\r\nReturns the index with the smallest value across dimensions of a tensor.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `qint8`, `quint8`, `qint32`.\r\n* `dimension`: A `Tensor` of type `int32`.\r\n int32, 0 <= dimension < rank(input). Describes which dimension\r\n of the input Tensor to reduce across. For vectors, use dimension = 0.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `int64`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.argmax(input, dimension, name=None)` \r\n\r\nReturns the index with the largest value across dimensions of a tensor.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `qint8`, `quint8`, `qint32`.\r\n* `dimension`: A `Tensor` of type `int32`.\r\n int32, 0 <= dimension < rank(input). Describes which dimension\r\n of the input Tensor to reduce across. For vectors, use dimension = 0.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `int64`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.listdiff(x, y, name=None)` \r\n\r\nComputes the difference between two lists of numbers.\r\n\r\nGiven a list `x` and a list `y`, this operation returns a list `out` that\r\nrepresents all numbers that are in `x` but not in `y`. The returned list `out`\r\nis sorted in the same order that the numbers appear in `x` (duplicates are\r\npreserved). This operation also returns a list `idx` that represents the\r\nposition of each `out` element in `x`. In other words:\r\n\r\n`out[i] = x[idx[i]] for i in [0, 1, ..., len(out) - 1]`\r\n\r\nFor example, given this input:\r\n\r\n```prettyprint\r\nx = [1, 2, 3, 4, 5, 6]\r\ny = [1, 3, 5]\r\n```\r\n\r\nThis operation would return:\r\n\r\n```prettyprint\r\nout ==> [2, 4, 6]\r\nidx ==> [1, 3, 5]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. 1-D. Values to keep.\r\n* `y`: A `Tensor`. Must have the same type as `x`. 1-D. Values to remove.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tuple of `Tensor` objects (out, idx).\r\n\r\n* `out`: A `Tensor`. Has the same type as `x`. 1-D. Values present in `x` but not in `y`.\r\n* `idx`: A `Tensor` of type `int32`. 1-D. Positions of `x` values preserved in `out`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.where(input, name=None)` \r\n\r\nReturns locations of true values in a boolean tensor.\r\n\r\nThis operation returns the coordinates of true elements in `input`. The\r\ncoordinates are returned in a 2-D tensor where the first dimension (rows)\r\nrepresents the number of true elements, and the second dimension (columns)\r\nrepresents the coordinates of the true elements. Keep in mind, the shape of\r\nthe output tensor can vary depending on how many true values there are in\r\n`input`. Indices are output in row-major order.\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# 'input' tensor is [[True, False]\r\n# [True, False]]\r\n# 'input' has two true values, so output has two coordinates.\r\n# 'input' has rank of 2, so coordinates have two indices.\r\nwhere(input) ==> [[0, 0],\r\n [1, 0]]\r\n\r\n# `input` tensor is [[[True, False]\r\n# [True, False]]\r\n# [[False, True]\r\n# [False, True]]\r\n# [[False, False]\r\n# [False, True]]]\r\n# 'input' has 5 true values, so output has 5 coordinates.\r\n# 'input' has rank of 3, so coordinates have three indices.\r\nwhere(input) ==> [[0, 0, 0],\r\n [0, 1, 0],\r\n [1, 0, 1],\r\n [1, 1, 1],\r\n [2, 1, 1]]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor` of type `bool`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `int64`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.unique(x, name=None)` \r\n\r\nFinds unique elements in a 1-D tensor.\r\n\r\nThis operation returns a tensor `y` containing all of the unique elements of `x`\r\nsorted in the same order that they occur in `x`. This operation also returns a\r\ntensor `idx` the same size as `x` that contains the index of each value of `x`\r\nin the unique output `y`. In other words:\r\n\r\n`y[idx[i]] = x[i] for i in [0, 1,...,rank(x) - 1]`\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# tensor 'x' is [1, 1, 2, 4, 4, 4, 7, 8, 8]\r\ny, idx = unique(x)\r\ny ==> [1, 2, 4, 7, 8]\r\nidx ==> [0, 0, 1, 2, 2, 2, 3, 4, 4]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`. 1-D.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tuple of `Tensor` objects (y, idx).\r\n\r\n* `y`: A `Tensor`. Has the same type as `x`. 1-D.\r\n* `idx`: A `Tensor` of type `int32`. 1-D.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.edit_distance(hypothesis, truth, normalize=True, name='edit_distance')` \r\n\r\nComputes the Levenshtein distance between sequences.\r\n\r\nThis operation takes variable-length sequences (`hypothesis` and `truth`),\r\neach provided as a `SparseTensor`, and computes the Levenshtein distance.\r\nYou can normalize the edit distance by length of `truth` by setting\r\n`normalize` to true.\r\n\r\nFor example, given the following input:\r\n\r\n```python\r\n# 'hypothesis' is a tensor of shape `[2, 1]` with variable-length values:\r\n# (0,0) = [\"a\"]\r\n# (1,0) = [\"b\"]\r\nhypothesis = tf.SparseTensor(\r\n [[0, 0, 0],\r\n [1, 0, 0]],\r\n [\"a\", \"b\"]\r\n (2, 1, 1))\r\n\r\n# 'truth' is a tensor of shape `[2, 2]` with variable-length values:\r\n# (0,0) = []\r\n# (0,1) = [\"a\"]\r\n# (1,0) = [\"b\", \"c\"]\r\n# (1,1) = [\"a\"]\r\ntruth = tf.SparseTensor(\r\n [[0, 1, 0],\r\n [1, 0, 0],\r\n [1, 0, 1],\r\n [1, 1, 0]]\r\n [\"a\", \"b\", \"c\", \"a\"],\r\n (2, 2, 2))\r\n\r\nnormalize = True\r\n```\r\n\r\nThis operation would return the following:\r\n\r\n```python\r\n# 'output' is a tensor of shape `[2, 2]` with edit distances normalized\r\n# by 'truth' lengths.\r\noutput ==> [[inf, 1.0], # (0,0): no truth, (0,1): no hypothesis\r\n [0.5, 1.0]] # (1,0): addition, (1,1): no hypothesis\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `hypothesis`: A `SparseTensor` containing hypothesis sequences.\r\n* `truth`: A `SparseTensor` containing truth sequences.\r\n* `normalize`: A `bool`. If `True`, normalizes the Levenshtein distance by\r\n length of `truth.`\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A dense `Tensor` with rank `R - 1`, where R is the rank of the\r\n `SparseTensor` inputs `hypothesis` and `truth`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If either `hypothesis` or `truth` are not a `SparseTensor`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.invert_permutation(x, name=None)` \r\n\r\nComputes the inverse permutation of a tensor.\r\n\r\nThis operation computes the inverse of an index permutation. It takes a 1-D\r\ninteger tensor `x`, which represents the indices of a zero-based array, and\r\nswaps each value with its index position. In other words, for an ouput tensor\r\n`y` and an input tensor `x`, this operation computes the following:\r\n\r\n`y[x[i]] = i for i in [0, 1, ..., len(x) - 1]`\r\n\r\nThe values must include 0. There can be no duplicate values or negative values.\r\n\r\nFor example:\r\n\r\n```prettyprint\r\n# tensor `x` is [3, 4, 0, 2, 1]\r\ninvert_permutation(x) ==> [2, 4, 3, 0, 1]\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor` of type `int32`. 1-D.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `int32`. 1-D.\r\n\r\n\r\n"
},
{
"path": "SOURCE/api_docs/python/nn.md",
"content": "\r\n\r\n# Neural Network \r\n\r\nNote: Functions taking `Tensor` arguments can also take anything accepted by\r\n[`tf.convert_to_tensor`](../../api_docs/python/framework.md#convert_to_tensor).\r\n\r\n\r\n## Contents\r\n### [Neural Network](#AUTOGENERATED-neural-network)\r\n* [Activation Functions](#AUTOGENERATED-activation-functions)\r\n * [`tf.nn.relu(features, name=None)`](#relu)\r\n * [`tf.nn.relu6(features, name=None)`](#relu6)\r\n * [`tf.nn.softplus(features, name=None)`](#softplus)\r\n * [`tf.nn.dropout(x, keep_prob, noise_shape=None, seed=None, name=None)`](#dropout)\r\n * [`tf.nn.bias_add(value, bias, name=None)`](#bias_add)\r\n * [`tf.sigmoid(x, name=None)`](#sigmoid)\r\n * [`tf.tanh(x, name=None)`](#tanh)\r\n* [Convolution](#AUTOGENERATED-convolution)\r\n * [`tf.nn.conv2d(input, filter, strides, padding, use_cudnn_on_gpu=None, name=None)`](#conv2d)\r\n * [`tf.nn.depthwise_conv2d(input, filter, strides, padding, name=None)`](#depthwise_conv2d)\r\n * [`tf.nn.separable_conv2d(input, depthwise_filter, pointwise_filter, strides, padding, name=None)`](#separable_conv2d)\r\n* [Pooling](#AUTOGENERATED-pooling)\r\n * [`tf.nn.avg_pool(value, ksize, strides, padding, name=None)`](#avg_pool)\r\n * [`tf.nn.max_pool(value, ksize, strides, padding, name=None)`](#max_pool)\r\n * [`tf.nn.max_pool_with_argmax(input, ksize, strides, padding, Targmax=None, name=None)`](#max_pool_with_argmax)\r\n* [Normalization](#AUTOGENERATED-normalization)\r\n * [`tf.nn.l2_normalize(x, dim, epsilon=1e-12, name=None)`](#l2_normalize)\r\n * [`tf.nn.local_response_normalization(input, depth_radius=None, bias=None, alpha=None, beta=None, name=None)`](#local_response_normalization)\r\n * [`tf.nn.moments(x, axes, name=None)`](#moments)\r\n* [Losses](#AUTOGENERATED-losses)\r\n * [`tf.nn.l2_loss(t, name=None)`](#l2_loss)\r\n* [Classification](#AUTOGENERATED-classification)\r\n * [`tf.nn.sigmoid_cross_entropy_with_logits(logits, targets, name=None)`](#sigmoid_cross_entropy_with_logits)\r\n * [`tf.nn.softmax(logits, name=None)`](#softmax)\r\n * [`tf.nn.softmax_cross_entropy_with_logits(logits, labels, name=None)`](#softmax_cross_entropy_with_logits)\r\n* [Embeddings](#AUTOGENERATED-embeddings)\r\n * [`tf.nn.embedding_lookup(params, ids, name=None)`](#embedding_lookup)\r\n* [Evaluation](#AUTOGENERATED-evaluation)\r\n * [`tf.nn.top_k(input, k, name=None)`](#top_k)\r\n * [`tf.nn.in_top_k(predictions, targets, k, name=None)`](#in_top_k)\r\n* [Candidate Sampling](#AUTOGENERATED-candidate-sampling)\r\n * [Sampled Loss Functions](#AUTOGENERATED-sampled-loss-functions)\r\n * [`tf.nn.nce_loss(weights, biases, inputs, labels, num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=False, name='nce_loss')`](#nce_loss)\r\n * [`tf.nn.sampled_softmax_loss(weights, biases, inputs, labels, num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=True, name='sampled_softmax_loss')`](#sampled_softmax_loss)\r\n * [Candidate Samplers](#AUTOGENERATED-candidate-samplers)\r\n * [`tf.nn.uniform_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, seed=None, name=None)`](#uniform_candidate_sampler)\r\n * [`tf.nn.log_uniform_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, seed=None, name=None)`](#log_uniform_candidate_sampler)\r\n * [`tf.nn.learned_unigram_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, seed=None, name=None)`](#learned_unigram_candidate_sampler)\r\n * [`tf.nn.fixed_unigram_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, vocab_file='', distortion=0.0, num_reserved_ids=0, num_shards=1, shard=0, unigrams=[], seed=None, name=None)`](#fixed_unigram_candidate_sampler)\r\n * [Miscellaneous candidate sampling utilities](#AUTOGENERATED-miscellaneous-candidate-sampling-utilities)\r\n * [`tf.nn.compute_accidental_hits(true_classes, sampled_candidates, num_true, seed=None, name=None)`](#compute_accidental_hits)\r\n\r\n\r\n\r\n\r\n## Activation Functions \r\n\r\nThe activation ops provide different types of nonlinearities for use in\r\nneural networks. These include smooth nonlinearities (`sigmoid`,\r\n`tanh`, and `softplus`), continuous but not everywhere differentiable\r\nfunctions (`relu`, `relu6`, and `relu_x`), and random regularization\r\n(`dropout`).\r\n\r\nAll activation ops apply componentwise, and produce a tensor of the same\r\nshape as the input tensor.\r\n\r\n- - -\r\n\r\n### `tf.nn.relu(features, name=None)` \r\n\r\nComputes rectified linear: `max(features, 0)`.\r\n\r\n##### Args: \r\n\r\n\r\n* `features`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`, `uint8`, `int16`, `int8`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `features`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.relu6(features, name=None)` \r\n\r\nComputes Rectified Linear 6: `min(max(features, 0), 6)`.\r\n\r\n##### Args: \r\n\r\n\r\n* `features`: A `Tensor` with type `float`, `double`, `int32`, `int64`, `uint8`,\r\n `int16`, or `int8`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` with the same type as `features`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.softplus(features, name=None)` \r\n\r\nComputes softplus: `log(exp(features) + 1)`.\r\n\r\n##### Args: \r\n\r\n\r\n* `features`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`, `uint8`, `int16`, `int8`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `features`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.dropout(x, keep_prob, noise_shape=None, seed=None, name=None)` \r\n\r\nComputes dropout.\r\n\r\nWith probability `keep_prob`, outputs the input element scaled up by\r\n`1 / keep_prob`, otherwise outputs `0`. The scaling is so that the expected\r\nsum is unchanged.\r\n\r\nBy default, each element is kept or dropped independently. If `noise_shape`\r\nis specified, it must be\r\n[broadcastable](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html)\r\nto the shape of `x`, and only dimensions with `noise_shape[i] == shape(x)[i]`\r\nwill make independent decisions. For example, if `shape(x) = [k, l, m, n]`\r\nand `noise_shape = [k, 1, 1, n]`, each batch and channel component will be\r\nkept independently and each row and column will be kept or not kept together.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A tensor.\r\n* `keep_prob`: A Python float. The probability that each element is kept.\r\n* `noise_shape`: A 1-D `Tensor` of type `int32`, representing the\r\n shape for randomly generated keep/drop flags.\r\n* `seed`: A Python integer. Used to create random seeds. See\r\n [`set_random_seed`](../../api_docs/python/constant_op.md#set_random_seed)\r\n for behavior.\r\n* `name`: A name for this operation (optional).\r\n\r\n##### Returns: \r\n\r\n A Tensor of the same shape of `x`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `keep_prob` is not in `(0, 1]`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.bias_add(value, bias, name=None)` \r\n\r\nAdds `bias` to `value`.\r\n\r\nThis is (mostly) a special case of `tf.add` where `bias` is restricted to 1-D.\r\nBroadcasting is supported, so `value` may have any number of dimensions.\r\nUnlike `tf.add`, the type of `bias` is allowed to differ from `value` in the\r\ncase where both types are quantized.\r\n\r\n##### Args: \r\n\r\n\r\n* `value`: A `Tensor` with type `float`, `double`, `int64`, `int32`, `uint8`,\r\n `int16`, `int8`, or `complex64`.\r\n* `bias`: A 1-D `Tensor` with size matching the last dimension of `value`.\r\n Must be the same type as `value` unless `value` is a quantized type,\r\n in which case a different quantized type may be used.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` with the same type as `value`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.sigmoid(x, name=None)` \r\n\r\nComputes sigmoid of `x` element-wise.\r\n\r\nSpecifically, `y = 1 / (1 + exp(-x))`.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A Tensor with type `float`, `double`, `int32`, `complex64`, `int64`,\r\n or `qint32`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A Tensor with the same type as `x` if `x.dtype != qint32`\r\n otherwise the return type is `quint8`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.tanh(x, name=None)` \r\n\r\nComputes hyperbolic tangent of `x` element-wise.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A Tensor with type `float`, `double`, `int32`, `complex64`, `int64`,\r\n or `qint32`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A Tensor with the same type as `x` if `x.dtype != qint32` otherwise\r\n the return type is `quint8`.\r\n\r\n\r\n\r\n## Convolution \r\n\r\nThe convolution ops sweep a 2-D filter over a batch of images, applying the\r\nfilter to each window of each image of the appropriate size. The different\r\nops trade off between generic vs. specific filters:\r\n\r\n* `conv2d`: Arbitrary filters that can mix channels together.\r\n* `depthwise_conv2d`: Filters that operate on each channel independently.\r\n* `separable_conv2d`: A depthwise spatial filter followed by a pointwise filter.\r\n\r\nNote that although these ops are called \"convolution\", they are strictly\r\nspeaking \"cross-correlation\" since the filter is combined with an input window\r\nwithout reversing the filter. For details, see [the properties of\r\ncross-correlation](https://en.wikipedia.org/wiki/Cross-correlation#Properties).\r\n\r\nThe filter is applied to image patches of the same size as the filter and\r\nstrided according to the `strides` argument. `strides = [1, 1, 1, 1]` applies\r\nthe filter to a patch at every offset, `strides = [1, 2, 2, 1]` applies the\r\nfilter to every other image patch in each dimension, etc.\r\n\r\nIgnoring channels for the moment, the spatial semantics of the convolution ops\r\nare as follows. If the 4-D `input` has shape\r\n`[batch, in_height, in_width, ...]` and the 4-D `filter` has shape\r\n`[filter_height, filter_width, ...]`, then\r\n\r\n shape(output) = [batch,\r\n (in_height - filter_height + 1) / strides[1],\r\n (in_width - filter_width + 1) / strides[2],\r\n ...]\r\n\r\n output[b, i, j, :] =\r\n sum_{di, dj} input[b, strides[1] * i + di, strides[2] * j + dj, ...] *\r\n filter[di, dj, ...]\r\n\r\nSince `input` is 4-D, each `input[b, i, j, :]` is a vector. For `conv2d`, these\r\nvectors are multiplied by the `filter[di, dj, :, :]` matrices to produce new\r\nvectors. For `depthwise_conv_2d`, each scalar component `input[b, i, j, k]`\r\nis multiplied by a vector `filter[di, dj, k]`, and all the vectors are\r\nconcatenated.\r\n\r\nIn the formula for `shape(output)`, the rounding direction depends on padding:\r\n\r\n* `padding = 'SAME'`: Round down (only full size windows are considered).\r\n* `padding = 'VALID'`: Round up (partial windows are included).\r\n\r\n- - -\r\n\r\n### `tf.nn.conv2d(input, filter, strides, padding, use_cudnn_on_gpu=None, name=None)` \r\n\r\nComputes a 2-D convolution given 4-D `input` and `filter` tensors.\r\n\r\nGiven an input tensor of shape `[batch, in_height, in_width, in_channels]`\r\nand a filter / kernel tensor of shape\r\n`[filter_height, filter_width, in_channels, out_channels]`, this op\r\nperforms the following:\r\n\r\n1. Flattens the filter to a 2-D matrix with shape\r\n `[filter_height * filter_width * in_channels, output_channels]`.\r\n2. Extracts image patches from the the input tensor to form a *virtual*\r\n tensor of shape `[batch, out_height, out_width,\r\n filter_height * filter_width * in_channels]`.\r\n3. For each patch, right-multiplies the filter matrix and the image patch\r\n vector.\r\n\r\nIn detail,\r\n\r\n output[b, i, j, k] =\r\n sum_{di, dj, q} input[b, strides[1] * i + di, strides[2] * j + dj, q] *\r\n filter[di, dj, q, k]\r\n\r\nMust have `strides[0] = strides[3] = 1`. For the most common case of the same\r\nhorizontal and vertices strides, `strides = [1, stride, stride, 1]`.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n* `filter`: A `Tensor`. Must have the same type as `input`.\r\n* `strides`: A list of `ints`.\r\n 1-D of length 4. The stride of the sliding window for each dimension\r\n of `input`.\r\n* `padding`: A `string` from: `\"SAME\", \"VALID\"`.\r\n The type of padding algorithm to use.\r\n* `use_cudnn_on_gpu`: An optional `bool`. Defaults to `True`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.depthwise_conv2d(input, filter, strides, padding, name=None)` \r\n\r\nDepthwise 2-D convolution.\r\n\r\nGiven an input tensor of shape `[batch, in_height, in_width, in_channels]`\r\nand a filter tensor of shape\r\n`[filter_height, filter_width, in_channels, channel_multiplier]`\r\ncontaining `in_channels` convolutional filters of depth 1, `depthwise_conv2d`\r\napplies a different filter to each input channel (expanding from 1 channel\r\nto `channel_multiplier` channels for each), then concatenates the results\r\ntogether. The output has `in_channels * channel_multiplier` channels.\r\n\r\nIn detail,\r\n\r\n output[b, i, j, k * channel_multiplier + q] =\r\n sum_{di, dj} input[b, strides[1] * i + di, strides[2] * j + dj, k] *\r\n filter[di, dj, k, q]\r\n\r\nMust have `strides[0] = strides[3] = 1`. For the most common case of the\r\nsame horizontal and vertical strides, `strides = [1, stride, stride, 1]`.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: 4-D with shape `[batch, in_height, in_width, in_channels]`.\r\n* `filter`: 4-D with shape\r\n `[filter_height, filter_width, in_channels, channel_multiplier]`.\r\n* `strides`: 1-D of size 4. The stride of the sliding window for each\r\n dimension of `input`.\r\n* `padding`: A string, either `'VALID'` or `'SAME'`. The padding algorithm.\r\n* `name`: A name for this operation (optional).\r\n\r\n##### Returns: \r\n\r\n A 4-D `Tensor` of shape\r\n `[batch, out_height, out_width, in_channels * channel_multiplier].`\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.separable_conv2d(input, depthwise_filter, pointwise_filter, strides, padding, name=None)` \r\n\r\n2-D convolution with separable filters.\r\n\r\nPerforms a depthwise convolution that acts separately on channels followed by\r\na pointwise convolution that mixes channels. Note that this is separability\r\nbetween dimensions `[1, 2]` and `3`, not spatial separability between\r\ndimensions `1` and `2`.\r\n\r\nIn detail,\r\n\r\n output[b, i, j, k] = sum_{di, dj, q, r]\r\n input[b, strides[1] * i + di, strides[2] * j + dj, q] *\r\n depthwise_filter[di, dj, q, r] *\r\n pointwise_filter[0, 0, q * channel_multiplier + r, k]\r\n\r\n`strides` controls the strides for the depthwise convolution only, since\r\nthe pointwise convolution has implicit strides of `[1, 1, 1, 1]`. Must have\r\n`strides[0] = strides[3] = 1`. For the most common case of the same\r\nhorizontal and vertical strides, `strides = [1, stride, stride, 1]`.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: 4-D `Tensor` with shape `[batch, in_height, in_width, in_channels]`.\r\n* `depthwise_filter`: 4-D `Tensor` with shape\r\n `[filter_height, filter_width, in_channels, channel_multiplier]`.\r\n Contains `in_channels` convolutional filters of depth 1.\r\n* `pointwise_filter`: 4-D `Tensor` with shape\r\n `[1, 1, channel_multiplier * in_channels, out_channels]`. Pointwise\r\n filter to mix channels after `depthwise_filter` has convolved spatially.\r\n* `strides`: 1-D of size 4. The strides for the depthwise convolution for\r\n each dimension of `input`.\r\n* `padding`: A string, either `'VALID'` or `'SAME'`. The padding algorithm.\r\n* `name`: A name for this operation (optional).\r\n\r\n##### Returns: \r\n\r\n A 4-D `Tensor` of shape `[batch, out_height, out_width, out_channels]`.\r\n\r\n\r\n\r\n## Pooling \r\n\r\nThe pooling ops sweep a rectangular window over the input tensor, computing a\r\nreduction operation for each window (average, max, or max with argmax). Each\r\npooling op uses rectangular windows of size `ksize` separated by offset\r\n`strides`. For example, if `strides` is all ones every window is used, if\r\n`strides` is all twos every other window is used in each dimension, etc.\r\n\r\nIn detail, the output is\r\n\r\n output[i] = reduce(value[strides * i:strides * i + ksize])\r\n\r\nfor each tuple of indices `i`. The output shape is\r\n\r\n shape(output) = (shape(value) - ksize + 1) / strides\r\n\r\nwhere the rounding direction depends on padding:\r\n\r\n* `padding = 'SAME'`: Round down (only full size windows are considered).\r\n* `padding = 'VALID'`: Round up (partial windows are included).\r\n\r\n- - -\r\n\r\n### `tf.nn.avg_pool(value, ksize, strides, padding, name=None)` \r\n\r\nPerforms the average pooling on the input.\r\n\r\nEach entry in `output` is the mean of the corresponding size `ksize`\r\nwindow in `value`.\r\n\r\n##### Args: \r\n\r\n\r\n* `value`: A 4-D `Tensor` of shape `[batch, height, width, channels]` and type\r\n `float32`, `float64`, `qint8`, `quint8`, or `qint32`.\r\n* `ksize`: A list of ints that has length >= 4.\r\n The size of the window for each dimension of the input tensor.\r\n* `strides`: A list of ints that has length >= 4.\r\n The stride of the sliding window for each dimension of the\r\n input tensor.\r\n* `padding`: A string, either `'VALID'` or `'SAME'`. The padding algorithm.\r\n* `name`: Optional name for the operation.\r\n\r\n##### Returns: \r\n\r\n A `Tensor` with the same type as `value`. The average pooled output tensor.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.max_pool(value, ksize, strides, padding, name=None)` \r\n\r\nPerforms the max pooling on the input.\r\n\r\n##### Args: \r\n\r\n\r\n* `value`: A 4-D `Tensor` with shape `[batch, height, width, channels]` and\r\n type `float32`, `float64`, `qint8`, `quint8`, `qint32`.\r\n* `ksize`: A list of ints that has length >= 4. The size of the window for\r\n each dimension of the input tensor.\r\n* `strides`: A list of ints that has length >= 4. The stride of the sliding\r\n window for each dimension of the input tensor.\r\n* `padding`: A string, either `'VALID'` or `'SAME'`. The padding algorithm.\r\n* `name`: Optional name for the operation.\r\n\r\n##### Returns: \r\n\r\n A `Tensor` with the same type as `value`. The max pooled output tensor.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.max_pool_with_argmax(input, ksize, strides, padding, Targmax=None, name=None)` \r\n\r\nPerforms max pooling on the input and outputs both max values and indices.\r\n\r\nThe indices in `argmax` are flattened, so that a maximum value at position\r\n`[b, y, x, c]` becomes flattened index\r\n`((b * height + y) * width + x) * channels + c`.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor` of type `float32`.\r\n 4-D with shape `[batch, height, width, channels]`. Input to pool over.\r\n* `ksize`: A list of `ints` that has length `>= 4`.\r\n The size of the window for each dimension of the input tensor.\r\n* `strides`: A list of `ints` that has length `>= 4`.\r\n The stride of the sliding window for each dimension of the\r\n input tensor.\r\n* `padding`: A `string` from: `\"SAME\", \"VALID\"`.\r\n The type of padding algorithm to use.\r\n* `Targmax`: An optional `tf.DType` from: `tf.int32, tf.int64`. Defaults to `tf.int64`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tuple of `Tensor` objects (output, argmax).\r\n\r\n* `output`: A `Tensor` of type `float32`. The max pooled output tensor.\r\n* `argmax`: A `Tensor` of type `Targmax`. 4-D. The flattened indices of the max values chosen for each output.\r\n\r\n\r\n\r\n## Normalization \r\n\r\nNormalization is useful to prevent neurons from saturating when inputs may\r\nhave varying scale, and to aid generalization.\r\n\r\n- - -\r\n\r\n### `tf.nn.l2_normalize(x, dim, epsilon=1e-12, name=None)` \r\n\r\nNormalizes along dimension `dim` using an L2 norm.\r\n\r\nFor a 1-D tensor with `dim = 0`, computes\r\n\r\n output = x / sqrt(max(sum(x**2), epsilon))\r\n\r\nFor `x` with more dimensions, independently normalizes each 1-D slice along\r\ndimension `dim`.\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`.\r\n* `dim`: Dimension along which to normalize.\r\n* `epsilon`: A lower bound value for the norm. Will use `sqrt(epsilon)` as the\r\n divisor if `norm < sqrt(epsilon)`.\r\n* `name`: A name for this operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` with the same shape as `x`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.local_response_normalization(input, depth_radius=None, bias=None, alpha=None, beta=None, name=None)` \r\n\r\nLocal Response Normalization.\r\n\r\nThe 4-D `input` tensor is treated as a 3-D array of 1-D vectors (along the last\r\ndimension), and each vector is normalized independently. Within a given vector,\r\neach component is divided by the weighted, squared sum of inputs within\r\n`depth_radius`. In detail,\r\n\r\n sqr_sum[a, b, c, d] =\r\n sum(input[a, b, c, d - depth_radius : d + depth_radius + 1] ** 2)\r\n output = input / (bias + alpha * sqr_sum ** beta)\r\n\r\nFor details, see [Krizhevsky et al., ImageNet classification with deep\r\nconvolutional neural networks (NIPS 2012)]\r\n(http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks).\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor` of type `float32`. 4-D.\r\n* `depth_radius`: An optional `int`. Defaults to `5`.\r\n 0-D. Half-width of the 1-D normalization window.\r\n* `bias`: An optional `float`. Defaults to `1`.\r\n An offset (usually positive to avoid dividing by 0).\r\n* `alpha`: An optional `float`. Defaults to `1`.\r\n A scale factor, usually positive.\r\n* `beta`: An optional `float`. Defaults to `0.5`. An exponent.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `float32`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.moments(x, axes, name=None)` \r\n\r\nCalculate the mean and variance of `x`.\r\n\r\nThe mean and variance are calculated by aggregating the contents of `x`\r\nacross `axes`. If `x` is 1-D and `axes = [0]` this is just the mean\r\nand variance of a vector.\r\n\r\nFor so-called \"global normalization\" needed for convolutional filters pass\r\n`axes=[0, 1, 2]` (batch, height, width). For batch normalization pass\r\n`axes=[0]` (batch).\r\n\r\n##### Args: \r\n\r\n\r\n* `x`: A `Tensor`.\r\n* `axes`: array of ints. Axes along which to compute mean and\r\n variance.\r\n* `name`: Name used to scope the operations that compute the moments.\r\n\r\n##### Returns: \r\n\r\n Two `Tensors`: `mean` and `variance`.\r\n\r\n\r\n\r\n## Losses \r\n\r\nThe loss ops measure error between two tensors, or between a tensor and zero.\r\nThese can be used for measuring accuracy of a network in a regression task\r\nor for regularization purposes (weight decay).\r\n\r\n- - -\r\n\r\n### `tf.nn.l2_loss(t, name=None)` \r\n\r\nL2 Loss.\r\n\r\nComputes half the L2 norm of a tensor without the `sqrt`:\r\n\r\n output = sum(t ** 2) / 2\r\n\r\n##### Args: \r\n\r\n\r\n* `t`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `qint8`, `quint8`, `qint32`.\r\n Typically 2-D, but may have any dimensions.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `t`. 0-D.\r\n\r\n\r\n\r\n## Classification \r\n\r\nTensorFlow provides several operations that help you perform classification.\r\n\r\n- - -\r\n\r\n### `tf.nn.sigmoid_cross_entropy_with_logits(logits, targets, name=None)` \r\n\r\nComputes sigmoid cross entropy given `logits`.\r\n\r\nMeasures the probability error in discrete classification tasks in which each\r\nclass is independent and not mutually exclusive. For instance, one could\r\nperform multilabel classification where a picture can contain both an elephant\r\nand a dog at the same time.\r\n\r\nFor brevity, let `x = logits`, `z = targets`. The logistic loss is\r\n\r\n x - x * z + log(1 + exp(-x))\r\n\r\nTo ensure stability and avoid overflow, the implementation uses\r\n\r\n max(x, 0) - x * z + log(1 + exp(-abs(x)))\r\n\r\n`logits` and `targets` must have the same type and shape.\r\n\r\n##### Args: \r\n\r\n\r\n* `logits`: A `Tensor` of type `float32` or `float64`.\r\n* `targets`: A `Tensor` of the same type and shape as `logits`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of the same shape as `logits` with the componentwise\r\n logistic losses.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.softmax(logits, name=None)` \r\n\r\nComputes softmax activations.\r\n\r\nFor each batch `i` and class `j` we have\r\n\r\n softmax[i, j] = exp(logits[i, j]) / sum(exp(logits[i]))\r\n\r\n##### Args: \r\n\r\n\r\n* `logits`: A `Tensor`. Must be one of the following types: `float32`, `float64`.\r\n 2-D with shape `[batch_size, num_classes]`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `logits`. Same shape as `logits`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.softmax_cross_entropy_with_logits(logits, labels, name=None)` \r\n\r\nComputes softmax cross entropy between `logits` and `labels`.\r\n\r\nMeasures the probability error in discrete classification tasks in which the\r\nclasses are mutually exclusive (each entry is in exactly one class). For\r\nexample, each CIFAR-10 image is labeled with one and only one label: an image\r\ncan be a dog or a truck, but not both.\r\n\r\n**WARNING:** This op expects unscaled logits, since it performs a `softmax`\r\non `logits` internally for efficiency. Do not call this op with the\r\noutput of `softmax`, as it will produce incorrect results.\r\n\r\n`logits` and `labels` must have the same shape `[batch_size, num_classes]`\r\nand the same dtype (either `float32` or `float64`).\r\n\r\n##### Args: \r\n\r\n\r\n* `logits`: Unscaled log probabilities.\r\n* `labels`: Each row `labels[i]` must be a valid probability distribution.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A 1-D `Tensor` of length `batch_size` of the same type as `logits` with the\r\n softmax cross entropy loss.\r\n\r\n\r\n\r\n## Embeddings \r\n\r\nTensorFlow provides library support for looking up values in embedding\r\ntensors.\r\n\r\n- - -\r\n\r\n### `tf.nn.embedding_lookup(params, ids, name=None)` \r\n\r\nLooks up `ids` in a list of embedding tensors.\r\n\r\nThis function is used to perform parallel lookups on the list of\r\ntensors in `params`. It is a generalization of\r\n[`tf.gather()`](../../api_docs/python/array_ops.md#gather), where `params` is\r\ninterpreted as a partition of a larger embedding tensor.\r\n\r\nIf `len(params) > 1`, each element `id` of `ids` is partitioned between\r\nthe elements of `params` by computing `p = id % len(params)`, and is\r\nthen used to look up the slice `params[p][id // len(params), ...]`.\r\n\r\nThe results of the lookup are then concatenated into a dense\r\ntensor. The returned tensor has shape `shape(ids) + shape(params)[1:]`.\r\n\r\n##### Args: \r\n\r\n\r\n* `params`: A list of tensors with the same shape and type.\r\n* `ids`: A `Tensor` with type `int32` containing the ids to be looked\r\n up in `params`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` with the same type as the tensors in `params`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If `params` is empty.\r\n\r\n\r\n\r\n## Evaluation \r\n\r\nThe evaluation ops are useful for measuring the performance of a network.\r\nSince they are nondifferentiable, they are typically used at evaluation time.\r\n\r\n- - -\r\n\r\n### `tf.nn.top_k(input, k, name=None)` \r\n\r\nReturns the values and indices of the k largest elements for each row.\r\n\r\n\\\\(values_{i, j}\\\\) represents the j-th largest element in \\\\(input_i\\\\).\r\n\r\n\\\\(indices_{i, j}\\\\) gives the column index of the corresponding element,\r\nsuch that \\\\(input_{i, indices_{i, j}} = values_{i, j}\\\\). If two\r\nelements are equal, the lower-index element appears first.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `int64`, `uint8`, `int16`, `int8`.\r\n A batch_size x classes tensor\r\n* `k`: An `int` that is `>= 1`.\r\n Number of top elements to look for within each row\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A tuple of `Tensor` objects (values, indices).\r\n\r\n* `values`: A `Tensor`. Has the same type as `input`. A batch_size x k tensor with the k largest elements for each row,\r\n sorted in descending order\r\n* `indices`: A `Tensor` of type `int32`. A batch_size x k tensor with the index of each value within each row\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.in_top_k(predictions, targets, k, name=None)` \r\n\r\nSays whether the targets are in the top K predictions.\r\n\r\nThis outputs a batch_size bool array, an entry out[i] is true if the\r\nprediction for the target class is among the top k predictions among\r\nall predictions for example i. Note that the behavior of InTopK differs\r\nfrom the TopK op in its handling of ties; if multiple classes have the\r\nsame prediction value and straddle the top-k boundary, all of those\r\nclasses are considered to be in the top k.\r\n\r\nMore formally, let\r\n\r\n \\\\(predictions_i\\\\) be the predictions for all classes for example i,\r\n \\\\(targets_i\\\\) be the target class for example i,\r\n \\\\(out_i\\\\) be the output for example i,\r\n\r\n$$out_i = predictions_{i, targets_i} \\in TopKIncludingTies(predictions_i)$$\r\n\r\n##### Args: \r\n\r\n\r\n* `predictions`: A `Tensor` of type `float32`. A batch_size x classes tensor\r\n* `targets`: A `Tensor` of type `int32`. A batch_size vector of class ids\r\n* `k`: An `int`. Number of top elements to look at for computing precision\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor` of type `bool`. Computed Precision at k as a bool Tensor\r\n\r\n\r\n\r\n## Candidate Sampling \r\n\r\nDo you want to train a multiclass or multilabel model with thousands\r\nor millions of output classes (for example, a language model with a\r\nlarge vocabulary)? Training with a full Softmax is slow in this case,\r\nsince all of the classes are evaluated for every training example.\r\nCandidate Sampling training algorithms can speed up your step times by\r\nonly considering a small randomly-chosen subset of contrastive classes\r\n(called candidates) for each batch of training examples.\r\n\r\nSee our [Candidate Sampling Algorithms Reference]\r\n(../../extras/candidate_sampling.pdf)\r\n\r\n### Sampled Loss Functions \r\n\r\nTensorFlow provides the following sampled loss functions for faster training.\r\n\r\n- - -\r\n\r\n### `tf.nn.nce_loss(weights, biases, inputs, labels, num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=False, name='nce_loss')` \r\n\r\nComputes and returns the noise-contrastive estimation training loss.\r\n\r\nSee [Noise-contrastive estimation: A new estimation principle for\r\nunnormalized statistical models]\r\n(http://www.jmlr.org/proceedings/papers/v9/gutmann10a/gutmann10a.pdf).\r\nAlso see our [Candidate Sampling Algorithms Reference]\r\n(http://www.tensorflow.org/extras/candidate_sampling.pdf)\r\n\r\nNote: In the case where num_true > 1, we assign to each target class\r\nthe target probability 1 / num_true so that the target probabilities\r\nsum to 1 per-example.\r\n\r\nNote: It would be useful to allow a variable number of target classes per\r\nexample. We hope to provide this functionality in a future release.\r\nFor now, if you have a variable number of target classes, you can pad them\r\nout to a constant number by either repeating them or by padding\r\nwith an otherwise unused class.\r\n\r\n##### Args: \r\n\r\n\r\n* `weights`: A `Tensor` of shape [num_classes, dim]. The class embeddings.\r\n* `biases`: A `Tensor` of shape [num_classes]. The class biases.\r\n* `inputs`: A `Tensor` of shape [batch_size, dim]. The forward\r\n activations of the input network.\r\n* `labels`: A `Tensor` of type `int64` and shape `[batch_size,\r\n num_true]`. The target classes.\r\n* `num_sampled`: An `int`. The number of classes to randomly sample per batch.\r\n* `num_classes`: An `int`. The number of possible classes.\r\n* `num_true`: An `int`. The number of target classes per training example.\r\n* `sampled_values`: a tuple of `(sampled_candidates, true_expected_count,\r\n sampled_expected_count)` returned by a *_candidate_sampler function.\r\n (if None, we default to LogUniformCandidateSampler)\r\n* `remove_accidental_hits`: A `bool`. Whether to remove \"accidental hits\"\r\n where a sampled class equals one of the target classes. If set to\r\n `True`, this is a \"Sampled Logistic\" loss instead of NCE, and we are\r\n learning to generate log-odds instead of log probabilities. See\r\n our [Candidate Sampling Algorithms Reference]\r\n (http://www.tensorflow.org/extras/candidate_sampling.pdf).\r\n Default is False.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A batch_size 1-D tensor of per-example NCE losses.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.sampled_softmax_loss(weights, biases, inputs, labels, num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=True, name='sampled_softmax_loss')` \r\n\r\nComputes and returns the sampled softmax training loss.\r\n\r\nThis is a faster way to train a softmax classifier over a huge number of\r\nclasses.\r\n\r\nThis operation is for training only. It is generally an underestimate of\r\nthe full softmax loss.\r\n\r\nAt inference time, you can compute full softmax probabilities with the\r\nexpression `tf.nn.softmax(tf.matmul(inputs, weights) + biases)`.\r\n\r\nSee our [Candidate Sampling Algorithms Reference]\r\n(http://www.tensorflow.org/extras/candidate_sampling.pdf)\r\n\r\nAlso see Section 3 of http://arxiv.org/abs/1412.2007 for the math.\r\n\r\n##### Args: \r\n\r\n\r\n* `weights`: A `Tensor` of shape [num_classes, dim]. The class embeddings.\r\n* `biases`: A `Tensor` of shape [num_classes]. The class biases.\r\n* `inputs`: A `Tensor` of shape [batch_size, dim]. The forward\r\n activations of the input network.\r\n* `labels`: A `Tensor` of type `int64` and shape `[batch_size,\r\n num_true]`. The target classes. Note that this format differs from\r\n the `labels` argument of `nn.softmax_cross_entropy_with_logits`.\r\n* `num_sampled`: An `int`. The number of classes to randomly sample per batch.\r\n* `num_classes`: An `int`. The number of possible classes.\r\n* `num_true`: An `int`. The number of target classes per training example.\r\n* `sampled_values`: a tuple of `(sampled_candidates, true_expected_count,\r\n sampled_expected_count)` returned by a *_candidate_sampler function.\r\n (if None, we default to LogUniformCandidateSampler)\r\n* `remove_accidental_hits`: A `bool`. whether to remove \"accidental hits\"\r\n where a sampled class equals one of the target classes. Default is\r\n True.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A batch_size 1-D tensor of per-example sampled softmax losses.\r\n\r\n\r\n\r\n### Candidate Samplers \r\n\r\nTensorFlow provides the following samplers for randomly sampling candidate\r\nclasses when using one of the sampled loss functions above.\r\n\r\n- - -\r\n\r\n### `tf.nn.uniform_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, seed=None, name=None)` \r\n\r\nSamples a set of classes using a uniform base distribution.\r\n\r\nThis operation randomly samples a tensor of sampled classes\r\n(`sampled_candidates`) from the range of integers `[0, range_max]`.\r\n\r\nThe elements of `sampled_candidates` are drawn without replacement\r\n(if `unique=True`) or with replacement (if `unique=False`) from\r\nthe base distribution.\r\n\r\nThe base distribution for this operation is the uniform distribution\r\nover the range of integers `[0, range_max]`.\r\n\r\nIn addition, this operation returns tensors `true_expected_count`\r\nand `sampled_expected_count` representing the number of times each\r\nof the target classes (`true_classes`) and the sampled\r\nclasses (`sampled_candidates`) is expected to occur in an average\r\ntensor of sampled classes. These values correspond to `Q(y|x)`\r\ndefined in [this\r\ndocument](http://www.tensorflow.org/extras/candidate_sampling.pdf).\r\nIf `unique=True`, then these are post-rejection probabilities and we\r\ncompute them approximately.\r\n\r\n##### Args: \r\n\r\n\r\n* `true_classes`: A `Tensor` of type `int64` and shape `[batch_size,\r\n num_true]`. The target classes.\r\n* `num_true`: An `int`. The number of target classes per training example.\r\n* `num_sampled`: An `int`. The number of classes to randomly sample per batch.\r\n* `unique`: A `bool`. Determines whether all sampled classes in a batch are\r\n unique.\r\n* `range_max`: An `int`. The number of possible classes.\r\n* `seed`: An `int`. An operation-specific seed. Default is 0.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n\r\n* `sampled_candidates`: A tensor of type `int64` and shape `[num_sampled]`.\r\n The sampled classes.\r\n* `true_expected_count`: A tensor of type `float`. Same shape as\r\n `true_classes`. The expected counts under the sampling distribution\r\n of each of `true_classes`.\r\n* `sampled_expected_count`: A tensor of type `float`. Same shape as\r\n `sampled_candidates`. The expected counts under the sampling distribution\r\n of each of `sampled_candidates`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.log_uniform_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, seed=None, name=None)` \r\n\r\nSamples a set of classes using a log-uniform (Zipfian) base distribution.\r\n\r\nThis operation randomly samples a tensor of sampled classes\r\n(`sampled_candidates`) from the range of integers `[0, range_max]`.\r\n\r\nThe elements of `sampled_candidates` are drawn without replacement\r\n(if `unique=True`) or with replacement (if `unique=False`) from\r\nthe base distribution.\r\n\r\nThe base distribution for this operation is an approximately log-uniform\r\nor Zipfian distribution:\r\n\r\n`P(class) = (log(class + 2) - log(class + 1)) / log(range_max + 1)`\r\n\r\nThis sampler is useful when the target classes approximately follow such\r\na distribution - for example, if the classes represent words in a lexicon\r\nsorted in decreasing order of frequency. If your classes are not ordered by\r\ndecreasing frequency, do not use this op.\r\n\r\nIn addition, this operation returns tensors `true_expected_count`\r\nand `sampled_expected_count` representing the number of times each\r\nof the target classes (`true_classes`) and the sampled\r\nclasses (`sampled_candidates`) is expected to occur in an average\r\ntensor of sampled classes. These values correspond to `Q(y|x)`\r\ndefined in [this\r\ndocument](http://www.tensorflow.org/extras/candidate_sampling.pdf).\r\nIf `unique=True`, then these are post-rejection probabilities and we\r\ncompute them approximately.\r\n\r\n##### Args: \r\n\r\n\r\n* `true_classes`: A `Tensor` of type `int64` and shape `[batch_size,\r\n num_true]`. The target classes.\r\n* `num_true`: An `int`. The number of target classes per training example.\r\n* `num_sampled`: An `int`. The number of classes to randomly sample per batch.\r\n* `unique`: A `bool`. Determines whether all sampled classes in a batch are\r\n unique.\r\n* `range_max`: An `int`. The number of possible classes.\r\n* `seed`: An `int`. An operation-specific seed. Default is 0.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n\r\n* `sampled_candidates`: A tensor of type `int64` and shape `[num_sampled]`.\r\n The sampled classes.\r\n* `true_expected_count`: A tensor of type `float`. Same shape as\r\n `true_classes`. The expected counts under the sampling distribution\r\n of each of `true_classes`.\r\n* `sampled_expected_count`: A tensor of type `float`. Same shape as\r\n `sampled_candidates`. The expected counts under the sampling distribution\r\n of each of `sampled_candidates`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.learned_unigram_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, seed=None, name=None)` \r\n\r\nSamples a set of classes from a distribution learned during training.\r\n\r\nThis operation randomly samples a tensor of sampled classes\r\n(`sampled_candidates`) from the range of integers `[0, range_max]`.\r\n\r\nThe elements of `sampled_candidates` are drawn without replacement\r\n(if `unique=True`) or with replacement (if `unique=False`) from\r\nthe base distribution.\r\n\r\nThe base distribution for this operation is constructed on the fly\r\nduring training. It is a unigram distribution over the target\r\nclasses seen so far during training. Every integer in `[0, range_max]`\r\nbegins with a weight of 1, and is incremented by 1 each time it is\r\nseen as a target class. The base distribution is not saved to checkpoints,\r\nso it is reset when the model is reloaded.\r\n\r\nIn addition, this operation returns tensors `true_expected_count`\r\nand `sampled_expected_count` representing the number of times each\r\nof the target classes (`true_classes`) and the sampled\r\nclasses (`sampled_candidates`) is expected to occur in an average\r\ntensor of sampled classes. These values correspond to `Q(y|x)`\r\ndefined in [this\r\ndocument](http://www.tensorflow.org/extras/candidate_sampling.pdf).\r\nIf `unique=True`, then these are post-rejection probabilities and we\r\ncompute them approximately.\r\n\r\n##### Args: \r\n\r\n\r\n* `true_classes`: A `Tensor` of type `int64` and shape `[batch_size,\r\n num_true]`. The target classes.\r\n* `num_true`: An `int`. The number of target classes per training example.\r\n* `num_sampled`: An `int`. The number of classes to randomly sample per batch.\r\n* `unique`: A `bool`. Determines whether all sampled classes in a batch are\r\n unique.\r\n* `range_max`: An `int`. The number of possible classes.\r\n* `seed`: An `int`. An operation-specific seed. Default is 0.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n\r\n* `sampled_candidates`: A tensor of type `int64` and shape `[num_sampled]`.\r\n The sampled classes.\r\n* `true_expected_count`: A tensor of type `float`. Same shape as\r\n `true_classes`. The expected counts under the sampling distribution\r\n of each of `true_classes`.\r\n* `sampled_expected_count`: A tensor of type `float`. Same shape as\r\n `sampled_candidates`. The expected counts under the sampling distribution\r\n of each of `sampled_candidates`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.fixed_unigram_candidate_sampler(true_classes, num_true, num_sampled, unique, range_max, vocab_file='', distortion=0.0, num_reserved_ids=0, num_shards=1, shard=0, unigrams=[], seed=None, name=None)` \r\n\r\nSamples a set of classes using the provided (fixed) base distribution.\r\n\r\nThis operation randomly samples a tensor of sampled classes\r\n(`sampled_candidates`) from the range of integers `[0, range_max]`.\r\n\r\nThe elements of `sampled_candidates` are drawn without replacement\r\n(if `unique=True`) or with replacement (if `unique=False`) from\r\nthe base distribution.\r\n\r\nThe base distribution is read from a file or passed in as an\r\nin-memory array. There is also an option to skew the distribution by\r\napplying a distortion power to the weights.\r\n\r\nIn addition, this operation returns tensors `true_expected_count`\r\nand `sampled_expected_count` representing the number of times each\r\nof the target classes (`true_classes`) and the sampled\r\nclasses (`sampled_candidates`) is expected to occur in an average\r\ntensor of sampled classes. These values correspond to `Q(y|x)`\r\ndefined in [this\r\ndocument](http://www.tensorflow.org/extras/candidate_sampling.pdf).\r\nIf `unique=True`, then these are post-rejection probabilities and we\r\ncompute them approximately.\r\n\r\n##### Args: \r\n\r\n\r\n* `true_classes`: A `Tensor` of type `int64` and shape `[batch_size,\r\n num_true]`. The target classes.\r\n* `num_true`: An `int`. The number of target classes per training example.\r\n* `num_sampled`: An `int`. The number of classes to randomly sample per batch.\r\n* `unique`: A `bool`. Determines whether all sampled classes in a batch are\r\n unique.\r\n* `range_max`: An `int`. The number of possible classes.\r\n* `vocab_file`: Each valid line in this file (which should have a CSV-like\r\n format) corresponds to a valid word ID. IDs are in sequential order,\r\n starting from num_reserved_ids. The last entry in each line is expected\r\n to be a value corresponding to the count or relative probability. Exactly\r\n one of `vocab_file` and `unigrams` needs to be passed to this operation.\r\n* `distortion`: The distortion is used to skew the unigram probability\r\n distribution. Each weight is first raised to the distortion's power\r\n before adding to the internal unigram distribution. As a result,\r\n `distortion = 1.0` gives regular unigram sampling (as defined by the vocab\r\n file), and `distortion = 0.0` gives a uniform distribution.\r\n* `num_reserved_ids`: Optionally some reserved IDs can be added in the range\r\n `[0, num_reserved_ids]` by the users. One use case is that a special\r\n unknown word token is used as ID 0. These IDs will have a sampling\r\n probability of 0.\r\n* `num_shards`: A sampler can be used to sample from a subset of the original\r\n range in order to speed up the whole computation through parallelism. This\r\n parameter (together with `shard`) indicates the number of partitions that\r\n are being used in the overall computation.\r\n* `shard`: A sampler can be used to sample from a subset of the original range\r\n in order to speed up the whole computation through parallelism. This\r\n parameter (together with `num_shards`) indicates the particular partition\r\n number of the operation, when partitioning is being used.\r\n* `unigrams`: A list of unigram counts or probabilities, one per ID in\r\n sequential order. Exactly one of `vocab_file` and `unigrams` should be\r\n passed to this operation.\r\n* `seed`: An `int`. An operation-specific seed. Default is 0.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n\r\n* `sampled_candidates`: A tensor of type `int64` and shape `[num_sampled]`.\r\n The sampled classes.\r\n* `true_expected_count`: A tensor of type `float`. Same shape as\r\n `true_classes`. The expected counts under the sampling distribution\r\n of each of `true_classes`.\r\n* `sampled_expected_count`: A tensor of type `float`. Same shape as\r\n `sampled_candidates`. The expected counts under the sampling distribution\r\n of each of `sampled_candidates`.\r\n\r\n\r\n\r\n### Miscellaneous candidate sampling utilities \r\n\r\n- - -\r\n\r\n### `tf.nn.compute_accidental_hits(true_classes, sampled_candidates, num_true, seed=None, name=None)` \r\n\r\nCompute the ids of positions in sampled_candidates matching true_classes.\r\n\r\nIn Candidate Sampling, this operation facilitates virtually removing\r\nsampled classes which happen to match target classes. This is done\r\nin Sampled Softmax and Sampled Logistic.\r\n\r\nSee our [Candidate Sampling Algorithms\r\nReference](http://www.tensorflow.org/extras/candidate_sampling.pdf).\r\n\r\nWe presuppose that the `sampled_candidates` are unique.\r\n\r\nWe call it an 'accidental hit' when one of the target classes\r\nmatches one of the sampled classes. This operation reports\r\naccidental hits as triples `(index, id, weight)`, where `index`\r\nrepresents the row number in `true_classes`, `id` represents the\r\nposition in `sampled_candidates`, and weight is `-FLOAT_MAX`.\r\n\r\nThe result of this op should be passed through a `sparse_to_dense`\r\noperation, then added to the logits of the sampled classes. This\r\nremoves the contradictory effect of accidentally sampling the true\r\ntarget classes as noise classes for the same example.\r\n\r\n##### Args: \r\n\r\n\r\n* `true_classes`: A `Tensor` of type `int64` and shape `[batch_size,\r\n num_true]`. The target classes.\r\n* `sampled_candidates`: A tensor of type `int64` and shape `[num_sampled]`.\r\n The sampled_candidates output of CandidateSampler.\r\n* `num_true`: An `int`. The number of target classes per training example.\r\n* `seed`: An `int`. An operation-specific seed. Default is 0.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n\r\n* `indices`: A `Tensor` of type `int32` and shape `[num_accidental_hits]`.\r\n Values indicate rows in `true_classes`.\r\n* `ids`: A `Tensor` of type `int64` and shape `[num_accidental_hits]`.\r\n Values indicate positions in `sampled_candidates`.\r\n* `weights`: A `Tensor` of type `float` and shape `[num_accidental_hits]`.\r\n Each value is `-FLOAT_MAX`.\r\n\r\n\r\n"
},
{
"path": "SOURCE/api_docs/python/python_io.md",
"content": "\r\n\r\n# Data IO (Python functions) \r\n\r\n## Contents\r\n### [Data IO (Python functions)](#AUTOGENERATED-data-io--python-functions-)\r\n* [Data IO (Python Functions)](#AUTOGENERATED-data-io--python-functions-)\r\n * [`class tf.python_io.TFRecordWriter`](#TFRecordWriter)\r\n * [`tf.python_io.tf_record_iterator(path)`](#tf_record_iterator)\r\n * [TFRecords Format Details](#AUTOGENERATED-tfrecords-format-details)\r\n\r\n\r\n\r\n\r\n## Data IO (Python Functions) \r\n\r\nA TFRecords file represents a sequence of (binary) strings. The format is not\r\nrandom access, so it is suitable for streaming large amounts of data but not\r\nsuitable if fast sharding or other non-sequential access is desired.\r\n\r\n- - -\r\n\r\n### `class tf.python_io.TFRecordWriter` \r\n\r\nA class to write records to a TFRecords file.\r\n\r\nThis class implements `__enter__` and `__exit__`, and can be used\r\nin `with` blocks like a normal file.\r\n\r\n- - -\r\n\r\n#### `tf.python_io.TFRecordWriter.__init__(path)` \r\n\r\nOpens file `path` and creates a `TFRecordWriter` writing to it.\r\n\r\n##### Args: \r\n\r\n\r\n* `path`: The path to the TFRecords file.\r\n\r\n##### Raises: \r\n\r\n\r\n* `IOError`: If `path` cannot be opened for writing.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.python_io.TFRecordWriter.write(record)` \r\n\r\nWrite a string record to the file.\r\n\r\n##### Args: \r\n\r\n\r\n* `record`: str\r\n\r\n\r\n- - -\r\n\r\n#### `tf.python_io.TFRecordWriter.close()` \r\n\r\nClose the file.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.python_io.tf_record_iterator(path)` \r\n\r\nAn iterator that read the records from a TFRecords file.\r\n\r\n##### Args: \r\n\r\n\r\n* `path`: The path to the TFRecords file.\r\n\r\n##### Yields: \r\n\r\n Strings.\r\n\r\n##### Raises: \r\n\r\n\r\n* `IOError`: If `path` cannot be opened for reading.\r\n\r\n\r\n\r\n- - -\r\n\r\n### TFRecords Format Details \r\n\r\nA TFRecords file contains a sequence of strings with CRC hashes. Each record\r\nhas the format\r\n\r\n uint64 length\r\n uint32 masked_crc32_of_length\r\n byte data[length]\r\n uint32 masked_crc32_of_data\r\n\r\nand the records are concatenated together to produce the file. The CRC32s\r\nare [described here](https://en.wikipedia.org/wiki/Cyclic_redundancy_check),\r\nand the mask of a CRC is\r\n\r\n masked_crc = ((crc >> 15) | (crc << 17)) + 0xa282ead8ul\r\n"
},
{
"path": "SOURCE/api_docs/python/sparse_ops.md",
"content": "\r\n\r\n# Sparse Tensors \r\n\r\nNote: Functions taking `Tensor` arguments can also take anything accepted by\r\n[`tf.convert_to_tensor`](../../api_docs/python/framework.md#convert_to_tensor).\r\n\r\n\r\n## Contents\r\n### [Sparse Tensors](#AUTOGENERATED-sparse-tensors)\r\n* [Sparse Tensor Representation](#AUTOGENERATED-sparse-tensor-representation)\r\n * [`class tf.SparseTensor`](#SparseTensor)\r\n * [`class tf.SparseTensorValue`](#SparseTensorValue)\r\n* [Sparse to Dense Conversion](#AUTOGENERATED-sparse-to-dense-conversion)\r\n * [`tf.sparse_to_dense(sparse_indices, output_shape, sparse_values, default_value, name=None)`](#sparse_to_dense)\r\n * [`tf.sparse_tensor_to_dense(sp_input, default_value, name=None)`](#sparse_tensor_to_dense)\r\n * [`tf.sparse_to_indicator(sp_input, vocab_size, name=None)`](#sparse_to_indicator)\r\n* [Manipulation](#AUTOGENERATED-manipulation)\r\n * [`tf.sparse_concat(concat_dim, sp_inputs, name=None)`](#sparse_concat)\r\n * [`tf.sparse_reorder(sp_input, name=None)`](#sparse_reorder)\r\n * [`tf.sparse_retain(sp_input, to_retain)`](#sparse_retain)\r\n * [`tf.sparse_fill_empty_rows(sp_input, default_value, name=None)`](#sparse_fill_empty_rows)\r\n\r\n\r\n\r\n\r\n## Sparse Tensor Representation \r\n\r\nTensorflow supports a `SparseTensor` representation for data that is sparse\r\nin multiple dimensions. Contrast this representation with `IndexedSlices`,\r\nwhich is efficient for representing tensors that are sparse in their first\r\ndimension, and dense along all other dimensions.\r\n\r\n- - -\r\n\r\n### `class tf.SparseTensor` \r\n\r\nRepresents a sparse tensor.\r\n\r\nTensorflow represents a sparse tensor as three separate dense tensors:\r\n`indices`, `values`, and `dense_shape`. In Python, the three tensors are\r\ncollected into a `SparseTensor` class for ease of use. If you have separate\r\n`indices`, `values`, and `dense_shape` tensors, wrap them in a `SparseTensor`\r\nobject before passing to the Ops below.\r\n\r\nConcretely, the sparse tensor `SparseTensor(values, indices, dense_shape)` is\r\n\r\n* `indices`: A 2-D int64 tensor of shape `[N, ndims]`.\r\n* `values`: A 1-D tensor of any type and shape `[N]`.\r\n* `dense_shape`: A 1-D int64 tensor of shape `[ndims]`.\r\n\r\nwhere `N` and `ndims` are the number of values, and number of dimensions in\r\nthe `SparseTensor` respectively.\r\n\r\nThe corresponding dense tensor satisfies\r\n\r\n```python\r\ndense.shape = dense_shape\r\ndense[tuple(indices[i])] = values[i]\r\n```\r\n\r\nBy convention, `indices` should be sorted in row-major order (or equivalently\r\nlexigraphic order on the tuples `indices[i]`). This is not enforced when\r\n`SparseTensor` objects are constructed, but most Ops assume correct ordering.\r\nIf the ordering is wrong, it can be fixed by calling `sparse_reorder` on the\r\nmisordered `SparseTensor`.\r\n\r\nExample: The sparse tensor\r\n\r\n```python\r\n SparseTensor(values=[1, 2], indices=[[0, 0], [1, 2]], shape=[3, 4])\r\n```\r\n\r\nrepresents the dense tensor\r\n\r\n```python\r\n [[1, 0, 0, 0]\r\n [0, 0, 2, 0]\r\n [0, 0, 0, 0]]\r\n```\r\n\r\n- - -\r\n\r\n#### `tf.SparseTensor.__init__(indices, values, shape)` \r\n\r\nCreates a `SparseTensor`.\r\n\r\n##### Args: \r\n\r\n\r\n* `indices`: A 2-D int64 tensor of shape `[N, ndims]`.\r\n* `values`: A 1-D tensor of any type and shape `[N]`.\r\n* `dense_shape`: A 1-D int64 tensor of shape `[ndims]`.\r\n\r\n##### Returns: \r\n\r\n A `SparseTensor`\r\n\r\n\r\n- - -\r\n\r\n#### `tf.SparseTensor.indices` \r\n\r\nThe indices of non-zero values in the represented dense tensor.\r\n\r\n##### Returns: \r\n\r\n A 2-D Tensor of int64 with shape `[N, ndims]`, where `N` is the\r\n number of non-zero values in the tensor, and `ndims` is the rank.\r\n\r\n- - -\r\n\r\n#### `tf.SparseTensor.values` \r\n\r\nThe non-zero values in the represented dense tensor.\r\n\r\n##### Returns: \r\n\r\n A 1-D Tensor of any data type.\r\n\r\n- - -\r\n\r\n#### `tf.SparseTensor.dtype` \r\n\r\nThe `DType` of elements in this tensor.\r\n\r\n- - -\r\n\r\n#### `tf.SparseTensor.shape` \r\n\r\nA 1-D Tensor of int64 representing the shape of the dense tensor.\r\n\r\n- - -\r\n\r\n#### `tf.SparseTensor.graph` \r\n\r\nThe `Graph` that contains the index, value, and shape tensors.\r\n\r\n\r\n- - -\r\n\r\n### `class tf.SparseTensorValue` \r\n\r\nSparseTensorValue(indices, values, shape)\r\n- - -\r\n\r\n#### `tf.SparseTensorValue.indices` \r\n\r\nAlias for field number 0\r\n\r\n- - -\r\n\r\n#### `tf.SparseTensorValue.shape` \r\n\r\nAlias for field number 2\r\n\r\n- - -\r\n\r\n#### `tf.SparseTensorValue.values` \r\n\r\nAlias for field number 1\r\n\r\n\r\n\r\n## Sparse to Dense Conversion \r\n\r\n- - -\r\n\r\n### `tf.sparse_to_dense(sparse_indices, output_shape, sparse_values, default_value, name=None)` \r\n\r\nConverts a sparse representation into a dense tensor.\r\n\r\nBuilds an array `dense` with shape `output_shape` such that\r\n\r\n```prettyprint\r\n# If sparse_indices is scalar\r\ndense[i] = (i == sparse_indices ? sparse_values : default_value)\r\n\r\n# If sparse_indices is a vector, then for each i\r\ndense[sparse_indices[i]] = sparse_values[i]\r\n\r\n# If sparse_indices is an n by d matrix, then for each i in [0, n)\r\ndense[sparse_indices[i][0], ..., sparse_indices[i][d-1]] = sparse_values[i]\r\n```\r\n\r\nAll other values in `dense` are set to `default_value`. If `sparse_values` is a\r\nscalar, all sparse indices are set to this single value.\r\n\r\n##### Args: \r\n\r\n\r\n* `sparse_indices`: A `Tensor`. Must be one of the following types: `int32`, `int64`.\r\n 0-D, 1-D, or 2-D. `sparse_indices[i]` contains the complete\r\n index where `sparse_values[i]` will be placed.\r\n* `output_shape`: A `Tensor`. Must have the same type as `sparse_indices`.\r\n 1-D. Shape of the dense output tensor.\r\n* `sparse_values`: A `Tensor`.\r\n 1-D. Values corresponding to each row of `sparse_indices`,\r\n or a scalar value to be used for all sparse indices.\r\n* `default_value`: A `Tensor`. Must have the same type as `sparse_values`.\r\n Scalar value to set for indices not specified in\r\n `sparse_indices`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `sparse_values`.\r\n Dense output tensor of shape `output_shape`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.sparse_tensor_to_dense(sp_input, default_value, name=None)` \r\n\r\nConverts a `SparseTensor` into a dense tensor.\r\n\r\nThis op is a convenience wrapper around `sparse_to_dense` for `SparseTensor`s.\r\n\r\nFor example, if `sp_input` has shape `[3, 5]` and non-empty string values:\r\n\r\n [0, 1]: a\r\n [0, 3]: b\r\n [2, 0]: c\r\n\r\nand `default_value` is `x`, then the output will be a dense `[3, 5]`\r\nstring tensor with values:\r\n\r\n [[x a x b x]\r\n [x x x x x]\r\n [c x x x x]]\r\n\r\n##### Args: \r\n\r\n\r\n* `sp_input`: The input `SparseTensor`.\r\n* `default_value`: Scalar value to set for indices not specified in\r\n `sp_input`.\r\n* `name`: A name prefix for the returned tensors (optional).\r\n\r\n##### Returns: \r\n\r\n A dense tensor with shape `sp_input.shape` and values specified by\r\n the non-empty values in `sp_input`. Indices not in `sp_input` are assigned\r\n `default_value`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `sp_input` is not a `SparseTensor`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.sparse_to_indicator(sp_input, vocab_size, name=None)` \r\n\r\nConverts a `SparseTensor` of ids into a dense bool indicator tensor.\r\n\r\nThe last dimension of `sp_input` is discarded and replaced with the values of\r\n`sp_input`. If `sp_input.shape = [D0, D1, ..., Dn, K]`, then\r\n`output.shape = [D0, D1, ..., Dn, vocab_size]`, where\r\n\r\n output[d_0, d_1, ..., d_n, sp_input[d_0, d_1, ..., d_n, k]] = True\r\n\r\nand False elsewhere in `output`.\r\n\r\nFor example, if `sp_input.shape = [2, 3, 4]` with non-empty values:\r\n\r\n [0, 0, 0]: 0\r\n [0, 1, 0]: 10\r\n [1, 0, 3]: 103\r\n [1, 1, 2]: 112\r\n [1, 1, 3]: 113\r\n [1, 2, 1]: 121\r\n\r\nand `vocab_size = 200`, then the output will be a `[2, 3, 200]` dense bool\r\ntensor with False everywhere except at positions\r\n\r\n (0, 0, 0), (0, 1, 10), (1, 0, 103), (1, 1, 112), (1, 1, 113), (1, 2, 121).\r\n\r\nThis op is useful for converting `SparseTensor`s into dense formats for\r\ncompatibility with ops that expect dense tensors.\r\n\r\nThe input `SparseTensor` must be in row-major order.\r\n\r\n##### Args: \r\n\r\n\r\n* `sp_input`: A `SparseTensor` of type `int32` or `int64`.\r\n* `vocab_size`: The new size of the last dimension, with\r\n `all(0 <= sp_input.values < vocab_size)`.\r\n* `name`: A name prefix for the returned tensors (optional)\r\n\r\n##### Returns: \r\n\r\n A dense bool indicator tensor representing the indices with specified value.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `sp_input` is not a `SparseTensor`.\r\n\r\n\r\n\r\n## Manipulation \r\n\r\n- - -\r\n\r\n### `tf.sparse_concat(concat_dim, sp_inputs, name=None)` \r\n\r\nConcatenates a list of `SparseTensor` along the specified dimension.\r\n\r\nConcatenation is with respect to the dense versions of each sparse input.\r\nIt is assumed that each inputs is a `SparseTensor` whose elements are ordered\r\nalong increasing dimension number.\r\n\r\nAll inputs' shapes must match, except for the concat dimension. The\r\n`indices`, `values`, and `shapes` lists must have the same length.\r\n\r\nThe output shape is identical to the inputs', except along the concat\r\ndimension, where it is the sum of the inputs' sizes along that dimension.\r\n\r\nThe output elements will be resorted to preserve the sort order along\r\nincreasing dimension number.\r\n\r\nThis op runs in `O(M log M)` time, where `M` is the total number of non-empty\r\nvalues across all inputs. This is due to the need for an internal sort in\r\norder to concatenate efficiently across an arbitrary dimension.\r\n\r\nFor example, if `concat_dim = 1` and the inputs are\r\n\r\n sp_inputs[0]: shape = [2, 3]\r\n [0, 2]: \"a\"\r\n [1, 0]: \"b\"\r\n [1, 1]: \"c\"\r\n\r\n sp_inputs[1]: shape = [2, 4]\r\n [0, 1]: \"d\"\r\n [0, 2]: \"e\"\r\n\r\nthen the output will be\r\n\r\n shape = [2, 7]\r\n [0, 2]: \"a\"\r\n [0, 4]: \"d\"\r\n [0, 5]: \"e\"\r\n [1, 0]: \"b\"\r\n [1, 1]: \"c\"\r\n\r\nGraphically this is equivalent to doing\r\n\r\n [ a] concat [ d e ] = [ a d e ]\r\n [b c ] [ ] [b c ]\r\n\r\n##### Args: \r\n\r\n\r\n* `concat_dim`: Dimension to concatenate along.\r\n* `sp_inputs`: List of `SparseTensor` to concatenate.\r\n* `name`: A name prefix for the returned tensors (optional).\r\n\r\n##### Returns: \r\n\r\n A `SparseTensor` with the concatenated output.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `sp_inputs` is not a list of `SparseTensor`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.sparse_reorder(sp_input, name=None)` \r\n\r\nReorders a `SparseTensor` into the canonical, row-major ordering.\r\n\r\nNote that by convention, all sparse ops preserve the canonical ordering\r\nalong increasing dimension number. The only time ordering can be violated\r\nis during manual manipulation of the indices and values to add entries.\r\n\r\nReordering does not affect the shape of the `SparseTensor`.\r\n\r\nFor example, if sp_input has shape `[4, 5]` and `indices` / `values`:\r\n\r\n [0, 3]: b\r\n [0, 1]: a\r\n [3, 1]: d\r\n [2, 0]: c\r\n\r\nthen the output will be a `SparseTensor` of shape `[4, 5]` and\r\n`indices` / `values`:\r\n\r\n [0, 1]: a\r\n [0, 3]: b\r\n [2, 0]: c\r\n [3, 1]: d\r\n\r\n##### Args: \r\n\r\n\r\n* `sp_input`: The input `SparseTensor`.\r\n* `name`: A name prefix for the returned tensors (optional)\r\n\r\n##### Returns: \r\n\r\n A `SparseTensor` with the same shape and non-empty values, but in\r\n canonical ordering.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `sp_input` is not a `SparseTensor`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.sparse_retain(sp_input, to_retain)` \r\n\r\nRetains specified non-empty values within a `SparseTensor`.\r\n\r\nFor example, if `sp_input` has shape `[4, 5]` and 4 non-empty string values:\r\n\r\n [0, 1]: a\r\n [0, 3]: b\r\n [2, 0]: c\r\n [3, 1]: d\r\n\r\nand `to_retain = [True, False, False, True]`, then the output will\r\nbe a `SparseTensor` of shape `[4, 5]` with 2 non-empty values:\r\n\r\n [0, 1]: a\r\n [3, 1]: d\r\n\r\n##### Args: \r\n\r\n\r\n* `sp_input`: The input `SparseTensor` with `N` non-empty elements.\r\n* `to_retain`: A bool vector of length `N` with `M` true values.\r\n\r\n##### Returns: \r\n\r\n A `SparseTensor` with the same shape as the input and `M` non-empty\r\n elements corresponding to the true positions in `to_retain`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `sp_input` is not a `SparseTensor`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.sparse_fill_empty_rows(sp_input, default_value, name=None)` \r\n\r\nFills empty rows in the input 2-D `SparseTensor` with a default value.\r\n\r\nThis op adds entries with the specified `default_value` at index\r\n`[row, 0]` for any row in the input that does not already have a value.\r\n\r\nFor example, suppose `sp_input` has shape `[5, 6]` and non-empty values:\r\n\r\n [0, 1]: a\r\n [0, 3]: b\r\n [2, 0]: c\r\n [3, 1]: d\r\n\r\nRows 1 and 4 are empty, so the output will be of shape `[5, 6]` with values:\r\n\r\n [0, 1]: a\r\n [0, 3]: b\r\n [1, 0]: default_value\r\n [2, 0]: c\r\n [3, 1]: d\r\n [4, 0]: default_value\r\n\r\nNote that the input may have empty columns at the end, with no effect on\r\nthis op.\r\n\r\nThe output `SparseTensor` will be in row-major order and will have the\r\nsame shape as the input.\r\n\r\nThis op also returns an indicator vector such that\r\n\r\n empty_row_indicator[i] = True iff row i was an empty row.\r\n\r\n##### Args: \r\n\r\n\r\n* `sp_input`: A `SparseTensor` with shape `[N, M]`.\r\n* `default_value`: The value to fill for empty rows, with the same type as\r\n `sp_input.`\r\n* `name`: A name prefix for the returned tensors (optional)\r\n\r\n##### Returns: \r\n\r\n\r\n* `sp_ordered_output`: A `SparseTensor` with shape `[N, M]`, and with all empty\r\n rows filled in with `default_value`.\r\n* `empty_row_indicator`: A bool vector of length `N` indicating whether each\r\n input row was empty.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `sp_input` is not a `SparseTensor`.\r\n\r\n\r\n"
},
{
"path": "SOURCE/api_docs/python/state_ops.md",
"content": "\r\n\r\n# Variables \r\n\r\nNote: Functions taking `Tensor` arguments can also take anything accepted by\r\n[`tf.convert_to_tensor`](../../api_docs/python/framework.md#convert_to_tensor).\r\n\r\n\r\n## Contents\r\n### [Variables](#AUTOGENERATED-variables)\r\n* [Variables](#AUTOGENERATED-variables)\r\n * [`class tf.Variable`](#Variable)\r\n* [Variable helper functions](#AUTOGENERATED-variable-helper-functions)\r\n * [`tf.all_variables()`](#all_variables)\r\n * [`tf.trainable_variables()`](#trainable_variables)\r\n * [`tf.initialize_all_variables()`](#initialize_all_variables)\r\n * [`tf.initialize_variables(var_list, name='init')`](#initialize_variables)\r\n * [`tf.assert_variables_initialized(var_list=None)`](#assert_variables_initialized)\r\n* [Saving and Restoring Variables](#AUTOGENERATED-saving-and-restoring-variables)\r\n * [`class tf.train.Saver`](#Saver)\r\n * [`tf.train.latest_checkpoint(checkpoint_dir, latest_filename=None)`](#latest_checkpoint)\r\n * [`tf.train.get_checkpoint_state(checkpoint_dir, latest_filename=None)`](#get_checkpoint_state)\r\n * [`tf.train.update_checkpoint_state(save_dir, model_checkpoint_path, all_model_checkpoint_paths=None, latest_filename=None)`](#update_checkpoint_state)\r\n* [Sharing Variables](#AUTOGENERATED-sharing-variables)\r\n * [`tf.get_variable(name, shape=None, dtype=tf.float32, initializer=None, trainable=True, collections=None)`](#get_variable)\r\n * [`tf.get_variable_scope()`](#get_variable_scope)\r\n * [`tf.variable_scope(name_or_scope, reuse=None, initializer=None)`](#variable_scope)\r\n * [`tf.constant_initializer(value=0.0)`](#constant_initializer)\r\n * [`tf.random_normal_initializer(mean=0.0, stddev=1.0, seed=None)`](#random_normal_initializer)\r\n * [`tf.truncated_normal_initializer(mean=0.0, stddev=1.0, seed=None)`](#truncated_normal_initializer)\r\n * [`tf.random_uniform_initializer(minval=0.0, maxval=1.0, seed=None)`](#random_uniform_initializer)\r\n * [`tf.uniform_unit_scaling_initializer(factor=1.0, seed=None)`](#uniform_unit_scaling_initializer)\r\n * [`tf.zeros_initializer(shape, dtype=tf.float32)`](#zeros_initializer)\r\n* [Sparse Variable Updates](#AUTOGENERATED-sparse-variable-updates)\r\n * [`tf.scatter_update(ref, indices, updates, use_locking=None, name=None)`](#scatter_update)\r\n * [`tf.scatter_add(ref, indices, updates, use_locking=None, name=None)`](#scatter_add)\r\n * [`tf.scatter_sub(ref, indices, updates, use_locking=None, name=None)`](#scatter_sub)\r\n * [`tf.sparse_mask(a, mask_indices, name=None)`](#sparse_mask)\r\n * [`class tf.IndexedSlices`](#IndexedSlices)\r\n\r\n\r\n\r\n\r\n## Variables \r\n\r\n- - -\r\n\r\n### `class tf.Variable` \r\n\r\nSee the [Variables How To](../../how_tos/variables/index.md) for a high\r\nlevel overview.\r\n\r\nA variable maintains state in the graph across calls to `run()`. You add a\r\nvariable to the graph by constructing an instance of the class `Variable`.\r\n\r\nThe `Variable()` constructor requires an initial value for the variable,\r\nwhich can be a `Tensor` of any type and shape. The initial value defines the\r\ntype and shape of the variable. After construction, the type and shape of\r\nthe variable are fixed. The value can be changed using one of the assign\r\nmethods.\r\n\r\nIf you want to change the shape of a variable later you have to use an\r\n`assign` Op with `validate_shape=False`.\r\n\r\nJust like any `Tensor`, variables created with `Variable()` can be used as\r\ninputs for other Ops in the graph. Additionally, all the operators\r\noverloaded for the `Tensor` class are carried over to variables, so you can\r\nalso add nodes to the graph by just doing arithmetic on variables.\r\n\r\n```python\r\nimport tensorflow as tf\r\n\r\n# Create a variable.\r\nw = tf.Variable(\r\n\r\n![](\"../images/ScatterUpdate.png\")
\r\n
\r\n\r\n##### Args: \r\n\r\n\r\n* `ref`: A mutable `Tensor`. Should be from a `Variable` node.\r\n* `indices`: A `Tensor`. Must be one of the following types: `int32`, `int64`.\r\n A tensor of indices into the first dimension of `ref`.\r\n* `updates`: A `Tensor`. Must have the same type as `ref`.\r\n A tensor of updated values to store in `ref`.\r\n* `use_locking`: An optional `bool`. Defaults to `True`.\r\n If True, the assignment will be protected by a lock;\r\n otherwise the behavior is undefined, but may exhibit less contention.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n Same as `ref`. Returned as a convenience for operations that want\r\n to use the updated values after the update is done.\r\n\r\n\r\n- - -\r\n\r\n### `tf.scatter_add(ref, indices, updates, use_locking=None, name=None)` \r\n\r\nAdds sparse updates to a variable reference.\r\n\r\nThis operation computes\r\n\r\n # Scalar indices\r\n ref[indices, ...] += updates[...]\r\n\r\n # Vector indices (for each i)\r\n ref[indices[i], ...] += updates[i, ...]\r\n\r\n # High rank indices (for each i, ..., j)\r\n ref[indices[i, ..., j], ...] += updates[i, ..., j, ...]\r\n\r\nThis operation outputs `ref` after the update is done.\r\nThis makes it easier to chain operations that need to use the reset value.\r\n\r\nDuplicate entries are handled correctly: if multiple `indices` reference\r\nthe same location, their contributions add.\r\n\r\nRequires `updates.shape = indices.shape + ref.shape[1:]`.\r\n\r\n\r\n\r\n![](\"../images/ScatterAdd.png\")
\r\n
\r\n\r\n##### Args: \r\n\r\n\r\n* `ref`: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `qint8`, `quint8`, `qint32`.\r\n Should be from a `Variable` node.\r\n* `indices`: A `Tensor`. Must be one of the following types: `int32`, `int64`.\r\n A tensor of indices into the first dimension of `ref`.\r\n* `updates`: A `Tensor`. Must have the same type as `ref`.\r\n A tensor of updated values to add to `ref`.\r\n* `use_locking`: An optional `bool`. Defaults to `False`.\r\n If True, the addition will be protected by a lock;\r\n otherwise the behavior is undefined, but may exhibit less contention.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n Same as `ref`. Returned as a convenience for operations that want\r\n to use the updated values after the update is done.\r\n\r\n\r\n- - -\r\n\r\n### `tf.scatter_sub(ref, indices, updates, use_locking=None, name=None)` \r\n\r\nSubtracts sparse updates to a variable reference.\r\n\r\n # Scalar indices\r\n ref[indices, ...] -= updates[...]\r\n\r\n # Vector indices (for each i)\r\n ref[indices[i], ...] -= updates[i, ...]\r\n\r\n # High rank indices (for each i, ..., j)\r\n ref[indices[i, ..., j], ...] -= updates[i, ..., j, ...]\r\n\r\nThis operation outputs `ref` after the update is done.\r\nThis makes it easier to chain operations that need to use the reset value.\r\n\r\nDuplicate entries are handled correctly: if multiple `indices` reference\r\nthe same location, their (negated) contributions add.\r\n\r\nRequires `updates.shape = indices.shape + ref.shape[1:]`.\r\n\r\n\r\n\r\n![](\"../images/ScatterSub.png\")
\r\n
\r\n\r\n##### Args: \r\n\r\n\r\n* `ref`: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `qint8`, `quint8`, `qint32`.\r\n Should be from a `Variable` node.\r\n* `indices`: A `Tensor`. Must be one of the following types: `int32`, `int64`.\r\n A tensor of indices into the first dimension of `ref`.\r\n* `updates`: A `Tensor`. Must have the same type as `ref`.\r\n A tensor of updated values to subtract from `ref`.\r\n* `use_locking`: An optional `bool`. Defaults to `False`.\r\n If True, the subtraction will be protected by a lock;\r\n otherwise the behavior is undefined, but may exhibit less contention.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n Same as `ref`. Returned as a convenience for operations that want\r\n to use the updated values after the update is done.\r\n\r\n\r\n- - -\r\n\r\n### `tf.sparse_mask(a, mask_indices, name=None)` \r\n\r\nMasks elements of `IndexedSlices`.\r\n\r\nGiven an `IndexedSlices` instance `a`, returns another `IndexedSlices` that\r\ncontains a subset of the slices of `a`. Only the slices at indices specified\r\nin `mask_indices` are returned.\r\n\r\nThis is useful when you need to extract a subset of slices in an\r\n`IndexedSlices` object.\r\n\r\nFor example:\r\n\r\n```python\r\n# `a` contains slices at indices [12, 26, 37, 45] from a large tensor\r\n# with shape [1000, 10]\r\na.indices => [12, 26, 37, 45]\r\ntf.shape(a.values) => [4, 10]\r\n\r\n# `b` will be the subset of `a` slices at its second and third indices, so\r\n# we want to mask of its first and last indices (which are at absolute\r\n# indices 12, 45)\r\nb = tf.sparse_mask(a, [12, 45])\r\n\r\nb.indices => [26, 37]\r\ntf.shape(b.values) => [2, 10]\r\n\r\n```\r\n\r\n##### Args: \r\n\r\n * `a`: An `IndexedSlices` instance.\r\n * `mask_indices`: Indices of elements to mask.\r\n * `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The masked `IndexedSlices` instance.\r\n\r\n\r\n- - -\r\n\r\n### `class tf.IndexedSlices` \r\n\r\nA sparse representation of a set of tensor slices at given indices.\r\n\r\nThis class is a simple wrapper for a pair of `Tensor` objects:\r\n\r\n* `values`: A `Tensor` of any dtype with shape `[D0, D1, ..., Dn]`.\r\n* `indices`: A 1-D integer `Tensor` with shape `[D0]`.\r\n\r\nAn `IndexedSlices` is typically used to represent a subset of a larger\r\ntensor `dense` of shape `[LARGE0, D1, .. , DN]` where `LARGE0 >> D0`.\r\nThe values in `indices` are the indices in the first dimension of\r\nthe slices that have been extracted from the larger tensor.\r\n\r\nThe dense tensor `dense` represented by an `IndexedSlices` `slices` has\r\n\r\n```python\r\ndense[slices.indices[i], :, :, :, ...] = slices.values[i, :, :, :, ...]\r\n```\r\n\r\nThe `IndexedSlices` class is used principally in the definition of\r\ngradients for operations that have sparse gradients\r\n(e.g. [`tf.gather`](../../api_docs/python/array_ops.md#gather)).\r\n\r\nContrast this representation with\r\n[`SparseTensor`](../../api_docs/python/sparse_ops.md#SparseTensor),\r\nwhich uses multi-dimensional indices and scalar values.\r\n\r\n- - -\r\n\r\n#### `tf.IndexedSlices.__init__(values, indices, dense_shape=None)` \r\n\r\nCreates an `IndexedSlices`.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.IndexedSlices.values` \r\n\r\nA `Tensor` containing the values of the slices.\r\n\r\n- - -\r\n\r\n#### `tf.IndexedSlices.indices` \r\n\r\nA 1-D `Tensor` containing the indices of the slices.\r\n\r\n- - -\r\n\r\n#### `tf.IndexedSlices.dense_shape` \r\n\r\nA 1-D `Tensor` containing the shape of the corresponding dense tensor.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.IndexedSlices.name` \r\n\r\nThe name of this `IndexedSlices`.\r\n\r\n- - -\r\n\r\n#### `tf.IndexedSlices.dtype` \r\n\r\nThe `DType` of elements in this tensor.\r\n\r\n- - -\r\n\r\n#### `tf.IndexedSlices.device` \r\n\r\nThe name of the device on which `values` will be produced, or `None`.\r\n\r\n- - -\r\n\r\n#### `tf.IndexedSlices.op` \r\n\r\nThe `Operation` that produces `values` as an output.\r\n\r\n\r\n"
},
{
"path": "SOURCE/api_docs/python/train.md",
"content": "\r\n\r\n# Training \r\n\r\n## Contents\r\n### [Training](#AUTOGENERATED-training)\r\n* [Optimizers](#AUTOGENERATED-optimizers)\r\n * [`class tf.train.Optimizer`](#Optimizer)\r\n * [Usage](#AUTOGENERATED-usage)\r\n * [Processing gradients before applying them.](#AUTOGENERATED-processing-gradients-before-applying-them.)\r\n * [Gating Gradients](#AUTOGENERATED-gating-gradients)\r\n * [Slots](#AUTOGENERATED-slots)\r\n * [`class tf.train.GradientDescentOptimizer`](#GradientDescentOptimizer)\r\n * [`class tf.train.AdagradOptimizer`](#AdagradOptimizer)\r\n * [`class tf.train.MomentumOptimizer`](#MomentumOptimizer)\r\n * [`class tf.train.AdamOptimizer`](#AdamOptimizer)\r\n * [`class tf.train.FtrlOptimizer`](#FtrlOptimizer)\r\n * [`class tf.train.RMSPropOptimizer`](#RMSPropOptimizer)\r\n* [Gradient Computation](#AUTOGENERATED-gradient-computation)\r\n * [`tf.gradients(ys, xs, grad_ys=None, name='gradients', colocate_gradients_with_ops=False, gate_gradients=False, aggregation_method=None)`](#gradients)\r\n * [`class tf.AggregationMethod`](#AggregationMethod)\r\n * [`tf.stop_gradient(input, name=None)`](#stop_gradient)\r\n* [Gradient Clipping](#AUTOGENERATED-gradient-clipping)\r\n * [`tf.clip_by_value(t, clip_value_min, clip_value_max, name=None)`](#clip_by_value)\r\n * [`tf.clip_by_norm(t, clip_norm, name=None)`](#clip_by_norm)\r\n * [`tf.clip_by_average_norm(t, clip_norm, name=None)`](#clip_by_average_norm)\r\n * [`tf.clip_by_global_norm(t_list, clip_norm, use_norm=None, name=None)`](#clip_by_global_norm)\r\n * [`tf.global_norm(t_list, name=None)`](#global_norm)\r\n* [Decaying the learning rate](#AUTOGENERATED-decaying-the-learning-rate)\r\n * [`tf.train.exponential_decay(learning_rate, global_step, decay_steps, decay_rate, staircase=False, name=None)`](#exponential_decay)\r\n* [Moving Averages](#AUTOGENERATED-moving-averages)\r\n * [`class tf.train.ExponentialMovingAverage`](#ExponentialMovingAverage)\r\n* [Coordinator and QueueRunner](#AUTOGENERATED-coordinator-and-queuerunner)\r\n * [`class tf.train.Coordinator`](#Coordinator)\r\n * [`class tf.train.QueueRunner`](#QueueRunner)\r\n * [`tf.train.add_queue_runner(qr, collection='queue_runners')`](#add_queue_runner)\r\n * [`tf.train.start_queue_runners(sess=None, coord=None, daemon=True, start=True, collection='queue_runners')`](#start_queue_runners)\r\n* [Summary Operations](#AUTOGENERATED-summary-operations)\r\n * [`tf.scalar_summary(tags, values, collections=None, name=None)`](#scalar_summary)\r\n * [`tf.image_summary(tag, tensor, max_images=None, collections=None, name=None)`](#image_summary)\r\n * [`tf.histogram_summary(tag, values, collections=None, name=None)`](#histogram_summary)\r\n * [`tf.nn.zero_fraction(value, name=None)`](#zero_fraction)\r\n * [`tf.merge_summary(inputs, collections=None, name=None)`](#merge_summary)\r\n * [`tf.merge_all_summaries(key='summaries')`](#merge_all_summaries)\r\n* [Adding Summaries to Event Files](#AUTOGENERATED-adding-summaries-to-event-files)\r\n * [`class tf.train.SummaryWriter`](#SummaryWriter)\r\n * [`tf.train.summary_iterator(path)`](#summary_iterator)\r\n* [Training utilities](#AUTOGENERATED-training-utilities)\r\n * [`tf.train.global_step(sess, global_step_tensor)`](#global_step)\r\n * [`tf.train.write_graph(graph_def, logdir, name, as_text=True)`](#write_graph)\r\n\r\n\r\n\r\n\r\nThis library provides a set of classes and functions that helps train models.\r\n\r\n## Optimizers \r\n\r\nThe Optimizer base class provides methods to compute gradients for a loss and\r\napply gradients to variables. A collection of subclasses implement classic\r\noptimization algorithms such as GradientDescent and Adagrad.\r\n\r\nYou never instantiate the Optimizer class itself, but instead instantiate one\r\nof the subclasses.\r\n\r\n- - -\r\n\r\n### `class tf.train.Optimizer` \r\n\r\nBase class for optimizers.\r\n\r\nThis class defines the API to add Ops to train a model. You never use this\r\nclass directly, but instead instantiate one of its subclasses such as\r\n`GradientDescentOptimizer`, `AdagradOptimizer`, or `MomentumOptimizer`.\r\n\r\n### Usage \r\n\r\n```\r\n# Create an optimizer with the desired parameters.\r\nopt = GradientDescentOptimizer(learning_rate=0.1)\r\n# Add Ops to the graph to minimize a cost by updating a list of variables.\r\n# \"cost\" is a Tensor, and the list of variables contains variables.Variable\r\n# objects.\r\nopt_op = opt.minimize(cost, \r\n- )\r\n```\r\n\r\nIn the training program you will just have to run the returned Op.\r\n\r\n```\r\n# Execute opt_op to do one step of training:\r\nopt_op.run()\r\n```\r\n\r\n### Processing gradients before applying them. \r\n\r\nCalling `minimize()` takes care of both computing the gradients and\r\napplying them to the variables. If you want to process the gradients\r\nbefore applying them you can instead use the optimizer in three steps:\r\n\r\n1. Compute the gradients with `compute_gradients()`.\r\n2. Process the gradients as you wish.\r\n3. Apply the processed gradients with `apply_gradients()`.\r\n\r\nExample:\r\n\r\n```\r\n# Create an optimizer.\r\nopt = GradientDescentOptimizer(learning_rate=0.1)\r\n\r\n# Compute the gradients for a list of variables.\r\ngrads_and_vars = opt.compute_gradients(loss,
- )\r\n\r\n# grads_and_vars is a list of tuples (gradient, variable). Do whatever you\r\n# need to the 'gradient' part, for example cap them, etc.\r\ncapped_grads_and_vars = [(MyCapper(gv[0]), gv[1])) for gv in grads_and_vars]\r\n\r\n# Ask the optimizer to apply the capped gradients.\r\nopt.apply_gradients(capped_grads_and_vars)\r\n```\r\n\r\n- - -\r\n\r\n#### `tf.train.Optimizer.__init__(use_locking, name)` \r\n\r\nCreate a new Optimizer.\r\n\r\nThis must be called by the constructors of subclasses.\r\n\r\n##### Args: \r\n\r\n\r\n* `use_locking`: Bool. If True apply use locks to prevent concurrent updates\r\n to variables.\r\n* `name`: A non-empty string. The name to use for accumulators created\r\n for the optimizer.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if name is malformed.\r\n\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.Optimizer.minimize(loss, global_step=None, var_list=None, gate_gradients=1, name=None)` \r\n\r\nAdd operations to minimize 'loss' by updating 'var_list'.\r\n\r\nThis method simply combines calls compute_gradients() and\r\napply_gradients(). If you want to process the gradient before applying them\r\ncall compute_gradients() and apply_gradients() explicitly instead of using\r\nthis function.\r\n\r\n##### Args: \r\n\r\n\r\n* `loss`: A Tensor containing the value to minimize.\r\n* `global_step`: Optional Variable to increment by one after the\r\n variables have been updated.\r\n* `var_list`: Optional list of variables.Variable to update to minimize\r\n 'loss'. Defaults to the list of variables collected in the graph\r\n under the key GraphKeys.TRAINABLE_VARIABLES.\r\n* `gate_gradients`: How to gate the computation of gradients. Can be\r\n GATE_NONE, GATE_OP, or GATE_GRAPH.\r\n* `name`: Optional name for the returned operation.\r\n\r\n##### Returns: \r\n\r\n An Operation that updates the variables in 'var_list'. If 'global_step'\r\n was not None, that operation also increments global_step.\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if some of the variables are not variables.Variable objects.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.Optimizer.compute_gradients(loss, var_list=None, gate_gradients=1)` \r\n\r\nCompute gradients of \"loss\" for the variables in \"var_list\".\r\n\r\nThis is the first part of minimize(). It returns a list\r\nof (gradient, variable) pairs where \"gradient\" is the gradient\r\nfor \"variable\". Note that \"gradient\" can be a Tensor, a\r\nIndexedSlices, or None if there is no gradient for the\r\ngiven variable.\r\n\r\n##### Args: \r\n\r\n\r\n* `loss`: A Tensor containing the value to minimize.\r\n* `var_list`: Optional list of variables.Variable to update to minimize\r\n \"loss\". Defaults to the list of variables collected in the graph\r\n under the key GraphKey.TRAINABLE_VARIABLES.\r\n* `gate_gradients`: How to gate the computation of gradients. Can be\r\n GATE_NONE, GATE_OP, or GATE_GRAPH.\r\n\r\n##### Returns: \r\n\r\n A list of (gradient, variable) pairs.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If var_list contains anything else than variables.Variable.\r\n* `ValueError`: If some arguments are invalid.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.Optimizer.apply_gradients(grads_and_vars, global_step=None, name=None)` \r\n\r\nApply gradients to variables.\r\n\r\nThis is the second part of minimize(). It returns an Operation that\r\napplies gradients.\r\n\r\n##### Args: \r\n\r\n\r\n* `grads_and_vars`: List of (gradient, variable) pairs as returned by\r\n compute_gradients().\r\n* `global_step`: Optional Variable to increment by one after the\r\n variables have been updated.\r\n* `name`: Optional name for the returned operation. Default to the\r\n name passed to the Optimizer constructor.\r\n\r\n##### Returns: \r\n\r\n An Operation that applies the specified gradients. If 'global_step'\r\n was not None, that operation also increments global_step.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: if grads_and_vars is malformed.\r\n\r\n\r\n\r\n### Gating Gradients \r\n\r\nBoth `minimize()` and `compute_gradients()` accept a `gate_gradient` argument\r\nthat controls the degree of parallelism during the application of the\r\ngradients.\r\n\r\nThe possible values are: `GATE_NONE`, `GATE_OP`, and `GATE_GRAPH`.\r\n\r\nGATE_NONE: Compute and apply gradients in parallel. This provides the\r\nmaximum parallelism in execution, at the cost of some non-reproducibility in\r\nthe results. For example the two gradients of MatMul depend on the input\r\nvalues: With `GATE_NONE` one of the gradients could be applied to one of the\r\ninputs _before_ the other gradient is computed resulting in non-reproducible\r\nresults.\r\n\r\nGATE_OP: For each Op, make sure all gradients are computed before they\r\nare used. This prevents race conditions for Ops that generate gradients for\r\nmultiple inputs where the gradients depend on the inputs.\r\n\r\nGATE_GRAPH: Make sure all gradients for all variables are computed\r\nbefore any one of them is used. This provides the least parallelism but can\r\nbe useful if you want to process all gradients before applying any of them.\r\n\r\n### Slots \r\n\r\nSome optimizer subclasses, such as `MomentumOptimizer` and `AdagradOptimizer`\r\nallocate and manage additional variables associated with the variables to\r\ntrain. These are called Slots. Slots have names and you can ask the\r\noptimizer for the names of the slots that it uses. Once you have a slot name\r\nyou can ask the optimizer for the variable it created to hold the slot value.\r\n\r\nThis can be useful if you want to log debug a training algorithm, report stats\r\nabout the slots, etc.\r\n\r\n- - -\r\n\r\n#### `tf.train.Optimizer.get_slot_names()` \r\n\r\nReturn a list of the names of slots created by the Optimizer.\r\n\r\nSee get_slot().\r\n\r\n##### Returns: \r\n\r\n A list of strings.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.Optimizer.get_slot(var, name)` \r\n\r\nReturn a slot named \"name\" created for \"var\" by the Optimizer.\r\n\r\nSome Optimizer subclasses use additional variables. For example\r\nMomentum and Adagrad use variables to accumulate updates. This method\r\ngives access to these Variables if for some reason you need them.\r\n\r\nUse get_slot_names() to get the list of slot names created by the Optimizer.\r\n\r\n##### Args: \r\n\r\n\r\n* `var`: A variable passed to minimize() or apply_gradients().\r\n* `name`: A string.\r\n\r\n##### Returns: \r\n\r\n The Variable for the slot if it was created, None otherwise.\r\n\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.train.GradientDescentOptimizer` \r\n\r\nOptimizer that implements the gradient descent algorithm.\r\n\r\n- - -\r\n\r\n#### `tf.train.GradientDescentOptimizer.__init__(learning_rate, use_locking=False, name='GradientDescent')` \r\n\r\nConstruct a new gradient descent optimizer.\r\n\r\n##### Args: \r\n\r\n\r\n* `learning_rate`: A Tensor or a floating point value. The learning\r\n rate to use.\r\n* `use_locking`: If True use locks for update operation.s\r\n* `name`: Optional name prefix for the operations created when applying\r\n gradients. Defaults to \"GradientDescent\".\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.train.AdagradOptimizer` \r\n\r\nOptimizer that implements the Adagrad algorithm.\r\n\r\n- - -\r\n\r\n#### `tf.train.AdagradOptimizer.__init__(learning_rate, initial_accumulator_value=0.1, use_locking=False, name='Adagrad')` \r\n\r\nConstruct a new Adagrad optimizer.\r\n\r\n##### Args: \r\n\r\n\r\n* `learning_rate`: A `Tensor` or a floating point value. The learning rate.\r\n* `initial_accumulator_value`: A floating point value.\r\n Starting value for the accumulators, must be positive.\r\n* `use_locking`: If `True` use locks for update operations.\r\n* `name`: Optional name prefix for the operations created when applying\r\n gradients. Defaults to \"Adagrad\".\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: If the initial_accumulator_value is invalid.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.train.MomentumOptimizer` \r\n\r\nOptimizer that implements the Momentum algorithm.\r\n\r\n- - -\r\n\r\n#### `tf.train.MomentumOptimizer.__init__(learning_rate, momentum, use_locking=False, name='Momentum')` \r\n\r\nConstruct a new Momentum optimizer.\r\n\r\n##### Args: \r\n\r\n\r\n* `learning_rate`: A `Tensor` or a floating point value. The learning rate.\r\n* `momentum`: A `Tensor` or a floating point value. The momentum.\r\n* `use_locking`: If `True` use locks for update operations.\r\n* `name`: Optional name prefix for the operations created when applying\r\n gradients. Defaults to \"Momentum\".\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.train.AdamOptimizer` \r\n\r\nOptimizer that implements the Adam algorithm.\r\n\r\n- - -\r\n\r\n#### `tf.train.AdamOptimizer.__init__(learning_rate=0.001, beta1=0.9, beta2=0.999, epsilon=1e-08, use_locking=False, name='Adam')` \r\n\r\nConstruct a new Adam optimizer.\r\n\r\nImplementation is based on: http://arxiv.org/pdf/1412.6980v7.pdf\r\n\r\nInitialization:\r\n\r\n```\r\nm_0 <- 0 (Initialize initial 1st moment vector)\r\nv_0 <- 0 (Initialize initial 2nd moment vector)\r\nt <- 0 (Initialize timestep)\r\n```\r\n\r\nThe update rule for `variable` with gradient `g` uses an optimization\r\ndescribed at the end of section2 of the paper:\r\n\r\n```\r\nt <- t + 1\r\nlr_t <- learning_rate * sqrt(1 - beta2^t) / (1 - beta1^t)\r\n\r\nm_t <- beta1 * m_{t-1} + (1 - beta1) * g\r\nv_t <- beta2 * v_{t-1} + (1 - beta2) * g * g\r\nvariable <- variable - lr_t * m_t / (sqrt(v_t) + epsilon)\r\n```\r\n\r\nThe default value of 1e-8 for epsilon might not be a good default in\r\ngeneral. For example, when training an Inception network on ImageNet a\r\ncurrent good choice is 1.0 or 0.1.\r\n\r\n##### Args: \r\n\r\n\r\n* `learning_rate`: A Tensor or a floating point value. The learning rate.\r\n* `beta1`: A float value or a constant float tensor.\r\n The exponential decay rate for the 1st moment estimates.\r\n* `beta2`: A float value or a constant float tensor.\r\n The exponential decay rate for the 2st moment estimates.\r\n* `epsilon`: A small constant for numerical stability.\r\n* `use_locking`: If True use locks for update operation.s\r\n* `name`: Optional name for the operations created when applying gradients.\r\n Defaults to \"Adam\".\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.train.FtrlOptimizer` \r\n\r\nOptimizer that implements the FTRL algorithm.\r\n\r\n- - -\r\n\r\n#### `tf.train.FtrlOptimizer.__init__(learning_rate, learning_rate_power=-0.5, initial_accumulator_value=0.1, l1_regularization_strength=0.0, l2_regularization_strength=0.0, use_locking=False, name='Ftrl')` \r\n\r\nConstruct a new FTRL optimizer.\r\n\r\nThe Ftrl-proximal algorithm, abbreviated for Follow-the-regularized-leader,\r\nis described in the paper [Ad Click Prediction: a View from the Trenches](\r\nhttps://www.eecs.tufts.edu/~dsculley/papers/ad-click-prediction.pdf).\r\n\r\nIt can give a good performance vs. sparsity tradeoff.\r\n\r\nFtrl-proximal uses its own global base learning rate and can behave like\r\nAdagrad with `learning_rate_power=-0.5`, or like gradient descent with\r\n`learning_rate_power=0.0`.\r\n\r\nThe effective learning rate is adjusted per parameter, relative to this\r\nbase learning rate as:\r\n\r\n```\r\neffective_learning_rate_i = (learning_rate /\r\n pow(k + summed_squared_gradients_for_i, learning_rate_power));\r\n```\r\n\r\nwhere k is the small constant `initial_accumulator_value`.\r\n\r\nNote that the real regularization coefficient of `|w|^2` for objective\r\nfunction is `1 / lambda_2` if specifying `l2 = lambda_2` as argument when\r\nusing this function.\r\n\r\n##### Args: \r\n\r\n\r\n* `learning_rate`: A float value or a constant float `Tensor`.\r\n* `learning_rate_power`: A float value, must be less or equal to zero.\r\n* `initial_accumulator_value`: The starting value for accumulators.\r\n Only positive values are allowed.\r\n* `l1_regularization_strength`: A float value, must be greater than or\r\n equal to zero.\r\n* `l2_regularization_strength`: A float value, must be greater than or\r\n equal to zero.\r\n* `use_locking`: If `True` use locks for update operations.\r\n* `name`: Optional name prefix for the operations created when applying\r\n gradients. Defaults to \"Ftrl\".\r\n\r\n##### Raises: \r\n\r\n\r\n* `ValueError`: if one of the arguments is invalid.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.train.RMSPropOptimizer` \r\n\r\nOptimizer that implements the RMSProp algorithm.\r\n\r\n- - -\r\n\r\n#### `tf.train.RMSPropOptimizer.__init__(learning_rate, decay, momentum=0.0, epsilon=1e-10, use_locking=False, name='RMSProp')` \r\n\r\nConstruct a new RMSProp optimizer.\r\n\r\n##### Args: \r\n\r\n\r\n* `learning_rate`: A Tensor or a floating point value. The learning rate.\r\n* `decay`: discounting factor for the history/coming gradient\r\n* `momentum`: a scalar tensor.\r\n* `epsilon`: small value to avoid zero denominator.\r\n* `use_locking`: If True use locks for update operation.\r\n* `name`: Optional name prefic for the operations created when applying\r\n gradients. Defaults to \"RMSProp\".\r\n\r\n\r\n\r\n\r\n## Gradient Computation \r\n\r\nTensorFlow provides functions to compute the derivatives for a given\r\nTensorFlow computation graph, adding operations to the graph. The\r\noptimizer classes automatically compute derivatives on your graph, but\r\ncreators of new Optimizers or expert users can call the lower-level\r\nfunctions below.\r\n\r\n- - -\r\n\r\n### `tf.gradients(ys, xs, grad_ys=None, name='gradients', colocate_gradients_with_ops=False, gate_gradients=False, aggregation_method=None)` \r\n\r\nConstructs symbolic partial derivatives of `ys` w.r.t. x in `xs`.\r\n\r\n`ys` and `xs` are each a `Tensor` or a list of tensors. `grad_ys`\r\nis a list of `Tensor`, holding the gradients received by the\r\n`ys`. The list must be the same length as `ys`.\r\n\r\n`gradients()` adds ops to the graph to output the partial\r\nderivatives of `ys` with respect to `xs`. It returns a list of\r\n`Tensor` of length `len(xs)` where each tensor is the `sum(dy/dx)`\r\nfor y in `ys`.\r\n\r\n`grad_ys` is a list of tensors of the same length as `ys` that holds\r\nthe initial gradients for each y in `ys`. When `grad_ys` is None,\r\nwe fill in a tensor of '1's of the shape of y for each y in `ys`. A\r\nuser can provide their own initial 'grad_ys` to compute the\r\nderivatives using a different initial gradient for each y (e.g., if\r\none wanted to weight the gradient differently for each value in\r\neach y).\r\n\r\n##### Args: \r\n\r\n\r\n* `ys`: A `Tensor` or list of tensors to be differentiated.\r\n* `xs`: A `Tensor` or list of tensors to be used for differentiation.\r\n* `grad_ys`: Optional. A `Tensor` or list of tensors the same size as\r\n `ys` and holding the gradients computed for each y in `ys`.\r\n* `name`: Optional name to use for grouping all the gradient ops together.\r\n defaults to 'gradients'.\r\n* `colocate_gradients_with_ops`: If True, try colocating gradients with\r\n the corresponding op.\r\n* `gate_gradients`: If True, add a tuple around the gradients returned\r\n for an operations. This avoids some race conditions.\r\n* `aggregation_method`: Specifies the method used to combine gradient terms.\r\n Accepted values are constants defined in the class `AggregationMethod`.\r\n\r\n##### Returns: \r\n\r\n A list of `sum(dy/dx)` for each x in `xs`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `LookupError`: if one of the operations between `x` and `y` does not\r\n have a registered gradient function.\r\n* `ValueError`: if the arguments are invalid.\r\n\r\n\r\n- - -\r\n\r\n### `class tf.AggregationMethod` \r\n\r\nA class listing aggregation methods used to combine gradients.\r\n\r\nComputing partial derivatives can require aggregating gradient\r\ncontributions. This class lists the various methods that can\r\nbe used to combine gradients in the graph:\r\n\r\n* `ADD_N`: All of the gradient terms are summed as part of one\r\n operation using the \"AddN\" op. It has the property that all\r\n gradients must be ready before any aggregation is performed.\r\n* `DEFAULT`: The system-chosen default aggregation method.\r\n\r\n\r\n- - -\r\n\r\n### `tf.stop_gradient(input, name=None)` \r\n\r\nStops gradient computation.\r\n\r\nWhen executed in a graph, this op outputs its input tensor as-is.\r\n\r\nWhen building ops to compute gradients, this op prevents the contribution of\r\nits inputs to be taken into account. Normally, the gradient generator adds ops\r\nto a graph to compute the derivatives of a specified 'loss' by recursively\r\nfinding out inputs that contributed to its computation. If you insert this op\r\nin the graph it inputs are masked from the gradient generator. They are not\r\ntaken into account for computing gradients.\r\n\r\nThis is useful any time you want to compute a value with TensorFlow but need\r\nto pretend that the value was a constant. Some examples include:\r\n\r\n* The *EM* algorithm where the *M-step* should not involve backpropagation\r\n through the output of the *E-step*.\r\n* Contrastive divergence training of Boltzmann machines where, when\r\n differentiating the energy function, the training must not backpropagate\r\n through the graph that generated the samples from the model.\r\n* Adversarial training, where no backprop should happen through the adversarial\r\n example generation process.\r\n\r\n##### Args: \r\n\r\n\r\n* `input`: A `Tensor`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A `Tensor`. Has the same type as `input`.\r\n\r\n\r\n\r\n\r\n## Gradient Clipping \r\n\r\nTensorFlow provides several operations that you can use to add clipping\r\nfunctions to your graph. You can use these functions to perform general data\r\nclipping, but they're particularly useful for handling exploding or vanishing\r\ngradients.\r\n\r\n- - -\r\n\r\n### `tf.clip_by_value(t, clip_value_min, clip_value_max, name=None)` \r\n\r\nClips tensor values to a specified min and max.\r\n\r\nGiven a tensor `t`, this operation returns a tensor of the same type and\r\nshape as `t` with its values clipped to `clip_value_min` and `clip_value_max`.\r\nAny values less than `clip_value_min` are set to `clip_value_min`. Any values\r\ngreater than `clip_value_max` are set to `clip_value_max`.\r\n\r\n##### Args: \r\n\r\n\r\n* `t`: A `Tensor`.\r\n* `clip_value_min`: A 0-D (scalar) `Tensor`. The minimum value to clip by.\r\n* `clip_value_max`: A 0-D (scalar) `Tensor`. The maximum value to clip by.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A clipped `Tensor`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.clip_by_norm(t, clip_norm, name=None)` \r\n\r\nClips tensor values to a maximum L2-norm.\r\n\r\nGiven a tensor `t`, and a maximum clip value `clip_norm`, this operation\r\nnormalizes `t` so that its L2-norm is less than or equal to `clip_norm'.\r\nSpecifically, if the L2-norm is already less than or equal to `clip_norm`,\r\nthen `t` is not modified. If the L2-norm is greater than `clip_norm`, then\r\nthis operation returns a tensor of the same type and shape as `t` with its\r\nvalues set to:\r\n\r\n`t * clip_norm / l2norm(t)`\r\n\r\nIn this case, the L2-norm of the output tensor is `clip_norm`.\r\n\r\nThis operation is typically used to clip gradients before applying them with\r\nan optimizer.\r\n\r\n##### Args: \r\n\r\n\r\n* `t`: A `Tensor`.\r\n* `clip_norm`: A 0-D (scalar) `Tensor` > 0. A maximum clipping value.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A clipped `Tensor`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.clip_by_average_norm(t, clip_norm, name=None)` \r\n\r\nClips tensor values to a maximum average L2-norm.\r\n\r\nGiven a tensor `t`, and a maximum clip value `clip_norm`, this operation\r\nnormalizes `t` so that its average L2-norm is less than or equal to\r\n`clip_norm'. Specifically, if the average L2-norm is already less than or\r\nequal to `clip_norm`, then `t` is not modified. If the average L2-norm is\r\ngreater than `clip_norm`, then this operation returns a tensor of the same\r\ntype and shape as `t` with its values set to:\r\n\r\n`t * clip_norm / l2norm_avg(t)`\r\n\r\nIn this case, the average L2-norm of the output tensor is `clip_norm`.\r\n\r\nThis operation is typically used to clip gradients before applying them with\r\nan optimizer.\r\n\r\n##### Args: \r\n\r\n\r\n* `t`: A `Tensor`.\r\n* `clip_norm`: A 0-D (scalar) `Tensor` > 0. A maximum clipping value.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A clipped `Tensor`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.clip_by_global_norm(t_list, clip_norm, use_norm=None, name=None)` \r\n\r\nClips values of multiple tensors by the ratio of the sum of their norms.\r\n\r\nGiven a tuple or list of tensors `t_list`, and a clipping ratio `clip_norm`,\r\nthis operation returns a list of clipped tensors `list_clipped`\r\nand the global norm (`global_norm`) of all tensors in `t_list`. Optionally,\r\nif you've already computed the global norm for `t_list`, you can specify\r\nthe global norm with `use_norm`.\r\n\r\nTo perform the clipping, the values t_list[i] are set to:\r\n\r\n`t_list[i] * clip_norm / max(global_norm, clip_norm)`\r\n\r\nwhere:\r\n\r\n`global_norm = sqrt(sum([l2norm(t)**2 for t in t_list]))`\r\n\r\nIf `clip_norm > global_norm` then the entries in `t_list` remain as they are,\r\notherwise they're all shrunk by the global ratio.\r\n\r\nAny of the entries of `t_list` that are of type None are ignored.\r\n\r\nThis is the correct way to perform gradient clipping (for example, see\r\nR. Pascanu, T. Mikolov, and Y. Bengio, \"On the difficulty of training\r\nRecurrent Neural Networks\". http://arxiv.org/abs/1211.5063)\r\n\r\nHowever, it is slower than `clip_by_norm()` because all the parameters must be\r\nready before the clipping operation can be performed.\r\n\r\n##### Args: \r\n\r\n\r\n* `t_list`: A tuple or list of mixed `Tensors`, `IndexedSlices`, or None.\r\n* `clip_norm`: A 0-D (scalar) `Tensor` > 0. The clipping ratio.\r\n* `use_norm`: A 0-D (scalar) `Tensor` of type `float` (optional). The global\r\n norm to use. If not provided, `global_norm()` is used to compute the norm.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n\r\n* `list_clipped`: A list of `Tensors` of the same type as `list_t`.\r\n* `global_norm`: A 0-D (scalar) `Tensor` representing the global norm.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `t_list` is not a sequence.\r\n\r\n\r\n- - -\r\n\r\n### `tf.global_norm(t_list, name=None)` \r\n\r\nComputes the global norm of multiple tensors.\r\n\r\nGiven a tuple or list of tensors `t_list`, this operation returns the\r\nglobal norm of the elements in all tensors in `t_list`. The global norm is\r\ncomputed as:\r\n\r\n`global_norm = sqrt(sum([l2norm(t)**2 for t in t_list]))`\r\n\r\nAny entries in `t_list` that are of type None are ignored.\r\n\r\n##### Args: \r\n\r\n\r\n* `t_list`: A tuple or list of mixed `Tensors`, `IndexedSlices`, or None.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A 0-D (scalar) `Tensor` of type `float`.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If `t_list` is not a sequence.\r\n\r\n\r\n\r\n## Decaying the learning rate \r\n- - -\r\n\r\n### `tf.train.exponential_decay(learning_rate, global_step, decay_steps, decay_rate, staircase=False, name=None)` \r\n\r\nApplies exponential decay to the learning rate.\r\n\r\nWhen training a model, it is often recommended to lower the learning rate as\r\nthe training progresses. This function applies an exponential decay function\r\nto a provided initial learning rate. It requires a `global_step` value to\r\ncompute the decayed learning rate. You can just pass a TensorFlow variable\r\nthat you increment at each training step.\r\n\r\nThe function returns the decayed learning rate. It is computed as:\r\n\r\n```python\r\ndecayed_learning_rate = learning_rate *\r\n decay_rate ^ (global_step / decay_steps)\r\n```\r\n\r\nIf the argument `staircase` is `True`, then `global_step /decay_steps` is an\r\ninteger division and the decayed learning rate follows a staircase function.\r\n\r\nExample: decay every 100000 steps with a base of 0.96:\r\n\r\n```python\r\n...\r\nglobal_step = tf.Variable(0, trainable=False)\r\nstarter_learning_rate = 0.1\r\nlearning_rate = tf.exponential_decay(starter_learning_rate, global_step,\r\n 100000, 0.96, staircase=True)\r\noptimizer = tf.GradientDescent(learning_rate)\r\n# Passing global_step to minimize() will increment it at each step.\r\noptimizer.minimize(...my loss..., global_step=global_step)\r\n```\r\n\r\n##### Args: \r\n\r\n\r\n* `learning_rate`: A scalar `float32` or `float64` `Tensor` or a\r\n Python number. The initial learning rate.\r\n* `global_step`: A scalar `int32` or `int64` `Tensor` or a Python number.\r\n Global step to use for the decay computation. Must not be negative.\r\n* `decay_steps`: A scalar `int32` or `int64` `Tensor` or a Python number.\r\n Must be positive. See the decay computation above.\r\n* `decay_rate`: A scalar `float32` or `float64` `Tensor` or a\r\n Python number. The decay rate.\r\n* `staircase`: Boolean. It `True` decay the learning rate at discrete intervals.\r\n* `name`: string. Optional name of the operation. Defaults to 'ExponentialDecay'\r\n\r\n##### Returns: \r\n\r\n A scalar `Tensor` of the same type as `learning_rate`. The decayed\r\n learning rate.\r\n\r\n\r\n\r\n## Moving Averages \r\n\r\nSome training algorithms, such as GradientDescent and Momentum often benefit\r\nfrom maintaining a moving average of variables during optimization. Using the\r\nmoving averages for evaluations often improve results significantly.\r\n\r\n- - -\r\n\r\n### `class tf.train.ExponentialMovingAverage` \r\n\r\nMaintains moving averages of variables by employing and exponential decay.\r\n\r\nWhen training a model, it is often beneficial to maintain moving averages of\r\nthe trained parameters. Evaluations that use averaged parameters sometimes\r\nproduce significantly better results than the final trained values.\r\n\r\nThe `apply()` method adds shadow copies of trained variables and add ops that\r\nmaintain a moving average of the trained variables in their shadow copies.\r\nIt is used when building the training model. The ops that maintain moving\r\naverages are typically run after each training step.\r\nThe `average()` and `average_name()` methods give access to the shadow\r\nvariables and their names. They are useful when building an evaluation\r\nmodel, or when restoring a model from a checkpoint file. They help use the\r\nmoving averages in place of the last trained values for evaluations.\r\n\r\nThe moving averages are computed using exponential decay. You specify the\r\ndecay value when creating the `ExponentialMovingAverage` object. The shadow\r\nvariables are initialized with the same initial values as the trained\r\nvariables. When you run the ops to maintain the moving averages, each\r\nshadow variable is updated with the formula:\r\n\r\n `shadow_variable -= (1 - decay) * (shadow_variable - variable)`\r\n\r\nThis is mathematically equivalent to the classic formula below, but the use\r\nof an `assign_sub` op (the `\"-=\"` in the formula) allows concurrent lockless\r\nupdates to the variables:\r\n\r\n `shadow_variable = decay * shadow_variable + (1 - decay) * variable`\r\n\r\nReasonable values for `decay` are close to 1.0, typically in the\r\nmultiple-nines range: 0.999, 0.9999, etc.\r\n\r\nExample usage when creating a training model:\r\n\r\n```python\r\n# Create variables.\r\nvar0 = tf.Variable(...)\r\nvar1 = tf.Variable(...)\r\n# ... use the variables to build a training model...\r\n...\r\n# Create an op that applies the optimizer. This is what we usually\r\n# would use as a training op.\r\nopt_op = opt.minimize(my_loss, [var0, var1])\r\n\r\n# Create an ExponentialMovingAverage object\r\nema = tf.train.ExponentialMovingAverage(decay=0.9999)\r\n\r\n# Create the shadow variables, and add ops to maintain moving averages\r\n# of var0 and var1.\r\nmaintain_averages_op = ema.apply([var0, var1])\r\n\r\n# Create an op that will update the moving averages after each training\r\n# step. This is what we will use in place of the usuall trainig op.\r\nwith tf.control_dependencies([opt_op]):\r\n training_op = tf.group(maintain_averages_op)\r\n\r\n...train the model by running training_op...\r\n```\r\n\r\nThere are two ways to use the moving averages for evaluations:\r\n\r\n* Build a model that uses the shadow variables instead of the variables.\r\n For this, use the `average()` method which returns the shadow variable\r\n for a given variable.\r\n* Build a model normally but load the checkpoint files to evaluate by using\r\n the shadow variable names. For this use the `average_name()` method. See\r\n the [Saver class](../../api_docs/python/train.md#Saver) for more\r\n information on restoring saved variables.\r\n\r\nExample of restoring the shadow variable values:\r\n\r\n```python\r\n# Create a Saver that loads variables from their saved shadow values.\r\nshadow_var0_name = ema.average_name(var0)\r\nshadow_var1_name = ema.average_name(var1)\r\nsaver = tf.train.Saver({shadow_var0_name: var0, shadow_var1_name: var1})\r\nsaver.restore(...checkpoint filename...)\r\n# var0 and var1 now hold the moving average values\r\n```\r\n\r\n- - -\r\n\r\n#### `tf.train.ExponentialMovingAverage.__init__(decay, num_updates=None, name='ExponentialMovingAverage')` \r\n\r\nCreates a new ExponentialMovingAverage object.\r\n\r\nThe `Apply()` method has to be called to create shadow variables and add\r\nops to maintain moving averages.\r\n\r\nThe optional `num_updates` parameter allows one to tweak the decay rate\r\ndynamically. . It is typical to pass the count of training steps, usually\r\nkept in a variable that is incremented at each step, in which case the\r\ndecay rate is lower at the start of training. This makes moving averages\r\nmove faster. If passed, the actual decay rate used is:\r\n\r\n `min(decay, (1 + num_updates) / (10 + num_updates))`\r\n\r\n##### Args: \r\n\r\n\r\n* `decay`: Float. The decay to use.\r\n* `num_updates`: Optional count of number of updates applied to variables.\r\n* `name`: String. Optional prefix name to use for the name of ops added in\r\n `Apply()`.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.ExponentialMovingAverage.apply(var_list=None)` \r\n\r\nMaintains moving averages of variables.\r\n\r\n`var_list` must be a list of `Variable` or `Tensor` objects. This method\r\ncreates shadow variables for all elements of `var_list`. Shadow variables\r\nfor `Variable` objects are initialized to the variable's initial value.\r\nFor `Tensor` objects, the shadow variables are initialized to 0.\r\n\r\nshadow variables are created with `trainable=False` and added to the\r\n`GraphKeys.ALL_VARIABLES` collection. They will be returned by calls to\r\n`tf.all_variables()`.\r\n\r\nReturns an op that updates all shadow variables as described above.\r\n\r\nNote that `apply()` can be called multiple times with different lists of\r\nvariables.\r\n\r\n##### Args: \r\n\r\n\r\n* `var_list`: A list of Variable or Tensor objects. The variables\r\n and Tensors must be of types float32 or float64.\r\n\r\n##### Returns: \r\n\r\n An Operation that updates the moving averages.\r\n\r\n##### Raises: \r\n\r\n\r\n* `TypeError`: If the arguments are not all float32 or float64.\r\n* `ValueError`: If the moving average of one of the variables is already\r\n being computed.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.ExponentialMovingAverage.average_name(var)` \r\n\r\nReturns the name of the `Variable` holding the average for `var`.\r\n\r\nThe typical scenario for `ExponentialMovingAverage` is to compute moving\r\naverages of variables during training, and restore the variables from the\r\ncomputed moving averages during evaluations.\r\n\r\nTo restore variables, you have to know the name of the shadow variables.\r\nThat name and the original variable can then be passed to a `Saver()` object\r\nto restore the variable from the moving average value with:\r\n `saver = tf.train.Saver({ema.average_name(var): var})`\r\n\r\n`average_name()` can be called whether or not `apply()` has been called.\r\n\r\n##### Args: \r\n\r\n\r\n* `var`: A `Variable` object.\r\n\r\n##### Returns: \r\n\r\n A string: the name of the variable that will be used or was used\r\n by the `ExponentialMovingAverage class` to hold the moving average of\r\n `var`.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.ExponentialMovingAverage.average(var)` \r\n\r\nReturns the `Variable` holding the average of `var`.\r\n\r\n##### Args: \r\n\r\n\r\n* `var`: A `Variable` object.\r\n\r\n##### Returns: \r\n\r\n A `Variable` object or `None` if the moving average of `var`\r\n is not maintained..\r\n\r\n\r\n\r\n\r\n## Coordinator and QueueRunner \r\n\r\nSee [Threading and Queues](../../how_tos/threading_and_queues/index.md)\r\nfor how to use threads and queues. For documentation on the Queue API,\r\nsee [Queues](../../api_docs/python/io_ops.md#queues).\r\n\r\n- - -\r\n\r\n### `class tf.train.Coordinator` \r\n\r\nA coordinator for threads.\r\n\r\nThis class implements a simple mechanism to coordinate the termination of a\r\nset of threads.\r\n\r\n#### Usage: \r\n\r\n```python\r\n# Create a coordinator.\r\ncoord = Coordinator()\r\n# Start a number of threads, passing the coordinator to each of them.\r\n...start thread 1...(coord, ...)\r\n...start thread N...(coord, ...)\r\n# Wait for all the threads to terminate.\r\ncoord.join(threads)\r\n```\r\n\r\nAny of the threads can call `coord.request_stop()` to ask for all the threads\r\nto stop. To cooperate with the requests, each thread must check for\r\n`coord.should_stop()` on a regular basis. `coord.should_stop()` returns\r\n`True` as soon as `coord.request_stop()` has been called.\r\n\r\nA typical thread running with a Coordinator will do something like:\r\n\r\n```python\r\nwhile not coord.should_stop():\r\n ...do some work...\r\n```\r\n\r\n#### Exception handling: \r\n\r\nA thread can report an exception to the Coordinator as part of the\r\n`should_stop()` call. The exception will be re-raised from the\r\n`coord.join()` call.\r\n\r\nThread code:\r\n\r\n```python\r\ntry:\r\n while not coord.should_stop():\r\n ...do some work...\r\nexcept Exception, e:\r\n coord.request_stop(e)\r\n```\r\n\r\nMain code:\r\n\r\n```python\r\ntry:\r\n ...\r\n coord = Coordinator()\r\n # Start a number of threads, passing the coordinator to each of them.\r\n ...start thread 1...(coord, ...)\r\n ...start thread N...(coord, ...)\r\n # Wait for all the threads to terminate.\r\n coord.join(threads)\r\nexcept Exception, e:\r\n ...exception that was passed to coord.request_stop()\r\n```\r\n\r\n#### Grace period for stopping: \r\n\r\nAfter a thread has called `coord.request_stop()` the other threads have a\r\nfixed time to stop, this is called the 'stop grace period' and defaults to 2\r\nminutes. If any of the threads is still alive after the grace period expires\r\n`coord.join()` raises a RuntimeException reporting the laggards.\r\n\r\n```\r\ntry:\r\n ...\r\n coord = Coordinator()\r\n # Start a number of threads, passing the coordinator to each of them.\r\n ...start thread 1...(coord, ...)\r\n ...start thread N...(coord, ...)\r\n # Wait for all the threads to terminate, give them 10s grace period\r\n coord.join(threads, stop_grace_period_secs=10)\r\nexcept RuntimeException:\r\n ...one of the threads took more than 10s to stop after request_stop()\r\n ...was called.\r\nexcept Exception:\r\n ...exception that was passed to coord.request_stop()\r\n```\r\n- - -\r\n\r\n#### `tf.train.Coordinator.__init__()` \r\n\r\nCreate a new Coordinator.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.Coordinator.join(threads, stop_grace_period_secs=120)` \r\n\r\nWait for threads to terminate.\r\n\r\nBlocks until all 'threads' have terminated or request_stop() is called.\r\n\r\nAfter the threads stop, if an 'exc_info' was passed to request_stop, that\r\nexception is re-reaised.\r\n\r\nGrace period handling: When request_stop() is called, threads are given\r\n'stop_grace_period_secs' seconds to terminate. If any of them is still\r\nalive after that period expires, a RuntimeError is raised. Note that if\r\nan 'exc_info' was passed to request_stop() then it is raised instead of\r\nthat RuntimeError.\r\n\r\n##### Args: \r\n\r\n\r\n* `threads`: List threading.Threads. The started threads to join.\r\n* `stop_grace_period_secs`: Number of seconds given to threads to stop after\r\n request_stop() has been called.\r\n\r\n##### Raises: \r\n\r\n\r\n* `RuntimeError`: If any thread is still alive after request_stop()\r\n is called and the grace period expires.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.Coordinator.request_stop(ex=None)` \r\n\r\nRequest that the threads stop.\r\n\r\nAfter this is called, calls to should_stop() will return True.\r\n\r\n##### Args: \r\n\r\n\r\n* `ex`: Optional Exception, or Python 'exc_info' tuple as returned by\r\n sys.exc_info(). If this is the first call to request_stop() the\r\n corresponding exception is recorded and re-raised from join().\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.Coordinator.should_stop()` \r\n\r\nCheck if stop was requested.\r\n\r\n##### Returns: \r\n\r\n True if a stop was requested.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.Coordinator.wait_for_stop(timeout=None)` \r\n\r\nWait till the Coordinator is told to stop.\r\n\r\n##### Args: \r\n\r\n\r\n* `timeout`: float. Sleep for up to that many seconds waiting for\r\n should_stop() to become True.\r\n\r\n##### Returns: \r\n\r\n True if the Coordinator is told stop, False if the timeout expired.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `class tf.train.QueueRunner` \r\n\r\nHolds a list of enqueue operations for a queue, each to be run in a thread.\r\n\r\nQueues are a convenient TensorFlow mechanism to compute tensors\r\nasynchronously using multiple threads. For example in the canonical 'Input\r\nReader' setup one set of threads generates filenames in a queue; a second set\r\nof threads read records from the files, processes them, and enqueues tensors\r\non a second queue; a third set of threads dequeues these input records to\r\nconstruct batches and runs them through training operations.\r\n\r\nThere are several delicate issues when running multiple threads that way:\r\nclosing the queues in sequence as the input is exhausted, correctly catching\r\nand reporting exceptions, etc.\r\n\r\nThe `QueueRunner`, combined with the `Coordinator`, helps handle these issues.\r\n- - -\r\n\r\n#### `tf.train.QueueRunner.__init__(queue, enqueue_ops)` \r\n\r\nCreate a QueueRunner.\r\n\r\nOn construction the `QueueRunner` adds an op to close the queue. That op\r\nwill be run if the enqueue ops raise exceptions.\r\n\r\nWhen you later call the `create_threads()` method, the `QueueRunner` will\r\ncreate one thread for each op in `enqueue_ops`. Each thread will run its\r\nenqueue op in parallel with the other threads. The enqueue ops do not have\r\nto all be the same op, but it is expected that they all enqueue tensors in\r\n`queue`.\r\n\r\n##### Args: \r\n\r\n\r\n* `queue`: A `Queue`.\r\n* `enqueue_ops`: List of enqueue ops to run in threads later.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.QueueRunner.create_threads(sess, coord=None, daemon=False, start=False)` \r\n\r\nCreate threads to run the enqueue ops.\r\n\r\nThis method requires a session in which the graph was launched. It creates\r\na list of threads, optionally starting them. There is one thread for each\r\nop passed in `enqueue_ops`.\r\n\r\nThe `coord` argument is an optional coordinator, that the threads will use\r\nto terminate together and report exceptions. If a coordinator is given,\r\nthis method starts an additional thread to close the queue when the\r\ncoordinator requests a stop.\r\n\r\nThis method may be called again as long as all threads from a previous call\r\nhave stopped.\r\n\r\n##### Args: \r\n\r\n\r\n* `sess`: A `Session`.\r\n* `coord`: Optional `Coordinator` object for reporting errors and checking\r\n stop conditions.\r\n* `daemon`: Boolean. If `True` make the threads daemon threads.\r\n* `start`: Boolean. If `True` starts the threads. If `False` the\r\n caller must call the `start()` method of the returned threads.\r\n\r\n##### Returns: \r\n\r\n A list of threads.\r\n\r\n##### Raises: \r\n\r\n\r\n* `RuntimeError`: If threads from a previous call to `create_threads()` are\r\n still running.\r\n\r\n\r\n- - -\r\n\r\n#### `tf.train.QueueRunner.exceptions_raised` \r\n\r\nExceptions raised but not handled by the `QueueRunner` threads.\r\n\r\nExceptions raised in queue runner threads are handled in one of two ways\r\ndepending on whether or not a `Coordinator` was passed to\r\n`create_threads()`:\r\n\r\n* With a `Coordinator`, exceptions are reported to the coordinator and\r\n forgotten by the `QueueRunner`.\r\n* Without a `Coordinator`, exceptions are captured by the `QueueRunner` and\r\n made available in this `exceptions_raised` property.\r\n\r\n##### Returns: \r\n\r\n A list of Python `Exception` objects. The list is empty if no exception\r\n was captured. (No exceptions are captured when using a Coordinator.)\r\n\r\n\r\n- - -\r\n\r\n### `tf.train.add_queue_runner(qr, collection='queue_runners')` \r\n\r\nAdds a `QueueRunner` to a collection in the graph.\r\n\r\nWhen building a complex model that uses many queues it is often difficult to\r\ngather all the queue runners that need to be run. This convenience function\r\nallows you to add a queue runner to a well known collection in the graph.\r\n\r\nThe companion method `start_queue_runners()` can be used to start threads for\r\nall the collected queue runners.\r\n\r\n##### Args: \r\n\r\n\r\n* `qr`: A `QueueRunner`.\r\n* `collection`: A `GraphKey` specifying the graph collection to add\r\n the queue runner to. Defaults to `GraphKeys.QUEUE_RUNNERS`.\r\n\r\n\r\n- - -\r\n\r\n### `tf.train.start_queue_runners(sess=None, coord=None, daemon=True, start=True, collection='queue_runners')` \r\n\r\nStarts all queue runners collected in the graph.\r\n\r\nThis is a companion method to `add_queue_runner()`. It just starts\r\nthreads for all queue runners collected in the graph. It returns\r\nthe list of all threads.\r\n\r\n##### Args: \r\n\r\n\r\n* `sess`: `Session` used to run the queue ops. Defaults to the\r\n default session.\r\n* `coord`: Optional `Coordinator` for coordinating the started threads.\r\n* `daemon`: Whether the threads should be marked as `daemons`, meaning\r\n they don't block program exit.\r\n* `start`: Set to `False` to only create the threads, not start them.\r\n* `collection`: A `GraphKey` specifying the graph collection to\r\n get the queue runners from. Defaults to `GraphKeys.QUEUE_RUNNERS`.\r\n\r\n##### Returns: \r\n\r\n A list of threads.\r\n\r\n\r\n\r\n## Summary Operations \r\n\r\nThe following ops output\r\n[`Summary`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/framework/summary.proto)\r\nprotocol buffers as serialized string tensors.\r\n\r\nYou can fetch the output of a summary op in a session, and pass it to\r\na [SummaryWriter](../../api_docs/python/train.md#SummaryWriter) to append it\r\nto an event file. Event files contain\r\n[`Event`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/util/event.proto)\r\nprotos that can contain `Summary` protos along with the timestamp and\r\nstep. You can then use TensorBoard to visualize the contents of the\r\nevent files. See [TensorBoard and\r\nSummaries](../../how_tos/summaries_and_tensorboard/index.md) for more\r\ndetails.\r\n\r\n- - -\r\n\r\n### `tf.scalar_summary(tags, values, collections=None, name=None)` \r\n\r\nOutputs a `Summary` protocol buffer with scalar values.\r\n\r\nThe input `tags` and `values` must have the same shape. The generated\r\nsummary has a summary value for each tag-value pair in `tags` and `values`.\r\n\r\n##### Args: \r\n\r\n\r\n* `tags`: A 1-D `string` `Tensor`. Tags for the summaries.\r\n* `values`: A 1-D `float32` or `float64` Tensor. Values for the summaries.\r\n* `collections`: Optional list of graph collections keys. The new summary op is\r\n added to these collections. Defaults to `[GraphKeys.SUMMARIES]`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A scalar `Tensor` of type `string`. The serialized `Summary` protocol\r\n buffer.\r\n\r\n\r\n- - -\r\n\r\n### `tf.image_summary(tag, tensor, max_images=None, collections=None, name=None)` \r\n\r\nOutputs a `Summary` protocol buffer with images.\r\n\r\nThe summary has up to `max_images` summary values containing images. The\r\nimages are built from `tensor` which must be 4-D with shape `[batch_size,\r\nheight, width, channels]` and where `channels` can be:\r\n\r\n* 1: `tensor` is interpreted as Grayscale.\r\n* 3: `tensor` is interpreted as RGB.\r\n* 4: `tensor` is interpreted as RGBA.\r\n\r\nThe images have the same number of channels as the input tensor. Their values\r\nare normalized, one image at a time, to fit in the range `[0, 255]`. The\r\nop uses two different normalization algorithms:\r\n\r\n* If the input values are all positive, they are rescaled so the largest one\r\n is 255.\r\n\r\n* If any input value is negative, the values are shifted so input value 0.0\r\n is at 127. They are then rescaled so that either the smallest value is 0,\r\n or the largest one is 255.\r\n\r\nThe `tag` argument is a scalar `Tensor` of type `string`. It is used to\r\nbuild the `tag` of the summary values:\r\n\r\n* If `max_images` is 1, the summary value tag is '*tag*/image'.\r\n* If `max_images` is greater than 1, the summary value tags are\r\n generated sequentially as '*tag*/image/0', '*tag*/image/1', etc.\r\n\r\n##### Args: \r\n\r\n\r\n* `tag`: A scalar `Tensor` of type `string`. Used to build the `tag`\r\n of the summary values.\r\n* `tensor`: A 4-D `float32` `Tensor` of shape `[batch_size, height, width,\r\n channels]` where `channels` is 1, 3, or 4.\r\n* `max_images`: Max number of batch elements to generate images for.\r\n* `collections`: Optional list of ops.GraphKeys. The collections to add the\r\n summary to. Defaults to [ops.GraphKeys.SUMMARIES]\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A scalar `Tensor` of type `string`. The serialized `Summary` protocol\r\n buffer.\r\n\r\n\r\n- - -\r\n\r\n### `tf.histogram_summary(tag, values, collections=None, name=None)` \r\n\r\nOutputs a `Summary` protocol buffer with a histogram.\r\n\r\nThe generated\r\n[`Summary`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/framework/summary.proto)\r\nhas one summary value containing a histogram for `values`.\r\n\r\nThis op reports an `OutOfRange` error if any value is not finite.\r\n\r\n##### Args: \r\n\r\n\r\n* `tag`: A `string` `Tensor`. 0-D. Tag to use for the summary value.\r\n* `values`: A `float32` `Tensor`. Any shape. Values to use to build the\r\n histogram.\r\n* `collections`: Optional list of graph collections keys. The new summary op is\r\n added to these collections. Defaults to `[GraphKeys.SUMMARIES]`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A scalar `Tensor` of type `string`. The serialized `Summary` protocol\r\n buffer.\r\n\r\n\r\n- - -\r\n\r\n### `tf.nn.zero_fraction(value, name=None)` \r\n\r\nReturns the fraction of zeros in `value`.\r\n\r\nIf `value` is empty, the result is `nan`.\r\n\r\nThis is useful in summaries to measure and report sparsity. For example,\r\n\r\n z = tf.Relu(...)\r\n summ = tf.scalar_summary('sparsity', tf.zero_fraction(z))\r\n\r\n##### Args: \r\n\r\n\r\n* `value`: A tensor of numeric type.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n The fraction of zeros in `value`, with type `float32`.\r\n\r\n\r\n\r\n- - -\r\n\r\n### `tf.merge_summary(inputs, collections=None, name=None)` \r\n\r\nMerges summaries.\r\n\r\nThis op creates a\r\n[`Summary`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/framework/summary.proto)\r\nprotocol buffer that contains the union of all the values in the input\r\nsummaries.\r\n\r\nWhen the Op is run, it reports an `InvalidArgument` error if multiple values\r\nin the summaries to merge use the same tag.\r\n\r\n##### Args: \r\n\r\n\r\n* `inputs`: A list of `string` `Tensor` objects containing serialized `Summary`\r\n protocol buffers.\r\n* `collections`: Optional list of graph collections keys. The new summary op is\r\n added to these collections. Defaults to `[GraphKeys.SUMMARIES]`.\r\n* `name`: A name for the operation (optional).\r\n\r\n##### Returns: \r\n\r\n A scalar `Tensor` of type `string`. The serialized `Summary` protocol\r\n buffer resulting from the merging.\r\n\r\n\r\n- - -\r\n\r\n### `tf.merge_all_summaries(key='summaries')` \r\n\r\nMerges all summaries collected in the default graph.\r\n\r\n##### Args: \r\n\r\n\r\n* `key`: `GraphKey` used to collect the summaries. Defaults to\r\n `GraphKeys.SUMMARIES`.\r\n\r\n##### Returns: \r\n\r\n If no summaries were collected, returns None. Otherwise returns a scalar\r\n `Tensor` of type`string` containing the serialized `Summary` protocol\r\n buffer resulting from the merging.\r\n\r\n\r\n\r\n## Adding Summaries to Event Files \r\n\r\nSee [Summaries and\r\nTensorBoard](../../how_tos/summaries_and_tensorboard/index.md) for an\r\noverview of summaries, event files, and visualization in TensorBoard.\r\n\r\n- - -\r\n\r\n### `class tf.train.SummaryWriter` \r\n\r\nWrites `Summary` protocol buffers to event files.\r\n\r\nThe `SummaryWriter` class provides a mechanism to create an event file in a\r\ngiven directory and add summaries and events to it. The class updates the\r\nfile contents asynchronously. This allows a training program to call methods\r\nto add data to the file directly from the training loop, without slowing down\r\ntraining.\r\n\r\n- - -\r\n\r\n#### `tf.train.SummaryWriter.__init__(logdir, graph_def=None, max_queue=10, flush_secs=120)` \r\n\r\nCreates a `SummaryWriter` and an event file.\r\n\r\nOn construction the summary writer creates a new event file in `logdir`.\r\nThis event file will contain `Event` protocol buffers constructed when you\r\ncall one of the following functions: `add_summary()`, `add_event()`, or\r\n`add_graph()`.\r\n\r\nIf you pass a `graph_def` protocol buffer to the constructor it is added to\r\nthe event file. (This is equivalent to calling `add_graph()` later).\r\n\r\nTensorBoard will pick the graph from the file and display it graphically so\r\nyou can interactively explore the graph you built. You will usually pass\r\nthe graph from the session in which you launched it:\r\n\r\n```python\r\n...create a graph...\r\n# Launch the graph in a session.\r\nsess = tf.Session()\r\n# Create a summary writer, add the 'graph_def' to the event file.\r\nwriter = tf.train.SummaryWriter(
![\"面向机器学习初学者的](\"../images/blue_pill.png\")
\r\n ![\"面向机器学习专家的](\"../images/red_pill.png\")
\r\n 图片由 CC BY-SA 4.0 授权; 原作者 W. Carter
\r\n\r\n如果你已经下定决心, 准备学习和安装 TensorFlow, 你可以略过这些文字, 直接阅读\r\n后面的章节. 不用担心, 你仍然会看到 MNIST -- 在阐述 TensorFlow 的特性时,\r\n我们还会使用 MNIST 作为一个样例.\r\n\r\n## 推荐随后阅读: \r\n\r\n* [下载与安装](../get_started/os_setup.md)\r\n* [基本使用](../get_started/basic_usage.md)\r\n* [TensorFlow 技术指南](../tutorials/mnist/tf/index.md)\r\n\r\n\r\n\r\n\r\n
\r\n\r\n> 原文：[Introduction](http://tensorflow.org/get_started) 翻译：[@doc001](https://github.com/PFZheng) 校对：[@yangtze](https://github.com/sstruct)\r\n"
},
{
"path": "SOURCE/get_started/os_setup.md",
"content": "# 下载与安装 \n\n你可以使用我们提供的 Pip, Docker, Virtualenv, Anaconda 或 源码编译的方法安装 TensorFlow.\n\n## Pip 安装 \n\n[Pip](https://en.wikipedia.org/wiki/Pip_(package_manager)) 是一个 Python 的软件包安装与管理工具.\n\n在安装 TensorFlow 过程中要涉及安装或升级的包详见 [列表](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/pip_package/setup.py)\n\n首先安装 pip (或 Python3 的 pip3 ):\n\n```bash\n# Ubuntu/Linux 64-bit\n$ sudo apt-get install python-pip python-dev\n\n# Mac OS X\n$ sudo easy_install pip\n```\n\n安装 TensorFlow :\n\n```bash\n# Ubuntu/Linux 64-bit, CPU only, Python 2.7:\n$ sudo pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.8.0-cp27-none-linux_x86_64.whl\n\n# Ubuntu/Linux 64-bit, GPU enabled, Python 2.7. Requires CUDA toolkit 7.5 and CuDNN v4.\n# For other versions, see \"Install from sources\" below.\n$ sudo pip install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.8.0-cp27-none-linux_x86_64.whl\n\n# Mac OS X, CPU only:\n$ sudo easy_install --upgrade six\n$ sudo pip install --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.8.0-py2-none-any.whl\n```\n\n如果是 Python3 :\n\n```bash\n# Ubuntu/Linux 64-bit, CPU only, Python 3.4:\n$ sudo pip3 install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.8.0-cp34-cp34m-linux_x86_64.whl\n\n# Ubuntu/Linux 64-bit, GPU enabled, Python 3.4. Requires CUDA toolkit 7.5 and CuDNN v4.\n# For other versions, see \"Install from sources\" below.\n$ sudo pip3 install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.8.0-cp34-cp34m-linux_x86_64.whl\n\n# Mac OS X, CPU only:\n$ sudo easy_install --upgrade six\n$ sudo pip3 install --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.8.0-py3-none-any.whl\n```\n\n备注：如果之前安装过 TensorFlow < 0.7.1 的版本,应该先使用 `pip uninstall` 卸载 TensorFlow 和 protobuf ,保证获取的是一个最新 protobuf 依赖下的安装包.\n\n之后可以[测试](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/get_started/os_setup.md#test-the-tensorflow-installation)一下.\n\n## 基于 Docker 的安装 \n\n我们也支持通过 [Docker](http://docker.com/) 运行 TensorFlow. \n该方式的优点是不用操心软件依赖问题.\n\n首先, [安装 Docker](http://docs.docker.com/engine/installation/). 一旦 Docker\n已经启动运行, 可以通过命令启动一个容器:\n\n```bash\n$ docker run -it b.gcr.io/tensorflow/tensorflow\n```\n\n该命令将启动一个已经安装好 TensorFlow 及相关依赖的容器.\n\n### 其它镜像 \n\n默认的 Docker 镜像只包含启动和运行 TensorFlow 所需依赖库的一个最小集. 我们额外提供了\n下面的容器, 该容器同样可以通过上述 `docker run` 命令安装:\n\n* `b.gcr.io/tensorflow/tensorflow-full`: 镜像中的 TensorFlow 是从源代码完整安装的,\n 包含了编译和运行 TensorFlow 所需的全部工具. 在该镜像上, 可以直接使用源代码进行实验,\n 而不需要再安装上述的任何依赖.\n \n## 基于 VirtualEnv 的安装 \n\n我们推荐使用 [virtualenv](https://pypi.python.org/pypi/virtualenv) 创建一个隔离的容器, 来安装 TensorFlow. 这是可选的, 但是这样做能使排查安装问题变得更容易.\n\n首先, 安装所有必备工具:\n\n```bash\n# 在 Linux 上:\n$ sudo apt-get install python-pip python-dev python-virtualenv\n\n# 在 Mac 上:\n$ sudo easy_install pip # 如果还没有安装 pip\n$ sudo pip install --upgrade virtualenv\n```\n\n接下来, 建立一个全新的 virtualenv 环境. 为了将环境建在 `~/tensorflow`\n目录下, 执行:\n\n```bash\n$ virtualenv --system-site-packages ~/tensorflow\n$ cd ~/tensorflow\n```\n\n然后, 激活 virtualenv:\n\n```bash\n$ source bin/activate # 如果使用 bash\n$ source bin/activate.csh # 如果使用 csh\n(tensorflow)$ # 终端提示符应该发生变化\n```\n\n在 virtualenv 内, 安装 TensorFlow:\n\n```bash\n(tensorflow)$ pip install --upgrade <$url_to_binary.whl>\n```\n\n接下来, 使用类似命令运行 TensorFlow 程序:\n\n```bash\n(tensorflow)$ cd tensorflow/models/image/mnist\n(tensorflow)$ python convolutional.py\n\n# 当使用完 TensorFlow\n(tensorflow)$ deactivate # 停用 virtualenv\n\n$ # 你的命令提示符会恢复原样\n```\n\n## 基于 Anaconda 的安装 \n\n[Anaconda](https://www.continuum.io/why-anaconda) 是一个集成许多第三方科学计算库的 Python 科学计算环境,Anaconda 使用 conda 作为自己的包管理工具,同时具有自己的[计算环境](http://conda.pydata.org/docs/using/envs.html),类似 Virtualenv.\n\n和 Virtualenv 一样,不同 Python 工程需要的依赖包,conda 将他们存储在不同的地方。 TensorFlow 上安装的 Anaconda 不会对之前安装的 Python 包进行覆盖.\n\n* 安装 Anaconda\n* 建立一个 conda 计算环境\n* 激活环境,使用 conda 安装 TensorFlow\n* 安装成功后,每次使用 TensorFlow 的时候需要激活 conda 环境\n\n安装 Anaconda :\n\n参考 Anaconda 的下载页面的[指导](https://www.continuum.io/downloads)\n\n建立一个 conda 计算环境名字叫`tensorflow`:\n\n```bash\n# Python 2.7\n$ conda create -n tensorflow python=2.7\n\n# Python 3.4\n$ conda create -n tensorflow python=3.4\n```\n\n激活`tensorflow`环境,然后使用其中的 pip 安装 TensorFlow. 当使用`easy_install`使用`--ignore-installed`标记防止错误的产生。\n\n```bash\n$ source activate tensorflow\n(tensorflow)$ # Your prompt should change\n\n# Ubuntu/Linux 64-bit, CPU only, Python 2.7:\n(tensorflow)$ pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.8.0rc0-cp27-none-linux_x86_64.whl\n\n# Ubuntu/Linux 64-bit, GPU enabled, Python 2.7. Requires CUDA toolkit 7.5 and CuDNN v4.\n# For other versions, see \"Install from sources\" below.\n(tensorflow)$ pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.8.0rc0-cp27-none-linux_x86_64.whl\n\n# Mac OS X, CPU only:\n(tensorflow)$ pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.8.0rc0-py2-none-any.whl\n```\n\n对于 Python 3.x :\n\n```bash\n$ source activate tensorflow\n(tensorflow)$ # Your prompt should change\n\n# Ubuntu/Linux 64-bit, CPU only, Python 3.4:\n(tensorflow)$ pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.8.0rc0-cp34-cp34m-linux_x86_64.whl\n\n# Ubuntu/Linux 64-bit, GPU enabled, Python 3.4. Requires CUDA toolkit 7.5 and CuDNN v4.\n# For other versions, see \"Install from sources\" below.\n(tensorflow)$ pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.8.0rc0-cp34-cp34m-linux_x86_64.whl\n\n# Mac OS X, CPU only:\n(tensorflow)$ pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.8.0rc0-py3-none-any.whl\n```\n\nconda 环境激活后,你可以[测试](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/get_started/os_setup.md#test-the-tensorflow-installation)\n\n当你不用 TensorFlow 的时候,关闭环境:\n\n```bash\n(tensorflow)$ source deactivate\n\n$ # Your prompt should change back\n```\n\n再次使用的时候再激活 :-)\n\n```bash\n$ source activate tensorflow\n(tensorflow)$ # Your prompt should change.\n# Run Python programs that use TensorFlow.\n...\n# When you are done using TensorFlow, deactivate the environment.\n(tensorflow)$ source deactivate\n```\n\n\n\n## 尝试你的第一个 TensorFlow 程序 \n\n### (可选) 启用 GPU 支持 \n\n如果你使用 pip 二进制包安装了开启 GPU 支持的 TensorFlow, 你必须确保\n系统里安装了正确的 CUDA sdk 和 CUDNN 版本. 请参间 [CUDA 安装教程](#install_cuda)\n\n你还需要设置 `LD_LIBRARY_PATH` 和 `CUDA_HOME` 环境变量. 可以考虑将下面的命令\n添加到 `~/.bash_profile` 文件中, 这样每次登陆后自动生效. 注意, 下面的命令\n假定 CUDA 安装目录为 `/usr/local/cuda`:\n\n```bash\nexport LD_LIBRARY_PATH=\"$LD_LIBRARY_PATH:/usr/local/cuda/lib64\"\nexport CUDA_HOME=/usr/local/cuda\n```\n\n### 运行 TensorFlow \n\n打开一个 python 终端:\n\n```bash\n$ python\n\n>>> import tensorflow as tf\n>>> hello = tf.constant('Hello, TensorFlow!')\n>>> sess = tf.Session()\n>>> print sess.run(hello)\nHello, TensorFlow!\n>>> a = tf.constant(10)\n>>> b = tf.constant(32)\n>>> print sess.run(a+b)\n42\n>>>\n\n```\n\n## 从源码安装 \n\n### 克隆 TensorFlow 仓库 \n\n```bash\n$ git clone --recurse-submodules https://github.com/tensorflow/tensorflow\n```\n\n`--recurse-submodules` 参数是必须得, 用于获取 TesorFlow 依赖的 protobuf 库.\n\n### Linux 安装 \n\n#### 安装 Bazel \n\n首先依照 [教程](http://bazel.io/docs/install.html) 安装 Bazel 的依赖.\n然后在 [链接](https://github.com/bazelbuild/bazel/releases) 中下载适合你的操作系统的最新稳定版,\n最后按照下面脚本执行：\n\n```bash\n$ chmod +x PATH_TO_INSTALL.SH\n$ ./PATH_TO_INSTALL.SH --user\n```\n\n注意把 `PATH_TO_INSTALL.SH` 替换为你下载的安装包的文件路径.\n\n将执行路径 `output/bazel` 添加到 `$PATH` 环境变量中.\n\n#### 安装其他依赖 \n\n```bash\n# For Python 2.7:\n$ sudo apt-get install python-numpy swig python-dev python-wheel\n# For Python 3.x:\n$ sudo apt-get install python3-numpy swig python3-dev python3-wheel\n```\n\n#### 可选: 安装 CUDA (在 Linux 上开启 GPU 支持) \n\n为了编译并运行能够使用 GPU 的 TensorFlow, 需要先安装 NVIDIA 提供的 Cuda Toolkit 7.0\n和 CUDNN 6.5 V2.\n\nTensorFlow 的 GPU 特性只支持 NVidia Compute Capability >= 3.5 的显卡. 被支持的显卡\n包括但不限于:\n\n* NVidia Titan\n* NVidia Titan X\n* NVidia K20\n* NVidia K40\n\n##### 下载并安装 Cuda Toolkit 7.0 \n\n[下载地址](https://developer.nvidia.com/cuda-toolkit-70)\n\n将工具安装到诸如 `/usr/local/cuda` 之类的路径.\n\n##### 下载并安装 CUDNN Toolkit 6.5 \n\n[下载地址](https://developer.nvidia.com/rdp/cudnn-archive)\n\n解压并拷贝 CUDNN 文件到 Cuda Toolkit 7.0 安装路径下. 假设 Cuda Toolkit 7.0 安装\n在 `/usr/local/cuda`, 执行以下命令:\n\n``` bash\ntar xvzf cudnn-6.5-linux-x64-v2.tgz\nsudo cp cudnn-6.5-linux-x64-v2/cudnn.h /usr/local/cuda/include\nsudo cp cudnn-6.5-linux-x64-v2/libcudnn* /usr/local/cuda/lib64\n```\n\n##### 配置 TensorFlow 的 Cuda 选项 \n\n从源码树的根路径执行:\n\n``` bash\n$ ./configure\nDo you wish to bulid TensorFlow with GPU support? [y/n] y\nGPU support will be enabled for TensorFlow\n\nPlease specify the location where CUDA 7.0 toolkit is installed. Refer to\nREADME.md for more details. [default is: /usr/local/cuda]: /usr/local/cuda\n\nPlease specify the location where CUDNN 6.5 V2 library is installed. Refer to\nREADME.md for more details. [default is: /usr/local/cuda]: /usr/local/cuda\n\nSetting up Cuda include\nSetting up Cuda lib64\nSetting up Cuda bin\nSetting up Cuda nvvm\nConfiguration finished\n```\n\n这些配置将建立到系统 Cuda 库的符号链接. 每当 Cuda 库的路径发生变更时, 必须重新执行上述\n步骤, 否则无法调用 bazel 编译命令.\n\n##### 编译目标程序, 开启 GPU 支持 \n\n从源码树的根路径执行:\n\n```bash\n$ bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer\n\n$ bazel-bin/tensorflow/cc/tutorials_example_trainer --use_gpu\n# 大量的输出信息. 这个例子用 GPU 迭代计算一个 2x2 矩阵的主特征值 (major eigenvalue).\n# 最后几行输出和下面的信息类似.\n000009/000005 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]\n000006/000001 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]\n000009/000009 lambda = 2.000000 x = [0.894427 -0.447214] y = [1.788854 -0.894427]\n```\n\n注意, GPU 支持需通过编译选项 \"--config=cuda\" 开启.\n\n##### 已知问题 \n\n* 尽管可以在同一个源码树下编译开启 Cuda 支持和禁用 Cuda 支持的版本, 我们还是推荐在\n在切换这两种不同的编译配置时, 使用 \"bazel clean\" 清理环境.\n\n* 在执行 bazel 编译前必须先运行 configure, 否则编译会失败并提示错误信息. 未来, \n我们可能考虑将 configure 步骤包含在编译过程中, 以简化整个过程, 前提是 bazel 能够提供新的特性支持这样. \n\n### Mac OS X 安装 \n\nMac 和 Linux 需要的软件依赖完全一样, 但是安装过程区别很大. 以下链接用于帮助你\n在 Mac OS X 上安装这些依赖:\n\n#### Bazel \n\n参见[本网页](http://bazel.io/docs/install.html)的 Mac OS X 安装指南.\n\n#### SWIG \n\n[Mac OS X 安装教程](http://www.swig.org/Doc3.0/Preface.html#Preface_osx_installation).\n\n注意: 你需要安装[PCRE](ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/),\n而*不是* PCRE2.\n\n#### Numpy \n\n参见[安装教程](http://docs.scipy.org/doc/numpy/user/install.html).\n\n### 创建 pip 包并安装 \n\n```bash\n$ bazel build -c opt //tensorflow/tools/pip_package:build_pip_package\n\n$ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg\n\n# .whl 文件的实际名字与你所使用的平台有关\n$ pip install /tmp/tensorflow_pkg/tensorflow-0.5.0-cp27-none-linux_x86_64.whl\n```\n\n## 训练你的第一个 TensorFlow 神经网络模型 \n\n从源代码树的根路径执行:\n\n```python\n$ cd tensorflow/models/image/mnist\n$ python convolutional.py\nSuccesfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.\nSuccesfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.\nSuccesfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.\nSuccesfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.\nExtracting data/train-images-idx3-ubyte.gz\nExtracting data/train-labels-idx1-ubyte.gz\nExtracting data/t10k-images-idx3-ubyte.gz\nExtracting data/t10k-labels-idx1-ubyte.gz\nInitialized!\nEpoch 0.00\nMinibatch loss: 12.054, learning rate: 0.010000\nMinibatch error: 90.6%\nValidation error: 84.6%\nEpoch 0.12\nMinibatch loss: 3.285, learning rate: 0.010000\nMinibatch error: 6.2%\nValidation error: 7.0%\n...\n...\n```\n\n## 常见问题 \n\n### GPU 相关问题 \n\n如果在尝试运行一个 TensorFlow 程序时出现以下错误:\n\n```python\nImportError: libcudart.so.7.0: cannot open shared object file: No such file or directory\n```\n\n请确认你正确安装了 GPU 支持, 参见 [相关章节](#install_cuda).\n\n### 在 Linux 上 \n\n如果出现错误:\n\n```python\n...\n \"__add__\", \"__radd__\",\n ^\nSyntaxError: invalid syntax\n```\n\n解决方案: 确认正在使用的 Python 版本为 Python 2.7.\n\n### 在 Mac OS X 上 \n\n如果出现错误:\n\n```python\nimport six.moves.copyreg as copyreg\n\nImportError: No module named copyreg\n```\n\n解决方案: TensorFlow 使用的 protobuf 依赖 `six-1.10.0`. 但是, Apple 的默认 python 环境\n已经安装了 `six-1.4.1`, 该版本可能很难升级. 这里提供几种方法来解决该问题:\n\n1. 升级全系统的 `six`:\n\n ```bash\n sudo easy_install -U six\n ```\n\n2. 通过 homebrew 安装一个隔离的 python 副本:\n\n ```bash\n brew install python\n ```\n\n3. 在[`virtualenv`](#virtualenv_install) 内编译或使用 TensorFlow.\n\n\n如果出现错误:\n\n```\n>>> import tensorflow as tf\nTraceback (most recent call last):\n File \"\r\nREGISTER\\_OP(\"ZeroOut\")\r\n .Attr(\"preserve\\_index: int\")\r\n .Input(\"to\\_zero: int32\")\r\n .Output(\"zeroed: int32\");\r\n
\r\nclass ZeroOutOp : public OpKernel {\r\n public:\r\n explicit ZeroOutOp(OpKernelConstruction\\* context) : OpKernel(context) {\r\n // Get the index of the value to preserve\r\n OP\\_REQUIRES\\_OK(context,\r\n context->GetAttr(\"preserve\\_index\", &preserve\\_index\\_));\r\n // Check that preserve\\_index is positive\r\n OP\\_REQUIRES(context, preserve\\_index_ >= 0,\r\n errors::InvalidArgument(\"Need preserve\\_index >= 0, got \",\r\n preserve\\_index_));\r\n }\r\n void Compute(OpKernelContext\\* context) override {\r\n // ...\r\n }\r\n private:\r\n int preserve\\_index\\_;\r\n};\r\n\r\n void Compute(OpKernelContext\\* context) override {\r\n // ...\r\n
// Check that preserve_index is in range\r\n OP\\_REQUIRES(context, preserve\\_index_ < input.dimension(0),\r\n errors::InvalidArgument(\"preserve\\_index out of range\"));
\r\n // Set all the elements of the output tensor to 0\r\n const int N = input.size();\r\n for (int i = 0; i < N; i++) {\r\n output\\_flat(i) = 0;\r\n }
\r\n // Preserve the requested input value\r\n output\\_flat(preserve\\_index\\_) = input(preserve\\_index\\_);\r\n }\r\n\r\n> REGISTER\\_OP(\"ZeroOut\")\r\n> .Attr(\"preserve\\_index: int = 0\")\r\n> .Input(\"to_zero: int32\")\r\n> .Output(\"zeroed: int32\");\r\n>
\r\nREGISTER\\_OP(\"ZeroOut\")\r\n .Attr(\"T: {float, int32}\")\r\n .Input(\"to_zero: T\")\r\n .Output(\"zeroed: T\");\r\n\r\n\\#include \"tensorflow/core/framework/op_kernel.h\"
\r\nclass ZeroOutInt32Op : public OpKernel {\r\n // as before\r\n};
\r\nclass ZeroOutFloatOp : public OpKernel {\r\n public:\r\n explicit ZeroOutFloatOp(OpKernelConstruction\\* context)\r\n : OpKernel(context) {}
\r\n void Compute(OpKernelContext\\* context) override {\r\n // Grab the input tensor\r\n const Tensor& input\\_tensor = context->input(0);\r\n auto input = input\\_tensor.flat<float>();
\r\n // Create an output tensor\r\n Tensor* output = NULL;\r\n OP\\_REQUIRES\\_OK(context,\r\n context->allocate\\_output(0, input_tensor.shape(), &output));\r\n auto output\\_flat = output->template flat<float>();
\r\n // Set all the elements of the output tensor to 0\r\n const int N = input.size();\r\n for (int i = 0; i < N; i++) {\r\n output\\_flat(i) = 0;\r\n }
\r\n // Preserve the first input value\r\n if (N > 0) output\\_flat(0) = input(0);\r\n }\r\n};
\r\n// Note that TypeConstraint<int32>(\"T\") means that attr \"T\" (defined\r\n// in the Op registration above) must be \"int32\" to use this template\r\n// instantiation.\r\nREGISTER\\_KERNEL\\_BUILDER(\r\n Name(\"ZeroOut\")\r\n .Device(DEVICE\\_CPU)\r\n .TypeConstraint<int32>(\"T\"),\r\n ZeroOutOpInt32);\r\nREGISTER\\_KERNEL\\_BUILDER(\r\n Name(\"ZeroOut\")\r\n .Device(DEVICE\\_CPU)\r\n .TypeConstraint<float>(\"T\"),\r\n ZeroOutFloatOp);\r\n
\r\n> REGISTER\\_OP(\"ZeroOut\")\r\n> .Attr(\"T: {float, int32} = DT_INT32\")\r\n> .Input(\"to_zero: T\")\r\n> .Output(\"zeroed: T\")\r\n> \r\nREGISTER\\_OP(\"ZeroOut\")\r\n .Attr(\"T: {float, double, int32}\")\r\n .Input(\"to_zero: T\")\r\n .Output(\"zeroed: T\");\r\n\r\ntemplate <typename T>\r\nclass ZeroOutOp : public OpKernel {\r\n public:\r\n explicit ZeroOutOp(OpKernelConstruction\\* context) : OpKernel(context) {}
\r\n void Compute(OpKernelContext\\* context) override {\r\n // Grab the input tensor\r\n const Tensor& input\\_tensor = context->input(0);\r\n auto input = input\\_tensor.flat<T>();
\r\n // Create an output tensor\r\n Tensor* output = NULL;\r\n OP\\_REQUIRES\\_OK(context,\r\n context->allocate\\_output(0, input_tensor.shape(), &output));\r\n auto output\\_flat = output->template flat<T>();
\r\n // Set all the elements of the output tensor to 0\r\n const int N = input.size();\r\n for (int i = 0; i < N; i++) {\r\n output\\_flat(i) = 0;\r\n }
\r\n // Preserve the first input value\r\n if (N > 0) output\\_flat(0) = input(0);\r\n }\r\n};
\r\n// Note that TypeConstraint<int32>(\"T\") means that attr \"T\" (defined\r\n// in the Op registration above) must be \"int32\" to use this template\r\n// instantiation.\r\nREGISTER\\_KERNEL\\_BUILDER(\r\n Name(\"ZeroOut\")\r\n .Device(DEVICE\\_CPU)\r\n .TypeConstraint<int32>(\"T\"),\r\n ZeroOutOp<int32>);\r\nREGISTER\\_KERNEL\\_BUILDER(\r\n Name(\"ZeroOut\")\r\n .Device(DEVICE\\_CPU)\r\n .TypeConstraint<float>(\"T\"),\r\n ZeroOutOp<float>);\r\nREGISTER\\_KERNEL\\_BUILDER(\r\n Name(\"ZeroOut\")\r\n .Device(DEVICE\\_CPU)\r\n .TypeConstraint<double>(\"T\"),\r\n ZeroOutOp<double>);\r\n\r\n // 保留第一个输入值\r\n if (N > 0) output_flat(0) = input(0);\r\n }\r\n};\r\n// 注意, TypeConstraint
\r\n// 注意, TypeConstraint
\r\n ![\"Unexpanded](\"./pool1_collapsed.png\" "\\"Unexpanded") \r\n | \r\n \r\n ![\"Expanded](\"./pool1_expanded.png\" "\\"Expanded") \r\n | \r\n
\r\n Initial view of top-level name scope pool_1. Clicking on the orange + button on the top right or double-clicking on the node itself will expand it.\r\n | \r\n \r\n Expanded view of pool_1 name scope. Clicking on the orange - button on the top right or double-clicking on the node itself will collapse the name scope.\r\n | \r\n
\r\n ![\"conv_1](\"./conv_1.png\" "\\"conv_1") \r\n | \r\n \r\n ![\"save](\"./save.png\" "\\"save") \r\n | \r\n
\r\n Node conv_1 is connected to save. Note the little save node icon on its right.\r\n | \r\n \r\n save has a high degree, and will appear as an auxiliary node. The connection with conv_1 is shown as a node icon on its left. To further reduce clutter, since save has a lot of connections, we show the first 5 and abbreviate the others as ... 12 more.\r\n | \r\n
\r\n ![\"Sequence](\"./series.png\" "\\"Sequence") \r\n | \r\n \r\n ![\"Expanded](\"./series_expanded.png\" "\\"Expanded") \r\n | \r\n
| \r\n A collapsed view of a node sequence.\r\n | \r\n\r\n A small piece of the expanded view, after double-click.\r\n | \r\n
\r\n ![\"Info](\"./infocard.png\" "\\"Info") \r\n | \r\n \r\n ![\"Info](\"./infocard_op.png\" "\\"Info") \r\n | \r\n
\r\n Info card showing detailed information for the conv2 name scope. The inputs and outputs are combined from the inputs and outputs of the operation nodes inside the name scope. For name scopes no attributes are shown.\r\n | \r\n \r\n Info card showing detailed information for the DecodeRaw operation node. In addition to inputs and outputs, the card shows the device and the attributes associated with the current operation.\r\n | \r\n
\r\n ![\"Color](\"./colorby_structure.png\" "\\"Color") \r\n | \r\n \r\n ![\"Color](\"./colorby_device.png\" "\\"Color") \r\n | \r\n
\r\n Structure view: The gray nodes have unique structure. The orange conv1 and conv2 nodes have the same structure, and analogously for nodes with other colors.\r\n | \r\n \r\n Device view: Name scopes are colored proportionally to the fraction of devices of the operation nodes inside them. Here, purple means GPU and the green is CPU.\r\n | \r\n
\r\n ![\"未展开的名称域\"](\"../images/pool1_collapsed.png\" "\\"未展开的名称域\\"") \r\n | \r\n \r\n ![\"展开的名称域\"](\"../images/pool1_expanded.png\" "\\"展开的名称域\\"") \r\n | \r\n
\r\n 顶级名称域的初始视图pool_1，点击右上方橙色的+按钮或双击节点来展开。\r\n | \r\n \r\n 展开的pool_1名称域视图，点击右上方橙色的-按钮或双击节点来收起此名称域。\r\n | \r\n
\r\n ![\"conv_1是主图表的部分\"](\"../images/conv_1.png\" "\\"conv_1是主图表的部分\\"") \r\n | \r\n \r\n ![\"save被抽出为从属节点\"](\"../images/save.png\" "\\"save被抽出为从属节点\\"") \r\n | \r\n
\r\n 节点conv_1被连接到save，注意其右边save节点图标。\r\n | \r\n \r\n save has a high degree, 并会作为从属节点出现，与conv_1的连接作为一个节点图标显示在其左边。为了继续减少杂乱，既然save有很多连接，我们则只显示前5个，而把其余的缩略为... 12 more。\r\n | \r\n
\r\n ![\"节点序列\"](\"../images/series.png\" "\\"节点序列\\"") \r\n | \r\n \r\n ![\"展开的节点序列\"](\"../images/series_expanded.png\" "\\"展开的节点序列\\"") \r\n | \r\n
| \r\n 一个节点序列的折叠视图。\r\n | \r\n\r\n 视图的一小块, 双击后展开。\r\n | \r\n
\r\n ![\"一个名称域的详情卡片\"](\"../images/infocard.png\" "\\"一个名称域的详情卡片\\"") \r\n | \r\n \r\n ![\"操作节点的详情卡片\"](\"../images/infocard_op.png\" "\\"操作节点的详情卡片\\"") \r\n | \r\n
\r\n 详情卡片展示conv2名称域的详细信息，名称域中操作节点的输入和输出被结合在一起，适用于不显示属性的名称域。\r\n | \r\n \r\n 详情卡片展示DecodeRaw操作节点，除了输入和输出，卡片也会展示与当前节点相关的设备和属性。\r\n | \r\n
\r\n ![\"按结构着色\"](\"../images/colorby_structure.png\" "\\"按结构着色\\"") \r\n | \r\n \r\n ![\"按设备着色\"](\"../images/colorby_device.png\" "\\"按设备着色\\"") \r\n | \r\n
\r\n 结构视图：灰色节点的结构是唯一的。橙色的conv1和conv2节点有相同的结构, 其他颜色的节点也类似。\r\n | \r\n \r\n 设备视图：名称域根据其中的操作节点的设备片件来着色，在此紫色代表GPU，绿色代表CPU。\r\n | \r\n
\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
\r\n\r\n"
},
{
"path": "SOURCE/how_tos/reading_data/__init__.py",
"content": ""
},
{
"path": "SOURCE/how_tos/reading_data/convert_to_records.py",
"content": "\"\"\"Converts MNIST data to TFRecords file format with Example protos.\"\"\"\r\nfrom __future__ import print_function\r\n\r\nimport os\r\nimport tensorflow.python.platform\r\n\r\nimport numpy\r\nimport tensorflow as tf\r\nfrom tensorflow.g3doc.tutorials.mnist import input_data\r\n\r\n\r\nTRAIN_IMAGES = 'train-images-idx3-ubyte.gz' # MNIST filenames\r\nTRAIN_LABELS = 'train-labels-idx1-ubyte.gz'\r\nTEST_IMAGES = 't10k-images-idx3-ubyte.gz'\r\nTEST_LABELS = 't10k-labels-idx1-ubyte.gz'\r\n\r\n\r\ntf.app.flags.DEFINE_string('directory', 'data',\r\n 'Directory to download data files and write the '\r\n 'converted result')\r\ntf.app.flags.DEFINE_integer('validation_size', 5000,\r\n 'Number of examples to separate from the training '\r\n 'data for the validation set.')\r\nFLAGS = tf.app.flags.FLAGS\r\n\r\n\r\ndef _int64_feature(value):\r\n return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))\r\n\r\n\r\ndef _bytes_feature(value):\r\n return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))\r\n\r\n\r\ndef convert_to(images, labels, name):\r\n num_examples = labels.shape[0]\r\n if images.shape[0] != num_examples:\r\n raise ValueError(\"Images size %d does not match label size %d.\" %\r\n (dat.shape[0], num_examples))\r\n rows = images.shape[1]\r\n cols = images.shape[2]\r\n depth = images.shape[3]\r\n\r\n filename = os.path.join(FLAGS.directory, name + '.tfrecords')\r\n print('Writing', filename)\r\n writer = tf.python_io.TFRecordWriter(filename)\r\n for index in range(num_examples):\r\n image_raw = images[index].tostring()\r\n example = tf.train.Example(features=tf.train.Features(feature={\r\n 'height': _int64_feature(rows),\r\n 'width': _int64_feature(cols),\r\n 'depth': _int64_feature(depth),\r\n 'label': _int64_feature(int(labels[index])),\r\n 'image_raw': _bytes_feature(image_raw)}))\r\n writer.write(example.SerializeToString())\r\n\r\n\r\ndef main(argv):\r\n # Get the data.\r\n train_images_filename = input_data.maybe_download(\r\n TRAIN_IMAGES, FLAGS.directory)\r\n train_labels_filename = input_data.maybe_download(\r\n TRAIN_LABELS, FLAGS.directory)\r\n test_images_filename = input_data.maybe_download(\r\n TEST_IMAGES, FLAGS.directory)\r\n test_labels_filename = input_data.maybe_download(\r\n TEST_LABELS, FLAGS.directory)\r\n\r\n # Extract it into numpy arrays.\r\n train_images = input_data.extract_images(train_images_filename)\r\n train_labels = input_data.extract_labels(train_labels_filename)\r\n test_images = input_data.extract_images(test_images_filename)\r\n test_labels = input_data.extract_labels(test_labels_filename)\r\n\r\n # Generate a validation set.\r\n validation_images = train_images[:FLAGS.validation_size, :, :, :]\r\n validation_labels = train_labels[:FLAGS.validation_size]\r\n train_images = train_images[FLAGS.validation_size:, :, :, :]\r\n train_labels = train_labels[FLAGS.validation_size:]\r\n\r\n # Convert to Examples and write the result to TFRecords.\r\n convert_to(train_images, train_labels, 'train')\r\n convert_to(validation_images, validation_labels, 'validation')\r\n convert_to(test_images, test_labels, 'test')\r\n\r\n\r\nif __name__ == '__main__':\r\n tf.app.run()\r\n"
},
{
"path": "SOURCE/how_tos/reading_data/fully_connected_preloaded.py",
"content": "\"\"\"Trains the MNIST network using preloaded data in a constant.\r\n\r\nCommand to run this py_binary target:\r\n\r\nbazel run -c opt \\\r\n <...>/tensorflow/g3doc/how_tos/reading_data:fully_connected_preloaded\r\n\"\"\"\r\nfrom __future__ import print_function\r\nimport os.path\r\nimport time\r\n\r\nimport tensorflow.python.platform\r\nimport numpy\r\nimport tensorflow as tf\r\n\r\nfrom tensorflow.g3doc.tutorials.mnist import input_data\r\nfrom tensorflow.g3doc.tutorials.mnist import mnist\r\n\r\n\r\n# Basic model parameters as external flags.\r\nflags = tf.app.flags\r\nFLAGS = flags.FLAGS\r\nflags.DEFINE_float('learning_rate', 0.01, 'Initial learning rate.')\r\nflags.DEFINE_integer('num_epochs', 2, 'Number of epochs to run trainer.')\r\nflags.DEFINE_integer('hidden1', 128, 'Number of units in hidden layer 1.')\r\nflags.DEFINE_integer('hidden2', 32, 'Number of units in hidden layer 2.')\r\nflags.DEFINE_integer('batch_size', 100, 'Batch size. '\r\n 'Must divide evenly into the dataset sizes.')\r\nflags.DEFINE_string('train_dir', 'data', 'Directory to put the training data.')\r\nflags.DEFINE_boolean('fake_data', False, 'If true, uses fake data '\r\n 'for unit testing.')\r\n\r\n\r\ndef run_training():\r\n \"\"\"Train MNIST for a number of epochs.\"\"\"\r\n # Get the sets of images and labels for training, validation, and\r\n # test on MNIST.\r\n data_sets = input_data.read_data_sets(FLAGS.train_dir, FLAGS.fake_data)\r\n\r\n # Tell TensorFlow that the model will be built into the default Graph.\r\n with tf.Graph().as_default():\r\n with tf.name_scope('input'):\r\n # Input data\r\n input_images = tf.constant(data_sets.train.images)\r\n input_labels = tf.constant(data_sets.train.labels)\r\n\r\n image, label = tf.train.slice_input_producer(\r\n [input_images, input_labels], num_epochs=FLAGS.num_epochs)\r\n label = tf.cast(label, tf.int32)\r\n images, labels = tf.train.batch(\r\n [image, label], batch_size=FLAGS.batch_size)\r\n\r\n # Build a Graph that computes predictions from the inference model.\r\n logits = mnist.inference(images, FLAGS.hidden1, FLAGS.hidden2)\r\n\r\n # Add to the Graph the Ops for loss calculation.\r\n loss = mnist.loss(logits, labels)\r\n\r\n # Add to the Graph the Ops that calculate and apply gradients.\r\n train_op = mnist.training(loss, FLAGS.learning_rate)\r\n\r\n # Add the Op to compare the logits to the labels during evaluation.\r\n eval_correct = mnist.evaluation(logits, labels)\r\n\r\n # Build the summary operation based on the TF collection of Summaries.\r\n summary_op = tf.merge_all_summaries()\r\n\r\n # Create a saver for writing training checkpoints.\r\n saver = tf.train.Saver()\r\n\r\n # Create the op for initializing variables.\r\n init_op = tf.initialize_all_variables()\r\n\r\n # Create a session for running Ops on the Graph.\r\n sess = tf.Session()\r\n\r\n # Run the Op to initialize the variables.\r\n sess.run(init_op)\r\n\r\n # Instantiate a SummaryWriter to output summaries and the Graph.\r\n summary_writer = tf.train.SummaryWriter(FLAGS.train_dir,\r\n graph_def=sess.graph_def)\r\n\r\n # Start input enqueue threads.\r\n coord = tf.train.Coordinator()\r\n threads = tf.train.start_queue_runners(sess=sess, coord=coord)\r\n\r\n # And then after everything is built, start the training loop.\r\n try:\r\n step = 0\r\n while not coord.should_stop():\r\n start_time = time.time()\r\n\r\n # Run one step of the model.\r\n _, loss_value = sess.run([train_op, loss])\r\n\r\n duration = time.time() - start_time\r\n\r\n # Write the summaries and print an overview fairly often.\r\n if step % 100 == 0:\r\n # Print status to stdout.\r\n print('Step %d: loss = %.2f (%.3f sec)' % (step, loss_value,\r\n duration))\r\n # Update the events file.\r\n summary_str = sess.run(summary_op)\r\n summary_writer.add_summary(summary_str, step)\r\n step += 1\r\n\r\n # Save a checkpoint periodically.\r\n if (step + 1) % 1000 == 0:\r\n print('Saving')\r\n saver.save(sess, FLAGS.train_dir, global_step=step)\r\n\r\n step += 1\r\n except tf.errors.OutOfRangeError:\r\n print('Saving')\r\n saver.save(sess, FLAGS.train_dir, global_step=step)\r\n print('Done training for %d epochs, %d steps.' % (FLAGS.num_epochs, step))\r\n finally:\r\n # When done, ask the threads to stop.\r\n coord.request_stop()\r\n\r\n # Wait for threads to finish.\r\n coord.join(threads)\r\n sess.close()\r\n\r\n\r\ndef main(_):\r\n run_training()\r\n\r\n\r\nif __name__ == '__main__':\r\n tf.app.run()\r\n"
},
{
"path": "SOURCE/how_tos/reading_data/fully_connected_preloaded_var.py",
"content": "\"\"\"Trains the MNIST network using preloaded data stored in a variable.\r\n\r\nCommand to run this py_binary target:\r\n\r\nbazel run -c opt \\\r\n <...>/tensorflow/g3doc/how_tos/reading_data:fully_connected_preloaded_var\r\n\"\"\"\r\nfrom __future__ import print_function\r\nimport os.path\r\nimport time\r\n\r\nimport tensorflow.python.platform\r\nimport numpy\r\nimport tensorflow as tf\r\n\r\nfrom tensorflow.g3doc.tutorials.mnist import input_data\r\nfrom tensorflow.g3doc.tutorials.mnist import mnist\r\n\r\n\r\n# Basic model parameters as external flags.\r\nflags = tf.app.flags\r\nFLAGS = flags.FLAGS\r\nflags.DEFINE_float('learning_rate', 0.01, 'Initial learning rate.')\r\nflags.DEFINE_integer('num_epochs', 2, 'Number of epochs to run trainer.')\r\nflags.DEFINE_integer('hidden1', 128, 'Number of units in hidden layer 1.')\r\nflags.DEFINE_integer('hidden2', 32, 'Number of units in hidden layer 2.')\r\nflags.DEFINE_integer('batch_size', 100, 'Batch size. '\r\n 'Must divide evenly into the dataset sizes.')\r\nflags.DEFINE_string('train_dir', 'data', 'Directory to put the training data.')\r\nflags.DEFINE_boolean('fake_data', False, 'If true, uses fake data '\r\n 'for unit testing.')\r\n\r\n\r\ndef run_training():\r\n \"\"\"Train MNIST for a number of epochs.\"\"\"\r\n # Get the sets of images and labels for training, validation, and\r\n # test on MNIST.\r\n data_sets = input_data.read_data_sets(FLAGS.train_dir, FLAGS.fake_data)\r\n\r\n # Tell TensorFlow that the model will be built into the default Graph.\r\n with tf.Graph().as_default():\r\n with tf.name_scope('input'):\r\n # Input data\r\n images_initializer = tf.placeholder(\r\n dtype=data_sets.train.images.dtype,\r\n shape=data_sets.train.images.shape)\r\n labels_initializer = tf.placeholder(\r\n dtype=data_sets.train.labels.dtype,\r\n shape=data_sets.train.labels.shape)\r\n input_images = tf.Variable(\r\n images_initializer, trainable=False, collections=[])\r\n input_labels = tf.Variable(\r\n labels_initializer, trainable=False, collections=[])\r\n\r\n image, label = tf.train.slice_input_producer(\r\n [input_images, input_labels], num_epochs=FLAGS.num_epochs)\r\n label = tf.cast(label, tf.int32)\r\n images, labels = tf.train.batch(\r\n [image, label], batch_size=FLAGS.batch_size)\r\n\r\n # Build a Graph that computes predictions from the inference model.\r\n logits = mnist.inference(images, FLAGS.hidden1, FLAGS.hidden2)\r\n\r\n # Add to the Graph the Ops for loss calculation.\r\n loss = mnist.loss(logits, labels)\r\n\r\n # Add to the Graph the Ops that calculate and apply gradients.\r\n train_op = mnist.training(loss, FLAGS.learning_rate)\r\n\r\n # Add the Op to compare the logits to the labels during evaluation.\r\n eval_correct = mnist.evaluation(logits, labels)\r\n\r\n # Build the summary operation based on the TF collection of Summaries.\r\n summary_op = tf.merge_all_summaries()\r\n\r\n # Create a saver for writing training checkpoints.\r\n saver = tf.train.Saver()\r\n\r\n # Create the op for initializing variables.\r\n init_op = tf.initialize_all_variables()\r\n\r\n # Create a session for running Ops on the Graph.\r\n sess = tf.Session()\r\n\r\n # Run the Op to initialize the variables.\r\n sess.run(init_op)\r\n sess.run(input_images.initializer,\r\n feed_dict={images_initializer: data_sets.train.images})\r\n sess.run(input_labels.initializer,\r\n feed_dict={labels_initializer: data_sets.train.labels})\r\n\r\n # Instantiate a SummaryWriter to output summaries and the Graph.\r\n summary_writer = tf.train.SummaryWriter(FLAGS.train_dir,\r\n graph_def=sess.graph_def)\r\n\r\n # Start input enqueue threads.\r\n coord = tf.train.Coordinator()\r\n threads = tf.train.start_queue_runners(sess=sess, coord=coord)\r\n\r\n # And then after everything is built, start the training loop.\r\n try:\r\n step = 0\r\n while not coord.should_stop():\r\n start_time = time.time()\r\n\r\n # Run one step of the model.\r\n _, loss_value = sess.run([train_op, loss])\r\n\r\n duration = time.time() - start_time\r\n\r\n # Write the summaries and print an overview fairly often.\r\n if step % 100 == 0:\r\n # Print status to stdout.\r\n print('Step %d: loss = %.2f (%.3f sec)' % (step, loss_value,\r\n duration))\r\n # Update the events file.\r\n summary_str = sess.run(summary_op)\r\n summary_writer.add_summary(summary_str, step)\r\n step += 1\r\n\r\n # Save a checkpoint periodically.\r\n if (step + 1) % 1000 == 0:\r\n print('Saving')\r\n saver.save(sess, FLAGS.train_dir, global_step=step)\r\n\r\n step += 1\r\n except tf.errors.OutOfRangeError:\r\n print('Saving')\r\n saver.save(sess, FLAGS.train_dir, global_step=step)\r\n print('Done training for %d epochs, %d steps.' % (FLAGS.num_epochs, step))\r\n finally:\r\n # When done, ask the threads to stop.\r\n coord.request_stop()\r\n\r\n # Wait for threads to finish.\r\n coord.join(threads)\r\n sess.close()\r\n\r\n\r\ndef main(_):\r\n run_training()\r\n\r\n\r\nif __name__ == '__main__':\r\n tf.app.run()\r\n"
},
{
"path": "SOURCE/how_tos/reading_data/fully_connected_reader.py",
"content": "\"\"\"Train and Eval the MNIST network.\r\n\r\nThis version is like fully_connected_feed.py but uses data converted\r\nto a TFRecords file containing tf.train.Example protocol buffers.\r\nSee tensorflow/g3doc/how_tos/reading_data.md#reading-from-files\r\nfor context.\r\n\r\nYOU MUST run convert_to_records before running this (but you only need to\r\nrun it once).\r\n\"\"\"\r\nfrom __future__ import print_function\r\n\r\nimport os.path\r\nimport time\r\n\r\nimport tensorflow.python.platform\r\nimport numpy\r\nimport tensorflow as tf\r\n\r\nfrom tensorflow.g3doc.tutorials.mnist import mnist\r\n\r\n\r\n# Basic model parameters as external flags.\r\nflags = tf.app.flags\r\nFLAGS = flags.FLAGS\r\nflags.DEFINE_float('learning_rate', 0.01, 'Initial learning rate.')\r\nflags.DEFINE_integer('num_epochs', 2, 'Number of epochs to run trainer.')\r\nflags.DEFINE_integer('hidden1', 128, 'Number of units in hidden layer 1.')\r\nflags.DEFINE_integer('hidden2', 32, 'Number of units in hidden layer 2.')\r\nflags.DEFINE_integer('batch_size', 100, 'Batch size.')\r\nflags.DEFINE_string('train_dir', 'data', 'Directory with the training data.')\r\n\r\n# Constants used for dealing with the files, matches convert_to_records.\r\nTRAIN_FILE = 'train.tfrecords'\r\nVALIDATION_FILE = 'validation.tfrecords'\r\n\r\n\r\ndef read_and_decode(filename_queue):\r\n reader = tf.TFRecordReader()\r\n _, serialized_example = reader.read(filename_queue)\r\n features = tf.parse_single_example(\r\n serialized_example,\r\n dense_keys=['image_raw', 'label'],\r\n # Defaults are not specified since both keys are required.\r\n dense_types=[tf.string, tf.int64])\r\n\r\n # Convert from a scalar string tensor (whose single string has\r\n # length mnist.IMAGE_PIXELS) to a uint8 tensor with shape\r\n # [mnist.IMAGE_PIXELS].\r\n image = tf.decode_raw(features['image_raw'], tf.uint8)\r\n image.set_shape([mnist.IMAGE_PIXELS])\r\n\r\n # OPTIONAL: Could reshape into a 28x28 image and apply distortions\r\n # here. Since we are not applying any distortions in this\r\n # example, and the next step expects the image to be flattened\r\n # into a vector, we don't bother.\r\n\r\n # Convert from [0, 255] -> [-0.5, 0.5] floats.\r\n image = tf.cast(image, tf.float32) * (1. / 255) - 0.5\r\n\r\n # Convert label from a scalar uint8 tensor to an int32 scalar.\r\n label = tf.cast(features['label'], tf.int32)\r\n\r\n return image, label\r\n\r\n\r\ndef inputs(train, batch_size, num_epochs):\r\n \"\"\"Reads input data num_epochs times.\r\n\r\n Args:\r\n train: Selects between the training (True) and validation (False) data.\r\n batch_size: Number of examples per returned batch.\r\n num_epochs: Number of times to read the input data, or 0/None to\r\n train forever.\r\n\r\n Returns:\r\n A tuple (images, labels), where:\r\n * images is a float tensor with shape [batch_size, mnist.IMAGE_PIXELS]\r\n in the range [-0.5, 0.5].\r\n * labels is an int32 tensor with shape [batch_size] with the true label,\r\n a number in the range [0, mnist.NUM_CLASSES).\r\n Note that an tf.train.QueueRunner is added to the graph, which\r\n must be run using e.g. tf.train.start_queue_runners().\r\n \"\"\"\r\n if not num_epochs: num_epochs = None\r\n filename = os.path.join(FLAGS.train_dir,\r\n TRAIN_FILE if train else VALIDATION_FILE)\r\n\r\n with tf.name_scope('input'):\r\n filename_queue = tf.train.string_input_producer(\r\n [filename], num_epochs=num_epochs)\r\n\r\n # Even when reading in multiple threads, share the filename\r\n # queue.\r\n image, label = read_and_decode(filename_queue)\r\n\r\n # Shuffle the examples and collect them into batch_size batches.\r\n # (Internally uses a RandomShuffleQueue.)\r\n # We run this in two threads to avoid being a bottleneck.\r\n images, sparse_labels = tf.train.shuffle_batch(\r\n [image, label], batch_size=batch_size, num_threads=2,\r\n capacity=1000 + 3 * batch_size,\r\n # Ensures a minimum amount of shuffling of examples.\r\n min_after_dequeue=1000)\r\n\r\n return images, sparse_labels\r\n\r\n\r\ndef run_training():\r\n \"\"\"Train MNIST for a number of steps.\"\"\"\r\n\r\n # Tell TensorFlow that the model will be built into the default Graph.\r\n with tf.Graph().as_default():\r\n # Input images and labels.\r\n images, labels = inputs(train=True, batch_size=FLAGS.batch_size,\r\n num_epochs=FLAGS.num_epochs)\r\n\r\n # Build a Graph that computes predictions from the inference model.\r\n logits = mnist.inference(images,\r\n FLAGS.hidden1,\r\n FLAGS.hidden2)\r\n\r\n # Add to the Graph the loss calculation.\r\n loss = mnist.loss(logits, labels)\r\n\r\n # Add to the Graph operations that train the model.\r\n train_op = mnist.training(loss, FLAGS.learning_rate)\r\n\r\n # The op for initializing the variables.\r\n init_op = tf.initialize_all_variables()\r\n\r\n # Create a session for running operations in the Graph.\r\n sess = tf.Session()\r\n\r\n # Initialize the variables (the trained variables and the\r\n # epoch counter).\r\n sess.run(init_op)\r\n\r\n # Start input enqueue threads.\r\n coord = tf.train.Coordinator()\r\n threads = tf.train.start_queue_runners(sess=sess, coord=coord)\r\n\r\n try:\r\n step = 0\r\n while not coord.should_stop():\r\n start_time = time.time()\r\n\r\n # Run one step of the model. The return values are\r\n # the activations from the `train_op` (which is\r\n # discarded) and the `loss` op. To inspect the values\r\n # of your ops or variables, you may include them in\r\n # the list passed to sess.run() and the value tensors\r\n # will be returned in the tuple from the call.\r\n _, loss_value = sess.run([train_op, loss])\r\n\r\n duration = time.time() - start_time\r\n\r\n # Print an overview fairly often.\r\n if step % 100 == 0:\r\n print('Step %d: loss = %.2f (%.3f sec)' % (step, loss_value,\r\n duration))\r\n step += 1\r\n except tf.errors.OutOfRangeError:\r\n print('Done training for %d epochs, %d steps.' % (FLAGS.num_epochs, step))\r\n finally:\r\n # When done, ask the threads to stop.\r\n coord.request_stop()\r\n\r\n # Wait for threads to finish.\r\n coord.join(threads)\r\n sess.close()\r\n\r\n\r\ndef main(_):\r\n run_training()\r\n\r\n\r\nif __name__ == '__main__':\r\n tf.app.run()\r\n"
},
{
"path": "SOURCE/how_tos/reading_data/index.md",
"content": "# Reading data \r\n\r\nThere are three main methods of getting data into a TensorFlow program:\r\n\r\n* Feeding: Python code provides the data when running each step.\r\n* Reading from files: an input pipeline reads the data from files\r\n at the beginning of a TensorFlow graph.\r\n* Preloaded data: a constant or variable in the TensorFlow graph holds\r\n all the data (for small data sets).\r\n\r\n\r\n## Contents\r\n### [Reading data](#AUTOGENERATED-reading-data)\r\n* [Feeding](#Feeding)\r\n* [Reading from files](#AUTOGENERATED-reading-from-files)\r\n * [Filenames, shuffling, and epoch limits](#AUTOGENERATED-filenames--shuffling--and-epoch-limits)\r\n * [File formats](#AUTOGENERATED-file-formats)\r\n * [Preprocessing](#AUTOGENERATED-preprocessing)\r\n * [Batching](#AUTOGENERATED-batching)\r\n * [Creating threads to prefetch using `QueueRunner` objects](#QueueRunner)\r\n * [Filtering records or producing multiple examples per record](#AUTOGENERATED-filtering-records-or-producing-multiple-examples-per-record)\r\n * [Sparse input data](#AUTOGENERATED-sparse-input-data)\r\n* [Preloaded data](#AUTOGENERATED-preloaded-data)\r\n* [Multiple input pipelines](#AUTOGENERATED-multiple-input-pipelines)\r\n\r\n\r\n\r\n\r\n## Feeding \r\n\r\nTensorFlow's feed mechanism lets you inject data into any Tensor in a\r\ncomputation graph. A python computation can thus feed data directly into the\r\ngraph.\r\n\r\nSupply feed data through the `feed_dict` argument to a run() or eval() call\r\nthat initiates computation.\r\n\r\n```python\r\nwith tf.Session():\r\n input = tf.placeholder(tf.float32)\r\n classifier = ...\r\n print classifier.eval(feed_dict={input: my_python_preprocessing_fn()})\r\n```\r\n\r\nWhile you can replace any Tensor with feed data, including variables and\r\nconstants, the best practice is to use a\r\n[`placeholder` op](../../api_docs/python/io_ops.md#placeholder) node. A\r\n`placeholder` exists solely to serve as the target of feeds. It is not\r\ninitialized and contains no data. A placeholder generates an error if\r\nit is executed without a feed, so you won't forget to feed it.\r\n\r\nAn example using `placeholder` and feeding to train on MNIST data can be found\r\nin\r\n[`tensorflow/g3doc/tutorials/mnist/fully_connected_feed.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/tutorials/mnist/fully_connected_feed.py),\r\nand is described in the [MNIST tutorial](../../tutorials/mnist/tf/index.md).\r\n\r\n## Reading from files \r\n\r\nA typical pipeline for reading records from files has the following stages:\r\n\r\n1. The list of filenames\r\n2. *Optional* filename shuffling\r\n3. *Optional* epoch limit\r\n4. Filename queue\r\n5. A Reader for the file format\r\n6. A decoder for a record read by the reader\r\n7. *Optional* preprocessing\r\n8. Example queue\r\n\r\n### Filenames, shuffling, and epoch limits \r\n\r\nFor the list of filenames, use either a constant string Tensor (like\r\n`[\"file0\", \"file1\"]` or `[(\"file%d\" % i) for i in range(2)]`) or the\r\n[`tf.train.match_filenames_once`\r\nfunction](../../api_docs/python/io_ops.md#match_filenames_once).\r\n\r\nPass the list of filenames to the [`tf.train.string_input_producer`\r\nfunction](../../api_docs/python/io_ops.md#string_input_producer).\r\n`string_input_producer` creates a FIFO queue for holding the filenames until\r\nthe reader needs them.\r\n\r\n`string_input_producer` has options for shuffling and setting a maximum number\r\nof epochs. A queue runner adds the whole list of filenames to the queue once\r\nfor each epoch, shuffling the filenames within an epoch if `shuffle=True`.\r\nThis procedure provides a uniform sampling of files, so that examples are not\r\nunder- or over- sampled relative to each other.\r\n\r\nThe queue runner works in a thread separate from the reader that pulls\r\nfilenames from the queue, so the shuffling and enqueuing process does not\r\nblock the reader.\r\n\r\n### File formats \r\n\r\nSelect the reader that matches your input file format and pass the filename\r\nqueue to the reader's read method. The read method outputs a key identifying\r\nthe file and record (useful for debugging if you have some weird records), and\r\na scalar string value. Use one (or more) of the decoder and conversion ops to\r\ndecode this string into the tensors that make up an example.\r\n\r\n#### CSV files \r\n\r\nTo read text files in [comma-separated value (CSV)\r\nformat](https://tools.ietf.org/html/rfc4180), use a\r\n[`TextLineReader`](../../api_docs/python/io_ops.md#TextLineReader) with the\r\n[`decode_csv`](../../api_docs/python/io_ops.md#decode_csv) operation. For example:\r\n\r\n```python\r\nfilename_queue = tf.train.string_input_producer([\"file0.csv\", \"file1.csv\"])\r\n\r\nreader = tf.TextLineReader()\r\nkey, value = reader.read(filename_queue)\r\n\r\n# Default values, in case of empty columns. Also specifies the type of the\r\n# decoded result.\r\nrecord_defaults = [[1], [1], [1], [1], [1]]\r\ncol1, col2, col3, col4, col5 = tf.decode_csv(\r\n value, record_defaults=record_defaults)\r\nfeatures = tf.concat(0, [col1, col2, col3, col4])\r\n\r\nwith tf.Session() as sess:\r\n # Start populating the filename queue.\r\n coord = tf.train.Coordinator()\r\n threads = tf.train.start_queue_runners(coord=coord)\r\n\r\n for i in range(1200):\r\n # Retrieve a single instance:\r\n example, label = sess.run([features, col5])\r\n\r\n coord.request_stop()\r\n coord.join(threads)\r\n```\r\n\r\nEach execution of `read` reads a single line from the file. The\r\n`decode_csv` op then parses the result into a list of tensors. The\r\n`record_defaults` argument determines the type of the resulting tensors and\r\nsets the default value to use if a value is missing in the input string.\r\n\r\nYou must call `tf.train.start_queue_runners` to populate the queue before\r\nyou call `run` or `eval` to execute the `read`. Otherwise `read` will\r\nblock while it waits for filenames from the queue.\r\n\r\n#### Fixed length records \r\n\r\nTo read binary files in which each record is a fixed number of bytes, use\r\n[`tf.FixedLengthRecordReader`](../../api_docs/python/io_ops.md#FixedLengthRecordReader)\r\nwith the [`tf.decode_raw`](../../api_docs/python/io_ops.md#decode_raw) operation.\r\nThe `decode_raw` op converts from a string to a uint8 tensor.\r\n\r\nFor example, [the CIFAR-10 dataset](http://www.cs.toronto.edu/~kriz/cifar.html)\r\nuses a file format where each record is represented using a fixed number of\r\nbytes: 1 byte for the label followed by 3072 bytes of image data. Once you have\r\na uint8 tensor, standard operations can slice out each piece and reformat as\r\nneeded. For CIFAR-10, you can see how to do the reading and decoding in\r\n[`tensorflow/models/image/cifar10/cifar10_input.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/cifar10/cifar10_input.py)\r\nand described in\r\n[this tutorial](../../tutorials/deep_cnn/index.md#prepare-the-data).\r\n\r\n#### Standard TensorFlow format \r\n\r\nAnother approach is to convert whatever data you have into a supported format.\r\nThis approach makes it easier to mix and match data sets and network\r\narchitectures. The recommended format for TensorFlow is a TFRecords file\r\ncontaining\r\n[`tf.train.Example` protocol buffers](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/example/example.proto)\r\n(which contain\r\n[`Features`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/example/feature.proto)\r\nas a field). You write a little program that gets your data, stuffs it in an\r\n`Example` protocol buffer, serializes the protocol buffer to a string, and then\r\nwrites the string to a TFRecords file using the\r\n[`tf.python_io.TFRecordWriter` class](../../api_docs/python/python_io.md#TFRecordWriter).\r\nFor example,\r\n[`tensorflow/g3doc/how_tos/reading_data/convert_to_records.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/how_tos/reading_data/convert_to_records.py)\r\nconverts MNIST data to this format.\r\n\r\nTo read a file of TFRecords, use\r\n[`tf.TFRecordReader`](../../api_docs/python/io_ops.md#TFRecordReader) with\r\nthe [`tf.parse_single_example`](../../api_docs/python/io_ops.md#parse_single_example)\r\ndecoder. The `parse_single_example` op decodes the example protocol buffers into\r\ntensors. An MNIST example using the data produced by `convert_to_records` can be\r\nfound in\r\n[`tensorflow/g3doc/how_tos/reading_data/fully_connected_reader.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/how_tos/reading_data/fully_connected_reader.py),\r\nwhich you can compare with the `fully_connected_feed` version.\r\n\r\n### Preprocessing \r\n\r\nYou can then do any preprocessing of these examples you want. This would be any\r\nprocessing that doesn't depend on trainable parameters. Examples include\r\nnormalization of your data, picking a random slice, adding noise or distortions,\r\netc. See\r\n[`tensorflow/models/image/cifar10/cifar10.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/cifar10/cifar10.py)\r\nfor an example.\r\n\r\n### Batching \r\n\r\nAt the end of the pipeline we use another queue to batch together examples for\r\ntraining, evaluation, or inference. For this we use a queue that randomizes the\r\norder of examples, using the\r\n[`tf.train.shuffle_batch` function](../../api_docs/python/io_ops.md#shuffle_batch).\r\n\r\nExample:\r\n\r\n```\r\ndef read_my_file_format(filename_queue):\r\n reader = tf.SomeReader()\r\n key, record_string = reader.read(filename_queue)\r\n example, label = tf.some_decoder(record_string)\r\n processed_example = some_processing(example)\r\n return processed_example, label\r\n\r\ndef input_pipeline(filenames, batch_size, num_epochs=None):\r\n filename_queue = tf.train.string_input_producer(\r\n filenames, num_epochs=num_epochs, shuffle=True)\r\n example, label = read_my_file_format(filename_queue)\r\n # min_after_dequeue defines how big a buffer we will randomly sample\r\n # from -- bigger means better shuffling but slower start up and more\r\n # memory used.\r\n # capacity must be larger than min_after_dequeue and the amount larger\r\n # determines the maximum we will prefetch. Recommendation:\r\n # min_after_dequeue + (num_threads + a small safety margin) * batch_size\r\n min_after_dequeue = 10000\r\n capacity = min_after_dequeue + 3 * batch_size\r\n example_batch, label_batch = tf.train.shuffle_batch(\r\n [example, label], batch_size=batch_size, capacity=capacity,\r\n min_after_dequeue=min_after_dequeue)\r\n return example_batch, label_batch\r\n```\r\n\r\nIf you need more parallelism or shuffling of examples between files, use\r\nmultiple reader instances using the\r\n[`tf.train.shuffle_batch_join` function](../../api_docs/python/io_ops.md#shuffle_batch_join).\r\nFor example:\r\n\r\n```\r\ndef read_my_file_format(filename_queue):\r\n # Same as above\r\n\r\ndef input_pipeline(filenames, batch_size, read_threads, num_epochs=None):\r\n filename_queue = tf.train.string_input_producer(\r\n filenames, num_epochs=num_epochs, shuffle=True)\r\n example_list = [read_my_file_format(filename_queue)\r\n for _ in range(read_threads)]\r\n min_after_dequeue = 10000\r\n capacity = min_after_dequeue + 3 * batch_size\r\n example_batch, label_batch = tf.train.shuffle_batch_join(\r\n example_list, batch_size=batch_size, capacity=capacity,\r\n min_after_dequeue=min_after_dequeue)\r\n return example_batch, label_batch\r\n```\r\n\r\nYou still only use a single filename queue that is shared by all the readers.\r\nThat way we ensure that the different readers use different files from the same\r\nepoch until all the files from the epoch have been started. (It is also usually\r\nsufficient to have a single thread filling the filename queue.)\r\n\r\nAn alternative is to use a single reader via the\r\n[`tf.train.shuffle_batch` function](../../api_docs/python/io_ops.md#shuffle_batch)\r\nwith `num_threads` bigger than 1. This will make it read from a single file at\r\nthe same time (but faster than with 1 thread), instead of N files at once.\r\nThis can be important:\r\n\r\n* If you have more reading threads than input files, to avoid the risk that\r\n you will have two threads reading the same example from the same file near\r\n each other.\r\n* Or if reading N files in parallel causes too many disk seeks.\r\n\r\nHow many threads do you need? the `tf.train.shuffle_batch*` functions add a\r\nsummary to the graph that indicates how full the example queue is. If you have\r\nenough reading threads, that summary will stay above zero. You can\r\n[view your summaries as training progresses using TensorBoard](../../how_tos/summaries_and_tensorboard/index.md).\r\n\r\n### Creating threads to prefetch using `QueueRunner` objects \r\n\r\nThe short version: many of the `tf.train` functions listed above add\r\n[`QueueRunner`](../../api_docs/python/train.md#QueueRunner) objects to your\r\ngraph. These require that you call\r\n[`tf.train.start_queue_runners`](../../api_docs/python/train.md#start_queue_runners)\r\nbefore running any training or inference steps, or it will hang forever. This\r\nwill start threads that run the input pipeline, filling the example queue so\r\nthat the dequeue to get the examples will succeed. This is best combined with a\r\n[`tf.train.Coordinator`](../../api_docs/python/train.md#Coordinator) to cleanly\r\nshut down these threads when there are errors. If you set a limit on the number\r\nof epochs, that will use an epoch counter that will need to be intialized. The\r\nrecommended code pattern combining these is:\r\n\r\n```python\r\n# Create the graph, etc.\r\ninit_op = tf.initialize_all_variables()\r\n\r\n# Create a session for running operations in the Graph.\r\nsess = tf.Session()\r\n\r\n# Initialize the variables (like the epoch counter).\r\nsess.run(init_op)\r\n\r\n# Start input enqueue threads.\r\ncoord = tf.train.Coordinator()\r\nthreads = tf.train.start_queue_runners(sess=sess, coord=coord)\r\n\r\ntry:\r\n while not coord.should_stop():\r\n # Run training steps or whatever\r\n sess.run(train_op)\r\n\r\nexcept tf.errors.OutOfRangeError:\r\n print 'Done training -- epoch limit reached'\r\nfinally:\r\n # When done, ask the threads to stop.\r\n coord.request_stop()\r\n\r\n# Wait for threads to finish.\r\ncoord.join(threads)\r\nsess.close()\r\n```\r\n\r\n#### Aside: What is happening here? \r\n\r\nFirst we create the graph. It will have a few pipeline stages that are\r\nconnected by queues. The first stage will generate filenames to read and enqueue\r\nthem in the filename queue. The second stage consumes filenames (using a\r\n`Reader`), produces examples, and enqueues them in an example queue. Depending\r\non how you have set things up, you may actually have a few independent copies of\r\nthe second stage, so that you can read from multiple files in parallel. At the\r\nend of these stages is an enqueue operation, which enqueues into a queue that\r\nthe next stage dequeues from. We want to start threads running these enqueuing\r\noperations, so that our training loop can dequeue examples from the example\r\nqueue.\r\n\r\n\r\n![](\"AnimatedFileQueues.gif\")
\r\n
\r\n\r\nThe helpers in `tf.train` that create these queues and enqueuing operations add\r\na [`tf.train.QueueRunner`](../../api_docs/python/train.md#QueueRunner) to the\r\ngraph using the\r\n[`tf.train.add_queue_runner`](../../api_docs/python/train.md#add_queue_runner)\r\nfunction. Each `QueueRunner` is responsible for one stage, and holds the list of\r\nenqueue operations that need to be run in threads. Once the graph is\r\nconstructed, the\r\n[`tf.train.start_queue_runners`](../../api_docs/python/train.md#start_queue_runners)\r\nfunction asks each QueueRunner in the graph to start its threads running the\r\nenqueuing operations.\r\n\r\nIf all goes well, you can now run your training steps and the queues will be\r\nfilled by the background threads. If you have set an epoch limit, at some point\r\nan attempt to dequeue examples will get an\r\n[`tf.OutOfRangeError`](../../api_docs/python/client.md#OutOfRangeError). This\r\nis the TensorFlow equivalent of \"end of file\" (EOF) -- this means the epoch\r\nlimit has been reached and no more examples are available.\r\n\r\nThe last ingredient is the\r\n[`Coordinator`](../../api_docs/python/train.md#Coordinator). This is responsible\r\nfor letting all the threads know if anything has signalled a shut down. Most\r\ncommonly this would be because an exception was raised, for example one of the\r\nthreads got an error when running some operation (or an ordinary Python\r\nexception).\r\n\r\nFor more about threading, queues, QueueRunners, and Coordinators\r\n[see here](../../how_tos/threading_and_queues/index.md).\r\n\r\n#### Aside: How clean shut-down when limiting epochs works \r\n\r\nImagine you have a model that has set a limit on the number of epochs to train\r\non. That means that the thread generating filenames will only run that many\r\ntimes before generating an `OutOfRange` error. The QueueRunner will catch that\r\nerror, close the filename queue, and exit the thread. Closing the queue does two\r\nthings:\r\n\r\n* Any future attempt to enqueue in the filename queue will generate an error.\r\n At this point there shouldn't be any threads trying to do that, but this\r\n is helpful when queues are closed due to other errors.\r\n* Any current or future dequeue will either succeed (if there are enough\r\n elements left) or fail (with an `OutOfRange` error) immediately. They won't\r\n block waiting for more elements to be enqueued, since by the previous point\r\n that can't happen.\r\n\r\nThe point is that when the filename queue is closed, there will likely still be\r\nmany filenames in that queue, so the next stage of the pipeline (with the reader\r\nand other preprocessing) may continue running for some time. Once the filename\r\nqueue is exhausted, though, the next attempt to dequeue a filename (e.g. from a\r\nreader that has finished with the file it was working on) will trigger an\r\n`OutOfRange` error. In this case, though, you might have multiple threads\r\nassociated with a single QueueRunner. If this isn't the last thread in the\r\nQueueRunner, the `OutOfRange` error just causes the one thread to exit. This\r\nallows the other threads, which are still finishing up their last file, to\r\nproceed until they finish as well. (Assuming you are using a\r\n[`tf.train.Coordinator`](../../api_docs/python/train.md#Coordinator),\r\nother types of errors will cause all the threads to stop.) Once all the reader\r\nthreads hit the `OutOfRange` error, only then does the next queue, the example\r\nqueue, gets closed.\r\n\r\nAgain, the example queue will have some elements queued, so training will\r\ncontinue until those are exhausted. If the example queue is a\r\n[`RandomShuffleQueue`](../../api_docs/python/io_ops.md#RandomShuffleQueue), say\r\nbecause you are using `shuffle_batch` or `shuffle_batch_join`, it normally will\r\navoid ever going having fewer than its `min_after_dequeue` attr elements\r\nbuffered. However, once the queue is closed that restriction will be lifted and\r\nthe queue will eventually empty. At that point the actual training threads,\r\nwhen they try and dequeue from example queue, will start getting `OutOfRange`\r\nerrors and exiting. Once all the training threads are done,\r\n[`tf.train.Coordinator.join`](../../api_docs/python/train.md#Coordinator.join)\r\nwill return and you can exit cleanly.\r\n\r\n### Filtering records or producing multiple examples per record \r\n\r\nInstead of examples with shapes `[x, y, z]`, you will produce a batch of\r\nexamples with shape `[batch, x, y, z]`. The batch size can be 0 if you want to\r\nfilter this record out (maybe it is in a hold-out set?), or bigger than 1 if you\r\nare producing multiple examples per record. Then simply set `enqueue_many=True`\r\nwhen calling one of the batching functions (such as `shuffle_batch` or\r\n`shuffle_batch_join`).\r\n\r\n### Sparse input data \r\n\r\nSparseTensors don't play well with queues. If you use SparseTensors you have\r\nto decode the string records using\r\n[`tf.parse_example`](../../api_docs/python/io_ops.md#parse_example) **after**\r\nbatching (instead of using `tf.parse_single_example` before batching).\r\n\r\n## Preloaded data \r\n\r\nThis is only used for small data sets that can be loaded entirely in memory.\r\nThere are two approaches:\r\n\r\n* Store the data in a constant.\r\n* Store the data in a variable, that you initialize and then never change.\r\n\r\nUsing a constant is a bit simpler, but uses more memory (since the constant is\r\nstored inline in the graph data structure, which may be duplicated a few times).\r\n\r\n```python\r\ntraining_data = ...\r\ntraining_labels = ...\r\nwith tf.Session():\r\n input_data = tf.constant(training_data)\r\n input_labels = tf.constant(training_labels)\r\n ...\r\n```\r\n\r\nTo instead use a variable, you need to also initialize it after the graph has been built.\r\n\r\n```python\r\ntraining_data = ...\r\ntraining_labels = ...\r\nwith tf.Session() as sess:\r\n data_initializer = tf.placeholder(dtype=training_data.dtype,\r\n shape=training_data.shape)\r\n label_initializer = tf.placeholder(dtype=training_labels.dtype,\r\n shape=training_labels.shape)\r\n input_data = tf.Variable(data_initalizer, trainable=False, collections=[])\r\n input_labels = tf.Variable(label_initalizer, trainable=False, collections=[])\r\n ...\r\n sess.run(input_data.initializer,\r\n feed_dict={data_initializer: training_data})\r\n sess.run(input_labels.initializer,\r\n feed_dict={label_initializer: training_lables})\r\n```\r\n\r\nSetting `trainable=False` keeps the variable out of the\r\n`GraphKeys.TRAINABLE_VARIABLES` collection in the graph, so we won't try and\r\nupdate it when training. Setting `collections=[]` keeps the variable out of the\r\n`GraphKeys.VARIABLES` collection used for saving and restoring checkpoints.\r\n\r\nEither way,\r\n[`tf.train.slice_input_producer function`](../../api_docs/python/io_ops.md#slice_input_producer)\r\ncan be used to produce a slice at a time. This shuffles the examples across an\r\nentire epoch, so further shuffling when batching is undesirable. So instead of\r\nusing the `shuffle_batch` functions, we use the plain\r\n[`tf.train.batch` function](../../api_docs/python/io_ops.md#batch). To use\r\nmultiple preprocessing threads, set the `num_threads` parameter to a number\r\nbigger than 1.\r\n\r\nAn MNIST example that preloads the data using constants can be found in\r\n[`tensorflow/g3doc/how_tos/reading_data/fully_connected_preloaded.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/how_tos/reading_data/fully_connected_preloaded.py), and one that preloads the data using variables can be found in\r\n[`tensorflow/g3doc/how_tos/reading_data/fully_connected_preloaded_var.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/how_tos/reading_data/fully_connected_preloaded_var.py),\r\nYou can compare these with the `fully_connected_feed` and\r\n`fully_connected_reader` versions above.\r\n\r\n## Multiple input pipelines \r\n\r\nCommonly you will want to train on one dataset and evaluate (or \"eval\") on\r\nanother. One way to do this is to actually have two separate processes:\r\n\r\n* The training process reads training input data and periodically writes\r\n checkpoint files with all the trained variables.\r\n* The evaluation process restores the checkpoint files into an inference\r\n model that reads validation input data.\r\n\r\nThis is what is done in\r\n[the example CIFAR-10 model](../../tutorials/deep_cnn/index.md#save-and-restore-checkpoints). This has a couple of benefits:\r\n\r\n* The eval is performed on a single snapshot of the trained variables.\r\n* You can perform the eval even after training has completed and exited.\r\n\r\nYou can have the train and eval in the same graph in the same process, and share\r\ntheir trained variables. See\r\n[the shared variables tutorial](../../how_tos/variable_scope/index.md).\r\n"
},
{
"path": "SOURCE/how_tos/reading_data.md",
"content": "# 数据读取 \r\n\r\nTensorFlow程序读取数据一共有3种方法:\r\n\r\n* 供给数据(Feeding)： 在TensorFlow程序运行的每一步， 让Python代码来供给数据。\r\n* 从文件读取数据： 在TensorFlow图的起始， 让一个输入管线从文件中读取数据。\r\n* 预加载数据： 在TensorFlow图中定义常量或变量来保存所有数据(仅适用于数据量比较小的情况)。\r\n\r\n\r\n## 目录\r\n### [数据读取](#AUTOGENERATED-reading-data)\r\n* [供给数据(Feeding)](#Feeding)\r\n* [从文件读取数据](#AUTOGENERATED-reading-from-files)\r\n * [文件名, 乱序(shuffling), 和最大训练迭代数(epoch limits)](#AUTOGENERATED-filenames--shuffling--and-epoch-limits)\r\n * [文件格式](#AUTOGENERATED-file-formats)\r\n * [预处理](#AUTOGENERATED-preprocessing)\r\n * [批处理](#AUTOGENERATED-batching)\r\n * [使用`QueueRunner`创建预读线程](#QueueRunner)\r\n * [对记录进行过滤或者为每个纪录创建多个样本](#AUTOGENERATED-filtering-records-or-producing-multiple-examples-per-record)\r\n * [序列化输入数据(Sparse input data)](#AUTOGENERATED-sparse-input-data)\r\n* [预加载数据](#AUTOGENERATED-preloaded-data)\r\n* [多管线输入](#AUTOGENERATED-multiple-input-pipelines)\r\n\r\n\r\n\r\n\r\n## 供给数据 \r\n\r\nTensorFlow的数据供给机制允许你在TensorFlow运算图中将数据注入到任一张量中。因此，python运算可以把数据直接设置到TensorFlow图中。\r\n\r\n通过给run()或者eval()函数输入`feed_dict`参数， 可以启动运算过程。\r\n\r\n```python\r\nwith tf.Session():\r\n input = tf.placeholder(tf.float32)\r\n classifier = ...\r\n print classifier.eval(feed_dict={input: my_python_preprocessing_fn()})\r\n```\r\n\r\n虽然你可以使用常量和变量来替换任何一个张量， 但是最好的做法应该是使用[`placeholder` op](tensorflow-zh/SOURCE/api_docs/python/io_ops.md#placeholder)节点。设计`placeholder`节点的唯一的意图就是为了提供数据供给(feeding)的方法。`placeholder`节点被声明的时候是未初始化的， 也不包含数据， 如果没有为它供给数据， 则TensorFlow运算的时候会产生错误， 所以千万不要忘了为`placeholder`提供数据。\r\n\r\n可以在[`tensorflow/g3doc/tutorials/mnist/fully_connected_feed.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/tutorials/mnist/fully_connected_feed.py)找到使用`placeholder`和MNIST训练的例子，[MNIST tutorial](tensorflow-zh/SOURCE/tutorials/mnist/tf/index.md)也讲述了这一例子。\r\n\r\n## 从文件读取数据 \r\n\r\n一共典型的文件读取管线会包含下面这些步骤：\r\n\r\n1. 文件名列表\r\n2. *可配置的* 文件名乱序(shuffling)\r\n3. *可配置的* 最大训练迭代数(epoch limit)\r\n4. 文件名队列\r\n5. 针对输入文件格式的阅读器\r\n6. 纪录解析器\r\n7. *可配置的*预处理器\r\n8. 样本队列\r\n\r\n### 文件名, 乱序(shuffling), 和最大训练迭代数(epoch limits) \r\n\r\n可以使用字符串张量(比如`[\"file0\", \"file1\"]`, `[(\"file%d\" % i) for i in range(2)]`， `[(\"file%d\" % i) for i in range(2)]`) 或者[`tf.train.match_filenames_once` 函数](tensorflow-zh/SOURCE/api_docs/python/io_ops.md#match_filenames_once)来产生文件名列表。\r\n\r\n将文件名列表交给[`tf.train.string_input_producer` 函数](tensorflow-zh/SOURCE/api_docs/python/io_ops.md#string_input_producer).`string_input_producer`来生成一个先入先出的队列， 文件阅读器会需要它来读取数据。\r\n\r\n`string_input_producer` 提供的可配置参数来设置文件名乱序和最大的训练迭代数， `QueueRunner`会为每次迭代(epoch)将所有的文件名加入文件名队列中， 如果`shuffle=True`的话， 会对文件名进行乱序处理。这一过程是比较均匀的，因此它可以产生均衡的文件名队列。\r\n\r\n这个`QueueRunner`的工作线程是独立于文件阅读器的线程， 因此乱序和将文件名推入到文件名队列这些过程不会阻塞文件阅读器运行。\r\n\r\n### 文件格式 \r\n\r\n根据你的文件格式， 选择对应的文件阅读器， 然后将文件名队列提供给阅读器的`read`方法。阅读器的`read`方法会输出一个key来表征输入的文件和其中的纪录(对于调试非常有用)，同时得到一个字符串标量， 这个字符串标量可以被一个或多个解析器，或者转换操作将其解码为张量并且构造成为样本。\r\n\r\n#### CSV 文件 \r\n\r\n从CSV文件中读取数据， 需要使用[`TextLineReader`](tensorflow-zh/SOURCE/api_docs/python/io_ops.md#TextLineReader)和[`decode_csv`](tensorflow-zh/SOURCE/api_docs/python/io_ops.md#decode_csv) 操作， 如下面的例子所示：\r\n\r\n```python\r\nfilename_queue = tf.train.string_input_producer([\"file0.csv\", \"file1.csv\"])\r\n\r\nreader = tf.TextLineReader()\r\nkey, value = reader.read(filename_queue)\r\n\r\n# Default values, in case of empty columns. Also specifies the type of the\r\n# decoded result.\r\nrecord_defaults = [[1], [1], [1], [1], [1]]\r\ncol1, col2, col3, col4, col5 = tf.decode_csv(\r\n value, record_defaults=record_defaults)\r\nfeatures = tf.concat(0, [col1, col2, col3, col4])\r\n\r\nwith tf.Session() as sess:\r\n # Start populating the filename queue.\r\n coord = tf.train.Coordinator()\r\n threads = tf.train.start_queue_runners(coord=coord)\r\n\r\n for i in range(1200):\r\n # Retrieve a single instance:\r\n example, label = sess.run([features, col5])\r\n\r\n coord.request_stop()\r\n coord.join(threads)\r\n```\r\n每次`read`的执行都会从文件中读取一行内容， `decode_csv` 操作会解析这一行内容并将其转为张量列表。如果输入的参数有缺失，`record_default`参数可以根据张量的类型来设置默认值。\r\n\r\n在调用`run`或者`eval`去执行`read`之前， 你必须调用`tf.train.start_queue_runners`来将文件名填充到队列。否则`read`操作会被阻塞到文件名队列中有值为止。\r\n\r\n#### 固定长度的记录 \r\n\r\n从二进制文件中读取固定长度纪录， 可以使用[`tf.FixedLengthRecordReader`](tensorflow-zh/SOURCE/api_docs/python/io_ops.md#FixedLengthRecordReader)的[`tf.decode_raw`](tensorflow-zh/SOURCE/api_docs/python/io_ops.md#decode_raw)操作。`decode_raw`操作可以讲一个字符串转换为一个uint8的张量。\r\n\r\n举例来说，[the CIFAR-10 dataset](http://www.cs.toronto.edu/~kriz/cifar.html)的文件格式定义是：每条记录的长度都是固定的，一个字节的标签，后面是3072字节的图像数据。uint8的张量的标准操作就可以从中获取图像片并且根据需要进行重组。 例子代码可以在[`tensorflow/models/image/cifar10/cifar10_input.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/cifar10/cifar10_input.py)找到，具体讲述可参见[教程](tensorflow-zh/SOURCE/tutorials/deep_cnn/index.md#prepare-the-data).\r\n\r\n#### 标准TensorFlow格式 \r\n\r\n另一种保存记录的方法可以允许你讲任意的数据转换为TensorFlow所支持的格式， 这种方法可以使TensorFlow的数据集更容易与网络应用架构相匹配。这种建议的方法就是使用TFRecords文件，TFRecords文件包含了[`tf.train.Example` 协议内存块(protocol buffer)](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/example/example.proto)(协议内存块包含了字段\r\n[`Features`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/core/example/feature.proto))。你可以写一段代码获取你的数据， 将数据填入到`Example`协议内存块(protocol buffer)，将协议内存块序列化为一个字符串， 并且通过[`tf.python_io.TFRecordWriter` class](tensorflow-zh/SOURCE/api_docs/python/python_io.md#TFRecordWriter)写入到TFRecords文件。[`tensorflow/g3doc/how_tos/reading_data/convert_to_records.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/how_tos/reading_data/convert_to_records.py)就是这样的一个例子。\r\n\r\n从TFRecords文件中读取数据， 可以使用[`tf.TFRecordReader`](tensorflow-zh/SOURCE/api_docs/python/io_ops.md#TFRecordReader)的[`tf.parse_single_example`](tensorflow-zh/SOURCE/api_docs/python/io_ops.md#parse_single_example)解析器。这个`parse_single_example`操作可以将`Example`协议内存块(protocol buffer)解析为张量。 MNIST的例子就使用了`convert_to_records` 所构建的数据。 请参看[`tensorflow/g3doc/how_tos/reading_data/fully_connected_reader.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/how_tos/reading_data/fully_connected_reader.py), 您也可以将这个例子跟`fully_connected_feed`的版本加以比较。\r\n\r\n### 预处理 \r\n\r\n你可以对输入的样本进行任意的预处理， 这些预处理不依赖于训练参数， 你可以在[`tensorflow/models/image/cifar10/cifar10.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/cifar10/cifar10.py)找到数据归一化， 提取随机数据片，增加噪声或失真等等预处理的例子。\r\n\r\n### 批处理 \r\n\r\n在数据输入管线的末端， 我们需要有另一个队列来执行输入样本的训练，评价和推理。因此我们使用[`tf.train.shuffle_batch` 函数](tensorflow-zh/SOURCE/api_docs/python/io_ops.md#shuffle_batch)来对队列中的样本进行乱序处理\r\n\r\n示例:\r\n\r\n```\r\ndef read_my_file_format(filename_queue):\r\n reader = tf.SomeReader()\r\n key, record_string = reader.read(filename_queue)\r\n example, label = tf.some_decoder(record_string)\r\n processed_example = some_processing(example)\r\n return processed_example, label\r\n\r\ndef input_pipeline(filenames, batch_size, num_epochs=None):\r\n filename_queue = tf.train.string_input_producer(\r\n filenames, num_epochs=num_epochs, shuffle=True)\r\n example, label = read_my_file_format(filename_queue)\r\n # min_after_dequeue defines how big a buffer we will randomly sample\r\n # from -- bigger means better shuffling but slower start up and more\r\n # memory used.\r\n # capacity must be larger than min_after_dequeue and the amount larger\r\n # determines the maximum we will prefetch. Recommendation:\r\n # min_after_dequeue + (num_threads + a small safety margin) * batch_size\r\n min_after_dequeue = 10000\r\n capacity = min_after_dequeue + 3 * batch_size\r\n example_batch, label_batch = tf.train.shuffle_batch(\r\n [example, label], batch_size=batch_size, capacity=capacity,\r\n min_after_dequeue=min_after_dequeue)\r\n return example_batch, label_batch\r\n```\r\n\r\n如果你需要对不同文件中的样子有更强的乱序和并行处理，可以使用[`tf.train.shuffle_batch_join` 函数](tensorflow-zh/SOURCE/api_docs/python/io_ops.md#shuffle_batch_join).\r\n示例:\r\n\r\n```\r\ndef read_my_file_format(filename_queue):\r\n # Same as above\r\n\r\ndef input_pipeline(filenames, batch_size, read_threads, num_epochs=None):\r\n filename_queue = tf.train.string_input_producer(\r\n filenames, num_epochs=num_epochs, shuffle=True)\r\n example_list = [read_my_file_format(filename_queue)\r\n for _ in range(read_threads)]\r\n min_after_dequeue = 10000\r\n capacity = min_after_dequeue + 3 * batch_size\r\n example_batch, label_batch = tf.train.shuffle_batch_join(\r\n example_list, batch_size=batch_size, capacity=capacity,\r\n min_after_dequeue=min_after_dequeue)\r\n return example_batch, label_batch\r\n```\r\n\r\n在这个例子中， 你虽然只使用了一个文件名队列， 但是TensorFlow依然能保证多个文件阅读器从同一次迭代(epoch)的不同文件中读取数据，知道这次迭代的所有文件都被开始读取为止。（通常来说一个线程来对文件名队列进行填充的效率是足够的）\r\n\r\n另一种替代方案是： 使用[`tf.train.shuffle_batch` 函数](tensorflow-zh/SOURCE/api_docs/python/io_ops.md#shuffle_batch),设置`num_threads`的值大于1。 这种方案可以保证同一时刻只在一个文件中进行读取操作(但是读取速度依然优于单线程)，而不是之前的同时读取多个文件。这种方案的优点是：\r\n* 避免了两个不同的线程从同一个文件中读取同一个样本。\r\n* 避免了过多的磁盘搜索操作。\r\n\r\n你一共需要多少个读取线程呢？ 函数`tf.train.shuffle_batch*`为TensorFlow图提供了获取文件名队列中的元素个数之和的方法。 如果你有足够多的读取线程， 文件名队列中的元素个数之和应该一直是一个略高于0的数。具体可以参考[TensorBoard:可视化学习](tensorflow-zh/SOURCE/how_tos/summaries_and_tensorboard/index.md).\r\n\r\n### 创建线程并使用`QueueRunner`对象来预取 \r\n\r\n简单来说：使用上面列出的许多`tf.train`函数添加[`QueueRunner`](../../api_docs/python/train.md#QueueRunner)到你的数据流图中。在你运行任何训练步骤之前，需要调用[`tf.train.start_queue_runners`](../../api_docs/python/train.md#start_queue_runners)函数，否则数据流图将一直挂起。[`tf.train.start_queue_runners`](../../api_docs/python/train.md#start_queue_runners) 这个函数将会启动输入管道的线程，填充样本到队列中，以便出队操作可以从队列中拿到样本。这种情况下最好配合使用一个[`tf.train.Coordinator`](../../api_docs/python/train.md#Coordinator)，这样可以在发生错误的情况下正确地关闭这些线程。如果你对训练迭代数做了限制，那么需要使用一个训练迭代数计数器，并且需要被初始化。推荐的代码模板如下：\r\n\r\n```python\r\n# Create the graph, etc.\r\ninit_op = tf.initialize_all_variables()\r\n\r\n# Create a session for running operations in the Graph.\r\nsess = tf.Session()\r\n\r\n# Initialize the variables (like the epoch counter).\r\nsess.run(init_op)\r\n\r\n# Start input enqueue threads.\r\ncoord = tf.train.Coordinator()\r\nthreads = tf.train.start_queue_runners(sess=sess, coord=coord)\r\n\r\ntry:\r\n while not coord.should_stop():\r\n # Run training steps or whatever\r\n sess.run(train_op)\r\n\r\nexcept tf.errors.OutOfRangeError:\r\n print 'Done training -- epoch limit reached'\r\nfinally:\r\n # When done, ask the threads to stop.\r\n coord.request_stop()\r\n\r\n# Wait for threads to finish.\r\ncoord.join(threads)\r\nsess.close()\r\n```\r\n\r\n#### 疑问: 这是怎么回事? \r\n\r\n首先，我们先创建数据流图，这个数据流图由一些流水线的阶段组成，阶段间用队列连接在一起。第一阶段将生成文件名，我们读取这些文件名并且把他们排到文件名队列中。第二阶段从文件中读取数据（使用`Reader`），产生样本，而且把样本放在一个样本队列中。根据你的设置，实际上也可以拷贝第二阶段的样本，使得他们相互独立，这样就可以从多个文件中并行读取。在第二阶段的最后是一个排队操作，就是入队到队列中去，在下一阶段出队。因为我们是要开始运行这些入队操作的线程，所以我们的训练循环会使得样本队列中的样本不断地出队。\r\n\r\n\r\n\r\n![](\"../images/AnimatedFileQueues.gif\")
\r\n
\r\n\r\n在`tf.train`中要创建这些队列和执行入队操作，就要添加[`tf.train.QueueRunner`](../../api_docs/python/train.md#QueueRunner)到一个使用[`tf.train.add_queue_runner`](../../api_docs/python/train.md#add_queue_runner)函数的数据流图中。每个`QueueRunner`负责一个阶段，处理那些需要在线程中运行的入队操作的列表。一旦数据流图构造成功，[`tf.train.start_queue_runners`](../../api_docs/python/train.md#start_queue_runners)函数就会要求数据流图中每个`QueueRunner`去开始它的线程运行入队操作。\r\n\r\n如果一切顺利的话，你现在可以执行你的训练步骤，同时队列也会被后台线程来填充。如果您设置了最大训练迭代数，在某些时候，样本出队的操作可能会得到一个[`tf.OutOfRangeError`](../../api_docs/python/client.md#OutOfRangeError)的错误。这其实是TensorFlow的“文件结束”（EOF） ———— 这就意味着已经达到了最大训练迭代数，已经没有更多可用的样本了。\r\n\r\n最后一个因素是[`Coordinator`](../../api_docs/python/train.md#Coordinator)。这是负责在收到任何关闭信号的时候，让所有的线程都知道。最常用的是在发生异常时这种情况就会呈现出来，比如说其中一个线程在运行某些操作时出现错误（或一个普通的Python异常）。\r\n\r\n想要了解更多的关于threading, queues, QueueRunners, and Coordinators的内容可以[看这里](../../how_tos/threading_and_queues/index.md).\r\n\r\n#### 疑问: 在达到最大训练迭代数的时候如何清理关闭线程? \r\n\r\n想象一下，你有一个模型并且设置了最大训练迭代数。这意味着，生成文件的那个线程将只会在产生`OutOfRange`错误之前运行许多次。该`QueueRunner`会捕获该错误，并且关闭文件名的队列，最后退出线程。关闭队列做了两件事情：\r\n\r\n* 如果还试着对文件名队列执行入队操作时将发生错误。任何线程不应该尝试去这样做，但是当队列因为其他错误而关闭时，这就会有用了。\r\n* 任何当前或将来出队操作要么成功（如果队列中还有足够的元素）或立即失败（发生`OutOfRange`错误）。它们不会防止等待更多的元素被添加到队列中，因为上面的一点已经保证了这种情况不会发生。\r\n\r\n关键是，当在文件名队列被关闭时候，有可能还有许多文件名在该队列中，这样下一阶段的流水线（包括reader和其它预处理）还可以继续运行一段时间。 一旦文件名队列空了之后，如果后面的流水线还要尝试从文件名队列中取出一个文件名（例如，从一个已经处理完文件的reader中），这将会触发`OutOfRange`错误。在这种情况下，即使你可能有一个QueueRunner关联着多个线程。如果这不是在QueueRunner中的最后那个线程，`OutOfRange`错误仅仅只会使得一个线程退出。这使得其他那些正处理自己的最后一个文件的线程继续运行，直至他们完成为止。 （但如果假设你使用的是[`tf.train.Coordinator`](../../api_docs/python/train.md#Coordinator)，其他类型的错误将导致所有线程停止）。一旦所有的reader线程触发`OutOfRange`错误，然后才是下一个队列，再是样本队列被关闭。\r\n\r\n同样，样本队列中会有一些已经入队的元素，所以样本训练将一直持续直到样本队列中再没有样本为止。如果样本队列是一个[`RandomShuffleQueue`](../../api_docs/python/io_ops.md#RandomShuffleQueue)，因为你使用了`shuffle_batch` 或者 `shuffle_batch_join`，所以通常不会出现以往那种队列中的元素会比`min_after_dequeue` 定义的更少的情况。 然而，一旦该队列被关闭，`min_after_dequeue`设置的限定值将失效，最终队列将为空。在这一点来说，当实际训练线程尝试从样本队列中取出数据时，将会触发`OutOfRange`错误，然后训练线程会退出。一旦所有的培训线程完成，[`tf.train.Coordinator.join`](../../api_docs/python/train.md#Coordinator.join)会返回，你就可以正常退出了。\r\n\r\n### 筛选记录或产生每个记录的多个样本 \r\n\r\n举个例子，有形式为`[x, y, z]`的样本，我们可以生成一批形式为`[batch, x, y, z]`的样本。 如果你想滤除这个记录（或许不需要这样的设置），那么可以设置batch的大小为0；但如果你需要每个记录产生多个样本，那么batch的值可以大于1。 然后很简单，只需调用批处理函数（比如： `shuffle_batch` or `shuffle_batch_join`）去设置`enqueue_many=True`就可以实现。\r\n\r\n### 稀疏输入数据 \r\n\r\nSparseTensors这种数据类型使用队列来处理不是太好。如果要使用SparseTensors你就必须在批处理**之后**使用[`tf.parse_example`](../../api_docs/python/io_ops.md#parse_example) 去解析字符串记录 (而不是在批处理**之前**使用 `tf.parse_single_example`) 。\r\n\r\n## 预取数据 \r\n\r\n这仅用于可以完全加载到存储器中的小的数据集。有两种方法：\r\n\r\n* 存储在常数中。\r\n* 存储在变量中，初始化后，永远不要改变它的值。\r\n\r\n使用常数更简单一些，但是会使用更多的内存（因为常数会内联的存储在数据流图数据结构中，这个结构体可能会被复制几次）。\r\n\r\n```python\r\ntraining_data = ...\r\ntraining_labels = ...\r\nwith tf.Session():\r\n input_data = tf.constant(training_data)\r\n input_labels = tf.constant(training_labels)\r\n ...\r\n```\r\n\r\n要改为使用变量的方式，您就需要在数据流图建立后初始化这个变量。\r\n\r\n```python\r\ntraining_data = ...\r\ntraining_labels = ...\r\nwith tf.Session() as sess:\r\n data_initializer = tf.placeholder(dtype=training_data.dtype,\r\n shape=training_data.shape)\r\n label_initializer = tf.placeholder(dtype=training_labels.dtype,\r\n shape=training_labels.shape)\r\n input_data = tf.Variable(data_initalizer, trainable=False, collections=[])\r\n input_labels = tf.Variable(label_initalizer, trainable=False, collections=[])\r\n ...\r\n sess.run(input_data.initializer,\r\n feed_dict={data_initializer: training_data})\r\n sess.run(input_labels.initializer,\r\n feed_dict={label_initializer: training_lables})\r\n```\r\n\r\n设定`trainable=False` 可以防止该变量被数据流图的 `GraphKeys.TRAINABLE_VARIABLES` 收集, 这样我们就不会在训练的时候尝试更新它的值； 设定 `collections=[]` 可以防止`GraphKeys.VARIABLES` 收集后做为保存和恢复的中断点。\r\n\r\n无论哪种方式，[`tf.train.slice_input_producer function`](../../api_docs/python/io_ops.md#slice_input_producer)函数可以被用来每次产生一个切片。这样就会让样本在整个迭代中被打乱，所以在使用批处理的时候不需要再次打乱样本。所以我们不使用`shuffle_batch`函数，取而代之的是纯[`tf.train.batch` 函数](../../api_docs/python/io_ops.md#batch)。 如果要使用多个线程进行预处理，需要将`num_threads`参数设置为大于1的数字。\r\n\r\n在[`tensorflow/g3doc/how_tos/reading_data/fully_connected_preloaded.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/how_tos/reading_data/fully_connected_preloaded.py) 中可以找到一个MNIST例子，使用常数来预加载。 另外使用变量来预加载的例子在[`tensorflow/g3doc/how_tos/reading_data/fully_connected_preloaded_var.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/how_tos/reading_data/fully_connected_preloaded_var.py)，你可以用上面 `fully_connected_feed` 和 `fully_connected_reader` 的描述来进行比较。\r\n\r\n## 多输入管道 \r\n\r\n通常你会在一个数据集上面训练，然后在另外一个数据集上做评估计算(或称为 \"eval\")。 这样做的一种方法是，实际上包含两个独立的进程：\r\n\r\n* 训练过程中读取输入数据，并定期将所有的训练的变量写入还原点文件）。\r\n* 在计算过程中恢复还原点文件到一个推理模型中，读取有效的输入数据。\r\n\r\n这两个进程在下面的例子中已经完成了：[the example CIFAR-10 model](../../tutorials/deep_cnn/index.md#save-and-restore-checkpoints)，有以下几个好处：\r\n\r\n* eval被当做训练后变量的一个简单映射。\r\n* 你甚至可以在训练完成和退出后执行eval。\r\n\r\n您可以在同一个进程的相同的数据流图中有训练和eval，并分享他们的训练后的变量。参考[the shared variables tutorial](../../how_tos/variable_scope/index.md).\r\n\r\n原文地址：[Reading data](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/how_tos/reading_data/index.md) 翻译：[volvet](https://github.com/volvet) and [zhangkom](https://github.com/zhangkom) 校对：\r\n"
},
{
"path": "SOURCE/how_tos/summaries_and_tensorboard/index.md",
"content": "# TensorBoard: Visualizing Learning \r\n\r\nThe computations you'll use TensorBoard for - like training a massive\r\ndeep neural network - can be complex and confusing. To make it easier to\r\nunderstand, debug, and optimize TensorFlow programs, we've included a suite of\r\nvisualization tools called TensorBoard. You can use TensorBoard to visualize\r\nyour TensorFlow graph, plot quantitative metrics about the execution of your\r\ngraph, and show additional data like images that pass through it. When\r\nTensorBoard is fully configured, it looks like this:\r\n\r\n![MNIST TensorBoard](./mnist_tensorboard.png \"MNIST TensorBoard\")\r\n\r\n\r\n## Serializing the data \r\n\r\nTensorBoard operates by reading TensorFlow events files, which contain summary\r\ndata that you can generate when running TensorFlow. Here's the general\r\nlifecycle for summary data within TensorBoard.\r\n\r\nFirst, create the TensorFlow graph that you'd like to collect summary\r\ndata from, and decide which nodes you would like to annotate with\r\n[summary operations]\r\n(../../api_docs/python/train.md#summary-operations).\r\n\r\nFor example, suppose you are training a convolutional neural network for\r\nrecognizing MNIST digits. You'd like to record how the learning rate\r\nvaries over time, and how the objective function is changing. Collect these by\r\nattaching [`scalar_summary`](../../api_docs/python/train.md#scalar_summary) ops\r\nto the nodes that output the learning rate and loss respectively. Then, give\r\neach `scalar_summary` a meaningful `tag`, like `'learning rate'` or `'loss\r\nfunction'`.\r\n\r\nPerhaps you'd also like to visualize the distributions of activations coming\r\noff a particular layer, or the distribution of gradients or weights. Collect\r\nthis data by attaching\r\n[`histogram_summary`](../../api_docs/python/train.md#histogram_summary) ops to\r\nthe gradient outputs and to the variable that holds your weights, respectively.\r\n\r\nFor details on all of the summary operations avaiable, check out the docs on\r\n[summary operations]\r\n(../../api_docs/python/train.md#summary-operations).\r\n\r\nOperations in TensorFlow don't do anything until you run them, or an op that\r\ndepends on their output. And the summary nodes that we've just created are\r\nperipheral to your graph: none of the ops you are currently running depend on\r\nthem. So, to generate summaries, we need to run all of these summary nodes.\r\nManaging them by hand would be tedious, so use\r\n[`tf.merge_all_summaries`](../../api_docs/python/train.md#merge_all_summaries)\r\nto combine them into a single op that generates all the summary data.\r\n\r\nThen, you can just run the merged summary op, which will generate a serialized\r\n`Summary` protobuf object with all of your summary data at a given step.\r\nFinally, to write this summary data to disk, pass the summary protobuf to a\r\n[`tf.train.SummaryWriter`](../../api_docs/python/train.md#SummaryWriter).\r\n\r\nThe `SummaryWriter` takes a logdir in its constructor - this logdir is quite\r\nimportant, it's the directory where all of the events will be written out.\r\nAlso, the `SummaryWriter` can optionally take a `GraphDef` in its constructor.\r\nIf it receives one, then TensorBoard will visualize your graph as well.\r\n\r\nNow that you've modified your graph and have a `SummaryWriter`, you're ready to\r\nstart runing your network! If you want, you could run the merged summary op\r\nevery single step, and record a ton of training data. That's likely to be more\r\ndata than you need, though. Instead, consider running the merged summary op\r\nevery hundred steps or so, as in the following code example.\r\n\r\n```python\r\nmerged_summary_op = tf.merge_all_summaries()\r\nsummary_writer = tf.train.SummaryWriter('/tmp/mnist_logs', sess.graph)\r\ntotal_step = 0\r\nwhile training:\r\n total_step += 1\r\n session.run(training_op)\r\n if total_step % 100 == 0:\r\n summary_str = session.run(merged_summary_op)\r\n summary_writer.add_summary(summary_str, total_step)\r\n```\r\n\r\nYou're now all set to visualize this data using TensorBoard.\r\n\r\n\r\n## Launching TensorBoard \r\n\r\nTo run TensorBoard, use the command\r\n\r\n python tensorflow/tensorboard/tensorboard.py --logdir=path/to/log-directory\r\n\r\nwhere `logdir` points to the directory where the `SummaryWriter` serialized its\r\ndata. If this `logdir` directory contains subdirectories which contain\r\nserialized data from separate runs, then TensorBoard will visualize the data\r\nfrom all of those runs. Once TensorBoard is running, navigate your web browser\r\nto `localhost:6006` to view the TensorBoard.\r\n\r\nIf you have pip installed TensorBoard, you can use the simpler command\r\n\r\n tensorboard --logdir=/path/to/log-directory\r\n\r\nWhen looking at TensorBoard, you will see the navigation tabs in the top right\r\ncorner. Each tab represents a set of serialized data that can be visualized.\r\nFor any tab you are looking at, if the logs being looked at by TensorBoard do\r\nnot contain any data relevant to that tab, a message will be displayed\r\nindicating how to serialize data that is applicable to that tab.\r\n\r\nFor in depth information on how to use the *graph* tab to visualize your graph,\r\nsee [TensorBoard: Visualizing your graph](../../how_tos/graph_viz/index.md).\r\n"
},
{
"path": "SOURCE/how_tos/summaries_and_tensorboard.md",
"content": "# TensorBoard:可视化学习 \r\n\r\nTensorBoard 涉及到的运算，通常是在训练庞大的深度神经网络中出现的复杂而又难以理解的运算。\r\n\r\n为了更方便 TensorFlow 程序的理解、调试与优化，我们发布了一套叫做 TensorBoard 的可视化工具。你可以用 TensorBoard 来展现你的 TensorFlow 图像，绘制图像生成的定量指标图以及附加数据。\r\n\r\n当 TensorBoard 设置完成后，它应该是这样子的：\r\n\r\n\r\n\r\n## 数据序列化 \r\n\r\nTensorBoard 通过读取 TensorFlow 的事件文件来运行。TensorFlow 的事件文件包括了你会在 TensorFlow 运行中涉及到的主要数据。下面是 TensorBoard 中汇总数据（Summary data）的大体生命周期。\r\n\r\n首先，创建你想汇总数据的 TensorFlow 图，然后再选择你想在哪个节点进行[汇总(summary)操作](../api_docs/python/train.md#summary_options)。\r\n\r\n比如，假设你正在训练一个卷积神经网络，用于识别 MNISt 标签。你可能希望记录学习速度(learning rate)的如何变化，以及目标函数如何变化。通过向节点附加[scalar_summary](../api_docs/python/train.md#scalary_summary)操作来分别输出学习速度和期望误差。然后你可以给每个 scalary_summary 分配一个有意义的 `标签`，比如 `'learning rate'` 和 `'loss function'`。\r\n\r\n或者你还希望显示一个特殊层中激活的分布，或者梯度权重的分布。可以通过分别附加 [histogram_summary](../api_docs/python/train.md#histogram_summary) 运算来收集权重变量和梯度输出。\r\n\r\n所有可用的 summary 操作详细信息，可以查看[summary_operation](../api_docs/python/train.md#summary_operation)文档。\r\n\r\n在TensorFlow中，所有的操作只有当你执行，或者另一个操作依赖于它的输出时才会运行。我们刚才创建的这些节点（summary nodes）都围绕着你的图像：没有任何操作依赖于它们的结果。因此，为了生成汇总信息，我们需要运行所有这些节点。这样的手动工作是很乏味的，因此可以使用[tf.merge_all_summaries](../api_docs/python/train.md#scalary_summary)来将他们合并为一个操作。\r\n\r\n然后你可以执行合并命令，它会依据特点步骤将所有数据生成一个序列化的`Summary` protobuf对象。最后，为了将汇总数据写入磁盘，需要将汇总的protobuf对象传递给[tf.train.Summarywriter](../api_docs/python/train.md#SummaryWriter)。\r\n\r\n`SummaryWriter` 的构造函数中包含了参数 logdir。这个 logdir 非常重要，所有事件都会写到它所指的目录下。此外，`SummaryWriter` 中还包含了一个可选择的参数 `GraphDef`。如果输入了该参数，那么 TensorBoard 也会显示你的图像。\r\n\r\n现在已经修改了你的图，也有了 `SummaryWriter`，现在就可以运行你的神经网络了！如果你愿意的话，你可以每一步执行一次合并汇总，这样你会得到一大堆训练数据。这很有可能超过了你想要的数据量。你也可以每一百步执行一次合并汇总，或者如下面代码里示范的这样。\r\n\r\n```python\r\nmerged_summary_op = tf.merge_all_summaries()\r\nsummary_writer = tf.train.SummaryWriter('/tmp/mnist_logs', sess.graph)\r\ntotal_step = 0\r\nwhile training:\r\n total_step += 1\r\n session.run(training_op)\r\n if total_step % 100 == 0:\r\n summary_str = session.run(merged_summary_op)\r\n summary_writer.add_summary(summary_str, total_step)\r\n```\r\n\r\n现在已经准备好用 TensorBoard 来可视化这些数据了。\r\n\r\n## 启动TensorBoard \r\n\r\n输入下面的指令来启动TensorBoard\r\n\r\n```\r\npython tensorflow/tensorboard/tensorboard.py --logdir=path/to/log-directory\r\n```\r\n\r\n这里的参数 `logdir` 指向 `SummaryWriter` 序列化数据的存储路径。如果`logdir`目录的子目录中包含另一次运行时的数据，那么 TensorBoard 会展示所有运行的数据。一旦 TensorBoard 开始运行，你可以通过在浏览器中输入 `localhost:6006` 来查看 TensorBoard。\r\n\r\n如果你已经通过pip安装了 TensorBoard，你可以通过执行更为简单地命令来访问 TensorBoard\r\n\r\n```\r\ntensorboard --logdir=/path/to/log-directory\r\n```\r\n\r\n进入 TensorBoard 的界面时，你会在右上角看到导航选项卡，每一个选项卡将展现一组可视化的序列化数据集 。对于你查看的每一个选项卡，如果 TensorBoard 中没有数据与这个选项卡相关的话，则会显示一条提示信息指示你如何序列化相关数据。\r\n\r\n更多更详细的关于如何使用 graph 选项来显示你的图像的信息。参见 [TensorBoard:图表可视化](./graph_viz.md)\r\n\r\n原文地址：[TensorBoard:Visualizing Learning](http://tensorflow.org/how_tos/summaries_and_tensorboard/index.html#tensorboard-visualizing-learning) 翻译：[thylaco1eo](https://github.com/thylaco1eo) 校对：[lucky521](https://github.com/lucky521)\r\n"
},
{
"path": "SOURCE/how_tos/threading_and_queues/index.md",
"content": "# Threading and Queues \r\n\r\nQueues are a powerful mechanism for asynchronous computation using TensorFlow.\r\n\r\nLike everything in TensorFlow, a queue is a node in a TensorFlow graph. It's a\r\nstateful node, like variable: other nodes can modify its content. In\r\nparticular, nodes can enqueue new items in to the queue, or dequeue existing\r\nitems from the queue.\r\n\r\nTo get a feel for queues, let's consider a simple example. We will create a\r\n\"first in, first out\" queue (`FIFOQueue`) and fill it with zeros.\r\nThen we'll construct a graph\r\nthat takes an item off the queue, adds one to that item, and puts it back on the\r\nend of the queue. Slowly, the numbers on the queue increase.\r\n\r\n\r\n\r\n![](\"threading_and_queues/IncremeterFifoQueue.gif\")
\r\n
\r\n\r\n`Enqueue`, `EnqueueMany`, and `Dequeue` are special nodes. They take a pointer\r\nto the queue instead of a normal value, allowing them to change it. We recommend\r\nyou think of these as being like methods of the queue. In fact, in the Python\r\nAPI, they are methods of the queue object (eg. `q.enqueue(...)`).\r\n\r\nNow that you have a bit of a feel for queues, let's dive into the details...\r\n\r\n## Queue Use Overview \r\n\r\nQueues, such as `FIFOQueue` and `RandomShuffleQueue`, are important TensorFlow\r\nobjects for computing tensors asynchronously in a graph.\r\n\r\nFor example, a typical input architecture is to use a `RandomShuffleQueue` to\r\nprepare inputs for training a model:\r\n\r\n* Multiple threads prepare training examples and push them in the queue.\r\n* A training thread executes a training op that dequeues mini-batches from the\r\n queue\r\n\r\nThis architecture has many benefits, as highlighted in the\r\n[Reading data how to](../reading_data), which also gives an overview of\r\nfunctions that simplify the construction of input pipelines.\r\n\r\nThe TensorFlow `Session` object is multithreaded, so multiple threads can\r\neasily use the same session and run ops in parallel. However, it is not always\r\neasy to implement a Python program that drives threads as described above. All\r\nthreads must be able to stop together, exceptions must be caught and\r\nreported, and queues must be properly closed when stopping.\r\n\r\nTensorFlow provides two classes to help:\r\n[tf.Coordinator](../../api_docs/python/train.md#Coordinator) and\r\n[tf.QueueRunner](../../api_docs/python/train.md#QueueRunner). These two classes\r\nare designed to be used together. The `Coordinator` class helps multiple threads\r\nstop together and report exceptions to a program that waits for them to stop.\r\nThe `QueueRunner` class is used to create a number of threads cooperating to\r\nenqueue tensors in the same queue.\r\n\r\n## Coordinator \r\n\r\nThe Coordinator class helps multiple threads stop together.\r\n\r\nIts key methods are:\r\n\r\n* `should_stop()`: returns True if the threads should stop.\r\n* `request_stop(\r\n- )`: waits until the specified threads have stopped.\r\n\r\nYou first create a `Coordinator` object, and then create a number of threads\r\nthat use the coordinator. The threads typically run loops that stop when\r\n`should_stop()` returns `True`.\r\n\r\nAny thread can decide that the computation should stop. It only has to call\r\n`request_stop()` and the other threads will stop as `should_stop()` will then\r\nreturn `True`.\r\n\r\n```python\r\n# Thread body: loop until the coordinator indicates a stop was requested.\r\n# If some condition becomes true, ask the coordinator to stop.\r\ndef MyLoop(coord):\r\n while not coord.should_stop():\r\n ...do something...\r\n if ...some condition...:\r\n coord.request_stop()\r\n\r\n# Main code: create a coordinator.\r\ncoord = Coordinator()\r\n\r\n# Create 10 threads that run 'MyLoop()'\r\nthreads = [threading.Thread(target=MyLoop, args=(coord)) for i in xrange(10)]\r\n\r\n# Start the threads and wait for all of them to stop.\r\nfor t in threads: t.start()\r\ncoord.join(threads)\r\n```\r\n\r\nObviously, the coordinator can manage threads doing very different things.\r\nThey don't have to be all the same as in the example above. The coordinator\r\nalso has support to capture and report exceptions. See the [Coordinator class](../../api_docs/python/train.md#Coordinator) documentation for more details.\r\n\r\n## QueueRunner \r\n\r\nThe `QueueRunner` class creates a number of threads that repeatedly run an\r\nenqueue op. These threads can use a coordinator to stop together. In\r\naddition, a queue runner runs a *closer thread* that automatically closes the\r\nqueue if an exception is reported to the coordinator.\r\n\r\nYou can use a queue runner to implement the architecture described above.\r\n\r\nFirst build a graph that uses a `Queue` for input examples. Add ops that\r\nprocess examples and enqueue them in the queue. Add training ops that start by\r\ndequeueing from the queue.\r\n\r\n```python\r\nexample = ...ops to create one example...\r\n# Create a queue, and an op that enqueues examples one at a time in the queue.\r\nqueue = tf.RandomShuffleQueue(...)\r\nenqueue_op = queue.enqueue(example)\r\n# Create a training graph that starts by dequeuing a batch of examples.\r\ninputs = queue.dequeue_many(batch_size)\r\ntrain_op = ...use 'inputs' to build the training part of the graph...\r\n```\r\n\r\nIn the Python training program, create a `QueueRunner` that will run a few\r\nthreads to process and enqueue examples. Create a `Coordinator` and ask the\r\nqueue runner to start its threads with the coordinator. Write a training loop\r\nthat also uses the coordinator.\r\n\r\n```\r\n# Create a queue runner that will run 4 threads in parallel to enqueue\r\n# examples.\r\nqr = tf.train.QueueRunner(queue, [enqueue_op] * 4)\r\n\r\n# Launch the graph.\r\nsess = tf.Session()\r\n# Create a coordinator, launch the queue runner threads.\r\ncoord = tf.train.Coordinator()\r\nenqueue_threads = qr.create_threads(sess, coord=coord, start=True)\r\n# Run the training loop, controlling termination with the coordinator.\r\nfor step in xrange(1000000):\r\n if coord.should_stop():\r\n break\r\n sess.run(train_op)\r\n# When done, ask the threads to stop.\r\ncoord.request_stop()\r\n# And wait for them to actually do it.\r\ncoord.join(threads)\r\n```\r\n\r\n## Handling Exceptions \r\n\r\nThreads started by queue runners do more than just run the enqueue ops. They\r\nalso catch and handle exceptions generated by queues, including\r\n`OutOfRangeError` which is used to report that a queue was closed.\r\n\r\nA training program that uses a coordinator must similarly catch and report\r\nexceptions in its main loop.\r\n\r\nHere is an improved version of the training loop above.\r\n\r\n```python\r\ntry:\r\n for step in xrange(1000000):\r\n if coord.should_stop():\r\n break\r\n sess.run(train_op)\r\nexcept Exception, e:\r\n # Report exceptions to the coordinator.\r\n coord.request_stop(e)\r\n\r\n# Terminate as usual. It is innocuous to request stop twice.\r\ncoord.request_stop()\r\ncoord.join(threads)\r\n```\r\n\r\n原文地址：([Threading and Queues](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/how_tos/threading_and_queues/index.md)) 翻译：([zhangkom](https://github.com/zhangkom)) 校对：\r\n"
},
{
"path": "SOURCE/how_tos/threading_and_queues.md",
"content": "# 线程和队列 \r\n\r\n在使用TensorFlow进行异步计算时，队列是一种强大的机制。\r\n\r\n正如TensorFlow中的其他组件一样，队列就是TensorFlow图中的节点。这是一种有状态的节点，就像变量一样：其他节点可以修改它的内容。具体来说，其他节点可以把新元素插入到队列后端(rear)，也可以把队列前端(front)的元素删除。\r\n\r\n为了感受一下队列，让我们来看一个简单的例子。我们先创建一个“先入先出”的队列（FIFOQueue），并将其内部所有元素初始化为零。然后，我们构建一个TensorFlow图，它从队列前端取走一个元素，加上1之后，放回队列的后端。慢慢地，队列的元素的值就会增加。\r\n\r\n
\r\n![](\"../images/IncremeterFifoQueue.gif\")
\r\n
\r\n`Enqueue`、 `EnqueueMany`和`Dequeue`都是特殊的节点。他们需要获取队列指针，而非普通的值，如此才能修改队列内容。我们建议您将它们看作队列的方法。事实上，在Python API中，它们就是队列对象的方法（例如`q.enqueue(...)`）。\r\n\r\n现在你已经对队列有了一定的了解，让我们深入到细节...\r\n\r\n## 队列使用概述 \r\n\r\n队列，如`FIFOQueue`和`RandomShuffleQueue`，在TensorFlow的张量异步计算时都非常重要。\r\n\r\n例如，一个典型的输入结构：是使用一个`RandomShuffleQueue`来作为模型训练的输入：\r\n\r\n* 多个线程准备训练样本，并且把这些样本推入队列。\r\n* 一个训练线程执行一个训练操作，此操作会从队列中移除最小批次的样本（mini-batches)。\r\n\r\n这种结构具有许多优点，正如在[Reading data how to](../reading_data)中强调的，同时，[Reading data how to](../reading_data)也概括地描述了如何简化输入管道的构造过程。\r\n\r\nTensorFlow的`Session`对象是可以支持多线程的，因此多个线程可以很方便地使用同一个会话（Session）并且并行地执行操作。然而，在Python程序实现这样的并行运算却并不容易。所有线程都必须能被同步终止，异常必须能被正确捕获并报告，回话终止的时候， 队列必须能被正确地关闭。\r\n\r\n所幸TensorFlow提供了两个类来帮助多线程的实现：[tf.Coordinator](tensorflow-zh/SOURCE/api_docs/python/train.md#Coordinator)和\r\n[tf.QueueRunner](tensorflow-zh/SOURCE/api_docs/python/train.md#QueueRunner)。从设计上这两个类必须被一起使用。`Coordinator`类可以用来同时停止多个工作线程并且向那个在等待所有工作线程终止的程序报告异常。`QueueRunner`类用来协调多个工作线程同时将多个张量推入同一个队列中。\r\n\r\n## Coordinator \r\n\r\nCoordinator类用来帮助多个线程协同工作，多个线程同步终止。\r\n其主要方法有：\r\n\r\n* `should_stop()`:如果线程应该停止则返回True。\r\n* `request_stop(\r\n- )`:等待被指定的线程终止。\r\n\r\n首先创建一个`Coordinator`对象，然后建立一些使用`Coordinator`对象的线程。这些线程通常一直循环运行，一直到`should_stop()`返回True时停止。\r\n任何线程都可以决定计算什么时候应该停止。它只需要调用`request_stop()`，同时其他线程的`should_stop()`将会返回`True`，然后都停下来。\r\n\r\n```python\r\n# 线程体：循环执行，直到`Coordinator`收到了停止请求。\r\n# 如果某些条件为真，请求`Coordinator`去停止其他线程。\r\ndef MyLoop(coord):\r\n while not coord.should_stop():\r\n ...do something...\r\n if ...some condition...:\r\n coord.request_stop()\r\n\r\n# Main code: create a coordinator.\r\ncoord = Coordinator()\r\n\r\n# Create 10 threads that run 'MyLoop()'\r\nthreads = [threading.Thread(target=MyLoop, args=(coord)) for i in xrange(10)]\r\n\r\n# Start the threads and wait for all of them to stop.\r\nfor t in threads: t.start()\r\ncoord.join(threads)\r\n```\r\n\r\n显然，Coordinator可以管理线程去做不同的事情。上面的代码只是一个简单的例子，在设计实现的时候不必完全照搬。Coordinator还支持捕捉和报告异常, 具体可以参考[Coordinator class](tensorflow-zh/SOURCE/api_docs/python/train.md#Coordinator)的文档。\r\n\r\n## QueueRunner \r\n\r\n`QueueRunner`类会创建一组线程， 这些线程可以重复的执行Enquene操作， 他们使用同一个Coordinator来处理线程同步终止。此外，一个QueueRunner会运行一个*closer thread*，当Coordinator收到异常报告时，这个*closer thread*会自动关闭队列。\r\n\r\n您可以使用一个queue runner，来实现上述结构。\r\n首先建立一个TensorFlow图表，这个图表使用队列来输入样本。增加处理样本并将样本推入队列中的操作。增加training操作来移除队列中的样本。\r\n\r\n```python\r\nexample = ...ops to create one example...\r\n# Create a queue, and an op that enqueues examples one at a time in the queue.\r\nqueue = tf.RandomShuffleQueue(...)\r\nenqueue_op = queue.enqueue(example)\r\n# Create a training graph that starts by dequeuing a batch of examples.\r\ninputs = queue.dequeue_many(batch_size)\r\ntrain_op = ...use 'inputs' to build the training part of the graph...\r\n```\r\n\r\n在Python的训练程序中，创建一个`QueueRunner`来运行几个线程， 这几个线程处理样本，并且将样本推入队列。创建一个`Coordinator`，让queue runner使用`Coordinator`来启动这些线程，创建一个训练的循环， 并且使用`Coordinator`来控制`QueueRunner`的线程们的终止。\r\n\r\n```\r\n# Create a queue runner that will run 4 threads in parallel to enqueue\r\n# examples.\r\nqr = tf.train.QueueRunner(queue, [enqueue_op] * 4)\r\n\r\n# Launch the graph.\r\nsess = tf.Session()\r\n# Create a coordinator, launch the queue runner threads.\r\ncoord = tf.train.Coordinator()\r\nenqueue_threads = qr.create_threads(sess, coord=coord, start=True)\r\n# Run the training loop, controlling termination with the coordinator.\r\nfor step in xrange(1000000):\r\n if coord.should_stop():\r\n break\r\n sess.run(train_op)\r\n# When done, ask the threads to stop.\r\ncoord.request_stop()\r\n# And wait for them to actually do it.\r\ncoord.join(threads)\r\n```\r\n\r\n## 异常处理 \r\n\r\n通过queue runners启动的线程不仅仅只处理推送样本到队列。他们还捕捉和处理由队列产生的异常，包括`OutOfRangeError`异常，这个异常是用于报告队列被关闭。\r\n使用`Coordinator`的训练程序在主循环中必须同时捕捉和报告异常。\r\n下面是对上面训练循环的改进版本。\r\n\r\n```python\r\ntry:\r\n for step in xrange(1000000):\r\n if coord.should_stop():\r\n break\r\n sess.run(train_op)\r\nexcept Exception, e:\r\n # Report exceptions to the coordinator.\r\n coord.request_stop(e)\r\n\r\n# Terminate as usual. It is innocuous to request stop twice.\r\ncoord.request_stop()\r\ncoord.join(threads)\r\n```\r\n\r\n原文地址：[Threading and Queues](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/how_tos/threading_and_queues/index.md) 翻译：[zhangkom](https://github.com/zhangkom) 校对：[volvet](https://github.com/volvet)\r\n"
},
{
"path": "SOURCE/how_tos/using_gpu/index.md",
"content": "# Using GPUs \r\n\r\n## Supported devices \r\n\r\nOn a typical system, there are multiple computing devices. In TensorFlow, the\r\nsupported device types are `CPU` and `GPU`. They are represented as\r\n`strings`. For example:\r\n\r\n* `\"/cpu:0\"`: The CPU of your machine.\r\n* `\"/gpu:0\"`: The GPU of your machine, if you have one.\r\n* `\"/gpu:1\"`: The second GPU of your machine, etc.\r\n\r\nIf a TensorFlow operation has both CPU and GPU implementations, the\r\nGPU devices will be given priority when the operation is assigned to\r\na device. For example, `matmul` has both CPU and GPU kernels. On a\r\nsystem with devices `cpu:0` and `gpu:0`, `gpu:0` will be selected to run\r\n`matmul`.\r\n\r\n## Logging Device placement \r\n\r\nTo find out which devices your operations and tensors are assigned to, create\r\nthe session with `log_device_placement` configuration option set to `True`.\r\n\r\n```python\r\n# Creates a graph.\r\na = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')\r\nb = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')\r\nc = tf.matmul(a, b)\r\n# Creates a session with log_device_placement set to True.\r\nsess = tf.Session(config=tf.ConfigProto(log_device_placement=True))\r\n# Runs the op.\r\nprint sess.run(c)\r\n```\r\n\r\nYou should see the following output:\r\n\r\n```\r\nDevice mapping:\r\n/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: Tesla K40c, pci bus\r\nid: 0000:05:00.0\r\nb: /job:localhost/replica:0/task:0/gpu:0\r\na: /job:localhost/replica:0/task:0/gpu:0\r\nMatMul: /job:localhost/replica:0/task:0/gpu:0\r\n[[ 22. 28.]\r\n [ 49. 64.]]\r\n\r\n```\r\n\r\n## Manual device placement \r\n\r\nIf you would like a particular operation to run on a device of your\r\nchoice instead of what's automatically selected for you, you can use\r\n`with tf.device` to create a device context such that all the operations\r\nwithin that context will have the same device assignment.\r\n\r\n```python\r\n# Creates a graph.\r\nwith tf.device('/cpu:0'):\r\n a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')\r\n b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')\r\nc = tf.matmul(a, b)\r\n# Creates a session with log_device_placement set to True.\r\nsess = tf.Session(config=tf.ConfigProto(log_device_placement=True))\r\n# Runs the op.\r\nprint sess.run(c)\r\n```\r\n\r\nYou will see that now `a` and `b` are assigned to `cpu:0`.\r\n\r\n```\r\nDevice mapping:\r\n/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: Tesla K40c, pci bus\r\nid: 0000:05:00.0\r\nb: /job:localhost/replica:0/task:0/cpu:0\r\na: /job:localhost/replica:0/task:0/cpu:0\r\nMatMul: /job:localhost/replica:0/task:0/gpu:0\r\n[[ 22. 28.]\r\n [ 49. 64.]]\r\n```\r\n\r\n## Using a single GPU on a multi-GPU system \r\n\r\nIf you have more than one GPU in your system, the GPU with the lowest ID will be\r\nselected by default. If you would like to run on a different GPU, you will need\r\nto specify the preference explicitly:\r\n\r\n```python\r\n# Creates a graph.\r\nwith tf.device('/gpu:2'):\r\n a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')\r\n b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')\r\n c = tf.matmul(a, b)\r\n# Creates a session with log_device_placement set to True.\r\nsess = tf.Session(config=tf.ConfigProto(log_device_placement=True))\r\n# Runs the op.\r\nprint sess.run(c)\r\n```\r\n\r\nIf the device you have specified does not exist, you will get\r\n`InvalidArgumentError`:\r\n\r\n```\r\nInvalidArgumentError: Invalid argument: Cannot assign a device to node 'b':\r\nCould not satisfy explicit device specification '/gpu:2'\r\n [[Node: b = Const[dtype=DT_FLOAT, value=Tensor
\r\n\r\n\r\n\r\n\r\n\r\n
\r\n"
},
{
"path": "SOURCE/resources/uses.md",
"content": "# 应用实例 \r\n\r\n本页介绍了一些 TensorFlow 系统当前在实际中的应用。\r\n\r\n> 如果您在做研究、教育、或在某些产品中正在使用 TensorFlow，\r\n> 我们非常乐意在这里添加一些有关您的使用情况。\r\n> 请随时给我们发电子邮件简要说明您是如何使用TensorFlow的，\r\n> 或者给我们发 pull request来添加一个条目到本文件。\r\n\r\n下面列出了一些 TensorFlow 的用途。\r\n\r\n* **RankBrain**\r\n * **组织**: Google\r\n * **域名**: Information Retrieval\r\n * **描述**: 对www.google.com搜索排名大规模部署的深层神经网络。\r\n * **更多信息**: [\"Google Turning Over Its Lucrative Search to AI Machines\"](http://www.bloomberg.com/news/articles/2015-10-26/google-turning-its-lucrative-web-search-over-to-ai-machines)\r\n\r\n* **Inception Image Classification Model**\r\n * **组织**: Google\r\n * **描述**: 研究高精确的计算机视觉模型，赢得了2014年Imagenet图像分类的挑战 (ILSVRC 2014)\r\n * **更多信息**: 关于 Baseline model 的描述 [Arxiv paper](http://arxiv.org/abs/1409.4842)\r\n\r\n* **SmartReply**\r\n * **组织**: Google\r\n * **描述**: 基于深度 LSTM 模型的自动生成电子邮件回复\r\n * **更多信息**: [Google research blog post](http://googleresearch.blogspot.com/2015/11/computer-respond-to-this-email.html)\r\n\r\n* **Massively Multitask Networks for Drug Discovery**\r\n * **组织**: Google and Stanford University\r\n * **域名**: Drug discovery\r\n * **描述**: 基于深度神经网络模型的药物探索\r\n * **更多信息**: [Arxiv paper](http://arxiv.org/abs/1502.02072)\r\n\r\n* **On-Device Computer Vision for OCR**\r\n * **组织**: Google\r\n * **描述**: 用设备内置的计算机视觉模型来做‘光学字符识别’（OCR）以实现实时翻译。\r\n * **更多信息**: [Google Research blog post](http://googleresearch.blogspot.com/2015/07/how-google-translate-squeezes-deep.html)\r\n\r\n原文:[http://tensorflow.org/resources/uses.md](http://tensorflow.org/resources/uses.md) \r\n\r\n翻 译:[andyiac](https://github.com/andyiac)\r\n\r\n校 对:[lonlonago](https://github.com/lonlonago)\r\n\r\n"
},
{
"path": "SOURCE/tutorials/BUILD",
"content": "# Description:\r\n# Top-level tutorials files\r\n\r\npackage(default_visibility = [\"//tensorflow:internal\"])\r\n\r\nlicenses([\"notice\"]) # Apache 2.0\r\n\r\nexports_files([\"LICENSE\"])\r\n\r\nfilegroup(\r\n name = \"all_files\",\r\n srcs = glob(\r\n [\"**/*\"],\r\n exclude = [\r\n \"**/METADATA\",\r\n \"**/OWNERS\",\r\n ],\r\n ),\r\n)\r\n"
},
{
"path": "SOURCE/tutorials/__init__.py",
"content": ""
},
{
"path": "SOURCE/tutorials/deep_cnn/cifar_tensorboard.html",
"content": "\r\n\r\n\r\n \r\n ![](\"./cifar_image_summary.png\")
\r\n
\r\n\r\nReading images from disk and distorting them can use a non-trivial amount of\r\nprocessing time. To prevent these operations from slowing down training, we run\r\nthem inside 16 separate threads which continuously fill a TensorFlow\r\n[queue](../../api_docs/python/io_ops.md#shuffle_batch).\r\n\r\n### Model Prediction \r\n\r\nThe prediction part of the model is constructed by the `inference()` function\r\nwhich adds operations to compute the *logits* of the predictions. That part of\r\nthe model is organized as follows:\r\n\r\nLayer Name | Description\r\n--- | ---\r\n`conv1` | [convolution](../../api_docs/python/nn.md#conv2d) and [rectified linear](../../api_docs/python/nn.md#relu) activation.\r\n`pool1` | [max pooling](../../api_docs/python/nn.md#max_pool).\r\n`norm1` | [local response normalization](../../api_docs/python/nn.md#local_response_normalization).\r\n`conv2` | [convolution](../../api_docs/python/nn.md#conv2d) and [rectified linear](../../api_docs/python/nn.md#relu) activation.\r\n`norm2` | [local response normalization](../../api_docs/python/nn.md#local_response_normalization).\r\n`pool2` | [max pooling](../../api_docs/python/nn.md#max_pool).\r\n`local3` | [fully connected layer with rectified linear activation](../../api_docs/python/nn.md).\r\n`local4` | [fully connected layer with rectified linear activation](../../api_docs/python/nn.md).\r\n`softmax_linear` | linear transformation to produce logits.\r\n\r\nHere is a graph generated from TensorBoard describing the inference operation:\r\n\r\n\r\n\r\n ![](\"./cifar_graph.png\")
\r\n
\r\n\r\n> **EXERCISE**: The output of `inference` are un-normalized logits. Try editing\r\nthe network architecture to return normalized predictions using [`tf.softmax()`]\r\n(../../api_docs/python/nn.md#softmax).\r\n\r\nThe `inputs()` and `inference()` functions provide all the components\r\nnecessary to perform evaluation on a model. We now shift our focus towards\r\nbuilding operations for training a model.\r\n\r\n> **EXERCISE:** The model architecture in `inference()` differs slightly from\r\nthe CIFAR-10 model specified in\r\n[cuda-convnet](https://code.google.com/p/cuda-convnet/). In particular, the top\r\nlayers are locally connected and not fully connected. Try editing the\r\narchitecture to exactly replicate that fully connected model.\r\n\r\n### Model Training \r\n\r\nThe usual method for training a network to perform N-way classification is\r\n[multinomial logistic regression](https://en.wikipedia.org/wiki/Multinomial_logistic_regression),\r\naka. *softmax regression*. Softmax regression applies a\r\n[softmax](../../api_docs/python/nn.md#softmax) nonlinearity to the\r\noutput of the network and calculates the\r\n[cross-entropy](../../api_docs/python/nn.md#softmax_cross_entropy_with_logits)\r\nbetween the normalized predictions and a\r\n[1-hot encoding](../../api_docs/python/sparse_ops.md#sparse_to_dense) of the label.\r\nFor regularization, we also apply the usual\r\n[weight decay](../../api_docs/python/nn.md#l2_loss) losses to all learned\r\nvariables. The objective function for the model is the sum of the cross entropy\r\nloss and all these weight decay terms, as returned by the `loss()` function.\r\n\r\nWe visualize it in TensorBoard with a [scalar_summary](../../api_docs/python/train.md#scalar_summary):\r\n\r\n![CIFAR-10 Loss](./cifar_loss.png \"CIFAR-10 Total Loss\")\r\n\r\nWe train the model using standard\r\n[gradient descent](https://en.wikipedia.org/wiki/Gradient_descent)\r\nalgorithm (see [Training](../../api_docs/python/train.md) for other methods)\r\nwith a learning rate that\r\n[exponentially decays](../../api_docs/python/train.md#exponential_decay)\r\nover time.\r\n\r\n![CIFAR-10 Learning Rate Decay](./cifar_lr_decay.png \"CIFAR-10 Learning Rate Decay\")\r\n\r\nThe `train()` function adds the operations needed to minimize the objective by\r\ncalculating the gradient and updating the learned variables (see\r\n[`GradientDescentOptimizer`](../../api_docs/python/train.md#GradientDescentOptimizer)\r\nfor details). It returns an operation that executes all the calculations\r\nneeded to train and update the model for one batch of images.\r\n\r\n## Launching and Training the Model \r\n\r\nWe have built the model, let's now launch it and run the training operation with\r\nthe script `cifar10_train.py`.\r\n\r\n```shell\r\npython cifar10_train.py\r\n```\r\n\r\n**NOTE:** The first time you run any target in the CIFAR-10 tutorial,\r\nthe CIFAR-10 dataset is automatically downloaded. The data set is ~160MB\r\nso you may want to grab a quick cup of coffee for your first run.\r\n\r\nYou should see the output:\r\n\r\n```shell\r\nFilling queue with 20000 CIFAR images before starting to train. This will take a few minutes.\r\n2015-11-04 11:45:45.927302: step 0, loss = 4.68 (2.0 examples/sec; 64.221 sec/batch)\r\n2015-11-04 11:45:49.133065: step 10, loss = 4.66 (533.8 examples/sec; 0.240 sec/batch)\r\n2015-11-04 11:45:51.397710: step 20, loss = 4.64 (597.4 examples/sec; 0.214 sec/batch)\r\n2015-11-04 11:45:54.446850: step 30, loss = 4.62 (391.0 examples/sec; 0.327 sec/batch)\r\n2015-11-04 11:45:57.152676: step 40, loss = 4.61 (430.2 examples/sec; 0.298 sec/batch)\r\n2015-11-04 11:46:00.437717: step 50, loss = 4.59 (406.4 examples/sec; 0.315 sec/batch)\r\n...\r\n```\r\n\r\nThe script reports the total loss every 10 steps as well the speed at which\r\nthe last batch of data was processed. A few comments:\r\n\r\n* The first batch of data can be inordinately slow (e.g. several minutes) as the\r\npreprocessing threads fill up the shuffling queue with 20,000 processed CIFAR\r\nimages.\r\n\r\n* The reported loss is the average loss of the most recent batch. Remember that\r\nthis loss is the sum of the cross entropy and all weight decay terms.\r\n\r\n* Keep an eye on the processing speed of a batch. The numbers shown above were\r\nobtained on a Tesla K40c. If you are running on a CPU, expect slower performance.\r\n\r\n\r\n> **EXERCISE:** When experimenting, it is sometimes annoying that the first\r\ntraining step can take so long. Try decreasing the number of images initially\r\nthat initially fill up the queue. Search for `NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN`\r\nin `cifar10.py`.\r\n\r\n`cifar10_train.py` periodically [saves](../../api_docs/python/state_ops.md#Saver)\r\nall model parameters in\r\n[checkpoint files](../../how_tos/variables/index.md#saving-and-restoring)\r\nbut it does *not* evaluate the model. The checkpoint file\r\nwill be used by `cifar10_eval.py` to measure the predictive\r\nperformance (see [Evaluating a Model](#evaluating-a-model) below).\r\n\r\n\r\nIf you followed the previous steps, then you have now started training\r\na CIFAR-10 model. [Congratulations!](https://www.youtube.com/watch?v=9bZkp7q19f0)\r\n\r\nThe terminal text returned from `cifar10_train.py` provides minimal insight into\r\nhow the model is training. We want more insight into the model during training:\r\n\r\n* Is the loss *really* decreasing or is that just noise?\r\n* Is the model being provided appropriate images?\r\n* Are the gradients, activations and weights reasonable?\r\n* What is the learning rate currently at?\r\n\r\n[TensorBoard](../../how_tos/summaries_and_tensorboard/index.md) provides this\r\nfunctionality, displaying data exported periodically from `cifar10_train.py` via\r\na\r\n[`SummaryWriter`](../../api_docs/python/train.md#SummaryWriter).\r\n\r\nFor instance, we can watch how the distribution of activations and degree of\r\nsparsity in `local3` features evolve during training:\r\n\r\n\r\n\r\n ![](\"./cifar_sparsity.png\")
\r\n ![](\"./cifar_activations.png\")
\r\n
\r\n\r\nIndividual loss functions, as well as the total loss, are particularly\r\ninteresting to track over time. However, the loss exhibits a considerable amount\r\nof noise due to the small batch size employed by training. In practice we find\r\nit extremely useful to visualize their moving averages in addition to their raw\r\nvalues. See how the scripts use\r\n[`ExponentialMovingAverage`](../../api_docs/python/train.md#ExponentialMovingAverage)\r\nfor this purpose.\r\n\r\n## Evaluating a Model \r\n\r\nLet us now evaluate how well the trained model performs on a hold-out data set.\r\nthe model is evaluated by the script `cifar10_eval.py`. It constructs the model\r\nwith the `inference()` function and uses all 10,000 images in the evaluation set\r\nof CIFAR-10. It calculates the *precision at 1:* how often the top prediction\r\nmatches the true label of the image.\r\n\r\nTo monitor how the model improves during training, the evaluation script runs\r\nperiodically on the latest checkpoint files created by the `cifar10_train.py`.\r\n\r\n```shell\r\npython cifar10_eval.py\r\n```\r\n\r\n> Be careful not to run the evaluation and training binary on the same GPU or\r\nelse you might run out of memory. Consider running the evaluation on\r\na separate GPU if available or suspending the training binary while running\r\nthe evaluation on the same GPU.\r\n\r\nYou should see the output:\r\n\r\n```shell\r\n2015-11-06 08:30:44.391206: precision @ 1 = 0.860\r\n...\r\n```\r\n\r\nThe script merely returns the precision @ 1 periodically -- in this case\r\nit returned 86% accuracy. `cifar10_eval.py` also\r\nexports summaries that may be visualized in TensorBoard. These summaries\r\nprovide additional insight into the model during evaluation.\r\n\r\nThe training script calculates the\r\n[moving average](../../api_docs/python/train.md#ExponentialMovingAverage)\r\nversion of all learned variables. The evaluation script substitutes\r\nall learned model parameters with the moving average version. This\r\nsubstitution boosts model performance at evaluation time.\r\n\r\n> **EXERCISE:** Employing averaged parameters may boost predictive performance\r\nby about 3% as measured by precision @ 1. Edit `cifar10_eval.py` to not employ\r\nthe averaged parameters for the model and verify that the predictive performance\r\ndrops.\r\n\r\n\r\n## Training a Model Using Multiple GPU Cards \r\n\r\nModern workstations may contain multiple GPUs for scientific computation.\r\nTensorFlow can leverage this environment to run the training operation\r\nconcurrently across multiple cards.\r\n\r\nTraining a model in a parallel, distributed fashion requires\r\ncoordinating training processes. For what follows we term *model replica*\r\nto be one copy of a model training on a subset of data.\r\n\r\nNaively employing asynchronous updates of model parameters\r\nleads to sub-optimal training performance\r\nbecause an individual model replica might be trained on a stale\r\ncopy of the model parameters. Conversely, employing fully synchronous\r\nupdates will be as slow as the slowest model replica.\r\n\r\nIn a workstation with multiple GPU cards, each GPU will have similar speed\r\nand contain enough memory to run an entire CIFAR-10 model. Thus, we opt to\r\ndesign our training system in the following manner:\r\n\r\n* Place an individual model replica on each GPU.\r\n* Update model parameters synchronously by waiting for all GPUs to finish\r\nprocessing a batch of data.\r\n\r\nHere is a diagram of this model:\r\n\r\n\r\n \r\n\r\n ![](\"./Parallelism.png\")
\r\n
\r\n\r\nNote that each GPU computes inference as well as the gradients for a unique\r\nbatch of data. This setup effectively permits dividing up a larger batch\r\nof data across the GPUs.\r\n\r\nThis setup requires that all GPUs share the model parameters. A well-known\r\nfact is that transferring data to and from GPUs is quite slow. For this\r\nreason, we decide to store and update all model parameters on the CPU (see\r\ngreen box). A fresh set of model parameters is transferred to the GPU\r\nwhen a new batch of data is processed by all GPUs.\r\n\r\nThe GPUs are synchronized in operation. All gradients are accumulated from\r\nthe GPUs and averaged (see green box). The model parameters are updated with\r\nthe gradients averaged across all model replicas.\r\n\r\n### Placing Variables and Operations on Devices \r\n\r\nPlacing operations and variables on devices requires some special\r\nabstractions.\r\n\r\nThe first abstraction we require is a function for computing inference and\r\ngradients for a single model replica. In the code we term this abstraction\r\na \"tower\". We must set two attributes for each tower:\r\n\r\n* A unique name for all operations within a tower.\r\n[`tf.name_scope()`](../../api_docs/python/framework.md#name_scope) provides\r\nthis unique name by prepending a scope. For instance, all operations in\r\nthe first tower are prepended with `tower_0`, e.g. `tower_0/conv1/Conv2D`.\r\n\r\n* A preferred hardware device to run the operation within a tower.\r\n[`tf.device()`](../../api_docs/python/framework.md#device) specifies this. For\r\ninstance, all operations in the first tower reside within `device('/gpu:0')`\r\nscope indicating that they should be run on the first GPU.\r\n\r\nAll variables are pinned to the CPU and accessed via\r\n[`tf.get_variable()`](../../api_docs/python/state_ops.md#get_variable)\r\nin order to share them in a multi-GPU version.\r\nSee how-to on [Sharing Variables](../../how_tos/variable_scope/index.md).\r\n\r\n### Launching and Training the Model on Multiple GPU cards \r\n\r\nIf you have several GPU cards installed on your machine you can use them to\r\ntrain the model faster with the `cifar10_multi_gpu_train.py` script. It is a\r\nvariation of the training script that parallelizes the model across multiple GPU\r\ncards.\r\n\r\n```shell\r\npython cifar10_multi_gpu_train.py --num_gpus=2\r\n```\r\n\r\nThe training script should output:\r\n\r\n```shell\r\nFilling queue with 20000 CIFAR images before starting to train. This will take a few minutes.\r\n2015-11-04 11:45:45.927302: step 0, loss = 4.68 (2.0 examples/sec; 64.221 sec/batch)\r\n2015-11-04 11:45:49.133065: step 10, loss = 4.66 (533.8 examples/sec; 0.240 sec/batch)\r\n2015-11-04 11:45:51.397710: step 20, loss = 4.64 (597.4 examples/sec; 0.214 sec/batch)\r\n2015-11-04 11:45:54.446850: step 30, loss = 4.62 (391.0 examples/sec; 0.327 sec/batch)\r\n2015-11-04 11:45:57.152676: step 40, loss = 4.61 (430.2 examples/sec; 0.298 sec/batch)\r\n2015-11-04 11:46:00.437717: step 50, loss = 4.59 (406.4 examples/sec; 0.315 sec/batch)\r\n...\r\n```\r\n\r\nNote that the number of GPU cards used defaults to 1. Additionally, if only 1\r\nGPU is available on your machine, all computations will be placed on it, even if\r\nyou ask for more.\r\n\r\n> **EXERCISE:** The default settings for `cifar10_train.py` is to\r\nrun on a batch size of 128. Try running `cifar10_multi_gpu_train.py` on 2 GPUs\r\nwith a batch size of 64 and compare the training speed.\r\n\r\n## Next Steps \r\n\r\n[Congratulations!](https://www.youtube.com/watch?v=9bZkp7q19f0) You have\r\ncompleted the CIFAR-10 tutorial.\r\n\r\nIf you are now interested in developing and training your own image\r\nclassification system, we recommend forking this tutorial and replacing\r\ncomponents to build address your image classification problem.\r\n\r\n> **EXERCISE:** Download the\r\n[Street View House Numbers (SVHN)](http://ufldl.stanford.edu/housenumbers/) data set.\r\nFork the CIFAR-10 tutorial and swap in the SVHN as the input data. Try adapting\r\nthe network architecture to improve predictive performance.\r\n"
},
{
"path": "SOURCE/tutorials/deep_cnn.md",
"content": "# 卷积神经网络 \r\n\r\n> **注意:** 本教程适用于对Tensorflow有丰富经验的用户，并假定用户有机器学习相关领域的专业知识和经验。\r\n\r\n## 概述 \r\n\r\n对CIFAR-10 数据集的分类是机器学习中一个公开的基准测试问题，其任务是对一组大小为32x32的RGB图像进行分类，这些图像涵盖了10个类别： \r\n```飞机， 汽车， 鸟， 猫， 鹿， 狗， 青蛙， 马， 船以及卡车。```\r\n\r\n![CIFAR-10 Samples](../images/cifar_samples.png \"CIFAR-10 Samples, from http://www.cs.toronto.edu/~kriz/cifar.html\")\r\n\r\n想了解更多信息请参考[CIFAR-10 page](http://www.cs.toronto.edu/~kriz/cifar.html)，以及Alex Krizhevsky写的[技术报告](http://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf) \r\n\r\n### 目标 \r\n\r\n本教程的目标是建立一个用于识别图像的相对较小的卷积神经网络，在这一过程中，本教程会：\r\n\r\n1. 着重于建立一个规范的网络组织结构，训练并进行评估；\r\n2. 为建立更大规模更加复杂的模型提供一个范例\r\n\r\n选择CIFAR-10是因为它的复杂程度足以用来检验TensorFlow中的大部分功能，并可将其扩展为更大的模型。与此同时由于模型较小所以训练速度很快，比较适合用来测试新的想法，检验新的技术。\r\n\r\n### 本教程的重点 \r\nCIFAR-10 教程演示了在TensorFlow上构建更大更复杂模型的几个种重要内容： \r\n* 相关核心数学对象，如[卷积](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#conv2d)、[修正线性激活](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#relu)、[最大池化](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#max_pool)以及[局部响应归一化](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#local_response_normalization)； \r\n* 训练过程中一些网络行为的[可视化](https://github.com/jikexueyuanwiki/tensorflow-zh/tree/master/SOURCE/how_tos/summaries_and_tensorboard/index.md)，这些行为包括输入图像、损失情况、网络行为的分布情况以及梯度； \r\n* 算法学习参数的[移动平均值](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/train.md#ExponentialMovingAverage)的计算函数，以及在评估阶段使用这些平均值提高预测性能； \r\n* 实现了一种机制，使得[学习率](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/train.md#exponential_decay)随着时间的推移而递减； \r\n* 为输入数据设计预存取[队列](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/io_ops.md#shuffle_batch)，将磁盘延迟和高开销的图像预处理操作与模型分离开来处理； \r\n\r\n我们也提供了模型的多GPU版本，用以表明： \r\n* 可以配置模型后使其在多个GPU上并行的训练 \r\n* 可以在多个GPU之间共享和更新变量值 \r\n\r\n我们希望本教程给大家开了个头，使得在Tensorflow上可以为视觉相关工作建立更大型的CNN模型\r\n\r\n### 模型架构 \r\n\r\n本教程中的模型是一个多层架构，由卷积层和非线性层(nonlinearities)交替多次排列后构成。这些层最终通过全连通层对接到softmax分类器上。这一模型除了最顶部的几层外，基本跟[Alex Krizhevsky](https://code.google.com/p/cuda-convnet/)提出的模型一致。 \r\n\r\n在一个GPU上经过几个小时的训练后，该模型最高可以达到86%的精度。细节请查看[下面](#evaluating-a-model)的描述以及代码。模型中包含了1,068,298个学习参数，对一副图像进行分类大概需要19.5M个乘加操作。\r\n\r\n## 代码组织 \r\n\r\n本教程的代码位于[`tensorflow/models/image/cifar10/`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/cifar10/).\r\n\r\n文件 | 作用\r\n--- | ---\r\n[`cifar10_input.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/cifar10/cifar10_input.py) | 读取本地CIFAR-10的二进制文件格式的内容。\r\n[`cifar10.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/cifar10/cifar10.py) | 建立CIFAR-10的模型。\r\n[`cifar10_train.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/cifar10/cifar10_train.py) | 在CPU或GPU上训练CIFAR-10的模型。\r\n[`cifar10_multi_gpu_train.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/cifar10/cifar10_multi_gpu_train.py) | 在多GPU上训练CIFAR-10的模型。\r\n[`cifar10_eval.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/cifar10/cifar10_eval.py) | 评估CIFAR-10模型的预测性能。\r\n\r\n## CIFAR-10 模型 \r\n\r\nCIFAR-10 网络模型部分的代码位于\r\n[`cifar10.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/cifar10/cifar10.py).\r\n完整的训练图中包含约765个操作。但是我们发现通过下面的模块来构造训练图可以最大限度的提高代码复用率:\r\n\r\n1. [**模型输入:**](#model-inputs) 包括`inputs()` 、 `distorted_inputs()`等一些操作，分别用于读取CIFAR的图像并进行预处理，做为后续评估和训练的输入； \r\n2. [**模型预测:**](#model-prediction) 包括`inference()`等一些操作，用于进行统计计算，比如在提供的图像进行分类；\r\nadds operations that perform inference, i.e. classification, on supplied images.\r\n3. [**模型训练:**](#model-training) 包括`loss()` and `train()`等一些操作，用于计算损失、计算梯度、进行变量更新以及呈现最终结果。\r\n\r\n### 模型输入 \r\n\r\n输入模型是通过 `inputs()` 和`distorted_inputs()`函数建立起来的，这2个函数会从CIFAR-10二进制文件中读取图片文件，由于每个图片的存储字节数是固定的，因此可以使用[`tf.FixedLengthRecordReader`](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/io_ops.md#FixedLengthRecordReader)函数。更多的关于`Reader`类的功能可以查看[Reading Data](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/how_tos/reading_data/index.md#reading-from-files)。\r\n\r\n图片文件的处理流程如下： \r\n\r\n* 图片会被统一裁剪到24x24像素大小，裁剪中央区域用于评估或[随机](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/image.md#random_crop)裁剪用于训练；\r\n* 图片会进行[近似的白化处理](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/image.md#per_image_whitening)，使得模型对图片的动态范围变化不敏感。\r\n\r\n对于训练，我们另外采取了一系列随机变换的方法来人为的增加数据集的大小：\r\n\r\n* 对图像进行[随机的左右翻转](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/image.md#random_flip_left_right)；\r\n* 随机变换[图像的亮度](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/image.md#random_brightness)；\r\n* 随机变换[图像的对比度](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/image.md#tf_image_random_contrast)；\r\n\r\n可以在[Images](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/image.md)页的列表中查看所有可用的变换，对于每个原始图我们还附带了一个[`image_summary`](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/train.md#image_summary)，以便于在TensorBoard中查看。这对于检查输入图像是否正确十分有用。 \r\n\r\n\r\n\r\n ![](\"../images/cifar_image_summary.png\")
\r\n
\r\n\r\n从磁盘上加载图像并进行变换需要花费不少的处理时间。为了避免这些操作减慢训练过程，我们在16个独立的线程中并行进行这些操作，这16个线程被连续的安排在一个TensorFlow[队列](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/io_ops.md#shuffle_batch)中。 \r\n\r\n### 模型预测 \r\n\r\n模型的预测流程由`inference()`构造，该函数会添加必要的操作步骤用于计算预测值的 *logits*，其对应的模型组织方式如下所示：\r\n\r\nLayer 名称 | 描述\r\n--- | ---\r\n`conv1` | 实现[卷积](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#conv2d) 以及 [rectified linear](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#relu) activation.\r\n`pool1` | [max pooling](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#max_pool).\r\n`norm1` | [局部响应归一化](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#local_response_normalization).\r\n`conv2` | [卷积](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#conv2d) and [rectified linear](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#relu) activation.\r\n`norm2` | [局部响应归一化](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#local_response_normalization).\r\n`pool2` | [max pooling](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#max_pool).\r\n`local3` | [基于修正线性激活的全连接层](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md).\r\n`local4` | [基于修正线性激活的全连接层](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md).\r\n`softmax_linear` | 进行线性变换以输出 logits.\r\n\r\n这里有一个由TensorBoard绘制的图形，用于描述模型建立过程中经过的步骤：\r\n\r\n\r\n\r\n ![](\"../images/cifar_graph.png\")
\r\n
\r\n\r\n> **练习**: `inference`的输出是未归一化的logits，尝试使用[`tf.softmax()`](tensorflow-zh/SOURCE/api_docs/python/nn.md#softmax)修改网络架构后返回归一化的预测值。\r\n\r\n`inputs()` 和 `inference()` 函数提供了评估模型时所需的所有构件，现在我们把讲解的重点从构建一个模型转向训练一个模型。\r\n\r\n> **练习:** `inference()` 中的模型跟[cuda-convnet](https://code.google.com/p/cuda-convnet/)中描述的CIFAR-10模型有些许不同，其差异主要在于其顶层不是全连接层而是局部连接层，可以尝试修改网络架构来准确的复制全连接模型。\r\n\r\n### 模型训练 \r\n\r\n训练一个可进行N维分类的网络的常用方法是使用[多项式逻辑回归](https://en.wikipedia.org/wiki/Multinomial_logistic_regression),又被叫做*softmax 回归*。Softmax 回归在网络的输出层上附加了一个[softmax](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#softmax) nonlinearity，并且计算归一化的预测值和label的[1-hot encoding](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/sparse_ops.md#sparse_to_dense)的[交叉熵](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#softmax_cross_entropy_with_logits)。在正则化过程中，我们会对所有学习变量应用[权重衰减损失](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/nn.md#l2_loss)。模型的目标函数是求交叉熵损失和所有权重衰减项的和，`loss()`函数的返回值就是这个值。\r\n\r\n在TensorBoard中使用[scalar_summary](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/train.md#scalar_summary)来查看该值的变化情况：\r\n\r\n![CIFAR-10 Loss](../images/cifar_loss.png \"CIFAR-10 Total Loss\")\r\n\r\n我们使用标准的梯度下降算法来训练模型（也可以在[Training](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/train.md)中看看其他方法），其学习率随时间以指数形式衰减。\r\n\r\n![CIFAR-10 Learning Rate Decay](../images/cifar_lr_decay.png \"CIFAR-10 Learning Rate Decay\")\r\n\r\n`train()` 函数会添加一些操作使得目标函数最小化，这些操作包括计算梯度、更新学习变量（详细信息请查看[`GradientDescentOptimizer`](tensorflow-zh/SOURCE/api_docs/python/train.md#GradientDescentOptimizer)）。`train()` 函数最终会返回一个用以对一批图像执行所有计算的操作步骤，以便训练并更新模型。\r\n\r\n## 开始执行并训练模型 \r\n\r\n我们已经把模型建立好了，现在通过执行脚本`cifar10_train.py`来启动训练过程。\r\n\r\n```shell\r\npython cifar10_train.py\r\n```\r\n\r\n**注意:** 当第一次在CIFAR-10教程上启动任何任务时，会自动下载CIFAR-10数据集，该数据集大约有160M大小，因此第一次运行时泡杯咖啡小栖一会吧。\r\n\r\n你应该可以看到如下类似的输出:\r\n\r\n```shell\r\nFilling queue with 20000 CIFAR images before starting to train. This will take a few minutes.\r\n2015-11-04 11:45:45.927302: step 0, loss = 4.68 (2.0 examples/sec; 64.221 sec/batch)\r\n2015-11-04 11:45:49.133065: step 10, loss = 4.66 (533.8 examples/sec; 0.240 sec/batch)\r\n2015-11-04 11:45:51.397710: step 20, loss = 4.64 (597.4 examples/sec; 0.214 sec/batch)\r\n2015-11-04 11:45:54.446850: step 30, loss = 4.62 (391.0 examples/sec; 0.327 sec/batch)\r\n2015-11-04 11:45:57.152676: step 40, loss = 4.61 (430.2 examples/sec; 0.298 sec/batch)\r\n2015-11-04 11:46:00.437717: step 50, loss = 4.59 (406.4 examples/sec; 0.315 sec/batch)\r\n...\r\n```\r\n\r\n脚本会在每10步训练过程后打印出总损失值，以及最后一批数据的处理速度。下面是几点注释：\r\n\r\n* 第一批数据会非常的慢（大概要几分钟时间），因为预处理线程要把20,000个待处理的CIFAR图像填充到重排队列中；\r\n\r\n* 打印出来的损失值是最近一批数据的损失值的均值。请记住损失值是交叉熵和权重衰减项的和；\r\n\r\n* 上面打印结果中关于一批数据的处理速度是在Tesla K40C上统计出来的，如果你运行在CPU上，性能会比此要低；\r\n\r\n> **练习:** 当实验时，第一阶段的训练时间有时会非常的长，长到足以让人生厌。可以尝试减少初始化时初始填充到队列中图片数量来改变这种情况。在`cifar10.py`中搜索`NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN`并修改之。 \r\n\r\n`cifar10_train.py` 会周期性的在[检查点文件](https://github.com/jikexueyuanwiki/tensorflow-zh/tree/master/SOURCE/how_tos/variables/index.md#saving-and-restoring)中[保存](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/state_ops.md#Saver)模型中的所有参数，但是*不会*对模型进行评估。`cifar10_eval.py`会使用该检查点文件来测试预测性能（详见下面的描述：[评估模型](#评估模型)）。\r\n\r\n如果按照上面的步骤做下来，你应该已经开始训练一个CIFAR-10模型了。[恭喜你!](https://www.youtube.com/watch?v=9bZkp7q19f0)\r\n\r\n`cifar10_train.py`输出的终端信息中提供了关于模型如何训练的一些信息，但是我们可能希望了解更多关于模型训练时的信息，比如： \r\n* 损失是*真的*在减小还是看到的只是噪声数据？ \r\n* 为模型提供的图片是否合适？ \r\n* 梯度、激活、权重的值是否合理？ \r\n* 当前的学习率是多少？ \r\n\r\n[TensorBoard](https://github.com/jikexueyuanwiki/tensorflow-zh/tree/master/SOURCE/how_tos/summaries_and_tensorboard/index.md)提供了该功能，可以通过`cifar10_train.py`中的[`SummaryWriter`](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/train.md#SummaryWriter)周期性的获取并显示这些数据。\r\n \r\n比如我们可以在训练过程中查看`local3`的激活情况，以及其特征维度的稀疏情况：\r\n\r\n\r\n\r\n ![](\"../images/cifar_sparsity.png\")
\r\n ![](\"../images/cifar_activations.png\")
\r\n
\r\n\r\n相比于总损失，在训练过程中的单项损失尤其值得人们的注意。但是由于训练中使用的数据批量比较小，损失值中夹杂了相当多的噪声。在实践过程中，我们也发现相比于原始值，损失值的移动平均值显得更为有意义。请参阅脚本[`ExponentialMovingAverage`](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/train.md#ExponentialMovingAverage)了解如何实现。\r\n\r\n## 评估模型 \r\n\r\n现在可以在另一部分数据集上来评估训练模型的性能。脚本文件`cifar10_eval.py`对模型进行了评估，利用 `inference()`函数重构模型，并使用了在评估数据集所有10,000张CIFAR-10图片进行测试。最终计算出的精度为*1:N*，N=预测值中置信度最高的一项与图片真实label匹配的频次。(It calculates the *precision at 1:* how often the top prediction matches the true label of the image)。\r\n\r\n为了监控模型在训练过程中的改进情况，评估用的脚本文件会周期性的在最新的检查点文件上运行，这些检查点文件是由`cifar10_train.py`产生。\r\n\r\n```shell\r\npython cifar10_eval.py\r\n```\r\n\r\n>注意：不要在同一块GPU上同时运行训练程序和评估程序，因为可能会导致内存耗尽。尽可能的在其它单独的GPU上运行评估程序，或者在同一块GPU上运行评估程序时先挂起训练程序。\r\n\r\n你可能会看到如下所示输出:\r\n\r\n```shell\r\n2015-11-06 08:30:44.391206: precision @ 1 = 0.860\r\n...\r\n```\r\n\r\n评估脚本只是周期性的返回precision@1 (The script merely returns the precision @ 1 periodically)--在该例中返回的准确率是86%。`cifar10_eval.py` 同时也返回其它一些可以在TensorBoard中进行可视化的简要信息。可以通过这些简要信息在评估过程中进一步的了解模型。\r\n\r\n训练脚本会为所有学习变量计算其[移动均值](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/train.md#ExponentialMovingAverage),评估脚本则直接将所有学习到的模型参数替换成对应的移动均值。这一替代方式可以在评估过程中提升模型的性能。 \r\n\r\n> **练习:** 通过precision @ 1测试发现，使用均值参数可以将预测性能提高约3%，在`cifar10_eval.py`中尝试修改为不采用均值参数的方式，并确认由此带来的预测性能下降。 \r\n\r\n## 在多个GPU板卡上训练模型 \r\n\r\n现代的工作站可能包含多个GPU进行科学计算。TensorFlow可以利用这一环境在多个GPU卡上运行训练程序。\r\n\r\n在并行、分布式的环境中进行训练，需要对训练程序进行协调。对于接下来的描述，术语*模型拷贝*（*model replica*）特指在一个数据子集中训练出来的模型的一份拷贝。\r\n\r\n如果天真的对模型参数的采用异步方式更新将会导致次优的训练性能，这是因为我们可能会基于一个旧的模型参数的拷贝去训练一个模型。但与此相反采用完全同步更新的方式，其速度将会变得和最慢的模型一样慢(Conversely, employing fully synchronous updates will be as slow as the slowest model replica.)。\r\n\r\n在具有多个GPU的工作站中，每个GPU的速度基本接近，并且都含有足够的内存来运行整个CIFAR-10模型。因此我们选择以下方式来设计我们的训练系统： \r\n\r\n* 在每个GPU上放置单独的模型副本；\r\n \r\n* 等所有GPU处理完一批数据后再同步更新模型的参数；\r\n\r\n下图示意了该模型的结构：:\r\n\r\n\r\n \r\n\r\n ![](\"../images/Parallelism.png\")
\r\n
\r\n\r\n可以看到，每一个GPU会用一批独立的数据计算梯度和估计值。这种设置可以非常有效的将一大批数据分割到各个GPU上。\r\n\r\n这一机制要求所有GPU能够共享模型参数。但是众所周知在GPU之间传输数据非常的慢，因此我们决定在CPU上存储和更新所有模型的参数(对应图中绿色矩形的位置)。这样一来，GPU在处理一批新的数据之前会更新一遍的参数。\r\n\r\n图中所有的GPU是同步运行的。所有GPU中的梯度会累积并求平均值(绿色方框部分)。模型参数会利用所有模型副本梯度的均值来更新。 \r\n\r\n### 在多个设备中设置变量和操作 \r\n\r\n在多个设备中设置变量和操作时需要做一些特殊的抽象。\r\n\r\n我们首先需要把在单个模型拷贝中计算估计值和梯度的行为抽象到一个函数中。在代码中，我们称这个抽象对象为“tower”。对于每一个“tower”我们都需要设置它的两个属性： \r\n* 在一个tower中为所有操作设定一个唯一的名称。[`tf.name_scope()`](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/framework.md#name_scope)通过添加一个范围前缀来提供该唯一名称。比如，第一个tower中的所有操作都会附带一个前缀`tower_0`，示例：`tower_0/conv1/Conv2D`；\r\n\r\n* 在一个tower中运行操作的优先硬件设备。 [`tf.device()`](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/framework.md#device) 提供该信息。比如，在第一个tower中的所有操作都位于 `device('/gpu:0')`范围中，暗含的意思是这些操作应该运行在第一块GPU上；\r\n\r\n为了在多个GPU上共享变量，所有的变量都绑定在CPU上，并通过[`tf.get_variable()`](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/api_docs/python/state_ops.md#get_variable)访问。可以查看[Sharing Variables](https://github.com/jikexueyuanwiki/tensorflow-zh/blob/master/SOURCE/how_tos/variables/index.md)以了解如何共享变量。\r\n\r\n### 启动并在多个GPU上训练模型 \r\n\r\n如果你的机器上安装有多块GPU，你可以通过使用`cifar10_multi_gpu_train.py`脚本来加速模型训练。该脚本是训练脚本的一个变种，使用多个GPU实现模型并行训练。\r\n\r\n```shell\r\npython cifar10_multi_gpu_train.py --num_gpus=2\r\n```\r\n\r\n训练脚本的输出如下所示：\r\n\r\n```shell\r\nFilling queue with 20000 CIFAR images before starting to train. This will take a few minutes.\r\n2015-11-04 11:45:45.927302: step 0, loss = 4.68 (2.0 examples/sec; 64.221 sec/batch)\r\n2015-11-04 11:45:49.133065: step 10, loss = 4.66 (533.8 examples/sec; 0.240 sec/batch)\r\n2015-11-04 11:45:51.397710: step 20, loss = 4.64 (597.4 examples/sec; 0.214 sec/batch)\r\n2015-11-04 11:45:54.446850: step 30, loss = 4.62 (391.0 examples/sec; 0.327 sec/batch)\r\n2015-11-04 11:45:57.152676: step 40, loss = 4.61 (430.2 examples/sec; 0.298 sec/batch)\r\n2015-11-04 11:46:00.437717: step 50, loss = 4.59 (406.4 examples/sec; 0.315 sec/batch)\r\n...\r\n```\r\n需要注意的是默认的GPU使用数是1，此外，如果你的机器上只有一个GPU，那么所有的计算都只会在一个GPU上运行，即便你可能设置的是N个。\r\n\r\n> **练习:** `cifar10_train.py`中的批处理大小默认配置是128。尝试在2个GPU上运行`cifar10_multi_gpu_train.py`脚本，并且设定批处理大小为64，然后比较2种方式的训练速度。\r\n\r\n## 下一步 \r\n\r\n[恭喜你!](https://www.youtube.com/watch?v=9bZkp7q19f0) 你已经完成了CIFAR-10教程。\r\n如果你对开发和训练自己的图像分类系统感兴趣，我们推荐你新建一个基于该教程的分支，并修改其中的内容以建立解决您问题的图像分类系统。\r\n\r\n> **练习:** 下载[Street View House Numbers (SVHN)](http://ufldl.stanford.edu/housenumbers/) 数据集。新建一个CIFAR-10教程的分支，并将输入数据替换成SVHN。尝试改变网络结构以提高预测性能。\r\n\r\n> 原文：[Convolutional Neural Networks](http://tensorflow.org/tutorials/deep_cnn/index.md) 翻译：[oskycar](https://github.com/oskycar) 校对：[KK4SBB](https://github.com/zhyhooo)\r\n"
},
{
"path": "SOURCE/tutorials/mandelbrot/index.md",
"content": "# Mandelbrot Set \r\n\r\nVisualizing the Mandelbrot set doesn't have anything to do with machine\r\nlearning, but it makes for a fun example of how one can use TensorFlow for\r\ngeneral mathematics. This is actually a pretty naive implementation of the\r\nvisualization, but it makes the point. (We may end up providing a more\r\nelaborate implementation down the line to produce more truly beautiful images.)\r\n\r\nNote: This tutorial was originally prepared as an IPython notebook.\r\n\r\n## Basic Setup \r\n\r\nWe'll need a few imports to get started.\r\n\r\n```python\r\n# Import libraries for simulation\r\nimport tensorflow as tf\r\nimport numpy as np\r\n\r\n# Imports for visualization\r\nimport PIL.Image\r\nfrom cStringIO import StringIO\r\nfrom IPython.display import clear_output, Image, display\r\nimport scipy.ndimage as nd\r\n```\r\n\r\nNow we'll define a function to actually display the image once we have\r\niteration counts.\r\n\r\n```python\r\ndef DisplayFractal(a, fmt='jpeg'):\r\n \"\"\"Display an array of iteration counts as a\r\n colorful picture of a fractal.\"\"\"\r\n a_cyclic = (6.28*a/20.0).reshape(list(a.shape)+[1])\r\n img = np.concatenate([10+20*np.cos(a_cyclic),\r\n 30+50*np.sin(a_cyclic),\r\n 155-80*np.cos(a_cyclic)], 2)\r\n img[a==a.max()] = 0\r\n a = img\r\n a = np.uint8(np.clip(a, 0, 255))\r\n f = StringIO()\r\n PIL.Image.fromarray(a).save(f, fmt)\r\n display(Image(data=f.getvalue()))\r\n```\r\n\r\n## Session and Variable Initialization \r\n\r\nFor playing around like this, we often use an interactive session, but a regular\r\nsession would work as well.\r\n\r\n```python\r\n sess = tf.InteractiveSession()\r\n```\r\n\r\nIt's handy that we can freely mix NumPy and TensorFlow.\r\n\r\n```python\r\n# Use NumPy to create a 2D array of complex numbers on [-2,2]x[-2,2]\r\n\r\nY, X = np.mgrid[-1.3:1.3:0.005, -2:1:0.005]\r\nZ = X+1j*Y\r\n```\r\n\r\nNow we define and initialize TensorFlow tensors.\r\n\r\n```python\r\nxs = tf.constant(Z.astype(\"complex64\"))\r\nzs = tf.Variable(xs)\r\nns = tf.Variable(tf.zeros_like(xs, \"float32\"))\r\n```\r\n\r\nTensorFlow requires that you explicitly initialize variables before using them.\r\n\r\n```python\r\ntf.initialize_all_variables().run()\r\n```\r\n\r\n## Defining and Running the Computation \r\n\r\nNow we specify more of the computation...\r\n\r\n```python\r\n# Compute the new values of z: z^2 + x\r\nzs_ = zs*zs + xs\r\n\r\n# Have we diverged with this new value?\r\nnot_diverged = tf.complex_abs(zs_) < 4\r\n\r\n# Operation to update the zs and the iteration count.\r\n#\r\n# Note: We keep computing zs after they diverge! This\r\n# is very wasteful! There are better, if a little\r\n# less simple, ways to do this.\r\n#\r\nstep = tf.group(\r\n zs.assign(zs_),\r\n ns.assign_add(tf.cast(not_diverged, \"float32\"))\r\n )\r\n```\r\n\r\n... and run it for a couple hundred steps\r\n\r\n```python\r\nfor i in range(200): step.run()\r\n```\r\n\r\nLet's see what we've got.\r\n\r\n```python\r\nDisplayFractal(ns.eval())\r\n```\r\n\r\n\r\n\r\nNot bad!\r\n\r\n\r\n"
},
{
"path": "SOURCE/tutorials/mandelbrot.md",
"content": "# 曼德布洛特(Mandelbrot)集合 \r\n\r\n虽然可视化曼德布洛特(Mandelbrot)集合与机器学习没有任何关系，但这对于将TensorFlow应用在数学更广泛的领域是一个有趣的例子。实际上，这是tensorflow一个非常直截了当的可视化运用。（我们最终也许会提供一种更加精心设计的运用方式来生成真正更加美丽的图像。）\r\n\r\n说明：本教程使用了IPython的notebook。\r\n\r\n## 基本步骤 \r\n\r\n首先，我们需要导入一些库。\r\n\r\n```python\r\n# 导入仿真库\r\nimport tensorflow as tf\r\nimport numpy as np\r\n\r\n# 导入可视化库\r\nimport PIL.Image\r\nfrom cStringIO import StringIO\r\nfrom IPython.display import clear_output, Image, display\r\nimport scipy.ndimage as nd\r\n```\r\n\r\n现在我们将定义一个函数来显示迭代计算出的图像。\r\n\r\n```python\r\ndef DisplayFractal(a, fmt='jpeg'):\r\n \"\"\"显示迭代计算出的彩色分形图像。\"\"\"\r\n a_cyclic = (6.28*a/20.0).reshape(list(a.shape)+[1])\r\n img = np.concatenate([10+20*np.cos(a_cyclic),\r\n 30+50*np.sin(a_cyclic),\r\n 155-80*np.cos(a_cyclic)], 2)\r\n img[a==a.max()] = 0\r\n a = img\r\n a = np.uint8(np.clip(a, 0, 255))\r\n f = StringIO()\r\n PIL.Image.fromarray(a).save(f, fmt)\r\n display(Image(data=f.getvalue()))\r\n```\r\n\r\n## 会话（session）和变量（variable）初始化 \r\n\r\n为了操作的方便，我们常常使用交互式会话（interactive session），但普通会话（regular session）也能正常使用。\r\n\r\n```python\r\n sess = tf.InteractiveSession()\r\n```\r\n\r\n我们可以自由的混合使用NumPy和TensorFlow，这一点非常方便。\r\n\r\n```python\r\n# 使用NumPy创建一个在[-2,2]x[-2,2]范围内的2维复数数组\r\n\r\nY, X = np.mgrid[-1.3:1.3:0.005, -2:1:0.005]\r\nZ = X+1j*Y\r\n```\r\n\r\n现在我们定义并初始化一组TensorFlow的张量 （tensors）。\r\n\r\n```python\r\nxs = tf.constant(Z.astype(\"complex64\"))\r\nzs = tf.Variable(xs)\r\nns = tf.Variable(tf.zeros_like(xs, \"float32\"))\r\n```\r\n\r\nTensorFlow在使用之前需要你明确给定变量的初始值。\r\n\r\n```python\r\ntf.initialize_all_variables().run()\r\n```\r\n\r\n## 定义并运行计算 \r\n\r\n现在我们指定更多的计算...\r\n\r\n```python\r\n# 计算一个新值z: z^2 + x\r\nzs_ = zs*zs + xs\r\n\r\n# 这个新值会发散吗？\r\nnot_diverged = tf.complex_abs(zs_) < 4\r\n\r\n# 更新zs并且迭代计算。\r\n#\r\n# 说明：在这些值发散之后，我们仍然在计算zs，这个计算消耗特别大！\r\n# 如果稍微简单点，这里有更好的方法来处理。\r\n#\r\nstep = tf.group(\r\n zs.assign(zs_),\r\n ns.assign_add(tf.cast(not_diverged, \"float32\"))\r\n )\r\n```\r\n\r\n...继续执行几百个步骤\r\n\r\n```python\r\nfor i in range(200): step.run()\r\n```\r\n\r\n让我们看看我们得到了什么。\r\n\r\n```python\r\nDisplayFractal(ns.eval())\r\n```\r\n\r\n\r\n\r\n结果不错！\r\n\r\n> 原文：[Mandelbrot Set](http://tensorflow.org/tutorials/mandelbrot/index.md) 翻译：[ericxk](https://github.com/ericxk) 校对：[tensorfly](https://github.com/tensorfly)\r\n"
},
{
"path": "SOURCE/tutorials/mnist/__init__.py",
"content": ""
},
{
"path": "SOURCE/tutorials/mnist/beginners/index.md",
"content": "# MNIST For ML Beginners \r\n\r\n*This tutorial is intended for readers who are new to both machine learning and\r\nTensorFlow. If you already\r\nknow what MNIST is, and what softmax (multinomial logistic) regression is,\r\nyou might prefer this [faster paced tutorial](../../../tutorials/mnist/pros/index.md).*\r\n\r\nWhen one learns how to program, there's a tradition that the first thing you do\r\nis print \"Hello World.\" Just like programming has Hello World, machine learning\r\nhas MNIST.\r\n\r\nMNIST is a simple computer vision dataset. It consists of images of handwritten\r\ndigits like these:\r\n\r\n\r\n\r\n![](\"img/MNIST.png\")
\r\n
\r\n\r\nIt also includes labels for each image, telling us which digit it is. For\r\nexample, the labels for the above images are 5, 0, 4, and 1.\r\n\r\nIn this tutorial, we're going to train a model to look at images and predict\r\nwhat digits they are. Our goal isn't to train a really elaborate model that\r\nachieves state-of-the-art performance -- although we'll give you code to do that\r\nlater! -- but rather to dip a toe into using TensorFlow. As such, we're going\r\nto start with a very simple model, called a Softmax Regression.\r\n\r\nThe actual code for this tutorial is very short, and all the interesting\r\nstuff happens in just three lines. However, it is very\r\nimportant to understand the ideas behind it: both how TensorFlow works and the\r\ncore machine learning concepts. Because of this, we are going to very carefully\r\nwork through the code.\r\n\r\n## The MNIST Data \r\n\r\nThe MNIST data is hosted on\r\n[Yann LeCun's website](http://yann.lecun.com/exdb/mnist/). For your\r\nconvenience, we've included some python code to download and install the data\r\nautomatically. You can either download\r\n[the code](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/tutorials/mnist/input_data.py)\r\nand import it as below, or simply copy and paste it in.\r\n\r\n```python\r\nimport input_data\r\nmnist = input_data.read_data_sets(\"MNIST_data/\", one_hot=True)\r\n```\r\n\r\nThe downloaded data is split into two parts, 60,000 data points of training\r\ndata (`mnist.train`) and 10,000 points of test data (`mnist.test`). This\r\nsplit is very important: it's essential in machine learning that we\r\nhave separate data which we don't learn from so that we can make sure\r\nthat what we've learned actually generalizes!\r\n\r\nAs mentioned earlier, every MNIST data point has two parts: an image of a\r\nhandwritten digit and a corresponding label. We will call the images \"xs\" and\r\nthe labels \"ys\". Both the training set and test set contain xs and ys, for\r\nexample the training images are `mnist.train.images` and the train labels are\r\n`mnist.train.labels`.\r\n\r\nEach image is 28 pixels by 28 pixels. We can interpret this as a big array of\r\nnumbers:\r\n\r\n\r\n\r\n![](\"img/MNIST-Matrix.png\")
\r\n
\r\n\r\nWe can flatten this array into a vector of 28x28 = 784 numbers. It doesn't\r\nmatter how we flatten the array, as long as we're consistent between images.\r\nFrom this perspective, the MNIST images are just a bunch of points in a\r\n784-dimensional vector space, with a\r\n[very rich structure](http://colah.github.io/posts/2014-10-Visualizing-MNIST/)\r\n(warning: computationally intensive visualizations).\r\n\r\nFlattening the data throws away information about the 2D structure of the image.\r\nIsn't that bad? Well, the best computer vision methods do exploit this\r\nstructure, and we will in later tutorials. But the simple method we will be\r\nusing here, a softmax regression, won't.\r\n\r\nThe result is that `mnist.train.images` is a tensor (an n-dimensional array) with a\r\nshape of `[60000, 784]`. The first dimension indexes the images and the second\r\ndimension indexes the pixels in each image. Each entry in the tensor is the\r\npixel intensity between 0 and 1, for a particular pixel in a particular image.\r\n\r\n\r\n\r\n![](\"img/mnist-train-xs.png\")
\r\n
\r\n\r\nThe corresponding labels in MNIST are numbers between 0 and 9, describing\r\nwhich digit a given image is of.\r\nFor the purposes of this tutorial, we're going to want our labels\r\nas \"one-hot vectors\". A one-hot vector is a vector which is 0 in most\r\ndimensions, and 1 in a single dimension. In this case, the \\\\(n\\\\)th digit will be\r\nrepresented as a vector which is 1 in the \\\\(n\\\\)th dimensions. For example, 3 would be \\\\([0,0,0,1,0,0,0,0,0,0]\\\\).\r\nConsequently, `mnist.train.labels` is a\r\n`[60000, 10]` array of floats.\r\n\r\n\r\n\r\n![](\"img/mnist-train-ys.png\")
\r\n
\r\n\r\nWe're now ready to actually make our model!\r\n\r\n## Softmax Regressions \r\n\r\nWe know that every image in MNIST is a digit, whether it's a zero or a nine. We\r\nwant to be able to look at an image and give probabilities for it being each\r\ndigit. For example, our model might look at a picture of a nine and be 80% sure\r\nit's a nine, but give a 5% chance to it being an eight (because of the top loop)\r\nand a bit of probability to all the others because it isn't sure.\r\n\r\nThis is a classic case where a softmax regression is a natural, simple model.\r\nIf you want to assign probabilities to an object being one of several different\r\nthings, softmax is the thing to do. Even later on, when we train more\r\nsophisticated models, the final step will be a layer of softmax.\r\n\r\nA softmax regression has two steps: first we add up the evidence of our input\r\nbeing in certain classes, and then we convert that evidence into probabilities.\r\n\r\nTo tally up the evidence that a given image is in a particular class, we do a\r\nweighted sum of the pixel intensities. The weight is negative if that pixel\r\nhaving a high intensity is evidence against the image being in that class,\r\nand positive if it is evidence in favor.\r\n\r\nThe following diagram shows the weights one model learned for each of these\r\nclasses. Red represents negative weights, while blue represents positive\r\nweights.\r\n\r\n\r\n\r\n![](\"img/softmax-weights.png\")
\r\n
\r\n\r\nWe also add some extra evidence called a bias. Basically, we want to be able\r\nto say that some things are more likely independent of the input. The result is\r\nthat the evidence for a class \\\\(i\\\\) given an input \\\\(x\\\\) is:\r\n\r\n$$\\text{evidence}_i = \\sum_j W_{i,~ j} x_j + b_i$$\r\n\r\nwhere \\\\(W\\_i\\\\) is the weights and \\\\(b\\_i\\\\) is the bias for class \\\\(i\\\\),\r\nand \\\\(j\\\\) is an index for summing over the pixels in our input image \\\\(x\\\\).\r\nWe then convert the evidence tallies into our predicted probabilities\r\n\\\\(y\\\\) using the \"softmax\" function:\r\n\r\n$$y = \\text{softmax}(\\text{evidence})$$\r\n\r\nHere softmax is serving as an \"activation\" or \"link\" function, shaping\r\nthe output of our linear function into the form we want -- in this case, a\r\nprobability distribution over 10 cases.\r\nYou can think of it as converting tallies\r\nof evidence into probabilities of our input being in each class.\r\nIt's defined as:\r\n\r\n$$\\text{softmax}(x) = \\text{normalize}(\\exp(x))$$\r\n\r\nIf you expand that equation out, you get:\r\n\r\n$$\\text{softmax}(x)_i = \\frac{\\exp(x_i)}{\\sum_j \\exp(x_j)}$$\r\n\r\nBut it's often more helpful to think of softmax the first way:\r\nexponentiating its inputs and then normalizing them. The exponentiation\r\nmeans that one unit more evidence increases the weight given to any hypothesis\r\nmultiplicatively. And conversely, having one less unit of evidence means that a\r\nhypothesis gets a fraction of its earlier weight. No hypothesis ever has zero\r\nor negative weight. Softmax then normalizes these weights, so that they add up\r\nto one, forming a valid probability distribution. (To get more intuition about\r\nthe softmax function, check out the\r\n[section](http://neuralnetworksanddeeplearning.com/chap3.html#softmax)\r\non it in Michael Nieslen's book, complete with an interactive visualization.)\r\n\r\n\r\nYou can picture our softmax regression as looking something like the following,\r\nalthough with a lot more \\\\(x\\\\)s. For each output, we compute a weighted sum of\r\nthe \\\\(x\\\\)s, add a bias, and then apply softmax.\r\n\r\n\r\n\r\n![](\"img/softmax-regression-scalargraph.png\")
\r\n
\r\n\r\nIf we write that out as equations, we get:\r\n\r\n\r\n\r\n![](\"img/softmax-regression-scalarequation.png\")
\r\n
\r\n\r\nWe can \"vectorize\" this procedure, turning it into a matrix multiplication\r\nand vector addition. This is helpful for computational efficiency. (It's also\r\na useful way to think.)\r\n\r\n\r\n\r\n![](\"img/softmax-regression-vectorequation.png\")
\r\n
\r\n\r\nMore compactly, we can just write:\r\n\r\n$$y = \\text{softmax}(Wx + b)$$\r\n\r\n\r\n## Implementing the Regression \r\n\r\n\r\nTo do efficient numerical computing in Python, we typically use libraries like\r\nNumPy that do expensive operations such as matrix multiplication outside Python,\r\nusing highly efficient code implemented in another language.\r\nUnfortunately, there can still be a lot of overhead from switching back to\r\nPython every operation. This overhead is especially bad if you want to run\r\ncomputations on GPUs or in a distributed manner, where there can be a high cost\r\nto transferring data.\r\n\r\nTensorFlow also does its heavy lifting outside python,\r\nbut it takes things a step further to avoid this overhead.\r\nInstead of running a single expensive operation independently\r\nfrom Python, TensorFlow lets us describe a graph of interacting operations that\r\nrun entirely outside Python. (Approaches like this can be seen in a few\r\nmachine learning libraries.)\r\n\r\nTo use TensorFlow, we need to import it.\r\n\r\n```python\r\nimport tensorflow as tf\r\n```\r\n\r\nWe describe these interacting operations by manipulating symbolic variables.\r\nLet's create one:\r\n\r\n```python\r\nx = tf.placeholder(\"float\", [None, 784])\r\n```\r\n\r\n`x` isn't a specific value. It's a `placeholder`, a value that we'll input when\r\nwe ask TensorFlow to run a computation. We want to be able to input any number\r\nof MNIST images, each flattened into a 784-dimensional vector. We represent\r\nthis as a 2d tensor of floating point numbers, with a shape `[None, 784]`.\r\n(Here `None` means that a dimension can be of any length.)\r\n\r\nWe also need the weights and biases for our model. We could imagine treating\r\nthese like additional inputs, but TensorFlow has an even better way to handle\r\nit: `Variable`.\r\nA `Variable` is a modifiable tensor that lives in TensorFlow's graph of\r\ninteracting\r\noperations. It can be used and even modified by the computation. For machine\r\nlearning applications, one generally has the model parameters be `Variable`s.\r\n\r\n```python\r\nW = tf.Variable(tf.zeros([784,10]))\r\nb = tf.Variable(tf.zeros([10]))\r\n```\r\n\r\nWe create these `Variable`s by giving `tf.Variable` the initial value of the\r\n`Variable`: in this case, we initialize both `W` and `b` as tensors full of\r\nzeros. Since we are going to learn `W` and `b`, it doesn't matter very much\r\nwhat they initially are.\r\n\r\nNotice that `W` has a shape of [784, 10] because we want to multiply the\r\n784-dimensional image vectors by it to produce 10-dimensional vectors of\r\nevidence for the difference classes. `b` has a shape of [10] so we can add it\r\nto the output.\r\n\r\nWe can now implement our model. It only takes one line!\r\n\r\n```python\r\ny = tf.nn.softmax(tf.matmul(x,W) + b)\r\n```\r\n\r\nFirst, we multiply `x` by `W` with the expression `tf.matmul(x,W)`. This is\r\nflipped from when we multiplied them in our equation, where we had \\\\(Wx\\\\), as a\r\nsmall trick\r\nto deal with `x` being a 2D tensor with multiple inputs. We then add `b`, and\r\nfinally apply `tf.nn.softmax`.\r\n\r\nThat's it. It only took us one line to define our model, after a couple short\r\nlines of setup. That isn't because TensorFlow is designed to make a softmax\r\nregression particularly easy: it's just a very flexible way to describe many\r\nkinds of numerical computations, from machine learning models to physics\r\nsimulations. And once defined, our model can be run on different devices:\r\nyour computer's CPU, GPUs, and even phones!\r\n\r\n\r\n## Training \r\n\r\nIn order to train our model, we need to define what it means for the model to\r\nbe good. Well, actually, in machine learning we typically define what it means\r\nfor a model to be bad, called the cost or loss, and then try to minimize how bad\r\nit is. But the two are equivalent.\r\n\r\nOne very common, very nice cost function is \"cross-entropy.\" Surprisingly,\r\ncross-entropy arises from thinking about information compressing codes in\r\ninformation theory but it winds up being an important idea in lots of areas,\r\nfrom gambling to machine learning. It's defined:\r\n\r\n$$H_{y'}(y) = -\\sum_i y'_i \\log(y_i)$$\r\n\r\nWhere \\\\(y\\\\) is our predicted probability distribution, and \\\\(y'\\\\) is the true\r\ndistribution (the one-hot vector we'll input). In some rough sense, the\r\ncross-entropy is measuring how inefficient our predictions are for describing\r\nthe truth. Going into more detail about cross-entropy is beyond the scope of\r\nthis tutorial, but it's well worth\r\n[understanding](http://colah.github.io/posts/2015-09-Visual-Information/).\r\n\r\nTo implement cross-entropy we need to first add a new placeholder to input\r\nthe correct answers:\r\n\r\n```python\r\ny_ = tf.placeholder(\"float\", [None,10])\r\n```\r\n\r\nThen we can implement the cross-entropy, \\\\(-\\sum y'\\log(y)\\\\):\r\n\r\n```python\r\ncross_entropy = -tf.reduce_sum(y_*tf.log(y))\r\n```\r\n\r\nFirst, `tf.log` computes the logarithm of each element of `y`. Next, we multiply\r\neach element of `y_` with the corresponding element of `tf.log(y)`. Finally,\r\n`tf.reduce_sum` adds all the elements of the tensor. (Note that this isn't\r\njust the cross-entropy of the truth with a single prediction, but the sum of the\r\ncross-entropies for all 100 images we looked at. How well we are doing on 100\r\ndata points is a much better description of how good our model is than a single\r\ndata point.)\r\n\r\nNow that we know what we want our model to do, it's very easy to have TensorFlow\r\ntrain it to do so.\r\nBecause TensorFlow knows the entire graph of your computations, it\r\ncan automatically use the [backpropagation\r\nalgorithm](http://colah.github.io/posts/2015-08-Backprop/)\r\nto efficiently determine how your variables affect the cost you ask it minimize.\r\nThen it can apply your choice of optimization algorithm to modify the variables\r\nand reduce the cost.\r\n\r\n```python\r\ntrain_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)\r\n```\r\n\r\nIn this case, we ask TensorFlow to minimize `cross_entropy` using the gradient\r\ndescent algorithm with a learning rate of 0.01. Gradient descent is a simple\r\nprocedure, where TensorFlow simply shifts each variable a little bit in the\r\ndirection that reduces the cost. But TensorFlow also provides\r\n[many other optimization algorithms]\r\n(../../../api_docs/python/train.md#optimizers): using one is as simple as\r\ntweaking one line.\r\n\r\nWhat TensorFlow actually does here, behind the scenes, is it adds new operations\r\nto your graph which\r\nimplement backpropagation and gradient descent. Then it gives you back a\r\nsingle operation which, when run, will do a step of gradient descent training,\r\nslightly tweaking your variables to reduce the cost.\r\n\r\nNow we have our model set up to train. One last thing before we launch it,\r\nwe have to add an operation to initialize the variables we created:\r\n\r\n```python\r\ninit = tf.initialize_all_variables()\r\n```\r\n\r\nWe can now launch the model in a `Session`, and run the operation that\r\ninitializes the variables:\r\n\r\n```python\r\nsess = tf.Session()\r\nsess.run(init)\r\n```\r\n\r\nLet's train -- we'll run the training step 1000 times!\r\n\r\n```python\r\nfor i in range(1000):\r\n batch_xs, batch_ys = mnist.train.next_batch(100)\r\n sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})\r\n```\r\n\r\nEach step of the loop, we get a \"batch\" of one hundred random data points from\r\nour training set. We run `train_step` feeding in the batches data to replace\r\nthe `placeholder`s.\r\n\r\nUsing small batches of random data is called stochastic training -- in\r\nthis case, stochastic gradient descent. Ideally, we'd like to use all our data\r\nfor every step of training because that would give us a better sense of what\r\nwe should be doing, but that's expensive. So, instead, we use a different subset\r\nevery time. Doing this is cheap and has much of the same benefit.\r\n\r\n\r\n\r\n## Evaluating Our Model \r\n\r\nHow well does our model do?\r\n\r\nWell, first let's figure out where we predicted the correct label. `tf.argmax`\r\nis an extremely useful function which gives you the index of the highest entry\r\nin a tensor along some axis. For example, `tf.argmax(y,1)` is the label our\r\nmodel thinks is most likely for each input, while `tf.argmax(y_,1)` is the\r\ncorrect label. We can use `tf.equal` to check if our prediction matches the\r\ntruth.\r\n\r\n```python\r\ncorrect_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))\r\n```\r\n\r\nThat gives us a list of booleans. To determine what fraction are correct, we\r\ncast to floating point numbers and then take the mean. For example,\r\n`[True, False, True, True]` would become `[1,0,1,1]` which would become `0.75`.\r\n\r\n```python\r\naccuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\r\n```\r\n\r\nFinally, we ask for our accuracy on our test data.\r\n\r\n```python\r\nprint sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels})\r\n```\r\n\r\nThis should be about 91%.\r\n\r\nIs that good? Well, not really. In fact, it's pretty bad. This is because we're\r\nusing a very simple model. With some small changes, we can get to\r\n97%. The best models can get to over 99.7% accuracy! (For more information, have\r\na look at this\r\n[list of results](http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html).)\r\n\r\nWhat matters is that we learned from this model. Still, if you're feeling a bit\r\ndown about these results, check out [the next tutorial](../../../tutorials/overview.md) where we\r\ndo a lot better, and learn how to build more sophisticated models using\r\nTensorFlow!\r\n"
},
{
"path": "SOURCE/tutorials/mnist/download/index.md",
"content": "# MNIST Data Download \r\n\r\nCode: [tensorflow/g3doc/tutorials/mnist/](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/tutorials/mnist/)\r\n\r\nThe goal of this tutorial is to show how to download the dataset files required\r\nfor handwritten digit classification using the (classic) MNIST data set.\r\n\r\n## Tutorial Files \r\n\r\nThis tutorial references the following files:\r\n\r\nFile | Purpose\r\n--- | ---\r\n[`input_data.py`](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/tutorials/mnist/input_data.py) | The code to download the MNIST dataset for training and evaluation.\r\n\r\n## Prepare the Data \r\n\r\nMNIST is a classic problem in machine learning. The problem is to look at\r\ngreyscale 28x28 pixel images of handwritten digits and determine which digit\r\nthe image represents, for all the digits from zero to nine.\r\n\r\n![MNIST Digits](../tf/mnist_digits.png \"MNIST Digits\")\r\n\r\nFor more information, refer to [Yann LeCun's MNIST page](http://yann.lecun.com/exdb/mnist/)\r\nor [Chris Olah's visualizations of MNIST](http://colah.github.io/posts/2014-10-Visualizing-MNIST/).\r\n\r\n### Download \r\n\r\n[Yann LeCun's MNIST page](http://yann.lecun.com/exdb/mnist/)\r\nalso hosts the training and test data for download.\r\n\r\nFile | Purpose\r\n--- | ---\r\n[`train-images-idx3-ubyte.gz`](http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz) | training set images - 55000 training images, 5000 validation images\r\n[`train-labels-idx1-ubyte.gz`](http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz) | training set labels matching the images\r\n[`t10k-images-idx3-ubyte.gz`](http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz) | test set images - 10000 images\r\n[`t10k-labels-idx1-ubyte.gz`](http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz) | test set labels matching the images\r\n\r\nIn the `input_data.py` file, the `maybe_download()` function will ensure these\r\nfiles are downloaded into a local data folder for training.\r\n\r\nThe folder name is specified in a flag variable at the top of the\r\n`fully_connected_feed.py` file and may be changed to fit your needs.\r\n\r\n### Unpack and Reshape \r\n\r\nThe files themselves are not in any standard image format and are manually\r\nunpacked (following the instructions available at the website) by the\r\n`extract_images()` and `extract_labels()` functions in `input_data.py`.\r\n\r\nThe image data is extracted into a 2d tensor of: `[image index, pixel index]`\r\nwhere each entry is the intensity value of a specific pixel in a specific\r\nimage, rescaled from `[0, 255]` to `[-0.5, 0.5]`. The \"image index\" corresponds\r\nto an image in the dataset, counting up from zero to the size of the dataset.\r\nAnd the \"pixel index\" corresponds to a specific pixel in that image, ranging\r\nfrom zero to the number of pixels in the image.\r\n\r\nThe 60000 examples in the `train-*` files are then split into 55000 examples\r\nfor training and 5000 examples for validation. For all of the 28x28\r\npixel greyscale images in the datasets the image size is 784 and so the output\r\ntensor for the training set images is of shape `[55000, 784]`.\r\n\r\nThe label data is extracted into a 1d tensor of: `[image index]`\r\nwith the class identifier for each example as the value. For the training set\r\nlabels, this would then be of shape `[55000]`.\r\n\r\n### DataSet Object \r\n\r\nThe underlying code will download, unpack, and reshape images and labels for\r\nthe following datasets:\r\n\r\nDataset | Purpose\r\n--- | ---\r\n`data_sets.train` | 55000 images and labels, for primary training.\r\n`data_sets.validation` | 5000 images and labels, for iterative validation of training accuracy.\r\n`data_sets.test` | 10000 images and labels, for final testing of trained accuracy.\r\n\r\nThe `read_data_sets()` function will return a dictionary with a `DataSet`\r\ninstance for each of these three sets of data. The `DataSet.next_batch()`\r\nmethod can be used to fetch a tuple consisting of `batch_size` lists of images\r\nand labels to be fed into the running TensorFlow session.\r\n\r\n```python\r\nimages_feed, labels_feed = data_set.next_batch(FLAGS.batch_size)\r\n```\r\n"
},
{
"path": "SOURCE/tutorials/mnist/fully_connected_feed.py",
"content": "\"\"\"Trains and Evaluates the MNIST network using a feed dictionary.\r\n\r\nTensorFlow install instructions:\r\nhttps://tensorflow.org/get_started/os_setup.html\r\n\r\nMNIST tutorial:\r\nhttps://tensorflow.org/tutorials/mnist/tf/index.html\r\n\r\n\"\"\"\r\nfrom __future__ import print_function\r\n# pylint: disable=missing-docstring\r\nimport os.path\r\nimport time\r\n\r\nimport tensorflow.python.platform\r\nimport numpy\r\nimport tensorflow as tf\r\n\r\nfrom tensorflow.g3doc.tutorials.mnist import input_data\r\nfrom tensorflow.g3doc.tutorials.mnist import mnist\r\n\r\n\r\n# Basic model parameters as external flags.\r\nflags = tf.app.flags\r\nFLAGS = flags.FLAGS\r\nflags.DEFINE_float('learning_rate', 0.01, 'Initial learning rate.')\r\nflags.DEFINE_integer('max_steps', 2000, 'Number of steps to run trainer.')\r\nflags.DEFINE_integer('hidden1', 128, 'Number of units in hidden layer 1.')\r\nflags.DEFINE_integer('hidden2', 32, 'Number of units in hidden layer 2.')\r\nflags.DEFINE_integer('batch_size', 100, 'Batch size. '\r\n 'Must divide evenly into the dataset sizes.')\r\nflags.DEFINE_string('train_dir', 'data', 'Directory to put the training data.')\r\nflags.DEFINE_boolean('fake_data', False, 'If true, uses fake data '\r\n 'for unit testing.')\r\n\r\n\r\ndef placeholder_inputs(batch_size):\r\n \"\"\"Generate placeholder variables to represent the the input tensors.\r\n\r\n These placeholders are used as inputs by the rest of the model building\r\n code and will be fed from the downloaded data in the .run() loop, below.\r\n\r\n Args:\r\n batch_size: The batch size will be baked into both placeholders.\r\n\r\n Returns:\r\n images_placeholder: Images placeholder.\r\n labels_placeholder: Labels placeholder.\r\n \"\"\"\r\n # Note that the shapes of the placeholders match the shapes of the full\r\n # image and label tensors, except the first dimension is now batch_size\r\n # rather than the full size of the train or test data sets.\r\n images_placeholder = tf.placeholder(tf.float32, shape=(batch_size,\r\n mnist.IMAGE_PIXELS))\r\n labels_placeholder = tf.placeholder(tf.int32, shape=(batch_size))\r\n return images_placeholder, labels_placeholder\r\n\r\n\r\ndef fill_feed_dict(data_set, images_pl, labels_pl):\r\n \"\"\"Fills the feed_dict for training the given step.\r\n\r\n A feed_dict takes the form of:\r\n feed_dict = {\r\n \r\n\r\n ![](\"./mnist_subgraph.png\")
\r\n
\r\n\r\n### Inference \r\n\r\nThe `inference()` function builds the graph as far as needed to\r\nreturn the tensor that would contain the output predictions.\r\n\r\nIt takes the images placeholder as input and builds on top\r\nof it a pair of fully connected layers with ReLu activation followed by a ten\r\nnode linear layer specifying the output logits.\r\n\r\nEach layer is created beneath a unique [`tf.name_scope`](../../../api_docs/python/framework.md#name_scope)\r\nthat acts as a prefix to the items created within that scope.\r\n\r\n```python\r\nwith tf.name_scope('hidden1') as scope:\r\n```\r\n\r\nWithin the defined scope, the weights and biases to be used by each of these\r\nlayers are generated into [`tf.Variable`](../../../api_docs/python/state_ops.md#Variable)\r\ninstances, with their desired shapes:\r\n\r\n```python\r\nweights = tf.Variable(\r\n tf.truncated_normal([IMAGE_PIXELS, hidden1_units],\r\n stddev=1.0 / math.sqrt(float(IMAGE_PIXELS))),\r\n name='weights')\r\nbiases = tf.Variable(tf.zeros([hidden1_units]),\r\n name='biases')\r\n```\r\n\r\nWhen, for instance, these are created under the `hidden1` scope, the unique\r\nname given to the weights variable would be \"`hidden1/weights`\".\r\n\r\nEach variable is given initializer ops as part of their construction.\r\n\r\nIn this most common case, the weights are initialized with the\r\n[`tf.truncated_normal`](../../../api_docs/python/constant_op.md#truncated_normal)\r\nand given their shape of a 2-D tensor with\r\nthe first dim representing the number of units in the layer from which the\r\nweights connect and the second dim representing the number of\r\nunits in the layer to which the weights connect. For the first layer, named\r\n`hidden1`, the dimensions are `[IMAGE_PIXELS, hidden1_units]` because the\r\nweights are connecting the image inputs to the hidden1 layer. The\r\n`tf.truncated_normal` initializer generates a random distribution with a given\r\nmean and standard deviation.\r\n\r\nThen the biases are initialized with [`tf.zeros`](../../../api_docs/python/constant_op.md#zeros)\r\nto ensure they start with all zero values, and their shape is simply the number\r\nof units in the layer to which they connect.\r\n\r\nThe graph's three primary ops -- two [`tf.nn.relu`](../../../api_docs/python/nn.md#relu)\r\nops wrapping [`tf.matmul`](../../../api_docs/python/math_ops.md#matmul)\r\nfor the hidden layers and one extra `tf.matmul` for the logits -- are then\r\ncreated, each in turn, with their `tf.Variable` instances connected to the\r\ninput placeholder or the output tensor of the layer beneath each.\r\n\r\n```python\r\nhidden1 = tf.nn.relu(tf.matmul(images, weights) + biases)\r\n```\r\n\r\n```python\r\nhidden2 = tf.nn.relu(tf.matmul(hidden1, weights) + biases)\r\n```\r\n\r\n```python\r\nlogits = tf.matmul(hidden2, weights) + biases\r\n```\r\n\r\nFinally, the `logits` tensor that will contain the output is returned.\r\n\r\n### Loss \r\n\r\nThe `loss()` function further builds the graph by adding the required loss\r\nops.\r\n\r\nFirst, the values from the `labels_placeholder` are encoded as a tensor of 1-hot\r\nvalues. For example, if the class identifier is '3' the value is converted to:\r\n\r\n`[0, 0, 0, 1, 0, 0, 0, 0, 0, 0]`\r\n\r\n```python\r\nbatch_size = tf.size(labels)\r\nlabels = tf.expand_dims(labels, 1)\r\nindices = tf.expand_dims(tf.range(0, batch_size, 1), 1)\r\nconcated = tf.concat(1, [indices, labels])\r\nonehot_labels = tf.sparse_to_dense(\r\n concated, tf.pack([batch_size, NUM_CLASSES]), 1.0, 0.0)\r\n```\r\n\r\nA [`tf.nn.softmax_cross_entropy_with_logits`](../../../api_docs/python/nn.md#softmax_cross_entropy_with_logits)\r\nop is then added to compare the output logits from the `inference()` function\r\nand the 1-hot labels.\r\n\r\n```python\r\ncross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits,\r\n onehot_labels,\r\n name='xentropy')\r\n```\r\n\r\nIt then uses [`tf.reduce_mean`](../../../api_docs/python/math_ops.md#reduce_mean)\r\nto average the cross entropy values across the batch dimension (the first\r\ndimension) as the total loss.\r\n\r\n```python\r\nloss = tf.reduce_mean(cross_entropy, name='xentropy_mean')\r\n```\r\n\r\nAnd the tensor that will then contain the loss value is returned.\r\n\r\n> Note: Cross-entropy is an idea from information theory that allows us\r\n> to describe how bad it is to believe the predictions of the neural network,\r\n> given what is actually true. For more information, read the blog post Visual\r\n> Information Theory (http://colah.github.io/posts/2015-09-Visual-Information/)\r\n\r\n### Training \r\n\r\nThe `training()` function adds the operations needed to minimize the loss via\r\ngradient descent.\r\n\r\nFirstly, it takes the loss tensor from the `loss()` function and hands it to a\r\n[`tf.scalar_summary`](../../../api_docs/python/train.md#scalar_summary),\r\nan op for generating summary values into the events file when used with a\r\n`SummaryWriter` (see below). In this case, it will emit the snapshot value of\r\nthe loss every time the summaries are written out.\r\n\r\n```python\r\ntf.scalar_summary(loss.op.name, loss)\r\n```\r\n\r\nNext, we instantiate a [`tf.train.GradientDescentOptimizer`](../../../api_docs/python/train.md#GradientDescentOptimizer)\r\nresponsible for applying gradients with the requested learning rate.\r\n\r\n```python\r\noptimizer = tf.train.GradientDescentOptimizer(FLAGS.learning_rate)\r\n```\r\n\r\nWe then generate a single variable to contain a counter for the global\r\ntraining step and the [`minimize()`](../../../api_docs/python/train.md#Optimizer.minimize)\r\nop is used to both update the trainable weights in the system and increment the\r\nglobal step. This is, by convention, known as the `train_op` and is what must\r\nbe run by a TensorFlow session in order to induce one full step of training\r\n(see below).\r\n\r\n```python\r\nglobal_step = tf.Variable(0, name='global_step', trainable=False)\r\ntrain_op = optimizer.minimize(loss, global_step=global_step)\r\n```\r\n\r\nThe tensor containing the outputs of the training op is returned.\r\n\r\n## Train the Model \r\n\r\nOnce the graph is built, it can be iteratively trained and evaluated in a loop\r\ncontrolled by the user code in `fully_connected_feed.py`.\r\n\r\n### The Graph \r\n\r\nAt the top of the `run_training()` function is a python `with` command that\r\nindicates all of the built ops are to be associated with the default\r\nglobal [`tf.Graph`](../../../api_docs/python/framework.md#Graph)\r\ninstance.\r\n\r\n```python\r\nwith tf.Graph().as_default():\r\n```\r\n\r\nA `tf.Graph` is a collection of ops that may be executed together as a group.\r\nMost TensorFlow uses will only need to rely on the single default graph.\r\n\r\nMore complicated uses with multiple graphs are possible, but beyond the scope of\r\nthis simple tutorial.\r\n\r\n### The Session \r\n\r\nOnce all of the build preparation has been completed and all of the necessary\r\nops generated, a [`tf.Session`](../../../api_docs/python/client.md#Session)\r\nis created for running the graph.\r\n\r\n```python\r\nsess = tf.Session()\r\n```\r\n\r\nAlternately, a `Session` may be generated into a `with` block for scoping:\r\n\r\n```python\r\nwith tf.Session() as sess:\r\n```\r\n\r\nThe empty parameter to session indicates that this code will attach to\r\n(or create if not yet created) the default local session.\r\n\r\nImmediately after creating the session, all of the `tf.Variable`\r\ninstances are initialized by calling [`sess.run()`](../../../api_docs/python/client.md#Session.run)\r\non their initialization op.\r\n\r\n```python\r\ninit = tf.initialize_all_variables()\r\nsess.run(init)\r\n```\r\n\r\nThe [`sess.run()`](../../../api_docs/python/client.md#Session.run)\r\nmethod will run the complete subset of the graph that\r\ncorresponds to the op(s) passed as parameters. In this first call, the `init`\r\nop is a [`tf.group`](../../../api_docs/python/control_flow_ops.md#group)\r\nthat contains only the initializers for the variables. None of the rest of the\r\ngraph is run here; that happens in the training loop below.\r\n\r\n### Train Loop \r\n\r\nAfter initializing the variables with the session, training may begin.\r\n\r\nThe user code controls the training per step, and the simplest loop that\r\ncan do useful training is:\r\n\r\n```python\r\nfor step in xrange(max_steps):\r\n sess.run(train_op)\r\n```\r\n\r\nHowever, this tutorial is slightly more complicated in that it must also slice\r\nup the input data for each step to match the previously generated placeholders.\r\n\r\n#### Feed the Graph \r\n\r\nFor each step, the code will generate a feed dictionary that will contain the\r\nset of examples on which to train for the step, keyed by the placeholder\r\nops they represent.\r\n\r\nIn the `fill_feed_dict()` function, the given `DataSet` is queried for its next\r\n`batch_size` set of images and labels, and tensors matching the placeholders are\r\nfilled containing the next images and labels.\r\n\r\n```python\r\nimages_feed, labels_feed = data_set.next_batch(FLAGS.batch_size)\r\n```\r\n\r\nA python dictionary object is then generated with the placeholders as keys and\r\nthe representative feed tensors as values.\r\n\r\n```python\r\nfeed_dict = {\r\n images_placeholder: images_feed,\r\n labels_placeholder: labels_feed,\r\n}\r\n```\r\n\r\nThis is passed into the `sess.run()` function's `feed_dict` parameter to provide\r\nthe input examples for this step of training.\r\n\r\n#### Check the Status \r\n\r\nThe code specifies two values to fetch in its run call: `[train_op, loss]`.\r\n\r\n```python\r\nfor step in xrange(FLAGS.max_steps):\r\n feed_dict = fill_feed_dict(data_sets.train,\r\n images_placeholder,\r\n labels_placeholder)\r\n _, loss_value = sess.run([train_op, loss],\r\n feed_dict=feed_dict)\r\n```\r\n\r\nBecause there are two values to fetch, `sess.run()` returns a tuple with two\r\nitems. Each `Tensor` in the list of values to fetch corresponds to a numpy\r\narray in the returned tuple, filled with the value of that tensor during this\r\nstep of training. Since `train_op` is an `Operation` with no output value, the\r\ncorresponding element in the returned tuple is `None` and, thus,\r\ndiscarded. However, the value of the `loss` tensor may become NaN if the model\r\ndiverges during training, so we capture this value for logging.\r\n\r\nAssuming that the training runs fine without NaNs, the training loop also\r\nprints a simple status text every 100 steps to let the user know the state of\r\ntraining.\r\n\r\n```python\r\nif step % 100 == 0:\r\n print 'Step %d: loss = %.2f (%.3f sec)' % (step, loss_value, duration)\r\n```\r\n\r\n#### Visualize the Status \r\n\r\nIn order to emit the events files used by [TensorBoard](../../../how_tos/summaries_and_tensorboard/index.md),\r\nall of the summaries (in this case, only one) are collected into a single op\r\nduring the graph building phase.\r\n\r\n```python\r\nsummary_op = tf.merge_all_summaries()\r\n```\r\n\r\nAnd then after the session is created, a [`tf.train.SummaryWriter`](../../../api_docs/python/train.md#SummaryWriter)\r\nmay be instantiated to write the events files, which\r\ncontain both the graph itself and the values of the summaries.\r\n\r\n```python\r\nsummary_writer = tf.train.SummaryWriter(FLAGS.train_dir,\r\n graph_def=sess.graph_def)\r\n```\r\n\r\nLastly, the events file will be updated with new summary values every time the\r\n`summary_op` is run and the ouput passed to the writer's `add_summary()`\r\nfunction.\r\n\r\n```python\r\nsummary_str = sess.run(summary_op, feed_dict=feed_dict)\r\nsummary_writer.add_summary(summary_str, step)\r\n```\r\n\r\nWhen the events files are written, TensorBoard may be run against the training\r\nfolder to display the values from the summaries.\r\n\r\n![MNIST TensorBoard](./mnist_tensorboard.png \"MNIST TensorBoard\")\r\n\r\n**NOTE**: For more info about how to build and run Tensorboard, please see the accompanying tutorial [Tensorboard: Visualizing Your Training](../../../how_tos/summaries_and_tensorboard/index.md).\r\n\r\n#### Save a Checkpoint \r\n\r\nIn order to emit a checkpoint file that may be used to later restore a model\r\nfor further training or evaluation, we instantiate a\r\n[`tf.train.Saver`](../../../api_docs/python/state_ops.md#Saver).\r\n\r\n```python\r\nsaver = tf.train.Saver()\r\n```\r\n\r\nIn the training loop, the [`saver.save()`](../../../api_docs/python/state_ops.md#Saver.save)\r\nmethod will periodically be called to write a checkpoint file to the training\r\ndirectory with the current values of all the trainable variables.\r\n\r\n```python\r\nsaver.save(sess, FLAGS.train_dir, global_step=step)\r\n```\r\n\r\nAt some later point in the future, training might be resumed by using the\r\n[`saver.restore()`](../../../api_docs/python/state_ops.md#Saver.restore)\r\nmethod to reload the model parameters.\r\n\r\n```python\r\nsaver.restore(sess, FLAGS.train_dir)\r\n```\r\n\r\n## Evaluate the Model \r\n\r\nEvery thousand steps, the code will attempt to evaluate the model against both\r\nthe training and test datasets. The `do_eval()` function is called thrice, for\r\nthe training, validation, and test datasets.\r\n\r\n```python\r\nprint 'Training Data Eval:'\r\ndo_eval(sess,\r\n eval_correct,\r\n images_placeholder,\r\n labels_placeholder,\r\n data_sets.train)\r\nprint 'Validation Data Eval:'\r\ndo_eval(sess,\r\n eval_correct,\r\n images_placeholder,\r\n labels_placeholder,\r\n data_sets.validation)\r\nprint 'Test Data Eval:'\r\ndo_eval(sess,\r\n eval_correct,\r\n images_placeholder,\r\n labels_placeholder,\r\n data_sets.test)\r\n```\r\n\r\n> Note that more complicated usage would usually sequester the `data_sets.test`\r\n> to only be checked after significant amounts of hyperparameter tuning. For\r\n> the sake of a simple little MNIST problem, however, we evaluate against all of\r\n> the data.\r\n\r\n### Build the Eval Graph \r\n\r\nBefore opening the default Graph, the test data should have been fetched by\r\ncalling the `get_data(train=False)` function with the parameter set to grab\r\nthe test dataset.\r\n\r\n```python\r\ntest_all_images, test_all_labels = get_data(train=False)\r\n```\r\n\r\nBefore entering the training loop, the Eval op should have been built\r\nby calling the `evaluation()` function from `mnist.py` with the same\r\nlogits/labels parameters as the `loss()` function.\r\n\r\n```python\r\neval_correct = mnist.evaluation(logits, labels_placeholder)\r\n```\r\n\r\nThe `evaluation()` function simply generates a [`tf.nn.in_top_k`](../../../api_docs/python/nn.md#in_top_k)\r\nop that can automatically score each model output as correct if the true label\r\ncan be found in the K most-likely predictions. In this case, we set the value\r\nof K to 1 to only consider a prediction correct if it is for the true label.\r\n\r\n```python\r\neval_correct = tf.nn.in_top_k(logits, labels, 1)\r\n```\r\n\r\n### Eval Output \r\n\r\nOne can then create a loop for filling a `feed_dict` and calling `sess.run()`\r\nagainst the `eval_correct` op to evaluate the model on the given dataset.\r\n\r\n```python\r\nfor step in xrange(steps_per_epoch):\r\n feed_dict = fill_feed_dict(data_set,\r\n images_placeholder,\r\n labels_placeholder)\r\n true_count += sess.run(eval_correct, feed_dict=feed_dict)\r\n```\r\n\r\nThe `true_count` variable simply accumulates all of the predictions that the\r\n`in_top_k` op has determined to be correct. From there, the precision may be\r\ncalculated from simply dividing by the total number of examples.\r\n\r\n```python\r\nprecision = float(true_count) / float(num_examples)\r\nprint ' Num examples: %d Num correct: %d Precision @ 1: %0.02f' % (\r\n num_examples, true_count, precision)\r\n```\r\n" }, { "path": "SOURCE/tutorials/mnist_beginners.md", "content": "# MNIST机器学习入门 \r\n\r\n*这个教程的目标读者是对机器学习和TensorFlow都不太了解的新手。如果你已经了解MNIST和softmax回归(softmax regression)的相关知识，你可以阅读这个[快速上手教程](./mnist_pros.md)。*\r\n \r\n当我们开始学习编程的时候，第一件事往往是学习打印\"Hello World\"。就好比编程入门有Hello World，机器学习入门有MNIST。\r\n\r\nMNIST是一个入门级的计算机视觉数据集，它包含各种手写数字图片：\r\n
\r\n![](\"../images/MNIST.png\")
\r\n
\r\n \r\n它也包含每一张图片对应的标签，告诉我们这个是数字几。比如，上面这四张图片的标签分别是5，0，4，1。\r\n \r\n在此教程中，我们将训练一个机器学习模型用于预测图片里面的数字。我们的目的不是要设计一个世界一流的复杂模型 -- 尽管我们会在之后给你源代码去实现一流的预测模型 -- 而是要介绍下如何使用TensorFlow。所以，我们这里会从一个很简单的数学模型开始，它叫做Softmax Regression。\r\n\r\n对应这个教程的实现代码很短，而且真正有意思的内容只包含在三行代码里面。但是，去理解包含在这些代码里面的设计思想是非常重要的：TensorFlow工作流程和机器学习的基本概念。因此，这个教程会很详细地介绍这些代码的实现原理。\r\n\r\n## MNIST数据集 \r\n\r\nMNIST数据集的官网是[Yann LeCun's website](http://yann.lecun.com/exdb/mnist/)。在这里，我们提供了一份python源代码用于自动下载和安装这个数据集。你可以下载[这份代码](https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/examples/tutorials/mnist/input_data.py)，然后用下面的代码导入到你的项目里面，也可以直接复制粘贴到你的代码文件里面。\r\n\r\n```python\r\nimport tensorflow.examples.tutorials.mnist.input_data as input_data\r\nmnist = input_data.read_data_sets(\"MNIST_data/\", one_hot=True)\r\n```\r\n\r\n下载下来的数据集被分成两部分：60000行的训练数据集（`mnist.train`）和10000行的测试数据集（`mnist.test`）。这样的切分很重要，在机器学习模型设计时必须有一个单独的测试数据集不用于训练而是用来评估这个模型的性能，从而更加容易把设计的模型推广到其他数据集上（泛化）。\r\n\r\n正如前面提到的一样，每一个MNIST数据单元有两部分组成：一张包含手写数字的图片和一个对应的标签。我们把这些图片设为“xs”，把这些标签设为“ys”。训练数据集和测试数据集都包含xs和ys，比如训练数据集的图片是 `mnist.train.images` ，训练数据集的标签是 `mnist.train.labels`。\r\n \r\n每一张图片包含28X28个像素点。我们可以用一个数字数组来表示这张图片：\r\n\r\n\r\n\r\n![](\"../images/MNIST-Matrix.png\")
\r\n
\r\n\r\n我们把这个数组展开成一个向量，长度是 28x28 = 784。如何展开这个数组（数字间的顺序）不重要，只要保持各个图片采用相同的方式展开。从这个角度来看，MNIST数据集的图片就是在784维向量空间里面的点, 并且拥有比较[复杂的结构](http://colah.github.io/posts/2014-10-Visualizing-MNIST/) (提醒: 此类数据的可视化是计算密集型的)。 \r\n\r\n展平图片的数字数组会丢失图片的二维结构信息。这显然是不理想的，最优秀的计算机视觉方法会挖掘并利用这些结构信息，我们会在后续教程中介绍。但是在这个教程中我们忽略这些结构，所介绍的简单数学模型，softmax回归(softmax regression)，不会利用这些结构信息。\r\n \r\n因此，在MNIST训练数据集中，`mnist.train.images` 是一个形状为 `[60000, 784]` 的张量，第一个维度数字用来索引图片，第二个维度数字用来索引每张图片中的像素点。在此张量里的每一个元素，都表示某张图片里的某个像素的强度值，值介于0和1之间。\r\n\r\n\r\n\r\n![](\"../images/mnist-train-xs.png\")
\r\n
\r\n\r\n相对应的MNIST数据集的标签是介于0到9的数字，用来描述给定图片里表示的数字。为了用于这个教程，我们使标签数据是\"one-hot vectors\"。 一个one-hot向量除了某一位的数字是1以外其余各维度数字都是0。所以在此教程中，数字n将表示成一个只有在第n维度（从0开始）数字为1的10维向量。比如，标签0将表示成([1,0,0,0,0,0,0,0,0,0,0])。因此， `mnist.train.labels` 是一个 `[60000, 10]` 的数字矩阵。 \r\n\r\n\r\n\r\n![](\"../images/mnist-train-ys.png\")
\r\n
\r\n\r\n现在，我们准备好可以开始构建我们的模型啦！\r\n\r\n## Softmax回归介绍 \r\n \r\n我们知道MNIST的每一张图片都表示一个数字，从0到9。我们希望得到给定图片代表每个数字的概率。比如说，我们的模型可能推测一张包含9的图片代表数字9的概率是80%但是判断它是8的概率是5%（因为8和9都有上半部分的小圆），然后给予它代表其他数字的概率更小的值。\r\n\r\n这是一个使用softmax回归（softmax regression）模型的经典案例。softmax模型可以用来给不同的对象分配概率。即使在之后，我们训练更加精细的模型时，最后一步也需要用softmax来分配概率。\r\n \r\nsoftmax回归（softmax regression）分两步：第一步\r\n \r\n为了得到一张给定图片属于某个特定数字类的证据（evidence），我们对图片像素值进行加权求和。如果这个像素具有很强的证据说明这张图片不属于该类，那么相应的权值为负数，相反如果这个像素拥有有利的证据支持这张图片属于这个类，那么权值是正数。\r\n\r\n下面的图片显示了一个模型学习到的图片上每个像素对于特定数字类的权值。红色代表负数权值，蓝色代表正数权值。\r\n\r\n\r\n\r\n![](\"../images/softmax-weights.png\")
\r\n
\r\n\r\n我们也需要加入一个额外的偏置量（bias），因为输入往往会带有一些无关的干扰量。因此对于给定的输入图片 **x** 它代表的是数字 **i** 的证据可以表示为\r\n\r\n\r\n\r\n其中  代表权重， 代表数字 **i** 类的偏置量，**j** 代表给定图片 **x** 的像素索引用于像素求和。然后用softmax函数可以把这些证据转换成概率 **y**：\r\n\r\n\r\n\r\n这里的softmax可以看成是一个激励（activation）函数或者链接（link）函数，把我们定义的线性函数的输出转换成我们想要的格式，也就是关于10个数字类的概率分布。因此，给定一张图片，它对于每一个数字的吻合度可以被softmax函数转换成为一个概率值。softmax函数可以定义为：\r\n\r\n\r\n\r\n展开等式右边的子式，可以得到：\r\n\r\n\r\n \r\n但是更多的时候把softmax模型函数定义为前一种形式：把输入值当成幂指数求值，再正则化这些结果值。这个幂运算表示，更大的证据对应更大的假设模型（hypothesis）里面的乘数权重值。反之，拥有更少的证据意味着在假设模型里面拥有更小的乘数系数。假设模型里的权值不可以是0值或者负值。Softmax然后会正则化这些权重值，使它们的总和等于1，以此构造一个有效的概率分布。（更多的关于Softmax函数的信息，可以参考Michael Nieslen的书里面的这个[部分](http://neuralnetworksanddeeplearning.com/chap3.html#softmax)，其中有关于softmax的可交互式的可视化解释。）\r\n\r\n对于softmax回归模型可以用下面的图解释，对于输入的`xs`加权求和，再分别加上一个偏置量，最后再输入到softmax函数中：\r\n\r\n\r\n\r\n![](\"../images/softmax-regression-scalargraph.png\")
\r\n
\r\n\r\n如果把它写成一个等式，我们可以得到：\r\n\r\n\r\n\r\n![](\"../images/softmax-regression-scalarequation.png\")
\r\n
\r\n\r\n我们也可以用向量表示这个计算过程：用矩阵乘法和向量相加。这有助于提高计算效率。（也是一种更有效的思考方式）\r\n\r\n\r\n\r\n![](\"../images/softmax-regression-vectorequation.png\")
\r\n
\r\n\r\n更进一步，可以写成更加紧凑的方式：\r\n\r\n\r\n\r\n## 实现回归模型 \r\n\r\n为了用python实现高效的数值计算，我们通常会使用函数库，比如NumPy，会把类似矩阵乘法这样的复杂运算使用其他外部语言实现。不幸的是，从外部计算切换回Python的每一个操作，仍然是一个很大的开销。如果你用GPU来进行外部计算，这样的开销会更大。用分布式的计算方式，也会花费更多的资源用来传输数据。\r\n\r\nTensorFlow也把复杂的计算放在python之外完成，但是为了避免前面说的那些开销，它做了进一步完善。Tensorflow不单独地运行单一的复杂计算，而是让我们可以先用图描述一系列可交互的计算操作，然后全部一起在Python之外运行。（这样类似的运行方式，可以在不少的机器学习库中看到。）\r\n\r\n使用TensorFlow之前，首先导入它：\r\n\r\n```python\r\nimport tensorflow as tf\r\n```\r\n\r\n我们通过操作符号变量来描述这些可交互的操作单元，可以用下面的方式创建一个：\r\n\r\n```python\r\nx = tf.placeholder(tf.float32, [None, 784])\r\n```\r\n\r\n`x`不是一个特定的值，而是一个占位符`placeholder`，我们在TensorFlow运行计算时输入这个值。我们希望能够输入任意数量的MNIST图像，每一张图展平成784维的向量。我们用2维的浮点数张量来表示这些图，这个张量的形状是`[None，784 ]`。（这里的`None`表示此张量的第一个维度可以是任何长度的。）\r\n\r\n我们的模型也需要权重值和偏置量，当然我们可以把它们当做是另外的输入（使用占位符），但TensorFlow有一个更好的方法来表示它们：`Variable` 。\r\n一个`Variable`代表一个可修改的张量，存在在TensorFlow的用于描述交互性操作的图中。它们可以用于计算输入值，也可以在计算中被修改。对于各种机器学习应用，一般都会有模型参数，可以用`Variable`表示。\r\n\r\n```python\r\nW = tf.Variable(tf.zeros([784,10]))\r\nb = tf.Variable(tf.zeros([10]))\r\n```\r\n\r\n我们赋予`tf.Variable`不同的初值来创建不同的`Variable`：在这里，我们都用全为零的张量来初始化`W`和`b`。因为我们要学习`W`和`b`的值，它们的初值可以随意设置。\r\n\r\n注意，`W`的维度是[784，10]，因为我们想要用784维的图片向量乘以它以得到一个10维的证据值向量，每一位对应不同数字类。`b`的形状是[10]，所以我们可以直接把它加到输出上面。\r\n\r\n现在，我们可以实现我们的模型啦。只需要一行代码！\r\n\r\n```python\r\ny = tf.nn.softmax(tf.matmul(x,W) + b)\r\n```\r\n\r\n首先，我们用`tf.matmul(X，W)`表示`x`乘以`W`，对应之前等式里面的，这里`x`是一个2维张量拥有多个输入。然后再加上`b`，把和输入到`tf.nn.softmax`函数里面。\r\n\r\n至此，我们先用了几行简短的代码来设置变量，然后只用了一行代码来定义我们的模型。TensorFlow不仅仅可以使softmax回归模型计算变得特别简单，它也用这种非常灵活的方式来描述其他各种数值计算，从机器学习模型对物理学模拟仿真模型。一旦被定义好之后，我们的模型就可以在不同的设备上运行：计算机的CPU，GPU，甚至是手机！\r\n\r\n## 训练模型 \r\n\r\n为了训练我们的模型，我们首先需要定义一个指标来评估这个模型是好的。其实，在机器学习，我们通常定义指标来表示一个模型是坏的，这个指标称为成本（cost）或损失（loss），然后尽量最小化这个指标。但是，这两种方式是相同的。\r\n\r\n一个非常常见的，非常漂亮的成本函数是“交叉熵”（cross-entropy）。交叉熵产生于信息论里面的信息压缩编码技术，但是它后来演变成为从博弈论到机器学习等其他领域里的重要技术手段。它的定义如下：\r\n\r\n\r\n\r\n**y** 是我们预测的概率分布, **y'** 是实际的分布（我们输入的one-hot vector)。比较粗糙的理解是，交叉熵是用来衡量我们的预测用于描述真相的低效性。更详细的关于交叉熵的解释超出本教程的范畴，但是你很有必要好好[理解它](http://colah.github.io/posts/2015-09-Visual-Information/)。\r\n\r\n为了计算交叉熵，我们首先需要添加一个新的占位符用于输入正确值：\r\n\r\n```python\r\ny_ = tf.placeholder(\"float\", [None,10])\r\n```\r\n\r\n然后我们可以用  计算交叉熵:\r\n\r\n```python\r\ncross_entropy = -tf.reduce_sum(y_*tf.log(y))\r\n```\r\n\r\n首先，用 `tf.log` 计算 `y` 的每个元素的对数。接下来，我们把 `y_` 的每一个元素和 `tf.log(y)` 的对应元素相乘。最后，用 `tf.reduce_sum` 计算张量的所有元素的总和。（注意，这里的交叉熵不仅仅用来衡量单一的一对预测和真实值，而是所有100幅图片的交叉熵的总和。对于100个数据点的预测表现比单一数据点的表现能更好地描述我们的模型的性能。\r\n\r\n现在我们知道我们需要我们的模型做什么啦，用TensorFlow来训练它是非常容易的。因为TensorFlow拥有一张描述你各个计算单元的图，它可以自动地使用[反向传播算法(backpropagation algorithm)](http://colah.github.io/posts/2015-08-Backprop/)来有效地确定你的变量是如何影响你想要最小化的那个成本值的。然后，TensorFlow会用你选择的优化算法来不断地修改变量以降低成本。\r\n\r\n```python\r\ntrain_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)\r\n```\r\n\r\n在这里，我们要求TensorFlow用梯度下降算法（gradient descent algorithm）以0.01的学习速率最小化交叉熵。梯度下降算法（gradient descent algorithm）是一个简单的学习过程，TensorFlow只需将每个变量一点点地往使成本不断降低的方向移动。当然TensorFlow也提供了[其他许多优化算法](../api_docs/python/train.md#optimizers)：只要简单地调整一行代码就可以使用其他的算法。\r\n\r\nTensorFlow在这里实际上所做的是，它会在后台给描述你的计算的那张图里面增加一系列新的计算操作单元用于实现反向传播算法和梯度下降算法。然后，它返回给你的只是一个单一的操作，当运行这个操作时，它用梯度下降算法训练你的模型，微调你的变量，不断减少成本。\r\n\r\n现在，我们已经设置好了我们的模型。在运行计算之前，我们需要添加一个操作来初始化我们创建的变量：\r\n\r\n```python\r\ninit = tf.initialize_all_variables()\r\n```\r\n\r\n现在我们可以在一个`Session`里面启动我们的模型，并且初始化变量：\r\n\r\n```python\r\nsess = tf.Session()\r\nsess.run(init)\r\n```\r\n\r\n然后开始训练模型，这里我们让模型循环训练1000次！\r\n\r\n```python\r\nfor i in range(1000):\r\n batch_xs, batch_ys = mnist.train.next_batch(100)\r\n sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})\r\n```\r\n\r\n该循环的每个步骤中，我们都会随机抓取训练数据中的100个批处理数据点，然后我们用这些数据点作为参数替换之前的占位符来运行`train_step`。 \r\n\r\n使用一小部分的随机数据来进行训练被称为随机训练（stochastic training）- 在这里更确切的说是随机梯度下降训练。在理想情况下，我们希望用我们所有的数据来进行每一步的训练，因为这能给我们更好的训练结果，但显然这需要很大的计算开销。所以，每一次训练我们可以使用不同的数据子集，这样做既可以减少计算开销，又可以最大化地学习到数据集的总体特性。\r\n\r\n## 评估我们的模型 \r\n\r\n那么我们的模型性能如何呢？\r\n\r\n首先让我们找出那些预测正确的标签。`tf.argmax` 是一个非常有用的函数，它能给出某个tensor对象在某一维上的其数据最大值所在的索引值。由于标签向量是由0,1组成，因此最大值1所在的索引位置就是类别标签，比如`tf.argmax(y,1)`返回的是模型对于任一输入x预测到的标签值，而 `tf.argmax(y_,1)` 代表正确的标签，我们可以用 `tf.equal` 来检测我们的预测是否真实标签匹配(索引位置一样表示匹配)。\r\n\r\n```python\r\ncorrect_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))\r\n```\r\n\r\n这行代码会给我们一组布尔值。为了确定正确预测项的比例，我们可以把布尔值转换成浮点数，然后取平均值。例如，`[True, False, True, True]` 会变成 `[1,0,1,1]` ，取平均值后得到 `0.75`.\r\n\r\n```python\r\naccuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\r\n```\r\n\r\n最后，我们计算所学习到的模型在测试数据集上面的正确率。\r\n\r\n```python\r\nprint sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels})\r\n```\r\n\r\n这个最终结果值应该大约是91%。\r\n\r\n这个结果好吗？嗯，并不太好。事实上，这个结果是很差的。这是因为我们仅仅使用了一个非常简单的模型。不过，做一些小小的改进，我们就可以得到97％的正确率。最好的模型甚至可以获得超过99.7％的准确率！（想了解更多信息，可以看看这个关于各种模型的[性能对比列表](http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html)。)\r\n\r\n比结果更重要的是，我们从这个模型中学习到的设计思想。不过，如果你仍然对这里的结果有点失望，可以查看[下一个教程](./mnist_pros.md)，在那里你可以学习如何用TensorFlow构建更加复杂的模型以获得更好的性能！\r\n\r\n原文地址：[MNIST For ML Beginners](http://tensorflow.org/tutorials/mnist/beginners/index.md) 翻译：[linbojin](https://github.com/linbojin) 校对：\r\n"
},
{
"path": "SOURCE/tutorials/mnist_download.md",
"content": "# MNIST 数据下载 \r\n\r\n源码: [tensorflow/g3doc/tutorials/mnist/](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/tutorials/mnist)\r\n\r\n本教程的目标是展示如何下载用于手写数字分类问题所要用到的（经典）MNIST数据集。\r\n\r\n## 教程 文件 \r\n\r\n本教程需要使用以下文件：\r\n\r\n文件 | 目的\r\n--- | ---\r\n[`input_data.py`](https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/examples/tutorials/mnist/input_data.py) | 下载用于训练和测试的MNIST数据集的源码\r\n\r\n## 准备数据 \r\n\r\nMNIST是在机器学习领域中的一个经典问题。该问题解决的是把28x28像素的灰度手写数字图片识别为相应的数字，其中数字的范围从0到9.\r\n\r\n![MNIST Digits](../images/mnist_digits.png \"MNIST Digits\")\r\n\r\n更多详情, 请参考 [Yann LeCun's MNIST page](http://yann.lecun.com/exdb/mnist/)\r\n或 [Chris Olah's visualizations of MNIST](http://colah.github.io/posts/2014-10-Visualizing-MNIST/).\r\n\r\n### 下载 \r\n\r\n[Yann LeCun's MNIST page](http://yann.lecun.com/exdb/mnist/)\r\n也提供了训练集与测试集数据的下载。\r\n\r\n文件 | 内容\r\n--- | ---\r\n[`train-images-idx3-ubyte.gz`](http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz) | 训练集图片 - 55000 张 训练图片, 5000 张 验证图片\r\n[`train-labels-idx1-ubyte.gz`](http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz) | 训练集图片对应的数字标签\r\n[`t10k-images-idx3-ubyte.gz`](http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz) | 测试集图片 - 10000 张 图片\r\n[`t10k-labels-idx1-ubyte.gz`](http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz) | 测试集图片对应的数字标签\r\n\r\n在 `input_data.py` 文件中, `maybe_download()` 函数可以确保这些训练数据下载到本地文件夹中。\r\n\r\n文件夹的名字在\r\n`fully_connected_feed.py` 文件的顶部由一个标记变量指定，你可以根据自己的需要进行修改。\r\n### 解压 与 重构 \r\n\r\n这些文件本身并没有使用标准的图片格式储存，并且需要使用`input_data.py`文件中`extract_images()` 和`extract_labels()`函数来手动解压（页面中有相关说明）。\r\n\r\n图片数据将被解压成2维的tensor：`[image index, pixel index]`\r\n其中每一项表示某一图片中特定像素的强度值, 范围从 `[0, 255]` 到 `[-0.5, 0.5]`。 \"image index\"代表数据集中图片的编号, 从0到数据集的上限值。\"pixel index\"代表该图片中像素点得个数, 从0到图片的像素上限值。\r\n\r\n以`train-*`开头的文件中包括60000个样本，其中分割出55000个样本作为训练集，其余的5000个样本作为验证集。因为所有数据集中28x28像素的灰度图片的尺寸为784，所以训练集输出的tensor格式为`[55000, 784]`。\r\n\r\n数字标签数据被解压称1维的tensor: `[image index]`，它定义了每个样本数值的类别分类。对于训练集的标签来说，这个数据规模就是:`[55000]`。\r\n\r\n### 数据集 对象 \r\n\r\n底层的源码将会执行下载、解压、重构图片和标签数据来组成以下的数据集对象:\r\n\r\n数据集 | 目的\r\n--- | ---\r\n`data_sets.train` | 55000 组 图片和标签, 用于训练。\r\n`data_sets.validation` | 5000 组 图片和标签, 用于迭代验证训练的准确性。\r\n`data_sets.test` | 10000 组 图片和标签, 用于最终测试训练的准确性。\r\n\r\n执行`read_data_sets()`函数将会返回一个`DataSet`实例，其中包含了以上三个数据集。函数`DataSet.next_batch()`是用于获取以`batch_size`为大小的一个元组，其中包含了一组图片和标签，该元组会被用于当前的TensorFlow运算会话中。\r\n```python\r\nimages_feed, labels_feed = data_set.next_batch(FLAGS.batch_size)\r\n```\r\n原文地址：[MNIST Data Download](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/tutorials/mnist/download/index.md) 翻译：[btpeter](https://github.com/btpeter) 校对：waiwaizheng"
},
{
"path": "SOURCE/tutorials/mnist_pros.md",
"content": "# 深入MNIST \r\n\r\nTensorFlow是一个非常强大的用来做大规模数值计算的库。其所擅长的任务之一就是实现以及训练深度神经网络。\r\n\r\n在本教程中，我们将学到构建一个TensorFlow模型的基本步骤，并将通过这些步骤为MNIST构建一个深度卷积神经网络。\r\n\r\n*这个教程假设你已经熟悉神经网络和MNIST数据集。如果你尚未了解，请查看[新手指南](./mnist_beginners.md).*\r\n\r\n## 安装 \r\n\r\n在创建模型之前，我们会先加载MNIST数据集，然后启动一个TensorFlow的session。\r\n\r\n### 加载MNIST数据 \r\n\r\n为了方便起见，我们已经准备了[一个脚本](https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/examples/tutorials/mnist/input_data.py)来自动下载和导入MNIST数据集。它会自动创建一个`'MNIST_data'`的目录来存储数据。\r\n\r\n```python\r\nimport input_data\r\nmnist = input_data.read_data_sets('MNIST_data', one_hot=True)\r\n```\r\n\r\n这里，`mnist`是一个轻量级的类。它以Numpy数组的形式存储着训练、校验和测试数据集。同时提供了一个函数，用于在迭代中获得minibatch，后面我们将会用到。\r\n\r\n### 运行TensorFlow的InteractiveSession \r\n\r\nTensorflow依赖于一个高效的C++后端来进行计算。与后端的这个连接叫做session。一般而言，使用TensorFlow程序的流程是先创建一个图，然后在session中启动它。\r\n\r\n这里，我们使用更加方便的`InteractiveSession`类。通过它，你可以更加灵活地构建你的代码。它能让你在运行图的时候，插入一些[计算图](../get_started/basic_usage.md#the-computation-graph)，这些计算图是由某些操作(operations)构成的。这对于工作在交互式环境中的人们来说非常便利，比如使用IPython。如果你没有使用`InteractiveSession`，那么你需要在启动session之前构建整个计算图，然后[启动该计算图](../get_started/basic_usage.md#launching-the-graph-in-a-session)。\r\n\r\n```python\r\nimport tensorflow as tf\r\nsess = tf.InteractiveSession()\r\n```\r\n\r\n#### 计算图 \r\n\r\n为了在Python中进行高效的数值计算，我们通常会使用像NumPy一类的库，将一些诸如矩阵乘法的耗时操作在Python环境的外部来计算，这些计算通常会通过其它语言并用更为高效的代码来实现。\r\n\r\n但遗憾的是，每一个操作切换回Python环境时仍需要不小的开销。如果你想在GPU或者分布式环境中计算时，这一开销更加可怖，这一开销主要可能是用来进行数据迁移。\r\n\r\nTensorFlow也是在Python外部完成其主要工作，但是进行了改进以避免这种开销。其并没有采用在Python外部独立运行某个耗时操作的方式，而是先让我们描述一个交互操作图，然后完全将其运行在Python外部。这与Theano或Torch的做法类似。\r\n\r\n因此Python代码的目的是用来构建这个可以在外部运行的计算图，以及安排计算图的哪一部分应该被运行。详情请查看[基本用法](../../../get_started/basic_usage.md)中的[计算图表](../../../get_started/basic_usage.md#the-computation-graph)一节。\r\n\r\n## 构建Softmax 回归模型 \r\n\r\n在这一节中我们将建立一个拥有一个线性层的softmax回归模型。在下一节，我们会将其扩展为一个拥有多层卷积网络的softmax回归模型。\r\n\r\n### 占位符 \r\n\r\n我们通过为输入图像和目标输出类别创建节点，来开始构建计算图。\r\n\r\n```python\r\nx = tf.placeholder(\"float\", shape=[None, 784])\r\ny_ = tf.placeholder(\"float\", shape=[None, 10])\r\n```\r\n\r\n这里的`x`和`y`并不是特定的值，相反，他们都只是一个`占位符`，可以在TensorFlow运行某一计算时根据该占位符输入具体的值。\r\n\r\n输入图片`x`是一个2维的浮点数张量。这里，分配给它的`shape`为`[None, 784]`，其中`784`是一张展平的MNIST图片的维度。`None`表示其值大小不定，在这里作为第一个维度值，用以指代batch的大小，意即`x`的数量不定。输出类别值`y_`也是一个2维张量，其中每一行为一个10维的one-hot向量,用于代表对应某一MNIST图片的类别。\r\n\r\n虽然`placeholder`的`shape`参数是可选的，但有了它，TensorFlow能够自动捕捉因数据维度不一致导致的错误。\r\n\r\n### 变量 \r\n\r\n我们现在为模型定义权重`W`和偏置`b`。可以将它们当作额外的输入量，但是TensorFlow有一个更好的处理方式：`变量`。一个`变量`代表着TensorFlow计算图中的一个值，能够在计算过程中使用，甚至进行修改。在机器学习的应用过程中，模型参数一般用`Variable`来表示。\r\n\r\n```python\r\nW = tf.Variable(tf.zeros([784,10]))\r\nb = tf.Variable(tf.zeros([10]))\r\n```\r\n\r\n我们在调用`tf.Variable`的时候传入初始值。在这个例子里，我们把`W`和`b`都初始化为零向量。`W`是一个784x10的矩阵（因为我们有784个特征和10个输出值）。`b`是一个10维的向量（因为我们有10个分类）。\r\n\r\n\r\nBefore `Variable`s can be used within a session, they must be initialized using\r\nthat session.\r\nThis step takes the initial values (in this case tensors full of zeros) that\r\nhave already been specified, and assigns them to each `Variable`. This can be\r\ndone for all `Variables` at once.\r\n\r\n`变量`需要通过seesion初始化后，才能在session中使用。这一初始化步骤为，为初始值指定具体值（本例当中是全为零），并将其分配给每个`变量`,可以一次性为所有`变量`完成此操作。\r\n\r\n```python\r\nsess.run(tf.initialize_all_variables())\r\n```\r\n\r\n### 类别预测与损失函数 \r\n\r\n现在我们可以实现我们的回归模型了。这只需要一行！我们把向量化后的图片`x`和权重矩阵`W`相乘，加上偏置`b`，然后计算每个分类的softmax概率值。\r\n\r\n```python\r\ny = tf.nn.softmax(tf.matmul(x,W) + b)\r\n```\r\n可以很容易的为训练过程指定最小化误差用的损失函数，我们的损失函数是目标类别和预测类别之间的交叉熵。\r\n\r\n```python\r\ncross_entropy = -tf.reduce_sum(y_*tf.log(y))\r\n```\r\n\r\n注意，`tf.reduce_sum`把minibatch里的每张图片的交叉熵值都加起来了。我们计算的交叉熵是指整个minibatch的。\r\n\r\n## 训练模型 \r\n\r\n我们已经定义好模型和训练用的损失函数，那么用TensorFlow进行训练就很简单了。因为TensorFlow知道整个计算图，它可以使用自动微分法找到对于各个变量的损失的梯度值。TensorFlow有[大量内置的优化算法](../../../api_docs/python/train.md#optimizers) 这个例子中，我们用最速下降法让交叉熵下降，步长为0.01.\r\n\r\n```python\r\ntrain_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)\r\n```\r\n\r\n这一行代码实际上是用来往计算图上添加一个新操作，其中包括计算梯度，计算每个参数的步长变化，并且计算出新的参数值。\r\n\r\n返回的`train_step`操作对象，在运行时会使用梯度下降来更新参数。因此，整个模型的训练可以通过反复地运行`train_step`来完成。\r\n\r\n```python\r\nfor i in range(1000):\r\n batch = mnist.train.next_batch(50)\r\n train_step.run(feed_dict={x: batch[0], y_: batch[1]})\r\n```\r\n\r\n每一步迭代，我们都会加载50个训练样本，然后执行一次`train_step`，并通过`feed_dict`将`x` 和 `y_`张量`占位符`用训练训练数据替代。\r\n\r\n注意，在计算图中，你可以用`feed_dict`来替代任何张量，并不仅限于替换`占位符`。\r\n\r\n### 评估模型 \r\n\r\n那么我们的模型性能如何呢？\r\n\r\n首先让我们找出那些预测正确的标签。`tf.argmax` 是一个非常有用的函数，它能给出某个tensor对象在某一维上的其数据最大值所在的索引值。由于标签向量是由0,1组成，因此最大值1所在的索引位置就是类别标签，比如`tf.argmax(y,1)`返回的是模型对于任一输入x预测到的标签值，而 `tf.argmax(y_,1)` 代表正确的标签，我们可以用 `tf.equal` 来检测我们的预测是否真实标签匹配(索引位置一样表示匹配)。\r\n\r\n```python\r\ncorrect_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))\r\n```\r\n\r\n这里返回一个布尔数组。为了计算我们分类的准确率，我们将布尔值转换为浮点数来代表对、错，然后取平均值。例如：`[True, False, True, True]`变为`[1,0,1,1]`，计算出平均值为`0.75`。\r\n\r\n```python\r\naccuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\r\n```\r\n\r\n最后，我们可以计算出在测试数据上的准确率，大概是91%。\r\n\r\n```python\r\nprint accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels})\r\n```\r\n\r\n## 构建一个多层卷积网络 \r\n\r\n在MNIST上只有91%正确率，实在太糟糕。在这个小节里，我们用一个稍微复杂的模型：卷积神经网络来改善效果。这会达到大概99.2%的准确率。虽然不是最高，但是还是比较让人满意。\r\n\r\n### 权重初始化 \r\n\r\n为了创建这个模型，我们需要创建大量的权重和偏置项。这个模型中的权重在初始化时应该加入少量的噪声来打破对称性以及避免0梯度。由于我们使用的是ReLU神经元，因此比较好的做法是用一个较小的正数来初始化偏置项，以避免神经元节点输出恒为0的问题（dead neurons）。为了不在建立模型的时候反复做初始化操作，我们定义两个函数用于初始化。\r\n\r\n```python\r\ndef weight_variable(shape):\r\n initial = tf.truncated_normal(shape, stddev=0.1)\r\n return tf.Variable(initial)\r\n\r\ndef bias_variable(shape):\r\n initial = tf.constant(0.1, shape=shape)\r\n return tf.Variable(initial)\r\n```\r\n\r\n### 卷积和池化 \r\n\r\nTensorFlow在卷积和池化上有很强的灵活性。我们怎么处理边界？步长应该设多大？在这个实例里，我们会一直使用vanilla版本。我们的卷积使用1步长（stride size），0边距（padding size）的模板，保证输出和输入是同一个大小。我们的池化用简单传统的2x2大小的模板做max pooling。为了代码更简洁，我们把这部分抽象成一个函数。\r\n\r\n```python\r\ndef conv2d(x, W):\r\n return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')\r\n\r\ndef max_pool_2x2(x):\r\n return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],\r\n strides=[1, 2, 2, 1], padding='SAME')\r\n```\r\n\r\n### 第一层卷积 \r\n\r\n现在我们可以开始实现第一层了。它由一个卷积接一个max pooling完成。卷积在每个5x5的patch中算出32个特征。卷积的权重张量形状是`[5, 5, 1, 32]`，前两个维度是patch的大小，接着是输入的通道数目，最后是输出的通道数目。 而对于每一个输出通道都有一个对应的偏置量。\r\n\r\n```python\r\nW_conv1 = weight_variable([5, 5, 1, 32])\r\nb_conv1 = bias_variable([32])\r\n```\r\n为了用这一层，我们把`x`变成一个4d向量，其第2、第3维对应图片的宽、高，最后一维代表图片的颜色通道数(因为是灰度图所以这里的通道数为1，如果是rgb彩色图，则为3)。\r\n\r\n```python\r\nx_image = tf.reshape(x, [-1,28,28,1])\r\n```\r\n\r\nWe then convolve `x_image` with the weight tensor, add the\r\nbias, apply the ReLU function, and finally max pool.\r\n我们把`x_image`和权值向量进行卷积，加上偏置项，然后应用ReLU激活函数，最后进行max pooling。 \r\n\r\n```python\r\nh_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)\r\nh_pool1 = max_pool_2x2(h_conv1)\r\n```\r\n\r\n### 第二层卷积 \r\n\r\n为了构建一个更深的网络，我们会把几个类似的层堆叠起来。第二层中，每个5x5的patch会得到64个特征。\r\n\r\n```python\r\nW_conv2 = weight_variable([5, 5, 32, 64])\r\nb_conv2 = bias_variable([64])\r\n\r\nh_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)\r\nh_pool2 = max_pool_2x2(h_conv2)\r\n```\r\n\r\n### 密集连接层 \r\n\r\n现在，图片尺寸减小到7x7，我们加入一个有1024个神经元的全连接层，用于处理整个图片。我们把池化层输出的张量reshape成一些向量，乘上权重矩阵，加上偏置，然后对其使用ReLU。\r\n\r\n```python\r\nW_fc1 = weight_variable([7 * 7 * 64, 1024])\r\nb_fc1 = bias_variable([1024])\r\n\r\nh_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])\r\nh_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)\r\n```\r\n\r\n#### Dropout \r\n\r\n为了减少过拟合，我们在输出层之前加入dropout。我们用一个`placeholder`来代表一个神经元的输出在dropout中保持不变的概率。这样我们可以在训练过程中启用dropout，在测试过程中关闭dropout。\r\nTensorFlow的`tf.nn.dropout`操作除了可以屏蔽神经元的输出外，还会自动处理神经元输出值的scale。所以用dropout的时候可以不用考虑scale。\r\n\r\n```python\r\nkeep_prob = tf.placeholder(\"float\")\r\nh_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)\r\n```\r\n\r\n### 输出层 \r\n\r\n最后，我们添加一个softmax层，就像前面的单层softmax regression一样。\r\n\r\n```python\r\nW_fc2 = weight_variable([1024, 10])\r\nb_fc2 = bias_variable([10])\r\n\r\ny_conv=tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)\r\n```\r\n\r\n### 训练和评估模型 \r\n\r\n这个模型的效果如何呢？\r\n\r\n为了进行训练和评估，我们使用与之前简单的单层SoftMax神经网络模型几乎相同的一套代码，只是我们会用更加复杂的ADAM优化器来做梯度最速下降，在`feed_dict`中加入额外的参数`keep_prob`来控制dropout比例。然后每100次迭代输出一次日志。\r\n\r\n```python\r\ncross_entropy = -tf.reduce_sum(y_*tf.log(y_conv))\r\ntrain_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)\r\ncorrect_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(y_,1))\r\naccuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))\r\nsess.run(tf.initialize_all_variables())\r\nfor i in range(20000):\r\n batch = mnist.train.next_batch(50)\r\n if i%100 == 0:\r\n train_accuracy = accuracy.eval(feed_dict={\r\n x:batch[0], y_: batch[1], keep_prob: 1.0})\r\n print \"step %d, training accuracy %g\"%(i, train_accuracy)\r\n train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})\r\n\r\nprint \"test accuracy %g\"%accuracy.eval(feed_dict={\r\n x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0})\r\n```\r\n\r\n以上代码，在最终测试集上的准确率大概是99.2%。\r\n\r\n目前为止，我们已经学会了用TensorFlow快捷地搭建、训练和评估一个复杂一点儿的深度学习模型。\r\n\r\n原文地址：[Deep MNIST for Experts](http://tensorflow.org/tutorials/mnist/pros/index.html) 翻译：[chenweican](https://github.com/chenweican) 校对：[HongyangWang](https://github.com/wanghong-yang)\r\n\r\n\n"
},
{
"path": "SOURCE/tutorials/mnist_tf.md",
"content": "# TensorFlow运作方式入门 \r\n\r\n代码：[tensorflow/g3doc/tutorials/mnist/](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/tutorials/mnist)\r\n\r\n本篇教程的目的，是向大家展示如何利用TensorFlow使用（经典）MNIST数据集训练并评估一个用于识别手写数字的简易前馈神经网络（feed-forward neural network）。我们的目标读者，是有兴趣使用TensorFlow的资深机器学习人士。\r\n\r\n因此，撰写该系列教程并不是为了教大家机器学习领域的基础知识。\r\n\r\n在学习本教程之前，请确保您已按照[安装TensorFlow](../get_started/os_setup.md)教程中的要求，完成了安装。\r\n\r\n## 教程使用的文件 \r\n\r\n\r\n本教程引用如下文件：\r\n\r\n文件 | 目的\r\n--- | ---\r\n[`mnist.py`](https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/examples/tutorials/mnist/mnist.py) | 构建一个完全连接（fully connected）的MINST模型所需的代码。\r\n[`fully_connected_feed.py`](https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/examples/tutorials/mnist/fully_connected_feed.py) | 利用下载的数据集训练构建好的MNIST模型的主要代码，以数据反馈字典（feed dictionary）的形式作为输入模型。\r\n\r\n\r\n只需要直接运行`fully_connected_feed.py`文件，就可以开始训练：\r\n\r\n`python fully_connected_feed.py`\r\n\r\n## 准备数据 \r\n\r\nMNIST是机器学习领域的一个经典问题，指的是让机器查看一系列大小为28x28像素的手写数字灰度图像，并判断这些图像代表0-9中的哪一个数字。\r\n\r\n![MNIST手写数字](.././images/mnist_digits.png \"MNIST手写数字\")\r\n\r\n更多相关信息，请查阅[Yann LeCun网站中关于MNIST的介绍](http://yann.lecun.com/exdb/mnist/)\r\n或者[Chris Olah对MNIST的可视化探索](http://colah.github.io/posts/2014-10-Visualizing-MNIST/)。\r\n\r\n### 下载 \r\n\r\n在`run_training()`方法的一开始，`input_data.read_data_sets()`函数会确保你的本地训练文件夹中，已经下载了正确的数据，然后将这些数据解压并返回一个含有`DataSet`实例的字典。\r\n\r\n```python\r\ndata_sets = input_data.read_data_sets(FLAGS.train_dir, FLAGS.fake_data)\r\n```\r\n\r\n**注意**：`fake_data`标记是用于单元测试的，读者可以不必理会。\r\n\r\n数据集 | 目的\r\n--- | ---\r\n`data_sets.train` | 55000个图像和标签（labels），作为主要训练集。\r\n`data_sets.validation` | 5000个图像和标签，用于迭代验证训练准确度。\r\n`data_sets.test` | 10000个图像和标签，用于最终测试训练准确度（trained accuracy）。\r\n\r\n了解更多数据有关信息，请查阅此系列教程的[数据下载](mnist/download/index.md)\r\n部分.\r\n\r\n### 输入与占位符（Inputs and Placeholders） \r\n\r\n`placeholder_inputs()`函数将生成两个[`tf.placeholder`](../api_docs/python/io_ops.md#placeholder)操作，定义传入图表中的shape参数，shape参数中包括`batch_size `值，后续还会将实际的训练用例传入图表。\r\n\r\n```python\r\nimages_placeholder = tf.placeholder(tf.float32, shape=(batch_size,\r\n IMAGE_PIXELS))\r\nlabels_placeholder = tf.placeholder(tf.int32, shape=(batch_size))\r\n```\r\n\r\n在训练循环（training loop）的后续步骤中，传入的整个图像和标签数据集会被切片，以符合每一个操作所设置的`batch_size`值，占位符操作将会填补以符合这个`batch_size`值。然后使用`feed_dict`参数，将数据传入`sess.run()`函数。\r\n\r\n## 构建图表 （Build the Graph）\r\n\r\n在为数据创建占位符之后，就可以运行`mnist.py`文件，经过三阶段的模式函数操作：`inference()`， `loss()`，和`training()`。图表就构建完成了。\r\n\r\n1.`inference()` —— 尽可能地构建好图表，满足促使神经网络向前反馈并做出预测的要求。\r\n\r\n2.`loss()` —— 往inference图表中添加生成损失（loss）所需要的操作（ops）。\r\n\r\n3.`training()` —— 往损失图表中添加计算并应用梯度（gradients）所需的操作。\r\n\r\n\r\n\r\n ![](\".././images/mnist_subgraph.png\")
\r\n
\r\n\r\n### 推理（Inference） \r\n\r\n`inference()`函数会尽可能地构建图表，做到返回包含了预测结果（output prediction）的Tensor。\r\n\r\n它接受图像占位符为输入，在此基础上借助ReLu(Rectified Linear Units)激活函数，构建一对完全连接层（layers），以及一个有着十个节点（node）、指明了输出logits模型的线性层。\r\n\r\n每一层都创建于一个唯一的[`tf.name_scope`](../api_docs/python/framework.md#name_scope)之下，创建于该作用域之下的所有元素都将带有其前缀。\r\n\r\n```python\r\nwith tf.name_scope('hidden1') as scope:\r\n```\r\n\r\n在定义的作用域中，每一层所使用的权重和偏差都在[`tf.Variable`](../api_docs/python/state_ops.md#Variable)实例中生成，并且包含了各自期望的shape。\r\n\r\n```python\r\nweights = tf.Variable(\r\n tf.truncated_normal([IMAGE_PIXELS, hidden1_units],\r\n stddev=1.0 / math.sqrt(float(IMAGE_PIXELS))),\r\n name='weights')\r\nbiases = tf.Variable(tf.zeros([hidden1_units]),\r\n name='biases')\r\n```\r\n\r\n例如，当这些层是在`hidden1`作用域下生成时，赋予权重变量的独特名称将会是\"`hidden1/weights`\"。\r\n\r\n每个变量在构建时，都会获得初始化操作（initializer ops）。\r\n\r\n在这种最常见的情况下，通过[`tf.truncated_normal`](../api_docs/python/constant_op.md#truncated_normal)函数初始化权重变量，给赋予的shape则是一个二维tensor，其中第一个维度代表该层中权重变量所连接（connect from）的单元数量，第二个维度代表该层中权重变量所连接到的（connect to）单元数量。对于名叫`hidden1`的第一层，相应的维度则是`[IMAGE_PIXELS, hidden1_units]`，因为权重变量将图像输入连接到了`hidden1`层。`tf.truncated_normal`初始函数将根据所得到的均值和标准差，生成一个随机分布。\r\n\r\n然后，通过[`tf.zeros`](../api_docs/python/constant_op.md#zeros)函数初始化偏差变量（biases），确保所有偏差的起始值都是0，而它们的shape则是其在该层中所接到的（connect to）单元数量。\r\n\r\n图表的三个主要操作，分别是两个[`tf.nn.relu`](../api_docs/python/nn.md#relu)操作，它们中嵌入了隐藏层所需的[`tf.matmul`](../api_docs/python/math_ops.md#matmul)；以及logits模型所需的另外一个`tf.matmul`。三者依次生成，各自的`tf.Variable`实例则与输入占位符或下一层的输出tensor所连接。\r\n\r\n```python\r\nhidden1 = tf.nn.relu(tf.matmul(images, weights) + biases)\r\n```\r\n\r\n```python\r\nhidden2 = tf.nn.relu(tf.matmul(hidden1, weights) + biases)\r\n```\r\n\r\n```python\r\nlogits = tf.matmul(hidden2, weights) + biases\r\n```\r\n\r\n最后，程序会返回包含了输出结果的`logits`Tensor。\r\n\r\n### 损失（Loss）\r\n\r\n`loss()`函数通过添加所需的损失操作，进一步构建图表。\r\n\r\n首先，`labels_placeholer`中的值，将被编码为一个含有1-hot values的Tensor。例如，如果类标识符为“3”，那么该值就会被转换为：\r\n\r\n`[0, 0, 0, 1, 0, 0, 0, 0, 0, 0]`\r\n\r\n```python\r\nbatch_size = tf.size(labels)\r\nlabels = tf.expand_dims(labels, 1)\r\nindices = tf.expand_dims(tf.range(0, batch_size, 1), 1)\r\nconcated = tf.concat(1, [indices, labels])\r\nonehot_labels = tf.sparse_to_dense(\r\n concated, tf.pack([batch_size, NUM_CLASSES]), 1.0, 0.0)\r\n```\r\n\r\n之后，又添加一个[`tf.nn.softmax_cross_entropy_with_logits`](../api_docs/python/nn.md#softmax_cross_entropy_with_logits)操作，用来比较`inference()`函数与1-hot标签所输出的logits Tensor。\r\n\r\n```python\r\ncross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits,\r\n onehot_labels,\r\n name='xentropy')\r\n```\r\n\r\n然后，使用[`tf.reduce_mean`](../api_docs/python/math_ops.md#reduce_mean)函数，计算batch维度（第一维度）下交叉熵（cross entropy）的平均值，将将该值作为总损失。\r\n\r\n```python\r\nloss = tf.reduce_mean(cross_entropy, name='xentropy_mean')\r\n```\r\n\r\n最后，程序会返回包含了损失值的Tensor。\r\n\r\n> 注意：交叉熵是信息理论中的概念，可以让我们描述如果基于已有事实，相信神经网络所做的推测最坏会导致什么结果。更多详情，请查阅博文《可视化信息理论》(http://colah.github.io/posts/2015-09-Visual-Information/)\r\n\r\n### 训练 \r\n\r\n`training()`函数添加了通过梯度下降（gradient descent）将损失最小化所需的操作。\r\n\r\n首先，该函数从`loss()`函数中获取损失Tensor，将其交给[`tf.scalar_summary`](../api_docs/python/train.md#scalar_summary)，后者在与`SummaryWriter`（见下文）配合使用时，可以向事件文件（events file）中生成汇总值（summary values）。在本篇教程中，每次写入汇总值时，它都会释放损失Tensor的当前值（snapshot value）。\r\n\r\n```python\r\ntf.scalar_summary(loss.op.name, loss)\r\n```\r\n\r\n接下来，我们实例化一个[`tf.train.GradientDescentOptimizer`](../api_docs/python/train.md#GradientDescentOptimizer)，负责按照所要求的学习效率（learning rate）应用梯度下降法（gradients）。\r\n\r\n```python\r\noptimizer = tf.train.GradientDescentOptimizer(FLAGS.learning_rate)\r\n```\r\n\r\n之后，我们生成一个变量用于保存全局训练步骤（global training step）的数值，并使用[`minimize()`](../api_docs/python/train.md#Optimizer.minimize)函数更新系统中的三角权重（triangle weights）、增加全局步骤的操作。根据惯例，这个操作被称为 `train_op`，是TensorFlow会话（session）诱发一个完整训练步骤所必须运行的操作（见下文）。\r\n\r\n```python\r\nglobal_step = tf.Variable(0, name='global_step', trainable=False)\r\ntrain_op = optimizer.minimize(loss, global_step=global_step)\r\n```\r\n\r\n最后，程序返回包含了训练操作（training op）输出结果的Tensor。\r\n\r\n## 训练模型 \r\n\r\n一旦图表构建完毕，就通过`fully_connected_feed.py`文件中的用户代码进行循环地迭代式训练和评估。\r\n\r\n### 图表 \r\n\r\n在`run_training()`这个函数的一开始，是一个Python语言中的`with`命令，这个命令表明所有已经构建的操作都要与默认的[`tf.Graph`](../api_docs/python/framework.md#Graph)全局实例关联起来。\r\n\r\n```python\r\nwith tf.Graph().as_default():\r\n```\r\n\r\n`tf.Graph`实例是一系列可以作为整体执行的操作。TensorFlow的大部分场景只需要依赖默认图表一个实例即可。\r\n\r\n利用多个图表的更加复杂的使用场景也是可能的，但是超出了本教程的范围。\r\n\r\n### 会话 \r\n\r\n完成全部的构建准备、生成全部所需的操作之后，我们就可以创建一个[`tf.Session`](../api_docs/python/client.md#Session)，用于运行图表。\r\n\r\n```python\r\nsess = tf.Session()\r\n```\r\n\r\n另外，也可以利用`with`代码块生成`Session`，限制作用域：\r\n\r\n```python\r\nwith tf.Session() as sess:\r\n```\r\n\r\n`Session`函数中没有传入参数，表明该代码将会依附于（如果还没有创建会话，则会创建新的会话）默认的本地会话。\r\n\r\n生成会话之后，所有`tf.Variable`实例都会立即通过调用各自初始化操作中的[`sess.run()`](../api_docs/python/client.md#Session.run)函数进行初始化。\r\n\r\n```python\r\ninit = tf.initialize_all_variables()\r\nsess.run(init)\r\n```\r\n\r\n[`sess.run()`](../api_docs/python/client.md#Session.run)方法将会运行图表中与作为参数传入的操作相对应的完整子集。在初次调用时，`init`操作只包含了变量初始化程序[`tf.group`](../api_docs/python/control_flow_ops.md#group)。图表的其他部分不会在这里，而是在下面的训练循环运行。\r\n\r\n### 训练循环 \r\n\r\n完成会话中变量的初始化之后，就可以开始训练了。\r\n\r\n训练的每一步都是通过用户代码控制，而能实现有效训练的最简单循环就是：\r\n\r\n```python\r\nfor step in xrange(max_steps):\r\n sess.run(train_op)\r\n```\r\n\r\n但是，本教程中的例子要更为复杂一点，原因是我们必须把输入的数据根据每一步的情况进行切分，以匹配之前生成的占位符。\r\n\r\n#### 向图表提供反馈 \r\n\r\n执行每一步时，我们的代码会生成一个反馈字典（feed dictionary），其中包含对应步骤中训练所要使用的例子，这些例子的哈希键就是其所代表的占位符操作。\r\n\r\n`fill_feed_dict`函数会查询给定的`DataSet`，索要下一批次`batch_size`的图像和标签，与占位符相匹配的Tensor则会包含下一批次的图像和标签。\r\n\r\n```python\r\nimages_feed, labels_feed = data_set.next_batch(FLAGS.batch_size)\r\n```\r\n\r\n然后，以占位符为哈希键，创建一个Python字典对象，键值则是其代表的反馈Tensor。\r\n\r\n```python\r\nfeed_dict = {\r\n images_placeholder: images_feed,\r\n labels_placeholder: labels_feed,\r\n}\r\n```\r\n\r\n这个字典随后作为`feed_dict`参数，传入`sess.run()`函数中，为这一步的训练提供输入样例。\r\n\r\n#### 检查状态 \r\n\r\n在运行`sess.run`函数时，要在代码中明确其需要获取的两个值：`[train_op, loss]`。\r\n\r\n```python\r\nfor step in xrange(FLAGS.max_steps):\r\n feed_dict = fill_feed_dict(data_sets.train,\r\n images_placeholder,\r\n labels_placeholder)\r\n _, loss_value = sess.run([train_op, loss],\r\n feed_dict=feed_dict)\r\n```\r\n\r\n因为要获取这两个值，`sess.run()`会返回一个有两个元素的元组。其中每一个`Tensor`对象，对应了返回的元组中的numpy数组，而这些数组中包含了当前这步训练中对应Tensor的值。由于`train_op`并不会产生输出，其在返回的元祖中的对应元素就是`None`，所以会被抛弃。但是，如果模型在训练中出现偏差，`loss` Tensor的值可能会变成NaN，所以我们要获取它的值，并记录下来。\r\n\r\n假设训练一切正常，没有出现NaN，训练循环会每隔100个训练步骤，就打印一行简单的状态文本，告知用户当前的训练状态。\r\n\r\n```python\r\nif step % 100 == 0:\r\n print 'Step %d: loss = %.2f (%.3f sec)' % (step, loss_value, duration)\r\n```\r\n\r\n#### 状态可视化 \r\n\r\n为了释放[TensorBoard](../how_tos/summaries_and_tensorboard.md)所使用的事件文件（events file），所有的即时数据（在这里只有一个）都要在图表构建阶段合并至一个操作（op）中。\r\n\r\n```python\r\nsummary_op = tf.merge_all_summaries()\r\n```\r\n\r\n在创建好会话（session）之后，可以实例化一个[`tf.train.SummaryWriter`](../api_docs/python/train.md#SummaryWriter)，用于写入包含了图表本身和即时数据具体值的事件文件。\r\n\r\n```python\r\nsummary_writer = tf.train.SummaryWriter(FLAGS.train_dir,\r\n graph_def=sess.graph_def)\r\n```\r\n\r\n最后，每次运行`summary_op`时，都会往事件文件中写入最新的即时数据，函数的输出会传入事件文件读写器（writer）的`add_summary()`函数。。\r\n\r\n```python\r\nsummary_str = sess.run(summary_op, feed_dict=feed_dict)\r\nsummary_writer.add_summary(summary_str, step)\r\n```\r\n\r\n事件文件写入完毕之后，可以就训练文件夹打开一个TensorBoard，查看即时数据的情况。\r\n\r\n![MNIST TensorBoard](../images/mnist_tensorboard.png \"MNIST TensorBoard\")\r\n\r\n**注意**：了解更多如何构建并运行TensorBoard的信息，请查看相关教程[Tensorboard：训练过程可视化](../how_tos/summaries_and_tensorboard.md)。\r\n\r\n#### 保存检查点（checkpoint）\r\n\r\n为了得到可以用来后续恢复模型以进一步训练或评估的检查点文件（checkpoint file），我们实例化一个[`tf.train.Saver`](../api_docs/python/state_ops.md#Saver)。\r\n\r\n```python\r\nsaver = tf.train.Saver()\r\n```\r\n\r\n在训练循环中，将定期调用[`saver.save()`](../api_docs/python/state_ops.md#Saver.save)方法，向训练文件夹中写入包含了当前所有可训练变量值得检查点文件。\r\n\r\n```python\r\nsaver.save(sess, FLAGS.train_dir, global_step=step)\r\n```\r\n\r\n这样，我们以后就可以使用[`saver.restore()`](../api_docs/python/state_ops.md#Saver.restore)方法，重载模型的参数，继续训练。\r\n\r\n```python\r\nsaver.restore(sess, FLAGS.train_dir)\r\n```\r\n## 评估模型 \r\n\r\n每隔一千个训练步骤，我们的代码会尝试使用训练数据集与测试数据集，对模型进行评估。`do_eval`函数会被调用三次，分别使用训练数据集、验证数据集合测试数据集。\r\n\r\n```python\r\nprint 'Training Data Eval:'\r\ndo_eval(sess,\r\n eval_correct,\r\n images_placeholder,\r\n labels_placeholder,\r\n data_sets.train)\r\nprint 'Validation Data Eval:'\r\ndo_eval(sess,\r\n eval_correct,\r\n images_placeholder,\r\n labels_placeholder,\r\n data_sets.validation)\r\nprint 'Test Data Eval:'\r\ndo_eval(sess,\r\n eval_correct,\r\n images_placeholder,\r\n labels_placeholder,\r\n data_sets.test)\r\n```\r\n\r\n>注意，更复杂的使用场景通常是，先隔绝`data_sets.test`测试数据集，只有在大量的超参数优化调整（hyperparameter tuning）之后才进行检查。但是，由于MNIST问题比较简单，我们在这里一次性评估所有的数据。\r\n\r\n### 构建评估图表（Eval Graph）\r\n\r\n在打开默认图表（Graph）之前，我们应该先调用`get_data(train=False)`函数，抓取测试数据集。\r\n\r\n```python\r\ntest_all_images, test_all_labels = get_data(train=False)\r\n```\r\n\r\n在进入训练循环之前，我们应该先调用`mnist.py`文件中的`evaluation`函数，传入的logits和标签参数要与`loss`函数的一致。这样做事为了先构建Eval操作。\r\n\r\n```python\r\neval_correct = mnist.evaluation(logits, labels_placeholder)\r\n```\r\n\r\n`evaluation`函数会生成[`tf.nn.in_top_k`](../api_docs/python/nn.md#in_top_k)\r\n操作，如果在K个最有可能的预测中可以发现真的标签，那么这个操作就会将模型输出标记为正确。在本文中，我们把K的值设置为1，也就是只有在预测是真的标签时，才判定它是正确的。\r\n\r\n```python\r\neval_correct = tf.nn.in_top_k(logits, labels, 1)\r\n```\r\n\r\n### 评估图表的输出（Eval Output） \r\n\r\n之后，我们可以创建一个循环，往其中添加`feed_dict`，并在调用`sess.run()`函数时传入`eval_correct`操作，目的就是用给定的数据集评估模型。\r\n\r\n```python\r\nfor step in xrange(steps_per_epoch):\r\n feed_dict = fill_feed_dict(data_set,\r\n images_placeholder,\r\n labels_placeholder)\r\n true_count += sess.run(eval_correct, feed_dict=feed_dict)\r\n```\r\n\r\n`true_count`变量会累加所有`in_top_k`操作判定为正确的预测之和。接下来，只需要将正确测试的总数，除以例子总数，就可以得出准确率了。\r\n\r\n```python\r\nprecision = float(true_count) / float(num_examples)\r\nprint ' Num examples: %d Num correct: %d Precision @ 1: %0.02f' % (\r\n num_examples, true_count, precision)\r\n```\r\n\r\n> 原文：[TensorFlow Mechanics 101](http://www.tensorflow.org/tutorials/mnist/tf/index.md)\r\n> 翻译：[bingjin](https://github.com/bingjin)\r\n> 校对：[LichAmnesia](https://github.com/LichAmnesia)\r\n" }, { "path": "SOURCE/tutorials/overview.md", "content": "# 综述\r\n\r\n## 面向机器学习初学者的 MNIST 初级教程\r\n\r\n如果你是机器学习领域的新手, 我们推荐你从本文开始阅读. 本文通过讲述一个经典的问题, 手写数字识别 (MNIST), 让你对多类分类 (multiclass classification) 问题有直观的了解.\r\n\r\n[阅读教程](../tutorials/mnist_beginners.md)\r\n\r\n## 面向机器学习专家的 MNIST 高级教程\r\n\r\n如果你已经对其它深度学习软件比较熟悉, 并且也对 MNIST 很熟悉, 这篇教程能够引导你对 TensorFlow 有初步了解.\r\n\r\n[阅读教程](../tutorials/mnist_pros.md)\r\n\r\n## TensorFlow 使用指南\r\n\r\n这是一篇技术教程, 详细介绍了如何使用 TensorFlow 架构训练大规模模型. 本文继续使用MNIST 作为例子.\r\n\r\n[阅读教程](../tutorials/mnist_tf.md)\r\n\r\n## 卷积神经网络 \r\n\r\n这篇文章介绍了如何使用 TensorFlow 在 CIFAR-10 数据集上训练卷积神经网络. 卷积神经网络是为图像识别量身定做的一个模型. 相比其它模型, 该模型利用了平移不变性(translation invariance), 从而能够更更简洁有效地表示视觉内容.\r\n\r\n[阅读教程](../tutorials/deep_cnn.md)\r\n\r\n## 单词的向量表示\r\n\r\n本文让你了解为什么学会使用向量来表示单词, 即单词嵌套 (word embedding), 是一件很有用的事情. 文章中介绍的 word2vec 模型, 是一种高效学习嵌套的方法. 本文还涉及了对比噪声(noise-contrastive) 训练方法的一些高级细节, 该训练方法是训练嵌套领域最近最大的进展. \r\n\r\n[阅读教程](../tutorials/word2vec.md)\r\n\r\n## 循环神经网络 (Recurrent Neural Network, 简称 RNN)\r\n\r\n一篇 RNN 的介绍文章, 文章中训练了一个 LSTM 网络来预测一个英文句子的下一个单词(该任务有时候被称作语言建模).\r\n\r\n[阅读教程](../tutorials/recurrent.md)\r\n\r\n## 序列到序列模型 (Sequence-to-Sequence Model)\r\n\r\nRNN 教程的后续, 该教程采用序列到序列模型进行机器翻译. 你将学会构建一个完全基于机器学习,端到端的 `英语-法语` 翻译器.\r\n\r\n[阅读教程](../tutorials/seq2seq.md)\r\n\r\n## Mandelbrot 集合\r\n\r\nTensorFlow 可以用于与机器学习完全无关的其它计算领域. 这里实现了一个原生的 Mandelbrot 集合的可视化程序.\r\n\r\n[阅读教程](../tutorials/mandelbrot.md)\r\n\r\n## 偏微分方程\r\n\r\n这是另外一个非机器学习计算的例子, 我们利用一个原生实现的偏微分方程, 对雨滴落在池塘上的过程进行仿真.\r\n\r\n[阅读教程](../tutorials/pdes.md)\r\n\r\n## MNIST 数据下载\r\n\r\n一篇关于下载 MNIST 手写识别数据集的详细教程.\r\n\r\n[阅读教程](../tutorials/mnist_download.md)\r\n\r\n## 视觉物体识别 (Visual Object Recognition)\r\n\r\n我们将毫无保留地发布已经选训练好的, 目前最先进的 Inception 物体识别模型.\r\n\r\n敬请期待...\r\n\r\n## Deep Dream 视幻觉软件\r\n\r\n我们将发布一个 TensorFlow 版本的 [Deep Dream](https://github.com/google/deepdream),这是一款基于 Inception 识别模型的神经网络视幻觉软件.\r\n\r\n敬请期待...\r\n\r\n\r\n
\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
\r\n\r\n> 原文：[Overview](http://tensorflow.org/tutorials) 翻译：[@doc001](https://github.com/PFZheng) 校对：[@eric_xu](https://github.com/ericxk)\r\n\r\n\r\n"
},
{
"path": "SOURCE/tutorials/pdes/index.md",
"content": "# 偏积分方程 \r\n\r\n ***TensorFlow*** 不仅仅是用来机器学习，它更可以用来模拟仿真。在这里，我们将通过模拟仿真几滴落入一块方形水池的雨点的例子，来引导您如何使用 ***TensorFlow*** 中的偏积分方程来模拟仿真的基本使用方法。\r\n\r\n>注：本教程最初是准备做为一个 **IPython** 的手册。\r\n>>译者注:关于偏积分方程的相关知识，译者推荐读者查看 [**网易公开课**](http://open.163.com/) 上的[**《麻省理工学院公开课：多变量微积分》**](http://open.163.com/special/opencourse/multivariable.html)课程。\r\n\r\n## 基本设置 \r\n\r\n首先,我们需要导入一些必要的引用。\r\n\r\n```python\r\n#Import libraries for simulation\r\nimport tensorflow as tf\r\nimport numpy as np\r\n\r\n#Imports for visualization\r\nimport PIL.Image\r\nfrom cStringIO import StringIO\r\nfrom IPython.display import clear_output, Image, display\r\n```\r\n\r\n然后，我们还需要一个用于表示池塘表面状态的函数。\r\n\r\n```python\r\ndef DisplayArray(a, fmt='jpeg', rng=[0,1]):\r\n \"\"\"Display an array as a picture.\"\"\"\r\n a = (a - rng[0])/float(rng[1] - rng[0])*255\r\n a = np.uint8(np.clip(a, 0, 255))\r\n f = StringIO()\r\n PIL.Image.fromarray(a).save(f, fmt)\r\n display(Image(data=f.getvalue()))\r\n```\r\n\r\n最后，为了方便演示，这里我们需要打开一个交互的 ***TensorFlow*** 会话。当然为了以后能方便调用，我们可以把相关代码写到一个可以执行的***Python***文件中。\r\n\r\n```python\r\nsess = tf.InteractiveSession()\r\n```\r\n\r\n## 定义计算函数 \r\n\r\n\r\n```python\r\ndef make_kernel(a):\r\n \"\"\"Transform a 2D array into a convolution kernel\"\"\"\r\n a = np.asarray(a)\r\n a = a.reshape(list(a.shape) + [1,1])\r\n return tf.constant(a, dtype=1)\r\n\r\ndef simple_conv(x, k):\r\n \"\"\"A simplified 2D convolution operation\"\"\"\r\n x = tf.expand_dims(tf.expand_dims(x, 0), -1)\r\n y = tf.nn.depthwise_conv2d(x, k, [1, 1, 1, 1], padding='SAME')\r\n return y[0, :, :, 0]\r\n\r\ndef laplace(x):\r\n \"\"\"Compute the 2D laplacian of an array\"\"\"\r\n laplace_k = make_kernel([[0.5, 1.0, 0.5],\r\n [1.0, -6., 1.0],\r\n [0.5, 1.0, 0.5]])\r\n return simple_conv(x, laplace_k)\r\n```\r\n\r\n## 定义偏积分方程 \r\n\r\n首先,我们需要创建一个完美的 500 × 500 的正方形池塘,就像是我们在现实中找到的一样。.\r\n\r\n```python\r\nN = 500\r\n```\r\n\r\n然后，我们需要创建了一个池塘和几滴将要坠入池塘的雨滴。\r\n\r\n```python\r\n# Initial Conditions -- some rain drops hit a pond\r\n\r\n# Set everything to zero\r\nu_init = np.zeros([N, N], dtype=\"float32\")\r\nut_init = np.zeros([N, N], dtype=\"float32\")\r\n\r\n# Some rain drops hit a pond at random points\r\nfor n in range(40):\r\n a,b = np.random.randint(0, N, 2)\r\n u_init[a,b] = np.random.uniform()\r\n\r\nDisplayArray(u_init, rng=[-0.1, 0.1])\r\n```\r\n\r\n\r\n\r\n现在，让我们来指定该微分方程的一些详细参数。\r\n\r\n```python\r\n# Parameters:\r\n# eps -- time resolution\r\n# damping -- wave damping\r\neps = tf.placeholder(tf.float32, shape=())\r\ndamping = tf.placeholder(tf.float32, shape=())\r\n\r\n# Create variables for simulation state\r\nU = tf.Variable(u_init)\r\nUt = tf.Variable(ut_init)\r\n\r\n# Discretized PDE update rules\r\nU_ = U + eps * Ut\r\nUt_ = Ut + eps * (laplace(U) - damping * Ut)\r\n\r\n# Operation to update the state\r\nstep = tf.group(\r\n U.assign(U_),\r\n Ut.assign(Ut_))\r\n```\r\n\r\n## 开始仿真 \r\n\r\n为了能看清仿真效果，我们可以用一个简单的 **for** 循环来远行我们的仿真程序。\r\n\r\n```python\r\n# Initialize state to initial conditions\r\ntf.initialize_all_variables().run()\r\n\r\n# Run 1000 steps of PDE\r\nfor i in range(1000):\r\n # Step simulation\r\n step.run({eps: 0.03, damping: 0.04})\r\n # Visualize every 50 steps\r\n if i % 50 == 0:\r\n clear_output()\r\n DisplayArray(U.eval(), rng=[-0.1, 0.1])\r\n```\r\n\r\n\r\n\r\n看！！ 雨点落在池塘中,和现实中泛起了无数涟漪。\r\n\r\n> 原文链接:[http://tensorflow.org/tutorials/pdes/index.md](http://tensorflow.org/tutorials/pdes/index.md) 翻译:[@wangaicc](https://github.com/wangaicc) 校对： \r\n"
},
{
"path": "SOURCE/tutorials/pdes.md",
"content": "# 偏微分方程 \r\n\r\n***TensorFlow*** 不仅仅是用来机器学习，它更可以用来模拟仿真。在这里，我们将通过模拟仿真几滴落入一块方形水池的雨点的例子，来引导您如何使用 ***TensorFlow*** 中的偏微分方程来模拟仿真的基本使用方法。\r\n\r\n>注：本教程最初是准备做为一个 **IPython** 的手册。\r\n>>译者注:关于偏微分方程的相关知识，译者推荐读者查看 [**网易公开课**](http://open.163.com/) 上的[**《麻省理工学院公开课：多变量微积分》**](http://open.163.com/special/opencourse/multivariable.html)课程。\r\n\r\n## 基本设置 \r\n\r\n首先,我们需要导入一些必要的引用。\r\n\r\n```python\r\n#导入模拟仿真需要的库\r\nimport tensorflow as tf\r\nimport numpy as np\r\n\r\n#导入可视化需要的库\r\nimport PIL.Image\r\nfrom cStringIO import StringIO\r\nfrom IPython.display import clear_output, Image, display\r\n```\r\n\r\n然后，我们还需要一个用于表示池塘表面状态的函数。\r\n\r\n```python\r\ndef DisplayArray(a, fmt='jpeg', rng=[0,1]):\r\n \"\"\"Display an array as a picture.\"\"\"\r\n a = (a - rng[0])/float(rng[1] - rng[0])*255\r\n a = np.uint8(np.clip(a, 0, 255))\r\n f = StringIO()\r\n PIL.Image.fromarray(a).save(f, fmt)\r\n display(Image(data=f.getvalue()))\r\n```\r\n\r\n最后，为了方便演示，这里我们需要打开一个 ***TensorFlow*** 的交互会话（interactive session）。当然为了以后能方便调用，我们可以把相关代码写到一个可以执行的***Python***文件中。\r\n\r\n```python\r\nsess = tf.InteractiveSession()\r\n```\r\n\r\n## 定义计算函数 \r\n\r\n```python\r\ndef make_kernel(a):\r\n \"\"\"Transform a 2D array into a convolution kernel\"\"\"\r\n a = np.asarray(a)\r\n a = a.reshape(list(a.shape) + [1,1])\r\n return tf.constant(a, dtype=1)\r\n\r\ndef simple_conv(x, k):\r\n \"\"\"A simplified 2D convolution operation\"\"\"\r\n x = tf.expand_dims(tf.expand_dims(x, 0), -1)\r\n y = tf.nn.depthwise_conv2d(x, k, [1, 1, 1, 1], padding='SAME')\r\n return y[0, :, :, 0]\r\n\r\ndef laplace(x):\r\n \"\"\"Compute the 2D laplacian of an array\"\"\"\r\n laplace_k = make_kernel([[0.5, 1.0, 0.5],\r\n [1.0, -6., 1.0],\r\n [0.5, 1.0, 0.5]])\r\n return simple_conv(x, laplace_k)\r\n```\r\n\r\n## 定义偏微分方程 \r\n\r\n首先,我们需要创建一个完美的 500 × 500 的正方形池塘,就像是我们在现实中找到的一样。\r\n\r\n```python\r\nN = 500\r\n```\r\n\r\n然后，我们需要创建了一个池塘和几滴将要坠入池塘的雨滴。\r\n\r\n```python\r\n# Initial Conditions -- some rain drops hit a pond\r\n\r\n# Set everything to zero\r\nu_init = np.zeros([N, N], dtype=\"float32\")\r\nut_init = np.zeros([N, N], dtype=\"float32\")\r\n\r\n# Some rain drops hit a pond at random points\r\nfor n in range(40):\r\n a,b = np.random.randint(0, N, 2)\r\n u_init[a,b] = np.random.uniform()\r\n\r\nDisplayArray(u_init, rng=[-0.1, 0.1])\r\n```\r\n\r\n\r\n\r\n现在，让我们来指定该微分方程的一些详细参数。\r\n\r\n```python\r\n# Parameters:\r\n# eps -- time resolution\r\n# damping -- wave damping\r\neps = tf.placeholder(tf.float32, shape=())\r\ndamping = tf.placeholder(tf.float32, shape=())\r\n\r\n# Create variables for simulation state\r\nU = tf.Variable(u_init)\r\nUt = tf.Variable(ut_init)\r\n\r\n# Discretized PDE update rules\r\nU_ = U + eps * Ut\r\nUt_ = Ut + eps * (laplace(U) - damping * Ut)\r\n\r\n# Operation to update the state\r\nstep = tf.group(\r\n U.assign(U_),\r\n Ut.assign(Ut_))\r\n```\r\n\r\n## 开始仿真 \r\n\r\n为了能看清仿真效果，我们可以用一个简单的 **for** 循环来远行我们的仿真程序。\r\n\r\n```python\r\n# Initialize state to initial conditions\r\ntf.initialize_all_variables().run()\r\n\r\n# Run 1000 steps of PDE\r\nfor i in range(1000):\r\n # Step simulation\r\n step.run({eps: 0.03, damping: 0.04})\r\n # Visualize every 50 steps\r\n if i % 50 == 0:\r\n clear_output()\r\n DisplayArray(U.eval(), rng=[-0.1, 0.1])\r\n```\r\n\r\n\r\n\r\n看！！ 雨点落在池塘中,和现实中一样的泛起了涟漪。\r\n\r\n> 原文链接:[http://tensorflow.org/tutorials/pdes/index.md](http://tensorflow.org/tutorials/pdes/index.md) 翻译:[@wangaicc](https://github.com/wangaicc) 校对:[@tensorfly](https://github.com/tensorfly)\r\n"
},
{
"path": "SOURCE/tutorials/recurrent/index.md",
"content": "# Recurrent Neural Networks \r\n\r\n## Introduction \r\n\r\nTake a look at [this great article]\r\n(http://colah.github.io/posts/2015-08-Understanding-LSTMs/)\r\nfor an introduction to recurrent neural networks and LSTMs in particular.\r\n\r\n## Language Modeling \r\n\r\nIn this tutorial we will show how to train a recurrent neural network on\r\na challenging task of language modeling. The goal of the problem is to fit a\r\nprobabilistic model which assigns probablities to sentences. It does so by\r\npredicting next words in a text given a history of previous words. For this\r\npurpose we will use the Penn Tree Bank (PTB) dataset, which is a popular\r\nbenchmark for measuring quality of these models, whilst being small and\r\nrelatively fast to train.\r\n\r\nLanguage modeling is key to many interesting problems such as speech\r\nrecognition, machine translation, or image captioning. It is also fun, too --\r\ntake a look [here] (http://karpathy.github.io/2015/05/21/rnn-effectiveness/).\r\n\r\nFor the purpose of this tutorial, we will reproduce the results from\r\n[Zaremba et al., 2014] (http://arxiv.org/abs/1409.2329), which achieves very\r\ngood results on the PTB dataset.\r\n\r\n## Tutorial Files \r\n\r\nThis tutorial references the following files from `models/rnn/ptb`:\r\n\r\nFile | Purpose\r\n--- | ---\r\n`ptb_word_lm.py` | The code to train a language model on the PTB dataset.\r\n`reader.py` | The code to read the dataset.\r\n\r\n## Download and Prepare the Data \r\n\r\nThe data required for this tutorial is in the data/ directory of the\r\nPTB dataset from Tomas Mikolov's webpage:\r\nhttp://www.fit.vutbr.cz/~imikolov/rnnlm/simple-examples.tgz\r\n\r\nThe dataset is already preprocessed and contains overall 10000 different words,\r\nincluding the end-of-sentence marker and a special symbol (\\\r\n![](\"basic_seq2seq.png\")
\r\n
\r\n\r\nEach box in the picture above represents a cell of the RNN, most commonly\r\na GRU cell or an LSTM cell (see the [RNN Tutorial](../../tutorials/recurrent/index.md)\r\nfor an explanation of those). Encoder and decoder can share weights or,\r\nas is more common, use a different set of parameters. Mutli-layer cells\r\nhave been successfully used in sequence-to-sequence models too, e.g. for\r\ntranslation [Sutskever et al., 2014](http://arxiv.org/abs/1409.3215).\r\n\r\nIn the basic model depicted above, every input has to be encoded into\r\na fixed-size state vector, as that is the only thing passed to the decoder.\r\nTo allow the decoder more direct access to the input, an *attention* mechanism\r\nwas introduced in [Bahdanu et al., 2014](http://arxiv.org/abs/1409.0473).\r\nWe will not go into the details of the attention mechanism (see the paper),\r\nsuffice it to say that it allows the decoder to peek into the input at every\r\ndecoding step. A multi-layer sequence-to-sequence network with LSTM cells and\r\nattention mechanism in the decoder looks like this.\r\n\r\n\r\n\r\n![](\"attention_seq2seq.png\")
\r\n
\r\n\r\n## TensorFlow seq2seq Library \r\n\r\nAs you can see above, there are many different sequence-to-sequence\r\nmodels. Each of these models can use different RNN cells, but all\r\nof them accept encoder inputs and decoder inputs. This motivates\r\nthe interfaces in the TensorFlow seq2seq library (`models/rnn/seq2seq.py`).\r\nThe basic RNN encoder-decoder sequence-to-sequence model works as follows.\r\n\r\n```python\r\noutputs, states = basic_rnn_seq2seq(encoder_inputs, decoder_inputs, cell)\r\n```\r\n\r\nIn the above call, `encoder_inputs` are a list of tensors representing inputs\r\nto the encoder, i.e., corresponding to the letters *A, B, C* in the first\r\npicture above. Similarly, `decoder_inputs` are tensors representing inputs\r\nto the decoder, *GO, W, X, Y, Z* on the first picture.\r\n\r\nThe `cell` argument is an instance of the `models.rnn.rnn_cell.RNNCell` class\r\nthat determines which cell will be used inside the model. You can use\r\nan existing cell, such as `GRUCell` or `LSTMCell`, or you can write your own.\r\nMoreover, `rnn_cell` provides wrappers to construct multi-layer cells,\r\nadd dropout to cell inputs or outputs, or to do other transformations,\r\nsee the [RNN Tutorial](../../tutorials/recurrent/index.md) for examples.\r\n\r\nThe call to `basic_rnn_seq2seq` returns two arguments: `outputs` and `states`.\r\nBoth of them are lists of tensors of the same length as `decoder_inputs`.\r\nNaturally, `outputs` correspond to the outputs of the decoder in each time-step,\r\nin the first picture above that would be *W, X, Y, Z, EOS*. The returned\r\n`states` represent the internal state of the decoder at every time-step.\r\n\r\nIn many applications of sequence-to-sequence models, the output of the decoder\r\nat time t is fed back and becomes the input of the decoder at time t+1. At test\r\ntime, when decoding a sequence, this is how the sequence is constructed.\r\nDuring training, on the other hand, it is common to provide the correct input\r\nto the decoder at every time-step, even if the decoder made a mistake before.\r\nFunctions in `seq2seq.py` support both modes using the `feed_previous` argument.\r\nFor example, let's analyze the following use of an embedding RNN model.\r\n\r\n```python\r\noutputs, states = embedding_rnn_seq2seq(\r\n encoder_inputs, decoder_inputs, cell,\r\n num_encoder_symbols, num_decoder_symbols,\r\n output_projection=None, feed_previous=False)\r\n```\r\n\r\nIn the `embedding_rnn_seq2seq` model, all inputs (both `encoder_inputs` and\r\n`decoder_inputs`) are integer-tensors that represent discrete values.\r\nThey will be embedded into a dense representation (see the\r\n[Vectors Representations Tutorial](../../tutorials/word2vec/index.md) for more details\r\non embeddings), but to construct these embeddings we need to specify\r\nthe maximum number of discrete symbols that will appear: `num_encoder_symbols`\r\non the encoder side, and `num_decoder_symbols` on the decoder side.\r\n\r\nIn the above invocation, we set `feed_previous` to False. This means that the\r\ndecoder will use `decoder_inputs` tensors as provided. If we set `feed_previous`\r\nto True, the decoder would only use the first element of `decoder_inputs`.\r\nAll other tensors from this list would be ignored, and instead the previous\r\noutput of the encoder would be used. This is used for decoding translations\r\nin our translation model, but it can also be used during training, to make\r\nthe model more robust to its own mistakes, similar\r\nto [Bengio et al., 2015](http://arxiv.org/pdf/1506.03099v2.pdf).\r\n\r\nOne more important argument used above is `output_projection`. If not specified,\r\nthe outputs of the embedding model will be tensors of shape batch-size by\r\n`num_decoder_symbols` as they represent the logits for each generated symbol.\r\nWhen training models with large output vocabularies, i.e., when\r\n`num_decoder_symbols` is large, it is not practical to store these large\r\ntensors. Instead, it is better to return smaller output tensors, which will\r\nlater be projected onto a large output tensor using `output_projection`.\r\nThis allows to use our seq2seq models with a sampled softmax loss, as described\r\nin [Jean et. al., 2015](http://arxiv.org/pdf/1412.2007v2.pdf).\r\n\r\nIn addition to `basic_rnn_seq2seq` and `embedding_rnn_seq2seq` there are a few\r\nmore sequence-to-sequence models in `seq2seq.py`, take a look there. They all\r\nhave similar interfaces, so we will not describe them in detail. We will use\r\n`embedding_attention_seq2seq` for our translation model below.\r\n\r\n## Neural Translation Model \r\n\r\nWhile the core of the sequence-to-sequence model is constructed by\r\nthe functions in `models/rnn/seq2seq.py`, there are still a few tricks\r\nthat are worth mentioning that are used in our translation model in\r\n`models/rnn/translate/seq2seq_model.py`.\r\n\r\n### Sampled softmax and output projection \r\n\r\nFor one, as already mentioned above, we want to use sampled softmax to\r\nhandle large output vocabulary. To decode from it, we need to keep track\r\nof the output projection. Both the sampled softmax loss and the output\r\nprojections are constructed by the following code in `seq2seq_model.py`.\r\n\r\n```python\r\n if num_samples > 0 and num_samples < self.target_vocab_size:\r\n w = tf.get_variable(\"proj_w\", [size, self.target_vocab_size])\r\n w_t = tf.transpose(w)\r\n b = tf.get_variable(\"proj_b\", [self.target_vocab_size])\r\n output_projection = (w, b)\r\n\r\n def sampled_loss(inputs, labels):\r\n labels = tf.reshape(labels, [-1, 1])\r\n return tf.nn.sampled_softmax_loss(w_t, b, inputs, labels, num_samples,\r\n self.target_vocab_size)\r\n```\r\n\r\nFirst, note that we only construct a sampled softmax if the number of samples\r\n(512 by default) is smaller that the target vocabulary size. For vocabularies\r\nsmaller than 512 it might be a better idea to just use a standard softmax loss.\r\n\r\nThen, as you can see, we construct an output projection. It is a pair,\r\nconsisting of a weight matrix and a bias vector. If used, the rnn cell\r\nwill return vectors of shape batch-size by `size`, rather than batch-size\r\nby `target_vocab_size`. To recover logits, we need to multiply by the weight\r\nmatrix and add the biases, as is done in lines 124-126 in `seq2seq_model.py`.\r\n\r\n```python\r\nif output_projection is not None:\r\n self.outputs[b] = [tf.matmul(output, output_projection[0]) +\r\n output_projection[1] for ...]\r\n```\r\n\r\n### Bucketing and padding \r\n\r\nIn addition to sampled softmax, our translation model also makes use\r\nof *bucketing*, which is a method to efficiently handle sentences of\r\ndifferent lengths. Let us first clarify the problem. When translating\r\nEnglish to French, we will have English sentences of different lengths L1\r\non input, and French sentences of different lengths L2 on output. Since\r\nthe English sentence is passed as `encoder_inputs`, and the French sentence\r\ncomes as `decoder_inputs` (prefixed by a GO symbol), we should in principle\r\ncreate a seq2seq model for every pair (L1, L2+1) of lengths of an English\r\nand French sentence. This would result in an enormous graph consisting of\r\nmany very similar subgraphs. On the other hand, we could just pad every\r\nsentence with a special PAD symbol. Then we'd need only one seq2seq model,\r\nfor the padded lengths. But on shorter sentence our model would be inefficient,\r\nencoding and decoding many PAD symbols that are useless.\r\n\r\nAs a compromise between contructing a graph for every pair of lengths and\r\npadding to a single length, we use a number of *buckets* and pad each sentence\r\nto the length of the bucket above it. In `translate.py` we use the following\r\ndefault buckets.\r\n\r\n```python\r\nbuckets = [(5, 10), (10, 15), (20, 25), (40, 50)]\r\n```\r\n\r\nThis means that if the input is an English sentence with 3 tokens,\r\nand the corresponding output is a French sentence with 6 tokens,\r\nthen they will be put in the first bucket and padded to length 5 for\r\nencoder inputs, and length 10 for decoder inputs. If we have an English\r\nsentence with 8 tokens and the corresponding French sentence has 18 tokens,\r\nthen they will not fit into the (10, 15) bucket, and so the (20, 25) bucket\r\nwill be used, i.e. the English sentence will be padded to 20, and the French\r\none to 25.\r\n\r\nRemember that when constructing decoder inputs we prepend the special `GO`\r\nsymbol to the input data. This is done in the `get_batch()` function in\r\n`seq2seq_model.py`, which also reverses the input English sentence.\r\nReversing the inputs was shown to improve results for the neural translation\r\nmodel in [Sutskever et al., 2014](http://arxiv.org/abs/1409.3215).\r\nTo put it all together, imagine we have the sentence \"I go.\", tokenized\r\nas `[\"I\", \"go\", \".\"]` as input and the sentence \"Je vais.\" as output,\r\ntokenized `[\"Je\", \"vais\", \".\"]`. It will be put in the (5, 10) bucket,\r\nwith encoder inputs representing `[PAD PAD \".\" \"go\" \"I\"]` and decoder\r\ninputs `[GO \"Je\" \"vais\" \".\" EOS PAD PAD PAD PAD PAD]`.\r\n\r\n\r\n## Let's Run It \r\n\r\nTo train the model described above, we need to a large English-French corpus.\r\nWe will use the *10^9-French-English corpus* from the\r\n[WMT'15 Website](http://www.statmt.org/wmt15/translation-task.html)\r\nfor training, and the 2013 news test from the same site as development set.\r\nBoth data-sets will be downloaded to `data_dir` and training will start,\r\nsaving checkpoints in `train_dir`, when this command is run.\r\n\r\n```\r\nbazel run -c opt <...>/models/rnn/translate:translate\r\n --data_dir [your_data_directory] --train_dir [checkpoints_directory]\r\n --en_vocab_size=40000 --fr_vocab_size=40000\r\n```\r\n\r\nIt takes about 18GB of disk space and several hours to prepare the training\r\ncorpus. It is unpacked, vocabulary files are created in `data_dir`, and then\r\nthe corpus is tokenized and converted to integer ids. Note the parameters\r\nthat determine vocabulary sizes. In the example above, all words outside\r\nthe 40K most common ones will be converted to an `UNK` token representing\r\nunknown words. So if you change vocabulary size, the binary will re-map\r\nthe corpus to token-ids again.\r\n\r\nAfter the data is prepared, training starts. Default parameters in `translate`\r\nare set to quite large values. Large models trained over a long time give good\r\nresults, but it might take too long or use too much memory for your GPU.\r\nYou can request to train a smaller model as in the following example.\r\n\r\n```\r\nbazel run -c opt <...>/models/rnn/translate:translate\r\n --data_dir [your_data_directory] --train_dir [checkpoints_directory]\r\n --size=256 --num_layers=2 --steps_per_checkpoint=50\r\n```\r\n\r\nThe above command will train a model with 2 layers (the default is 3),\r\neach layer with 256 units (default is 1024), and will save a checkpoint\r\nevery 50 steps (the default is 200). You can play with these parameters\r\nto find out how large a model can be to fit into the memory of your GPU.\r\n\r\nDuring training, every `steps_per_checkpoint` steps the binary will print\r\nout statistics from recent steps. With the default parameters (3 layers\r\nof size 1024), first messages look like this.\r\n\r\n```\r\nglobal step 200 learning rate 0.5000 step-time 1.39 perplexity 1720.62\r\n eval: bucket 0 perplexity 184.97\r\n eval: bucket 1 perplexity 248.81\r\n eval: bucket 2 perplexity 341.64\r\n eval: bucket 3 perplexity 469.04\r\nglobal step 400 learning rate 0.5000 step-time 1.38 perplexity 379.89\r\n eval: bucket 0 perplexity 151.32\r\n eval: bucket 1 perplexity 190.36\r\n eval: bucket 2 perplexity 227.46\r\n eval: bucket 3 perplexity 238.66\r\n```\r\n\r\nYou can see that each step takes just under 1.4 seconds, the perplexity\r\non the training set and the perplexities on the development set\r\nfor each bucket. After about 30K steps, we see perplexities on short\r\nsentences (bucket 0 and 1) going into single digits.\r\nSince the training corpus contains ~22M sentences, one epoch (going through\r\nthe training data once) takes about 340K steps with batch-size of 64. At this\r\npoint the model can be used for translating English sentences to French\r\nusing the `--decode` option.\r\n\r\n```\r\nbazel run -c opt <...>/models/rnn/translate:translate --decode\r\n --data_dir [your_data_directory] --train_dir [checkpoints_directory]\r\n\r\nReading model parameters from /tmp/translate.ckpt-340000\r\n> Who is the president of the United States?\r\n Qui est le président des États-Unis ?\r\n```\r\n\r\n## What Next? \r\n\r\nThe example above shows how you can build your own English-to-French\r\ntranslator, end-to-end. Run it and see how the model performs for yourself.\r\nWhile it has reasonable quality, the default parameters will not give you\r\nthe best translation model. Here are a few things you can improve.\r\n\r\nFirst of all, we use a very promitive tokenizer, the `basic_tokenizer` function\r\nin `data_utils`. A better tokenizer can be found on the\r\n[WMT'15 Website](http://www.statmt.org/wmt15/translation-task.html).\r\nUsing that tokenizer, and a larger vocabulary, should improve your translations.\r\n\r\nAlso, the default parameters of the translation model are not tuned.\r\nYou can try changing the learning rate, decay, or initializing the weights\r\nof your model in a different way. You can also change the default\r\n`GradientDescentOptimizer` in `seq2seq_model.py` to a more advanced one, such\r\nas `AdagradOptimizer`. Try these things and see how they improve your results!\r\n\r\nFinally, the model presented above can be used for any sequence-to-sequence\r\ntask, not only for translation. Even if you want to transform a sequence to\r\na tree, for example to generate a parsing tree, the same model as above can\r\ngive state-of-the-art results, as demonstrated in\r\n[Vinyals & Kaiser et al., 2015](http://arxiv.org/abs/1412.7449).\r\nSo you can not only build your own translator, you can also build a parser,\r\na chat-bot, or any program that comes to your mind. Experiment!\r\n"
},
{
"path": "SOURCE/tutorials/seq2seq.md",
"content": "# Sequence-to-Sequence Models \r\n\r\nRecurrent neural networks can learn to model language, as already discussed\r\nin the [RNN Tutorial](tensorflow-zh/SOURCE/tutorials/recurrent/index.md)\r\n(if you did not read it, please go through it before proceeding with this one).\r\nThis raises an interesting question: could we condition the generated words on\r\nsome input and generate a meaningful response? For example, could we train\r\na neural network to translate from English to French? It turns out that\r\nthe answer is *yes*.\r\n\r\nThis tutorial will show you how to build and train such a system end-to-end.\r\nYou can start by running this binary.\r\n\r\n```\r\nbazel run -c opt <...>/models/rnn/translate/translate.py\r\n --data_dir [your_data_directory]\r\n```\r\n\r\nIt will download English-to-French translation data from the\r\n[WMT'15 Website](http://www.statmt.org/wmt15/translation-task.html)\r\nprepare it for training and train. It takes about 20GB of disk space,\r\nand a while to download and prepare (see [later](#run_it) for details),\r\nso you can start and leave it running while reading this tutorial.\r\n\r\nThis tutorial references the following files from `models/rnn`.\r\n\r\nFile | What's in it?\r\n--- | ---\r\n`seq2seq.py` | Library for building sequence-to-sequence models.\r\n`translate/seq2seq_model.py` | Neural translation sequence-to-sequence model.\r\n`translate/data_utils.py` | Helper functions for preparing translation data.\r\n`translate/translate.py` | Binary that trains and runs the translation model.\r\n\r\n\r\n## Sequence-to-Sequence Basics \r\n\r\nA basic sequence-to-sequence model, as introduced in\r\n[Cho et al., 2014](http://arxiv.org/pdf/1406.1078v3.pdf),\r\nconsists of two recurrent neural networks (RNNs): an *encoder* that\r\nprocesses the input and a *decoder* that generates the output.\r\nThis basic architecture is depicted below.\r\n\r\n\r\n\r\n\r\n
\r\n\r\nEach box in the picture above represents a cell of the RNN, most commonly\r\na GRU cell or an LSTM cell (see the [RNN Tutorial](tensorflow-zh/SOURCE/tutorials/recurrent/index.md)\r\nfor an explanation of those). Encoder and decoder can share weights or,\r\nas is more common, use a different set of parameters. Mutli-layer cells\r\nhave been successfully used in sequence-to-sequence models too, e.g. for\r\ntranslation [Sutskever et al., 2014](http://arxiv.org/abs/1409.3215).\r\n\r\nIn the basic model depicted above, every input has to be encoded into\r\na fixed-size state vector, as that is the only thing passed to the decoder.\r\nTo allow the decoder more direct access to the input, an *attention* mechanism\r\nwas introduced in [Bahdanu et al., 2014](http://arxiv.org/abs/1409.0473).\r\nWe will not go into the details of the attention mechanism (see the paper),\r\nsuffice it to say that it allows the decoder to peek into the input at every\r\ndecoding step. A multi-layer sequence-to-sequence network with LSTM cells and\r\nattention mechanism in the decoder looks like this.\r\n\r\n\r\n\r\n\r\n
\r\n\r\n## TensorFlow seq2seq Library \r\n\r\nAs you can see above, there are many different sequence-to-sequence\r\nmodels. Each of these models can use different RNN cells, but all\r\nof them accept encoder inputs and decoder inputs. This motivates\r\nthe interfaces in the TensorFlow seq2seq library (`models/rnn/seq2seq.py`).\r\nThe basic RNN encoder-decoder sequence-to-sequence model works as follows.\r\n\r\n```python\r\noutputs, states = basic_rnn_seq2seq(encoder_inputs, decoder_inputs, cell)\r\n```\r\n\r\nIn the above call, `encoder_inputs` are a list of tensors representing inputs\r\nto the encoder, i.e., corresponding to the letters *A, B, C* in the first\r\npicture above. Similarly, `decoder_inputs` are tensors representing inputs\r\nto the decoder, *GO, W, X, Y, Z* on the first picture.\r\n\r\nThe `cell` argument is an instance of the `models.rnn.rnn_cell.RNNCell` class\r\nthat determines which cell will be used inside the model. You can use\r\nan existing cell, such as `GRUCell` or `LSTMCell`, or you can write your own.\r\nMoreover, `rnn_cell` provides wrappers to construct multi-layer cells,\r\nadd dropout to cell inputs or outputs, or to do other transformations,\r\nsee the [RNN Tutorial](tensorflow-zh/SOURCE/tutorials/recurrent/index.md) for examples.\r\n\r\nThe call to `basic_rnn_seq2seq` returns two arguments: `outputs` and `states`.\r\nBoth of them are lists of tensors of the same length as `decoder_inputs`.\r\nNaturally, `outputs` correspond to the outputs of the decoder in each time-step,\r\nin the first picture above that would be *W, X, Y, Z, EOS*. The returned\r\n`states` represent the internal state of the decoder at every time-step.\r\n\r\nIn many applications of sequence-to-sequence models, the output of the decoder\r\nat time t is fed back and becomes the input of the decoder at time t+1. At test\r\ntime, when decoding a sequence, this is how the sequence is constructed.\r\nDuring training, on the other hand, it is common to provide the correct input\r\nto the decoder at every time-step, even if the decoder made a mistake before.\r\nFunctions in `seq2seq.py` support both modes using the `feed_previous` argument.\r\nFor example, let's analyze the following use of an embedding RNN model.\r\n\r\n```python\r\noutputs, states = embedding_rnn_seq2seq(\r\n encoder_inputs, decoder_inputs, cell,\r\n num_encoder_symbols, num_decoder_symbols,\r\n output_projection=None, feed_previous=False)\r\n```\r\n\r\nIn the `embedding_rnn_seq2seq` model, all inputs (both `encoder_inputs` and\r\n`decoder_inputs`) are integer-tensors that represent discrete values.\r\nThey will be embedded into a dense representation (see the\r\n[Vectors Representations Tutorial](tensorflow-zh/SOURCE/tutorials/word2vec/index.md) for more details\r\non embeddings), but to construct these embeddings we need to specify\r\nthe maximum number of discrete symbols that will appear: `num_encoder_symbols`\r\non the encoder side, and `num_decoder_symbols` on the decoder side.\r\n\r\nIn the above invocation, we set `feed_previous` to False. This means that the\r\ndecoder will use `decoder_inputs` tensors as provided. If we set `feed_previous`\r\nto True, the decoder would only use the first element of `decoder_inputs`.\r\nAll other tensors from this list would be ignored, and instead the previous\r\noutput of the encoder would be used. This is used for decoding translations\r\nin our translation model, but it can also be used during training, to make\r\nthe model more robust to its own mistakes, similar\r\nto [Bengio et al., 2015](http://arxiv.org/pdf/1506.03099v2.pdf).\r\n\r\nOne more important argument used above is `output_projection`. If not specified,\r\nthe outputs of the embedding model will be tensors of shape batch-size by\r\n`num_decoder_symbols` as they represent the logits for each generated symbol.\r\nWhen training models with large output vocabularies, i.e., when\r\n`num_decoder_symbols` is large, it is not practical to store these large\r\ntensors. Instead, it is better to return smaller output tensors, which will\r\nlater be projected onto a large output tensor using `output_projection`.\r\nThis allows to use our seq2seq models with a sampled softmax loss, as described\r\nin [Jean et. al., 2015](http://arxiv.org/pdf/1412.2007v2.pdf).\r\n\r\nIn addition to `basic_rnn_seq2seq` and `embedding_rnn_seq2seq` there are a few\r\nmore sequence-to-sequence models in `seq2seq.py`, take a look there. They all\r\nhave similar interfaces, so we will not describe them in detail. We will use\r\n`embedding_attention_seq2seq` for our translation model below.\r\n\r\n## Neural Translation Model \r\n\r\nWhile the core of the sequence-to-sequence model is constructed by\r\nthe functions in `models/rnn/seq2seq.py`, there are still a few tricks\r\nthat are worth mentioning that are used in our translation model in\r\n`models/rnn/translate/seq2seq_model.py`.\r\n\r\n### Sampled softmax and output projection \r\n\r\nFor one, as already mentioned above, we want to use sampled softmax to\r\nhandle large output vocabulary. To decode from it, we need to keep track\r\nof the output projection. Both the sampled softmax loss and the output\r\nprojections are constructed by the following code in `seq2seq_model.py`.\r\n\r\n```python\r\n if num_samples > 0 and num_samples < self.target_vocab_size:\r\n w = tf.get_variable(\"proj_w\", [size, self.target_vocab_size])\r\n w_t = tf.transpose(w)\r\n b = tf.get_variable(\"proj_b\", [self.target_vocab_size])\r\n output_projection = (w, b)\r\n\r\n def sampled_loss(inputs, labels):\r\n labels = tf.reshape(labels, [-1, 1])\r\n return tf.nn.sampled_softmax_loss(w_t, b, inputs, labels, num_samples,\r\n self.target_vocab_size)\r\n```\r\n\r\nFirst, note that we only construct a sampled softmax if the number of samples\r\n(512 by default) is smaller that the target vocabulary size. For vocabularies\r\nsmaller than 512 it might be a better idea to just use a standard softmax loss.\r\n\r\nThen, as you can see, we construct an output projection. It is a pair,\r\nconsisting of a weight matrix and a bias vector. If used, the rnn cell\r\nwill return vectors of shape batch-size by `size`, rather than batch-size\r\nby `target_vocab_size`. To recover logits, we need to multiply by the weight\r\nmatrix and add the biases, as is done in lines 124-126 in `seq2seq_model.py`.\r\n\r\n```python\r\nif output_projection is not None:\r\n self.outputs[b] = [tf.matmul(output, output_projection[0]) +\r\n output_projection[1] for ...]\r\n```\r\n\r\n### Bucketing and padding \r\n\r\nIn addition to sampled softmax, our translation model also makes use\r\nof *bucketing*, which is a method to efficiently handle sentences of\r\ndifferent lengths. Let us first clarify the problem. When translating\r\nEnglish to French, we will have English sentences of different lengths L1\r\non input, and French sentences of different lengths L2 on output. Since\r\nthe English sentence is passed as `encoder_inputs`, and the French sentence\r\ncomes as `decoder_inputs` (prefixed by a GO symbol), we should in principle\r\ncreate a seq2seq model for every pair (L1, L2+1) of lengths of an English\r\nand French sentence. This would result in an enormous graph consisting of\r\nmany very similar subgraphs. On the other hand, we could just pad every\r\nsentence with a special PAD symbol. Then we'd need only one seq2seq model,\r\nfor the padded lengths. But on shorter sentence our model would be inefficient,\r\nencoding and decoding many PAD symbols that are useless.\r\n\r\nAs a compromise between contructing a graph for every pair of lengths and\r\npadding to a single length, we use a number of *buckets* and pad each sentence\r\nto the length of the bucket above it. In `translate.py` we use the following\r\ndefault buckets.\r\n\r\n```python\r\nbuckets = [(5, 10), (10, 15), (20, 25), (40, 50)]\r\n```\r\n\r\nThis means that if the input is an English sentence with 3 tokens,\r\nand the corresponding output is a French sentence with 6 tokens,\r\nthen they will be put in the first bucket and padded to length 5 for\r\nencoder inputs, and length 10 for decoder inputs. If we have an English\r\nsentence with 8 tokens and the corresponding French sentence has 18 tokens,\r\nthen they will not fit into the (10, 15) bucket, and so the (20, 25) bucket\r\nwill be used, i.e. the English sentence will be padded to 20, and the French\r\none to 25.\r\n\r\nRemember that when constructing decoder inputs we prepend the special `GO`\r\nsymbol to the input data. This is done in the `get_batch()` function in\r\n`seq2seq_model.py`, which also reverses the input English sentence.\r\nReversing the inputs was shown to improve results for the neural translation\r\nmodel in [Sutskever et al., 2014](http://arxiv.org/abs/1409.3215).\r\nTo put it all together, imagine we have the sentence \"I go.\", tokenized\r\nas `[\"I\", \"go\", \".\"]` as input and the sentence \"Je vais.\" as output,\r\ntokenized `[\"Je\", \"vais\", \".\"]`. It will be put in the (5, 10) bucket,\r\nwith encoder inputs representing `[PAD PAD \".\" \"go\" \"I\"]` and decoder\r\ninputs `[GO \"Je\" \"vais\" \".\" EOS PAD PAD PAD PAD PAD]`.\r\n\r\n\r\n## Let's Run It \r\n\r\nTo train the model described above, we need to a large English-French corpus.\r\nWe will use the *10^9-French-English corpus* from the\r\n[WMT'15 Website](http://www.statmt.org/wmt15/translation-task.html)\r\nfor training, and the 2013 news test from the same site as development set.\r\nBoth data-sets will be downloaded to `data_dir` and training will start,\r\nsaving checkpoints in `train_dir`, when this command is run.\r\n\r\n```\r\nbazel run -c opt <...>/models/rnn/translate:translate\r\n --data_dir [your_data_directory] --train_dir [checkpoints_directory]\r\n --en_vocab_size=40000 --fr_vocab_size=40000\r\n```\r\n\r\nIt takes about 18GB of disk space and several hours to prepare the training\r\ncorpus. It is unpacked, vocabulary files are created in `data_dir`, and then\r\nthe corpus is tokenized and converted to integer ids. Note the parameters\r\nthat determine vocabulary sizes. In the example above, all words outside\r\nthe 40K most common ones will be converted to an `UNK` token representing\r\nunknown words. So if you change vocabulary size, the binary will re-map\r\nthe corpus to token-ids again.\r\n\r\nAfter the data is prepared, training starts. Default parameters in `translate`\r\nare set to quite large values. Large models trained over a long time give good\r\nresults, but it might take too long or use too much memory for your GPU.\r\nYou can request to train a smaller model as in the following example.\r\n\r\n```\r\nbazel run -c opt <...>/models/rnn/translate:translate\r\n --data_dir [your_data_directory] --train_dir [checkpoints_directory]\r\n --size=256 --num_layers=2 --steps_per_checkpoint=50\r\n```\r\n\r\nThe above command will train a model with 2 layers (the default is 3),\r\neach layer with 256 units (default is 1024), and will save a checkpoint\r\nevery 50 steps (the default is 200). You can play with these parameters\r\nto find out how large a model can be to fit into the memory of your GPU.\r\n\r\nDuring training, every `steps_per_checkpoint` steps the binary will print\r\nout statistics from recent steps. With the default parameters (3 layers\r\nof size 1024), first messages look like this.\r\n\r\n```\r\nglobal step 200 learning rate 0.5000 step-time 1.39 perplexity 1720.62\r\n eval: bucket 0 perplexity 184.97\r\n eval: bucket 1 perplexity 248.81\r\n eval: bucket 2 perplexity 341.64\r\n eval: bucket 3 perplexity 469.04\r\nglobal step 400 learning rate 0.5000 step-time 1.38 perplexity 379.89\r\n eval: bucket 0 perplexity 151.32\r\n eval: bucket 1 perplexity 190.36\r\n eval: bucket 2 perplexity 227.46\r\n eval: bucket 3 perplexity 238.66\r\n```\r\n\r\nYou can see that each step takes just under 1.4 seconds, the perplexity\r\non the training set and the perplexities on the development set\r\nfor each bucket. After about 30K steps, we see perplexities on short\r\nsentences (bucket 0 and 1) going into single digits.\r\nSince the training corpus contains ~22M sentences, one epoch (going through\r\nthe training data once) takes about 340K steps with batch-size of 64. At this\r\npoint the model can be used for translating English sentences to French\r\nusing the `--decode` option.\r\n\r\n```\r\nbazel run -c opt <...>/models/rnn/translate:translate --decode\r\n --data_dir [your_data_directory] --train_dir [checkpoints_directory]\r\n\r\nReading model parameters from /tmp/translate.ckpt-340000\r\n> Who is the president of the United States?\r\n Qui est le président des États-Unis ?\r\n```\r\n\r\n## What Next? \r\n\r\nThe example above shows how you can build your own English-to-French\r\ntranslator, end-to-end. Run it and see how the model performs for yourself.\r\nWhile it has reasonable quality, the default parameters will not give you\r\nthe best translation model. Here are a few things you can improve.\r\n\r\nFirst of all, we use a very promitive tokenizer, the `basic_tokenizer` function\r\nin `data_utils`. A better tokenizer can be found on the\r\n[WMT'15 Website](http://www.statmt.org/wmt15/translation-task.html).\r\nUsing that tokenizer, and a larger vocabulary, should improve your translations.\r\n\r\nAlso, the default parameters of the translation model are not tuned.\r\nYou can try changing the learning rate, decay, or initializing the weights\r\nof your model in a different way. You can also change the default\r\n`GradientDescentOptimizer` in `seq2seq_model.py` to a more advanced one, such\r\nas `AdagradOptimizer`. Try these things and see how they improve your results!\r\n\r\nFinally, the model presented above can be used for any sequence-to-sequence\r\ntask, not only for translation. Even if you want to transform a sequence to\r\na tree, for example to generate a parsing tree, the same model as above can\r\ngive state-of-the-art results, as demonstrated in\r\n[Vinyals & Kaiser et al., 2015](http://arxiv.org/abs/1412.7449).\r\nSo you can not only build your own translator, you can also build a parser,\r\na chat-bot, or any program that comes to your mind. Experiment!\r\n"
},
{
"path": "SOURCE/tutorials/word2vec/__init__.py",
"content": ""
},
{
"path": "SOURCE/tutorials/word2vec/index.md",
"content": "# Vector Representations of Words \r\n\r\n\r\n在本教程我们来看一下[Mikolov et al](http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf)中提到的word2vec模型。该模型是用于学习文字的向量表示，称之为“word embedding”。\r\n\r\n## 亮点 \r\n\r\n\r\n本教程意在展现出在TensorfLow中构建word2vec模型有趣、本质的部分。\r\n\r\n\r\n* 我们从我们为何需要使用向量表示文字开始。\r\n* 我们通过直观地例子观察模型背后的本质，以及它是如何训练的（通过一些数学方法评估）。\r\n* 同时我们也展示了TensorFlow对该模型的简单实现。\r\n* 最后，我们着眼于让给这个简单版本的模型表现更好。\r\n\r\n我们会在教程的推进中循序渐进地解释代码，但是如果你更希望直入主题，可以在\r\n[tensorflow/examples/tutorials/word2vec/word2vec_basic.py](https://www.tensorflow.org/code/tensorflow/examples/tutorials/word2vec/word2vec_basic.py)查看到一个最简单的实现。这个基本的例子提供的代码可以完成下载一些数据，简单训练后展示结果。一旦你觉得已经完全掌握了这个简单版本，你可以查看\r\n[tensorflow/models/embedding/word2vec.py](https://www.tensorflow.org/code/tensorflow/models/embedding/word2vec.py)，这里提供了一些更复杂的实现，同时也展示了TensorFlow的一些更进阶的特性，比如如何更高效地使用线程将数据送入文本模型，再比如如何在训练中设置检查点等等。\r\n\r\n但是首先，让我们来看一下为何需要学习word embeddings。如果你对word embeddings相关内容已经是个专家了，那么请安心跳过本节内容，直接深入细节干一些脏活吧。\r\n\r\n## 动机: 为什么需要学习 Word Embeddings? \r\n\r\n通常图像或音频系统处理的是由图片中所有单个原始像素点强度值或者音频中功率谱密度的强度值，把它们编码成丰富、高纬度的向量数据集。对于物体或语音识别这一类的任务，我们所需的全部信息已经都存储在原始数据中（显然人类本身就是依赖原始数据进行日常的物体或语音识别的）。然后，自然语言处理系统通常将词汇作为离散的单一符号，例如 \"cat\" 一词或可表示为 `Id537` ，而 \"dog\" 一词或可表示为 `Id143`。这些符号编码毫无规律，无法提供不同词汇之间可能存在的关联信息。换句话说，在处理关于 \"dogs\" 一词的信息时，模型将无法利用已知的关于 \"cats\" 的信息（例如，它们都是动物，有四条腿，可作为宠物等等）。可见，将词汇表达为上述的独立离散符号将进一步导致数据稀疏，使我们在训练统计模型时不得不寻求更多的数据。而词汇的向量表示将克服上述的难题。\r\n\r\n\r\n\r\n![](\"img/audio-image-text.png\")
\r\n
\r\n\r\n[向量空间模型](https://en.wikipedia.org/wiki/Vector_space_model) (VSMs)将词汇表达（嵌套）于一个连续的向量空间中，语义近似的词汇被映射为相邻的数据点。向量空间模型在自然语言处理领域中有着漫长且丰富的历史，不过几乎所有利用这一模型的方法都依赖于\r\n[分布式假设](https://en.wikipedia.org/wiki/Distributional_semantics#Distributional_Hypothesis)，其核心思想为出现于上下文情景中的词汇都有相类似的语义。采用这一假设的研究方法大致分为以下两类：*基于技术的方法* (e.g.\r\n[潜在语义分析](https://en.wikipedia.org/wiki/Latent_semantic_analysis))，\r\n和 *预测方法* (e.g.\r\n[神经概率化语言模型](http://www.scholarpedia.org/article/Neural_net_language_models)).\r\n\r\n其中它们的区别在如下论文中又详细阐述\r\n[Baroni et al.](http://clic.cimec.unitn.it/marco/publications/acl2014/baroni-etal-countpredict-acl2014.pdf)，不过简而言之：基于计数的方法计算某词汇与其邻近词汇在一个大型语料库中共同出现的频率及其他统计量，然后将这些统计量映射到一个小型且稠密的向量中。预测方法则试图直接从某词汇的邻近词汇对其进行预测，在此过程中利用已经学习到的小型且稠密的*嵌套向量*。 \r\n\r\nWord2vec是一种可以进行高效率词嵌套学习的预测模型。其两种变体分别为：连续词袋模型（CBOW）及Skip-Gram模型。从算法角度看，这两种方法非常相似，其区别为CBOW根据源词上下文词汇（'the cat sits on the'）来预测目标词汇（例如，‘mat’），而Skip-Gram模型做法相反，它通过目标词汇来预测源词汇。Skip-Gram模型采取CBOW的逆过程的动机在于：CBOW算法对于很多分布式信息进行了平滑处理（例如将一整段上下文信息视为一个单一观察量）。很多情况下，对于小型的数据集，这一处理是有帮助的。相形之下，Skip-Gram模型将每个“上下文-目标词汇”的组合视为一个新观察量，这种做法在大型数据集中会更为有效。本教程余下部分将着重讲解Skip-Gram模型。\r\n\r\n\r\n## 处理噪声对比训练 \r\n\r\n神经概率化语言模型通常使用[极大似然法](https://en.wikipedia.org/wiki/Maximum_likelihood) (ML) 进行训练，其中通过 [*softmax* function](https://en.wikipedia.org/wiki/Softmax_function) 来最大化当提供前一个单词 \\\\(h\\\\) (代表 \"history\")，后一个单词的概率 \\\\(w_t\\\\) (代表 \"target\")，\r\n\r\n$$\r\n\\begin{align}\r\nP(w_t | h) &= \\text{softmax}(\\exp \\{ \\text{score}(w_t, h) \\}) \\\\\r\n &= \\frac{\\exp \\{ \\text{score}(w_t, h) \\} }\r\n {\\sum_\\text{Word w' in Vocab} \\exp \\{ \\text{score}(w', h) \\} }.\r\n\\end{align}\r\n$$\r\n\r\n当 \\\\(\\text{score}(w\\_t, h)\\\\) 计算了文字 \\\\(w\\_t\\\\) 和 上下文 \\\\(h\\\\) 的相容性（通常使用向量积）。我们使用对数似然函数来训练训练集的最大值，比如通过：\r\n\r\n$$\r\n\\begin{align}\r\n J_\\text{ML} &= \\log P(w_t | h) \\\\\r\n &= \\text{score}(w_t, h) -\r\n \\log \\left( \\sum_\\text{Word w' in Vocab} \\exp \\{ \\text{score}(w', h) \\} \\right)\r\n\\end{align}\r\n$$\r\n\r\n这里提出了一个解决语言概率模型的合适的通用方法。然而这个方法实际执行起来开销非常大，因为我们需要去计算并正则化当前上下文环境 \\\\(h\\\\) 中所有其他 \\\\(V\\\\) 单词 \\\\(w'\\\\) 的概率得分，*在每一步训练迭代中*。\r\n\r\n\r\n\r\n![](\"img/softmax-nplm.png\")
\r\n
\r\n\r\n从另一个角度来说，当使用word2vec模型时，我们并不需要对概率模型中的所有特征进行学习。而CBOW模型和Skip-Gram模型为了避免这种情况发生，使用一个二分类器（逻辑回归）在同一个上下文环境里从 \\\\(k\\\\) 虚构的 (噪声) 单词 \\\\(\\tilde w\\\\) 区分出真正的目标单词 \\\\(w_t\\\\)。我们下面详细阐述一下CBOW模型，对于Skip-Gram模型只要简单地做相反的操作即可。\r\n\r\n\r\n\r\n![](\"img/nce-nplm.png\")
\r\n
\r\n\r\n从数学角度来说，我们的目标是对每个样本最大化：\r\n\r\n$$J_\\text{NEG} = \\log Q_\\theta(D=1 |w_t, h) +\r\n k \\mathop{\\mathbb{E}}_{\\tilde w \\sim P_\\text{noise}}\r\n \\left[ \\log Q_\\theta(D = 0 |\\tilde w, h) \\right]$$\r\n\r\n其中 \\\\(Q_\\theta(D=1 | w, h)\\\\) 代表的是数据集在当前上下文 \\\\(h\\\\) ，根据所学习的嵌套向量 \\\\(\\theta\\\\) ，目标单词 \\\\(w\\\\) 使用二分类逻辑回归计算得出的概率。在实践中，我们通过在噪声分布中绘制比对文字来获得近似的期望值（通过计算[蒙特卡洛平均值](https://en.wikipedia.org/wiki/Monte_Carlo_integration)）。\r\n\r\n当真实地目标单词被分配到较高的概率，同时噪声单词的概率很低时，目标函数也就达到最大值了。从技术层面来说，这种方法叫做\r\n[负抽样](http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf)，而且使用这个损失函数在数学层面上也有很好的解释：这个更新过程也近似于softmax函数的更新。这在计算上将会有很大的优势，因为当计算这个损失函数时，只是有我们挑选出来的 (\\\\(k\\\\)) 个 *噪声单词*，而没有使用整个语料库 (\\\\(V\\\\))。这使得训练变得非常快。我们实际上使用了与[noise-contrastive estimation (NCE)](http://papers.nips.cc/paper/5165-learning-word-embeddings-efficiently-with-noise-contrastive-estimation.pdf)介绍的非常相似的方法，这在TensorFlow中已经封装了一个很便捷的函数`tf.nn.nce_loss()`。\r\n\r\n让我们在实践中来直观地体会它是如何运作的！\r\n\r\n## Skip-gram 模型\r\n\r\n下面来看一下这个数据集\r\n\r\n`the quick brown fox jumped over the lazy dog`\r\n\r\n我们首先对一些单词以及它们的上下文环境建立一个数据集。我们可以以任何合理的方式定义‘上下文’，而通常上这个方式是根据文字的句法语境的（使用语法原理的方式处理当前目标单词可以看一下这篇文献 [Levy et al.](https://levyomer.files.wordpress.com/2014/04/dependency-based-word-embeddings-acl-2014.pdf)，比如说把目标单词左边的内容当做一个‘上下文’，或者以目标单词右边的内容，等等。现在我们把目标单词的左右单词视作一个上下文， 使用大小为1的窗口，这样就得到这样一个由`(上下文, 目标单词)` 组成的数据集：\r\n\r\n`([the, brown], quick), ([quick, fox], brown), ([brown, jumped], fox), ...`\r\n\r\n前文提到Skip-Gram模型是把目标单词和上下文颠倒过来，所以在这个问题中，举个例子，就是用'quick'来预测 'the' 和 'brown' ，用 'brown' 预测 'quick' 和 'brown' 。因此这个数据集就变成由`(输入, 输出)`组成的：\r\n\r\n`(quick, the), (quick, brown), (brown, quick), (brown, fox), ...`\r\n\r\n目标函数通常是对整个数据集建立的，但是本问题中要对每一个样本（或者是一个`batch_size` 很小的样本集，通常设置为`16 <= batch_size <= 512`）在同一时间执行特别的操作，称之为[随机梯度下降](https://en.wikipedia.org/wiki/Stochastic_gradient_descent)\r\n(SGD)。我们来看一下训练过程中每一步的执行。\r\n\r\n假设用 \\\\(t\\\\)表示上面这个例子中`quick` 来预测 `the` 的训练的单个循环。用 `num_noise` 定义从噪声分布中挑选出来的噪声（相反的）单词的个数，通常使用一元分布，\\\\(P(w)\\\\)。为了简单起见，我们就定`num_noise=1`，用 `sheep` 选作噪声词。接下来就可以计算每一对观察值和噪声值的损失函数了，每一个执行步骤就可表示为：\r\n\r\n$$J^{(t)}_\\text{NEG} = \\log Q_\\theta(D=1 | \\text{the, quick}) +\r\n \\log(Q_\\theta(D=0 | \\text{sheep, quick}))$$.\r\n\r\n整个计算过程的目标是通过更新嵌套参数 \\\\(\\theta\\\\) 来逼近目标函数（这个这个例子中就是使目标函数最大化）。为此我们要计算损失函数中嵌套参数 \\\\(\\theta\\\\) 的梯度，比如，\r\n\\\\(\\frac{\\partial}{\\partial \\theta} J_\\text{NEG}\\\\) (幸好TensorFlow封装了工具函数可以简单调用!)。对于整个数据集，当梯度下降的过程中不断地更新参数，对应产生的效果就是不断地移动每个单词的嵌套向量，直到可以把真实单词和噪声单词很好得区分开。\r\n\r\n我们可以把学习向量映射到2维中以便我们观察，其中用到的技术可以参考\r\n[t-SNE 降纬技术](http://lvdmaaten.github.io/tsne/)。当我们用可视化的方式来观察这些向量，就可以很明显的获取单词之间语义信息的关系，这实际上是非常有用的。当我们第一次发现这样的诱导向量空间中，展示了一些特定的语义关系，这是非常有趣的，比如文字中 *male-female*，*gender* 甚至还有 *country-capital* 的关系, 如下方的图所示 (也可以参考\r\n[Mikolov et al., 2013](http://www.aclweb.org/anthology/N13-1090)论文中的例子)。\r\n\r\n\r\n\r\n![](\"img/linear-relationships.png\")
\r\n
\r\n\r\n这也解释了为什么这些向量在传统的NLP问题中可作为特性使用，比如用在对一个演讲章节打个标签，或者对一个专有名词的识别\r\n(看看如下这个例子\r\n[Collobert et al.](http://arxiv.org/pdf/1103.0398v1.pdf)或者\r\n[Turian et al.](http://www.aclweb.org/anthology/P10-1040))。\r\n\r\n不过现在让我们用它们来画漂亮的图表吧！\r\n\r\n## 建立图形 \r\n\r\n这里谈得都是嵌套，那么先来定义一个嵌套参数矩阵。我们用均匀分布的随机值来初始化这个大矩阵。\r\n\r\n\r\n```python\r\nembeddings = tf.Variable(\r\n tf.random_uniform([vocabulary_size, embedding_size], -1.0, 1.0))\r\n```\r\n\r\n对噪声-比对的损失计算就使用一个逻辑回归模型。对此，我们需要对语料库中的每个单词定义一个权重值和偏差值。(也可称之为`输出权重` 与之对应的 `输入嵌套值`)。定义如下。\r\n\r\n```python\r\nnce_weights = tf.Variable(\r\n tf.truncated_normal([vocabulary_size, embedding_size],\r\n stddev=1.0 / math.sqrt(embedding_size)))\r\nnce_biases = tf.Variable(tf.zeros([vocabulary_size]))\r\n```\r\n\r\n我们有了这些参数之后，就可以定义Skip-Gram模型了。简单起见，假设我们已经把语料库中的文字整型化了，这样每个整型代表一个单词(细节请查看\r\n[tensorflow/g3doc/tutorials/word2vec/word2vec_basic.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/tutorials/word2vec/word2vec_basic.py))。Skip-Gram模型有两个输入。一个是一组用整型表示的上下文单词，另一个是目标单词。给这些输入建立占位符节点，之后就可以填入数据了。\r\n\r\n```python\r\n# 建立输入占位符\r\ntrain_inputs = tf.placeholder(tf.int32, shape=[batch_size])\r\ntrain_labels = tf.placeholder(tf.int32, shape=[batch_size, 1])\r\n```\r\n\r\n然后我们需要对批数据中的单词建立嵌套向量，TensorFlow提供了方便的工具函数。\r\n\r\n```python\r\nembed = tf.nn.embedding_lookup(embeddings, train_inputs)\r\n```\r\n\r\n好了，现在我们有了每个单词的嵌套向量，接下来就是使用噪声-比对的训练方式来预测目标单词。\r\n\r\n```python\r\n# 计算 NCE 损失函数, 每次使用负标签的样本.\r\nloss = tf.reduce_mean(\r\n tf.nn.nce_loss(nce_weights, nce_biases, embed, train_labels,\r\n num_sampled, vocabulary_size))\r\n```\r\n\r\n我们对损失函数建立了图形节点，然后我们需要计算相应梯度和更新参数的节点，比如说在这里我们会使用随机梯度下降法，TensorFlow也已经封装好了该过程。\r\n\r\n```python\r\n# 使用 SGD 控制器.\r\noptimizer = tf.train.GradientDescentOptimizer(learning_rate=1.0).minimize(loss)\r\n```\r\n\r\n## 训练模型 \r\n\r\n训练的过程很简单，只要在循环中使用`feed_dict`不断给占位符填充数据，同时调用\r\n[`session.run`](../../api_docs/python/client.md#Session.run)即可。\r\n\r\n```python\r\nfor inputs, labels in generate_batch(...):\r\n feed_dict = {training_inputs: inputs, training_labels: labels}\r\n _, cur_loss = session.run([optimizer, loss], feed_dict=feed_dict)\r\n```\r\n\r\n完整地例子可参考\r\n[tensorflow/g3doc/tutorials/word2vec/word2vec_basic.py](./word2vec_basic.py).\r\n\r\n## 嵌套学习结果可视化 \r\n\r\n使用t-SNE来看一下嵌套学习完成的结果。\r\n\r\n\r\n\r\n![](\"img/tsne.png\")
\r\n
\r\n\r\nEt voila! 与预期的一样，相似的单词被聚类在一起。对word2vec模型更复杂的实现需要用到TensorFlow一些更高级的特性，具体是实现可以参考\r\n[tensorflow/models/embedding/word2vec.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/embedding/word2vec.py)。\r\n\r\n## 嵌套学习的评估: 类比推理 \r\n\r\n词嵌套在NLP的预测问题中是非常有用且使用广泛地。如果要检测一个模型是否是可以成熟地区分词性或者区分专有名词的模型，最简单的办法就是直接检验它的预测词性、语义关系的能力，比如让它解决形如`king is to queen as father is to ?`这样的问题。这种方法叫做*类比推理* ，可参考[Mikolov and colleagues](http://msr-waypoint.com/en-us/um/people/gzweig/Pubs/NAACL2013Regularities.pdf)，数据集下载地址为: \r\nhttps://word2vec.googlecode.com/svn/trunk/questions-words.txt。\r\n\r\nTo see how we do this evaluation如何执行这样的评估，可以看`build_eval_graph()`和\r\n`eval()`这两个函数在下面源码中的使用\r\n[tensorflow/models/embedding/word2vec.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/embedding/word2vec.py).\r\n\r\n超参数的选择对该问题解决的准确性有巨大的影响。想要模型具有很好的表现，需要有一个巨大的训练数据集，同时仔细调整参数的选择并且使用例如二次抽样的一些技巧。不过这些问题已经超出了本教程的范围。\r\n\r\n## 优化实现 \r\n\r\n以上简单的例子展示了TensorFlow的灵活性。比如说，我们可以很轻松得用现成的`tf.nn.sampled_softmax_loss()`来代替`tf.nn.nce_loss()`构成目标函数。如果你对损失函数想做新的尝试，你可以用TensorFlow手动编写新的目标函数的表达式，然后用控制器执行计算。这种灵活性的价值体现在，当我们探索一个机器学习模型时，我们可以很快地遍历这些尝试，从中选出最优。\r\n\r\n一旦你有了一个满意的模型结构，或许它就可以使实现运行地更高效（在短时间内覆盖更多的数据）。比如说，在本教程中使用的简单代码，实际运行速度都不错，因为我们使用Python来读取和填装数据，而这些在TensorFlow后台只需执行非常少的工作。如果你发现你的模型在输入数据时存在严重的瓶颈，你可以根据自己的实际问题自行实现一个数据阅读器，参考\r\n[新的数据格式](../../how_tos/new_data_formats/index.md)。对于Skip-Gram\r\n模型，我们已经完成了如下这个例子\r\n[tensorflow/models/embedding/word2vec.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/embedding/word2vec.py)。\r\n\r\n如果I/O问题对你的模型已经不再是个问题，并且想进一步地优化性能，或许你可以自行编写TensorFlow操作单元，详见\r\n[添加一个新的操作](../../how_tos/adding_an_op/index.md)。相应的，我们也提供了Skip-Gram模型的例子\r\n[tensorflow/models/embedding/word2vec_optimized.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/embedding/word2vec_optimized.py)。请自行调节以上几个过程的标准，使模型在每个运行阶段有更好地性能。\r\n\r\n## 总结 \r\n\r\n在本教程中我们介绍了word2vec模型,它在解决词嵌套问题中具有良好的性能。我们解释了使用词嵌套模型的实用性，并且讨论了如何使用TensorFlow实现该模型的高效训练。总的来说，我们希望这个例子能够让向你展示TensorFlow可以提供实验初期的灵活性，以及在后期优化模型时对模型内部的可操控性。\r\n\r\n原文地址：[Vector Representation of Words](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/tutorials/word2vec/index.md) 翻译：[btpeter](https://github.com/btpeter) 校对：waiwaizheng\r\n"
},
{
"path": "SOURCE/tutorials/word2vec/word2vec_basic.py",
"content": "from __future__ import print_function\r\nimport tensorflow.python.platform\r\n\r\nimport collections\r\nimport math\r\nimport numpy as np\r\nimport os\r\nimport random\r\nimport tensorflow as tf\r\nimport urllib\r\nimport zipfile\r\n\r\n# Step 1: Download the data.\r\nurl = 'http://mattmahoney.net/dc/'\r\n\r\ndef maybe_download(filename, expected_bytes):\r\n \"\"\"Download a file if not present, and make sure it's the right size.\"\"\"\r\n if not os.path.exists(filename):\r\n filename, _ = urllib.urlretrieve(url + filename, filename)\r\n statinfo = os.stat(filename)\r\n if statinfo.st_size == expected_bytes:\r\n print('Found and verified', filename)\r\n else:\r\n print(statinfo.st_size)\r\n raise Exception(\r\n 'Failed to verify ' + filename + '. Can you get to it with a browser?')\r\n return filename\r\n\r\nfilename = maybe_download('text8.zip', 31344016)\r\n\r\n\r\n# Read the data into a string.\r\ndef read_data(filename):\r\n f = zipfile.ZipFile(filename)\r\n for name in f.namelist():\r\n return f.read(name).split()\r\n f.close()\r\n\r\nwords = read_data(filename)\r\nprint('Data size', len(words))\r\n\r\n# Step 2: Build the dictionary and replace rare words with UNK token.\r\nvocabulary_size = 50000\r\n\r\ndef build_dataset(words):\r\n count = [['UNK', -1]]\r\n count.extend(collections.Counter(words).most_common(vocabulary_size - 1))\r\n dictionary = dict()\r\n for word, _ in count:\r\n dictionary[word] = len(dictionary)\r\n data = list()\r\n unk_count = 0\r\n for word in words:\r\n if word in dictionary:\r\n index = dictionary[word]\r\n else:\r\n index = 0 # dictionary['UNK']\r\n unk_count = unk_count + 1\r\n data.append(index)\r\n count[0][1] = unk_count\r\n reverse_dictionary = dict(zip(dictionary.values(), dictionary.keys()))\r\n return data, count, dictionary, reverse_dictionary\r\n\r\ndata, count, dictionary, reverse_dictionary = build_dataset(words)\r\ndel words # Hint to reduce memory.\r\nprint('Most common words (+UNK)', count[:5])\r\nprint('Sample data', data[:10])\r\n\r\ndata_index = 0\r\n\r\n\r\n# Step 4: Function to generate a training batch for the skip-gram model.\r\ndef generate_batch(batch_size, num_skips, skip_window):\r\n global data_index\r\n assert batch_size % num_skips == 0\r\n assert num_skips <= 2 * skip_window\r\n batch = np.ndarray(shape=(batch_size), dtype=np.int32)\r\n labels = np.ndarray(shape=(batch_size, 1), dtype=np.int32)\r\n span = 2 * skip_window + 1 # [ skip_window target skip_window ]\r\n buffer = collections.deque(maxlen=span)\r\n for _ in range(span):\r\n buffer.append(data[data_index])\r\n data_index = (data_index + 1) % len(data)\r\n for i in range(batch_size / num_skips):\r\n target = skip_window # target label at the center of the buffer\r\n targets_to_avoid = [ skip_window ]\r\n for j in range(num_skips):\r\n while target in targets_to_avoid:\r\n target = random.randint(0, span - 1)\r\n targets_to_avoid.append(target)\r\n batch[i * num_skips + j] = buffer[skip_window]\r\n labels[i * num_skips + j, 0] = buffer[target]\r\n buffer.append(data[data_index])\r\n data_index = (data_index + 1) % len(data)\r\n return batch, labels\r\n\r\nbatch, labels = generate_batch(batch_size=8, num_skips=2, skip_window=1)\r\nfor i in range(8):\r\n print(batch[i], '->', labels[i, 0])\r\n print(reverse_dictionary[batch[i]], '->', reverse_dictionary[labels[i, 0]])\r\n\r\n# Step 5: Build and train a skip-gram model.\r\n\r\nbatch_size = 128\r\nembedding_size = 128 # Dimension of the embedding vector.\r\nskip_window = 1 # How many words to consider left and right.\r\nnum_skips = 2 # How many times to reuse an input to generate a label.\r\n\r\n# We pick a random validation set to sample nearest neighbors. Here we limit the\r\n# validation samples to the words that have a low numeric ID, which by\r\n# construction are also the most frequent.\r\nvalid_size = 16 # Random set of words to evaluate similarity on.\r\nvalid_window = 100 # Only pick dev samples in the head of the distribution.\r\nvalid_examples = np.array(random.sample(xrange(valid_window), valid_size))\r\nnum_sampled = 64 # Number of negative examples to sample.\r\n\r\ngraph = tf.Graph()\r\n\r\nwith graph.as_default():\r\n\r\n # Input data.\r\n train_inputs = tf.placeholder(tf.int32, shape=[batch_size])\r\n train_labels = tf.placeholder(tf.int32, shape=[batch_size, 1])\r\n valid_dataset = tf.constant(valid_examples, dtype=tf.int32)\r\n\r\n # Construct the variables.\r\n embeddings = tf.Variable(\r\n tf.random_uniform([vocabulary_size, embedding_size], -1.0, 1.0))\r\n nce_weights = tf.Variable(\r\n tf.truncated_normal([vocabulary_size, embedding_size],\r\n stddev=1.0 / math.sqrt(embedding_size)))\r\n nce_biases = tf.Variable(tf.zeros([vocabulary_size]))\r\n\r\n # Look up embeddings for inputs.\r\n embed = tf.nn.embedding_lookup(embeddings, train_inputs)\r\n\r\n # Compute the average NCE loss for the batch.\r\n # tf.nce_loss automatically draws a new sample of the negative labels each\r\n # time we evaluate the loss.\r\n loss = tf.reduce_mean(\r\n tf.nn.nce_loss(nce_weights, nce_biases, train_labels,embed,\r\n num_sampled, vocabulary_size))\r\n\r\n # Construct the SGD optimizer using a learning rate of 1.0.\r\n optimizer = tf.train.GradientDescentOptimizer(1.0).minimize(loss)\r\n\r\n # Compute the cosine similarity between minibatch examples and all embeddings.\r\n norm = tf.sqrt(tf.reduce_sum(tf.square(embeddings), 1, keep_dims=True))\r\n normalized_embeddings = embeddings / norm\r\n valid_embeddings = tf.nn.embedding_lookup(\r\n normalized_embeddings, valid_dataset)\r\n similarity = tf.matmul(\r\n valid_embeddings, normalized_embeddings, transpose_b=True)\r\n\r\n# Step 6: Begin training\r\nnum_steps = 100001\r\n\r\nwith tf.Session(graph=graph) as session:\r\n # We must initialize all variables before we use them.\r\n tf.initialize_all_variables().run()\r\n print(\"Initialized\")\r\n\r\n average_loss = 0\r\n for step in xrange(num_steps):\r\n batch_inputs, batch_labels = generate_batch(\r\n batch_size, num_skips, skip_window)\r\n feed_dict = {train_inputs : batch_inputs, train_labels : batch_labels}\r\n\r\n # We perform one update step by evaluating the optimizer op (including it\r\n # in the list of returned values for session.run()\r\n _, loss_val = session.run([optimizer, loss], feed_dict=feed_dict)\r\n average_loss += loss_val\r\n\r\n if step % 2000 == 0:\r\n if step > 0:\r\n average_loss = average_loss / 2000\r\n # The average loss is an estimate of the loss over the last 2000 batches.\r\n print(\"Average loss at step \", step, \": \", average_loss)\r\n average_loss = 0\r\n\r\n # note that this is expensive (~20% slowdown if computed every 500 steps)\r\n if step % 10000 == 0:\r\n sim = similarity.eval()\r\n for i in xrange(valid_size):\r\n valid_word = reverse_dictionary[valid_examples[i]]\r\n top_k = 8 # number of nearest neighbors\r\n nearest = (-sim[i, :]).argsort()[1:top_k+1]\r\n log_str = \"Nearest to %s:\" % valid_word\r\n for k in xrange(top_k):\r\n close_word = reverse_dictionary[nearest[k]]\r\n log_str = \"%s %s,\" % (log_str, close_word)\r\n print(log_str)\r\n final_embeddings = normalized_embeddings.eval()\r\n\r\n# Step 7: Visualize the embeddings.\r\n\r\ndef plot_with_labels(low_dim_embs, labels, filename='tsne.png'):\r\n assert low_dim_embs.shape[0] >= len(labels), \"More labels than embeddings\"\r\n plt.figure(figsize=(18, 18)) #in inches\r\n for i, label in enumerate(labels):\r\n x, y = low_dim_embs[i,:]\r\n plt.scatter(x, y)\r\n plt.annotate(label,\r\n xy=(x, y),\r\n xytext=(5, 2),\r\n textcoords='offset points',\r\n ha='right',\r\n va='bottom')\r\n\r\n plt.savefig(filename)\r\n\r\ntry:\r\n from sklearn.manifold import TSNE\r\n import matplotlib.pyplot as plt\r\n\r\n tsne = TSNE(perplexity=30, n_components=2, init='pca', n_iter=5000)\r\n plot_only = 500\r\n low_dim_embs = tsne.fit_transform(final_embeddings[:plot_only,:])\r\n labels = list(dictionary.keys())[:plot_only]\r\n plot_with_labels(low_dim_embs, labels)\r\n\r\nexcept ImportError:\r\n print(\"Please install sklearn and matplotlib to visualize embeddings.\")\r\n"
},
{
"path": "SOURCE/tutorials/word2vec.md",
"content": "# Vector Representations of Words \r\n\r\n在本教程我们来看一下[Mikolov et al](http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf)中提到的word2vec模型。该模型是用于学习文字的向量表示，称之为“word embedding”。\r\n\r\n## 亮点 \r\n\r\n本教程意在展现出在TensorfLow中构建word2vec模型有趣、本质的部分。\r\n\r\n* 我们从我们为何需要使用向量表示文字开始。\r\n* 我们通过直观地例子观察模型背后的本质，以及它是如何训练的（通过一些数学方法评估）。\r\n* 同时我们也展示了TensorFlow对该模型的简单实现。\r\n* 最后，我们着眼于让给这个简单版本的模型表现更好。\r\n\r\n我们会在教程的推进中循序渐进地解释代码，但是如果你更希望直入主题，可以在\r\n[tensorflow/g3doc/tutorials/word2vec/word2vec_basic.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/tutorials/word2vec/word2vec_basic.py)查看到一个最简单的实现。这个基本的例子提供的代码可以完成下载一些数据，简单训练后展示结果。一旦你觉得已经完全掌握了这个简单版本，你可以查看\r\n[tensorflow/models/embedding/word2vec.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/embedding/word2vec.py)，这里提供了一些更复杂的实现，同时也展示了TensorFlow的一些更进阶的特性，比如如何更高效地使用线程将数据送入文本模型，再比如如何在训练中设置检查点等等。\r\n\r\n但是首先，让我们来看一下为何需要学习word embeddings。如果你对word embeddings相关内容已经是个专家了，那么请安心跳过本节内容，直接深入细节干一些脏活吧。\r\n\r\n## 动机: 为什么需要学习 Word Embeddings? \r\n\r\n通常图像或音频系统处理的是由图片中所有单个原始像素点强度值或者音频中功率谱密度的强度值，把它们编码成丰富、高纬度的向量数据集。对于物体或语音识别这一类的任务，我们所需的全部信息已经都存储在原始数据中（显然人类本身就是依赖原始数据进行日常的物体或语音识别的）。然后，自然语言处理系统通常将词汇作为离散的单一符号，例如 \"cat\" 一词或可表示为 `Id537` ，而 \"dog\" 一词或可表示为 `Id143`。这些符号编码毫无规律，无法提供不同词汇之间可能存在的关联信息。换句话说，在处理关于 \"dogs\" 一词的信息时，模型将无法利用已知的关于 \"cats\" 的信息（例如，它们都是动物，有四条腿，可作为宠物等等）。可见，将词汇表达为上述的独立离散符号将进一步导致数据稀疏，使我们在训练统计模型时不得不寻求更多的数据。而词汇的向量表示将克服上述的难题。\r\n\r\n\r\n\r\n![](\"../images/audio-image-text.png\")
\r\n
\r\n\r\n[向量空间模型](https://en.wikipedia.org/wiki/Vector_space_model) (VSMs)将词汇表达（嵌套）于一个连续的向量空间中，语义近似的词汇被映射为相邻的数据点。向量空间模型在自然语言处理领域中有着漫长且丰富的历史，不过几乎所有利用这一模型的方法都依赖于\r\n[分布式假设](https://en.wikipedia.org/wiki/Distributional_semantics#Distributional_Hypothesis)，其核心思想为出现于上下文情景中的词汇都有相类似的语义。采用这一假设的研究方法大致分为以下两类：*基于计数的方法* (e.g.\r\n[潜在语义分析](https://en.wikipedia.org/wiki/Latent_semantic_analysis))，\r\n和 *预测方法* (e.g.\r\n[神经概率化语言模型](http://www.scholarpedia.org/article/Neural_net_language_models)).\r\n\r\n其中它们的区别在如下论文中又详细阐述\r\n[Baroni et al.](http://clic.cimec.unitn.it/marco/publications/acl2014/baroni-etal-countpredict-acl2014.pdf)，不过简而言之：基于计数的方法计算某词汇与其邻近词汇在一个大型语料库中共同出现的频率及其他统计量，然后将这些统计量映射到一个小型且稠密的向量中。预测方法则试图直接从某词汇的邻近词汇对其进行预测，在此过程中利用已经学习到的小型且稠密的*嵌套向量*。 \r\n\r\nWord2vec是一种可以进行高效率词嵌套学习的预测模型。其两种变体分别为：连续词袋模型（CBOW）及Skip-Gram模型。从算法角度看，这两种方法非常相似，其区别为CBOW根据源词上下文词汇（'the cat sits on the'）来预测目标词汇（例如，‘mat’），而Skip-Gram模型做法相反，它通过目标词汇来预测源词汇。Skip-Gram模型采取CBOW的逆过程的动机在于：CBOW算法对于很多分布式信息进行了平滑处理（例如将一整段上下文信息视为一个单一观察量）。很多情况下，对于小型的数据集，这一处理是有帮助的。相形之下，Skip-Gram模型将每个“上下文-目标词汇”的组合视为一个新观察量，这种做法在大型数据集中会更为有效。本教程余下部分将着重讲解Skip-Gram模型。\r\n\r\n## 处理噪声对比训练 \r\n\r\n神经概率化语言模型通常使用[极大似然法](https://en.wikipedia.org/wiki/Maximum_likelihood) (ML) 进行训练，其中通过 [*softmax* function](https://en.wikipedia.org/wiki/Softmax_function) 来最大化当提供前一个单词 **h** (代表 \"history\")，后一个单词的概率  (代表 \"target\")，\r\n\r\n\r\n\r\n当 **score(w_t,h)** 计算了文字 **w_t** 和 上下文 **h** 的相容性（通常使用向量积）。我们使用对数似然函数来训练训练集的最大值，比如通过：\r\n\r\n\r\n\r\n这里提出了一个解决语言概率模型的合适的通用方法。然而这个方法实际执行起来开销非常大，因为我们需要去计算并正则化当前上下文环境 **h** 中所有其他 **V** 单词 **w'** 的概率得分，*在每一步训练迭代中*。\r\n\r\n\r\n\r\n![](\"../images/softmax-nplm.png\")
\r\n
\r\n\r\n从另一个角度来说，当使用word2vec模型时，我们并不需要对概率模型中的所有特征进行学习。而CBOW模型和Skip-Gram模型为了避免这种情况发生，使用一个二分类器（逻辑回归）在同一个上下文环境里从 **k** 虚构的 (噪声) 单词  区分出真正的目标单词 。我们下面详细阐述一下CBOW模型，对于Skip-Gram模型只要简单地做相反的操作即可。\r\n\r\n\r\n\r\n![](\"../images/nce-nplm.png\")
\r\n
\r\n\r\n从数学角度来说，我们的目标是对每个样本最大化：\r\n\r\n\r\n\r\n其中  代表的是数据集在当前上下文 **h** ，根据所学习的嵌套向量  ，目标单词 **w** 使用二分类逻辑回归计算得出的概率。在实践中，我们通过在噪声分布中绘制比对文字来获得近似的期望值（通过计算[蒙特卡洛平均值](https://en.wikipedia.org/wiki/Monte_Carlo_integration)）。\r\n\r\n当真实地目标单词被分配到较高的概率，同时噪声单词的概率很低时，目标函数也就达到最大值了。从技术层面来说，这种方法叫做\r\n[负抽样](http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf)，而且使用这个损失函数在数学层面上也有很好的解释：这个更新过程也近似于softmax函数的更新。这在计算上将会有很大的优势，因为当计算这个损失函数时，只是有我们挑选出来的 **k** 个 *噪声单词*，而没有使用整个语料库 **V**。这使得训练变得非常快。我们实际上使用了与[noise-contrastive estimation (NCE)](http://papers.nips.cc/paper/5165-learning-word-embeddings-efficiently-with-noise-contrastive-estimation.pdf)介绍的非常相似的方法，这在TensorFlow中已经封装了一个很便捷的函数`tf.nn.nce_loss()`。\r\n\r\n让我们在实践中来直观地体会它是如何运作的！\r\n\r\n## Skip-gram 模型\r\n\r\n下面来看一下这个数据集\r\n\r\n`the quick brown fox jumped over the lazy dog`\r\n\r\n我们首先对一些单词以及它们的上下文环境建立一个数据集。我们可以以任何合理的方式定义‘上下文’，而通常上这个方式是根据文字的句法语境的（使用语法原理的方式处理当前目标单词可以看一下这篇文献 [Levy et al.](https://levyomer.files.wordpress.com/2014/04/dependency-based-word-embeddings-acl-2014.pdf)，比如说把目标单词左边的内容当做一个‘上下文’，或者以目标单词右边的内容，等等。现在我们把目标单词的左右单词视作一个上下文， 使用大小为1的窗口，这样就得到这样一个由`(上下文, 目标单词)` 组成的数据集：\r\n\r\n`([the, brown], quick), ([quick, fox], brown), ([brown, jumped], fox), ...`\r\n\r\n前文提到Skip-Gram模型是把目标单词和上下文颠倒过来，所以在这个问题中，举个例子，就是用'quick'来预测 'the' 和 'brown' ，用 'brown' 预测 'quick' 和 'brown' 。因此这个数据集就变成由`(输入, 输出)`组成的：\r\n\r\n`(quick, the), (quick, brown), (brown, quick), (brown, fox), ...`\r\n\r\n目标函数通常是对整个数据集建立的，但是本问题中要对每一个样本（或者是一个`batch_size` 很小的样本集，通常设置为`16 <= batch_size <= 512`）在同一时间执行特别的操作，称之为[随机梯度下降](https://en.wikipedia.org/wiki/Stochastic_gradient_descent)\r\n(SGD)。我们来看一下训练过程中每一步的执行。\r\n\r\n假设用 **t** 表示上面这个例子中`quick` 来预测 `the` 的训练的单个循环。用 `num_noise` 定义从噪声分布中挑选出来的噪声（相反的）单词的个数，通常使用一元分布，**P(w)**。为了简单起见，我们就定`num_noise=1`，用 `sheep` 选作噪声词。接下来就可以计算每一对观察值和噪声值的损失函数了，每一个执行步骤就可表示为：\r\n\r\n\r\n\r\n整个计算过程的目标是通过更新嵌套参数  来逼近目标函数（这个这个例子中就是使目标函数最大化）。为此我们要计算损失函数中嵌套参数  的梯度，比如，\r\n (幸好TensorFlow封装了工具函数可以简单调用!)。对于整个数据集，当梯度下降的过程中不断地更新参数，对应产生的效果就是不断地移动每个单词的嵌套向量，直到可以把真实单词和噪声单词很好得区分开。\r\n\r\n我们可以把学习向量映射到2维中以便我们观察，其中用到的技术可以参考\r\n[t-SNE 降纬技术](http://lvdmaaten.github.io/tsne/)。当我们用可视化的方式来观察这些向量，就可以很明显的获取单词之间语义信息的关系，这实际上是非常有用的。当我们第一次发现这样的诱导向量空间中，展示了一些特定的语义关系，这是非常有趣的，比如文字中 *male-female*，*gender* 甚至还有 *country-capital* 的关系, 如下方的图所示 (也可以参考\r\n[Mikolov et al., 2013](http://www.aclweb.org/anthology/N13-1090)论文中的例子)。\r\n\r\n\r\n\r\n![](\"../images/linear-relationships.png\")
\r\n
\r\n\r\n这也解释了为什么这些向量在传统的NLP问题中可作为特性使用，比如用在对一个演讲章节打个标签，或者对一个专有名词的识别\r\n(看看如下这个例子\r\n[Collobert et al.](http://arxiv.org/pdf/1103.0398v1.pdf)或者\r\n[Turian et al.](http://www.aclweb.org/anthology/P10-1040))。\r\n\r\n不过现在让我们用它们来画漂亮的图表吧！\r\n\r\n## 建立图形 \r\n\r\n这里谈得都是嵌套，那么先来定义一个嵌套参数矩阵。我们用唯一的随机值来初始化这个大矩阵。\r\n\r\n\r\n```python\r\nembeddings = tf.Variable(\r\n tf.random_uniform([vocabulary_size, embedding_size], -1.0, 1.0))\r\n```\r\n\r\n对噪声-比对的损失计算就使用一个逻辑回归模型。对此，我们需要对语料库中的每个单词定义一个权重值和偏差值。(也可称之为`输出权重` 与之对应的 `输入嵌套值`)。定义如下。\r\n\r\n```python\r\nnce_weights = tf.Variable(\r\n tf.truncated_normal([vocabulary_size, embedding_size],\r\n stddev=1.0 / math.sqrt(embedding_size)))\r\nnce_biases = tf.Variable(tf.zeros([vocabulary_size]))\r\n```\r\n\r\n我们有了这些参数之后，就可以定义Skip-Gram模型了。简单起见，假设我们已经把语料库中的文字整型化了，这样每个整型代表一个单词(细节请查看\r\n[tensorflow/g3doc/tutorials/word2vec/word2vec_basic.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/g3doc/tutorials/word2vec/word2vec_basic.py))。Skip-Gram模型有两个输入。一个是一组用整型表示的上下文单词，另一个是目标单词。给这些输入建立占位符节点，之后就可以填入数据了。\r\n\r\n```python\r\n# 建立输入占位符\r\ntrain_inputs = tf.placeholder(tf.int32, shape=[batch_size])\r\ntrain_labels = tf.placeholder(tf.int32, shape=[batch_size, 1])\r\n```\r\n\r\n然后我们需要对批数据中的单词建立嵌套向量，TensorFlow提供了方便的工具函数。\r\n\r\n```python\r\nembed = tf.nn.embedding_lookup(embeddings, train_inputs)\r\n```\r\n\r\n好了，现在我们有了每个单词的嵌套向量，接下来就是使用噪声-比对的训练方式来预测目标单词。\r\n\r\n```python\r\n# 计算 NCE 损失函数, 每次使用负标签的样本.\r\nloss = tf.reduce_mean(\r\n tf.nn.nce_loss(nce_weights, nce_biases, embed, train_labels,\r\n num_sampled, vocabulary_size))\r\n```\r\n\r\n我们对损失函数建立了图形节点，然后我们需要计算相应梯度和更新参数的节点，比如说在这里我们会使用随机梯度下降法，TensorFlow也已经封装好了该过程。\r\n\r\n```python\r\n# 使用 SGD 控制器.\r\noptimizer = tf.train.GradientDescentOptimizer(learning_rate=1.0).minimize(loss)\r\n```\r\n\r\n## 训练模型 \r\n\r\n训练的过程很简单，只要在循环中使用`feed_dict`不断给占位符填充数据，同时调用\r\n[`session.run`](tensorflow-zh/SOURCE/api_docs/python/client.md#Session.run)即可。\r\n\r\n```python\r\nfor inputs, labels in generate_batch(...):\r\n feed_dict = {training_inputs: inputs, training_labels: labels}\r\n _, cur_loss = session.run([optimizer, loss], feed_dict=feed_dict)\r\n```\r\n\r\n完整地例子可参考\r\n[tensorflow/g3doc/tutorials/word2vec/word2vec_basic.py](./word2vec_basic.py).\r\n\r\n## 嵌套学习结果可视化 \r\n\r\n使用t-SNE来看一下嵌套学习完成的结果。\r\n\r\n\r\n\r\n![](\"../images/tsne.png\")
\r\n
\r\n\r\nEt voila! 与预期的一样，相似的单词被聚类在一起。对word2vec模型更复杂的实现需要用到TensorFlow一些更高级的特性，具体是实现可以参考\r\n[tensorflow/models/embedding/word2vec.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/embedding/word2vec.py)。\r\n\r\n## 嵌套学习的评估: 类比推理 \r\n\r\n词嵌套在NLP的预测问题中是非常有用且使用广泛地。如果要检测一个模型是否是可以成熟地区分词性或者区分专有名词的模型，最简单的办法就是直接检验它的预测词性、语义关系的能力，比如让它解决形如`king is to queen as father is to ?`这样的问题。这种方法叫做*类比推理* ，可参考[Mikolov and colleagues](http://msr-waypoint.com/en-us/um/people/gzweig/Pubs/NAACL2013Regularities.pdf)，数据集下载地址为: \r\nhttps://word2vec.googlecode.com/svn/trunk/questions-words.txt。\r\n\r\nTo see how we do this evaluation如何执行这样的评估，可以看`build_eval_graph()`和\r\n`eval()`这两个函数在下面源码中的使用\r\n[tensorflow/models/embedding/word2vec.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/embedding/word2vec.py).\r\n\r\n超参数的选择对该问题解决的准确性有巨大的影响。想要模型具有很好的表现，需要有一个巨大的训练数据集，同时仔细调整参数的选择并且使用例如二次抽样的一些技巧。不过这些问题已经超出了本教程的范围。\r\n\r\n## 优化实现 \r\n\r\n以上简单的例子展示了TensorFlow的灵活性。比如说，我们可以很轻松得用现成的`tf.nn.sampled_softmax_loss()`来代替`tf.nn.nce_loss()`构成目标函数。如果你对损失函数想做新的尝试，你可以用TensorFlow手动编写新的目标函数的表达式，然后用控制器执行计算。这种灵活性的价值体现在，当我们探索一个机器学习模型时，我们可以很快地遍历这些尝试，从中选出最优。\r\n\r\n一旦你有了一个满意的模型结构，或许它就可以使实现运行地更高效（在短时间内覆盖更多的数据）。比如说，在本教程中使用的简单代码，实际运行速度都不错，因为我们使用Python来读取和填装数据，而这些在TensorFlow后台只需执行非常少的工作。如果你发现你的模型在输入数据时存在严重的瓶颈，你可以根据自己的实际问题自行实现一个数据阅读器，参考\r\n[新的数据格式](tensorflow-zh/SOURCE/how_tos/new_data_formats/index.md)。对于Skip-Gram\r\n模型，我们已经完成了如下这个例子\r\n[tensorflow/models/embedding/word2vec.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/embedding/word2vec.py)。\r\n\r\n如果I/O问题对你的模型已经不再是个问题，并且想进一步地优化性能，或许你可以自行编写TensorFlow操作单元，详见\r\n[添加一个新的操作](tensorflow-zh/SOURCE/how_tos/adding_an_op/index.md)。相应的，我们也提供了Skip-Gram模型的例子\r\n[tensorflow/models/embedding/word2vec_optimized.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/embedding/word2vec_optimized.py)。请自行调节以上几个过程的标准，使模型在每个运行阶段有更好地性能。\r\n\r\n## 总结 \r\n\r\n在本教程中我们介绍了word2vec模型,它在解决词嵌套问题中具有良好的性能。我们解释了使用词嵌套模型的实用性，并且讨论了如何使用TensorFlow实现该模型的高效训练。总的来说，我们希望这个例子能够让向你展示TensorFlow可以提供实验初期的灵活性，以及在后期优化模型时对模型内部的可操控性。\r\n\r\n原文地址：[Vector Representation of Words](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/tutorials/word2vec/index.md) 翻译：[btpeter](https://github.com/btpeter) 校对：waiwaizheng\r\n"
},
{
"path": "TOC.md",
"content": "- 起步\r\n - [介绍](SOURCE/get_started/introduction.md)\r\n - [下载及安装](SOURCE/get_started/os_setup.md)\r\n - [基本用法](SOURCE/get_started/basic_usage.md)\r\n- 教程\r\n - [总览](SOURCE/tutorials/overview.md)\r\n - [MNIST 机器学习入门](SOURCE/tutorials/mnist_beginners.md)\r\n - [深入 MNIST](SOURCE/tutorials/mnist_pros.md)\r\n - [TensorFlow 运作方式入门](SOURCE/tutorials/mnist_tf.md)\r\n - [卷积神经网络](SOURCE/tutorials/deep_cnn.md)\r\n - [字词的向量表示](SOURCE/tutorials/word2vec.md)\r\n - [递归神经网络](SOURCE/tutorials/recurrent.md)\r\n - [曼德布洛特(Mandelbrot)集合](SOURCE/tutorials/mandelbrot.md)\r\n - [偏微分方程](SOURCE/tutorials/pdes.md) \r\n - [MNIST数据下载](SOURCE/tutorials/mnist_download.md)\r\n- 运作方式\r\n - [总览](SOURCE/how_tos/overview.md) \r\n - [变量:创建、初始化、保存和加载](SOURCE/how_tos/variables.md) \r\n - [TensorBoard:可视化学习](SOURCE/how_tos/summaries_and_tensorboard.md) \r\n - [TensorBoard:图表可视化](SOURCE/how_tos/graph_viz.md) \r\n - [读取数据](SOURCE/how_tos/reading_data.md) \r\n - [线程和队列](SOURCE/how_tos/threading_and_queues.md) \r\n - [添加新的Op](SOURCE/how_tos/adding_an_op.md) \r\n - [自定义数据读取](SOURCE/how_tos/new_data_formats.md) \r\n - [使用gpu](SOURCE/how_tos/using_gpu.md) \r\n - [共享变量](SOURCE/how_tos/variable_scope.md) \r\n- 资源\r\n - [总览](SOURCE/resources/overview.md) \r\n - [BibTex 引用](SOURCE/resources/bib.md) \r\n - [示例使用](SOURCE/resources/uses.md)\r\n - [FAQ](SOURCE/resources/faq.md)\r\n - [术语表](SOURCE/resources/glossary.md)\r\n - [Tensor排名、形状和类型](SOURCE/resources/dims_types.md)\r\n- 其他\r\n - [常见问题汇总](SOURCE/faq.md)\r\n - [相关资源](SOURCE/resource.md)\r\n - [个人学习心得](SOURCE/personal.md)\r\n\r\n \r\n \n"
},
{
"path": "config.json",
"content": "{\r\n \"name\": \"TensorFlow 官方文档中文版\",\r\n \"introduction\": \"Google 人工智能系统 TensorFlow 官方文档中文版协同翻译。\",\r\n \"path\": {\r\n \"content\": \"SOURCE\",\r\n \"images\": \"SOURCE/images\"\r\n }\r\n\r\n}\r\n"
},
{
"path": "learn-github.md",
"content": "\r\n## 注册 GitHub 账号\r\n\r\n\r\n\r\n```\r\n// 代码区域的上下分别用三个 ` 括起来\r\npublic class Person {\r\n // 代码缩进请使用 四个空格，不要使用 Tab\r\n}\r\n```\r\n\r\n\r\n效果： \r\n\r\n```\r\n// 代码区域的上下分别用三个 ` 括起来\r\npublic class Person {\r\n // 代码缩进请使用 四个空格，不要使用 Tab\r\n}\r\n```\r\n\r\n## 锚点使用\r\n\r\n书写示例：\r\n\r\n```\r\n极客学院是中国最大的 IT 职业在线教育平台。[[锚点名]](#footnode)\r\n\r\n[锚点名]我的示例。\r\n\r\n```\r\n\r\n效果：\r\n\r\n极客学院是中国最大的 IT 职业在线教育平台。[[锚点名]](#footnode)\r\n\r\n[锚点名]我的示例。\r\n\r\n## 特别注意\r\n\r\n1. 英文字母与中文之间需要两边空格（英文与符号之间不空格）\r\n2. 高亮英文之间需要两边空格\r\n3. 代码符号\\`\\`\\`上面需要空一行，结束```下面需要空一行\r\n4. 中文之间和短英文之间使用中文标点\r\n5. 一篇文章中只能出现一个＃号标题\r\n6. 标题＃号之后需要空一格\r\n\r\n\r\n## 表格\r\n\r\n书写示例：\r\n\r\n```\r\n| Prefix | Framework |\r\n| ------------- |:-------------:| -----:|\r\n| NS | Foundation (OS X and iOS) and Application Kit (OS X) |\r\n| UI | UIKit (iOS) |\r\n| AB | Address Book |\r\n| CA | Core Animation |\r\n| CI | Core Image |\r\n```\r\n\r\n效果： \r\n\r\n| Prefix | Framework |\r\n| ------------- |:-------------:| -----:|\r\n| NS | Foundation (OS X and iOS) and Application Kit (OS X) |\r\n| UI | UIKit (iOS) |\r\n| AB | Address Book |\r\n| CA | Core Animation |\r\n| CI | Core Image |\r\n"
},
{
"path": "tex_pdf/api/c4s00.tex",
"content": "\r\n\r\n\\section{Overview}\\label{api_overview}\r\n\r\nTensorFlow has APIs available in several languages both for constructing\r\nand executing a TensorFlow graph. The Python API is at present the most\r\ncomplete and the easiest to use, but the C++ API may offer some\r\nperformance advantages in graph execution, and supports deployment to\r\nsmall devices such as Android.\r\n\r\nOver time, we hope that the TensorFlow community will develop front ends\r\nfor languages like Go, Java, JavaScript, Lua R, and perhaps others. With\r\n\\href{http://swig.org}{SWIG}, it's relatively easy to develop a\r\nTensorFlow interface for your favorite language.\r\n\r\nNote: Many practical aspects of usage are covered in the Mechanics tab,\r\nand some additional documentation not specific to any particular\r\nlanguage API is available in the Resources tab."
},
{
"path": "tex_pdf/api/cc/ClassEnv.md",
"content": "# Class `tensorflow::Env` \r\n\r\nAn interface used by the tensorflow implementation to access operating system functionality like the filesystem etc.\r\n\r\nCallers may wish to provide a custom Env object to get fine grain control.\r\n\r\nAll Env implementations are safe for concurrent access from multiple threads without any external synchronization.\r\n\r\n##Member Summary \r\n\r\n* [`tensorflow::Env::Env()`](#tensorflow_Env_Env)\r\n* [`virtual tensorflow::Env::~Env()`](#virtual_tensorflow_Env_Env)\r\n* [`virtual Status tensorflow::Env::NewRandomAccessFile(const string &fname, RandomAccessFile **result)=0`](#virtual_Status_tensorflow_Env_NewRandomAccessFile)\r\n * Creates a brand new random access read-only file with the specified name.\r\n* [`virtual Status tensorflow::Env::NewWritableFile(const string &fname, WritableFile **result)=0`](#virtual_Status_tensorflow_Env_NewWritableFile)\r\n * Creates an object that writes to a new file with the specified name.\r\n* [`virtual Status tensorflow::Env::NewAppendableFile(const string &fname, WritableFile **result)=0`](#virtual_Status_tensorflow_Env_NewAppendableFile)\r\n * Creates an object that either appends to an existing file, or writes to a new file (if the file does not exist to begin with).\r\n* [`virtual bool tensorflow::Env::FileExists(const string &fname)=0`](#virtual_bool_tensorflow_Env_FileExists)\r\n * Returns true iff the named file exists.\r\n* [`virtual Status tensorflow::Env::GetChildren(const string &dir, std::vector< string > *result)=0`](#virtual_Status_tensorflow_Env_GetChildren)\r\n * Stores in *result the names of the children of the specified directory. The names are relative to \"dir\".\r\n* [`virtual Status tensorflow::Env::DeleteFile(const string &fname)=0`](#virtual_Status_tensorflow_Env_DeleteFile)\r\n * Deletes the named file.\r\n* [`virtual Status tensorflow::Env::CreateDir(const string &dirname)=0`](#virtual_Status_tensorflow_Env_CreateDir)\r\n * Creates the specified directory.\r\n* [`virtual Status tensorflow::Env::DeleteDir(const string &dirname)=0`](#virtual_Status_tensorflow_Env_DeleteDir)\r\n * Deletes the specified directory.\r\n* [`virtual Status tensorflow::Env::GetFileSize(const string &fname, uint64 *file_size)=0`](#virtual_Status_tensorflow_Env_GetFileSize)\r\n * Stores the size of fname in *file_size.\r\n* [`virtual Status tensorflow::Env::RenameFile(const string &src, const string &target)=0`](#virtual_Status_tensorflow_Env_RenameFile)\r\n * Renames file src to target. If target already exists, it will be replaced.\r\n* [`virtual uint64 tensorflow::Env::NowMicros()=0`](#virtual_uint64_tensorflow_Env_NowMicros)\r\n * Returns the number of micro-seconds since some fixed point in time. Only useful for computing deltas of time.\r\n* [`virtual void tensorflow::Env::SleepForMicroseconds(int micros)=0`](#virtual_void_tensorflow_Env_SleepForMicroseconds)\r\n * Sleeps/delays the thread for the prescribed number of micro-seconds.\r\n* [`virtual Thread* tensorflow::Env::StartThread(const ThreadOptions &thread_options, const string &name, std::function< void()> fn) TF_MUST_USE_RESULT=0`](#virtual_Thread_tensorflow_Env_StartThread)\r\n * Returns a new thread that is running fn() and is identified (for debugging/performance-analysis) by \"name\".\r\n* [`static Env* tensorflow::Env::Default()`](#static_Env_tensorflow_Env_Default)\r\n * Returns a default environment suitable for the current operating system.\r\n\r\n##Member Details \r\n\r\n#### `tensorflow::Env::Env()` \r\n\r\n\r\n\r\n\r\n\r\n#### `virtual tensorflow::Env::~Env()` \r\n\r\n\r\n\r\n\r\n\r\n#### `virtual Status tensorflow::Env::NewRandomAccessFile(const string &fname, RandomAccessFile **result)=0` \r\n\r\nCreates a brand new random access read-only file with the specified name.\r\n\r\nOn success, stores a pointer to the new file in *result and returns OK. On failure stores NULL in *result and returns non-OK. If the file does not exist, returns a non-OK status.\r\n\r\nThe returned file may be concurrently accessed by multiple threads.\r\n\r\n#### `virtual Status tensorflow::Env::NewWritableFile(const string &fname, WritableFile **result)=0` \r\n\r\nCreates an object that writes to a new file with the specified name.\r\n\r\nDeletes any existing file with the same name and creates a new file. On success, stores a pointer to the new file in *result and returns OK. On failure stores NULL in *result and returns non-OK.\r\n\r\nThe returned file will only be accessed by one thread at a time.\r\n\r\n#### `virtual Status tensorflow::Env::NewAppendableFile(const string &fname, WritableFile **result)=0` \r\n\r\nCreates an object that either appends to an existing file, or writes to a new file (if the file does not exist to begin with).\r\n\r\nOn success, stores a pointer to the new file in *result and returns OK. On failure stores NULL in *result and returns non-OK.\r\n\r\nThe returned file will only be accessed by one thread at a time.\r\n\r\n#### `virtual bool tensorflow::Env::FileExists(const string &fname)=0` \r\n\r\nReturns true iff the named file exists.\r\n\r\n\r\n\r\n#### `virtual Status tensorflow::Env::GetChildren(const string &dir, std::vector< string > *result)=0` \r\n\r\nStores in *result the names of the children of the specified directory. The names are relative to \"dir\".\r\n\r\nOriginal contents of *results are dropped.\r\n\r\n#### `virtual Status tensorflow::Env::DeleteFile(const string &fname)=0` \r\n\r\nDeletes the named file.\r\n\r\n\r\n\r\n#### `virtual Status tensorflow::Env::CreateDir(const string &dirname)=0` \r\n\r\nCreates the specified directory.\r\n\r\n\r\n\r\n#### `virtual Status tensorflow::Env::DeleteDir(const string &dirname)=0` \r\n\r\nDeletes the specified directory.\r\n\r\n\r\n\r\n#### `virtual Status tensorflow::Env::GetFileSize(const string &fname, uint64 *file_size)=0` \r\n\r\nStores the size of fname in *file_size.\r\n\r\n\r\n\r\n#### `virtual Status tensorflow::Env::RenameFile(const string &src, const string &target)=0` \r\n\r\nRenames file src to target. If target already exists, it will be replaced.\r\n\r\n\r\n\r\n#### `virtual uint64 tensorflow::Env::NowMicros()=0` \r\n\r\nReturns the number of micro-seconds since some fixed point in time. Only useful for computing deltas of time.\r\n\r\n\r\n\r\n#### `virtual void tensorflow::Env::SleepForMicroseconds(int micros)=0` \r\n\r\nSleeps/delays the thread for the prescribed number of micro-seconds.\r\n\r\n\r\n\r\n#### `virtual Thread* tensorflow::Env::StartThread(const ThreadOptions &thread_options, const string &name, std::function< void()> fn) TF_MUST_USE_RESULT=0` \r\n\r\nReturns a new thread that is running fn() and is identified (for debugging/performance-analysis) by \"name\".\r\n\r\nCaller takes ownership of the result and must delete it eventually (the deletion will block until fn() stops running).\r\n\r\n#### `static Env* tensorflow::Env::Default()` \r\n\r\nReturns a default environment suitable for the current operating system.\r\n\r\nSophisticated users may wish to provide their own Env implementation instead of relying on this default environment.\r\n\r\nThe result of Default() belongs to this library and must never be deleted.\r\n"
},
{
"path": "tex_pdf/api/cc/ClassEnvWrapper.md",
"content": "# Class `tensorflow::EnvWrapper` \r\n\r\nAn implementation of Env that forwards all calls to another Env .\r\n\r\nMay be useful to clients who wish to override just part of the functionality of another Env .\r\n\r\n##Member Summary \r\n\r\n* [`tensorflow::EnvWrapper::EnvWrapper(Env *t)`](#tensorflow_EnvWrapper_EnvWrapper)\r\n * Initializes an EnvWrapper that delegates all calls to *t.\r\n* [`virtual tensorflow::EnvWrapper::~EnvWrapper()`](#virtual_tensorflow_EnvWrapper_EnvWrapper)\r\n* [`Env* tensorflow::EnvWrapper::target() const`](#Env_tensorflow_EnvWrapper_target)\r\n * Returns the target to which this Env forwards all calls.\r\n* [`Status tensorflow::EnvWrapper::NewRandomAccessFile(const string &f, RandomAccessFile **r) override`](#Status_tensorflow_EnvWrapper_NewRandomAccessFile)\r\n * Creates a brand new random access read-only file with the specified name.\r\n* [`Status tensorflow::EnvWrapper::NewWritableFile(const string &f, WritableFile **r) override`](#Status_tensorflow_EnvWrapper_NewWritableFile)\r\n * Creates an object that writes to a new file with the specified name.\r\n* [`Status tensorflow::EnvWrapper::NewAppendableFile(const string &f, WritableFile **r) override`](#Status_tensorflow_EnvWrapper_NewAppendableFile)\r\n * Creates an object that either appends to an existing file, or writes to a new file (if the file does not exist to begin with).\r\n* [`bool tensorflow::EnvWrapper::FileExists(const string &f) override`](#bool_tensorflow_EnvWrapper_FileExists)\r\n * Returns true iff the named file exists.\r\n* [`Status tensorflow::EnvWrapper::GetChildren(const string &dir, std::vector< string > *r) override`](#Status_tensorflow_EnvWrapper_GetChildren)\r\n * Stores in *result the names of the children of the specified directory. The names are relative to \"dir\".\r\n* [`Status tensorflow::EnvWrapper::DeleteFile(const string &f) override`](#Status_tensorflow_EnvWrapper_DeleteFile)\r\n * Deletes the named file.\r\n* [`Status tensorflow::EnvWrapper::CreateDir(const string &d) override`](#Status_tensorflow_EnvWrapper_CreateDir)\r\n * Creates the specified directory.\r\n* [`Status tensorflow::EnvWrapper::DeleteDir(const string &d) override`](#Status_tensorflow_EnvWrapper_DeleteDir)\r\n * Deletes the specified directory.\r\n* [`Status tensorflow::EnvWrapper::GetFileSize(const string &f, uint64 *s) override`](#Status_tensorflow_EnvWrapper_GetFileSize)\r\n * Stores the size of fname in *file_size.\r\n* [`Status tensorflow::EnvWrapper::RenameFile(const string &s, const string &t) override`](#Status_tensorflow_EnvWrapper_RenameFile)\r\n * Renames file src to target. If target already exists, it will be replaced.\r\n* [`uint64 tensorflow::EnvWrapper::NowMicros() override`](#uint64_tensorflow_EnvWrapper_NowMicros)\r\n * Returns the number of micro-seconds since some fixed point in time. Only useful for computing deltas of time.\r\n* [`void tensorflow::EnvWrapper::SleepForMicroseconds(int micros) override`](#void_tensorflow_EnvWrapper_SleepForMicroseconds)\r\n * Sleeps/delays the thread for the prescribed number of micro-seconds.\r\n* [`Thread* tensorflow::EnvWrapper::StartThread(const ThreadOptions &thread_options, const string &name, std::function< void()> fn) override`](#Thread_tensorflow_EnvWrapper_StartThread)\r\n * Returns a new thread that is running fn() and is identified (for debugging/performance-analysis) by \"name\".\r\n\r\n##Member Details \r\n\r\n#### `tensorflow::EnvWrapper::EnvWrapper(Env *t)` \r\n\r\nInitializes an EnvWrapper that delegates all calls to *t.\r\n\r\n\r\n\r\n#### `virtual tensorflow::EnvWrapper::~EnvWrapper()` \r\n\r\n\r\n\r\n\r\n\r\n#### `Env* tensorflow::EnvWrapper::target() const` \r\n\r\nReturns the target to which this Env forwards all calls.\r\n\r\n\r\n\r\n#### `Status tensorflow::EnvWrapper::NewRandomAccessFile(const string &f, RandomAccessFile **r) override` \r\n\r\nCreates a brand new random access read-only file with the specified name.\r\n\r\nOn success, stores a pointer to the new file in *result and returns OK. On failure stores NULL in *result and returns non-OK. If the file does not exist, returns a non-OK status.\r\n\r\nThe returned file may be concurrently accessed by multiple threads.\r\n\r\n#### `Status tensorflow::EnvWrapper::NewWritableFile(const string &f, WritableFile **r) override` \r\n\r\nCreates an object that writes to a new file with the specified name.\r\n\r\nDeletes any existing file with the same name and creates a new file. On success, stores a pointer to the new file in *result and returns OK. On failure stores NULL in *result and returns non-OK.\r\n\r\nThe returned file will only be accessed by one thread at a time.\r\n\r\n#### `Status tensorflow::EnvWrapper::NewAppendableFile(const string &f, WritableFile **r) override` \r\n\r\nCreates an object that either appends to an existing file, or writes to a new file (if the file does not exist to begin with).\r\n\r\nOn success, stores a pointer to the new file in *result and returns OK. On failure stores NULL in *result and returns non-OK.\r\n\r\nThe returned file will only be accessed by one thread at a time.\r\n\r\n#### `bool tensorflow::EnvWrapper::FileExists(const string &f) override` \r\n\r\nReturns true iff the named file exists.\r\n\r\n\r\n\r\n#### `Status tensorflow::EnvWrapper::GetChildren(const string &dir, std::vector< string > *r) override` \r\n\r\nStores in *result the names of the children of the specified directory. The names are relative to \"dir\".\r\n\r\nOriginal contents of *results are dropped.\r\n\r\n#### `Status tensorflow::EnvWrapper::DeleteFile(const string &f) override` \r\n\r\nDeletes the named file.\r\n\r\n\r\n\r\n#### `Status tensorflow::EnvWrapper::CreateDir(const string &d) override` \r\n\r\nCreates the specified directory.\r\n\r\n\r\n\r\n#### `Status tensorflow::EnvWrapper::DeleteDir(const string &d) override` \r\n\r\nDeletes the specified directory.\r\n\r\n\r\n\r\n#### `Status tensorflow::EnvWrapper::GetFileSize(const string &f, uint64 *s) override` \r\n\r\nStores the size of fname in *file_size.\r\n\r\n\r\n\r\n#### `Status tensorflow::EnvWrapper::RenameFile(const string &s, const string &t) override` \r\n\r\nRenames file src to target. If target already exists, it will be replaced.\r\n\r\n\r\n\r\n#### `uint64 tensorflow::EnvWrapper::NowMicros() override` \r\n\r\nReturns the number of micro-seconds since some fixed point in time. Only useful for computing deltas of time.\r\n\r\n\r\n\r\n#### `void tensorflow::EnvWrapper::SleepForMicroseconds(int micros) override` \r\n\r\nSleeps/delays the thread for the prescribed number of micro-seconds.\r\n\r\n\r\n\r\n#### `Thread* tensorflow::EnvWrapper::StartThread(const ThreadOptions &thread_options, const string &name, std::function< void()> fn) override` \r\n\r\nReturns a new thread that is running fn() and is identified (for debugging/performance-analysis) by \"name\".\r\n\r\nCaller takes ownership of the result and must delete it eventually (the deletion will block until fn() stops running).\r\n"
},
{
"path": "tex_pdf/api/cc/ClassRandomAccessFile.md",
"content": "# Class `tensorflow::RandomAccessFile` \r\n\r\nA file abstraction for randomly reading the contents of a file.\r\n\r\n\r\n\r\n##Member Summary \r\n\r\n* [`tensorflow::RandomAccessFile::RandomAccessFile()`](#tensorflow_RandomAccessFile_RandomAccessFile)\r\n* [`virtual tensorflow::RandomAccessFile::~RandomAccessFile()`](#virtual_tensorflow_RandomAccessFile_RandomAccessFile)\r\n* [`virtual Status tensorflow::RandomAccessFile::Read(uint64 offset, size_t n, StringPiece *result, char *scratch) const =0`](#virtual_Status_tensorflow_RandomAccessFile_Read)\r\n * Reads up to \"n\" bytes from the file starting at \"offset\".\r\n\r\n##Member Details \r\n\r\n#### `tensorflow::RandomAccessFile::RandomAccessFile()` \r\n\r\n\r\n\r\n\r\n\r\n#### `virtual tensorflow::RandomAccessFile::~RandomAccessFile()` \r\n\r\n\r\n\r\n\r\n\r\n#### `virtual Status tensorflow::RandomAccessFile::Read(uint64 offset, size_t n, StringPiece *result, char *scratch) const =0` \r\n\r\nReads up to \"n\" bytes from the file starting at \"offset\".\r\n\r\n\"scratch[0..n-1]\" may be written by this routine. Sets \"*result\" to the data that was read (including if fewer than \"n\" bytes were successfully read). May set \"*result\" to point at data in \"scratch[0..n-1]\", so \"scratch[0..n-1]\" must be live when \"*result\" is used.\r\n\r\nOn OK returned status: \"n\" bytes have been stored in \"*result\". On non-OK returned status: [0..n] bytes have been stored in \"*result\".\r\n\r\nReturns `OUT_OF_RANGE` if fewer than n bytes were stored in \"*result\" because of EOF.\r\n\r\nSafe for concurrent use by multiple threads.\r\n"
},
{
"path": "tex_pdf/api/cc/ClassSession.md",
"content": "# Class `tensorflow::Session` \r\n\r\nA Session instance lets a caller drive a TensorFlow graph computation.\r\n\r\nWhen a Session is created with a given target, a new Session object is bound to the universe of resources specified by that target. Those resources are available to this session to perform computation described in the GraphDef. After extending the session with a graph, the caller uses the Run() API to perform the computation and potentially fetch outputs as Tensors.\r\n\r\nExample:\r\n\r\n```c++ tensorflow::GraphDef graph;\r\n// ... Create or load graph into \"graph\".\r\n\r\n// This example uses the default options which connects\r\n// to a local runtime.\r\ntensorflow::SessionOptions options;\r\nstd::unique_ptr