![\"icon\"](\"assets/icon.png\"){kind=icon}
\n\n \n
\n ![\"teaser\"](\"assets/teaser.png\")
\n
\n ![\"method\"](\"assets/method.png\")
\n
![](\"assets/demo1.png\")
](https://imgsli.com/MzU2MjU1) [![](\"assets/demo2.png\")
](https://imgsli.com/MzU2MjU2) [![](\"assets/demo3.png\")
](https://imgsli.com/MzU2MjU3)\n\n[![](\"assets/demo4.png\")
](https://imgsli.com/MzU2NTg4) [![](\"assets/demo5.png\")
](https://imgsli.com/MzU2NTkw) [![](\"assets/demo6.png\")
](https://imgsli.com/MzU2NTk1)\n\n[![](\"assets/demo7.png\")
](https://imgsli.com/MzU2OTE0) [![](\"assets/demo8.png\")
](https://imgsli.com/MzU2OTE1)\n\nhttps://github.com/user-attachments/assets/1211cefa-8704-47f5-82cd-ec4ef084b9ec\n\n![\"comp\"](\"assets/comp.png\")
\n\n## ā Installation\n\n```shell\ngit clone https://github.com/Guaishou74851/AdcSR.git\ncd AdcSR\nconda create -n AdcSR python=3.10\nconda activate AdcSR\npip install --upgrade pip\npip install -r requirements.txt\nchmod +x train.sh train_debug.sh test_debug.sh evaluate_debug.sh\n```\n\n## ā” Test\n\n1. **Download test datasets** (`DIV2K-Val.zip`, `DRealSR.zip`, `RealSR.zip`) from [Hugging Face](https://huggingface.co/Guaishou74851/AdcSR) or [PKU Disk](https://disk.pku.edu.cn/link/AAD499197CBF054392BC4061F904CC4026).\n2. **Unzip** them into `./testset/`, ensuring the structure:\n ```\n ./testset/DIV2K-Val/LR/xxx.png\n ./testset/DIV2K-Val/HR/xxx.png\n ./testset/DRealSR/LR/xxx.png\n ./testset/DRealSR/HR/xxx.png\n ./testset/RealSR/LR/xxx.png\n ./testset/RealSR/HR/xxx.png\n ```\n3. **Download model weights** (`net_params_200.pkl`) from the same link and place it in `./weight/`. \n4. **Run the test script** (or modify and execute `./test_debug.sh` for convenience): \n ```bash\n python test.py --LR_dir=path_to_LR_images --SR_dir=path_to_SR_images\n ```\n The results will be saved in `path_to_SR_images`.\n5. **Test Your Own Images**:\n - Place your **low-resolution (LR)** images into `./testset/xxx/`.\n - Run the command with `--LR_dir=./testset/xxx/ --SR_dir=./yyy/`, and the model will perform **x4 super-resolution**.\n\n## š Evaluation\n\nRun the evaluation script (or modify and execute `./evaluate_debug.sh` for convenience):\n```bash\npython evaluate.py --HR_dir=path_to_HR_images --SR_dir=path_to_SR_images\n```\n\n## š„ Train\n\nThis repo provides code for **Stage 2** training (**adversarial distillation**). For **Stage 1** (pretraining the channel-pruned VAE decoder), refer to our paper and use the code of [Latent Diffusion Models](https://github.com/CompVis/latent-diffusion) repo.\n\n1. **Download pretrained model weights** (`DAPE.pth`, `halfDecoder.ckpt`, `osediff.pkl`, `ram_swin_large_14m.pth`) from [Hugging Face](https://huggingface.co/Guaishou74851/AdcSR) or [PKU Disk](https://disk.pku.edu.cn/link/AAD499197CBF054392BC4061F904CC4026), and place them in `./weight/pretrained/`.\n2. **Download the [LSDIR](https://huggingface.co/ofsoundof/LSDIR) dataset** and store it in your preferred location.\n3. **Modify the dataset path** in `config.yml`:\n ```yaml\n dataroot_gt: path_to_HR_images_of_LSDIR\n ```\n4. **Run the training script** (or modify and execute `./train.sh` or `./train_debug.sh`):\n ```bash\n CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.run --nproc_per_node=8 --master_port=23333 train.py\n ```\n The trained model will be saved in `./weight/`.\n \n## š„° Acknowledgement\n\nThis project is built upon the codes of [Latent Diffusion Models](https://github.com/CompVis/latent-diffusion), [Diffusers](https://github.com/huggingface/diffusers), [BasicSR](https://github.com/XPixelGroup/BasicSR), and [OSEDiff](https://github.com/cswry/OSEDiff). We sincerely thank the authors of these repos for their significant contributions.\n\n## š Citation\n\nIf you find our work helpful, please consider citing:\n\n```latex\n@inproceedings{chen2025adversarial,\n title={Adversarial Diffusion Compression for Real-World Image Super-Resolution},\n author={Chen, Bin and Li, Gehui and Wu, Rongyuan and Zhang, Xindong and Chen, Jie and Zhang, Jian and Zhang, Lei},\n booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n year={2025}\n}\n```\n"
},
{
"path": "bsr/degradations.py",
"content": "import cv2\nimport math\nimport numpy as np\nimport random\nimport torch\nfrom scipy import special\nfrom scipy.stats import multivariate_normal\nfrom torchvision.transforms._functional_tensor import rgb_to_grayscale\n\n# -------------------------------------------------------------------- #\n# --------------------------- blur kernels --------------------------- #\n# -------------------------------------------------------------------- #\n\n\n# --------------------------- util functions --------------------------- #\ndef sigma_matrix2(sig_x, sig_y, theta):\n \"\"\"Calculate the rotated sigma matrix (two dimensional matrix).\n\n Args:\n sig_x (float):\n sig_y (float):\n theta (float): Radian measurement.\n\n Returns:\n ndarray: Rotated sigma matrix.\n \"\"\"\n d_matrix = np.array([[sig_x**2, 0], [0, sig_y**2]])\n u_matrix = np.array([[np.cos(theta), -np.sin(theta)], [np.sin(theta), np.cos(theta)]])\n return np.dot(u_matrix, np.dot(d_matrix, u_matrix.T))\n\n\ndef mesh_grid(kernel_size):\n \"\"\"Generate the mesh grid, centering at zero.\n\n Args:\n kernel_size (int):\n\n Returns:\n xy (ndarray): with the shape (kernel_size, kernel_size, 2)\n xx (ndarray): with the shape (kernel_size, kernel_size)\n yy (ndarray): with the shape (kernel_size, kernel_size)\n \"\"\"\n ax = np.arange(-kernel_size // 2 + 1., kernel_size // 2 + 1.)\n xx, yy = np.meshgrid(ax, ax)\n xy = np.hstack((xx.reshape((kernel_size * kernel_size, 1)), yy.reshape(kernel_size * kernel_size,\n 1))).reshape(kernel_size, kernel_size, 2)\n return xy, xx, yy\n\n\ndef pdf2(sigma_matrix, grid):\n \"\"\"Calculate PDF of the bivariate Gaussian distribution.\n\n Args:\n sigma_matrix (ndarray): with the shape (2, 2)\n grid (ndarray): generated by :func:`mesh_grid`,\n with the shape (K, K, 2), K is the kernel size.\n\n Returns:\n kernel (ndarrray): un-normalized kernel.\n \"\"\"\n inverse_sigma = np.linalg.inv(sigma_matrix)\n kernel = np.exp(-0.5 * np.sum(np.dot(grid, inverse_sigma) * grid, 2))\n return kernel\n\n\ndef cdf2(d_matrix, grid):\n \"\"\"Calculate the CDF of the standard bivariate Gaussian distribution.\n Used in skewed Gaussian distribution.\n\n Args:\n d_matrix (ndarrasy): skew matrix.\n grid (ndarray): generated by :func:`mesh_grid`,\n with the shape (K, K, 2), K is the kernel size.\n\n Returns:\n cdf (ndarray): skewed cdf.\n \"\"\"\n rv = multivariate_normal([0, 0], [[1, 0], [0, 1]])\n grid = np.dot(grid, d_matrix)\n cdf = rv.cdf(grid)\n return cdf\n\n\ndef bivariate_Gaussian(kernel_size, sig_x, sig_y, theta, grid=None, isotropic=True):\n \"\"\"Generate a bivariate isotropic or anisotropic Gaussian kernel.\n\n In the isotropic mode, only `sig_x` is used. `sig_y` and `theta` is ignored.\n\n Args:\n kernel_size (int):\n sig_x (float):\n sig_y (float):\n theta (float): Radian measurement.\n grid (ndarray, optional): generated by :func:`mesh_grid`,\n with the shape (K, K, 2), K is the kernel size. Default: None\n isotropic (bool):\n\n Returns:\n kernel (ndarray): normalized kernel.\n \"\"\"\n if grid is None:\n grid, _, _ = mesh_grid(kernel_size)\n if isotropic:\n sigma_matrix = np.array([[sig_x**2, 0], [0, sig_x**2]])\n else:\n sigma_matrix = sigma_matrix2(sig_x, sig_y, theta)\n kernel = pdf2(sigma_matrix, grid)\n kernel = kernel / np.sum(kernel)\n return kernel\n\n\ndef bivariate_generalized_Gaussian(kernel_size, sig_x, sig_y, theta, beta, grid=None, isotropic=True):\n \"\"\"Generate a bivariate generalized Gaussian kernel.\n\n ``Paper: Parameter Estimation For Multivariate Generalized Gaussian Distributions``\n\n In the isotropic mode, only `sig_x` is used. `sig_y` and `theta` is ignored.\n\n Args:\n kernel_size (int):\n sig_x (float):\n sig_y (float):\n theta (float): Radian measurement.\n beta (float): shape parameter, beta = 1 is the normal distribution.\n grid (ndarray, optional): generated by :func:`mesh_grid`,\n with the shape (K, K, 2), K is the kernel size. Default: None\n\n Returns:\n kernel (ndarray): normalized kernel.\n \"\"\"\n if grid is None:\n grid, _, _ = mesh_grid(kernel_size)\n if isotropic:\n sigma_matrix = np.array([[sig_x**2, 0], [0, sig_x**2]])\n else:\n sigma_matrix = sigma_matrix2(sig_x, sig_y, theta)\n inverse_sigma = np.linalg.inv(sigma_matrix)\n kernel = np.exp(-0.5 * np.power(np.sum(np.dot(grid, inverse_sigma) * grid, 2), beta))\n kernel = kernel / np.sum(kernel)\n return kernel\n\n\ndef bivariate_plateau(kernel_size, sig_x, sig_y, theta, beta, grid=None, isotropic=True):\n \"\"\"Generate a plateau-like anisotropic kernel.\n\n 1 / (1+x^(beta))\n\n Reference: https://stats.stackexchange.com/questions/203629/is-there-a-plateau-shaped-distribution\n\n In the isotropic mode, only `sig_x` is used. `sig_y` and `theta` is ignored.\n\n Args:\n kernel_size (int):\n sig_x (float):\n sig_y (float):\n theta (float): Radian measurement.\n beta (float): shape parameter, beta = 1 is the normal distribution.\n grid (ndarray, optional): generated by :func:`mesh_grid`,\n with the shape (K, K, 2), K is the kernel size. Default: None\n\n Returns:\n kernel (ndarray): normalized kernel.\n \"\"\"\n if grid is None:\n grid, _, _ = mesh_grid(kernel_size)\n if isotropic:\n sigma_matrix = np.array([[sig_x**2, 0], [0, sig_x**2]])\n else:\n sigma_matrix = sigma_matrix2(sig_x, sig_y, theta)\n inverse_sigma = np.linalg.inv(sigma_matrix)\n kernel = np.reciprocal(np.power(np.sum(np.dot(grid, inverse_sigma) * grid, 2), beta) + 1)\n kernel = kernel / np.sum(kernel)\n return kernel\n\n\ndef random_bivariate_Gaussian(kernel_size,\n sigma_x_range,\n sigma_y_range,\n rotation_range,\n noise_range=None,\n isotropic=True):\n \"\"\"Randomly generate bivariate isotropic or anisotropic Gaussian kernels.\n\n In the isotropic mode, only `sigma_x_range` is used. `sigma_y_range` and `rotation_range` is ignored.\n\n Args:\n kernel_size (int):\n sigma_x_range (tuple): [0.6, 5]\n sigma_y_range (tuple): [0.6, 5]\n rotation range (tuple): [-math.pi, math.pi]\n noise_range(tuple, optional): multiplicative kernel noise,\n [0.75, 1.25]. Default: None\n\n Returns:\n kernel (ndarray):\n \"\"\"\n assert kernel_size % 2 == 1, 'Kernel size must be an odd number.'\n assert sigma_x_range[0] < sigma_x_range[1], 'Wrong sigma_x_range.'\n sigma_x = np.random.uniform(sigma_x_range[0], sigma_x_range[1])\n if isotropic is False:\n assert sigma_y_range[0] < sigma_y_range[1], 'Wrong sigma_y_range.'\n assert rotation_range[0] < rotation_range[1], 'Wrong rotation_range.'\n sigma_y = np.random.uniform(sigma_y_range[0], sigma_y_range[1])\n rotation = np.random.uniform(rotation_range[0], rotation_range[1])\n else:\n sigma_y = sigma_x\n rotation = 0\n\n kernel = bivariate_Gaussian(kernel_size, sigma_x, sigma_y, rotation, isotropic=isotropic)\n\n # add multiplicative noise\n if noise_range is not None:\n assert noise_range[0] < noise_range[1], 'Wrong noise range.'\n noise = np.random.uniform(noise_range[0], noise_range[1], size=kernel.shape)\n kernel = kernel * noise\n kernel = kernel / np.sum(kernel)\n return kernel\n\n\ndef random_bivariate_generalized_Gaussian(kernel_size,\n sigma_x_range,\n sigma_y_range,\n rotation_range,\n beta_range,\n noise_range=None,\n isotropic=True):\n \"\"\"Randomly generate bivariate generalized Gaussian kernels.\n\n In the isotropic mode, only `sigma_x_range` is used. `sigma_y_range` and `rotation_range` is ignored.\n\n Args:\n kernel_size (int):\n sigma_x_range (tuple): [0.6, 5]\n sigma_y_range (tuple): [0.6, 5]\n rotation range (tuple): [-math.pi, math.pi]\n beta_range (tuple): [0.5, 8]\n noise_range(tuple, optional): multiplicative kernel noise,\n [0.75, 1.25]. Default: None\n\n Returns:\n kernel (ndarray):\n \"\"\"\n assert kernel_size % 2 == 1, 'Kernel size must be an odd number.'\n assert sigma_x_range[0] < sigma_x_range[1], 'Wrong sigma_x_range.'\n sigma_x = np.random.uniform(sigma_x_range[0], sigma_x_range[1])\n if isotropic is False:\n assert sigma_y_range[0] < sigma_y_range[1], 'Wrong sigma_y_range.'\n assert rotation_range[0] < rotation_range[1], 'Wrong rotation_range.'\n sigma_y = np.random.uniform(sigma_y_range[0], sigma_y_range[1])\n rotation = np.random.uniform(rotation_range[0], rotation_range[1])\n else:\n sigma_y = sigma_x\n rotation = 0\n\n # assume beta_range[0] < 1 < beta_range[1]\n if np.random.uniform() < 0.5:\n beta = np.random.uniform(beta_range[0], 1)\n else:\n beta = np.random.uniform(1, beta_range[1])\n\n kernel = bivariate_generalized_Gaussian(kernel_size, sigma_x, sigma_y, rotation, beta, isotropic=isotropic)\n\n # add multiplicative noise\n if noise_range is not None:\n assert noise_range[0] < noise_range[1], 'Wrong noise range.'\n noise = np.random.uniform(noise_range[0], noise_range[1], size=kernel.shape)\n kernel = kernel * noise\n kernel = kernel / np.sum(kernel)\n return kernel\n\n\ndef random_bivariate_plateau(kernel_size,\n sigma_x_range,\n sigma_y_range,\n rotation_range,\n beta_range,\n noise_range=None,\n isotropic=True):\n \"\"\"Randomly generate bivariate plateau kernels.\n\n In the isotropic mode, only `sigma_x_range` is used. `sigma_y_range` and `rotation_range` is ignored.\n\n Args:\n kernel_size (int):\n sigma_x_range (tuple): [0.6, 5]\n sigma_y_range (tuple): [0.6, 5]\n rotation range (tuple): [-math.pi/2, math.pi/2]\n beta_range (tuple): [1, 4]\n noise_range(tuple, optional): multiplicative kernel noise,\n [0.75, 1.25]. Default: None\n\n Returns:\n kernel (ndarray):\n \"\"\"\n assert kernel_size % 2 == 1, 'Kernel size must be an odd number.'\n assert sigma_x_range[0] < sigma_x_range[1], 'Wrong sigma_x_range.'\n sigma_x = np.random.uniform(sigma_x_range[0], sigma_x_range[1])\n if isotropic is False:\n assert sigma_y_range[0] < sigma_y_range[1], 'Wrong sigma_y_range.'\n assert rotation_range[0] < rotation_range[1], 'Wrong rotation_range.'\n sigma_y = np.random.uniform(sigma_y_range[0], sigma_y_range[1])\n rotation = np.random.uniform(rotation_range[0], rotation_range[1])\n else:\n sigma_y = sigma_x\n rotation = 0\n\n # TODO: this may be not proper\n if np.random.uniform() < 0.5:\n beta = np.random.uniform(beta_range[0], 1)\n else:\n beta = np.random.uniform(1, beta_range[1])\n\n kernel = bivariate_plateau(kernel_size, sigma_x, sigma_y, rotation, beta, isotropic=isotropic)\n # add multiplicative noise\n if noise_range is not None:\n assert noise_range[0] < noise_range[1], 'Wrong noise range.'\n noise = np.random.uniform(noise_range[0], noise_range[1], size=kernel.shape)\n kernel = kernel * noise\n kernel = kernel / np.sum(kernel)\n\n return kernel\n\n\ndef random_mixed_kernels(kernel_list,\n kernel_prob,\n kernel_size=21,\n sigma_x_range=(0.6, 5),\n sigma_y_range=(0.6, 5),\n rotation_range=(-math.pi, math.pi),\n betag_range=(0.5, 8),\n betap_range=(0.5, 8),\n noise_range=None):\n \"\"\"Randomly generate mixed kernels.\n\n Args:\n kernel_list (tuple): a list name of kernel types,\n support ['iso', 'aniso', 'skew', 'generalized', 'plateau_iso',\n 'plateau_aniso']\n kernel_prob (tuple): corresponding kernel probability for each\n kernel type\n kernel_size (int):\n sigma_x_range (tuple): [0.6, 5]\n sigma_y_range (tuple): [0.6, 5]\n rotation range (tuple): [-math.pi, math.pi]\n beta_range (tuple): [0.5, 8]\n noise_range(tuple, optional): multiplicative kernel noise,\n [0.75, 1.25]. Default: None\n\n Returns:\n kernel (ndarray):\n \"\"\"\n kernel_type = random.choices(kernel_list, kernel_prob)[0]\n if kernel_type == 'iso':\n kernel = random_bivariate_Gaussian(\n kernel_size, sigma_x_range, sigma_y_range, rotation_range, noise_range=noise_range, isotropic=True)\n elif kernel_type == 'aniso':\n kernel = random_bivariate_Gaussian(\n kernel_size, sigma_x_range, sigma_y_range, rotation_range, noise_range=noise_range, isotropic=False)\n elif kernel_type == 'generalized_iso':\n kernel = random_bivariate_generalized_Gaussian(\n kernel_size,\n sigma_x_range,\n sigma_y_range,\n rotation_range,\n betag_range,\n noise_range=noise_range,\n isotropic=True)\n elif kernel_type == 'generalized_aniso':\n kernel = random_bivariate_generalized_Gaussian(\n kernel_size,\n sigma_x_range,\n sigma_y_range,\n rotation_range,\n betag_range,\n noise_range=noise_range,\n isotropic=False)\n elif kernel_type == 'plateau_iso':\n kernel = random_bivariate_plateau(\n kernel_size, sigma_x_range, sigma_y_range, rotation_range, betap_range, noise_range=None, isotropic=True)\n elif kernel_type == 'plateau_aniso':\n kernel = random_bivariate_plateau(\n kernel_size, sigma_x_range, sigma_y_range, rotation_range, betap_range, noise_range=None, isotropic=False)\n return kernel\n\n\nnp.seterr(divide='ignore', invalid='ignore')\n\n\ndef circular_lowpass_kernel(cutoff, kernel_size, pad_to=0):\n \"\"\"2D sinc filter\n\n Reference: https://dsp.stackexchange.com/questions/58301/2-d-circularly-symmetric-low-pass-filter\n\n Args:\n cutoff (float): cutoff frequency in radians (pi is max)\n kernel_size (int): horizontal and vertical size, must be odd.\n pad_to (int): pad kernel size to desired size, must be odd or zero.\n \"\"\"\n assert kernel_size % 2 == 1, 'Kernel size must be an odd number.'\n kernel = np.fromfunction(\n lambda x, y: cutoff * special.j1(cutoff * np.sqrt(\n (x - (kernel_size - 1) / 2)**2 + (y - (kernel_size - 1) / 2)**2)) / (2 * np.pi * np.sqrt(\n (x - (kernel_size - 1) / 2)**2 + (y - (kernel_size - 1) / 2)**2)), [kernel_size, kernel_size])\n kernel[(kernel_size - 1) // 2, (kernel_size - 1) // 2] = cutoff**2 / (4 * np.pi)\n kernel = kernel / np.sum(kernel)\n if pad_to > kernel_size:\n pad_size = (pad_to - kernel_size) // 2\n kernel = np.pad(kernel, ((pad_size, pad_size), (pad_size, pad_size)))\n return kernel\n\n\n# ------------------------------------------------------------- #\n# --------------------------- noise --------------------------- #\n# ------------------------------------------------------------- #\n\n# ----------------------- Gaussian Noise ----------------------- #\n\n\ndef generate_gaussian_noise(img, sigma=10, gray_noise=False):\n \"\"\"Generate Gaussian noise.\n\n Args:\n img (Numpy array): Input image, shape (h, w, c), range [0, 1], float32.\n sigma (float): Noise scale (measured in range 255). Default: 10.\n\n Returns:\n (Numpy array): Returned noisy image, shape (h, w, c), range[0, 1],\n float32.\n \"\"\"\n if gray_noise:\n noise = np.float32(np.random.randn(*(img.shape[0:2]))) * sigma / 255.\n noise = np.expand_dims(noise, axis=2).repeat(3, axis=2)\n else:\n noise = np.float32(np.random.randn(*(img.shape))) * sigma / 255.\n return noise\n\n\ndef add_gaussian_noise(img, sigma=10, clip=True, rounds=False, gray_noise=False):\n \"\"\"Add Gaussian noise.\n\n Args:\n img (Numpy array): Input image, shape (h, w, c), range [0, 1], float32.\n sigma (float): Noise scale (measured in range 255). Default: 10.\n\n Returns:\n (Numpy array): Returned noisy image, shape (h, w, c), range[0, 1],\n float32.\n \"\"\"\n noise = generate_gaussian_noise(img, sigma, gray_noise)\n out = img + noise\n if clip and rounds:\n out = np.clip((out * 255.0).round(), 0, 255) / 255.\n elif clip:\n out = np.clip(out, 0, 1)\n elif rounds:\n out = (out * 255.0).round() / 255.\n return out\n\n\ndef generate_gaussian_noise_pt(img, sigma=10, gray_noise=0):\n \"\"\"Add Gaussian noise (PyTorch version).\n\n Args:\n img (Tensor): Shape (b, c, h, w), range[0, 1], float32.\n scale (float | Tensor): Noise scale. Default: 1.0.\n\n Returns:\n (Tensor): Returned noisy image, shape (b, c, h, w), range[0, 1],\n float32.\n \"\"\"\n b, _, h, w = img.size()\n if not isinstance(sigma, (float, int)):\n sigma = sigma.view(img.size(0), 1, 1, 1)\n if isinstance(gray_noise, (float, int)):\n cal_gray_noise = gray_noise > 0\n else:\n gray_noise = gray_noise.view(b, 1, 1, 1)\n cal_gray_noise = torch.sum(gray_noise) > 0\n\n if cal_gray_noise:\n noise_gray = torch.randn(*img.size()[2:4], dtype=img.dtype, device=img.device) * sigma / 255.\n noise_gray = noise_gray.view(b, 1, h, w)\n\n # always calculate color noise\n noise = torch.randn(*img.size(), dtype=img.dtype, device=img.device) * sigma / 255.\n\n if cal_gray_noise:\n noise = noise * (1 - gray_noise) + noise_gray * gray_noise\n return noise\n\n\ndef add_gaussian_noise_pt(img, sigma=10, gray_noise=0, clip=True, rounds=False):\n \"\"\"Add Gaussian noise (PyTorch version).\n\n Args:\n img (Tensor): Shape (b, c, h, w), range[0, 1], float32.\n scale (float | Tensor): Noise scale. Default: 1.0.\n\n Returns:\n (Tensor): Returned noisy image, shape (b, c, h, w), range[0, 1],\n float32.\n \"\"\"\n noise = generate_gaussian_noise_pt(img, sigma, gray_noise)\n out = img + noise\n if clip and rounds:\n out = torch.clamp((out * 255.0).round(), 0, 255) / 255.\n elif clip:\n out = torch.clamp(out, 0, 1)\n elif rounds:\n out = (out * 255.0).round() / 255.\n return out\n\n\n# ----------------------- Random Gaussian Noise ----------------------- #\ndef random_generate_gaussian_noise(img, sigma_range=(0, 10), gray_prob=0):\n sigma = np.random.uniform(sigma_range[0], sigma_range[1])\n if np.random.uniform() < gray_prob:\n gray_noise = True\n else:\n gray_noise = False\n return generate_gaussian_noise(img, sigma, gray_noise)\n\n\ndef random_add_gaussian_noise(img, sigma_range=(0, 1.0), gray_prob=0, clip=True, rounds=False):\n noise = random_generate_gaussian_noise(img, sigma_range, gray_prob)\n out = img + noise\n if clip and rounds:\n out = np.clip((out * 255.0).round(), 0, 255) / 255.\n elif clip:\n out = np.clip(out, 0, 1)\n elif rounds:\n out = (out * 255.0).round() / 255.\n return out\n\n\ndef random_generate_gaussian_noise_pt(img, sigma_range=(0, 10), gray_prob=0):\n sigma = torch.rand(\n img.size(0), dtype=img.dtype, device=img.device) * (sigma_range[1] - sigma_range[0]) + sigma_range[0]\n gray_noise = torch.rand(img.size(0), dtype=img.dtype, device=img.device)\n gray_noise = (gray_noise < gray_prob).float()\n return generate_gaussian_noise_pt(img, sigma, gray_noise)\n\n\ndef random_add_gaussian_noise_pt(img, sigma_range=(0, 1.0), gray_prob=0, clip=True, rounds=False):\n noise = random_generate_gaussian_noise_pt(img, sigma_range, gray_prob)\n out = img + noise\n if clip and rounds:\n out = torch.clamp((out * 255.0).round(), 0, 255) / 255.\n elif clip:\n out = torch.clamp(out, 0, 1)\n elif rounds:\n out = (out * 255.0).round() / 255.\n return out\n\n\n# ----------------------- Poisson (Shot) Noise ----------------------- #\n\n\ndef generate_poisson_noise(img, scale=1.0, gray_noise=False):\n \"\"\"Generate poisson noise.\n\n Reference: https://github.com/scikit-image/scikit-image/blob/main/skimage/util/noise.py#L37-L219\n\n Args:\n img (Numpy array): Input image, shape (h, w, c), range [0, 1], float32.\n scale (float): Noise scale. Default: 1.0.\n gray_noise (bool): Whether generate gray noise. Default: False.\n\n Returns:\n (Numpy array): Returned noisy image, shape (h, w, c), range[0, 1],\n float32.\n \"\"\"\n if gray_noise:\n img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n # round and clip image for counting vals correctly\n img = np.clip((img * 255.0).round(), 0, 255) / 255.\n vals = len(np.unique(img))\n vals = 2**np.ceil(np.log2(vals))\n out = np.float32(np.random.poisson(img * vals) / float(vals))\n noise = out - img\n if gray_noise:\n noise = np.repeat(noise[:, :, np.newaxis], 3, axis=2)\n return noise * scale\n\n\ndef add_poisson_noise(img, scale=1.0, clip=True, rounds=False, gray_noise=False):\n \"\"\"Add poisson noise.\n\n Args:\n img (Numpy array): Input image, shape (h, w, c), range [0, 1], float32.\n scale (float): Noise scale. Default: 1.0.\n gray_noise (bool): Whether generate gray noise. Default: False.\n\n Returns:\n (Numpy array): Returned noisy image, shape (h, w, c), range[0, 1],\n float32.\n \"\"\"\n noise = generate_poisson_noise(img, scale, gray_noise)\n out = img + noise\n if clip and rounds:\n out = np.clip((out * 255.0).round(), 0, 255) / 255.\n elif clip:\n out = np.clip(out, 0, 1)\n elif rounds:\n out = (out * 255.0).round() / 255.\n return out\n\n\ndef generate_poisson_noise_pt(img, scale=1.0, gray_noise=0):\n \"\"\"Generate a batch of poisson noise (PyTorch version)\n\n Args:\n img (Tensor): Input image, shape (b, c, h, w), range [0, 1], float32.\n scale (float | Tensor): Noise scale. Number or Tensor with shape (b).\n Default: 1.0.\n gray_noise (float | Tensor): 0-1 number or Tensor with shape (b).\n 0 for False, 1 for True. Default: 0.\n\n Returns:\n (Tensor): Returned noisy image, shape (b, c, h, w), range[0, 1],\n float32.\n \"\"\"\n b, _, h, w = img.size()\n if isinstance(gray_noise, (float, int)):\n cal_gray_noise = gray_noise > 0\n else:\n gray_noise = gray_noise.view(b, 1, 1, 1)\n cal_gray_noise = torch.sum(gray_noise) > 0\n if cal_gray_noise:\n img_gray = rgb_to_grayscale(img, num_output_channels=1)\n # round and clip image for counting vals correctly\n img_gray = torch.clamp((img_gray * 255.0).round(), 0, 255) / 255.\n # use for-loop to get the unique values for each sample\n vals_list = [len(torch.unique(img_gray[i, :, :, :])) for i in range(b)]\n vals_list = [2**np.ceil(np.log2(vals)) for vals in vals_list]\n vals = img_gray.new_tensor(vals_list).view(b, 1, 1, 1)\n out = torch.poisson(img_gray * vals) / vals\n noise_gray = out - img_gray\n noise_gray = noise_gray.expand(b, 3, h, w)\n\n # always calculate color noise\n # round and clip image for counting vals correctly\n img = torch.clamp((img * 255.0).round(), 0, 255) / 255.\n # use for-loop to get the unique values for each sample\n vals_list = [len(torch.unique(img[i, :, :, :])) for i in range(b)]\n vals_list = [2**np.ceil(np.log2(vals)) for vals in vals_list]\n vals = img.new_tensor(vals_list).view(b, 1, 1, 1)\n out = torch.poisson(img * vals) / vals\n noise = out - img\n if cal_gray_noise:\n noise = noise * (1 - gray_noise) + noise_gray * gray_noise\n if not isinstance(scale, (float, int)):\n scale = scale.view(b, 1, 1, 1)\n return noise * scale\n\n\ndef add_poisson_noise_pt(img, scale=1.0, clip=True, rounds=False, gray_noise=0):\n \"\"\"Add poisson noise to a batch of images (PyTorch version).\n\n Args:\n img (Tensor): Input image, shape (b, c, h, w), range [0, 1], float32.\n scale (float | Tensor): Noise scale. Number or Tensor with shape (b).\n Default: 1.0.\n gray_noise (float | Tensor): 0-1 number or Tensor with shape (b).\n 0 for False, 1 for True. Default: 0.\n\n Returns:\n (Tensor): Returned noisy image, shape (b, c, h, w), range[0, 1],\n float32.\n \"\"\"\n noise = generate_poisson_noise_pt(img, scale, gray_noise)\n out = img + noise\n if clip and rounds:\n out = torch.clamp((out * 255.0).round(), 0, 255) / 255.\n elif clip:\n out = torch.clamp(out, 0, 1)\n elif rounds:\n out = (out * 255.0).round() / 255.\n return out\n\n\n# ----------------------- Random Poisson (Shot) Noise ----------------------- #\n\n\ndef random_generate_poisson_noise(img, scale_range=(0, 1.0), gray_prob=0):\n scale = np.random.uniform(scale_range[0], scale_range[1])\n if np.random.uniform() < gray_prob:\n gray_noise = True\n else:\n gray_noise = False\n return generate_poisson_noise(img, scale, gray_noise)\n\n\ndef random_add_poisson_noise(img, scale_range=(0, 1.0), gray_prob=0, clip=True, rounds=False):\n noise = random_generate_poisson_noise(img, scale_range, gray_prob)\n out = img + noise\n if clip and rounds:\n out = np.clip((out * 255.0).round(), 0, 255) / 255.\n elif clip:\n out = np.clip(out, 0, 1)\n elif rounds:\n out = (out * 255.0).round() / 255.\n return out\n\n\ndef random_generate_poisson_noise_pt(img, scale_range=(0, 1.0), gray_prob=0):\n scale = torch.rand(\n img.size(0), dtype=img.dtype, device=img.device) * (scale_range[1] - scale_range[0]) + scale_range[0]\n gray_noise = torch.rand(img.size(0), dtype=img.dtype, device=img.device)\n gray_noise = (gray_noise < gray_prob).float()\n return generate_poisson_noise_pt(img, scale, gray_noise)\n\n\ndef random_add_poisson_noise_pt(img, scale_range=(0, 1.0), gray_prob=0, clip=True, rounds=False):\n noise = random_generate_poisson_noise_pt(img, scale_range, gray_prob)\n out = img + noise\n if clip and rounds:\n out = torch.clamp((out * 255.0).round(), 0, 255) / 255.\n elif clip:\n out = torch.clamp(out, 0, 1)\n elif rounds:\n out = (out * 255.0).round() / 255.\n return out\n\n\n# ------------------------------------------------------------------------ #\n# --------------------------- JPEG compression --------------------------- #\n# ------------------------------------------------------------------------ #\n\n\ndef add_jpg_compression(img, quality=90):\n \"\"\"Add JPG compression artifacts.\n\n Args:\n img (Numpy array): Input image, shape (h, w, c), range [0, 1], float32.\n quality (float): JPG compression quality. 0 for lowest quality, 100 for\n best quality. Default: 90.\n\n Returns:\n (Numpy array): Returned image after JPG, shape (h, w, c), range[0, 1],\n float32.\n \"\"\"\n img = np.clip(img, 0, 1)\n encode_param = [int(cv2.IMWRITE_JPEG_QUALITY), quality]\n _, encimg = cv2.imencode('.jpg', img * 255., encode_param)\n img = np.float32(cv2.imdecode(encimg, 1)) / 255.\n return img\n\n\ndef random_add_jpg_compression(img, quality_range=(90, 100)):\n \"\"\"Randomly add JPG compression artifacts.\n\n Args:\n img (Numpy array): Input image, shape (h, w, c), range [0, 1], float32.\n quality_range (tuple[float] | list[float]): JPG compression quality\n range. 0 for lowest quality, 100 for best quality.\n Default: (90, 100).\n\n Returns:\n (Numpy array): Returned image after JPG, shape (h, w, c), range[0, 1],\n float32.\n \"\"\"\n quality = np.random.uniform(quality_range[0], quality_range[1])\n return add_jpg_compression(img, quality)\n"
},
{
"path": "bsr/transforms.py",
"content": "import cv2\nimport random\nimport torch\n\n\ndef mod_crop(img, scale):\n \"\"\"Mod crop images, used during testing.\n\n Args:\n img (ndarray): Input image.\n scale (int): Scale factor.\n\n Returns:\n ndarray: Result image.\n \"\"\"\n img = img.copy()\n if img.ndim in (2, 3):\n h, w = img.shape[0], img.shape[1]\n h_remainder, w_remainder = h % scale, w % scale\n img = img[:h - h_remainder, :w - w_remainder, ...]\n else:\n raise ValueError(f'Wrong img ndim: {img.ndim}.')\n return img\n\n\ndef paired_random_crop(img_gts, img_lqs, gt_patch_size, scale, gt_path=None):\n \"\"\"Paired random crop. Support Numpy array and Tensor inputs.\n\n It crops lists of lq and gt images with corresponding locations.\n\n Args:\n img_gts (list[ndarray] | ndarray | list[Tensor] | Tensor): GT images. Note that all images\n should have the same shape. If the input is an ndarray, it will\n be transformed to a list containing itself.\n img_lqs (list[ndarray] | ndarray): LQ images. Note that all images\n should have the same shape. If the input is an ndarray, it will\n be transformed to a list containing itself.\n gt_patch_size (int): GT patch size.\n scale (int): Scale factor.\n gt_path (str): Path to ground-truth. Default: None.\n\n Returns:\n list[ndarray] | ndarray: GT images and LQ images. If returned results\n only have one element, just return ndarray.\n \"\"\"\n\n if not isinstance(img_gts, list):\n img_gts = [img_gts]\n if not isinstance(img_lqs, list):\n img_lqs = [img_lqs]\n\n # determine input type: Numpy array or Tensor\n input_type = 'Tensor' if torch.is_tensor(img_gts[0]) else 'Numpy'\n\n if input_type == 'Tensor':\n h_lq, w_lq = img_lqs[0].size()[-2:]\n h_gt, w_gt = img_gts[0].size()[-2:]\n else:\n h_lq, w_lq = img_lqs[0].shape[0:2]\n h_gt, w_gt = img_gts[0].shape[0:2]\n lq_patch_size = gt_patch_size // scale\n\n if h_gt != h_lq * scale or w_gt != w_lq * scale:\n raise ValueError(f'Scale mismatches. GT ({h_gt}, {w_gt}) is not {scale}x ',\n f'multiplication of LQ ({h_lq}, {w_lq}).')\n if h_lq < lq_patch_size or w_lq < lq_patch_size:\n raise ValueError(f'LQ ({h_lq}, {w_lq}) is smaller than patch size '\n f'({lq_patch_size}, {lq_patch_size}). '\n f'Please remove {gt_path}.')\n\n # randomly choose top and left coordinates for lq patch\n top = random.randint(0, h_lq - lq_patch_size)\n left = random.randint(0, w_lq - lq_patch_size)\n\n # crop lq patch\n if input_type == 'Tensor':\n img_lqs = [v[:, :, top:top + lq_patch_size, left:left + lq_patch_size] for v in img_lqs]\n else:\n img_lqs = [v[top:top + lq_patch_size, left:left + lq_patch_size, ...] for v in img_lqs]\n\n # crop corresponding gt patch\n top_gt, left_gt = int(top * scale), int(left * scale)\n if input_type == 'Tensor':\n img_gts = [v[:, :, top_gt:top_gt + gt_patch_size, left_gt:left_gt + gt_patch_size] for v in img_gts]\n else:\n img_gts = [v[top_gt:top_gt + gt_patch_size, left_gt:left_gt + gt_patch_size, ...] for v in img_gts]\n if len(img_gts) == 1:\n img_gts = img_gts[0]\n if len(img_lqs) == 1:\n img_lqs = img_lqs[0]\n return img_gts, img_lqs\n\n\ndef augment(imgs, hflip=True, rotation=True, flows=None, return_status=False):\n \"\"\"Augment: horizontal flips OR rotate (0, 90, 180, 270 degrees).\n\n We use vertical flip and transpose for rotation implementation.\n All the images in the list use the same augmentation.\n\n Args:\n imgs (list[ndarray] | ndarray): Images to be augmented. If the input\n is an ndarray, it will be transformed to a list.\n hflip (bool): Horizontal flip. Default: True.\n rotation (bool): Ratotation. Default: True.\n flows (list[ndarray]: Flows to be augmented. If the input is an\n ndarray, it will be transformed to a list.\n Dimension is (h, w, 2). Default: None.\n return_status (bool): Return the status of flip and rotation.\n Default: False.\n\n Returns:\n list[ndarray] | ndarray: Augmented images and flows. If returned\n results only have one element, just return ndarray.\n\n \"\"\"\n hflip = hflip and random.random() < 0.5\n vflip = rotation and random.random() < 0.5\n rot90 = rotation and random.random() < 0.5\n\n def _augment(img):\n if hflip: # horizontal\n cv2.flip(img, 1, img)\n if vflip: # vertical\n cv2.flip(img, 0, img)\n if rot90:\n img = img.transpose(1, 0, 2)\n return img\n\n def _augment_flow(flow):\n if hflip: # horizontal\n cv2.flip(flow, 1, flow)\n flow[:, :, 0] *= -1\n if vflip: # vertical\n cv2.flip(flow, 0, flow)\n flow[:, :, 1] *= -1\n if rot90:\n flow = flow.transpose(1, 0, 2)\n flow = flow[:, :, [1, 0]]\n return flow\n\n if not isinstance(imgs, list):\n imgs = [imgs]\n imgs = [_augment(img) for img in imgs]\n if len(imgs) == 1:\n imgs = imgs[0]\n\n if flows is not None:\n if not isinstance(flows, list):\n flows = [flows]\n flows = [_augment_flow(flow) for flow in flows]\n if len(flows) == 1:\n flows = flows[0]\n return imgs, flows\n else:\n if return_status:\n return imgs, (hflip, vflip, rot90)\n else:\n return imgs\n\n\ndef img_rotate(img, angle, center=None, scale=1.0):\n \"\"\"Rotate image.\n\n Args:\n img (ndarray): Image to be rotated.\n angle (float): Rotation angle in degrees. Positive values mean\n counter-clockwise rotation.\n center (tuple[int]): Rotation center. If the center is None,\n initialize it as the center of the image. Default: None.\n scale (float): Isotropic scale factor. Default: 1.0.\n \"\"\"\n (h, w) = img.shape[:2]\n\n if center is None:\n center = (w // 2, h // 2)\n\n matrix = cv2.getRotationMatrix2D(center, angle, scale)\n rotated_img = cv2.warpAffine(img, matrix, (w, h))\n return rotated_img\n"
},
{
"path": "bsr/utils/__init__.py",
"content": "from .color_util import bgr2ycbcr, rgb2ycbcr, rgb2ycbcr_pt, ycbcr2bgr, ycbcr2rgb\nfrom .diffjpeg import DiffJPEG\nfrom .file_client import FileClient\nfrom .img_process_util import USMSharp, usm_sharp\nfrom .img_util import crop_border, imfrombytes, img2tensor, imwrite, tensor2img\nfrom .logger import AvgTimer, MessageLogger, get_env_info, get_root_logger, init_tb_logger, init_wandb_logger\nfrom .misc import check_resume, get_time_str, make_exp_dirs, mkdir_and_rename, scandir, set_random_seed, sizeof_fmt\nfrom .options import yaml_load\n\n__all__ = [\n # color_util.py\n 'bgr2ycbcr',\n 'rgb2ycbcr',\n 'rgb2ycbcr_pt',\n 'ycbcr2bgr',\n 'ycbcr2rgb',\n # file_client.py\n 'FileClient',\n # img_util.py\n 'img2tensor',\n 'tensor2img',\n 'imfrombytes',\n 'imwrite',\n 'crop_border',\n # logger.py\n 'MessageLogger',\n 'AvgTimer',\n 'init_tb_logger',\n 'init_wandb_logger',\n 'get_root_logger',\n 'get_env_info',\n # misc.py\n 'set_random_seed',\n 'get_time_str',\n 'mkdir_and_rename',\n 'make_exp_dirs',\n 'scandir',\n 'check_resume',\n 'sizeof_fmt',\n # diffjpeg\n 'DiffJPEG',\n # img_process_util\n 'USMSharp',\n 'usm_sharp',\n # options\n 'yaml_load'\n]\n"
},
{
"path": "bsr/utils/color_util.py",
"content": "import numpy as np\nimport torch\n\n\ndef rgb2ycbcr(img, y_only=False):\n \"\"\"Convert a RGB image to YCbCr image.\n\n This function produces the same results as Matlab's `rgb2ycbcr` function.\n It implements the ITU-R BT.601 conversion for standard-definition\n television. See more details in\n https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion.\n\n It differs from a similar function in cv2.cvtColor: `RGB <-> YCrCb`.\n In OpenCV, it implements a JPEG conversion. See more details in\n https://en.wikipedia.org/wiki/YCbCr#JPEG_conversion.\n\n Args:\n img (ndarray): The input image. It accepts:\n 1. np.uint8 type with range [0, 255];\n 2. np.float32 type with range [0, 1].\n y_only (bool): Whether to only return Y channel. Default: False.\n\n Returns:\n ndarray: The converted YCbCr image. The output image has the same type\n and range as input image.\n \"\"\"\n img_type = img.dtype\n img = _convert_input_type_range(img)\n if y_only:\n out_img = np.dot(img, [65.481, 128.553, 24.966]) + 16.0\n else:\n out_img = np.matmul(\n img, [[65.481, -37.797, 112.0], [128.553, -74.203, -93.786], [24.966, 112.0, -18.214]]) + [16, 128, 128]\n out_img = _convert_output_type_range(out_img, img_type)\n return out_img\n\n\ndef bgr2ycbcr(img, y_only=False):\n \"\"\"Convert a BGR image to YCbCr image.\n\n The bgr version of rgb2ycbcr.\n It implements the ITU-R BT.601 conversion for standard-definition\n television. See more details in\n https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion.\n\n It differs from a similar function in cv2.cvtColor: `BGR <-> YCrCb`.\n In OpenCV, it implements a JPEG conversion. See more details in\n https://en.wikipedia.org/wiki/YCbCr#JPEG_conversion.\n\n Args:\n img (ndarray): The input image. It accepts:\n 1. np.uint8 type with range [0, 255];\n 2. np.float32 type with range [0, 1].\n y_only (bool): Whether to only return Y channel. Default: False.\n\n Returns:\n ndarray: The converted YCbCr image. The output image has the same type\n and range as input image.\n \"\"\"\n img_type = img.dtype\n img = _convert_input_type_range(img)\n if y_only:\n out_img = np.dot(img, [24.966, 128.553, 65.481]) + 16.0\n else:\n out_img = np.matmul(\n img, [[24.966, 112.0, -18.214], [128.553, -74.203, -93.786], [65.481, -37.797, 112.0]]) + [16, 128, 128]\n out_img = _convert_output_type_range(out_img, img_type)\n return out_img\n\n\ndef ycbcr2rgb(img):\n \"\"\"Convert a YCbCr image to RGB image.\n\n This function produces the same results as Matlab's ycbcr2rgb function.\n It implements the ITU-R BT.601 conversion for standard-definition\n television. See more details in\n https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion.\n\n It differs from a similar function in cv2.cvtColor: `YCrCb <-> RGB`.\n In OpenCV, it implements a JPEG conversion. See more details in\n https://en.wikipedia.org/wiki/YCbCr#JPEG_conversion.\n\n Args:\n img (ndarray): The input image. It accepts:\n 1. np.uint8 type with range [0, 255];\n 2. np.float32 type with range [0, 1].\n\n Returns:\n ndarray: The converted RGB image. The output image has the same type\n and range as input image.\n \"\"\"\n img_type = img.dtype\n img = _convert_input_type_range(img) * 255\n out_img = np.matmul(img, [[0.00456621, 0.00456621, 0.00456621], [0, -0.00153632, 0.00791071],\n [0.00625893, -0.00318811, 0]]) * 255.0 + [-222.921, 135.576, -276.836] # noqa: E126\n out_img = _convert_output_type_range(out_img, img_type)\n return out_img\n\n\ndef ycbcr2bgr(img):\n \"\"\"Convert a YCbCr image to BGR image.\n\n The bgr version of ycbcr2rgb.\n It implements the ITU-R BT.601 conversion for standard-definition\n television. See more details in\n https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion.\n\n It differs from a similar function in cv2.cvtColor: `YCrCb <-> BGR`.\n In OpenCV, it implements a JPEG conversion. See more details in\n https://en.wikipedia.org/wiki/YCbCr#JPEG_conversion.\n\n Args:\n img (ndarray): The input image. It accepts:\n 1. np.uint8 type with range [0, 255];\n 2. np.float32 type with range [0, 1].\n\n Returns:\n ndarray: The converted BGR image. The output image has the same type\n and range as input image.\n \"\"\"\n img_type = img.dtype\n img = _convert_input_type_range(img) * 255\n out_img = np.matmul(img, [[0.00456621, 0.00456621, 0.00456621], [0.00791071, -0.00153632, 0],\n [0, -0.00318811, 0.00625893]]) * 255.0 + [-276.836, 135.576, -222.921] # noqa: E126\n out_img = _convert_output_type_range(out_img, img_type)\n return out_img\n\n\ndef _convert_input_type_range(img):\n \"\"\"Convert the type and range of the input image.\n\n It converts the input image to np.float32 type and range of [0, 1].\n It is mainly used for pre-processing the input image in colorspace\n conversion functions such as rgb2ycbcr and ycbcr2rgb.\n\n Args:\n img (ndarray): The input image. It accepts:\n 1. np.uint8 type with range [0, 255];\n 2. np.float32 type with range [0, 1].\n\n Returns:\n (ndarray): The converted image with type of np.float32 and range of\n [0, 1].\n \"\"\"\n img_type = img.dtype\n img = img.astype(np.float32)\n if img_type == np.float32:\n pass\n elif img_type == np.uint8:\n img /= 255.\n else:\n raise TypeError(f'The img type should be np.float32 or np.uint8, but got {img_type}')\n return img\n\n\ndef _convert_output_type_range(img, dst_type):\n \"\"\"Convert the type and range of the image according to dst_type.\n\n It converts the image to desired type and range. If `dst_type` is np.uint8,\n images will be converted to np.uint8 type with range [0, 255]. If\n `dst_type` is np.float32, it converts the image to np.float32 type with\n range [0, 1].\n It is mainly used for post-processing images in colorspace conversion\n functions such as rgb2ycbcr and ycbcr2rgb.\n\n Args:\n img (ndarray): The image to be converted with np.float32 type and\n range [0, 255].\n dst_type (np.uint8 | np.float32): If dst_type is np.uint8, it\n converts the image to np.uint8 type with range [0, 255]. If\n dst_type is np.float32, it converts the image to np.float32 type\n with range [0, 1].\n\n Returns:\n (ndarray): The converted image with desired type and range.\n \"\"\"\n if dst_type not in (np.uint8, np.float32):\n raise TypeError(f'The dst_type should be np.float32 or np.uint8, but got {dst_type}')\n if dst_type == np.uint8:\n img = img.round()\n else:\n img /= 255.\n return img.astype(dst_type)\n\n\ndef rgb2ycbcr_pt(img, y_only=False):\n \"\"\"Convert RGB images to YCbCr images (PyTorch version).\n\n It implements the ITU-R BT.601 conversion for standard-definition television. See more details in\n https://en.wikipedia.org/wiki/YCbCr#ITU-R_BT.601_conversion.\n\n Args:\n img (Tensor): Images with shape (n, 3, h, w), the range [0, 1], float, RGB format.\n y_only (bool): Whether to only return Y channel. Default: False.\n\n Returns:\n (Tensor): converted images with the shape (n, 3/1, h, w), the range [0, 1], float.\n \"\"\"\n if y_only:\n weight = torch.tensor([[65.481], [128.553], [24.966]]).to(img)\n out_img = torch.matmul(img.permute(0, 2, 3, 1), weight).permute(0, 3, 1, 2) + 16.0\n else:\n weight = torch.tensor([[65.481, -37.797, 112.0], [128.553, -74.203, -93.786], [24.966, 112.0, -18.214]]).to(img)\n bias = torch.tensor([16, 128, 128]).view(1, 3, 1, 1).to(img)\n out_img = torch.matmul(img.permute(0, 2, 3, 1), weight).permute(0, 3, 1, 2) + bias\n\n out_img = out_img / 255.\n return out_img\n"
},
{
"path": "bsr/utils/diffjpeg.py",
"content": "\"\"\"\nModified from https://github.com/mlomnitz/DiffJPEG\n\nFor images not divisible by 8\nhttps://dsp.stackexchange.com/questions/35339/jpeg-dct-padding/35343#35343\n\"\"\"\nimport itertools\nimport numpy as np\nimport torch\nimport torch.nn as nn\nfrom torch.nn import functional as F\n\n# ------------------------ utils ------------------------#\ny_table = np.array(\n [[16, 11, 10, 16, 24, 40, 51, 61], [12, 12, 14, 19, 26, 58, 60, 55], [14, 13, 16, 24, 40, 57, 69, 56],\n [14, 17, 22, 29, 51, 87, 80, 62], [18, 22, 37, 56, 68, 109, 103, 77], [24, 35, 55, 64, 81, 104, 113, 92],\n [49, 64, 78, 87, 103, 121, 120, 101], [72, 92, 95, 98, 112, 100, 103, 99]],\n dtype=np.float32).T\ny_table = nn.Parameter(torch.from_numpy(y_table))\nc_table = np.empty((8, 8), dtype=np.float32)\nc_table.fill(99)\nc_table[:4, :4] = np.array([[17, 18, 24, 47], [18, 21, 26, 66], [24, 26, 56, 99], [47, 66, 99, 99]]).T\nc_table = nn.Parameter(torch.from_numpy(c_table))\n\n\ndef diff_round(x):\n \"\"\" Differentiable rounding function\n \"\"\"\n return torch.round(x) + (x - torch.round(x))**3\n\n\ndef quality_to_factor(quality):\n \"\"\" Calculate factor corresponding to quality\n\n Args:\n quality(float): Quality for jpeg compression.\n\n Returns:\n float: Compression factor.\n \"\"\"\n if quality < 50:\n quality = 5000. / quality\n else:\n quality = 200. - quality * 2\n return quality / 100.\n\n\n# ------------------------ compression ------------------------#\nclass RGB2YCbCrJpeg(nn.Module):\n \"\"\" Converts RGB image to YCbCr\n \"\"\"\n\n def __init__(self):\n super(RGB2YCbCrJpeg, self).__init__()\n matrix = np.array([[0.299, 0.587, 0.114], [-0.168736, -0.331264, 0.5], [0.5, -0.418688, -0.081312]],\n dtype=np.float32).T\n self.shift = nn.Parameter(torch.tensor([0., 128., 128.]))\n self.matrix = nn.Parameter(torch.from_numpy(matrix))\n\n def forward(self, image):\n \"\"\"\n Args:\n image(Tensor): batch x 3 x height x width\n\n Returns:\n Tensor: batch x height x width x 3\n \"\"\"\n image = image.permute(0, 2, 3, 1)\n result = torch.tensordot(image, self.matrix, dims=1) + self.shift\n return result.view(image.shape)\n\n\nclass ChromaSubsampling(nn.Module):\n \"\"\" Chroma subsampling on CbCr channels\n \"\"\"\n\n def __init__(self):\n super(ChromaSubsampling, self).__init__()\n\n def forward(self, image):\n \"\"\"\n Args:\n image(tensor): batch x height x width x 3\n\n Returns:\n y(tensor): batch x height x width\n cb(tensor): batch x height/2 x width/2\n cr(tensor): batch x height/2 x width/2\n \"\"\"\n image_2 = image.permute(0, 3, 1, 2).clone()\n cb = F.avg_pool2d(image_2[:, 1, :, :].unsqueeze(1), kernel_size=2, stride=(2, 2), count_include_pad=False)\n cr = F.avg_pool2d(image_2[:, 2, :, :].unsqueeze(1), kernel_size=2, stride=(2, 2), count_include_pad=False)\n cb = cb.permute(0, 2, 3, 1)\n cr = cr.permute(0, 2, 3, 1)\n return image[:, :, :, 0], cb.squeeze(3), cr.squeeze(3)\n\n\nclass BlockSplitting(nn.Module):\n \"\"\" Splitting image into patches\n \"\"\"\n\n def __init__(self):\n super(BlockSplitting, self).__init__()\n self.k = 8\n\n def forward(self, image):\n \"\"\"\n Args:\n image(tensor): batch x height x width\n\n Returns:\n Tensor: batch x h*w/64 x h x w\n \"\"\"\n height, _ = image.shape[1:3]\n batch_size = image.shape[0]\n image_reshaped = image.view(batch_size, height // self.k, self.k, -1, self.k)\n image_transposed = image_reshaped.permute(0, 1, 3, 2, 4)\n return image_transposed.contiguous().view(batch_size, -1, self.k, self.k)\n\n\nclass DCT8x8(nn.Module):\n \"\"\" Discrete Cosine Transformation\n \"\"\"\n\n def __init__(self):\n super(DCT8x8, self).__init__()\n tensor = np.zeros((8, 8, 8, 8), dtype=np.float32)\n for x, y, u, v in itertools.product(range(8), repeat=4):\n tensor[x, y, u, v] = np.cos((2 * x + 1) * u * np.pi / 16) * np.cos((2 * y + 1) * v * np.pi / 16)\n alpha = np.array([1. / np.sqrt(2)] + [1] * 7)\n self.tensor = nn.Parameter(torch.from_numpy(tensor).float())\n self.scale = nn.Parameter(torch.from_numpy(np.outer(alpha, alpha) * 0.25).float())\n\n def forward(self, image):\n \"\"\"\n Args:\n image(tensor): batch x height x width\n\n Returns:\n Tensor: batch x height x width\n \"\"\"\n image = image - 128\n result = self.scale * torch.tensordot(image, self.tensor, dims=2)\n result.view(image.shape)\n return result\n\n\nclass YQuantize(nn.Module):\n \"\"\" JPEG Quantization for Y channel\n\n Args:\n rounding(function): rounding function to use\n \"\"\"\n\n def __init__(self, rounding):\n super(YQuantize, self).__init__()\n self.rounding = rounding\n self.y_table = y_table\n\n def forward(self, image, factor=1):\n \"\"\"\n Args:\n image(tensor): batch x height x width\n\n Returns:\n Tensor: batch x height x width\n \"\"\"\n if isinstance(factor, (int, float)):\n image = image.float() / (self.y_table * factor)\n else:\n b = factor.size(0)\n table = self.y_table.expand(b, 1, 8, 8) * factor.view(b, 1, 1, 1)\n image = image.float() / table\n image = self.rounding(image)\n return image\n\n\nclass CQuantize(nn.Module):\n \"\"\" JPEG Quantization for CbCr channels\n\n Args:\n rounding(function): rounding function to use\n \"\"\"\n\n def __init__(self, rounding):\n super(CQuantize, self).__init__()\n self.rounding = rounding\n self.c_table = c_table\n\n def forward(self, image, factor=1):\n \"\"\"\n Args:\n image(tensor): batch x height x width\n\n Returns:\n Tensor: batch x height x width\n \"\"\"\n if isinstance(factor, (int, float)):\n image = image.float() / (self.c_table * factor)\n else:\n b = factor.size(0)\n table = self.c_table.expand(b, 1, 8, 8) * factor.view(b, 1, 1, 1)\n image = image.float() / table\n image = self.rounding(image)\n return image\n\n\nclass CompressJpeg(nn.Module):\n \"\"\"Full JPEG compression algorithm\n\n Args:\n rounding(function): rounding function to use\n \"\"\"\n\n def __init__(self, rounding=torch.round):\n super(CompressJpeg, self).__init__()\n self.l1 = nn.Sequential(RGB2YCbCrJpeg(), ChromaSubsampling())\n self.l2 = nn.Sequential(BlockSplitting(), DCT8x8())\n self.c_quantize = CQuantize(rounding=rounding)\n self.y_quantize = YQuantize(rounding=rounding)\n\n def forward(self, image, factor=1):\n \"\"\"\n Args:\n image(tensor): batch x 3 x height x width\n\n Returns:\n dict(tensor): Compressed tensor with batch x h*w/64 x 8 x 8.\n \"\"\"\n y, cb, cr = self.l1(image * 255)\n components = {'y': y, 'cb': cb, 'cr': cr}\n for k in components.keys():\n comp = self.l2(components[k])\n if k in ('cb', 'cr'):\n comp = self.c_quantize(comp, factor=factor)\n else:\n comp = self.y_quantize(comp, factor=factor)\n\n components[k] = comp\n\n return components['y'], components['cb'], components['cr']\n\n\n# ------------------------ decompression ------------------------#\n\n\nclass YDequantize(nn.Module):\n \"\"\"Dequantize Y channel\n \"\"\"\n\n def __init__(self):\n super(YDequantize, self).__init__()\n self.y_table = y_table\n\n def forward(self, image, factor=1):\n \"\"\"\n Args:\n image(tensor): batch x height x width\n\n Returns:\n Tensor: batch x height x width\n \"\"\"\n if isinstance(factor, (int, float)):\n out = image * (self.y_table * factor)\n else:\n b = factor.size(0)\n table = self.y_table.expand(b, 1, 8, 8) * factor.view(b, 1, 1, 1)\n out = image * table\n return out\n\n\nclass CDequantize(nn.Module):\n \"\"\"Dequantize CbCr channel\n \"\"\"\n\n def __init__(self):\n super(CDequantize, self).__init__()\n self.c_table = c_table\n\n def forward(self, image, factor=1):\n \"\"\"\n Args:\n image(tensor): batch x height x width\n\n Returns:\n Tensor: batch x height x width\n \"\"\"\n if isinstance(factor, (int, float)):\n out = image * (self.c_table * factor)\n else:\n b = factor.size(0)\n table = self.c_table.expand(b, 1, 8, 8) * factor.view(b, 1, 1, 1)\n out = image * table\n return out\n\n\nclass iDCT8x8(nn.Module):\n \"\"\"Inverse discrete Cosine Transformation\n \"\"\"\n\n def __init__(self):\n super(iDCT8x8, self).__init__()\n alpha = np.array([1. / np.sqrt(2)] + [1] * 7)\n self.alpha = nn.Parameter(torch.from_numpy(np.outer(alpha, alpha)).float())\n tensor = np.zeros((8, 8, 8, 8), dtype=np.float32)\n for x, y, u, v in itertools.product(range(8), repeat=4):\n tensor[x, y, u, v] = np.cos((2 * u + 1) * x * np.pi / 16) * np.cos((2 * v + 1) * y * np.pi / 16)\n self.tensor = nn.Parameter(torch.from_numpy(tensor).float())\n\n def forward(self, image):\n \"\"\"\n Args:\n image(tensor): batch x height x width\n\n Returns:\n Tensor: batch x height x width\n \"\"\"\n image = image * self.alpha\n result = 0.25 * torch.tensordot(image, self.tensor, dims=2) + 128\n result.view(image.shape)\n return result\n\n\nclass BlockMerging(nn.Module):\n \"\"\"Merge patches into image\n \"\"\"\n\n def __init__(self):\n super(BlockMerging, self).__init__()\n\n def forward(self, patches, height, width):\n \"\"\"\n Args:\n patches(tensor) batch x height*width/64, height x width\n height(int)\n width(int)\n\n Returns:\n Tensor: batch x height x width\n \"\"\"\n k = 8\n batch_size = patches.shape[0]\n image_reshaped = patches.view(batch_size, height // k, width // k, k, k)\n image_transposed = image_reshaped.permute(0, 1, 3, 2, 4)\n return image_transposed.contiguous().view(batch_size, height, width)\n\n\nclass ChromaUpsampling(nn.Module):\n \"\"\"Upsample chroma layers\n \"\"\"\n\n def __init__(self):\n super(ChromaUpsampling, self).__init__()\n\n def forward(self, y, cb, cr):\n \"\"\"\n Args:\n y(tensor): y channel image\n cb(tensor): cb channel\n cr(tensor): cr channel\n\n Returns:\n Tensor: batch x height x width x 3\n \"\"\"\n\n def repeat(x, k=2):\n height, width = x.shape[1:3]\n x = x.unsqueeze(-1)\n x = x.repeat(1, 1, k, k)\n x = x.view(-1, height * k, width * k)\n return x\n\n cb = repeat(cb)\n cr = repeat(cr)\n return torch.cat([y.unsqueeze(3), cb.unsqueeze(3), cr.unsqueeze(3)], dim=3)\n\n\nclass YCbCr2RGBJpeg(nn.Module):\n \"\"\"Converts YCbCr image to RGB JPEG\n \"\"\"\n\n def __init__(self):\n super(YCbCr2RGBJpeg, self).__init__()\n\n matrix = np.array([[1., 0., 1.402], [1, -0.344136, -0.714136], [1, 1.772, 0]], dtype=np.float32).T\n self.shift = nn.Parameter(torch.tensor([0, -128., -128.]))\n self.matrix = nn.Parameter(torch.from_numpy(matrix))\n\n def forward(self, image):\n \"\"\"\n Args:\n image(tensor): batch x height x width x 3\n\n Returns:\n Tensor: batch x 3 x height x width\n \"\"\"\n result = torch.tensordot(image + self.shift, self.matrix, dims=1)\n return result.view(image.shape).permute(0, 3, 1, 2)\n\n\nclass DeCompressJpeg(nn.Module):\n \"\"\"Full JPEG decompression algorithm\n\n Args:\n rounding(function): rounding function to use\n \"\"\"\n\n def __init__(self, rounding=torch.round):\n super(DeCompressJpeg, self).__init__()\n self.c_dequantize = CDequantize()\n self.y_dequantize = YDequantize()\n self.idct = iDCT8x8()\n self.merging = BlockMerging()\n self.chroma = ChromaUpsampling()\n self.colors = YCbCr2RGBJpeg()\n\n def forward(self, y, cb, cr, imgh, imgw, factor=1):\n \"\"\"\n Args:\n compressed(dict(tensor)): batch x h*w/64 x 8 x 8\n imgh(int)\n imgw(int)\n factor(float)\n\n Returns:\n Tensor: batch x 3 x height x width\n \"\"\"\n components = {'y': y, 'cb': cb, 'cr': cr}\n for k in components.keys():\n if k in ('cb', 'cr'):\n comp = self.c_dequantize(components[k], factor=factor)\n height, width = int(imgh / 2), int(imgw / 2)\n else:\n comp = self.y_dequantize(components[k], factor=factor)\n height, width = imgh, imgw\n comp = self.idct(comp)\n components[k] = self.merging(comp, height, width)\n #\n image = self.chroma(components['y'], components['cb'], components['cr'])\n image = self.colors(image)\n\n image = torch.min(255 * torch.ones_like(image), torch.max(torch.zeros_like(image), image))\n return image / 255\n\n\n# ------------------------ main DiffJPEG ------------------------ #\n\n\nclass DiffJPEG(nn.Module):\n \"\"\"This JPEG algorithm result is slightly different from cv2.\n DiffJPEG supports batch processing.\n\n Args:\n differentiable(bool): If True, uses custom differentiable rounding function, if False, uses standard torch.round\n \"\"\"\n\n def __init__(self, differentiable=True):\n super(DiffJPEG, self).__init__()\n if differentiable:\n rounding = diff_round\n else:\n rounding = torch.round\n\n self.compress = CompressJpeg(rounding=rounding)\n self.decompress = DeCompressJpeg(rounding=rounding)\n\n def forward(self, x, quality):\n \"\"\"\n Args:\n x (Tensor): Input image, bchw, rgb, [0, 1]\n quality(float): Quality factor for jpeg compression scheme.\n \"\"\"\n factor = quality\n if isinstance(factor, (int, float)):\n factor = quality_to_factor(factor)\n else:\n for i in range(factor.size(0)):\n factor[i] = quality_to_factor(factor[i])\n h, w = x.size()[-2:]\n h_pad, w_pad = 0, 0\n # why should use 16\n if h % 16 != 0:\n h_pad = 16 - h % 16\n if w % 16 != 0:\n w_pad = 16 - w % 16\n x = F.pad(x, (0, w_pad, 0, h_pad), mode='constant', value=0)\n\n y, cb, cr = self.compress(x, factor=factor)\n recovered = self.decompress(y, cb, cr, (h + h_pad), (w + w_pad), factor=factor)\n recovered = recovered[:, :, 0:h, 0:w]\n return recovered\n\n\nif __name__ == '__main__':\n import cv2\n\n from bsr.utils import img2tensor, tensor2img\n\n img_gt = cv2.imread('test.png') / 255.\n\n # -------------- cv2 -------------- #\n encode_param = [int(cv2.IMWRITE_JPEG_QUALITY), 20]\n _, encimg = cv2.imencode('.jpg', img_gt * 255., encode_param)\n img_lq = np.float32(cv2.imdecode(encimg, 1))\n cv2.imwrite('cv2_JPEG_20.png', img_lq)\n\n # -------------- DiffJPEG -------------- #\n jpeger = DiffJPEG(differentiable=False).cuda()\n img_gt = img2tensor(img_gt)\n img_gt = torch.stack([img_gt, img_gt]).cuda()\n quality = img_gt.new_tensor([20, 40])\n out = jpeger(img_gt, quality=quality)\n\n cv2.imwrite('pt_JPEG_20.png', tensor2img(out[0]))\n cv2.imwrite('pt_JPEG_40.png', tensor2img(out[1]))\n"
},
{
"path": "bsr/utils/dist_util.py",
"content": "# Modified from https://github.com/open-mmlab/mmcv/blob/master/mmcv/runner/dist_utils.py # noqa: E501\nimport functools\nimport os\nimport subprocess\nimport torch\nimport torch.distributed as dist\nimport torch.multiprocessing as mp\n\n\ndef init_dist(launcher, backend='nccl', **kwargs):\n if mp.get_start_method(allow_none=True) is None:\n mp.set_start_method('spawn')\n if launcher == 'pytorch':\n _init_dist_pytorch(backend, **kwargs)\n elif launcher == 'slurm':\n _init_dist_slurm(backend, **kwargs)\n else:\n raise ValueError(f'Invalid launcher type: {launcher}')\n\n\ndef _init_dist_pytorch(backend, **kwargs):\n rank = int(os.environ['RANK'])\n num_gpus = torch.cuda.device_count()\n torch.cuda.set_device(rank % num_gpus)\n dist.init_process_group(backend=backend, **kwargs)\n\n\ndef _init_dist_slurm(backend, port=None):\n \"\"\"Initialize slurm distributed training environment.\n\n If argument ``port`` is not specified, then the master port will be system\n environment variable ``MASTER_PORT``. If ``MASTER_PORT`` is not in system\n environment variable, then a default port ``29500`` will be used.\n\n Args:\n backend (str): Backend of torch.distributed.\n port (int, optional): Master port. Defaults to None.\n \"\"\"\n proc_id = int(os.environ['SLURM_PROCID'])\n ntasks = int(os.environ['SLURM_NTASKS'])\n node_list = os.environ['SLURM_NODELIST']\n num_gpus = torch.cuda.device_count()\n torch.cuda.set_device(proc_id % num_gpus)\n addr = subprocess.getoutput(f'scontrol show hostname {node_list} | head -n1')\n # specify master port\n if port is not None:\n os.environ['MASTER_PORT'] = str(port)\n elif 'MASTER_PORT' in os.environ:\n pass # use MASTER_PORT in the environment variable\n else:\n # 29500 is torch.distributed default port\n os.environ['MASTER_PORT'] = '29500'\n os.environ['MASTER_ADDR'] = addr\n os.environ['WORLD_SIZE'] = str(ntasks)\n os.environ['LOCAL_RANK'] = str(proc_id % num_gpus)\n os.environ['RANK'] = str(proc_id)\n dist.init_process_group(backend=backend)\n\n\ndef get_dist_info():\n if dist.is_available():\n initialized = dist.is_initialized()\n else:\n initialized = False\n if initialized:\n rank = dist.get_rank()\n world_size = dist.get_world_size()\n else:\n rank = 0\n world_size = 1\n return rank, world_size\n\n\ndef master_only(func):\n\n @functools.wraps(func)\n def wrapper(*args, **kwargs):\n rank, _ = get_dist_info()\n if rank == 0:\n return func(*args, **kwargs)\n\n return wrapper\n"
},
{
"path": "bsr/utils/download_util.py",
"content": "import math\nimport os\nimport requests\nfrom torch.hub import download_url_to_file, get_dir\nfrom tqdm import tqdm\nfrom urllib.parse import urlparse\n\nfrom .misc import sizeof_fmt\n\n\ndef download_file_from_google_drive(file_id, save_path):\n \"\"\"Download files from google drive.\n\n Reference: https://stackoverflow.com/questions/25010369/wget-curl-large-file-from-google-drive\n\n Args:\n file_id (str): File id.\n save_path (str): Save path.\n \"\"\"\n\n session = requests.Session()\n URL = 'https://docs.google.com/uc?export=download'\n params = {'id': file_id}\n\n response = session.get(URL, params=params, stream=True)\n token = get_confirm_token(response)\n if token:\n params['confirm'] = token\n response = session.get(URL, params=params, stream=True)\n\n # get file size\n response_file_size = session.get(URL, params=params, stream=True, headers={'Range': 'bytes=0-2'})\n if 'Content-Range' in response_file_size.headers:\n file_size = int(response_file_size.headers['Content-Range'].split('/')[1])\n else:\n file_size = None\n\n save_response_content(response, save_path, file_size)\n\n\ndef get_confirm_token(response):\n for key, value in response.cookies.items():\n if key.startswith('download_warning'):\n return value\n return None\n\n\ndef save_response_content(response, destination, file_size=None, chunk_size=32768):\n if file_size is not None:\n pbar = tqdm(total=math.ceil(file_size / chunk_size), unit='chunk')\n\n readable_file_size = sizeof_fmt(file_size)\n else:\n pbar = None\n\n with open(destination, 'wb') as f:\n downloaded_size = 0\n for chunk in response.iter_content(chunk_size):\n downloaded_size += chunk_size\n if pbar is not None:\n pbar.update(1)\n pbar.set_description(f'Download {sizeof_fmt(downloaded_size)} / {readable_file_size}')\n if chunk: # filter out keep-alive new chunks\n f.write(chunk)\n if pbar is not None:\n pbar.close()\n\n\ndef load_file_from_url(url, model_dir=None, progress=True, file_name=None):\n \"\"\"Load file form http url, will download models if necessary.\n\n Reference: https://github.com/1adrianb/face-alignment/blob/master/face_alignment/utils.py\n\n Args:\n url (str): URL to be downloaded.\n model_dir (str): The path to save the downloaded model. Should be a full path. If None, use pytorch hub_dir.\n Default: None.\n progress (bool): Whether to show the download progress. Default: True.\n file_name (str): The downloaded file name. If None, use the file name in the url. Default: None.\n\n Returns:\n str: The path to the downloaded file.\n \"\"\"\n if model_dir is None: # use the pytorch hub_dir\n hub_dir = get_dir()\n model_dir = os.path.join(hub_dir, 'checkpoints')\n\n os.makedirs(model_dir, exist_ok=True)\n\n parts = urlparse(url)\n filename = os.path.basename(parts.path)\n if file_name is not None:\n filename = file_name\n cached_file = os.path.abspath(os.path.join(model_dir, filename))\n if not os.path.exists(cached_file):\n print(f'Downloading: \"{url}\" to {cached_file}\\n')\n download_url_to_file(url, cached_file, hash_prefix=None, progress=progress)\n return cached_file\n"
},
{
"path": "bsr/utils/file_client.py",
"content": "# Modified from https://github.com/open-mmlab/mmcv/blob/master/mmcv/fileio/file_client.py # noqa: E501\nfrom abc import ABCMeta, abstractmethod\n\n\nclass BaseStorageBackend(metaclass=ABCMeta):\n \"\"\"Abstract class of storage backends.\n\n All backends need to implement two apis: ``get()`` and ``get_text()``.\n ``get()`` reads the file as a byte stream and ``get_text()`` reads the file\n as texts.\n \"\"\"\n\n @abstractmethod\n def get(self, filepath):\n pass\n\n @abstractmethod\n def get_text(self, filepath):\n pass\n\n\nclass MemcachedBackend(BaseStorageBackend):\n \"\"\"Memcached storage backend.\n\n Attributes:\n server_list_cfg (str): Config file for memcached server list.\n client_cfg (str): Config file for memcached client.\n sys_path (str | None): Additional path to be appended to `sys.path`.\n Default: None.\n \"\"\"\n\n def __init__(self, server_list_cfg, client_cfg, sys_path=None):\n if sys_path is not None:\n import sys\n sys.path.append(sys_path)\n try:\n import mc\n except ImportError:\n raise ImportError('Please install memcached to enable MemcachedBackend.')\n\n self.server_list_cfg = server_list_cfg\n self.client_cfg = client_cfg\n self._client = mc.MemcachedClient.GetInstance(self.server_list_cfg, self.client_cfg)\n # mc.pyvector servers as a point which points to a memory cache\n self._mc_buffer = mc.pyvector()\n\n def get(self, filepath):\n filepath = str(filepath)\n import mc\n self._client.Get(filepath, self._mc_buffer)\n value_buf = mc.ConvertBuffer(self._mc_buffer)\n return value_buf\n\n def get_text(self, filepath):\n raise NotImplementedError\n\n\nclass HardDiskBackend(BaseStorageBackend):\n \"\"\"Raw hard disks storage backend.\"\"\"\n\n def get(self, filepath):\n filepath = str(filepath)\n with open(filepath, 'rb') as f:\n value_buf = f.read()\n return value_buf\n\n def get_text(self, filepath):\n filepath = str(filepath)\n with open(filepath, 'r') as f:\n value_buf = f.read()\n return value_buf\n\n\nclass LmdbBackend(BaseStorageBackend):\n \"\"\"Lmdb storage backend.\n\n Args:\n db_paths (str | list[str]): Lmdb database paths.\n client_keys (str | list[str]): Lmdb client keys. Default: 'default'.\n readonly (bool, optional): Lmdb environment parameter. If True,\n disallow any write operations. Default: True.\n lock (bool, optional): Lmdb environment parameter. If False, when\n concurrent access occurs, do not lock the database. Default: False.\n readahead (bool, optional): Lmdb environment parameter. If False,\n disable the OS filesystem readahead mechanism, which may improve\n random read performance when a database is larger than RAM.\n Default: False.\n\n Attributes:\n db_paths (list): Lmdb database path.\n _client (list): A list of several lmdb envs.\n \"\"\"\n\n def __init__(self, db_paths, client_keys='default', readonly=True, lock=False, readahead=False, **kwargs):\n try:\n import lmdb\n except ImportError:\n raise ImportError('Please install lmdb to enable LmdbBackend.')\n\n if isinstance(client_keys, str):\n client_keys = [client_keys]\n\n if isinstance(db_paths, list):\n self.db_paths = [str(v) for v in db_paths]\n elif isinstance(db_paths, str):\n self.db_paths = [str(db_paths)]\n assert len(client_keys) == len(self.db_paths), ('client_keys and db_paths should have the same length, '\n f'but received {len(client_keys)} and {len(self.db_paths)}.')\n\n self._client = {}\n for client, path in zip(client_keys, self.db_paths):\n self._client[client] = lmdb.open(path, readonly=readonly, lock=lock, readahead=readahead, **kwargs)\n\n def get(self, filepath, client_key):\n \"\"\"Get values according to the filepath from one lmdb named client_key.\n\n Args:\n filepath (str | obj:`Path`): Here, filepath is the lmdb key.\n client_key (str): Used for distinguishing different lmdb envs.\n \"\"\"\n filepath = str(filepath)\n assert client_key in self._client, (f'client_key {client_key} is not in lmdb clients.')\n client = self._client[client_key]\n with client.begin(write=False) as txn:\n value_buf = txn.get(filepath.encode('ascii'))\n return value_buf\n\n def get_text(self, filepath):\n raise NotImplementedError\n\n\nclass FileClient(object):\n \"\"\"A general file client to access files in different backend.\n\n The client loads a file or text in a specified backend from its path\n and return it as a binary file. it can also register other backend\n accessor with a given name and backend class.\n\n Attributes:\n backend (str): The storage backend type. Options are \"disk\",\n \"memcached\" and \"lmdb\".\n client (:obj:`BaseStorageBackend`): The backend object.\n \"\"\"\n\n _backends = {\n 'disk': HardDiskBackend,\n 'memcached': MemcachedBackend,\n 'lmdb': LmdbBackend,\n }\n\n def __init__(self, backend='disk', **kwargs):\n if backend not in self._backends:\n raise ValueError(f'Backend {backend} is not supported. Currently supported ones'\n f' are {list(self._backends.keys())}')\n self.backend = backend\n self.client = self._backends[backend](**kwargs)\n\n def get(self, filepath, client_key='default'):\n # client_key is used only for lmdb, where different fileclients have\n # different lmdb environments.\n if self.backend == 'lmdb':\n return self.client.get(filepath, client_key)\n else:\n return self.client.get(filepath)\n\n def get_text(self, filepath):\n return self.client.get_text(filepath)\n"
},
{
"path": "bsr/utils/flow_util.py",
"content": "# Modified from https://github.com/open-mmlab/mmcv/blob/master/mmcv/video/optflow.py # noqa: E501\nimport cv2\nimport numpy as np\nimport os\n\n\ndef flowread(flow_path, quantize=False, concat_axis=0, *args, **kwargs):\n \"\"\"Read an optical flow map.\n\n Args:\n flow_path (ndarray or str): Flow path.\n quantize (bool): whether to read quantized pair, if set to True,\n remaining args will be passed to :func:`dequantize_flow`.\n concat_axis (int): The axis that dx and dy are concatenated,\n can be either 0 or 1. Ignored if quantize is False.\n\n Returns:\n ndarray: Optical flow represented as a (h, w, 2) numpy array\n \"\"\"\n if quantize:\n assert concat_axis in [0, 1]\n cat_flow = cv2.imread(flow_path, cv2.IMREAD_UNCHANGED)\n if cat_flow.ndim != 2:\n raise IOError(f'{flow_path} is not a valid quantized flow file, its dimension is {cat_flow.ndim}.')\n assert cat_flow.shape[concat_axis] % 2 == 0\n dx, dy = np.split(cat_flow, 2, axis=concat_axis)\n flow = dequantize_flow(dx, dy, *args, **kwargs)\n else:\n with open(flow_path, 'rb') as f:\n try:\n header = f.read(4).decode('utf-8')\n except Exception:\n raise IOError(f'Invalid flow file: {flow_path}')\n else:\n if header != 'PIEH':\n raise IOError(f'Invalid flow file: {flow_path}, header does not contain PIEH')\n\n w = np.fromfile(f, np.int32, 1).squeeze()\n h = np.fromfile(f, np.int32, 1).squeeze()\n flow = np.fromfile(f, np.float32, w * h * 2).reshape((h, w, 2))\n\n return flow.astype(np.float32)\n\n\ndef flowwrite(flow, filename, quantize=False, concat_axis=0, *args, **kwargs):\n \"\"\"Write optical flow to file.\n\n If the flow is not quantized, it will be saved as a .flo file losslessly,\n otherwise a jpeg image which is lossy but of much smaller size. (dx and dy\n will be concatenated horizontally into a single image if quantize is True.)\n\n Args:\n flow (ndarray): (h, w, 2) array of optical flow.\n filename (str): Output filepath.\n quantize (bool): Whether to quantize the flow and save it to 2 jpeg\n images. If set to True, remaining args will be passed to\n :func:`quantize_flow`.\n concat_axis (int): The axis that dx and dy are concatenated,\n can be either 0 or 1. Ignored if quantize is False.\n \"\"\"\n if not quantize:\n with open(filename, 'wb') as f:\n f.write('PIEH'.encode('utf-8'))\n np.array([flow.shape[1], flow.shape[0]], dtype=np.int32).tofile(f)\n flow = flow.astype(np.float32)\n flow.tofile(f)\n f.flush()\n else:\n assert concat_axis in [0, 1]\n dx, dy = quantize_flow(flow, *args, **kwargs)\n dxdy = np.concatenate((dx, dy), axis=concat_axis)\n os.makedirs(os.path.dirname(filename), exist_ok=True)\n cv2.imwrite(filename, dxdy)\n\n\ndef quantize_flow(flow, max_val=0.02, norm=True):\n \"\"\"Quantize flow to [0, 255].\n\n After this step, the size of flow will be much smaller, and can be\n dumped as jpeg images.\n\n Args:\n flow (ndarray): (h, w, 2) array of optical flow.\n max_val (float): Maximum value of flow, values beyond\n [-max_val, max_val] will be truncated.\n norm (bool): Whether to divide flow values by image width/height.\n\n Returns:\n tuple[ndarray]: Quantized dx and dy.\n \"\"\"\n h, w, _ = flow.shape\n dx = flow[..., 0]\n dy = flow[..., 1]\n if norm:\n dx = dx / w # avoid inplace operations\n dy = dy / h\n # use 255 levels instead of 256 to make sure 0 is 0 after dequantization.\n flow_comps = [quantize(d, -max_val, max_val, 255, np.uint8) for d in [dx, dy]]\n return tuple(flow_comps)\n\n\ndef dequantize_flow(dx, dy, max_val=0.02, denorm=True):\n \"\"\"Recover from quantized flow.\n\n Args:\n dx (ndarray): Quantized dx.\n dy (ndarray): Quantized dy.\n max_val (float): Maximum value used when quantizing.\n denorm (bool): Whether to multiply flow values with width/height.\n\n Returns:\n ndarray: Dequantized flow.\n \"\"\"\n assert dx.shape == dy.shape\n assert dx.ndim == 2 or (dx.ndim == 3 and dx.shape[-1] == 1)\n\n dx, dy = [dequantize(d, -max_val, max_val, 255) for d in [dx, dy]]\n\n if denorm:\n dx *= dx.shape[1]\n dy *= dx.shape[0]\n flow = np.dstack((dx, dy))\n return flow\n\n\ndef quantize(arr, min_val, max_val, levels, dtype=np.int64):\n \"\"\"Quantize an array of (-inf, inf) to [0, levels-1].\n\n Args:\n arr (ndarray): Input array.\n min_val (scalar): Minimum value to be clipped.\n max_val (scalar): Maximum value to be clipped.\n levels (int): Quantization levels.\n dtype (np.type): The type of the quantized array.\n\n Returns:\n tuple: Quantized array.\n \"\"\"\n if not (isinstance(levels, int) and levels > 1):\n raise ValueError(f'levels must be a positive integer, but got {levels}')\n if min_val >= max_val:\n raise ValueError(f'min_val ({min_val}) must be smaller than max_val ({max_val})')\n\n arr = np.clip(arr, min_val, max_val) - min_val\n quantized_arr = np.minimum(np.floor(levels * arr / (max_val - min_val)).astype(dtype), levels - 1)\n\n return quantized_arr\n\n\ndef dequantize(arr, min_val, max_val, levels, dtype=np.float64):\n \"\"\"Dequantize an array.\n\n Args:\n arr (ndarray): Input array.\n min_val (scalar): Minimum value to be clipped.\n max_val (scalar): Maximum value to be clipped.\n levels (int): Quantization levels.\n dtype (np.type): The type of the dequantized array.\n\n Returns:\n tuple: Dequantized array.\n \"\"\"\n if not (isinstance(levels, int) and levels > 1):\n raise ValueError(f'levels must be a positive integer, but got {levels}')\n if min_val >= max_val:\n raise ValueError(f'min_val ({min_val}) must be smaller than max_val ({max_val})')\n\n dequantized_arr = (arr + 0.5).astype(dtype) * (max_val - min_val) / levels + min_val\n\n return dequantized_arr\n"
},
{
"path": "bsr/utils/img_process_util.py",
"content": "import cv2\nimport numpy as np\nimport torch\nfrom torch.nn import functional as F\n\n\ndef filter2D(img, kernel):\n \"\"\"PyTorch version of cv2.filter2D\n\n Args:\n img (Tensor): (b, c, h, w)\n kernel (Tensor): (b, k, k)\n \"\"\"\n k = kernel.size(-1)\n b, c, h, w = img.size()\n if k % 2 == 1:\n img = F.pad(img, (k // 2, k // 2, k // 2, k // 2), mode='reflect')\n else:\n raise ValueError('Wrong kernel size')\n\n ph, pw = img.size()[-2:]\n\n if kernel.size(0) == 1:\n # apply the same kernel to all batch images\n img = img.view(b * c, 1, ph, pw)\n kernel = kernel.view(1, 1, k, k)\n return F.conv2d(img, kernel, padding=0).view(b, c, h, w)\n else:\n img = img.view(1, b * c, ph, pw)\n kernel = kernel.view(b, 1, k, k).repeat(1, c, 1, 1).view(b * c, 1, k, k)\n return F.conv2d(img, kernel, groups=b * c).view(b, c, h, w)\n\n\ndef usm_sharp(img, weight=0.5, radius=50, threshold=10):\n \"\"\"USM sharpening.\n\n Input image: I; Blurry image: B.\n 1. sharp = I + weight * (I - B)\n 2. Mask = 1 if abs(I - B) > threshold, else: 0\n 3. Blur mask:\n 4. Out = Mask * sharp + (1 - Mask) * I\n\n\n Args:\n img (Numpy array): Input image, HWC, BGR; float32, [0, 1].\n weight (float): Sharp weight. Default: 1.\n radius (float): Kernel size of Gaussian blur. Default: 50.\n threshold (int):\n \"\"\"\n if radius % 2 == 0:\n radius += 1\n blur = cv2.GaussianBlur(img, (radius, radius), 0)\n residual = img - blur\n mask = np.abs(residual) * 255 > threshold\n mask = mask.astype('float32')\n soft_mask = cv2.GaussianBlur(mask, (radius, radius), 0)\n\n sharp = img + weight * residual\n sharp = np.clip(sharp, 0, 1)\n return soft_mask * sharp + (1 - soft_mask) * img\n\n\nclass USMSharp(torch.nn.Module):\n\n def __init__(self, radius=50, sigma=0):\n super(USMSharp, self).__init__()\n if radius % 2 == 0:\n radius += 1\n self.radius = radius\n kernel = cv2.getGaussianKernel(radius, sigma)\n kernel = torch.FloatTensor(np.dot(kernel, kernel.transpose())).unsqueeze_(0)\n self.register_buffer('kernel', kernel)\n\n def forward(self, img, weight=0.5, threshold=10):\n blur = filter2D(img, self.kernel)\n residual = img - blur\n\n mask = torch.abs(residual) * 255 > threshold\n mask = mask.float()\n soft_mask = filter2D(mask, self.kernel)\n sharp = img + weight * residual\n sharp = torch.clip(sharp, 0, 1)\n return soft_mask * sharp + (1 - soft_mask) * img\n"
},
{
"path": "bsr/utils/img_util.py",
"content": "import cv2\nimport math\nimport numpy as np\nimport os\nimport torch\nfrom torchvision.utils import make_grid\n\n\ndef img2tensor(imgs, bgr2rgb=True, float32=True):\n \"\"\"Numpy array to tensor.\n\n Args:\n imgs (list[ndarray] | ndarray): Input images.\n bgr2rgb (bool): Whether to change bgr to rgb.\n float32 (bool): Whether to change to float32.\n\n Returns:\n list[tensor] | tensor: Tensor images. If returned results only have\n one element, just return tensor.\n \"\"\"\n\n def _totensor(img, bgr2rgb, float32):\n if img.shape[2] == 3 and bgr2rgb:\n if img.dtype == 'float64':\n img = img.astype('float32')\n img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n img = torch.from_numpy(img.transpose(2, 0, 1))\n if float32:\n img = img.float()\n return img\n\n if isinstance(imgs, list):\n return [_totensor(img, bgr2rgb, float32) for img in imgs]\n else:\n return _totensor(imgs, bgr2rgb, float32)\n\n\ndef tensor2img(tensor, rgb2bgr=True, out_type=np.uint8, min_max=(0, 1)):\n \"\"\"Convert torch Tensors into image numpy arrays.\n\n After clamping to [min, max], values will be normalized to [0, 1].\n\n Args:\n tensor (Tensor or list[Tensor]): Accept shapes:\n 1) 4D mini-batch Tensor of shape (B x 3/1 x H x W);\n 2) 3D Tensor of shape (3/1 x H x W);\n 3) 2D Tensor of shape (H x W).\n Tensor channel should be in RGB order.\n rgb2bgr (bool): Whether to change rgb to bgr.\n out_type (numpy type): output types. If ``np.uint8``, transform outputs\n to uint8 type with range [0, 255]; otherwise, float type with\n range [0, 1]. Default: ``np.uint8``.\n min_max (tuple[int]): min and max values for clamp.\n\n Returns:\n (Tensor or list): 3D ndarray of shape (H x W x C) OR 2D ndarray of\n shape (H x W). The channel order is BGR.\n \"\"\"\n if not (torch.is_tensor(tensor) or (isinstance(tensor, list) and all(torch.is_tensor(t) for t in tensor))):\n raise TypeError(f'tensor or list of tensors expected, got {type(tensor)}')\n\n if torch.is_tensor(tensor):\n tensor = [tensor]\n result = []\n for _tensor in tensor:\n _tensor = _tensor.squeeze(0).float().detach().cpu().clamp_(*min_max)\n _tensor = (_tensor - min_max[0]) / (min_max[1] - min_max[0])\n\n n_dim = _tensor.dim()\n if n_dim == 4:\n img_np = make_grid(_tensor, nrow=int(math.sqrt(_tensor.size(0))), normalize=False).numpy()\n img_np = img_np.transpose(1, 2, 0)\n if rgb2bgr:\n img_np = cv2.cvtColor(img_np, cv2.COLOR_RGB2BGR)\n elif n_dim == 3:\n img_np = _tensor.numpy()\n img_np = img_np.transpose(1, 2, 0)\n if img_np.shape[2] == 1: # gray image\n img_np = np.squeeze(img_np, axis=2)\n else:\n if rgb2bgr:\n img_np = cv2.cvtColor(img_np, cv2.COLOR_RGB2BGR)\n elif n_dim == 2:\n img_np = _tensor.numpy()\n else:\n raise TypeError(f'Only support 4D, 3D or 2D tensor. But received with dimension: {n_dim}')\n if out_type == np.uint8:\n # Unlike MATLAB, numpy.unit8() WILL NOT round by default.\n img_np = (img_np * 255.0).round()\n img_np = img_np.astype(out_type)\n result.append(img_np)\n if len(result) == 1:\n result = result[0]\n return result\n\n\ndef tensor2img_fast(tensor, rgb2bgr=True, min_max=(0, 1)):\n \"\"\"This implementation is slightly faster than tensor2img.\n It now only supports torch tensor with shape (1, c, h, w).\n\n Args:\n tensor (Tensor): Now only support torch tensor with (1, c, h, w).\n rgb2bgr (bool): Whether to change rgb to bgr. Default: True.\n min_max (tuple[int]): min and max values for clamp.\n \"\"\"\n output = tensor.squeeze(0).detach().clamp_(*min_max).permute(1, 2, 0)\n output = (output - min_max[0]) / (min_max[1] - min_max[0]) * 255\n output = output.type(torch.uint8).cpu().numpy()\n if rgb2bgr:\n output = cv2.cvtColor(output, cv2.COLOR_RGB2BGR)\n return output\n\n\ndef imfrombytes(content, flag='color', float32=False):\n \"\"\"Read an image from bytes.\n\n Args:\n content (bytes): Image bytes got from files or other streams.\n flag (str): Flags specifying the color type of a loaded image,\n candidates are `color`, `grayscale` and `unchanged`.\n float32 (bool): Whether to change to float32., If True, will also norm\n to [0, 1]. Default: False.\n\n Returns:\n ndarray: Loaded image array.\n \"\"\"\n img_np = np.frombuffer(content, np.uint8)\n imread_flags = {'color': cv2.IMREAD_COLOR, 'grayscale': cv2.IMREAD_GRAYSCALE, 'unchanged': cv2.IMREAD_UNCHANGED}\n img = cv2.imdecode(img_np, imread_flags[flag])\n if float32:\n img = img.astype(np.float32) / 255.\n return img\n\n\ndef imwrite(img, file_path, params=None, auto_mkdir=True):\n \"\"\"Write image to file.\n\n Args:\n img (ndarray): Image array to be written.\n file_path (str): Image file path.\n params (None or list): Same as opencv's :func:`imwrite` interface.\n auto_mkdir (bool): If the parent folder of `file_path` does not exist,\n whether to create it automatically.\n\n Returns:\n bool: Successful or not.\n \"\"\"\n if auto_mkdir:\n dir_name = os.path.abspath(os.path.dirname(file_path))\n os.makedirs(dir_name, exist_ok=True)\n ok = cv2.imwrite(file_path, img, params)\n if not ok:\n raise IOError('Failed in writing images.')\n\n\ndef crop_border(imgs, crop_border):\n \"\"\"Crop borders of images.\n\n Args:\n imgs (list[ndarray] | ndarray): Images with shape (h, w, c).\n crop_border (int): Crop border for each end of height and weight.\n\n Returns:\n list[ndarray]: Cropped images.\n \"\"\"\n if crop_border == 0:\n return imgs\n else:\n if isinstance(imgs, list):\n return [v[crop_border:-crop_border, crop_border:-crop_border, ...] for v in imgs]\n else:\n return imgs[crop_border:-crop_border, crop_border:-crop_border, ...]\n"
},
{
"path": "bsr/utils/lmdb_util.py",
"content": "import cv2\nimport lmdb\nimport sys\nfrom multiprocessing import Pool\nfrom os import path as osp\nfrom tqdm import tqdm\n\n\ndef make_lmdb_from_imgs(data_path,\n lmdb_path,\n img_path_list,\n keys,\n batch=5000,\n compress_level=1,\n multiprocessing_read=False,\n n_thread=40,\n map_size=None):\n \"\"\"Make lmdb from images.\n\n Contents of lmdb. The file structure is:\n\n ::\n\n example.lmdb\n āāā data.mdb\n āāā lock.mdb\n āāā meta_info.txt\n\n The data.mdb and lock.mdb are standard lmdb files and you can refer to\n https://lmdb.readthedocs.io/en/release/ for more details.\n\n The meta_info.txt is a specified txt file to record the meta information\n of our datasets. It will be automatically created when preparing\n datasets by our provided dataset tools.\n Each line in the txt file records 1)image name (with extension),\n 2)image shape, and 3)compression level, separated by a white space.\n\n For example, the meta information could be:\n `000_00000000.png (720,1280,3) 1`, which means:\n 1) image name (with extension): 000_00000000.png;\n 2) image shape: (720,1280,3);\n 3) compression level: 1\n\n We use the image name without extension as the lmdb key.\n\n If `multiprocessing_read` is True, it will read all the images to memory\n using multiprocessing. Thus, your server needs to have enough memory.\n\n Args:\n data_path (str): Data path for reading images.\n lmdb_path (str): Lmdb save path.\n img_path_list (str): Image path list.\n keys (str): Used for lmdb keys.\n batch (int): After processing batch images, lmdb commits.\n Default: 5000.\n compress_level (int): Compress level when encoding images. Default: 1.\n multiprocessing_read (bool): Whether use multiprocessing to read all\n the images to memory. Default: False.\n n_thread (int): For multiprocessing.\n map_size (int | None): Map size for lmdb env. If None, use the\n estimated size from images. Default: None\n \"\"\"\n\n assert len(img_path_list) == len(keys), ('img_path_list and keys should have the same length, '\n f'but got {len(img_path_list)} and {len(keys)}')\n print(f'Create lmdb for {data_path}, save to {lmdb_path}...')\n print(f'Totoal images: {len(img_path_list)}')\n if not lmdb_path.endswith('.lmdb'):\n raise ValueError(\"lmdb_path must end with '.lmdb'.\")\n if osp.exists(lmdb_path):\n print(f'Folder {lmdb_path} already exists. Exit.')\n sys.exit(1)\n\n if multiprocessing_read:\n # read all the images to memory (multiprocessing)\n dataset = {} # use dict to keep the order for multiprocessing\n shapes = {}\n print(f'Read images with multiprocessing, #thread: {n_thread} ...')\n pbar = tqdm(total=len(img_path_list), unit='image')\n\n def callback(arg):\n \"\"\"get the image data and update pbar.\"\"\"\n key, dataset[key], shapes[key] = arg\n pbar.update(1)\n pbar.set_description(f'Read {key}')\n\n pool = Pool(n_thread)\n for path, key in zip(img_path_list, keys):\n pool.apply_async(read_img_worker, args=(osp.join(data_path, path), key, compress_level), callback=callback)\n pool.close()\n pool.join()\n pbar.close()\n print(f'Finish reading {len(img_path_list)} images.')\n\n # create lmdb environment\n if map_size is None:\n # obtain data size for one image\n img = cv2.imread(osp.join(data_path, img_path_list[0]), cv2.IMREAD_UNCHANGED)\n _, img_byte = cv2.imencode('.png', img, [cv2.IMWRITE_PNG_COMPRESSION, compress_level])\n data_size_per_img = img_byte.nbytes\n print('Data size per image is: ', data_size_per_img)\n data_size = data_size_per_img * len(img_path_list)\n map_size = data_size * 10\n\n env = lmdb.open(lmdb_path, map_size=map_size)\n\n # write data to lmdb\n pbar = tqdm(total=len(img_path_list), unit='chunk')\n txn = env.begin(write=True)\n txt_file = open(osp.join(lmdb_path, 'meta_info.txt'), 'w')\n for idx, (path, key) in enumerate(zip(img_path_list, keys)):\n pbar.update(1)\n pbar.set_description(f'Write {key}')\n key_byte = key.encode('ascii')\n if multiprocessing_read:\n img_byte = dataset[key]\n h, w, c = shapes[key]\n else:\n _, img_byte, img_shape = read_img_worker(osp.join(data_path, path), key, compress_level)\n h, w, c = img_shape\n\n txn.put(key_byte, img_byte)\n # write meta information\n txt_file.write(f'{key}.png ({h},{w},{c}) {compress_level}\\n')\n if idx % batch == 0:\n txn.commit()\n txn = env.begin(write=True)\n pbar.close()\n txn.commit()\n env.close()\n txt_file.close()\n print('\\nFinish writing lmdb.')\n\n\ndef read_img_worker(path, key, compress_level):\n \"\"\"Read image worker.\n\n Args:\n path (str): Image path.\n key (str): Image key.\n compress_level (int): Compress level when encoding images.\n\n Returns:\n str: Image key.\n byte: Image byte.\n tuple[int]: Image shape.\n \"\"\"\n\n img = cv2.imread(path, cv2.IMREAD_UNCHANGED)\n if img.ndim == 2:\n h, w = img.shape\n c = 1\n else:\n h, w, c = img.shape\n _, img_byte = cv2.imencode('.png', img, [cv2.IMWRITE_PNG_COMPRESSION, compress_level])\n return (key, img_byte, (h, w, c))\n\n\nclass LmdbMaker():\n \"\"\"LMDB Maker.\n\n Args:\n lmdb_path (str): Lmdb save path.\n map_size (int): Map size for lmdb env. Default: 1024 ** 4, 1TB.\n batch (int): After processing batch images, lmdb commits.\n Default: 5000.\n compress_level (int): Compress level when encoding images. Default: 1.\n \"\"\"\n\n def __init__(self, lmdb_path, map_size=1024**4, batch=5000, compress_level=1):\n if not lmdb_path.endswith('.lmdb'):\n raise ValueError(\"lmdb_path must end with '.lmdb'.\")\n if osp.exists(lmdb_path):\n print(f'Folder {lmdb_path} already exists. Exit.')\n sys.exit(1)\n\n self.lmdb_path = lmdb_path\n self.batch = batch\n self.compress_level = compress_level\n self.env = lmdb.open(lmdb_path, map_size=map_size)\n self.txn = self.env.begin(write=True)\n self.txt_file = open(osp.join(lmdb_path, 'meta_info.txt'), 'w')\n self.counter = 0\n\n def put(self, img_byte, key, img_shape):\n self.counter += 1\n key_byte = key.encode('ascii')\n self.txn.put(key_byte, img_byte)\n # write meta information\n h, w, c = img_shape\n self.txt_file.write(f'{key}.png ({h},{w},{c}) {self.compress_level}\\n')\n if self.counter % self.batch == 0:\n self.txn.commit()\n self.txn = self.env.begin(write=True)\n\n def close(self):\n self.txn.commit()\n self.env.close()\n self.txt_file.close()\n"
},
{
"path": "bsr/utils/logger.py",
"content": "import datetime\nimport logging\nimport time\n\nfrom .dist_util import get_dist_info, master_only\n\ninitialized_logger = {}\n\n\nclass AvgTimer():\n\n def __init__(self, window=200):\n self.window = window # average window\n self.current_time = 0\n self.total_time = 0\n self.count = 0\n self.avg_time = 0\n self.start()\n\n def start(self):\n self.start_time = self.tic = time.time()\n\n def record(self):\n self.count += 1\n self.toc = time.time()\n self.current_time = self.toc - self.tic\n self.total_time += self.current_time\n # calculate average time\n self.avg_time = self.total_time / self.count\n\n # reset\n if self.count > self.window:\n self.count = 0\n self.total_time = 0\n\n self.tic = time.time()\n\n def get_current_time(self):\n return self.current_time\n\n def get_avg_time(self):\n return self.avg_time\n\n\nclass MessageLogger():\n \"\"\"Message logger for printing.\n\n Args:\n opt (dict): Config. It contains the following keys:\n name (str): Exp name.\n logger (dict): Contains 'print_freq' (str) for logger interval.\n train (dict): Contains 'total_iter' (int) for total iters.\n use_tb_logger (bool): Use tensorboard logger.\n start_iter (int): Start iter. Default: 1.\n tb_logger (obj:`tb_logger`): Tensorboard logger. Defaultļ¼ None.\n \"\"\"\n\n def __init__(self, opt, start_iter=1, tb_logger=None):\n self.exp_name = opt['name']\n self.interval = opt['logger']['print_freq']\n self.start_iter = start_iter\n self.max_iters = opt['train']['total_iter']\n self.use_tb_logger = opt['logger']['use_tb_logger']\n self.tb_logger = tb_logger\n self.start_time = time.time()\n self.logger = get_root_logger()\n\n def reset_start_time(self):\n self.start_time = time.time()\n\n @master_only\n def __call__(self, log_vars):\n \"\"\"Format logging message.\n\n Args:\n log_vars (dict): It contains the following keys:\n epoch (int): Epoch number.\n iter (int): Current iter.\n lrs (list): List for learning rates.\n\n time (float): Iter time.\n data_time (float): Data time for each iter.\n \"\"\"\n # epoch, iter, learning rates\n epoch = log_vars.pop('epoch')\n current_iter = log_vars.pop('iter')\n lrs = log_vars.pop('lrs')\n\n message = (f'[{self.exp_name[:5]}..][epoch:{epoch:3d}, iter:{current_iter:8,d}, lr:(')\n for v in lrs:\n message += f'{v:.3e},'\n message += ')] '\n\n # time and estimated time\n if 'time' in log_vars.keys():\n iter_time = log_vars.pop('time')\n data_time = log_vars.pop('data_time')\n\n total_time = time.time() - self.start_time\n time_sec_avg = total_time / (current_iter - self.start_iter + 1)\n eta_sec = time_sec_avg * (self.max_iters - current_iter - 1)\n eta_str = str(datetime.timedelta(seconds=int(eta_sec)))\n message += f'[eta: {eta_str}, '\n message += f'time (data): {iter_time:.3f} ({data_time:.3f})] '\n\n # other items, especially losses\n for k, v in log_vars.items():\n message += f'{k}: {v:.4e} '\n # tensorboard logger\n if self.use_tb_logger and 'debug' not in self.exp_name:\n if k.startswith('l_'):\n self.tb_logger.add_scalar(f'losses/{k}', v, current_iter)\n else:\n self.tb_logger.add_scalar(k, v, current_iter)\n self.logger.info(message)\n\n\n@master_only\ndef init_tb_logger(log_dir):\n from torch.utils.tensorboard import SummaryWriter\n tb_logger = SummaryWriter(log_dir=log_dir)\n return tb_logger\n\n\n@master_only\ndef init_wandb_logger(opt):\n \"\"\"We now only use wandb to sync tensorboard log.\"\"\"\n import wandb\n logger = get_root_logger()\n\n project = opt['logger']['wandb']['project']\n resume_id = opt['logger']['wandb'].get('resume_id')\n if resume_id:\n wandb_id = resume_id\n resume = 'allow'\n logger.warning(f'Resume wandb logger with id={wandb_id}.')\n else:\n wandb_id = wandb.util.generate_id()\n resume = 'never'\n\n wandb.init(id=wandb_id, resume=resume, name=opt['name'], config=opt, project=project, sync_tensorboard=True)\n\n logger.info(f'Use wandb logger with id={wandb_id}; project={project}.')\n\n\ndef get_root_logger(logger_name='basicsr', log_level=logging.INFO, log_file=None):\n \"\"\"Get the root logger.\n\n The logger will be initialized if it has not been initialized. By default a\n StreamHandler will be added. If `log_file` is specified, a FileHandler will\n also be added.\n\n Args:\n logger_name (str): root logger name. Default: 'basicsr'.\n log_file (str | None): The log filename. If specified, a FileHandler\n will be added to the root logger.\n log_level (int): The root logger level. Note that only the process of\n rank 0 is affected, while other processes will set the level to\n \"Error\" and be silent most of the time.\n\n Returns:\n logging.Logger: The root logger.\n \"\"\"\n logger = logging.getLogger(logger_name)\n # if the logger has been initialized, just return it\n if logger_name in initialized_logger:\n return logger\n\n format_str = '%(asctime)s %(levelname)s: %(message)s'\n stream_handler = logging.StreamHandler()\n stream_handler.setFormatter(logging.Formatter(format_str))\n logger.addHandler(stream_handler)\n logger.propagate = False\n rank, _ = get_dist_info()\n if rank != 0:\n logger.setLevel('ERROR')\n elif log_file is not None:\n logger.setLevel(log_level)\n # add file handler\n file_handler = logging.FileHandler(log_file, 'w')\n file_handler.setFormatter(logging.Formatter(format_str))\n file_handler.setLevel(log_level)\n logger.addHandler(file_handler)\n initialized_logger[logger_name] = True\n return logger\n\n\ndef get_env_info():\n \"\"\"Get environment information.\n\n Currently, only log the software version.\n \"\"\"\n import torch\n import torchvision\n\n from basicsr.version import __version__\n msg = r\"\"\"\n ____ _ _____ ____\n / __ ) ____ _ _____ (_)_____/ ___/ / __ \\\n / __ |/ __ `// ___// // ___/\\__ \\ / /_/ /\n / /_/ // /_/ /(__ )/ // /__ ___/ // _, _/\n /_____/ \\__,_//____//_/ \\___//____//_/ |_|\n ______ __ __ __ __\n / ____/____ ____ ____/ / / / __ __ _____ / /__ / /\n / / __ / __ \\ / __ \\ / __ / / / / / / // ___// //_/ / /\n / /_/ // /_/ // /_/ // /_/ / / /___/ /_/ // /__ / /< /_/\n \\____/ \\____/ \\____/ \\____/ /_____/\\____/ \\___//_/|_| (_)\n \"\"\"\n msg += ('\\nVersion Information: '\n f'\\n\\tBasicSR: {__version__}'\n f'\\n\\tPyTorch: {torch.__version__}'\n f'\\n\\tTorchVision: {torchvision.__version__}')\n return msg\n"
},
{
"path": "bsr/utils/matlab_functions.py",
"content": "import math\nimport numpy as np\nimport torch\n\n\ndef cubic(x):\n \"\"\"cubic function used for calculate_weights_indices.\"\"\"\n absx = torch.abs(x)\n absx2 = absx**2\n absx3 = absx**3\n return (1.5 * absx3 - 2.5 * absx2 + 1) * (\n (absx <= 1).type_as(absx)) + (-0.5 * absx3 + 2.5 * absx2 - 4 * absx + 2) * (((absx > 1) *\n (absx <= 2)).type_as(absx))\n\n\ndef calculate_weights_indices(in_length, out_length, scale, kernel, kernel_width, antialiasing):\n \"\"\"Calculate weights and indices, used for imresize function.\n\n Args:\n in_length (int): Input length.\n out_length (int): Output length.\n scale (float): Scale factor.\n kernel_width (int): Kernel width.\n antialisaing (bool): Whether to apply anti-aliasing when downsampling.\n \"\"\"\n\n if (scale < 1) and antialiasing:\n # Use a modified kernel (larger kernel width) to simultaneously\n # interpolate and antialias\n kernel_width = kernel_width / scale\n\n # Output-space coordinates\n x = torch.linspace(1, out_length, out_length)\n\n # Input-space coordinates. Calculate the inverse mapping such that 0.5\n # in output space maps to 0.5 in input space, and 0.5 + scale in output\n # space maps to 1.5 in input space.\n u = x / scale + 0.5 * (1 - 1 / scale)\n\n # What is the left-most pixel that can be involved in the computation?\n left = torch.floor(u - kernel_width / 2)\n\n # What is the maximum number of pixels that can be involved in the\n # computation? Note: it's OK to use an extra pixel here; if the\n # corresponding weights are all zero, it will be eliminated at the end\n # of this function.\n p = math.ceil(kernel_width) + 2\n\n # The indices of the input pixels involved in computing the k-th output\n # pixel are in row k of the indices matrix.\n indices = left.view(out_length, 1).expand(out_length, p) + torch.linspace(0, p - 1, p).view(1, p).expand(\n out_length, p)\n\n # The weights used to compute the k-th output pixel are in row k of the\n # weights matrix.\n distance_to_center = u.view(out_length, 1).expand(out_length, p) - indices\n\n # apply cubic kernel\n if (scale < 1) and antialiasing:\n weights = scale * cubic(distance_to_center * scale)\n else:\n weights = cubic(distance_to_center)\n\n # Normalize the weights matrix so that each row sums to 1.\n weights_sum = torch.sum(weights, 1).view(out_length, 1)\n weights = weights / weights_sum.expand(out_length, p)\n\n # If a column in weights is all zero, get rid of it. only consider the\n # first and last column.\n weights_zero_tmp = torch.sum((weights == 0), 0)\n if not math.isclose(weights_zero_tmp[0], 0, rel_tol=1e-6):\n indices = indices.narrow(1, 1, p - 2)\n weights = weights.narrow(1, 1, p - 2)\n if not math.isclose(weights_zero_tmp[-1], 0, rel_tol=1e-6):\n indices = indices.narrow(1, 0, p - 2)\n weights = weights.narrow(1, 0, p - 2)\n weights = weights.contiguous()\n indices = indices.contiguous()\n sym_len_s = -indices.min() + 1\n sym_len_e = indices.max() - in_length\n indices = indices + sym_len_s - 1\n return weights, indices, int(sym_len_s), int(sym_len_e)\n\n\n@torch.no_grad()\ndef imresize(img, scale, antialiasing=True):\n \"\"\"imresize function same as MATLAB.\n\n It now only supports bicubic.\n The same scale applies for both height and width.\n\n Args:\n img (Tensor | Numpy array):\n Tensor: Input image with shape (c, h, w), [0, 1] range.\n Numpy: Input image with shape (h, w, c), [0, 1] range.\n scale (float): Scale factor. The same scale applies for both height\n and width.\n antialisaing (bool): Whether to apply anti-aliasing when downsampling.\n Default: True.\n\n Returns:\n Tensor: Output image with shape (c, h, w), [0, 1] range, w/o round.\n \"\"\"\n squeeze_flag = False\n if type(img).__module__ == np.__name__: # numpy type\n numpy_type = True\n if img.ndim == 2:\n img = img[:, :, None]\n squeeze_flag = True\n img = torch.from_numpy(img.transpose(2, 0, 1)).float()\n else:\n numpy_type = False\n if img.ndim == 2:\n img = img.unsqueeze(0)\n squeeze_flag = True\n\n in_c, in_h, in_w = img.size()\n out_h, out_w = math.ceil(in_h * scale), math.ceil(in_w * scale)\n kernel_width = 4\n kernel = 'cubic'\n\n # get weights and indices\n weights_h, indices_h, sym_len_hs, sym_len_he = calculate_weights_indices(in_h, out_h, scale, kernel, kernel_width,\n antialiasing)\n weights_w, indices_w, sym_len_ws, sym_len_we = calculate_weights_indices(in_w, out_w, scale, kernel, kernel_width,\n antialiasing)\n # process H dimension\n # symmetric copying\n img_aug = torch.FloatTensor(in_c, in_h + sym_len_hs + sym_len_he, in_w)\n img_aug.narrow(1, sym_len_hs, in_h).copy_(img)\n\n sym_patch = img[:, :sym_len_hs, :]\n inv_idx = torch.arange(sym_patch.size(1) - 1, -1, -1).long()\n sym_patch_inv = sym_patch.index_select(1, inv_idx)\n img_aug.narrow(1, 0, sym_len_hs).copy_(sym_patch_inv)\n\n sym_patch = img[:, -sym_len_he:, :]\n inv_idx = torch.arange(sym_patch.size(1) - 1, -1, -1).long()\n sym_patch_inv = sym_patch.index_select(1, inv_idx)\n img_aug.narrow(1, sym_len_hs + in_h, sym_len_he).copy_(sym_patch_inv)\n\n out_1 = torch.FloatTensor(in_c, out_h, in_w)\n kernel_width = weights_h.size(1)\n for i in range(out_h):\n idx = int(indices_h[i][0])\n for j in range(in_c):\n out_1[j, i, :] = img_aug[j, idx:idx + kernel_width, :].transpose(0, 1).mv(weights_h[i])\n\n # process W dimension\n # symmetric copying\n out_1_aug = torch.FloatTensor(in_c, out_h, in_w + sym_len_ws + sym_len_we)\n out_1_aug.narrow(2, sym_len_ws, in_w).copy_(out_1)\n\n sym_patch = out_1[:, :, :sym_len_ws]\n inv_idx = torch.arange(sym_patch.size(2) - 1, -1, -1).long()\n sym_patch_inv = sym_patch.index_select(2, inv_idx)\n out_1_aug.narrow(2, 0, sym_len_ws).copy_(sym_patch_inv)\n\n sym_patch = out_1[:, :, -sym_len_we:]\n inv_idx = torch.arange(sym_patch.size(2) - 1, -1, -1).long()\n sym_patch_inv = sym_patch.index_select(2, inv_idx)\n out_1_aug.narrow(2, sym_len_ws + in_w, sym_len_we).copy_(sym_patch_inv)\n\n out_2 = torch.FloatTensor(in_c, out_h, out_w)\n kernel_width = weights_w.size(1)\n for i in range(out_w):\n idx = int(indices_w[i][0])\n for j in range(in_c):\n out_2[j, :, i] = out_1_aug[j, :, idx:idx + kernel_width].mv(weights_w[i])\n\n if squeeze_flag:\n out_2 = out_2.squeeze(0)\n if numpy_type:\n out_2 = out_2.numpy()\n if not squeeze_flag:\n out_2 = out_2.transpose(1, 2, 0)\n\n return out_2\n"
},
{
"path": "bsr/utils/misc.py",
"content": "import numpy as np\nimport os\nimport random\nimport time\nimport torch\nfrom os import path as osp\n\nfrom .dist_util import master_only\n\n\ndef set_random_seed(seed):\n \"\"\"Set random seeds.\"\"\"\n random.seed(seed)\n np.random.seed(seed)\n torch.manual_seed(seed)\n torch.cuda.manual_seed(seed)\n torch.cuda.manual_seed_all(seed)\n\n\ndef get_time_str():\n return time.strftime('%Y%m%d_%H%M%S', time.localtime())\n\n\ndef mkdir_and_rename(path):\n \"\"\"mkdirs. If path exists, rename it with timestamp and create a new one.\n\n Args:\n path (str): Folder path.\n \"\"\"\n if osp.exists(path):\n new_name = path + '_archived_' + get_time_str()\n print(f'Path already exists. Rename it to {new_name}', flush=True)\n os.rename(path, new_name)\n os.makedirs(path, exist_ok=True)\n\n\n@master_only\ndef make_exp_dirs(opt):\n \"\"\"Make dirs for experiments.\"\"\"\n path_opt = opt['path'].copy()\n if opt['is_train']:\n mkdir_and_rename(path_opt.pop('experiments_root'))\n else:\n mkdir_and_rename(path_opt.pop('results_root'))\n for key, path in path_opt.items():\n if ('strict_load' in key) or ('pretrain_network' in key) or ('resume' in key) or ('param_key' in key):\n continue\n else:\n os.makedirs(path, exist_ok=True)\n\n\ndef scandir(dir_path, suffix=None, recursive=False, full_path=False):\n \"\"\"Scan a directory to find the interested files.\n\n Args:\n dir_path (str): Path of the directory.\n suffix (str | tuple(str), optional): File suffix that we are\n interested in. Default: None.\n recursive (bool, optional): If set to True, recursively scan the\n directory. Default: False.\n full_path (bool, optional): If set to True, include the dir_path.\n Default: False.\n\n Returns:\n A generator for all the interested files with relative paths.\n \"\"\"\n\n if (suffix is not None) and not isinstance(suffix, (str, tuple)):\n raise TypeError('\"suffix\" must be a string or tuple of strings')\n\n root = dir_path\n\n def _scandir(dir_path, suffix, recursive):\n for entry in os.scandir(dir_path):\n if not entry.name.startswith('.') and entry.is_file():\n if full_path:\n return_path = entry.path\n else:\n return_path = osp.relpath(entry.path, root)\n\n if suffix is None:\n yield return_path\n elif return_path.endswith(suffix):\n yield return_path\n else:\n if recursive:\n yield from _scandir(entry.path, suffix=suffix, recursive=recursive)\n else:\n continue\n\n return _scandir(dir_path, suffix=suffix, recursive=recursive)\n\n\ndef check_resume(opt, resume_iter):\n \"\"\"Check resume states and pretrain_network paths.\n\n Args:\n opt (dict): Options.\n resume_iter (int): Resume iteration.\n \"\"\"\n if opt['path']['resume_state']:\n # get all the networks\n networks = [key for key in opt.keys() if key.startswith('network_')]\n flag_pretrain = False\n for network in networks:\n if opt['path'].get(f'pretrain_{network}') is not None:\n flag_pretrain = True\n if flag_pretrain:\n print('pretrain_network path will be ignored during resuming.')\n # set pretrained model paths\n for network in networks:\n name = f'pretrain_{network}'\n basename = network.replace('network_', '')\n if opt['path'].get('ignore_resume_networks') is None or (network\n not in opt['path']['ignore_resume_networks']):\n opt['path'][name] = osp.join(opt['path']['models'], f'net_{basename}_{resume_iter}.pth')\n print(f\"Set {name} to {opt['path'][name]}\")\n\n # change param_key to params in resume\n param_keys = [key for key in opt['path'].keys() if key.startswith('param_key')]\n for param_key in param_keys:\n if opt['path'][param_key] == 'params_ema':\n opt['path'][param_key] = 'params'\n print(f'Set {param_key} to params')\n\n\ndef sizeof_fmt(size, suffix='B'):\n \"\"\"Get human readable file size.\n\n Args:\n size (int): File size.\n suffix (str): Suffix. Default: 'B'.\n\n Return:\n str: Formatted file size.\n \"\"\"\n for unit in ['', 'K', 'M', 'G', 'T', 'P', 'E', 'Z']:\n if abs(size) < 1024.0:\n return f'{size:3.1f} {unit}{suffix}'\n size /= 1024.0\n return f'{size:3.1f} Y{suffix}'\n"
},
{
"path": "bsr/utils/options.py",
"content": "import argparse\nimport os\nimport random\nimport torch\nimport yaml\nfrom collections import OrderedDict\nfrom os import path as osp\n\nfrom bsr.utils import set_random_seed\nfrom bsr.utils.dist_util import get_dist_info, init_dist, master_only\n\n\ndef ordered_yaml():\n \"\"\"Support OrderedDict for yaml.\n\n Returns:\n tuple: yaml Loader and Dumper.\n \"\"\"\n try:\n from yaml import CDumper as Dumper\n from yaml import CLoader as Loader\n except ImportError:\n from yaml import Dumper, Loader\n\n _mapping_tag = yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG\n\n def dict_representer(dumper, data):\n return dumper.represent_dict(data.items())\n\n def dict_constructor(loader, node):\n return OrderedDict(loader.construct_pairs(node))\n\n Dumper.add_representer(OrderedDict, dict_representer)\n Loader.add_constructor(_mapping_tag, dict_constructor)\n return Loader, Dumper\n\n\ndef yaml_load(f):\n \"\"\"Load yaml file or string.\n\n Args:\n f (str): File path or a python string.\n\n Returns:\n dict: Loaded dict.\n \"\"\"\n if os.path.isfile(f):\n with open(f, 'r') as f:\n return yaml.load(f, Loader=ordered_yaml()[0])\n else:\n return yaml.load(f, Loader=ordered_yaml()[0])\n\n\ndef dict2str(opt, indent_level=1):\n \"\"\"dict to string for printing options.\n\n Args:\n opt (dict): Option dict.\n indent_level (int): Indent level. Default: 1.\n\n Return:\n (str): Option string for printing.\n \"\"\"\n msg = '\\n'\n for k, v in opt.items():\n if isinstance(v, dict):\n msg += ' ' * (indent_level * 2) + k + ':['\n msg += dict2str(v, indent_level + 1)\n msg += ' ' * (indent_level * 2) + ']\\n'\n else:\n msg += ' ' * (indent_level * 2) + k + ': ' + str(v) + '\\n'\n return msg\n\n\ndef _postprocess_yml_value(value):\n # None\n if value == '~' or value.lower() == 'none':\n return None\n # bool\n if value.lower() == 'true':\n return True\n elif value.lower() == 'false':\n return False\n # !!float number\n if value.startswith('!!float'):\n return float(value.replace('!!float', ''))\n # number\n if value.isdigit():\n return int(value)\n elif value.replace('.', '', 1).isdigit() and value.count('.') < 2:\n return float(value)\n # list\n if value.startswith('['):\n return eval(value)\n # str\n return value\n\n\ndef parse_options(root_path, is_train=True):\n parser = argparse.ArgumentParser()\n parser.add_argument('-opt', type=str, required=True, help='Path to option YAML file.')\n parser.add_argument('--launcher', choices=['none', 'pytorch', 'slurm'], default='none', help='job launcher')\n parser.add_argument('--auto_resume', action='store_true')\n parser.add_argument('--debug', action='store_true')\n parser.add_argument('--local_rank', type=int, default=0)\n parser.add_argument(\n '--force_yml', nargs='+', default=None, help='Force to update yml files. Examples: train:ema_decay=0.999')\n args = parser.parse_args()\n\n # parse yml to dict\n opt = yaml_load(args.opt)\n\n # distributed settings\n if args.launcher == 'none':\n opt['dist'] = False\n print('Disable distributed.', flush=True)\n else:\n opt['dist'] = True\n if args.launcher == 'slurm' and 'dist_params' in opt:\n init_dist(args.launcher, **opt['dist_params'])\n else:\n init_dist(args.launcher)\n opt['rank'], opt['world_size'] = get_dist_info()\n\n # random seed\n seed = opt.get('manual_seed')\n if seed is None:\n seed = random.randint(1, 10000)\n opt['manual_seed'] = seed\n set_random_seed(seed + opt['rank'])\n\n # force to update yml options\n if args.force_yml is not None:\n for entry in args.force_yml:\n # now do not support creating new keys\n keys, value = entry.split('=')\n keys, value = keys.strip(), value.strip()\n value = _postprocess_yml_value(value)\n eval_str = 'opt'\n for key in keys.split(':'):\n eval_str += f'[\"{key}\"]'\n eval_str += '=value'\n # using exec function\n exec(eval_str)\n\n opt['auto_resume'] = args.auto_resume\n opt['is_train'] = is_train\n\n # debug setting\n if args.debug and not opt['name'].startswith('debug'):\n opt['name'] = 'debug_' + opt['name']\n\n if opt['num_gpu'] == 'auto':\n opt['num_gpu'] = torch.cuda.device_count()\n\n # datasets\n for phase, dataset in opt['datasets'].items():\n # for multiple datasets, e.g., val_1, val_2; test_1, test_2\n phase = phase.split('_')[0]\n dataset['phase'] = phase\n if 'scale' in opt:\n dataset['scale'] = opt['scale']\n if dataset.get('dataroot_gt') is not None:\n dataset['dataroot_gt'] = osp.expanduser(dataset['dataroot_gt'])\n if dataset.get('dataroot_lq') is not None:\n dataset['dataroot_lq'] = osp.expanduser(dataset['dataroot_lq'])\n\n # paths\n for key, val in opt['path'].items():\n if (val is not None) and ('resume_state' in key or 'pretrain_network' in key):\n opt['path'][key] = osp.expanduser(val)\n\n if is_train:\n experiments_root = opt['path'].get('experiments_root')\n if experiments_root is None:\n experiments_root = osp.join(root_path, 'experiments')\n experiments_root = osp.join(experiments_root, opt['name'])\n\n opt['path']['experiments_root'] = experiments_root\n opt['path']['models'] = osp.join(experiments_root, 'models')\n opt['path']['training_states'] = osp.join(experiments_root, 'training_states')\n opt['path']['log'] = experiments_root\n opt['path']['visualization'] = osp.join(experiments_root, 'visualization')\n\n # change some options for debug mode\n if 'debug' in opt['name']:\n if 'val' in opt:\n opt['val']['val_freq'] = 8\n opt['logger']['print_freq'] = 1\n opt['logger']['save_checkpoint_freq'] = 8\n else: # test\n results_root = opt['path'].get('results_root')\n if results_root is None:\n results_root = osp.join(root_path, 'results')\n results_root = osp.join(results_root, opt['name'])\n\n opt['path']['results_root'] = results_root\n opt['path']['log'] = results_root\n opt['path']['visualization'] = osp.join(results_root, 'visualization')\n\n return opt, args\n\n\n@master_only\ndef copy_opt_file(opt_file, experiments_root):\n # copy the yml file to the experiment root\n import sys\n import time\n from shutil import copyfile\n cmd = ' '.join(sys.argv)\n filename = osp.join(experiments_root, osp.basename(opt_file))\n copyfile(opt_file, filename)\n\n with open(filename, 'r+') as f:\n lines = f.readlines()\n lines.insert(0, f'# GENERATE TIME: {time.asctime()}\\n# CMD:\\n# {cmd}\\n\\n')\n f.seek(0)\n f.writelines(lines)\n"
},
{
"path": "bsr/utils/plot_util.py",
"content": "import re\n\n\ndef read_data_from_tensorboard(log_path, tag):\n \"\"\"Get raw data (steps and values) from tensorboard events.\n\n Args:\n log_path (str): Path to the tensorboard log.\n tag (str): tag to be read.\n \"\"\"\n from tensorboard.backend.event_processing.event_accumulator import EventAccumulator\n\n # tensorboard event\n event_acc = EventAccumulator(log_path)\n event_acc.Reload()\n scalar_list = event_acc.Tags()['scalars']\n print('tag list: ', scalar_list)\n steps = [int(s.step) for s in event_acc.Scalars(tag)]\n values = [s.value for s in event_acc.Scalars(tag)]\n return steps, values\n\n\ndef read_data_from_txt_2v(path, pattern, step_one=False):\n \"\"\"Read data from txt with 2 returned values (usually [step, value]).\n\n Args:\n path (str): path to the txt file.\n pattern (str): re (regular expression) pattern.\n step_one (bool): add 1 to steps. Default: False.\n \"\"\"\n with open(path) as f:\n lines = f.readlines()\n lines = [line.strip() for line in lines]\n steps = []\n values = []\n\n pattern = re.compile(pattern)\n for line in lines:\n match = pattern.match(line)\n if match:\n steps.append(int(match.group(1)))\n values.append(float(match.group(2)))\n if step_one:\n steps = [v + 1 for v in steps]\n return steps, values\n\n\ndef read_data_from_txt_1v(path, pattern):\n \"\"\"Read data from txt with 1 returned values.\n\n Args:\n path (str): path to the txt file.\n pattern (str): re (regular expression) pattern.\n \"\"\"\n with open(path) as f:\n lines = f.readlines()\n lines = [line.strip() for line in lines]\n data = []\n\n pattern = re.compile(pattern)\n for line in lines:\n match = pattern.match(line)\n if match:\n data.append(float(match.group(1)))\n return data\n\n\ndef smooth_data(values, smooth_weight):\n \"\"\" Smooth data using 1st-order IIR low-pass filter (what tensorflow does).\n\n Reference: https://github.com/tensorflow/tensorboard/blob/f801ebf1f9fbfe2baee1ddd65714d0bccc640fb1/tensorboard/plugins/scalar/vz_line_chart/vz-line-chart.ts#L704 # noqa: E501\n\n Args:\n values (list): A list of values to be smoothed.\n smooth_weight (float): Smooth weight.\n \"\"\"\n values_sm = []\n last_sm_value = values[0]\n for value in values:\n value_sm = last_sm_value * smooth_weight + (1 - smooth_weight) * value\n values_sm.append(value_sm)\n last_sm_value = value_sm\n return values_sm\n"
},
{
"path": "bsr/utils/registry.py",
"content": "# Modified from: https://github.com/facebookresearch/fvcore/blob/master/fvcore/common/registry.py # noqa: E501\n\n\nclass Registry():\n \"\"\"\n The registry that provides name -> object mapping, to support third-party\n users' custom modules.\n\n To create a registry (e.g. a backbone registry):\n\n .. code-block:: python\n\n BACKBONE_REGISTRY = Registry('BACKBONE')\n\n To register an object:\n\n .. code-block:: python\n\n @BACKBONE_REGISTRY.register()\n class MyBackbone():\n ...\n\n Or:\n\n .. code-block:: python\n\n BACKBONE_REGISTRY.register(MyBackbone)\n \"\"\"\n\n def __init__(self, name):\n \"\"\"\n Args:\n name (str): the name of this registry\n \"\"\"\n self._name = name\n self._obj_map = {}\n\n def _do_register(self, name, obj, suffix=None):\n if isinstance(suffix, str):\n name = name + '_' + suffix\n\n assert (name not in self._obj_map), (f\"An object named '{name}' was already registered \"\n f\"in '{self._name}' registry!\")\n self._obj_map[name] = obj\n\n def register(self, obj=None, suffix=None):\n \"\"\"\n Register the given object under the the name `obj.__name__`.\n Can be used as either a decorator or not.\n See docstring of this class for usage.\n \"\"\"\n if obj is None:\n # used as a decorator\n def deco(func_or_class):\n name = func_or_class.__name__\n self._do_register(name, func_or_class, suffix)\n return func_or_class\n\n return deco\n\n # used as a function call\n name = obj.__name__\n self._do_register(name, obj, suffix)\n\n def get(self, name, suffix='basicsr'):\n ret = self._obj_map.get(name)\n if ret is None:\n ret = self._obj_map.get(name + '_' + suffix)\n print(f'Name {name} is not found, use name: {name}_{suffix}!')\n if ret is None:\n raise KeyError(f\"No object named '{name}' found in '{self._name}' registry!\")\n return ret\n\n def __contains__(self, name):\n return name in self._obj_map\n\n def __iter__(self):\n return iter(self._obj_map.items())\n\n def keys(self):\n return self._obj_map.keys()\n\n\nDATASET_REGISTRY = Registry('dataset')\nARCH_REGISTRY = Registry('arch')\nMODEL_REGISTRY = Registry('model')\nLOSS_REGISTRY = Registry('loss')\nMETRIC_REGISTRY = Registry('metric')\n"
},
{
"path": "config.yml",
"content": "dataroot_gt: path_to_HR_images_of_LSDIR\n\nscale: 4\n\n# the first degradation process\nresize_prob: [0.2, 0.7, 0.1] # up, down, keep\nresize_range: [0.3, 1.5]\ngaussian_noise_prob: 0.5\nnoise_range: [1, 15]\npoisson_scale_range: [0.05, 2.0]\ngray_noise_prob: 0.4\njpeg_range: [60, 95]\n\n# the second degradation process\nsecond_blur_prob: 0.5\nresize_prob2: [0.3, 0.4, 0.3] # up, down, keep\nresize_range2: [0.6, 1.2]\ngaussian_noise_prob2: 0.5\nnoise_range2: [1, 12]\npoisson_scale_range2: [0.05, 1.0]\ngray_noise_prob2: 0.4\njpeg_range2: [60, 100]\n\ngt_size: 512\n\nblur_kernel_size: 21\nkernel_list: ['iso', 'aniso', 'generalized_iso', 'generalized_aniso', 'plateau_iso', 'plateau_aniso']\nkernel_prob: [0.45, 0.25, 0.12, 0.03, 0.12, 0.03]\nsinc_prob: 0.1\nblur_sigma: [0.2, 1.5]\nbetag_range: [0.5, 2.0]\nbetap_range: [1, 1.5]\n\nblur_kernel_size2: 11\nkernel_list2: ['iso', 'aniso', 'generalized_iso', 'generalized_aniso', 'plateau_iso', 'plateau_aniso']\nkernel_prob2: [0.45, 0.25, 0.12, 0.03, 0.12, 0.03]\nsinc_prob2: 0.1\nblur_sigma2: [0.2, 1.0]\nbetag_range2: [0.5, 2.0]\nbetap_range2: [1, 1.5]\n\nfinal_sinc_prob: 0.8\n\nuse_hflip: True\nuse_rot: False\n\niter_num: 1000"
},
{
"path": "dataset.py",
"content": "import torch, random, cv2, os, math, glob\nimport torch.nn.functional as F\nimport numpy as np\nfrom bsr.degradations import circular_lowpass_kernel, random_mixed_kernels, random_add_gaussian_noise_pt, random_add_poisson_noise_pt\nfrom bsr.transforms import augment, paired_random_crop\nfrom bsr.utils import FileClient, imfrombytes, img2tensor, DiffJPEG\nfrom bsr.utils.img_process_util import filter2D\n\nclass RealESRGANDataset(torch.utils.data.Dataset):\n def __init__(self, opt, bsz):\n super(RealESRGANDataset, self).__init__()\n self.opt = opt\n self.file_client = FileClient(\"disk\")\n self.gt_folder = opt[\"dataroot_gt\"]\n self.len = bsz * opt[\"iter_num\"]\n self.paths = glob.glob(os.path.join(self.gt_folder, \"**/*\"), recursive=True)\n\n # blur settings for the first degradation\n self.blur_kernel_size = opt[\"blur_kernel_size\"]\n self.kernel_list = opt[\"kernel_list\"]\n self.kernel_prob = opt[\"kernel_prob\"] # a list for each kernel probability\n self.blur_sigma = opt[\"blur_sigma\"]\n self.betag_range = opt[\"betag_range\"] # betag used in generalized Gaussian blur kernels\n self.betap_range = opt[\"betap_range\"] # betap used in plateau blur kernels\n self.sinc_prob = opt[\"sinc_prob\"] # the probability for sinc filters\n\n # blur settings for the second degradation\n self.blur_kernel_size2 = opt[\"blur_kernel_size2\"]\n self.kernel_list2 = opt[\"kernel_list2\"]\n self.kernel_prob2 = opt[\"kernel_prob2\"]\n self.blur_sigma2 = opt[\"blur_sigma2\"]\n self.betag_range2 = opt[\"betag_range2\"]\n self.betap_range2 = opt[\"betap_range2\"]\n self.sinc_prob2 = opt[\"sinc_prob2\"]\n\n # a final sinc filter\n self.final_sinc_prob = opt[\"final_sinc_prob\"]\n\n self.kernel_range = [2 * v + 1 for v in range(3, 11)] # kernel size ranges from 7 to 21\n # TODO: kernel range is now hard-coded, should be in the configure file\n self.pulse_tensor = torch.zeros(21, 21).float() # convolving with pulse tensor brings no blurry effect\n self.pulse_tensor[10, 10] = 1\n\n def __getitem__(self, index):\n index = random.randint(0, len(self.paths) - 1)\n gt_path = self.paths[index]\n img_gt = imfrombytes(self.file_client.get(gt_path, \"gt\"), float32=True)\n img_gt = augment(img_gt, self.opt[\"use_hflip\"], self.opt[\"use_rot\"])\n h, w = img_gt.shape[0:2]\n crop_pad_size = self.opt.gt_size\n if h < crop_pad_size or w < crop_pad_size:\n pad_h = max(0, crop_pad_size - h)\n pad_w = max(0, crop_pad_size - w)\n img_gt = cv2.copyMakeBorder(img_gt, 0, pad_h, 0, pad_w, cv2.BORDER_REFLECT_101)\n if img_gt.shape[0] > crop_pad_size or img_gt.shape[1] > crop_pad_size:\n h, w = img_gt.shape[0:2]\n top = random.randint(0, h - crop_pad_size)\n left = random.randint(0, w - crop_pad_size)\n img_gt = img_gt[top:top + crop_pad_size, left:left + crop_pad_size, ...]\n\n # ------------------------ Generate kernels (used in the first degradation) ------------------------ #\n kernel_size = random.choice(self.kernel_range)\n if np.random.uniform() < self.opt[\"sinc_prob\"]:\n # this sinc filter setting is for kernels ranging from [7, 21]\n if kernel_size < 13:\n omega_c = np.random.uniform(np.pi / 3, np.pi)\n else:\n omega_c = np.random.uniform(np.pi / 5, np.pi)\n kernel = circular_lowpass_kernel(omega_c, kernel_size, pad_to=False)\n else:\n kernel = random_mixed_kernels(\n self.kernel_list,\n self.kernel_prob,\n kernel_size,\n self.blur_sigma,\n self.blur_sigma, [-math.pi, math.pi],\n self.betag_range,\n self.betap_range,\n noise_range=None)\n # pad kernel\n pad_size = (21 - kernel_size) // 2\n kernel = np.pad(kernel, ((pad_size, pad_size), (pad_size, pad_size)))\n\n # ------------------------ Generate kernels (used in the second degradation) ------------------------ #\n kernel_size = random.choice(self.kernel_range)\n if np.random.uniform() < self.opt[\"sinc_prob2\"]:\n if kernel_size < 13:\n omega_c = np.random.uniform(np.pi / 3, np.pi)\n else:\n omega_c = np.random.uniform(np.pi / 5, np.pi)\n kernel2 = circular_lowpass_kernel(omega_c, kernel_size, pad_to=False)\n else:\n kernel2 = random_mixed_kernels(\n self.kernel_list2,\n self.kernel_prob2,\n kernel_size,\n self.blur_sigma2,\n self.blur_sigma2, [-math.pi, math.pi],\n self.betag_range2,\n self.betap_range2,\n noise_range=None)\n\n # pad kernel\n pad_size = (21 - kernel_size) // 2\n kernel2 = np.pad(kernel2, ((pad_size, pad_size), (pad_size, pad_size)))\n\n # ------------------------------------- the final sinc kernel ------------------------------------- #\n if np.random.uniform() < self.opt[\"final_sinc_prob\"]:\n kernel_size = random.choice(self.kernel_range)\n omega_c = np.random.uniform(np.pi / 3, np.pi)\n sinc_kernel = circular_lowpass_kernel(omega_c, kernel_size, pad_to=21)\n sinc_kernel = torch.FloatTensor(sinc_kernel)\n else:\n sinc_kernel = self.pulse_tensor\n\n # BGR to RGB, HWC to CHW, numpy to tensor\n img_gt = img2tensor([img_gt], bgr2rgb=True, float32=True)[0]\n kernel = torch.FloatTensor(kernel)\n kernel2 = torch.FloatTensor(kernel2)\n\n return_d = {\"gt\": img_gt, \"kernel1\": kernel, \"kernel2\": kernel2, \"sinc_kernel\": sinc_kernel, \"gt_path\": gt_path}\n return return_d\n\n def __len__(self):\n return self.len\n\nclass RealESRGANDegrader:\n def __init__(self, opt, device):\n self.opt = opt\n self.device = device\n self.jpeger = DiffJPEG(differentiable=False).to(device) # simulate JPEG compression artifacts\n self.queue_size = 1200\n\n @torch.no_grad()\n def _dequeue_and_enqueue(self):\n \"\"\"It is the training pair pool for increasing the diversity in a batch.\n\n Batch processing limits the diversity of synthetic degradations in a batch. For example, samples in a\n batch could not have different resize scaling factors. Therefore, we employ this training pair pool\n to increase the degradation diversity in a batch.\n \"\"\"\n # initialize\n b, c, h, w = self.lq.size()\n if not hasattr(self, \"queue_lr\"):\n assert self.queue_size % b == 0, f\"queue size {self.queue_size} should be divisible by batch size {b}\"\n self.queue_lr = torch.zeros(self.queue_size, c, h, w).to(self.device)\n _, c, h, w = self.gt.size()\n self.queue_gt = torch.zeros(self.queue_size, c, h, w).to(self.device)\n self.queue_ptr = 0\n if self.queue_ptr == self.queue_size: # the pool is full\n # do dequeue and enqueue\n # shuffle\n idx = torch.randperm(self.queue_size)\n self.queue_lr = self.queue_lr[idx]\n self.queue_gt = self.queue_gt[idx]\n # get first b samples\n lq_dequeue = self.queue_lr[0:b, :, :, :].clone()\n gt_dequeue = self.queue_gt[0:b, :, :, :].clone()\n # update the queue\n self.queue_lr[0:b, :, :, :] = self.lq.clone()\n self.queue_gt[0:b, :, :, :] = self.gt.clone()\n\n self.lq = lq_dequeue\n self.gt = gt_dequeue\n else:\n # only do enqueue\n self.queue_lr[self.queue_ptr:self.queue_ptr + b, :, :, :] = self.lq.clone()\n self.queue_gt[self.queue_ptr:self.queue_ptr + b, :, :, :] = self.gt.clone()\n self.queue_ptr = self.queue_ptr + b\n\n @torch.no_grad()\n def degrade(self, data):\n \"\"\"Accept data from dataloader, and then add two-order degradations to obtain LQ images.\n \"\"\"\n # training data synthesis\n self.gt = data[\"gt\"].to(self.device)\n\n self.kernel1 = data[\"kernel1\"].to(self.device)\n self.kernel2 = data[\"kernel2\"].to(self.device)\n self.sinc_kernel = data[\"sinc_kernel\"].to(self.device)\n\n ori_h, ori_w = self.gt.size()[2:4]\n\n # ----------------------- The first degradation process ----------------------- #\n # blur\n out = filter2D(self.gt, self.kernel1)\n # random resize\n updown_type = random.choices([\"up\", \"down\", \"keep\"], self.opt[\"resize_prob\"])[0]\n if updown_type == \"up\":\n scale = np.random.uniform(1, self.opt[\"resize_range\"][1])\n elif updown_type == \"down\":\n scale = np.random.uniform(self.opt[\"resize_range\"][0], 1)\n else:\n scale = 1\n mode = random.choice([\"area\", \"bilinear\", \"bicubic\"])\n out = F.interpolate(out, scale_factor=scale, mode=mode)\n # add noise\n gray_noise_prob = self.opt[\"gray_noise_prob\"]\n if np.random.uniform() < self.opt[\"gaussian_noise_prob\"]:\n out = random_add_gaussian_noise_pt(\n out, sigma_range=self.opt[\"noise_range\"], clip=True, rounds=False, gray_prob=gray_noise_prob)\n else:\n out = random_add_poisson_noise_pt(\n out,\n scale_range=self.opt[\"poisson_scale_range\"],\n gray_prob=gray_noise_prob,\n clip=True,\n rounds=False)\n # JPEG compression\n jpeg_p = out.new_zeros(out.size(0)).uniform_(*self.opt[\"jpeg_range\"])\n out = torch.clamp(out, 0, 1) # clamp to [0, 1], otherwise JPEGer will result in unpleasant artifacts\n out = self.jpeger(out, quality=jpeg_p)\n\n # ----------------------- The second degradation process ----------------------- #\n # blur\n if np.random.uniform() < self.opt[\"second_blur_prob\"]:\n out = filter2D(out, self.kernel2)\n # random resize\n updown_type = random.choices([\"up\", \"down\", \"keep\"], self.opt[\"resize_prob2\"])[0]\n if updown_type == \"up\":\n scale = np.random.uniform(1, self.opt[\"resize_range2\"][1])\n elif updown_type == \"down\":\n scale = np.random.uniform(self.opt[\"resize_range2\"][0], 1)\n else:\n scale = 1\n mode = random.choice([\"area\", \"bilinear\", \"bicubic\"])\n out = F.interpolate(\n out, size=(int(ori_h / self.opt[\"scale\"] * scale), int(ori_w / self.opt[\"scale\"] * scale)), mode=mode)\n # add noise\n gray_noise_prob = self.opt[\"gray_noise_prob2\"]\n if np.random.uniform() < self.opt[\"gaussian_noise_prob2\"]:\n out = random_add_gaussian_noise_pt(\n out, sigma_range=self.opt[\"noise_range2\"], clip=True, rounds=False, gray_prob=gray_noise_prob)\n else:\n out = random_add_poisson_noise_pt(\n out,\n scale_range=self.opt[\"poisson_scale_range2\"],\n gray_prob=gray_noise_prob,\n clip=True,\n rounds=False)\n\n # JPEG compression + the final sinc filter\n # We also need to resize images to desired sizes. We group [resize back + sinc filter] together\n # as one operation.\n # We consider two orders:\n # 1. [resize back + sinc filter] + JPEG compression\n # 2. JPEG compression + [resize back + sinc filter]\n # Empirically, we find other combinations (sinc + JPEG + Resize) will introduce twisted lines.\n if np.random.uniform() < 0.5:\n # resize back + the final sinc filter\n mode = random.choice([\"area\", \"bilinear\", \"bicubic\"])\n out = F.interpolate(out, size=(ori_h // self.opt[\"scale\"], ori_w // self.opt[\"scale\"]), mode=mode)\n out = filter2D(out, self.sinc_kernel)\n # JPEG compression\n jpeg_p = out.new_zeros(out.size(0)).uniform_(*self.opt[\"jpeg_range2\"])\n out = torch.clamp(out, 0, 1)\n out = self.jpeger(out, quality=jpeg_p)\n else:\n # JPEG compression\n jpeg_p = out.new_zeros(out.size(0)).uniform_(*self.opt[\"jpeg_range2\"])\n out = torch.clamp(out, 0, 1)\n out = self.jpeger(out, quality=jpeg_p)\n # resize back + the final sinc filter\n mode = random.choice([\"area\", \"bilinear\", \"bicubic\"])\n out = F.interpolate(out, size=(ori_h // self.opt[\"scale\"], ori_w // self.opt[\"scale\"]), mode=mode)\n out = filter2D(out, self.sinc_kernel)\n\n # clamp and round\n self.lq = torch.clamp((out * 255.0).round(), 0, 255) / 255.\n\n # random crop\n gt_size = self.opt[\"gt_size\"]\n self.gt, self.lq = paired_random_crop(self.gt, self.lq, gt_size, self.opt[\"scale\"])\n\n # training pair pool\n self._dequeue_and_enqueue()\n # sharpen self.gt again, as we have changed the self.gt with self._dequeue_and_enqueue\n self.lq = self.lq.contiguous() # for the warning: grad and param do not obey the gradient layout contract\n\n return self.lq, self.gt\n"
},
{
"path": "evaluate.py",
"content": "import torch, os, glob, pyiqa\nfrom argparse import ArgumentParser\nimport numpy as np\nfrom PIL import Image\nfrom tqdm import tqdm\nfrom torchvision import transforms\n\nparser = ArgumentParser()\nparser.add_argument(\"--HR_dir\", type=str, default=\"testset/RealSR/HR\")\nparser.add_argument(\"--SR_dir\", type=str, default=\"result/RealSR\")\nargs = parser.parse_args()\n\ndevice = torch.device(\"cuda\")\n\npsnr = pyiqa.create_metric(\"psnr\", test_y_channel=True, color_space=\"ycbcr\", device=device)\nssim = pyiqa.create_metric(\"ssim\", test_y_channel=True, color_space=\"ycbcr\", device=device)\nlpips = pyiqa.create_metric(\"lpips\", device=device)\ndists = pyiqa.create_metric(\"dists\", device=device)\nfid = pyiqa.create_metric(\"fid\", device=device)\nniqe = pyiqa.create_metric(\"niqe\", device=device)\nmaniqa = pyiqa.create_metric(\"maniqa-pipal\", device=device)\nclipiqa = pyiqa.create_metric(\"clipiqa\", device=device)\nmusiq = pyiqa.create_metric(\"musiq\", device=device)\n\ntest_SR_paths = list(sorted(glob.glob(os.path.join(args.SR_dir, \"*\"))))\ntest_HR_paths = list(sorted(glob.glob(os.path.join(args.HR_dir, \"*\"))))\n\nmetrics = {\"psnr\": [], \"ssim\": [], \"lpips\": [], \"dists\": [], \"niqe\": [], \"maniqa\": [], \"musiq\": [], \"clipiqa\": []}\n\nfor i, (SR_path, HR_path) in tqdm(enumerate(zip(test_SR_paths, test_HR_paths))):\n SR = Image.open(SR_path).convert(\"RGB\")\n SR = transforms.ToTensor()(SR).to(device).unsqueeze(0)\n HR = Image.open(HR_path).convert(\"RGB\")\n HR = transforms.ToTensor()(HR).to(device).unsqueeze(0)\n metrics[\"psnr\"].append(psnr(SR, HR).item())\n metrics[\"ssim\"].append(ssim(SR, HR).item())\n metrics[\"lpips\"].append(lpips(SR, HR).item())\n metrics[\"dists\"].append(dists(SR, HR).item())\n metrics[\"niqe\"].append(niqe(SR).item())\n metrics[\"maniqa\"].append(maniqa(SR).item())\n metrics[\"clipiqa\"].append(clipiqa(SR).item())\n metrics[\"musiq\"].append(musiq(SR).item())\n\nfor k in metrics.keys():\n metrics[k] = np.mean(metrics[k])\n\nmetrics[\"fid\"] = fid(args.SR_dir, args.HR_dir)\n\nfor k, v in metrics.items():\n if k == \"niqe\":\n print(k, f\"{v:.3g}\")\n elif k == \"fid\":\n print(k, f\"{v:.5g}\")\n else:\n print(k, f\"{v:.4g}\")"
},
{
"path": "evaluate_debug.sh",
"content": "HF_ENDPOINT=https://hf-mirror.com \\\nCUDA_VISIBLE_DEVICES=0 \\\npython -u evaluate.py \\\n--HR_dir=testset/RealSR/HR \\\n--SR_dir=result/RealSR\n"
},
{
"path": "forward.py",
"content": "import torch\n\ndef MyUNet2DConditionModel_SD_forward(self, x):\n global skip\n x = self.conv_in(x)\n skip = [x]\n x = self.body(x)\n return x\n\ndef MyCrossAttnDownBlock2D_SD_forward(self, x):\n for i in range(2):\n x = self.resnets[i](x)\n x = self.attentions[i](x)\n skip.append(x)\n if self.downsamplers is not None:\n x = self.downsamplers[0](x)\n skip.append(x)\n return x\n\ndef MyCrossAttnUpBlock2D_SD_forward(self, x):\n for i in range(3):\n x = self.resnets[i](torch.cat([x, skip.pop()], dim=1))\n x = self.attentions[i](x)\n if self.upsamplers is not None:\n x = self.upsamplers[0](x)\n return x\n\ndef MyDownBlock2D_SD_forward(self, x):\n for i in range(2):\n x = self.resnets[i](x)\n skip.append(x)\n return x\n\ndef MyUNetMidBlock2DCrossAttn_SD_forward(self, x):\n x = self.resnets[0](x)\n x = self.attentions[0](x)\n x = self.resnets[1](x)\n return x\n\ndef MyUpBlock2D_SD_forward(self, x):\n for i in range(3):\n x = self.resnets[i](torch.cat([x, skip.pop()], dim=1))\n x = self.upsamplers[0](x)\n return x\n\ndef MyResnetBlock2D_SD_forward(self, x_in):\n x = self.norm1(x_in)\n x = self.nonlinearity(x)\n x = self.conv1(x)\n x = self.norm2(x)\n x = self.nonlinearity(x)\n x = self.conv2(x)\n if self.in_channels == self.out_channels:\n return x + x_in\n return x + self.conv_shortcut(x_in)\n\ndef MyTransformer2DModel_SD_forward(self, x_in):\n b, c, h, w = x_in.shape\n x = self.norm(x_in)\n x = x.permute(0, 2, 3, 1).reshape(b, h * w, c).contiguous()\n x = self.proj_in(x)\n for block in self.transformer_blocks:\n x = x + block.attn1(block.norm1(x))\n x = x + block.ff(block.norm3(x))\n x = self.proj_out(x)\n x = x.reshape(b, h, w, c).permute(0, 3, 1, 2).contiguous()\n return x + x_in"
},
{
"path": "model.py",
"content": "import torch, types, copy\r\nfrom torch import nn\r\nimport torch.nn.functional as F\r\nfrom diffusers.models.unets.unet_2d_blocks import CrossAttnDownBlock2D, \\\r\n CrossAttnUpBlock2D, \\\r\n DownBlock2D, \\\r\n UpBlock2D, \\\r\n UNetMidBlock2DCrossAttn\r\nfrom diffusers.models.resnet import ResnetBlock2D\r\nfrom diffusers.models.transformers.transformer_2d import Transformer2DModel\r\nfrom diffusers.models.attention import BasicTransformerBlock\r\nfrom diffusers.models.downsampling import Downsample2D\r\nfrom diffusers.models.upsampling import Upsample2D\r\nfrom forward import MyUNet2DConditionModel_SD_forward, \\\r\n MyCrossAttnDownBlock2D_SD_forward, \\\r\n MyDownBlock2D_SD_forward, \\\r\n MyUNetMidBlock2DCrossAttn_SD_forward, \\\r\n MyCrossAttnUpBlock2D_SD_forward, \\\r\n MyUpBlock2D_SD_forward, \\\r\n MyResnetBlock2D_SD_forward, \\\r\n MyTransformer2DModel_SD_forward\r\n\r\ndef find_parent(model, module_name):\r\n components = module_name.split(\".\")\r\n parent = model\r\n for comp in components[:-1]:\r\n parent = getattr(parent, comp)\r\n return parent, components[-1]\r\n\r\ndef halve_channels(model):\r\n for name, module in model.named_modules():\r\n if hasattr(module, \"pruned\"):\r\n continue\r\n if isinstance(module, nn.Conv2d):\r\n in_channels = int(module.in_channels * 0.75)\r\n out_channels = int(module.out_channels * 0.75)\r\n new_conv = nn.Conv2d(in_channels=in_channels,\r\n out_channels=out_channels,\r\n kernel_size=module.kernel_size,\r\n stride=module.stride,\r\n padding=module.padding,\r\n dilation=module.dilation,\r\n groups=module.groups,\r\n bias=module.bias is not None)\r\n with torch.no_grad():\r\n new_conv.weight.copy_(module.weight[:out_channels, :in_channels])\r\n if module.bias is not None:\r\n new_conv.bias.copy_(module.bias[:out_channels])\r\n parent, last_name = find_parent(model, name)\r\n setattr(parent, last_name, new_conv)\r\n new_conv.pruned = True\r\n elif isinstance(module, nn.Linear):\r\n in_features = int(module.in_features * 0.75)\r\n out_features = int(module.out_features * 0.75)\r\n new_linear = nn.Linear(in_features=in_features,\r\n out_features=out_features,\r\n bias=module.bias is not None)\r\n with torch.no_grad():\r\n new_linear.weight.copy_(module.weight[:out_features, :in_features])\r\n if module.bias is not None:\r\n new_linear.bias.copy_(module.bias[:out_features])\r\n parent, last_name = find_parent(model, name)\r\n setattr(parent, last_name, new_linear)\r\n new_linear.pruned = True\r\n elif isinstance(module, nn.GroupNorm):\r\n num_channels = int(module.num_channels * 0.75)\r\n for num_groups in [32, 24, 16, 12, 8, 6, 4, 2, 1]:\r\n if num_channels % num_groups == 0:\r\n break\r\n new_gn = nn.GroupNorm(num_groups=num_groups,\r\n num_channels=num_channels,\r\n eps=module.eps,\r\n affine=module.affine)\r\n with torch.no_grad():\r\n new_gn.weight.copy_(module.weight[:num_channels])\r\n new_gn.bias.copy_(module.bias[:num_channels])\r\n parent, last_name = find_parent(model, name)\r\n setattr(parent, last_name, new_gn)\r\n new_gn.pruned = True\r\n elif isinstance(module, nn.LayerNorm):\r\n normalized_shape = int(module.normalized_shape[0] * 0.75)\r\n new_ln = nn.LayerNorm(normalized_shape, \r\n eps=module.eps, \r\n elementwise_affine=module.elementwise_affine)\r\n with torch.no_grad():\r\n new_ln.weight.copy_(module.weight[:normalized_shape])\r\n new_ln.bias.copy_(module.bias[:normalized_shape])\r\n parent, last_name = find_parent(model, name)\r\n setattr(parent, last_name, new_ln)\r\n new_ln.pruned = True\r\n elif isinstance(module, Downsample2D) or isinstance(module, Upsample2D):\r\n module.channels = int(module.channels * 0.75)\r\n\r\nclass Net(nn.Module):\r\n def __init__(self, unet, decoder):\r\n super().__init__()\r\n del unet.time_embedding\r\n new_conv_in = nn.Conv2d(16, 320, 3, padding=1)\r\n new_conv_in.weight.data = unet.conv_in.weight.data.repeat(1, 4, 1, 1)\r\n new_conv_in.bias.data = unet.conv_in.bias.data\r\n unet.conv_in = new_conv_in\r\n new_conv_out = nn.Conv2d(320, 342, 3, padding=1)\r\n new_conv_out.weight.data = unet.conv_out.weight.data.repeat(86, 1, 1, 1)[:342]\r\n new_conv_out.bias.data = unet.conv_out.bias.data.repeat(86,)[:342]\r\n unet.conv_out = new_conv_out\r\n def ResnetBlock2D_remove_time_emb_proj(module):\r\n if isinstance(module, ResnetBlock2D):\r\n del module.time_emb_proj\r\n unet.apply(ResnetBlock2D_remove_time_emb_proj)\r\n def BasicTransformerBlock_remove_cross_attn(module):\r\n if isinstance(module, BasicTransformerBlock):\r\n del module.attn2, module.norm2\r\n unet.apply(BasicTransformerBlock_remove_cross_attn)\r\n def set_inplace_to_true(module):\r\n if isinstance(module, nn.Dropout) or isinstance(module, nn.SiLU):\r\n module.inplace = True\r\n unet.apply(set_inplace_to_true)\r\n def replace_forward_methods(module):\r\n if isinstance(module, CrossAttnDownBlock2D):\r\n module.forward = types.MethodType(MyCrossAttnDownBlock2D_SD_forward, module)\r\n elif isinstance(module, DownBlock2D):\r\n module.forward = types.MethodType(MyDownBlock2D_SD_forward, module)\r\n elif isinstance(module, UNetMidBlock2DCrossAttn):\r\n module.forward = types.MethodType(MyUNetMidBlock2DCrossAttn_SD_forward, module)\r\n elif isinstance(module, UpBlock2D):\r\n module.forward = types.MethodType(MyUpBlock2D_SD_forward, module)\r\n elif isinstance(module, CrossAttnUpBlock2D):\r\n module.forward = types.MethodType(MyCrossAttnUpBlock2D_SD_forward, module)\r\n elif isinstance(module, ResnetBlock2D):\r\n module.forward = types.MethodType(MyResnetBlock2D_SD_forward, module)\r\n elif isinstance(module, Transformer2DModel):\r\n module.forward = types.MethodType(MyTransformer2DModel_SD_forward, module)\r\n unet.apply(replace_forward_methods)\r\n unet.forward = types.MethodType(MyUNet2DConditionModel_SD_forward, unet)\r\n halve_channels(unet)\r\n unet.body = nn.Sequential(\r\n *unet.down_blocks,\r\n unet.mid_block,\r\n *unet.up_blocks,\r\n unet.conv_norm_out,\r\n unet.conv_act,\r\n unet.conv_out,\r\n )\r\n del decoder.conv_in, decoder.up_blocks, decoder.conv_norm_out, decoder.conv_act, decoder.conv_out\r\n self.body = nn.Sequential(\r\n nn.PixelUnshuffle(2),\r\n unet,\r\n decoder.mid_block,\r\n )\r\n \r\n def forward(self, x):\r\n return self.body(x)"
},
{
"path": "ram/configs/condition_config.json",
"content": "{\n \"nf\": 64\n }"
},
{
"path": "ram/configs/med_config.json",
"content": "{\n \"architectures\": [\n \"BertModel\"\n ],\n \"attention_probs_dropout_prob\": 0.1,\n \"hidden_act\": \"gelu\",\n \"hidden_dropout_prob\": 0.1,\n \"hidden_size\": 768,\n \"initializer_range\": 0.02,\n \"intermediate_size\": 3072,\n \"layer_norm_eps\": 1e-12,\n \"max_position_embeddings\": 512,\n \"model_type\": \"bert\",\n \"num_attention_heads\": 12,\n \"num_hidden_layers\": 12,\n \"pad_token_id\": 0,\n \"type_vocab_size\": 2,\n \"vocab_size\": 30524,\n \"encoder_width\": 768,\n \"add_cross_attention\": true \n }"
},
{
"path": "ram/configs/q2l_config.json",
"content": "{\n \"architectures\": [\n \"BertModel\"\n ],\n \"attention_probs_dropout_prob\": 0.1,\n \"hidden_act\": \"gelu\",\n \"hidden_dropout_prob\": 0.1,\n \"hidden_size\": 768,\n \"initializer_range\": 0.02,\n \"intermediate_size\": 3072,\n \"layer_norm_eps\": 1e-12,\n \"max_position_embeddings\": 512,\n \"model_type\": \"bert\",\n \"num_attention_heads\": 4,\n \"num_hidden_layers\": 2,\n \"pad_token_id\": 0,\n \"type_vocab_size\": 2,\n \"vocab_size\": 30522,\n \"encoder_width\": 768,\n \"add_cross_attention\": true,\n \"add_tag_cross_attention\": false\n }"
},
{
"path": "ram/configs/swin/config_swinB_384.json",
"content": "{\n \"ckpt\": \"pretrain_model/swin_base_patch4_window7_224_22k.pth\",\n \"vision_width\": 1024,\n \"image_res\": 384,\n \"window_size\": 12,\n \"embed_dim\": 128,\n \"depths\": [ 2, 2, 18, 2 ],\n \"num_heads\": [ 4, 8, 16, 32 ]\n }"
},
{
"path": "ram/configs/swin/config_swinL_384.json",
"content": "{\n \"ckpt\": \"pretrain_model/swin_large_patch4_window12_384_22k.pth\",\n \"vision_width\": 1536,\n \"image_res\": 384,\n \"window_size\": 12,\n \"embed_dim\": 192,\n \"depths\": [ 2, 2, 18, 2 ],\n \"num_heads\": [ 6, 12, 24, 48 ]\n }"
},
{
"path": "ram/configs/swin/config_swinL_444.json",
"content": "{\n \"ckpt\": \"pretrain_model/swin_large_patch4_window12_384_22k.pth\",\n \"vision_width\": 1536,\n \"image_res\": 444,\n \"window_size\": 12,\n \"embed_dim\": 192,\n \"depths\": [ 2, 2, 18, 2 ],\n \"num_heads\": [ 6, 12, 24, 48 ]\n }"
},
{
"path": "ram/data/ram_tag_list.txt",
"content": "3D CG rendering\n3D glasses\nabacus\nabalone\nmonastery\nbelly\nacademy\naccessory\naccident\naccordion\nacorn\nacrylic paint\nact\naction\naction film\nactivity\nactor\nadaptation\nadd\nadhesive tape\nadjust\nadult\nadventure\nadvertisement\nantenna\naerobics\nspray can\nafro\nagriculture\naid\nair conditioner\nair conditioning\nair sock\naircraft cabin\naircraft model\nair field\nair line\nairliner\nairman\nplane\nairplane window\nairport\nairport runway\nairport terminal\nairship\nairshow\naisle\nalarm\nalarm clock\nmollymawk\nalbum\nalbum cover\nalcohol\nalcove\nalgae\nalley\nalmond\naloe vera\nalp\nalpaca\nalphabet\ngerman shepherd\naltar\namber\nambulance\nbald eagle\nAmerican shorthair\namethyst\namphitheater\namplifier\namusement park\namusement ride\nanchor\nancient\nanemone\nangel\nangle\nanimal\nanimal sculpture\nanimal shelter\nanimation\nanimation film\nanimator\nanime\nankle\nanklet\nanniversary\ntrench coat\nant\nantelope\nantique\nantler\nanvil\napartment\nape\napp\napp icon\nappear\nappearance\nappetizer\napplause\napple\napple juice\napple pie\napple tree\napplesauce\nappliance\nappointment\napproach\napricot\napron\naqua\naquarium\naquarium fish\naqueduct\narcade\narcade machine\narch\narch bridge\narchaelogical excavation\narchery\narchipelago\narchitect\narchitecture\narchive\narchway\narea\narena\nargument\narm\narmadillo\narmband\narmchair\narmoire\narmor\narmy\narmy base\narmy tank\narray\narrest\narrow\nart\nart exhibition\nart gallery\nart print\nart school\nart studio\nart vector illustration\nartichoke\narticle\nartifact\nartist\nartists loft\nash\nashtray\nasia temple\nasparagus\nasphalt road\nassemble\nassembly\nassembly line\nassociation\nastronaut\nastronomer\nathlete\nathletic\natlas\natm\natmosphere\natrium\nattach\nfighter jet\nattend\nattraction\natv\neggplant\nauction\naudi\naudio\nauditorium\naurora\nauthor\nauto factory\nauto mechanic\nauto part\nauto show\nauto showroom\ncar battery\nautomobile make\nautomobile model\nmotor vehicle\nautumn\nautumn forest\nautumn leave\nautumn park\nautumn tree\navatar\navenue\naviator sunglasses\navocado\naward\naward ceremony\naward winner\nshed\nax\nazalea\nbaboon\nbaby\nbaby bottle\nbaby carriage\nbaby clothe\nbaby elephant\nbaby food\nbaby seat\nbaby shower\nback\nbackdrop\nbacklight\nbackpack\nbackyard\nbacon\nbadge\nbadger\nbadlands\nbadminton\nbadminton racket\nbag\nbagel\nbagpipe\nbaguette\nbait\nbaked goods\nbaker\nbakery\nbaking\nbaking sheet\nbalance\nbalance car\nbalcony\nball\nball pit\nballerina\nballet\nballet dancer\nballet skirt\nballoon\nballoon arch\nbaseball player\nballroom\nbamboo\nbamboo forest\nbanana\nbanana bread\nbanana leaf\nbanana tree\nband\nband aid\nbandage\nheadscarf\nbandeau\nbangs\nbracelet\nbalustrade\nbanjo\nbank\nbank card\nbank vault\nbanknote\nbanner\nbanquet\nbanquet hall\nbanyan tree\nbaozi\nbaptism\nbar\nbar code\nbar stool\nbarbecue\nbarbecue grill\nbarbell\nbarber\nbarber shop\nbarbie\nbarge\nbarista\nbark\nbarley\nbarn\nbarn owl\nbarn door\nbarrel\nbarricade\nbarrier\nhandcart\nbartender\nbaseball\nbaseball base\nbaseball bat\nbaseball hat\nbaseball stadium\nbaseball game\nbaseball glove\nbaseball pitcher\nbaseball team\nbaseball uniform\nbasement\nbasil\nbasin\nbasket\nbasket container\nbasketball\nbasketball backboard\nbasketball coach\nbasketball court\nbasketball game\nbasketball hoop\nbasketball player\nbasketball stadium\nbasketball team\nbass\nbass guitar\nbass horn\nbassist\nbat\nbath\nbath heater\nbath mat\nbath towel\nswimwear\nbathrobe\nbathroom\nbathroom accessory\nbathroom cabinet\nbathroom door\nbathroom mirror\nbathroom sink\ntoilet paper\nbathroom window\nbatman\nwand\nbatter\nbattery\nbattle\nbattle rope\nbattleship\nbay\nbay bridge\nbay window\nbayberry\nbazaar\nbeach\nbeach ball\nbeach chair\nbeach house\nbeach hut\nbeach towel\nbeach volleyball\nlighthouse\nbead\nbeagle\nbeak\nbeaker\nbeam\nbean\nbean bag chair\nbeanbag\nbear\nbear cub\nbeard\nbeast\nbeat\nbeautiful\nbeauty\nbeauty salon\nbeaver\nbed\nbedcover\nbed frame\nbedroom\nbedding\nbedpan\nbedroom window\nbedside lamp\nbee\nbeech tree\nbeef\nbeekeeper\nbeeper\nbeer\nbeer bottle\nbeer can\nbeer garden\nbeer glass\nbeer hall\nbeet\nbeetle\nbeige\nclock\nbell pepper\nbell tower\nbelt\nbelt buckle\nbench\nbend\nbengal tiger\nbento\nberet\nberry\nberth\nbeverage\nbib\nbibimbap\nbible\nbichon\nbicycle\nbicycle helmet\nbicycle wheel\nbiker\nbidet\nbig ben\nbike lane\nbike path\nbike racing\nbike ride\nbikini\nbikini top\nbill\nbillard\nbillboard\nbilliard table\nbin\nbinder\nbinocular\nbiology laboratory\nbiplane\nbirch\nbirch tree\nbird\nbird bath\nbird feeder\nbird house\nbird nest\nbirdbath\nbird cage\nbirth\nbirthday\nbirthday cake\nbirthday candle\nbirthday card\nbirthday party\nbiscuit\nbishop\nbison\nbit\nbite\nblack\nblack sheep\nblackberry\nblackbird\nblackboard\nblacksmith\nblade\nblanket\nsports coat\nbleacher\nblender\nblessing\nblind\neye mask\nflasher\nsnowstorm\nblock\nblog\nblood\nbloom\nblossom\nblouse\nblow\nhair drier\nblowfish\nblue\nblue artist\nblue jay\nblue sky\nblueberry\nbluebird\npig\nboard\nboard eraser\nboard game\nboardwalk\nboat\nboat deck\nboat house\npaddle\nboat ride\nbobfloat\nbobcat\nbody\nbodyboard\nbodybuilder\nboiled egg\nboiler\nbolo tie\nbolt\nbomb\nbomber\nbonasa umbellu\nbone\nbonfire\nbonnet\nbonsai\nbook\nbook cover\nbookcase\nfolder\nbookmark\nbookshelf\nbookstore\nboom microphone\nboost\nboot\nborder\nBorder collie\nbotanical garden\nbottle\nbottle cap\nbottle opener\nbottle screw\nbougainvillea\nboulder\nbouquet\nboutique\nboutique hotel\nbow\nbow tie\nbow window\nbowl\nbowling\nbowling alley\nbowling ball\nbowling equipment\nbox\nbox girder bridge\nbox turtle\nboxer\nunderdrawers\nboxing\nboxing glove\nboxing ring\nboy\nbrace\nbracket\nbraid\nbrain\nbrake\nbrake light\nbranch\nbrand\nbrandy\nbrass\nbrass plaque\nbread\nbreadbox\nbreak\nbreakfast\nseawall\nchest\nbrewery\nbrick\nbrick building\nwall\nbrickwork\nwedding dress\nbride\ngroom\nbridesmaid\nbridge\nbridle\nbriefcase\nbright\nbrim\nbroach\nbroadcasting\nbroccoli\nbronze\nbronze medal\nbronze sculpture\nbronze statue\nbrooch\ncreek\nbroom\nbroth\nbrown\nbrown bear\nbrownie\nbrunch\nbrunette\nbrush\ncoyote\nbrussels sprout\nbubble\nbubble gum\nbubble tea\nbucket cabinet\nshield\nbud\nbuddha\nbuffalo\nbuffet\nbug\nbuild\nbuilder\nbuilding\nbuilding block\nbuilding facade\nbuilding material\nlamp\nbull\nbulldog\nbullet\nbullet train\nbulletin board\nbulletproof vest\nbullfighting\nmegaphone\nbullring\nbumblebee\nbumper\nroll\nbundle\nbungee\nbunk bed\nbunker\nbunny\nbuoy\nbureau\nburial chamber\nburn\nburrito\nbus\nbus driver\nbus interior\nbus station\nbus stop\nbus window\nbush\nbusiness\nbusiness card\nbusiness executive\nbusiness suit\nbusiness team\nbusiness woman\nbusinessman\nbust\nbutcher\nbutchers shop\nbutte\nbutter\ncream\nbutterfly\nbutterfly house\nbutton\nbuttonwood\nbuy\ntaxi\ncabana\ncabbage\ncabin\ncabin car\ncabinet\ncabinetry\ncable\ncable car\ncactus\ncafe\ncanteen\ncage\ncake\ncake stand\ncalculator\ncaldron\ncalendar\ncalf\ncall\nphone box\ncalligraphy\ncalm\ncamcorder\ncamel\ncamera\ncamera lens\ncamouflage\ncamp\ncamper\ncampfire\ncamping\ncampsite\ncampus\ncan\ncan opener\ncanal\ncanary\ncancer\ncandle\ncandle holder\ncandy\ncandy bar\ncandy cane\ncandy store\ncane\njar\ncannon\ncanopy\ncanopy bed\ncantaloupe\ncantilever bridge\ncanvas\ncanyon\ncap\ncape\ncape cod\ncappuccino\ncapsule\ncaptain\ncapture\ncar\ncar dealership\ncar door\ncar interior\ncar logo\ncar mirror\nparking lot\ncar seat\ncar show\ncar wash\ncar window\ncaramel\ncard\ncard game\ncardboard\ncardboard box\ncardigan\ncardinal\ncargo\ncargo aircraft\ncargo ship\ncaribbean\ncarnation\ncarnival\ncarnivore\ncarousel\ncarp\ncarpenter\ncarpet\nslipper\nhouse finch\ncoach\ndalmatian\naircraft carrier\ncarrot\ncarrot cake\ncarry\ncart\ncarton\ncartoon\ncartoon character\ncartoon illustration\ncartoon style\ncarve\ncase\ncash\ncashew\ncasino\ncasserole\ncassette\ncassette deck\nplaster bandage\ncasting\ncastle\ncat\ncat bed\ncat food\ncat furniture\ncat tree\ncatacomb\ncatamaran\ncatamount\ncatch\ncatcher\ncaterpillar\ncatfish\ncathedral\ncattle\ncatwalk\ncatwalk show\ncauliflower\ncave\ncaviar\nCD\nCD player\ncedar\nceiling\nceiling fan\ncelebrate\ncelebration\ncelebrity\ncelery\ncello\nsmartphone\ncement\ngraveyard\ncenterpiece\ncentipede\nceramic\nceramic tile\ncereal\nceremony\ncertificate\nchain\nchain saw\nchair\nchairlift\ndaybed\nchalet\nchalice\nchalk\nchamber\nchameleon\nchampagne\nchampagne flute\nchampion\nchampionship\nchandelier\nchanging table\nchannel\nchap\nchapel\ncharacter sculpture\ncharcoal\ncharge\ncharger\nchariot\ncharity\ncharity event\ncharm\ngraph\nchase\nchassis\ncheck\ncheckbook\nchessboard\nchecklist\ncheer\ncheerlead\ncheese\ncheeseburger\ncheesecake\ncheetah\nchef\nchemical compound\nchemist\nchemistry\nchemistry lab\ncheongsam\ncherry\ncherry blossom\ncherry tomato\ncherry tree\nchess\nchestnut\nchicken\nchicken breast\nchicken coop\nchicken salad\nchicken wing\ngarbanzo\nchiffonier\nchihuahua\nchild\nchild actor\nchilds room\nchile\nchili dog\nchimney\nchimpanzee\nchinaware\nchinese cabbage\nchinese garden\nchinese knot\nchinese rose\nchinese tower\nchip\nchipmunk\nchisel\nchocolate\nchocolate bar\nchocolate cake\nchocolate chip\nchocolate chip cookie\nchocolate milk\nchocolate mousse\ntruffle\nchoir\nkitchen knife\ncutting board\nchopstick\nchristmas\nchristmas ball\nchristmas card\nchristmas decoration\nchristmas dinner\nchristmas eve\nchristmas hat\nchristmas light\nchristmas market\nchristmas ornament\nchristmas tree\nchrysanthemum\nchurch\nchurch tower\ncider\ncigar\ncigar box\ncigarette\ncigarette case\nwaistband\ncinema\nphotographer\ncinnamon\ncircle\ncircuit\ncircuit board\ncircus\nwater tank\ncitrus fruit\ncity\ncity bus\ncity hall\ncity nightview\ncity park\ncity skyline\ncity square\ncity street\ncity wall\ncity view\nclam\nclarinet\nclasp\nclass\nclassic\nclassroom\nclavicle\nclaw\nclay\npottery\nclean\nclean room\ncleaner\ncleaning product\nclear\ncleat\nclementine\nclient\ncliff\nclimb\nclimb mountain\nclimber\nclinic\nclip\nclip art\nclipboard\nclipper\nclivia\ncloak\nclogs\nclose-up\ncloset\ncloth\nclothe\nclothing\nclothespin\nclothesline\nclothing store\ncloud\ncloud forest\ncloudy\nclover\njoker\nclown fish\nclub\nclutch\nclutch bag\ncoal\ncoast\ncoat\ncoatrack\ncob\ncock\ncockatoo\ncocker\ncockpit\nroach\ncocktail\ncocktail dress\ncocktail shaker\ncocktail table\ncocoa\ncoconut\ncoconut tree\ncoffee\ncoffee bean\ncoffee cup\ncoffee machine\ncoffee shop\ncoffeepot\ncoffin\ncognac\nspiral\ncoin\ncoke\ncolander\ncold\nslaw\ncollaboration\ncollage\ncollection\ncollege student\nsheepdog\ncrash\ncolor\ncoloring book\ncoloring material\npony\npillar\ncomb\ncombination lock\ncomic\ncomedy\ncomedy film\ncomet\ncomfort\ncomfort food\ncomic book\ncomic book character\ncomic strip\ncommander\ncommentator\ncommunity\ncommuter\ncompany\ncompass\ncompete\ncontest\ncompetitor\ncomposer\ncomposition\ncompost\ncomputer\ncomputer box\ncomputer chair\ncomputer desk\nkeyboard\ncomputer monitor\ncomputer room\ncomputer screen\ncomputer tower\nconcept car\nconcert\nconcert hall\nconch\nconcrete\ncondiment\ncondom\ncondominium\nconductor\ncone\nmeeting\nconference center\nconference hall\nmeeting room\nconfetti\nconflict\nconfluence\nconnect\nconnector\nconservatory\nconstellation\nconstruction site\nconstruction worker\ncontain\ncontainer\ncontainer ship\ncontinent\nprofile\ncontract\ncontrol\ncontrol tower\nconvenience store\nconvention\nconversation\nconverter\nconvertible\ntransporter\ncook\ncooking\ncooking spray\ncooker\ncool\ncooler\ncopper\ncopy\ncoral\ncoral reef\nrope\ncorded phone\nliquor\ncorgi\ncork\ncorkboard\ncormorant\ncorn\ncorn field\ncornbread\ncorner\ntrumpet\ncornice\ncornmeal\ncorral\ncorridor\ncorset\ncosmetic\ncosmetics brush\ncosmetics mirror\ncosplay\ncostume\ncostumer film designer\ninfant bed\ncottage\ncotton\ncotton candy\ncouch\ncountdown\ncounter\ncounter top\ncountry artist\ncountry house\ncountry lane\ncountry pop artist\ncountryside\ncoupe\ncouple\ncouple photo\ncourgette\ncourse\ncourt\ncourthouse\ncourtyard\ncousin\ncoverall\ncow\ncowbell\ncowboy\ncowboy boot\ncowboy hat\ncrab\ncrabmeat\ncrack\ncradle\ncraft\ncraftsman\ncranberry\ncrane\ncrape\ncrapper\ncrate\ncrater lake\nlobster\ncrayon\ncream cheese\ncream pitcher\ncreate\ncreature\ncredit card\ncrescent\ncroissant\ncrest\ncrew\ncricket\ncricket ball\ncricket team\ncricketer\ncrochet\ncrock pot\ncrocodile\ncrop\ncrop top\ncross\ncrossbar\ncrossroad\ncrosstalk\ncrosswalk\ncrouton\ncrow\ncrowbar\ncrowd\ncrowded\ncrown\ncrt screen\ncrucifix\ncruise\ncruise ship\ncruiser\ncrumb\ncrush\ncrutch\ncrystal\ncub\ncube\ncucumber\ncue\ncuff\ncufflink\ncuisine\nfarmland\ncup\ncupcake\ncupid\ncurb\ncurl\nhair roller\ncurrant\ncurrency\ncurry\ncurtain\ncurve\npad\ncustomer\ncut\ncutlery\ncycle\ncycling\ncyclone\ncylinder\ncymbal\ncypress\ncypress tree\ndachshund\ndaffodil\ndagger\ndahlia\ndaikon\ndairy\ndaisy\ndam\ndamage\ndamp\ndance\ndance floor\ndance room\ndancer\ndandelion\ndark\ndarkness\ndart\ndartboard\ndashboard\ndate\ndaughter\ndawn\nday bed\ndaylight\ndeadbolt\ndeath\ndebate\ndebris\ndecanter\ndeck\ndecker bus\ndecor\ndecorate\ndecorative picture\ndeer\ndefender\ndeity\ndelicatessen\ndeliver\ndemolition\nmonster\ndemonstration\nden\ndenim jacket\ndentist\ndepartment store\ndepression\nderby\ndermopathy\ndesert\ndesert road\ndesign\ndesigner\ntable\ntable lamp\ndesktop\ndesktop computer\ndessert\ndestruction\ndetective\ndetergent\ndew\ndial\ndiamond\ndiaper\ndiaper bag\njournal\ndie\ndiet\nexcavator\nnumber\ndigital clock\ndill\ndinner\nrowboat\ndining room\ndinner party\ndinning table\ndinosaur\ndip\ndiploma\ndirect\ndirector\ndirt\ndirt bike\ndirt field\ndirt road\ndirt track\ndisaster\ndisciple\ndisco\ndisco ball\ndiscotheque\ndisease\nplate\ndish antenna\ndish washer\ndishrag\ndishes\ndishsoap\nDisneyland\ndispenser\ndisplay\ndisplay window\ntrench\ndive\ndiver\ndiving board\npaper cup\ndj\ndoberman\ndock\ndoctor\ndocument\ndocumentary\ndog\ndog bed\ndog breed\ndog collar\ndog food\ndog house\ndoll\ndollar\ndollhouse\ndolly\ndolphin\ndome\ndomicile\ndomino\ndonkey\ndonut\ndoodle\ndoor\ndoor handle\ndoormat\ndoorplate\ndoorway\ndormitory\ndough\ndowntown\ndozer\ndrag\ndragon\ndragonfly\ndrain\ndrama\ndrama film\ndraw\ndrawer\ndrawing\ndrawing pin\npigtail\ndress\ndress hat\ndress shirt\ndress shoe\ndress suit\ndresser\ndressing room\ndribble\ndrift\ndriftwood\ndrill\ndrink\ndrinking water\ndrive\ndriver\ndriveway\ndrone\ndrop\ndroplight\ndropper\ndrought\nmedicine\npharmacy\ndrum\ndrummer\ndrumstick\ndry\nduchess\nduck\nduckbill\nduckling\nduct tape\ndude\nduet\nduffel\ncanoe\ndumbbell\ndumpling\ndune\ndunk\ndurian\ndusk\ndust\ngarbage truck\ndustpan\nduvet\nDVD\ndye\neagle\near\nearmuff\nearphone\nearplug\nearring\nearthquake\neasel\neaster\neaster bunny\neaster egg\neat\nrestaurant\neclair\neclipse\necosystem\nedit\neducation\neducator\neel\negg\negg roll\negg tart\neggbeater\negret\nEiffel tower\nelastic band\nsenior\nelectric chair\nelectric drill\nelectrician\nelectricity\nelectron\nelectronic\nelephant\nelevation map\nelevator\nelevator car\nelevator door\nelevator lobby\nelevator shaft\nembankment\nembassy\nembellishment\nember\nemblem\nembroidery\nemerald\nemergency\nemergency service\nemergency vehicle\nemotion\nEmpire State Building\nenamel\nenclosure\nside table\nenergy\nengagement\nengagement ring\nengine\nengine room\nengineer\nengineering\nenglish shorthair\nensemble\nenter\nentertainer\nentertainment\nentertainment center\nentrance\nentrance hall\nenvelope\nequestrian\nequipment\neraser\nerhu\nerosion\nescalator\nescargot\nespresso\nestate\nestuary\neucalyptus tree\nevening\nevening dress\nevening light\nevening sky\nevening sun\nevent\nevergreen\newe\nexcavation\nexercise\nexhaust hood\nexhibition\nexit\nexplorer\nexplosion\nextension cord\nextinguisher\nextractor\nextrude\neye\neye shadow\neyebrow\neyeliner\nfabric\nfabric store\nfacade\nface\nface close-up\nface powder\nface towel\nfacial tissue holder\nfacility\nfactory\nfactory workshop\nfair\nfairground\nfairy\nfalcon\nfall\nfamily\nfamily car\nfamily photo\nfamily room\nfan\nfang\nfarm\nfarmer\nfarmer market\nfarmhouse\nfashion\nfashion accessory\nfashion designer\nfashion girl\nfashion illustration\nfashion look\nfashion model\nfashion show\nfast food\nfastfood restaurant\nfather\nfaucet\nfault\nfauna\nfawn\nfax\nfeast\nfeather\nfedora\nfeed\nfeedbag\nfeeding\nfeeding chair\nfeline\nmountain lion\nfence\nfender\nfern\nferret\nferris wheel\nferry\nfertilizer\nfestival\nfiber\nfiction\nfiction book\nfield\nfield road\nfig\nfight\nfigure skater\nfigurine\nfile\nfile photo\nfile cabinet\nfill\nfilm camera\nfilm director\nfilm format\nfilm premiere\nfilm producer\nfilming\nfilter\nfin\nhand\nfinish line\nfir\nfir tree\nfire\nfire alarm\nfire department\nfire truck\nfire escape\nfire hose\nfire pit\nfire station\nfirecracker\nfireman\nfireplace\nfirework\nfirework display\nfirst-aid kit\nfish\nfish boat\nfish market\nfish pond\nfishbowl\nfisherman\nfishing\nfishing boat\nfishing net\nfishing pole\nfishing village\nfitness\nfitness course\nfive\nfixture\nfjord\nflag\nflag pole\nflake\nflame\nflamingo\nflannel\nflap\nflare\nflash\nflask\nflat\nflatfish\nflavor\nflea\nflea market\nfleet\nflight\nflight attendant\nflip\nflip-flop\nflipchart\nfloat\nflock\nflood\nfloor\nfloor fan\nfloor mat\nfloor plan\nfloor window\nfloral arrangement\nflorist\nfloss\nflour\nflow\nflower\nflower basket\nflower bed\nflower box\nflower field\nflower girl\nflower market\nfluid\nflush\nflute\nfly\nfly fishing\nflyer\nhorse\nfoam\nfog\nfoggy\nfoie gra\nfoil\nfolding chair\nleaf\nfolk artist\nfolk dance\nfolk rock artist\nfondant\nhotpot\nfont\nfood\nfood coloring\nfood court\nfood processor\nfood stand\nfood truck\nfoosball\nfoot\nfoot bridge\nfootball\nfootball coach\nfootball college game\nfootball match\nfootball field\nfootball game\nfootball helmet\nfootball player\nfootball stadium\nfootball team\npath\nfootprint\nfootrest\nfootstall\nfootwear\nforbidden city\nford\nforehead\nforest\nforest fire\nforest floor\nforest path\nforest road\nforge\nfork\nforklift\nform\nformal garden\nformation\nformula 1\nfort\nfortification\nforward\nfossil\nfoundation\nfountain\nfountain pen\nfox\nframe\nfreckle\nhighway\nlorry\nFrench\nFrench bulldog\nFrench fries\nFrench toast\nfreshener\nfridge\nfried chicken\nfried egg\nfried rice\nfriendship\nfrisbee\nfrog\nfrost\nfrosting\nfrosty\nfrozen\nfruit\nfruit cake\nfruit dish\nfruit market\nfruit salad\nfruit stand\nfruit tree\nfruits shop\nfry\nfrying pan\nfudge\nfuel\nfume hood\nfun\nfuneral\nfungi\nfunnel\nfur\nfur coat\nfurniture\nfuton\ngadget\nmuzzle\ngalaxy\ngallery\ngame\ngame board\ngame controller\nham\ngang\ngarage\ngarage door\ngarage kit\ngarbage\ngarden\ngarden asparagus\ngarden hose\ngarden spider\ngardener\ngardening\ngarfield\ngargoyle\nwreath\ngarlic\ngarment\ngas\ngas station\ngas stove\ngasmask\ncollect\ngathering\ngauge\ngazebo\ngear\ngecko\ngeisha\ngel\ngeneral store\ngenerator\ngeranium\nghost\ngift\ngift bag\ngift basket\ngift box\ngift card\ngift shop\ngift wrap\ngig\ngin\nginger\ngingerbread\ngingerbread house\nginkgo tree\ngiraffe\ngirl\ngive\nglacier\ngladiator\nglass bead\nglass bottle\nglass bowl\nglass box\nglass building\nglass door\nglass floor\nglass house\nglass jar\nglass plate\nglass table\nglass vase\nglass wall\nglass window\nglasses\nglaze\nglider\nearth\nglove\nglow\nglue pudding\ngo\ngo for\ngoal\ngoalkeeper\ngoat\ngoat cheese\ngobi\ngoggles\ngold\ngold medal\nGolden Gate Bridge\ngolden retriever\ngoldfish\ngolf\ngolf cap\ngolf cart\ngolf club\ngolf course\ngolfer\ngoose\ngorilla\ngothic\ngourd\ngovernment\ngovernment agency\ngown\ngraduate\ngraduation\ngrain\ngrampus\ngrand prix\ngrandfather\ngrandmother\ngrandparent\ngranite\ngranola\ngrape\ngrapefruit\nwine\ngrass\ngrasshopper\ngrassland\ngrassy\ngrater\ngrave\ngravel\ngravestone\ngravy\ngravy boat\ngray\ngraze\ngrazing\ngreen\ngreenery\ngreet\ngreeting\ngreeting card\ngreyhound\ngrid\ngriddle\ngrill\ngrille\ngrilled eel\ngrind\ngrinder\ngrits\ngrocery bag\ngrotto\nground squirrel\ngroup\ngroup photo\ngrove\ngrow\nguacamole\nguard\nguard dog\nguest house\nguest room\nguide\nguinea pig\nguitar\nguitarist\ngulf\ngull\ngun\ngundam\ngurdwara\nguzheng\ngym\ngymnast\nhabitat\nhacker\nhail\nhair\nhair color\nhair spray\nhairbrush\nhaircut\nhairgrip\nhairnet\nhairpin\nhairstyle\nhalf\nhall\nhalloween\nhalloween costume\nhalloween pumpkin\nhalter top\nhamburg\nhamburger\nhami melon\nhammer\nhammock\nhamper\nhamster\nhand dryer\nhand glass\nhand towel\nhandbag\nhandball\nhandcuff\nhandgun\nhandkerchief\nhandle\nhandsaw\nhandshake\nhandstand\nhandwriting\nhanfu\nhang\nhangar\nhanger\nhappiness\nharbor\nharbor seal\nhard rock artist\nhardback book\nsafety helmet\nhardware\nhardware store\nhardwood\nhardwood floor\nmouth organ\npipe organ\nharpsichord\nharvest\nharvester\nhassock\nhat\nhatbox\nhautboy\nhawthorn\nhay\nhayfield\nhazelnut\nhead\nhead coach\nheadlight\nheadboard\nheaddress\nheadland\nheadquarter\nhearing\nheart\nheart shape\nheat\nheater\nheather\nhedge\nhedgehog\nheel\nhelicopter\nheliport\nhelmet\nhelp\nhen\nhenna\nherb\nherd\nhermit crab\nhero\nheron\nhibiscus\nhibiscus flower\nhide\nhigh bar\nhigh heel\nhighland\nhighlight\nhike\nhiker\nhiking boot\nhiking equipment\nhill\nhill country\nhill station\nhillside\nhindu temple\nhinge\nhip\nhip hop artist\nhippo\nhistorian\nhistoric\nhistory\nhockey\nhockey arena\nhockey game\nhockey player\nhockey stick\nhoe\nhole\nvacation\nholly\nholothurian\nhome\nhome appliance\nhome base\nhome decor\nhome interior\nhome office\nhome theater\nhomework\nhummus\nhoney\nbeehive\nhoneymoon\nhood\nhoodie\nhook\njump\nhorizon\nhornbill\nhorned cow\nhornet\nhorror\nhorror film\nhorse blanket\nhorse cart\nhorse farm\nhorse ride\nhorseback\nhorseshoe\nhose\nhospital\nhospital bed\nhospital room\nhost\ninn\nhot\nhot air balloon\nhot dog\nhot sauce\nhot spring\nhotel\nhotel lobby\nhotel room\nhotplate\nhourglass\nhouse\nhouse exterior\nhouseplant\nhoverboard\nhowler\nhuddle\nhug\nhula hoop\nperson\nhumidifier\nhummingbird\nhumpback whale\nhunt\nhunting lodge\nhurdle\nhurricane\nhusky\nhut\nhyaena\nhybrid\nhydrangea\nhydrant\nseaplane\nice\nice bag\npolar bear\nice cave\nicecream\nice cream cone\nice cream parlor\nice cube\nice floe\nice hockey player\nice hockey team\nlollipop\nice maker\nrink\nice sculpture\nice shelf\nskate\nice skating\niceberg\nicicle\nicing\nicon\nid photo\nidentity card\nigloo\nlight\niguana\nilluminate\nillustration\nimage\nimpala\nincense\nindependence day\nindividual\nindoor\nindoor rower\ninduction cooker\nindustrial area\nindustry\ninfantry\ninflatable boat\ninformation desk\ninfrastructure\ningredient\ninhalator\ninjection\ninjury\nink\ninking pad\ninlet\ninscription\ninsect\ninstall\ninstrument\ninsulated cup\ninteraction\ninterior design\nwebsite\nintersection\ninterview\ninvertebrate\ninvitation\nipad\niphone\nipod\niris\niron\nironing board\nirrigation system\nisland\nislet\nisopod\nivory\nivy\nizakaya\njack\njackcrab\njacket\njacuzzi\njade\njaguar\njail cell\njam\njapanese garden\njasmine\njaw\njay\njazz\njazz artist\njazz fusion artist\njeans\njeep\njelly\njelly bean\njellyfish\njet\nmotorboat\njewel\njewellery\njewelry shop\njigsaw puzzle\nrickshaw\njockey\njockey cap\njog\njoint\njournalist\njoystick\njudge\njug\njuggle\njuice\njuicer\njujube\njump rope\njumpsuit\njungle\njunkyard\nkale\nkaleidoscope\nkangaroo\nkaraoke\nkarate\nkarting\nkasbah\nkayak\nkebab\nkey\nkeycard\nkhaki\nkick\nkilt\nkimono\nkindergarden classroom\nkindergarten\nking\nking crab\nkiss\nkit\nkitchen\nkitchen cabinet\nkitchen counter\nkitchen floor\nkitchen hood\nkitchen island\nkitchen sink\nkitchen table\nkitchen utensil\nkitchen window\nkitchenware\nkite\nkiwi\nknee pad\nkneel\nknife\nrider\nknit\nknitting needle\nknob\nknocker\nknot\nkoala\nkoi\nktv\nlaboratory\nlab coat\nlabel\nlabrador\nmaze\nlace\nlace dress\nladder\nladle\nladybird\nlagoon\nlake\nlake district\nlake house\nlakeshore\nlamb\nlamb chop\nlamp post\nlamp shade\nspear\nland\nland vehicle\nlandfill\nlanding\nlanding deck\nlandmark\nlandscape\nlandslide\nlanyard\nlantern\nlap\nlaptop\nlaptop keyboard\nlarva\nlasagne\nlaser\nlash\nlasso\nlatch\nlatex\nlatte\nlaugh\nlaunch\nlaunch event\nlaunch party\nlaundromat\nlaundry\nlaundry basket\nlaundry room\nlava\nlavender\nlawn\nlawn wedding\nlawyer\nlay\nlead\nlead singer\nlead to\nleader\nleak\nlean\nlearn\nleash\nleather\nleather jacket\nleather shoe\nspeech\nlecture hall\nlecture room\nledge\nleftover\nleg\nlegend\nlegging\nlegislative chamber\nlego\nlegume\nlemon\nlemon juice\nlemonade\nlemur\nlens\nlens flare\nlentil\nleopard\nleotard\ntights\nleprechaun\nlesson\nletter\nmailbox\nletter logo\nlettering\nlettuce\nlevel\nlibrary\nlicense\nlicense plate\nlichen\nlick\nlid\nlie\nlife belt\nlife jacket\nlifeboat\nlifeguard\nlift\nlight fixture\nlight show\nlight switch\nlighting\nlightning\nlightning rod\nlilac\nlily\nlimb\nlime\nlimestone\nlimo\nline\nline art\nline up\nlinen\nliner\nlion\nlip balm\nlipstick\nliquid\nliquor store\nlist\nlitchi\nlive\nlivestock\nliving room\nliving space\nlizard\nload\nloading dock\nloafer\nhallway\nlocate\nlock\nlock chamber\nlocker\nloft\nlog\nlog cabin\nlogo\nloki\nlong hair\nlongboard\nloom\nloop\nlose\nlottery\nlotus\nlove\nloveseat\nluggage\nlumber\nlumberjack\nlunch\nlunch box\nlush\nluxury\nluxury yacht\nmac\nmacadamia\nmacaque\nmacaroni\nmacaw\nmachete\nmachine\nmachine gun\nmagazine\nmagic\nmagician\nmagnet\nmagnifying glass\nmagnolia\nmagpie\nmahjong\nmahout\nmaid\nchain mail\nmail slot\nmake\nmakeover\nmakeup artist\nmakeup tool\nmallard\nmallard duck\nmallet\nmammal\nmammoth\nman\nmanagement\nmanager\nmanatee\nmandala\nmandarin orange\nmandarine\nmane\nmanga\nmanger\nmango\nmangosteen\nmangrove\nmanhattan\nmanhole\nmanhole cover\nmanicure\nmannequin\nmanor house\nmansion\nmantid\nmantle\nmanufactured home\nmanufacturing\nmanuscript\nmap\nmaple\nmaple leaf\nmaple syrup\nmaraca\nmarathon\nmarble\nmarch\nmarching band\nmare\nmarigold\nmarine\nmarine invertebrate\nmarine mammal\npuppet\nmark\nmarket\nmarket square\nmarket stall\nmarriage\nmartial\nmartial artist\nmartial arts gym\nmartini\nmartini glass\nmascara\nmascot\nmashed potato\nmasher\nmask\nmassage\nmast\nmat\nmatador\nmatch\nmatchbox\nmaterial\nmattress\nmausoleum\nmaxi dress\nmeal\nmeasuring cup\nmeasuring tape\nmeat\nmeatball\nmechanic\nmechanical fan\nmedal\nmedia\nmedical equipment\nmedical image\nmedical staff\nmedicine cabinet\nmedieval\nmedina\nmeditation\nmeerkat\nmeet\nmelon\nmonument\nmenu\nmermaid\nnet\nmess\nmessenger bag\nmetal\nmetal artist\nmetal detector\nmeter\nmezzanine\nmicrophone\nmicroscope\nmicrowave\nmidnight\nmilestone\nmilitary uniform\nmilk\nmilk can\nmilk tea\nmilkshake\nmill\nmine\nminer\nmineral\nmineral water\nminiskirt\nminiature\nminibus\nminister\nminivan\nmint\nmint candy\nmirror\nmiss\nmissile\nmission\nmistletoe\nmix\nmixer\nmixing bowl\nmixture\nmoat\nmobility scooter\nmodel\nmodel car\nmodern\nmodern tower\nmoisture\nmold\nmolding\nmole\nmonarch\nmoney\nmonitor\nmonk\nmonkey\nmonkey wrench\nmonochrome\nmonocycle\nmonster truck\nmoon\nmoon cake\nmoonlight\nmoor\nmoose\nswab\nmoped\nmorning\nmorning fog\nmorning light\nmorning sun\nmortar\nmosaic\nmosque\nmosquito\nmoss\nmotel\nmoth\nmother\nmotherboard\nmotif\nsport\nmotor\nmotorbike\nmotorcycle\nmotorcycle helmet\nmotorcycle racer\nmotorcyclist\nmotorsport\nmound\nmountain\nmountain bike\nmountain biker\nmountain biking\nmountain gorilla\nmountain lake\nmountain landscape\nmountain pass\nmountain path\nmountain range\nmountain river\nmountain snowy\nmountain stream\nmountain view\nmountain village\nmountaineer\nmountaineering bag\nmouse\nmousepad\nmousetrap\nmouth\nmouthwash\nmove\nmovie poster\nmovie ticket\nmower\nmp3 player\nmr\nmud\nmuffin\nmug\nmulberry\nmulch\nmule\nmunicipality\nmural\nmuscle\nmuscle car\nmuseum\nmushroom\nmusic\nmusic festival\nmusic stool\nmusic studio\nmusic video performer\nmusical keyboard\nmusician\nmussel\nmustard\nmythology\nnacho\nnail polish\nnailfile\nnanny\nnapkin\nnarrow\nnational flag\nnativity scene\nnatural history museum\nnature\nnature reserve\nnavigation\nnavratri\nnavy\nnebula\nneck\nneckband\nnecklace\nneckline\nnectar\nnectarine\nneedle\nneighbor\nneighbourhood\nneon\nneon light\nnerve\nnest\nnew year\nnewborn\nnewfoundland\nnewlywed\nnews\nnews conference\nnewsstand\nnight\nnight market\nnight sky\nnight view\nnightclub\nnightstand\nnoodle\nnose\nnoseband\nnote\nnotebook\nnotepad\nnotepaper\nnotice\nnumber icon\nnun\nnurse\nnursery\nnursing home\nnut\nnutcracker\noak\noak tree\noar\noasis\noast house\noatmeal\noats\nobelisk\nobservation tower\nobservatory\nobstacle course\nsea\noctopus\noffer\noffice\noffice building\noffice chair\noffice cubicle\noffice desk\noffice supply\noffice window\nofficer\nofficial\noil\noil lamp\noil painting\noilrig\nokra\nold photo\nolive\nolive oil\nolive tree\nomelet\nonion\nonion ring\nopal\nopen\nopening\nopening ceremony\nopera\nopera house\noperate\noperating room\noperation\noptical shop\norangutan\norange\norange juice\norange tree\norangery\norbit\norchard\norchestra pit\norchid\norder\norganization\norigami\nornament\nosprey\nostrich\notter\nout\noutcrop\noutdoor\nouthouse\nelectric outlet\noutline\noval\noven\noverall\novercoat\noverpass\nowl\noyster\nteething ring\npack\npackage\npaddock\npolice van\npadlock\npaella\npagoda\npain\npaint brush\npainter\npaisley bandanna\npalace\npalette\npaling\npall\npalm tree\npan\npancake\npanda\npanel\npanorama\npansy\npant\npantry\npants\npantyhose\npapaya\npaper\npaper bag\npaper cutter\npaper lantern\npaper plate\npaper towel\npaperback book\npaperweight\nparachute\nparade\nparadise\nparrot\nparamedic\nparaquet\nparasail\nparatrooper\nparchment\nparish\npark\npark bench\nparking\nparking garage\nparking meter\nparking sign\nparliament\nparsley\nparticipant\npartner\npartridge\nparty\nparty hat\npass\npassage\npassbook\npassenger\npassenger ship\npassenger train\npassion fruit\npassport\npasta\npaste\npastry\npasture\npatch\npatient\npattern\npavement\npavilion\npaw\npay\npayphone\npea\npeace\npeach\npeacock\npeak\npeanut\npeanut butter\npear\npearl\npebble\npecan\npedestrian\npedestrian bridge\npedestrian street\npeel\npeeler\npegboard\npegleg\npelican\npen\npenalty kick\npencil\npencil case\npencil sharpener\npencil skirt\npendant\npendulum\npenguin\npeninsula\npennant\npenny\npiggy bank\npeony\npepper\npepper grinder\npeppercorn\npepperoni\nperch\nperform\nperformance\nperformance arena\nperfume\npergola\npersian cat\npersimmon\npersonal care\npersonal flotation device\npest\npet\npet shop\npet store\npetal\npetunia\nchurch bench\npheasant\nphenomenon\nphilosopher\nphone\nphonebook\nrecord player\nphoto\nphoto booth\nphoto frame\nphotography\nphysicist\nphysics laboratory\npianist\npiano\nplectrum\npick up\npickle\npicnic\npicnic area\npicnic basket\npicnic table\npicture\npicture frame\npie\npigeon\npilgrim\ntablet\npillow\npilot\npilot boat\npin\npine\npine cone\npine forest\npine nut\npineapple\ntable tennis table\ntable tennis\npink\npint\npipa\npipe\npipe bowl\npirate\npirate flag\npirate ship\npistachio\nski slope\npocket bread\npitaya\npitbull\npitch\npitcher\npitcher plant\npitchfork\npizza\npizza cutter\npizza pan\npizzeria\nplacard\nplace\nplace mat\nplaid\nplain\nplan\nplanet\nplanet earth\nplank\nplant\nplantation\nplanting\nplaque\nplaster\nplastic\nplasticine\nplateau\nplatform\nplatinum\nplatter\nplay\nplay badminton\nplay baseball\nplay basketball\nplay billiard\nplay football\nplay pong\nplay tennis\nplay volleyball\nplayer\nplayground\nplayhouse\nplaying card\nplaying chess\nplaying golf\nplaying mahjong\nplayingfield\nplaypen\nplayroom\nplaza\nplier\nplot\nplow\nplug\nplug hat\nplum\nplumber\nplumbing fixture\nplume\nplywood\npocket\npocket watch\npocketknife\npod\npodium\npoetry\npoinsettia\npoint\npointer\npoker card\npoker chip\npoker table\npole\npolecat\npolice\npolice car\npolice dog\npolice station\npolitician\npolka dot\npollen\npollution\npolo\npolo neck\npolo shirt\npomegranate\npomeranian\nponcho\npond\nponytail\npoodle\npool\npop\npop artist\npopcorn\npope\npoppy\nporcelain\nporch\npork\nporridge\nportable battery\nportal\nportfolio\nporthole\nportrait\nportrait session\npose\npossum\npost\npost office\nstamp\npostcard\nposter\nposter page\npot\npotato\npotato chip\npotato salad\npotholder\npotty\npouch\npoultry\npound\npour\npowder\npower line\npower plugs and sockets\npower see\npower station\npractice\nPrague Castle\nprayer\npreacher\npremiere\nprescription\nshow\npresentation\npresident\npress room\npressure cooker\npretzel\nprince\nprincess\nprint\nprinted page\nprinter\nprinting\nprison\nproduce\nproduct\nprofession\nprofessional\nprofessor\nproject picture\nprojection screen\nprojector\nprom\npromenade\npropeller\nprophet\nproposal\nprotective suit\nprotest\nprotester\npublication\npublicity portrait\nice hockey\npudding\npuddle\npuff\npuffin\npug\npull\npulpit\npulse\npump\npumpkin\npumpkin pie\npumpkin seed\npunch bag\npunch\nstudent\npurple\npush\nputt\npuzzle\ntower\npyramid\npython\nqr code\nquail\nquarry\nquarter\nquartz\nqueen\nquesadilla\nqueue\nquiche\nquilt\nquilting\nquote\nrabbit\nraccoon\nrace\nrace track\nraceway\nrace car\nracket\nradar\nradiator\nradio\nraft\nrag doll\nrail\nrailcar\nrailroad\nrailroad bridge\nrailway line\nrailway station\nrain\nrain boot\nrainbow\nrainbow trout\nraincoat\nrainforest\nrainy\nraisin\nrake\nram\nramp\nrapeseed\nrapid\nrapper\nraspberry\nrat\nratchet\nraven\nravine\nray\nrazor\nrazor blade\nread\nreading\nreamer\nrear\nrear light\nrear view\nrearview mirror\nreceipt\nreceive\nreception\nrecipe\nrecord\nrecord producer\nrecorder\nrecording studio\nrecreation room\nrecreational vehicle\nrectangle\nrecycling\nrecycling bin\nred\nred carpet\nred flag\nred panda\nred wine\nredwood\nreed\nreef\nreel\nreferee\nreflect\nreflection\nreflector\nregister\nrein\nreindeer\nrelax\nrelease\nrelief\nreligion\nreligious\nrelish\nremain\nremodel\nremote\nremove\nrepair\nrepair shop\nreptile\nrescue\nrescuer\nresearch\nresearcher\nreservoir\nresidence\nresidential neighborhood\nresin\nresort\nresort town\nrestaurant kitchen\nrestaurant patio\nrestroom\nretail\nretriever\nretro\nreveal\nrhinoceros\nrhododendron\nrib\nribbon\nrice\nrice cooker\nrice field\nride\nridge\nriding\nrifle\nrim\nring\nriot\nripple\nrise\nrise building\nriver\nriver bank\nriver boat\nriver valley\nriverbed\nroad\nroad sign\nroad trip\nroadside\nroast chicken\nrobe\nrobin\nrobot\nstone\nrock arch\nrock artist\nrock band\nrock climber\nrock climbing\nrock concert\nrock face\nrock formation\nrocker\nrocket\nrocking chair\nrocky\nrodent\nrodeo\nrodeo arena\nroe\nroe deer\nroller\ncoaster\nroller skate\nroller skates\nrolling pin\nromance\nromantic\nroof\nroof garden\nroom\nroom divider\nroot\nroot beer\nrope bridge\nrosary\nrose\nrosemary\nrosy cloud\nrottweiler\nround table\nrouter\nrow\nrowan\nroyal\nrubber stamp\nrubble\nrubik's cube\nruby\nruffle\nrugby\nrugby ball\nrugby player\nruins\nruler\nrum\nrun\nrunner\nrunning shoe\nrural\nrust\nrustic\nrye\nsack\nsaddle\nsaddlebag\nsafari\nsafe\nsafety vest\nsage\nsail\nsailboat\nsailing\nsailor\nsquirrel monkey\nsake\nsalad\nsalad bowl\nsalamander\nsalami\nsale\nsalmon\nsalon\nsalsa\nsalt\nsalt and pepper shakers\nsalt lake\nsalt marsh\nsalt shaker\nsalute\nsamoyed\nsamurai\nsand\nsand bar\nsand box\nsand castle\nsand sculpture\nsandal\nsandwich\nsanitary napkin\nsanta claus\nsapphire\nsardine\nsari\nsashimi\nsatay\nsatchel\nsatellite\nsatin\nsauce\nsaucer\nsauna\nsausage\nsavanna\nsaw\nsawbuck\nsax\nsaxophonist\nscaffold\nscale\nscale model\nscallop\nscar\nstrawman\nscarf\nscene\nscenery\nschnauzer\nschool\nschool bus\nschool uniform\nschoolhouse\nschooner\nscience\nscience fiction film\nscience museum\nscientist\nscissors\nwall lamp\nscone\nscoop\nscooter\nscore\nscoreboard\nscorpion\nscout\nscrambled egg\nscrap\nscraper\nscratch\nscreen\nscreen door\nscreenshot\nscrew\nscrewdriver\nscroll\nscrub\nscrubbing brush\nsculptor\nsculpture\nsea cave\nsea ice\nsea lion\nsea turtle\nsea urchin\nseabass\nseabed\nseabird\nseafood\nseahorse\nseal\nsea view\nseashell\nseaside resort\nseason\nseat\nseat belt\nseaweed\nsecretary\nsecurity\nsedan\nsee\nseed\nseesaw\nsegway\nselfie\nsell\nseminar\nsense\nsensor\nserver\nserver room\nservice\nset\nsewing machine\nshadow\nshake\nshaker\nshampoo\nshape\nshare\nshark\nsharpener\nsharpie\nshaver\nshaving cream\nshawl\nshear\nshears\nsheep\nsheet\nsheet music\nshelf\nshell\nshellfish\nshelter\nshelve\nshepherd\nsherbert\nshiba inu\nshine\nshipping\nshipping container\nshipwreck\nshipyard\nshirt\nshirtless\nshoal\nshoe\nshoe box\nshoe shop\nshoe tree\nshoot\nshooting basketball guard\nshop window\nshopfront\nshopper\nshopping\nshopping bag\nshopping basket\nshopping cart\nmall\nshopping street\nshore\nshoreline\nshort\nshort hair\nshorts\nshot glass\nshotgun\nshoulder\nshoulder bag\nshovel\nshowcase\nshower\nshower cap\nshower curtain\nshower door\nshower head\nshredder\nshrew\nshrimp\nshrine\nshrub\nshutter\nsiamese\nsiberia\nsibling\nside\nside cabinet\nside dish\nsidecar\nsideline\nsiding\nsign\nsignage\nsignal\nsignature\nsilk\nsilk stocking\nsilo\nsilver\nsilver medal\nsilverware\nsing\nsinge\nsinger\nsink\nsip\nsit\nsitting\nskate park\nskateboard\nskateboarder\nskater\nskating rink\nskeleton\nsketch\nskewer\nski\nski boot\nski equipment\nski jacket\nski lift\nski pole\nski resort\nsnowboard\nskier\nskiing shoes\nskin\nskull\nskullcap\nsky\nsky tower\nskylight\nskyline\nskyscraper\nslalom\nslate\nsleigh\nsleep\nsleeping bag\nsleepwear\nsleeve\nslice\nslide\nslider\nsling\nslope\nslot\nslot machine\nsloth\nslow cooker\nslug\nslum\nsmell\nsmile\nsmoke\nsnack\nsnail\nsnake\nsnapper\nsnapshot\nsnorkel\nsnout\nsnow\nsnow leopard\nsnow mountain\nsnowball\nsnowboarder\nsnowfield\nsnowflake\nsnowman\nsnowmobile\nsnowplow\nsnowshoe\nsnowy\nsoap\nsoap bubble\nsoap dispenser\nsoccer goalkeeper\nsocialite\nsock\nsocket\nsoda\nsoftball\nsoftware\nsolar battery\nsoldier\nsolo\nsolution\nsombrero\nsong\nsound\nsoup\nsoup bowl\nsoupspoon\nsour cream\nsouvenir\nsoybean milk\nspa\nspace\nspace shuttle\nspace station\nspacecraft\nspaghetti\nspan\nwrench\nspark\nsparkle\nsparkler\nsparkling wine\nsparrow\nspatula\nspeaker\nspectator\nspeech bubble\nspeed limit\nspeed limit sign\nspeedboat\nspeedometer\nsphere\nspice\nspice rack\nspider\nspider web\nspike\nspin\nspinach\nspire\nsplash\nsponge\nspoon\nsport association\nsport equipment\nsport team\nsports ball\nsports equipment\nsports meet\nsportswear\ndot\nspray\nspread\nspring\nspring roll\nsprinkle\nsprinkler\nsprout\nspruce\nspruce forest\nsquad\nsquare\nsquash\nsquat\nsqueeze\nsquid\nsquirrel\nwater gun\nstab\nstable\nstack\nstadium\nstaff\nstage\nstage light\nstagecoach\nstain\nstainless steel\nstair\nstairs\nstairwell\nstall\nstallion\nstand\nstanding\nstaple\nstapler\nstar\nstare\nstarfish\nstarfruit\nstarling\nstate park\nstate school\nstation\nstationary bicycle\nstationery\nstatue\nsteak\nsteak knife\nsteam\nsteam engine\nsteam locomotive\nsteam train\nsteamed bread\nsteel\nsteering wheel\nstem\nstencil\nstep stool\nstereo\nstethoscope\nstew\nstick\nstick insect\nsticker\nstill life\nstilt\nstingray\nstir\nstirrer\nstirrup\nsew\nstock\nstocking\nstomach\nstone building\nstone carving\nstone house\nstone mill\nstool\nstop\nstop at\nstop light\nstop sign\nstop watch\ntraffic light\nstorage box\nstorage room\ntank\nstore\nstorefront\nstork\nstorm\nstorm cloud\nstormy\nstove\npoker\nstraddle\nstrainer\nstrait\nstrap\nstraw\nstraw hat\nstrawberry\nstream\nstreet art\nstreet artist\nstreet corner\nstreet dog\nstreet food\nstreet light\nstreet market\nstreet photography\nstreet scene\nstreet sign\nstreet vendor\nstretch\nstretcher\nstrike\nstriker\nstring\nstring cheese\nstrip\nstripe\nstroll\nstructure\nstudio\nstudio shot\nstuff\nstuffed animal\nstuffed toy\nstuffing\nstump\nstunning\nstunt\nstupa\nstyle\nstylus\nsubmarine\nsubmarine sandwich\nsubmarine water\nsuburb\nsubway\nsubway station\nsubwoofer\nsucculent\nsuede\nsugar\nsugar bowl\nsugar cane\nsugar cube\nsuit\nsuite\nsummer\nsummer evening\nsummit\nsun\nsun hat\nsunbathe\nsunday\nsundial\nsunflower\nsunflower field\nsunflower seed\nsunglasses\nsunny\nsunrise\nsunset\nsunshade\nsunshine\nsuper bowl\nsports car\nsuperhero\nsupermarket\nsupermarket shelf\nsupermodel\nsupporter\nsurf\nsurface\nsurfboard\nsurfer\nsurgeon\nsurgery\nsurround\nsushi\nsushi bar\nsuspenders\nsuspension\nsuspension bridge\nsuv\nswallow\nswallowtail butterfly\nswamp\nswan\nswan boat\nsweat pant\nsweatband\nsweater\nsweatshirt\nsweet\nsweet potato\nswim\nswim cap\nswimmer\nswimming hole\nswimming pool\nswing\nswing bridge\nswinge\nswirl\nswitch\nswivel chair\nsword\nswordfish\nsymbol\nsymmetry\nsynagogue\nsyringe\nsyrup\nsystem\nt shirt\nt-shirt\ntabasco sauce\ntabby\ntable tennis racket\ntable top\ntablecloth\ntablet computer\ntableware\ntachometer\ntackle\ntaco\ntae kwon do\ntai chi\ntail\ntailor\ntake\ntakeoff\ntalk\ntambourine\ntan\ntangerine\ntape\ntapestry\ntarmac\ntaro\ntarp\ntart\ntassel\ntaste\ntatami\ntattoo\ntattoo artist\ntavern\ntea\ntea bag\ntea party\ntea plantation\ntea pot\ntea set\nteach\nteacher\nteacup\nteal\nteam photo\nteam presentation\ntear\ntechnician\ntechnology\nteddy\ntee\nteenager\ntelegraph pole\nzoom lens\ntelescope\ntelevision\ntelevision camera\ntelevision room\ntelevision studio\ntemperature\ntemple\ntempura\ntennis\ntennis court\ntennis match\ntennis net\ntennis player\ntennis racket\ntent\ntequila\nterminal\nterrace\nterrain\nterrarium\nterritory\ntest\ntest match\ntest tube\ntext\ntext message\ntextile\ntexture\nthanksgiving\nthanksgiving dinner\ntheater\ntheatre actor\ntherapy\nthermometer\nthermos\nthermos bottle\nthermostat\nthicket\nthimble\nthing\nthinking\nthistle\nthrone\nthrone room\nthrow\nthrow pillow\nthunder\nthunderstorm\nthyme\ntiara\ntick\nticket\nticket booth\ntide pool\ntie\ntiger\ntight\ntile\ntile flooring\ntile roof\ntile wall\ntin\ntinfoil\ntinsel\ntiramisu\ntire\ntissue\ntoast\ntoaster\ntobacco\ntobacco pipe\ntoddler\ntoe\ntofu\ntoilet bowl\ntoilet seat\ntoiletry\ntokyo tower\ntomato\ntomato sauce\ntomato soup\ntomb\ntong\ntongs\ntool\ntoolbox\ntoothbrush\ntoothpaste\ntoothpick\ntopiary garden\ntopping\ntorch\ntornado\ntortilla\ntortoise\ntote bag\ntotem pole\ntotoro\ntoucan\ntouch\ntouchdown\ntour\ntour bus\ntour guide\ntourist\ntourist attraction\ntournament\ntow truck\ntowel\ntowel bar\ntower block\ntower bridge\ntown\ntown square\ntoy\ntoy car\ntoy gun\ntoyshop\ntrack\ntractor\ntrade\ntradition\ntraditional\ntraffic\ntraffic cone\ntraffic congestion\ntraffic jam\ntraffic sign\ntrail\ntrailer\ntrailer truck\ntrain\ntrain bridge\ntrain car\ntrain interior\ntrain track\ntrain window\ntrainer\ntraining\ntraining bench\ntraining ground\ntrolley\ntrampoline\ntransformer\ntransparency\ntravel\ntray\ntreadmill\ntreat\ntree\ntree branch\ntree farm\ntree frog\ntree house\ntree root\ntree trunk\ntrial\ntriangle\ntriathlon\ntribe\ntributary\ntrick\ntricycle\ntrim\ntrio\ntripod\ntrombone\ntroop\ntrophy\ntrophy cup\ntropic\ntrout\ntruck\ntruck driver\ntub\ntube\ntugboat\ntulip\ntuna\ntundra\ntunnel\nturbine\nturkey\nturn\nturnip\nturquoise\nturret\nturtle\ntusk\ntv actor\ntv cabinet\ntv drama\ntv genre\ntv personality\ntv show\ntv sitcom\ntv tower\ntwig\ntwilight\ntwin\ntwine\ntwist\ntype\ntype on\ntypewriter\nukulele\nultraman\numbrella\nunderclothes\nunderwater\nunicorn\nuniform\nuniverse\nuniversity\nup\nurban\nurinal\nurn\nuse\nutensil\nutility room\nvacuum\nvalley\nvalve\nvampire\nvan\nvanilla\nvanity\nvariety\nvase\nvault\nvector cartoon illustration\nvector icon\nvegetable\nvegetable garden\nvegetable market\nvegetation\nvehicle\nveil\nvein\nvelvet\nvending machine\nvendor\nvent\nvespa\nvessel\nvest\nvet\nveteran\nveterinarians office\nviaduct\nvideo\nvideo camera\nvideo game\nvideotape\nview mirror\nvigil\nvilla\nvillage\nvine\nvinegar\nvineyard\nviolence\nviolet\nviolin\nviolinist\nviolist\nvision\nvisor\nvodka\nvolcano\nvolleyball\nvolleyball court\nvolleyball player\nvolunteer\nvoyage\nvulture\nwaffle\nwaffle iron\nwagon\nwagon wheel\nwaist\nwaiter\nwaiting hall\nwaiting room\nwalk\nwalking\nwalking cane\nwall clock\nwallpaper\nwalnut\nwalrus\nwar\nwarehouse\nwarm\nwarning sign\nwarrior\nwarship\nwarthog\nwash\nwasher\nwashing\nwashing machine\nwasp\nwaste\nwaste container\nwatch\nwater\nwater bird\nwater buffalo\nwater cooler\nwater drop\nwater feature\nwater heater\nwater level\nwater lily\nwater park\nwater pipe\nwater purifier\nwater ski\nwater sport\nwater surface\nwater tower\nwatercolor\nwatercolor illustration\nwatercolor painting\nwaterfall\nwatering can\nwatermark overlay stamp\nwatermelon\nwaterproof jacket\nwaterway\nwave\nwax\nweapon\nwear\nweather\nvane\nweb\nwebcam\nwedding\nwedding ring\nwedding bouquet\nwedding cake\nwedding couple\nwedding invitation\nwedding party\nwedding photo\nwedding photographer\nwedding photography\nwedding reception\nwedge\nweed\nweight\nweight scale\nwelder\nwell\nwestern food\nwestern restaurant\nwet\nwet bar\nwet suit\nwetland\nwetsuit\nwhale\nwhale shark\nwheat\nwheat field\nwheel\nwheelchair\nwheelie\nwhipped cream\nwhisk\nwhisker\nwhiskey\nwhistle\nwhite\nwhite house\nwhite wine\nwhiteboard\nwicket\nwide\nwield\nwig\nWii\nWii controller\nwild\nwildebeest\nwildfire\nwildflower\nwildlife\nwillow\nwind\nwind chime\nwind farm\nwind turbine\nwindmill\nwindow\nwindow box\nwindow display\nwindow frame\nwindow screen\nwindow seat\nwindow sill\nwiper\nwindshield\nwindy\nwine bottle\nwine cooler\nwine cabinet\nwine cellar\nwine glass\nwine rack\nwine tasting\nwinery\nwing\nwinter\nwinter melon\nwinter morning\nwinter scene\nwinter sport\nwinter storm\nwire\nwisteria\nwitch\nwitch hat\nwok\nwolf\nwoman\nwood\nwood duck\nwood floor\nwood wall\nwood-burning stove\nwooden spoon\nwoodland\nwoodpecker\nwoodworking plane\nwool\njob\nwork card\nworkbench\nworker\nworkplace\nworkshop\nworld\nworm\nworship\nwound\nwrap\nwrap dress\nwrapping paper\nwrestle\nwrestler\nwrinkle\nwristband\nwrite\nwriter\nwriting\nwriting brush\nwriting desk\nyacht\nyak\nyard\nyellow\nyoga\nyoga mat\nyoghurt\nyoke\nyolk\nyouth\nyouth hostel\nyurt\nzebra\nzebra crossing\nzen garden\nzip\nzipper\nzombie\nzongzi\nzoo"
},
{
"path": "ram/data/ram_tag_list_chinese.txt",
"content": "äøē»“CGęø²ę \n3dē¼é\nē®ē \né²é±¼ \näæ®éé¢ \nčå \nå¦é¢ \né件 \näŗę
\nęé£ē“ \nę©”å \näøēÆé¢ę\nč”Øę¼\nč”åØ \nåØä½ēµå½± \nę“»åØ \nę¼å \nę¹ē¼ę¬\nę·»å \nč¶åø¦ \nč°ę“ \nęäŗŗ \nåé© \nå¹æå \n天ēŗæ \nęę°§čæåØ \nå·é¾ē½\nēēø夓\nåäø \nåø®å©\nē©ŗč° \nē©ŗč°ē³»ē»\né£åę \né£ęŗå®¢č± \né£ęŗęØ”å \nęŗåŗ \nčŖēŗæ\n客ęŗ \né£č”å \né£ęŗ \né£ęŗēŖå£ \nęŗåŗ \nęŗåŗč·é \nčŖē«ę„¼ \né£č \nčŖå±\nčæé \nč¦ę„\né¹é \näæ”天ēæ \nå±ē\nå±ēå°é¢ \né
ē²¾ \nå£é¾\nę°“č»\nč”å/ēé\nęä» \nč¦č\né«å±± \nē¾é©¼ \nåęÆč”Ø\nå¾·å½ē§ē¾ē¬\nå£å\nē„ē\nęę¤č½¦ \nē§é¹°\nē¾å½ēęÆē« \nē“«ę°“ę¶ \nåå½¢å§åŗ \nę©é³åØ\nęøøä¹å \nęøøä¹č®¾ę½ \né \nå¤čē \nęµ·čµ \n天ä½æ \nč§ \nåØē© \nåØē©éå” \nåØē©ę¶å®¹ę \nåØē»ē\nåØē»ēµå½± \nåØē»åø \nåØę¼« \nččø \nēč¢ \nåØ幓åŗ\né£č”£ \nčč \nē¾ē¾ \nå¤č£ \né¹æč§ \néē § \nå
¬åÆ \nēæ \nåŗēØēØåŗ \nåŗēØå¾ę \nåŗē° \nå¤č§ \nå¼čč \nę声 \nč¹ę \nč¹ęę± \nč¹ęę“¾ \nč¹ęę \nč¹ęé
± \nč®¾å¤ \nēŗ¦å®\néé\nęå\nå“č£ \nęµ
ē»æč²\nę°“ęé¦ \nč§čµé±¼\nęø”ę§½ \nęøøä¹äøåæ\nååŗęøøęęŗ\nę±éØ\nę±ę”„ \nčå¤ē°åŗ\nå°ē® \nē¾¤å² \nå»ŗēåø\nå»ŗēč®¾č®”\nę”£ę”\nę±éØ \nå°åŗ\nē«ęåŗ \näŗč®ŗ\nęč \nē©æå±±ē²\nčē« \nę¶ęę¤
\nč”£ę\nēē²\nåé \nåäŗåŗå° \nå¦å
\néµå\né®ę \nē®å¤“ \nčŗęÆ \nčŗęÆå±\nē¾ęÆé¦\nčŗęÆå°å·å\nčŗęÆå¦ę ” \nčŗęÆå·„ä½å®¤ \nčŗęÆē¢éęå¾ \nę“č \nęē« \nęå·„čŗå\nčŗęÆ家 \nčŗęÆéę„¼ \nē° \nēē°ē¼ø \näŗę“²åÆŗåŗ \nč¦ē¬ \nę²„ééč·Æ\nē»č£
\néä¼\nēäŗ§ęµę°“ēŗæ\nåä¼ \nå®čŖå \n天ęå¦å®¶ \nčæåØå \nčæåØ \nå°å¾é\nčŖå©åę¬¾ęŗ \n大ę°å±\näøåŗ \nčæę„\nęęęŗ \nåå \nåøå¼å \nå
Øå°å½¢č½¦\nčå \nęå \nå„„čæŖę±½č½¦\né³é¢ \nē¤¼å \nęå
\nä½č
\nę±½č½¦å \nę±½č½¦äæ®ēå·„ \nę±½č½¦é¶ä»¶\nč½¦å± \nę±½č½¦å±å
\nę±½č½¦ēµę± \nę±½č½¦å¶é \nę±½č½¦ęØ”å \nę±½č½¦\nē§å¤© \nē§å¤©ēę£®ę \nē§å¤©ēå¶å\nē§å¤©ēå
¬å \nē§å¤©ēę \néæå”č¾¾\nęč«å¤§é\né£č”åå¤Ŗé³é \nēę²¹ę\nå„å\né¢å„å
øē¤¼ \nč·å„č
\nę£\nę§å¤“ \nęé¹č± \nēē \nå©“åæ \n儶ē¶ \nå©“åæč½¦ \nå©“åæč”£ę \nå°č±” \nå©“åæé£å \nå©“åæåŗ§ę¤
\nčæå©“ę“¾åƹ\nčå/åé¢\nčęÆ \nčå
\nčå
\nåé¢ \nå¹ę ¹ \nå¾½ē« \nē¾ \nčå° \nē¾½ęÆēčæåØ\nē¾½ęÆēę \nč¢å\né¢å
å\né£ē¬ \nę³ę£\nčƱ鄵 \nēē¤é£å \né¢å
åø\né¢å
åŗ \nēē \nē¤ē \nå¹³č”” \nå¹³č””č½¦ \né³å° \nē \nēę± \nčč¾č儳ę¼å \nčč¾č \nčč¾čę¼å \nčč¾čč£ \nę°ē \nę°ēę±éØ \nę£ēę \nčå
\nē«¹å \nē«¹ę \né¦č \né¦čé¢å
\né¦čå¶å \né¦čę \nä¹é \nååÆč““ \nē»·åø¦ \n夓巾 \nęååø¦ \nåęµ·\nęéÆ \nę ę\näŗ弦ē“\né¶č” \né¶č”å” \né¶č”éåŗ\nēŗøåø\nęØŖå¹
/ęåø\nå®“ä¼ \n宓ä¼å
\nę¦ę \nå
å \nę“ē¤¼ \né
å§ \nę”å½¢ē \né«čå³\nē§ē¤ \nē§ē¤ę¶ \nę é \nēååø \nēååŗ \nčęÆåØåØ \né©³č¹ \nåå”åø \nę ē® \n大éŗ¦ \nč°·ä» \nä»éø®\nę”å
ęæ\nꔶ \nč·Æé\nå±é\nęęØč½¦ \né
äæ \nę£ē \nę£ēåŗå° \nę£ēę£ \nę£ēåø½ \nę£ēåŗ\nę£ēęÆčµ \nę£ēęå„ \nę£ēęę \nę£ēé \nę£ēå¶ę \nå°äø室 \nē½å \nę°“ē\nēÆ®å \nēÆ®å\nēÆ®ē \nēÆ®ēēÆ®ęæ \nēÆ®ēęē» \nēÆ®ēåŗ \nēÆ®ēęÆčµ \nēÆ®ēę”\nēÆ®ēčæåØå \nēÆ®ēé¦ \nēÆ®ēé \nč“ęÆ\nä½é³åä» \nä½é³åå \nč“ęÆę \nēę£/ēę\nęµ“å®¤\n갓굓å ēåØ\nęµ“å« \nęµ“å·¾ \nę³³č£
\nęµ“č¢ \nęµ“å®¤ \nęµ“å®¤é
件 \nęµ“å®¤ę \nęµ“å®¤éØ \nęµ“å®¤éå\nęµ“å®¤ę°“ę§½ \nå«ēēŗø\nęµ“å®¤ēŖę·\nčč ä¾ \nę£å\nę„čæēę/å»ēå\nēµę± \nęę \nęē»³\nęč° \nęµ·ę¹¾\nęµ·ę¹¾å¤§ę”„ \nåøēŖ \nęØę¢
\néåø \n굷껩 \nę²ę»©ē \nę²ę»©ę¤
\nęµ·ę»Øå«å¢
\n굷껩å°å± \nę²ę»©ęÆå·¾ \nę²ę»©ęē \nēÆå” \nē å\nęÆę ¼ē¬\néøå“\nē§ęÆ \nęØŖę¢\nč±å\nč±č¢ę¤
\nč±č¢ \nē \nå¹¼ē \nč”å \néå
½ \nå»ę/å»č“„\nē¾äø½ē \nē¾äø½\nē¾å®¹é¢ \nęµ·ēø \nåŗ\nåŗå\nåŗę¶\nå§å®¤ \nåŗäøēØå \nä¾æē \nå§å®¤ēŖę· \nåŗ夓ēÆ \nčč \nå±±ęÆę¦\nēč \nå
»čäŗŗ \nčéø£åØ \nå¤é
\nå¤é
ē¶ \nå¤é
ē½ \nå¤é
č±å \nå¤é
ęÆ \nå¤é
é¦\nēč \nē²č« \nē±³č² \nę¶é \nēę¤ \néę„¼ \nē®åø¦\nē®åø¦ę£ \néæå³\nå¼Æę² \nåå ęč \nēé„ \nč“é·åø½ \nęµę \nåę³ä½ \né„®ę \nå“å“ \nęé„ \nå£ē» \nęÆē\nčŖč”č½¦ \nčŖč”č½¦å¤“ē \nčŖč”č½¦č½¦č½® \nčŖč”č½¦éŖ士\nå굓ē \n大ę¬é \nčŖč”č½¦é \nčŖč”č½¦é \nčŖč”č½¦čµ\néŖč½¦\nęÆåŗå°¼ \nęÆåŗå°¼äøč”£\n蓦å\nå°ē\nå¹æåē \nå°ēå°\nåå¾ē®±\nę“»é”µå¤¹\nåēęčæé \nēē©å¦å®éŖ室 \nåēæ¼é£ęŗ \nꔦęØ \nꔦę \néø \néøę± \nåéøåØ \néøč\néøå·¢ \néøę± \néøē¬¼ \nåŗē \nēę„ \nēę„čē³ \nēę„č”ē \nēę„č“ŗå” \nēę„čä¼ \n鄼干 \näø»ę \néē \né»å¤“\nå¬ \né»č²\né»å±±ē¾\né»č \nä¹éø¦\né»ęæ\néå \nå¶ē/åē\nęÆÆå/č¦ēå±\nčæåØå¤å„ \nēå°\nę
ęęŗ \nē„ē¦ \nēŖåø\nē¼ē½© \néŖå
\nę“é£éŖ \nå \nå客 \nč” \nå¼č±\nč±\n儳č£
蔬蔫\nå¹\nå¹é£ęŗ\nę²³č± \nčč²\nčč²čŗęÆ家\nčę¾éø¦\nč天\nčč \nčē„ę“éø \nēŖ \nęæå\nęæę¦\nę£ēęøøę \nęØęæč·Æ \nč¹ \nč¹ē²ęæ\nč¹å±\nę”Ø \nä¹č¹ \nęµ®ę \nå±±ē« \nčŗÆå¹²\nčŗ«ä½å²ęµŖęæ\nå„ē¾čæåØå \nę°“ē
®éø”č \né
ē \né„°ę£å¼é¢åø¦\néØé©\nēøå¼¹ \nč½°ēøęŗ \nę«č©ę¦éø”\néŖØéŖ¼\nēÆē« \néē \nēęÆ \n书 \n书ē±å°é¢\n书ę \nę件夹 \n书ē¾ \n书ę¶\n书åŗ \nčæēØę¾é³åØ\nęØåØ\né“å\nč¾¹ē\nč¾¹å¢ē§ē¾ē¬ \nę¤ē©å \nē¶ \nē¶ē \nå¼ē¶åØ \nčŗęå¼ē¶åØ\näøč§ę¢
\nå·Øē³\nč±ę \nę¶č£
åŗ\nē²¾åé
åŗ \né čŗ¬/č“č¶ē»\né¢ē» \nå¼å½¢ēŖ\nē¢ \näæé¾ēčæåØ\näæé¾ēé¦ \näæé¾ē \näæé¾ēč®¾å¤ \nēå \nē®±å½¢ę¢ę”„\nē®±é¾ \nę³å»ę \nå
č£¤ \nę³å» \nę³å»ęå„ \nę³å»å°\nē·å© \nęÆęē©\nęÆę¶ \nč¾«å\n大č \nå¹č½¦ \nå¹č½¦ēÆ \nę ę\nåę \nē½å
°å° \né»é \né»éēå¾ \né¢å
\né¢å
ē®± \nä¼ęÆ\nę©é¤ \né²ęµŖå ¤ \nčøéØ\nå¤é
å \nē å\nē å»ŗēē©\nå¢ \nē å\nå©ēŗ± \nę°åØ \nę°é \n伓åØ \nꔄ \nē¼°ē»³ \nå
¬ęå
\nęäŗ®ē \nč¾¹ę²æ\né»å¤“\nå¹æę \nč„æå
°č± \néé \néē \néééå” \néééå \nčøé \nå°ęŗŖ\nę«åø \nč걤 \nę£č²\nę£ē \nå·§å
åčē³ \nę©åé¤ \nęµ
é»č¤č²ē儳äŗŗ \nå·å\néē¼\nå
č\nę°ę³”\nę³”ę³”ē³ \nēē 儶č¶\nęę \nē¾ē\nč½ \nä½ \nę°“ē \nčŖå©é¤ \nęč«\nå»ŗé \nå»ŗé č
\nå»ŗē \nē§ÆęØ\nå»ŗēē«é¢ \nå»ŗēęę \nēÆ \nē \nęēē¬ \nåå¼¹ \nåØč½¦ \nå
¬åę \né²å¼¹čåæ \nęē \nę©é³åØ \nęēåŗ \n大é»č \näæé©ę \nå·/å°å½¢čµ·ä¼\nę\nč¹¦ę \nåå±åŗ \nå°å ”/å»ē\nå
å \nęµ®ę \n书ę”\nå¢å®¤ \nēē§ \nēē±³ē
鄼 \nå
¬äŗ¤č½¦\nå
¬äŗ¤č½¦åøęŗ\nå
¬äŗ¤č½¦å
éØ\nå
¬äŗ¤č½¦ē« \nå
¬äŗ¤č½¦ē« \nå
¬äŗ¤č½¦ēŖę·\nēęØ\nåäø\nåē \näøå”äø»ē®” \nåå”č„æč£
\näøå”å¢é \n儳åäŗŗ\nåäŗŗ \nåčŗ«å\nå± å¤« \nčéŗ\nå¤å³° \né»ę²¹ \nå„¶ę²¹ \nč“č¶ \nč“č¶é¦\nęé® \nę¢§ę”ę \nč“ä¹°\nåŗē§č½¦ \nå°å± \nå·åæč \nå°å±/ęŗč±\nå®č½¦\nåØčę\nꩱę\nēµē¼ \nē¼č½¦ \nä»äŗŗę \nåå”é¦ \né£å \nē¬¼å\nčē³ \nčē³å°\nč®”ē®åØ \n大é
\nę„å \nå°č
æ \néčÆ\nēµčÆäŗ\nä¹¦ę³ \nå¹³éē\nęåęŗ \néŖ驼 \nēøęŗ \nēøęŗé夓 \nčæ·å½©\né²č„\né²č„č
\nēÆē« \né²č„\nč„å° \nę ”å \nē½\nå¼ē½åØ \nčæę²³ \néäøé \nēē \nč”ē \nēå° \nē³ę \nåē¶ē³\nęŗęē³ \nē³ęåŗ \nęę\nē½å\n大ē® \nę å /锶ę£\nåę±åŗ\né¦ē \nę¬čꔄ \nåøåø \nå³”č°· \nåø½å\nęēÆ·\nē§å¾·č§ \nå”åøå„čÆŗ\nč¶å \nééæ \nęč· \nč½¦ \nę±½č½¦ē»éå \nč½¦éØ \nę±½č½¦å
é„°\nč½¦ę \nåč§é\nåč½¦åŗ \nę±½č½¦åŗ§ę¤
\nč½¦å±\nę“č½¦ \nč½¦ēŖ \nē¦ē³ \nå”ē\nēŗøēęøøę\nēŗøęæ \nēŗøęæē \nē¾ęÆ蔫\nēŗ¢č”£å¤å¤“éø\n蓧ē© \n蓧čæé£ęŗ \nč“§č¹ \nå åęÆ \nåŗ·ä¹é¦Ø \nēę¬¢č \né£čåØē© \nęč½¬ęØ马 \n鲤鱼 \nęØå \nå°ęÆÆ \nęé \nēŗ¢é\néæéå®¢č½¦\nęē¹ē\nčŖē©ŗęÆč°\nč”čå \nč”čåčē³ \nęŗåø¦ \nęęØč½¦\nēŗøē®±/ēŗøē\nå”é \nå”éäŗŗē© \nå”éęå¾ \nå”éé£ę ¼ \néå» \n容åØ\nē°é \nč
°ę \nčµåŗ \nē é
\nē£åø¦ \nēå¼å½é³ęŗ\nē³čē»·åø¦\néøé \nåå ” \nē« \nē«ēŖ\nē«ē²® \nē«åØå
·\nē«ę¶\nå°äøå¢ē©“ \nåä½č¹ \nē¾ę“²ē®\nę”ē/ęē\nęę \nęÆęÆč« \n鲶鱼 \nęå \nē \nē«ę„ \nčµ°ē§ \nčč± \nę“ē©“ \né±¼åé
± \nå
ē\nCDęę¾åØ\néŖę¾ \n天č±ęæ \nåę \nåŗē„ \nåŗå
ø\nåäŗŗ \nč¹č \n大ęē“ \nęęŗ \nę°“ę³„ \nå¢å° \näøåæč£
é„°å\nčč£ \né¶ē· \nē·ē \néŗ¦ē \nä»Ŗå¼ \nčÆ书 \né¾ę”\né¾éÆ \nę¤
å \nåéę¤
\nčŗŗę¤
\nęØå±\nå£ęÆ\nē²ē¬ \nęæé“\nåč²é¾ \né¦ę§é
\né¦ę§ęÆ\nå å\né¦ę čµ\nåēÆ\nå©“åæę¢å°æåøå°\néé\nē“č£å¤\nå°ęå \näŗŗē©éå”\nęØē\nå
ēµ\nå
ēµåØ\nęč½¦\nę
åęŗę\nę
åę“»åØ\né
å\nå¾č”Ø\nčæ½é\nåŗē\nę£ę„/ęÆē„Ø\nęÆē„Øē°æ\nę£ē\nę£ę„č”Ø\nę¬¢å¼å£°\né¼å±/å¦å¦é\n儶é
Ŗ\n儶é
Ŗę±å ”\n儶é
Ŗčē³\nēč±¹\nåØåø\nååē©\nåå¦å®¶\nåå¦\nåå¦å®éŖ室\nęč¢\nęرę”\nęرč±\nęرę”ēŖč\nęرę”ę \nå½é
č±”ę£\nę å\néø”\néø”čøč\néø”ē¬¼\néø”čę²ę\néø”ēæ
\né¹°å“č±\nå°č”£ę©±\nååØåØ\nå©å\nē«„ę\nå©åēęæé“\nēŗ¢ēŖę¤\nč¾£ēē\nēå±\né»ē©ē©\nē·åØ\nē½č\näøå½åę\näøå½ē»\nęå£\näøå½å”\nēøčÆę”/ēøčÆę”\nč±ę é¼ \nåæå\nå·§å
å\nå·§å
åę£\nå·§å
åčē³\nå·§å
åē¢ē\nå·§å
å鄼干\nå·§å
åē儶\nå·§å
åę
ęÆ\nę¾é²\nå±čÆē\nåØęæå\nē §ęæ\nē·å\nå£čÆč\nå£čÆē\nå£čÆč“ŗå”\nå£čÆč£
é„°\nå£čÆę宓\nå¹³å®å¤\nå£čÆåø½\nå£čÆēÆ\nå£čÆåøåŗ\nå£čÆč£
é„°\nå£čÆę \nčč±\nęå \nęå å”\nč¹ęé
\néŖč\néŖčē\né¦ē\nēē\nč
°åø¦\nēµå½±é¢\nęå½±åø\nčę”\nå\nēµč·Æ\nēµč·Æęæ\n马ęå¢\nę°“ē®±\nęę©ē±»ę°“ę\nååø\nååøå
¬äŗ¤\nåøęæå
\nååøå¤ęÆ\nååøå
¬å\nååø天é
ēŗæ\nååøå¹æåŗ\nååøč”é\nåå¢\nååøęÆč§\nč¤č\nåē°§ē®”\nę£å\nēēŗ§\nē»å
ø\nę室\nééŖØ\nēŖå\né»å\né¶åØ\nęø
ę“\nę“å室\nęø
ę“å·„äŗŗ\nęø
ę“ēØå\nęø
ę°ē\nę \nå
č±éØę°å°ęę©\n客ę·ē«Æ\nę¬å“\nē¬\nē¬å±±\nē»å±±č
\nčÆę\n夹å\nåŖč““ē»\nåŖč““ęæ\nåæ«éåøč¹\nååå
°\nęēÆ·\nęØåŗé\nē¹å\nå£ę©±\nåø\nē©æč”£\nč”£ę\nęč”£å¤¹\nęč”£ē»³\nęč£
åŗ\näŗ\näŗé¾ę£®ę\nå¤äŗ\näøå¶č\nå°äø\nå°äøé±¼\näæ±ä¹éØ\nē¦»ååØ\nęęæå
\nē
¤ē\nęµ·å²ø\nå¤å„\nč”£åø½ę¶\nēē±³\nå
¬éø”\nå¤å¤“鹦é¹\nåÆå”ē¬\n驾驶\nčč\néø”å°¾é
\nå°ē¤¼ę\néø”å°¾é
č°å¶åØ\néø”å°¾é
ę”\nåÆåÆ\nꤰå\nꤰåę \nåå”\nåå”č±\nåå”ęÆ\nåå”ęŗ\nåå”åŗ\nåå”壶\nę£ŗę\nę³å½ē½å
°å°\nčŗę\nē”¬åø\nåÆå£åÆä¹\n껤åØ\nå·ē\nå·åæčę²ę\nåä½\nę¼č““ē»\nę¶čå\n大å¦ē\nē§ē¾ē¬\nē¢°ę\né¢č²\nę¶č²ä¹¦\nęč²ęę\nē®ē§é©¬\nę±å\nę¢³å\nåÆē é\nåå§ę¼å\nåå§\nåå§ēµå½±\nå½ę\nčę\nå®ę
°é£ē©\nę¼«ē»ä¹¦\nę¼«ē»äŗŗē©\nčæēÆē»\nęę„å®\nčÆč®ŗå\nē¤¾åŗ\néå¤\nå
¬åø\nęåé\nęÆčµ\nęÆčµ\nē«äŗč
\nä½ę²å®¶\nä½ę\nå č„\nēµč\nēµčęŗē®±\nēµčę¤
\nēµčę”\né®ē\nč®”ē®ęŗę¾ē¤ŗåØ\nč®”ē®ęŗęæ\nēµčå±å¹\nęŗē®±\nę¦åæµč½¦\né³ä¹ä¼\né³ä¹å
\nč“壳\nę··åå\nč°å³å\néæåå„\nē¬ē«äŗ§ęēå
¬åÆ\nęę„\né„å½¢ē©\nä¼č®®\nä¼č®®äøåæ\nä¼č®®å
\nä¼č®®å®¤\näŗ彩ēŗøå±\nå²ēŖ\nåęµ\nčæę„\nčæę„åØ\nęø©å®¤\nęåŗ§\nå»ŗēå·„å°\nå»ŗēå·„äŗŗ\nå
å«\n容åØ\néč£
ē®±č¹\n大é\nč½®å»\nåå\nę§å¶\nę§å¶å”\nä¾æå©åŗ\néä¼\näŗ¤č°\nč½¬ę¢åØ\nåÆč½¬ę¢ē\nč¾éęŗ\nåØåø/ē¹é„Ŗ\nē¹é„Ŗ\nē¹é„Ŗå·é¾å\nēå
·\nåē\nå·å“åØ\né\näøę¬/äøå\nēē\nēēē¤\nē²ē»³\nęēŗæēµčÆ\né
\nåØå°å£«ē®čē\nē¶å”\nč½ÆęØęæ\néø¬é¹\nēē±³\nēē±³ē°\nēē±³é¢å
\nč§č½\nå°å·\né£ęŖ\nēéŗ¦ē\nå“ę \nčµ°å»\nē“§čŗ«č”£\nåå¦å\nåå¦å·\nåå¦é\nč§č²ę®ę¼\nęč£
\nęč£
ēµå½±č®¾č®”åø\nå©“åæåŗ\nå°å±\nę£č±\nę£č±ē³\nę²å\nåč®”ę¶\nęå°\nå°é¢\nę佳乔ęęę\nä¹”ęå«å¢
\nä¹”ęå
¬č·Æ\nä¹”ęęµč”ęę\nåę\nåéØå°č½æč½¦\n夫å¦/äø¤äŗŗ/å äøŖ\nę
ä¾£åē\nå°č”ē\nčƾēØ\nēåŗ\nę³é¢\né¢å\nå å
å¼\nå·„ä½ę\n儶ē\nęÆēēé¢é\nēä»\nēä»é“\nēä»åø½\nčč¹\nč¹č\nč£ēŗ¹\nęēÆ®\nå·„čŗ\nå·„å \nčč¶č\nčµ·éęŗ\né»ēŗ±\nåę\nęæę”ē®±\nē«å±±å£ę¹\né¾č¾\nč”ē¬\nå„¶ę²¹ä¹³é
Ŗ\nå„¶ę²¹ē½\nåå»ŗ\nēē©\näæ”ēØå”\nę°ęå½¢\nę°ęå½¢é¢å
\n山锶\nå
Øä½č¹å\nčč\nęæēēØē\nęæēé\nęæēéå\né©č¾¹\nå
ē½å
ēµé
\né³é±¼\nåŗēؼ\né²čäøč”£\näŗ¤å\nęØŖęØ\nååč·Æå£\nēø声\näŗŗč”ęØŖé\nę²¹ē
é¢å
å\nä¹éø¦\nę¬ę£\näŗŗē¾¤\nę„ę¤ē\nēå \né“ęå°ēŗæē®”å±å¹\nč¶ēØ£åé¾å\nå·”ęøø\nęøøč½®\nå·”ę“č\né¢å
å±\nåå\nęę\nę°“ę¶\nå¹¼å
½\nē«ę¹ä½\né»ē\nēę\nč¢å£\nč¢ę£\nē¹é„Ŗ\nåē°\nęÆå\nēŗøęÆčē³\näøęÆē¹\n马č·Æēå\nęåŗ¦\nå·ååØ\nę ē±½č”čå¹²\n蓧åø\nåå±\nēŖåø\nę²ēŗæ\nč½Æå«\n锾客\nå\né¤å
·\nčŖč”č½¦\néŖčŖč”č½¦\né¾å·é£\nę±½ē¼ø\néé¹\nęę \nęę \nč¾¾å
ęÆēē\nę°“ä»č±\nåé¦\n大äø½č±\nčå\nä¹³å¶å\néč\n大å\nę害\nę½®ę¹æē\nč·³č\nčę± \nčč¹å®¤\nčč
\nč²å
¬č±\né»ę\né»ę\né£é\nåé¶\nęē¤ŗęæ\nę„ę\n儳åæ\né»ę\n天åŗäø\nę„å
\néØę \nę»äŗ”\nč¾©č®ŗ\nē¢ē\nē»ēę°“ē¶\nē²ęæ\nåå±å·“士\nč£
é„°\nč£
äæ®/č£
é„°\nč£
é„°ē»\né¹æ\nåå«\nē„\nēé£\nęé\nęčæ\nęŖå
½\nę¼ē¤ŗ\nå
½ēŖ/ä¼é²å®¤\nēä»å¤¹å
\nēå»\nē¾č“§ååŗ\nęéē\nå¾·ęÆ\nē®č¤ē
\nę²ę¼ \nę²ę¼ å
¬č·Æ\nč®¾č®”\nč®¾č®”åø\nę”å/č”Øę ¼\nå°ēÆ\nę”é¢\nå°å¼ēµč\nēē¹\nē “å\n侦ę¢\nę“궤å\né²ę°“\nä»Ŗč”Øē\né»ē³\nå°æåø\nå°æåøå
\nęåæ\nę»\né„®é£\nęęęŗ\nę°å\nę°åę¶é\nč³č\nęé¤\nå°č¹\né¤å
\nę宓\né¤ę”\nęé¾\nęµø\nęå\nęå¼\nåƼę¼\nå°å\nč¶éę©ęč½¦\nę³„åå°\nę³„åč·Æ\nę³„č·Æ/åč·Æ\nē¾é¾\näæ”å¾\nčæŖęÆē§čå
\nčæŖęÆē§ēÆē§\nčæŖęÆē§čå
\nē¾ē
\nēå\nē¢å½¢å¤©ēŗæ\nę“ē¢ęŗ\nę¹åø\nčč“\nę“ē¢ę¶²\nčæŖęÆå°¼ä¹å\nčŖåØå®č“§ęŗ\nå±ē¤ŗ\néåēŖ\nå£ę²\nę½ę°“\nę½ę°“å\nč·³ę°“ęæ\nēŗøęÆ\nęµč”é³ä¹ęé³å\nę宾ē¬\nē 夓\nå»ē\nę件\nēŗŖå½ē\nē\nēēŖ\nē¬ē§\nē锹å\nēē²®\nēēŖ\nę“åØåØ\nē¾å
\nē©å¶ä¹å®¶\nę“åØåØ\nęµ·č±\nē©¹é”¶\nä½å®
\nå¤ē±³čÆŗéŖØē\né©“\nēēå\nę¶éø¦\néØ\néØęę\nåę°å
\néØē\néØå£\nå®æč\né¢å¢\nåøäøåæ\nęØåęŗ\nę\né¾\nč»č\nęę°“ę²\nå§ę¬\nęå§ēµå½±\nē»\nę½å±é\nå¾ē»/ē»ē»\nå¾é\nč¾«å\nčæč”£č£/ē¹å®åŗåēęč£
\nē¤¼åø½\nę£č£
蔬蔫\nē®é\n大ē¤¼ę\nę¢³å¦å°\nę“č”£å®¤\nčæē\nę¼ē§»\nęµ®ęØ\né»\né„®å/å\né„®ēØę°“\nå¼č½¦\nåøęŗ\nč½¦é\nę äŗŗęŗ\n갓껓/äøé\nåēÆ\n껓ē®”\nå¹²ę±\nčÆē©\nčÆåŗ\né¼\né¼ę\néø”č
æ\nå¹²ē\nå
¬ēµå¤«äŗŗ\néøå\néøå“å
½\nå°éøå\nåøåŗč¶åø¦\nä¼č®”\näŗéå±\nē²å¢\nē¬ęØč\nåé\né„ŗå\nę²äø\nę£ēÆ®\nꦓč²\né»ę\nē°å°\nåå¾č½¦\nē°øē®\nē¾½ē»č¢«\nDVD\nęę\né¹°\nč³ęµ\nå¾”åÆč³ē½©\nč³ęŗ\nč³å”\nč³ēÆ\nå°é\nē»ę¶\nå¤ę“»č\nå¤ę“»čå
å\nå¤ę“»č彩č\nå\né¤å
\nę³”č\nę„é£\nēęē³»ē»\nē¼č¾\nęč²\nęč²å®¶\né³é±¼\nč\nčå·\nčę\nęčåØ\nē½é¹\nåč²å°éå”\nę©”ē®ē\näøēŗ§\nēµę¤
\nēµé»\nēµå·„\nēµ\nēµå\nēµååØ件\nå¤§č±”\né«åŗ¦å¾\nēµę¢Æ\nēµę¢Æč½æå¢\nēµę¢ÆéØ\nēµę¢Æ大å \nēµę¢Æäŗ\nč·Æå ¤\n大ä½æé¦\nč£
é„°\nē°ē¬\nä¼å¾½\nåŗē»£\nēæ”ēæ \nē“§ę„\nē“§ę„ęå”\nē“§ę„č½¦č¾\nę
ę\nåøå½å¤§å¦\nęŖē·\nå¤å£³/å“å¢\nč¶å \nč½ęŗ\nč®¢å©\nč®¢å©ęę\nå¼ę\nęŗč±\nå·„ēØåø\nå·„ēØ\nč±å½ēęÆē«\nä¹å¢\nåč½¦é®\nę¼čŗäŗŗå\nåرä¹\nåرä¹äøåæ\nå
„å£\nå
„å£å¤§å
\näæ”å°\n马ęÆ\nč®¾å¤\nę©”ē®ę¦\näŗč”\nä¾µč\nčŖåØę¶ę¢Æ\né£ēØčē\nęµē¼©åå”\nęæå°äŗ§\nę²³å£\nę”ę \nęäø\nęē¤¼ę\nå¤å
\nåę天ē©ŗ\nęäøēå¤Ŗé³\näŗ件\nåøøē»æē\nęÆē¾\nęę\nčæåØ\nęę°ē½©\nå±č§\nåŗå£\nę¢é©č
\nēēø\n延éæēŗæ\nēē«åØ\nęę°ę\nę¤å\nē¼ē\nē¼å½±\nē\nē¼ēŗæē¬\nåøę\nēŗŗē»åååŗ\nå¤č§\nčø\nčøéØē¹å\nčē²\nęÆå·¾\né¢å·¾ēŗøę¶\nč®¾ę½\nå·„å\nå·„åč½¦é“\néåø\né²å¤©åøåŗ\nä»å„³\nēé¹°\nē§å¤©\n家åŗ\n家åŗč½æč½¦\nå
Ø家ē¦\n家åŗęæ\né£ę/ęå\nå°ē\nååŗ\nåę°\nåę°åøåŗ\nåč\nę¶å°\nę¶å°é
é„°\nę¶č£
č®¾č®”åø\nę¶å°ē儳å©\nę¶č£
ęå¾\nę¶č£
大ē\nę¶č£
ęØ”ē¹\nę¶č£
č”Øę¼\nåæ«é¤\nč„æå¼åæ«é¤\nē¶äŗ²\nę°“é¾å¤“\nę
é\nåØē©\nå°é¹æ\nä¼ ē\n宓ä¼\nē¾½ęÆ\nč½Æå¢åø½\n鄲ę\näøé¤\n鄲å
»\nåå
»ēę¤
å\nē«ē§\nē¾ę“²ē®\nę
ę \nč¬č¾¾\nčØē±»ę¤ē©\néŖč²\nę©å¤©č½®\nęø”č¹\nč„ę\nčę„\nēŗ¤ē»“\nå°čÆ“\nå°čƓ书\nē°é/åŗå°/éå¤\nē°é“éč·Æ\nę č±ę\nęę¶\nč±ę ·ę»å°čæåØå\nå°éå\nę件\nę”£ę”ē
§ē\nę件ę\nå”«ę»”\nč¶ēēøęŗ\nēµå½±åƼę¼\nēµå½±ę ¼å¼\nēµå½±é¦ę ē¤¼\nēµå½±å¶ēäŗŗ\nęę\nčæ껤åØ\né³\nę\nē»ē¹ēŗæ\nå·ę\nå·ęę \nē«\nē«ē¾ę„č¦\nę¶é²éØéØ\nę¶é²č½¦\nę¶é²éé\nę¶é²ę°“åø¦\nē«å\nę¶é²ē«\nēē«¹\nę¶é²éå\nå£ē\nēč±\nēč±č”Øę¼\nę„ęē®±\né±¼\né±¼č¹\nęµ·é²åøåŗ\né±¼å”\né±¼ē¼ø\nęø夫\néé±¼\nęøč¹\nęøē½\néé±¼\nęøę\nå„čŗ«\nå„čŗ«čƾēØ\näŗäøŖ\nåŗå®č£
ē½®\nå³”ę¹¾\nå½ę\nęę\nå°čē\nē«ē°\nē«ēéø\nę³å
°ē»\nęę\nčę\néŖå
\nē§ē¶\nå¹³\nęÆē®é±¼\né£å³\nč·³č¤\nč·³č¤åøåŗ\nč°é\né£č”\nē©ŗäøä¹å”å\nēæ»č½¬\n触ååØ\nēæ»č½¬å¾\nęµ®åØ\nē¾¤\nę“Ŗę°“\nå°ęæ/å°é¢\nč½å°ę\nčå«\nę„¼å±å¹³é¢å¾\nč½å°ēŖ\nęč±čŗęÆ\nč±åŗ\nēēŗæ\né¢ē²\nęµåØ\nč±\nč±ēÆ®\nč±å\nč±ē®±\nč±ē°\nč±ē«„\nč±ååøåŗ\nęµä½\nå²ę“\néæē¬\né£\né£č”éé±¼\nä¼ å\n马\nę³”ę²«\né¾\nå¤é¾ē\né¹
čé
±\nē®ēŗø\nęę¤
\nę å¶\nę°é“čŗęÆ家\nę°é“čč¹\nę°é“ęę»čŗęÆ家\nę¹ę¦ē³\nē«é
\nå£ę“ę± \né£ē©\né£ēØč²ē“ \nē¾é£å¹æåŗ\né£åå å·„ęŗ\nå°åę\nåæ«é¤č½¦\nę”äøč¶³ē\nč\näŗŗč”ꔄ\nč¶³ē\nč¶³ēęē»\n大å¦ę©ę¦ēčµ\nč¶³ēęÆčµ\nč¶³ēåŗ\nč¶³ēęÆčµ\nę©ę¦ē夓ē\nč¶³ēčæåØå\nč¶³ēåŗ\nč¶³ēé\nå°č·Æ\nčå°\nččøęæ\nå°åŗ§\néå\nę
宫\nęµ
껩\né¢å¤“ \nę£®ę \nę£®ę大ē« \nę£®ęå°é¢ \nę£®ęå°č·Æ\nę£®ęå
¬č·Æ\né»é \né¤å\nåč½¦ \nč”Øę ¼\nåę\néå/å½¢ęē©\nF1ę¹ēØå¼čµč½¦\nå ”å\nē¢å ”\nčæ½é\nåē³\nē²åŗ\nå·ę³\né¢ē¬\nēēø \nę”ę¶ \néę \né«éå
¬č·Æ \nå”č½¦ \nę³å½\nę³å½ęēē¬ \nčÆę” \nę³å¼ååø \nåå¦ę°“\nå°ē®± \nēøéø” \nē
č \nēé„ \nåč°\né£ē \néč \né \nē»é \näø„åÆ\nē»å°\nę°“ę \nę°“ęčē³ \nę°“ęē \nę°“ęåøåŗ \nę°“ęę²ę \nę°“ęę\nęę \nę°“ęååŗ \nę²¹ēøé£å\nē
é
\nč½Æē³\nēę \nåøēē½©\nęč¶£ē \nč¬ē¤¼ \nēč \nę¼ę \nęÆē®č”£ę\nęÆē®å¤§č”£ \n家å
· \nč²å¢ \nå°å·„å
· \nęŖå£\nęäŗ/ęē³»\nē¾ęÆé¦\nęøøę \nęøøęę£ē\nęøøęęę\nē«č
æ \nå¢ä¼\nč½¦åŗ \nč½¦åŗéØ \nęå·„ęØ”å\nåå¾ \nč±å \nč±åč¦ē¬ \nę©”č¶č½Æē®” \nč±åčč\nåäø \nåčŗ \nå č²ē«\nę»“ę°“å“ \nč±ēÆ \n大č \nč”£ę\nę°ä½ \nå ę²¹ē« \nē
¤ę°ē \né²ęÆé¢å
·\nę¶é \nčé\nęµéä»ŖåØ\né²å° \né½æč½® \nå£č \nčŗå¦\nåč¶ \nē¾č“§ååŗ \nåēµęŗ \n天ē«ŗčµ \nå¹½ēµ\nē¤¼ē© \nē¤¼åč¢ \nē¤¼åēÆ® \nē¤¼ē©ē \nē¤¼åå” \nē¤¼åååŗ \nē¤¼ē©å
č£
\nę¼å±ä¼\nęę¾åé
\nå§ \nå§é„¼ \nå§é„¼å± \né¶ęę \néæé¢é¹æ \nå„³å© \nē» \nå°å· \nč§ę士\nē»ēē \nē»ēē¶ \nē»ēē¢ \nē»ēē®±\nē»ēå»ŗē \nē»ēéØ \nē»ēå°ęæ \nē»ēå±\nē»ēē½ \nē»ēęæ \nē»ēę”å \nē»ēč±ē¶ \nē»ēå¢\nē»ēēŖ \nē¼é \nå
ę»é¢\nę»ēæęŗ \nå°ē \nęå„ \nåå
\n걤å \nå» \nč¢å»\nēéØ\nå®éØå \nå±±ē¾ \nē¾å„¶é
Ŗ\nęå£ \nę¤ē®é/å¢Øé\né»é \néē \nééØå¤§ę”„ \néęÆēē¬ \néé±¼ \né«å°å¤«čæåØ\né«å°å¤«ēåø½ \né«å°å¤«ēč½¦ \né«å°å¤«ēę\né«å°å¤«ēåŗ \né«å°å¤«ēę \né¹
\n大ē©ē© \nå„ē¹å¼\nč«č¦ \nęæåŗ \nęæåŗęŗę \nē¤¼ę\nęÆäøē\nęÆäøå
øē¤¼\nč°·ē©\néęé²ø \n大å„čµ \nē„ē¶ \nē„ęÆ \nē„ē¶ęÆ \nč±å²å²© \nę ¼å
°čÆŗęéŗ¦ē \nč”č \nč„æę\nč”čé
\nč \nč±č¢ \nčå \néæę»”čē \nę¦čåØ\nåå¢ \nē¢ē³\nå¢ē¢ \nčę± \nč°å³ę±ē¶\nē°č²\nåč \nę¾ē§ \nē»æč² \nē»æč²ę¤ē© \nę¬¢čæ\né®å \nč“ŗå” \nē°ē \nē½ę ¼ \nēå\nē§ē¤ę¶\nę ¼ę
\nē¤é³é±¼ \nē£Ø \nē ē£Øęŗ\nē²ēéŗ¦ē² \nę蓧č¢\nę“ē©“\nå°ę¾é¼ \nē¾¤ä½\nåå½± \nå°ę ę\nēéæ\nēę²¹ęé
±\nč¦å« \nēéØē \nå®¾é¦ \n客ęæ \nęå \nč±é¼ \nåä» \nåä»ę \nęµ·ę¹¾ \nęµ·éø„ \nęŖ \né«č¾¾\nč°åøę\nå¤ē \nå„čŗ«ęæ \nä½ęčæåØå \nę ęÆå° \né»å®¢ \nå°é¹ \n夓å \n夓åé¢č² \nåč¶ \nęÆå· \nåå \nå夹 \nåē½ \nå夹 \nåå \näøå \nē¤¼å \näøå£č \näøå£čęč£
\näøå£čåē \né²čč£
\nę±å ” \nę±å ”å
\nååÆē \né¤å\nååŗ \né»ē¢ \nä»é¼ \nēęęŗ\nę¾å¤§é\nę¦ęå·¾ \nęęå
\nęē \nęé\nęęŖ \nęåø \nęę\nęéÆ \nę”ę \nåē« \nęå\nę±ę \nę¬ę\né£ęŗåŗ\nč”£ę¶\nå¹øē¦ \nęµ·ęøÆ\nęęµ·č±¹\nē”¬ęę»čŗęÆ家 \nē²¾č£
书 \nå»ŗēå·„äŗŗ\nē”¬ä»¶ \näŗéåŗ \nē”¬ęØ \nē”¬ęØå°ęæ \nå£ē“ \nē®”é£ē“ \nē¾½ē®”é®ē“\nę¶č· \nę¶å²ęŗ \nåå«/ęčå³/čäø\nåø½å \nåø½ē \nåē°§ē®”\nå±±ę„ \nå¹²č \nå¹²čå°\nę¦å\n夓 \näø»ęē» \n大ēÆ\nåŗ夓ęæ \n夓鄰 \nęµ·å²¬ \nę»éØ \nå¬å \nåæč\nåæå½¢\nēč½\nå ēåØ \nåøē³ę„ \nę ēƱ\nåŗē¬ \nčåč·\nē“åęŗ \nē“åęŗęŗåŗ\n夓ē \nåø®å© \nęÆéø” \nęē²č± \nčÆč\nå
½ē¾¤\nåÆå±
č¹ \nč±é \nčé¹\nččč±\nččč± \néč/éč½å¤\né«ę \né«č·é \né«å° \nēŖåŗ \nå¾ę„ę
č” \nå¾ę„ę
č”č
\nå¾ę„é“\nē»å±±č®¾å¤ \nå±±äø\näøéµå°\nå«å¢
\nå±±å”\nå°åŗ¦ęåÆŗåŗ \né°é¾ \nčéØ \nå»åčŗäŗŗ \nę²³é©¬ \nåå²å¦å®¶ \nåå²éčæ¹\nåå² \nę²ę£ē \nå°ēé¦\nę²ę£ēęÆčµ \nę²ę£ēčæåØå \nę²ę£ēę£\né夓 \nę“ \nåę„\nå¬éę \nęµ·å \n家/ä½å®
\n家ēØēµåØ\nåŗå° \n家å±
č£
é„° \n室å
č®¾č®” \nå
ęæéØ\n家åŗå½±é¢ \n家åŗä½äø \né¹°å“č±ę³„\nčč \nčēŖ\nčę\né£åø½\nčæåø½č”« \nęé©/å¾ä½\nč·³ \nå°å¹³ēŗæ \nēéø\néæč§ē \n大é»č \néę\nęęēµå½± \n马é褄\né©¬č½¦ \n马åŗ \néŖ马\n马č\n马č¹é\nč½Æē®” \nå»é¢ \nå»é¢ē
åŗ\nē
ęæ \näø»ęäŗŗ\nå°ę
é¦\nē \nēę°ē \nēē \nč¾£ę¤é
± \nęø©ę³ \nę
é¦\né
åŗ大å \né
åŗęæé“ \nēµē\nę²ę¼ \nęæå \nęæåå¤éØ\n室å
ę¤ē© \nę¬ę»ęæ\nå¼ \nč·ē¼©\nę„ę±\nå¼å¦å\näŗŗ\nå¢ę¹æåØ\nčéø\nåŗ§å¤“é²ø \nęē\nē©ēå°å± \néē¢\né£é£ \nå士å„\nå°å± \n鬣ē \nę··åē©\nē»£ēč± \nę¶ē«ę \nę°“äøé£ęŗ \nå° \nå°č¢ \nåęē \nå°ę“ \nå°ę·ę· \nå°ę·ę·čå·\nå°ę·ę·ååŗ\nå°å \nęµ®å°\nå°ēčæåØå \nå°ēé \nę£ę£ē³ \nå¶å°ęŗ\nęŗå°åŗ \nå°é \nå°ę¶ \nęŗå°é\nę»å°\nå°å±± \nå°ę±\nē³č”£/é
„ē®\nå¾ę \nčŗ«ä»½čÆē
§ē \nčŗ«ä»½čÆ \nå°å±\nå
/ēÆå
/å
ēŗæ\n鬣č„č“\nē
§äŗ® \nęå¾\nå½¢č±”\né»ęē¾\nēé¦\nē¬ē«ę„ \näøŖäŗŗ\n室å
\nåč¹åØ\nēµē£ē \nå·„äøåŗ \nå·„äø\nę„å
µ \nå
ę°č \nęå”å° \nåŗē”č®¾ę½ \nęå \nåøå
„åØ \nę³Øå° \nå伤 \nå¢Øę°“ \nå°ę³„\nå°ę¹ę¹¾\né¢čÆ\nęč«\nå®č£
\nä¹åØ/åØę¢°\nē»ē¼ęÆ \näŗåØ\n室å
č®¾č®”\nē½ē« \nååč·Æå£ \né¢čÆ \nę čę¤åØē© \néčÆ· \nå¹³ęæēµč\nč¹ęęęŗ\nč¹ęé³ä¹ęę¾åØ\nč¹č \né \nēØč”£ęæ\nēęŗē³»ē» \nå² \nå°å²\nēč¶³ē±»åØē©\nč±”ē \nåøøéč¤\nå±
é
å±\nåę¤é”¶\nåøēč¹/č¹\n夹å
蔫\nęę©ęµ“ē¼ø\nē\nē¾ę“²č\nēē±ē¢ęæ\nęé
±\nę„å¼č±å\nččč±\näøå·“ \nę¾éø¦ \nēµå£«ä¹ \nēµå£«ä¹čŗęÆ家\nēµå£«čåčŗęÆ家\nēä»č£¤\nåę®č½¦ \nęå» \nęå»č±\nę°“ęÆ \nå·ę°å¼é£ęŗ\nę©ęč \nē å® \nē å® \nē å®åŗ\nę¼å¾ęøøę\näŗŗåč½¦ \nčµé©¬éŖåø\nčµé©¬åø½\nę
¢č· \nčåē\nč®°č
\nęēŗµę \nę³å® \nę°“å£¶\nē©ęč\nęę±\nę¦Øę±åØ\nę£å\nč·³ē»³\nčæčŗ«č£¤ \näøę\nåŗåå ę¾åŗ\nē¾½č”£ēč\näøč±ē\nč¢é¼ \nå”ęok \nē©ŗęé \nå”äøč½¦čæåØ \nę§ååŗ\nē®č¹\nē¤čäø² \nęé®/é„å\néØå”\nå”å
¶č²\nčø¢\nčę ¼å
°č£\nåę\nå¹¼åæåę室 \nå¹¼åæå \nå½ē\nåøēč¹ \näŗ²å»\nå·„å
·å
\nåØęæ \nåØęæꩱę\nåØęæå°é¢\nåØęæå°ęæ\nåØęæę½ę²¹ēęŗ\nåØęæå²\nåØęæę°“ę§½\nåØęæę”å\nåØęæēØå
·\nåØęæēŖę·\nåØęæēØå
·\né£ē \nēē“ę” \nę¤č\nč·Ŗäø\né¤å\néŖę\nē¼ē»\nē¼ē»é\nēå½¢ęę\néØēÆ\nē»\nčę \né¦é²¤\nktv\nå®éŖ室 \nå®éŖ室å¤å„\nę ē¾\nęåøęå¤\nčæ·å®«\nē½ē¼ē»ē©\nč¾äøčæč”£č£ \nę¢Æå\néæęę\nē¢č«\nēÆē¤ę¹ \nę¹ę³\nę¹åŗ \nę¹č¾¹å°å±\nę¹å²ø \nē¾č \nē¾ę \nēÆę± \nēÆē½© \nē \nåå°\néå°č½¦č¾ \nåŗē©å”«å\nēé\néč½ē²ęæ\nå°ę \né£ęÆ\n山哩\nęåø¦\nēÆē¬¼ \nč
æ/大č
æ\nē¬č®°ę¬ēµč\nē¬č®°ę¬é®ē\nå¹¼ä½\nē¤å®½é¢ę”\nęæå
\nē«ęÆ\nå„ē“¢ \néØé©\nä¹³č¶ \nęæéåå” \nē¬\nåå°\nååøä¼\näø¾åä¼č®®\nčŖå©ę“č”£åŗ \nę“č”£ęæ\nę“č”£ēÆ® \nę“č”£ęæ \nē岩 \nč°č”£č \nčåŖ\nčåŖå©ē¤¼ \nå¾åø \nčŗŗ \nå¼é¢ \näø»å± \néå\né¢č¢ \nę³ę¼ \nå¾ę/åé \nå¦ä¹ \nē®åø¦ \nē®é© \nē®å¤¹å
\nē®é \nę¼č®² \nę¼č®²å
\nęå¦å®¤\nēŖå° \nå©é„\nč
æ \nä¼ čÆ“ \nē“§čŗ«č£¤/ē§č£¤\nē«ę³é¢ \nä¹é« \nč±ē±» \nę ęŖ¬ \nę ęŖ¬ę± \nę ęŖ¬ę°“ \nēē“ \né夓 \nē©å
\nęč± \nč±¹ \nē“§čŗ«čæč”£č£¤ \nē“§čŗ«č£¤č¢\nå°å¦ē²¾ \nčƾēØ\näæ”å½\näæ”ē®± \näæ”ēę åæ \nå»å \nēč \nę°“å¹³ \nå¾ä¹¦é¦ \nč®øåÆčÆ \nč½¦ē \nå°č”£ \nč \nēå \nčŗŗē\nå®å
Øåø¦ \nęēč”£ \nęēč \nęēå \nęčµ·\nēÆå
· \nēÆå
ē§ \nēµēÆå¼å
³\nē
§ę/ē
§ęč®¾å¤\néŖēµ \néæé·é \nę·”ē“«č² \nē¾å\nč¢ä½ \nē³ē° \nē³ē°ē³ \nč±Ŗåč½æč½¦ \nēŗæę”\nčŗęÆēŗæę”\nęé \näŗéŗ» \né®č½®\nē®å \n궦åč \nå£ēŗ¢ \nę¶²ä½ \né
ē±»ååŗ\nåč”Ø \nčę \nēę“» \n家ē\n客å
\nēę“»ē©ŗé“ \nč„č“ \nč“č½½ \nč£
åøē 夓\nęøøę儽é²ēäŗŗ \nčµ°å»\nå®ä½ \né \néø室 \nåØē©ę \néę„¼ \nåęØ\nå°ęØå±\nę åæ \nę“åŗ \néæ夓å \nå²ęµŖęæ\néēŗ¦ę¾ē°/ē»åøęŗ\nēÆē¶\né失\n彩ē„Ø \nč²č± \nē± \nåäŗŗę²å\nč”ę \nęØę \nä¼ęØå·„äŗŗ \nåé¤ \nåé¤ē \nééč±č±ē \nå„¢ä¾å \nč±Ŗåęøøč \néØč”£\nę¾³ę“²č”ę”\nēå°¾ēæ \néåæē² \néåé¹¦é¹ \nå¼Æå \nęŗåØ\nęŗęŖ \nęåæ \néę³ \néęÆåø \nē£é \nę¾å¤§é \nęØå
°č±\nåé¹ \néŗ»å° \nč±”å¤«\nå„³ä» \né®ä»¶ \né®ä»¶ę§½ \nå¶ä½\nę¹é \nåå¦åø \nåå¦å·„å
· \nééø \nééø \nę§ę£\nåŗä¹³åØē© \nēēøč±”\nē·äŗŗ\nē®”ē \nē»ē \nęµ·ē \nę¼č¼ē½ \nę©å \nę®éčÆ \né¬ęÆ \nę¼«ē» \né£ę§½\nčę \nå±±ē«¹ę \nēŗ¢ę ę \nę¼åé”æ \nę£äæ®å\näŗē\näæ®ęē² \näŗŗä½ęØ”å \nåŗåäø»å®
\nå¤§å¦ \nč³č \nå°å¹ \nę“»åØęæå±\nå¶é äø \nęēØæ \nå°å¾ \nę«ęØ \nę«å¶ \nę«ē³ęµ \nę²ē \n马ęę¾ \n大ēē³ \nč”čæ\nč”čæä¹é\nęÆ马 \néēč± \nę°“å
µ\nęµ·ę“ę čę¤åØē© \nęµ·ę“åŗä¹³åØē© \nęØå¶ \nę åæ\néåø\nåøåŗå¹æåŗ\nåøåŗęä½ \nē»å©\nę¦ęÆ\nę¦ęÆ家 \nę¦ęÆé¦ \n马ęå°¼\n马äøå°¼é
ęÆ\nē«ęÆč\nåē„„ē© \nåč±ę³„ \nę
ē¢ęŗ \né¢å
·/å£ē½©\nęę© \nę”
ę \nå°å«\nęē士 \nęÆčµ\nē«ę“ē \nč”£ę\nåŗå« \néµå¢ \néæč£\näøé¤\néęÆ\nå·å°ŗ \nčē±»\nčäøø \nęŗę¢°åø \nęŗę¢°é£ę \nå„ē\nåŖä½ \nå»ēč®¾å¤ \nå»å¦å¾å \nå»å”äŗŗå \nå»čÆē®± \näøäøēŗŖē\néŗ¦å°é£åø\nå„ę³ \nē«é¼¬ \nčµäŗ\né¦ē\nēŗŖåæµē¢ \nčå \nē¾äŗŗé±¼ \nē½ \nč®č\näæ”ä½æč¢ \néå± \néå±čŗęÆ家\néå±ę¢ęµåØ \nč®”éåØ\näøå±ę„¼\néŗ¦å
é£ \nę¾å¾®é \nå¾®ę³¢ē\nåå¤ \néēØē¢\nåč£
\nē儶\nē儶ē½\nå„¶č¶ \n儶ę \nē£Øå\nēæäŗ\nēæå·„\nēæē©č“Ø\nēæę³ę°“ \nčæ·ä½ \nå¾®ē¼©ęØ”å\né¢å
č½¦\néØéæ \nå°åč“§č½¦\nčč· \nčč·ē³ \néå \nå°å§ \nęę·ē©\nä»»å” \nę§²åÆē\nę··å \nę
ęęŗ\nę
ęē¢\nę··åē© \nę¤åę²³ \nēµåØčøęæč½¦\nęØ”å/ęØ”ē¹\nę±½č½¦ęØ”å \nē°ä»£ \nē°ä»£å¤§å¦ \nę½®ę¹æ \nęØ”å
· \nęØ”å
·\né¼¹é¼ \nåäø» \né± \nēę§åØ\nåå° \nē“å \nę“»åØę³ę \né»ē½ē
§ē\nē¬č½®ččøč½¦ \nęŖē©å”č½¦ \nęäŗ® \nę鄼 \nęå
\nę²¼ę³½\n驼é¹æ \nęę\nå©åč½¦\nę©ęØ\nęØé¾ \nęØå
\nęé³\nē ęµ \n马čµå
\nęø
ēåÆŗ \nčå \nčē±»ę¤ē©\nę±½č½¦ę
é¦ \nč¾ \nęÆäŗ²\näø»ęæ \näø»é¢ \nåØä½\nēµåØęŗ \nę©ęč½¦ \nę©ęč½¦ \nę©ęč½¦å¤“ē \nę©ęč½¦čµč½¦ę \néŖę©ęč½¦ēäŗŗ \nčµč½¦čæåØ \nåå \nå±± \nå±±å°čŖč”č½¦ \nå±±å°čŖč”č½¦å\nå±±å°čŖč”č½¦čæåØ\nå±±å°å¤§ē©ē© \nå±±ę¹ \nå±±ęÆč§ \nå±±å£ \nå±±č·Æ \nå±±č \nå±±åŗę²³ęµ \nå±±éŖ \nå±±é“ęŗŖęµ\nå±±ęÆå \nå±±ę \nē»å±±č
\nē»å±±å
\né¼ ę /é¼ \né¼ ę å« \nęé¼ åØ \nå“\nę¼±å£ę°“ \nē§»åØ \nēµå½±ęµ·ę„ \nēµå½±ē„Ø \nå²čęŗ \nmp3ęę¾åØ \nå
ē \nę³„ \nę¾é„¼ \n马å
ęÆ\nę”ę \nč¦ēē© \néŖ”å \nē“č¾åø \nå£ē» \nčč \nččč½¦ \nåē©é¦ \nčč \né³ä¹ \né³ä¹č \né³ä¹å³å \né³ä¹å·„ä½å®¤ \né³ä¹å½å½±åø¦č”Øę¼č
\né³ä¹é®ē \né³ä¹å®¶ \nč“»č“ \nč„ę« \nē„čÆ \nē¤å¹²é
Ŗč¾£å³ēē±³ē \nęē²ę²¹ \nęē²é\näæå§ \né¤å·¾ \nēēŖē \nå½ę \nåŗē£čÆēēåŗęÆ \nčŖē¶åå²åē©é¦ \nčŖē¶ \nčŖē¶äæę¤åŗ \nåƼčŖ \nä¹å¤č\nęµ·å \nęäŗ \nčå \nå“é¢åø¦/é¢å£\né”¹é¾ \né¢å£ \nč±č \nę²¹ę” \néē¶ē©\né»å±
\näøęå¤é»čæēå°åŗ\néč¹ēÆ\néč¹ēÆ \nē„ē» \nå·¢ \nę°å¹“ \nę°ēē\nēŗ½č¬å
° \nę°å© \nę°é» \nč®°č
ęå¾
ä¼\nę„ę \nęäø \nå¤åø \nå¤ē©ŗ \nå¤ęÆ \nå¤ę»ä¼ \nåŗ夓ę\né¢ę” \né¼»å \né¼»ē¾ \nę³Øč§£\nē¬č®°ę¬ \nč®°äŗę¬ \näæ”ēŗø \nå
¬å\nę°åå¾ę \näæ®å„³ \nę¤å£« \nęåæę \nå
»čé¢ \nčŗęÆ \nč”ę”夹å \nę©”ęØ \nę©”ę \nę”Ø \nē»æę“² \nē干室 \nēéŗ¦ē \nēéŗ¦ \nę¹å°å” \nč§åÆå” \n天ęå° \nč¶
č¶éē¢č®ē»åŗ \nęµ·ę“ \nē« é±¼ \nęä¾ \nåå
¬å®¤ \nåå
¬å¤§ę„¼ \nåå
¬ę¤
\nåå
¬å®¤éé“ \nåå
¬ę”\nåå
¬ēØå \nåå
¬å®¤ēēŖę· \nåå®\nč”ęæå®å\nē³ę²¹ \nę²¹ēÆ \nę²¹ē» \nē³ę²¹é»å°\nē§čµ \nčē
§ē \nę©ę¦ \nę©ę¦ę²¹ \nę©ę¦ę \nē
čå· \nę“č± \nę“č±å \nčē½ē³ \nå¼éē/å¼ å¼\nå¼å§\nå¼å¹å¼ \nęå§ \nęå§é¢ \nęä½ \nęęÆ室 \nęä½ \nē¼éåŗ \nē©ē© \nę©å/ę©č²\nę©ę± \nę©ę \nę©å \nč½Øé \nęå \nä¹ę± \nå
°č± \nč®¢å \nē»ē» \nęēŗø \nē¹ē¼ \n鱼鹰 \néøµéø \nę°“ē \nå¤é¢ē\né²å¤“ \nę·å¤ \nåę \nēµęŗę夓\n大ēŗ² \nę¤åå½¢ \nē¤ē®± \nę“ä½ \nå¤§č”£ \nå¤©ę”„ \nē«å¤“é¹° \nē”č \nę©”ē®ēÆ \nå
č£¹\nå
/å
č£
/å
č£¹\nå“åŗ \nč¦č½¦ \nęé \nččé„ \nå®å” \nē¼ē \nę²¹ę¼å· \nē»å®¶ \n佩ęÆå©å°č±å¤§ęåø \n宫ę®æ\nč°č²ęæ \nę
ę \nę£ŗē½©\nę£ę¦ę \nå¹³åŗé
\nē
鄼 \nēē« \né¢ęæ \nå
ØęÆ \näøč²å \nåęÆ\nåØč室 \nč£¤å \nčæč£¤č¢ \nęØē \nēŗø \nēŗøč¢ \nåēŗøęŗ \nēŗøēÆē¬¼ \nēŗøēå \nēŗøå·¾ \nå¹³č£
书 \nåēŗøåØ \néč½ä¼ \nęøøč” \n天å \né¹¦é¹ \nę¤ēäŗŗå \néæå°¾å°é¹¦é¹\nę»ēæä¼ \nä¼å
µ \nē¾ē®ēŗø \nęåŗ \nå
¬å \nå
¬åéæę¤
\nåč½¦ \nåč½¦åŗ \nåč½¦č“¹ \nåč½¦ę åæ \nč®®ä¼ \n꬧č¹/é¦č\nåäøč
\nåä½ä¼ä¼“ \nåøē¹éå„ \nčä¼\nę“¾åƹåø½ \néčæ \néé \nåę \nä¹å®¢ \nå®¢č¹ \nę
客åč½¦ \nē¾é¦ę \nę¤ē
§ \né¢é£\nē²č““ \nē³ē¹ \nē§åŗ \n蔄äø \nē
äŗŗ \nå¾ę”/ę¬¾å¼\näŗŗč”é/ē”¬č·Æé¢\n大åøēÆ·\nēŖå \nęÆä» \nä»č“¹ēµčÆ \nč±č± \nåå¹³ \nę”å \nåé \n山峰/å°é”¶\nč±ē \nč±ēé
± \nę¢Ø \nēē \nåµē³ \nå±±ę øę” \nč”äŗŗ\näŗŗč”å¤©ę”„ \nę„č”č” \nęē®\nåē®åØ \nå°éęæ \nęØč“Øč
æ\né¹é¹ \nē¬/å“ę \nē¹ē \né
ē¬ \né
ē¬ē\nå·ē¬å \né
ē¬č£ \nåå \néę\nä¼é¹
\nåå² \né¦ę ę\nä¾æ士\nåØčē½ \nē”äø¹ \nč”ę¤/č¾£ę¤\nč”ę¤ē ē£Øęŗ \nč”ę¤å\nę大å©č¾£é¦č \nę ęÆ/é²é±¼\nč”Øę¼\nč”Øę¼\nč”Øę¼čå° \né¦ę°“ \nē»æå» \nę³¢ęÆē« \nęæå \näøŖäŗŗę¤ē \näøŖäŗŗę¼ęµ®č£
ē½® \nå®³č« \nå® ē© \nå® ē©åŗ \nå® ē©åŗ \nč±ē£ \n佩妮 \nęå ēéæę¤
\nééø” \nē°č±” \nå²å¦å®¶ \nēµčÆ \nēµčÆē°æ \nē声ęŗ \nē
§ē \nē
§ēøäŗ \nēøę” \nęå½± \nē©ēå¦å®¶ \nē©ēå®éŖ室 \né¢ē“家 \né¢ē“ \néę© \nę”čµ· \nę³”č \néé¤ \néé¤åŗ \néé¤ēÆ® \néé¤ę”\nå¾ē \nēøę” \né¦
鄼\néø½å \nęå£č
\nčÆē\nę夓 \né£č”å \né¢čŖč \nå«é\nę¾ę \nę¾ę \nę¾ę \nę¾å \nč č \nä¹ä¹ēę” \nä¹ä¹ē \nē²č²\näøåč±ēé \nēµē¶ \nē®”å\nē®”ē¢ \nęµ·ē \nęµ·ēę \nęµ·ēč¹ \néæęęµå \nę»éŖåŗ \nå£č¢éēé¢å
\nē«é¾ę \nęēē¬ \nēåŗ \nå¤§ę°“ē½\nēŖē¬¼č \nå¹²čå \nę«čØ \nę«čØå \nęÆčØé
\nę«čØåŗ \nęē \nå°ę¹ \né¤å
·å« \nę ¼å \nå¹³å \nē¤ŗęå¾\nč”ę\nč”ęå°ē\nåęØęæ\nę¤ē© \nē§ę¤å \nē§ę¤ \nå¾é¢\nē³č \nå”ę \nę©”ē®ę³„ \né«å \nå¹³å° \nē½é\n大ęµ
ē\nē©/ę¼å„/čæåØ\nęē¾½ęÆē \nęę£ē \nęēÆ®ē \nē©å°ē \nčø¢č¶³ē \nē©ä¹ä¹ē \nęē½ē \nęęē \néę/čæåØå\nęåŗ\nå§åŗ \nęå
ē \näøę£ \nęé«å°å¤«ē \nęéŗ»å° \nčæåØåŗ\nę¤ę \nęøøę室 \nå¹æåŗ \né³å \nę
äŗę
č\nē \nę夓 \nę夓åø½ \nęå \nę°“ē®”å·„ \nå«ēę“å
· \nē¾½ęÆ\n夹ęæ \nå£č¢\nęč”Ø \néčŗ«å°ęå \nåčä½ \nä¹éęę„å°\nčÆę \näøåēŗ¢ \nę/ęå\nęé \nęå
å” \nē¹ē \nęå
č”Ø \nę/ę±\nčē« \nč¦åÆ \nč¦č½¦ \nč¦ē¬ \nč¦åÆå± \nęæę²»å®¶ \nåē¹ \nč±ē² \nę±”ę \n马ē \n马ēé¢ \n马ē蔬蔫 \nē³ę¦“ \nę³¢ē¾ęå°¼äŗē \néØę« \nę± å” \né©¬å°¾č¾« \nč“µå®¾ē¬ \nę± \nęµč” \nęµč”čŗęÆ家 \nēē±³č± \nęē \nē½ē² \nē· \nēå
³ \nēŖč \nē²„ \nä¾æęŗå¼ēµę± \néØę·ē½ē« \nęčµē»å \nę±½éØ \nčå \nčåä¼čÆ \nęå§æåæęē
§\nč“é¼ \nåøå \né®å± \né®ē„Ø \nęäæ”ē \nęµ·ę„ \nęµ·ę„锵 \né
/ē½/é¶ē\nåč± \nåč±ē \nåč±ę²ę \nåøå«å \nä¾æ壶\nč¢ \n家ē¦½ \nč±é \nå¾ę³»\nē²ę«\nēµęŗēŗæ\nēµęŗę夓åęåŗ§ \nęåē \nēµē« \nē»ä¹ \nåøęę ¼åå ” \nē„ē„· \nē§åø \né¦ę \nå¤ę¹ \nę¾ē¤ŗ \nę¼č®² \nę»ē» \nę°é»ååø室\né«åé
\nę¤ēå·é„¼ \nēå \nå
¬äø» \nęå° \nęå°é”µé¢ \nęå°ęŗ \nå°å· \nēē± \nåäŗ§å/ēäŗ§\näŗ§å \nčäø \näøäøē\nęę \n锹ē®å¾ē \nęå½±å±å¹ \nęå½±ä»Ŗ \nęÆäøčä¼ \nę£ę„ \nčŗęę”Ø \nå
ē„ \nå»ŗč®® \né²ę¤ę \nęč®® \nęč®®č
\nåŗē \nå®£ä¼ ē»å\nå°äøę²ę£ē \nåøäø \nę°“å \nę³”č \nč§å“ęµ·é \nåå·“ē \nę \nč®²å \nčå² \nę³µ \nåē \nåē鄼 \nåēē§å \nę³å»åč¢\nę³å¤“ēå»/ē©æå\nå¦ē \nē“«č²\nęØ \nč½»č½»äøå»\nč°é¢ \nå” \néåå” \n大č\näŗē»“ē \né¹é¹ \néē³åŗ \nå£åŗ¦ \nē³č± \n儳ē \nę²¹ēøēē±³ē²é„¼ \néå \nä¹³č鄼 \nč¢«å \nē»ē¼ \nå¼ēØ\nå
å \nęµ£ē \nęÆčµ \nčµé \nę°“ę²/č·é\nčµč½¦ \nēę \né·č¾¾ \nę£ēåØ \nå¹æę \nęØē/ę©”ē®č\nåøåØåØ \nę ę/éč½Ø\nč½Øéč½¦ \néé\néč·Æę”„ę¢ \nč½Øéēŗæ\nē«č½¦ē« \néØ \néØé“\nå½©č¹ \nč¹é³é±¼ \néØč”£ \nēåø¦éØę \nå¤éØē \nč”čå¹² \nčå\nå
¬ē¾\nęå” \nę²¹čē±½\nåæ«é \nčÆ“å±ęę \nę č \nčé¼ \nę£č½® \nä¹éø¦ \nå³”č°· \né· \nåé”»å \néå©ē \né
čÆ»\né
čÆ»ęę\né»ååØ\nåé¢\nå°¾ēÆ \nåč§å¾\nåč§é \nę¶ę® \nę¶å° \nę„å¾
\né
ę¹ \nč®°å½ \nå±ēå¶ä½äŗŗ \nč®°å½åØ/ē«ē¬\nå½é³å®¤ \nåرä¹å®¤ \nä¼é²č½¦ \nē©å½¢ \nåę¶ \nåę¶ē« \nēŗ¢č²\nēŗ¢å°ęÆÆ \nēŗ¢ę \nēŗ¢ēē« \nēŗ¢é
\nēŗ¢ęØ \nč¦č\nē¤ē³\nå·č½“\nč£å¤ \nåå½±\nåå½±\nåå°åØ \nę³Øå \nę§å¶ \né©Æé¹æ \nę¾ę¾ \néę¾ \nęę“ \nå®ę \nå®ęē\näŗ«å \näæę \nę¹é \né„ę§åØ\nē§»é¤\näæ®å¤ \nē»“äæ®åŗ \nē¬č”åØē© \nęę“ \nęå©č
\nē ē©¶ \nē ē©¶å \nåØå± \nä½å®
\nå±
ę°åŗ \nę č \nåŗ¦åčå° \nåŗ¦åå°é\né¤å
ēåØęæ \né¤å
ēé²å° \nåę \né¶å® \nåÆ»åē¬ \nå¶åØē«ē®\nęē¤ŗ \nēē \nęé¹ \nčéŖØ \näøåø¦ \n大ē±³ \nēµé„ē
² \nēØ»ē° \néŖ/ęä¹\nč \néŖ马\nę„ęŖ \nč¾¹ē¼\nēÆ/ęę\nę“ä¹±\nę¶ę¼Ŗ \näøå \né«å±å»ŗē\nę²³ \nę²³å²ø \nę²³č¹ \nę²³č°· \nę²³åŗ \nč·Æ \nč·Æę \nå
¬č·Æę
č” \nč·Æč¾¹ \nē¤éø” \néæč¢ \nē½å®¾ \nęŗåØäŗŗ \nē³å¤“ \n岩ē³ę± \nęę»čŗęÆ家 \nęę»ä¹é \nę岩č
\nę岩 \nęę»é³ä¹ä¼ \n岩ē³č”Øé¢ \nå²©å± \nęę»ęę \nē«ē® \nęę¤
\n岩ē³ \nå®é½æåØē© \nēä»ē«ęč”Øę¼ \nē«ęčå° \nē½ä¼ \nēå \nč¾ \nčæå±±č½¦ \nč½®å¼ęŗå°é \nęŗå°é \nęé¢ę \nęµŖę¼« \nęµŖę¼«ē \nå±é”¶ \nå±é”¶č±å \nęæé“ \nęæé“åé¢åØ \nę ¹ \nę ¹å¤é
\nē»³ē“¢ę”„ \nåæµē \nē«ē° \nčæ·čæé¦ \nē«ē°č²ēäŗ \nē½ē¹é¦å°ē¬ \nåę” \nč·Æē±åØ \nč” \nē½ę \nē家 \nę©”ē®å¾ē« \nåŗå¢ \néę¹ \nēŗ¢å®ē³\nč±å¤« \nę©ę¦ē \nę©ę¦ē \nę©ę¦ēčæåØå \nęÆå\nå°ŗ\nęå§é
\nč·\nč·ę„č
\nč·ę„é \nåęē\né \nä¹”ęē\né»éŗ¦ \nč¢ \né \néå\nę
č”\nå®å
Ø \nå®å
Øčåæ \nå£äŗŗ \nåø \nåøč¹ \nčŖč” \nę°“ę \nę¾é¼ ē“ \nē¼ę
\nę²ę \nę²ęē¢ \nē«č„č“ \nę大å©čå³č
č \nåŗå® \näøęé±¼\nę²é¾ \nčØå°čØč \nē \nēåč”ę¤ē¶ \nēę¹ \nēę²¼ \nēē¶ \nę¬ē¤¼ \nčØč«č¶å¾·äŗŗ \nę¦å£« \nę²å \nę²ę“² \nē ē®± \nę²å ” \nę²é \nåé \näøęę²» \nå«ēå·¾ \nå£čÆčäŗŗ \nčå®ē³ \nę²äøé±¼ \nčäø½ \nēé±¼ē \nę²ē¹ \n书å
\nå«ę \nē¼ \né
±ę± \nē¢å\nę”ęæ \né¦č \nēØę 大čå \néÆ\néÆęØę¶\nčØå
ęÆē®”\nčØå
ęÆę \nčęę¶ \nē§¤/ę å°ŗ\nęÆä¾ęØ”å \nęč“ \nē¤ē \nēØ»čäŗŗ \nå“å·¾ \nåŗęÆ \né£ęÆ \néŖēŗ³ēē¬ \nå¦ę ” \nę ”č½¦ \nę ”ę \nę ”č \nēŗµåøč¹\nē§å¦ \nē§å¹»ēµå½± \nē§å¦åē©é¦ \nē§å¦å®¶ \nåŖå \nå£ēÆ \nåøåŗ·é„¼ \nåŗå\nčøęæč½¦/ę©ęč½¦\nåę° \nč®°åęæ \nčå \nē«„åå \nēč \nåŗå¼\nå®ęæ \nå®ä¼¤ \nå±å¹ \nēŗ±éØ \nęŖå¾ \nčŗę \nčŗäøå \néæå·ēŗø/å·č½“\nę¦ę“ \nē”¬ęÆå· \néå”家 \néå” \nęµ·ę“ē©“ \nęµ·å° \nęµ·ē® \nęµ·é¾ \nęµ·č \nå°å»é² \nęµ·åŗ \nęµ·éø \nęµ·é² \n굷马 \nęµ·č±¹\nęµ·ęÆ \nęµ·č“ \nęµ·ę»Øåŗ¦åčå° \nå£č \nåŗ§ä½ \nå®å
Øåø¦ \nęµ·č» \nē§ä¹¦ \nå®å
Ø \nå°č½æč½¦ \nēå° \nē§å \nč··č··ęæ \nčµę ¼åØ \nčŖę\nåŗå® \nē č®Øä¼ \nęč§ \nä¼ ęåØ \nęå”åØ \nęå”åØęŗęæ \nęå” \né \nē¼ēŗ«ęŗ \nå½±å \nę \nē¶ \nę“åę°“ \nå½¢ē¶ \nåäŗ« \né²Øé±¼ \nå·ē¬å \nč®°å·ē¬\nåé”»å \nåé”»č \nę«č©/å“å·¾\nåŖå \nåŖå\nē¾ \nåŗå \nä¹č°± \nę¶å\nč“壳\nč“ē±» \néæé¾ę \nęē½® \nē§ē¾äŗŗ \nęåé²\nę“ē¬ \nåå
\nčŖčæ \néč£
ē®± \nęµ·é¾ \nč¹å \n蔬蔫 \n赤čē\nęµ
껩\né \néē \néåŗ \néꄦ \nå°å»\nå¾åēÆ®ēåå« \nååŗꩱēŖ \néØé¢ \nč“ē©č
\nč“ē© \nč“ē©č¢ \nč“ē©ēÆ® \nč“ē©č½¦ \nč“ē©äøåæ \nč“ē©č” \nęµ·å²ø \nęµ·å²øēŗæ \nēē\nēå \nēč£¤ \nå°é
ęÆ\nę£å¼¹ęŖ \nč©č \nåč©å
\né² \néåę\nę·ęµ“ \n굓åø½\n굓åø \nę·ęµ“éØ \nę·ęµ“夓 \nē¢ēŗøęŗ \nę³¼å¦ \nč¾ \nē„ē¤¾ \nēęØ \nåæ«éØ \nę¹ē½ē«\nč„æä¼Æå©äŗ \nå
å¼å§å¦¹\n侧é¢\nč¾¹ę \né
č \nč¾¹č½¦\nč¾¹ēŗæ\nå£ęæ\nę åæ \nęē¤ŗē\näæ”å· \nē¾å \näøē»ø \näøč¢ \nēä» \né¶ \né¶ē \né¶åØ \nå±ę\nē§ē¦ \nęę \nę°“ę§½ \nå\nå/ę¾ē½®/åč½\nåē \nę»ęæå
¬å\nę»ęæ \nę»ęæč
\nęŗå°č
\nęŗå°åŗ \néŖØę¶ \nčå¾ \näø²äø²\nę»éŖ \nę»éŖé“ \nę»éŖč®¾å¤ \nę»éŖę\nę»éŖē¼č½¦ \nę»éŖę \nę»éŖčå° \nę»éŖęæ\nę»éŖ \nę»éŖé \nē®č¤ \n夓éŖØ \nę č¾¹ä¾æåø½ \n天ē©ŗ \n天ē©ŗå” \n天ēŖ \n天é
ēŗæ \nę©å¤©å¤§ę„¼ \nęæęµåę \nē³ęæ\néŖę© \nē”ē \nē”č¢ \nē”č”£\nč¢å\nē \nę»åØ\nę»å \nåē“¢ \nå” \nęåøå£\nččęŗ \nę ę\nę
¢ēé
\né¼»ę¶č« \nč“«ę°ēŖ \nę°å³ \nå¾®ē¬ \nēé¾/ę½ē\né¶é£ \nčē \nč \né²·é±¼ \nåæ«ē
§ \néę°ē®” \né¼»å \néŖ \néŖč±¹ \néŖå±± \néŖē \nåęæę»éŖč
\néŖå \néŖč± \néŖäŗŗ \néŖå°ę©ę\néŖē\néŖé \néŖ \nč„ē \nč„ēę³” \nē»ēåØ \nč¶³ēå®éØå \nē¤¾ä¼åęµ\nēč¢\nęåŗ§\nčęę°“ \nåē \nč½Æ件 \nå¤Ŗé³č½ēµę± éµå \n士å
µ \nē¬å„ \nč§£å³ę¹ę” \nå®½č¾¹åø½ \nęę²\nå£°é³ \n걤 \n걤ē¢\n걤å \né
øå„¶ę²¹ \nēŗŖåæµå \nč±ęµ \nę°“ēäøåæ \nē©ŗé“ \nčŖ天é£ęŗ \nē©ŗé“ē« \nå®å®é£č¹ \nę大å©é¢ \nęØŖč·Ø\nę³ę \nē«č± \néŖč \nēē«\nčµ·ę³”č”čé
\néŗ»é \nę¹å \nę¬å£°åØ\nč§ä¼ \nä¼čÆę”\néåŗ¦éå¶ \nééę åæ \nåæ«č \nč½¦éč”Ø\nē \né¦ę \nč°ęę¶\nčč \nččē½ \nę£ē\nęč½¬\nč č \nå°å” \né£ęŗ
\nęµ·ē»µ \nåŗå \nä½č²åä¼ \nčæåØåØę\nčæåØå¢é\nä½č²ē \nä½č²åØę\nčæåØä¼ \nčæåØęč£
\nē¹ \nå·é¾ \nä¼øå±\nę„天 \nę„å· \nę\nę“ę°“åØ\nåč½ \näŗę \näŗęę£®ę \né \nå¹æåŗ \nåē \nč¹² \nę¤\né±æé±¼ \nę¾é¼ \nę°“ęŖ \nåŗ \nēسå®ē \nļ¼ē ę¾ę“é½ēļ¼äøå \nä½č²åŗ \nå·„ä½äŗŗå \nčå°\nčå°ēÆ \né©æé©¬č½¦\nå¼č\näøéé¢ \nę„¼ę¢Æ \nę„¼ę¢Æ \nę„¼ę¢Æé“\nęä½/å°éé“\nē§é©¬ \nē«/ēē«/ęä½\nē« \näø»é£ \nč®¢ä¹¦ęŗ \nęę\nēÆē\nęµ·ę \nęØę” \nēå
«å„ \nå·ē«å
¬å \nå
¬ē«å¦ę ” \nč½¦ē«\nåŗå®čŖč”č½¦\nęå
· \néå \nēę \nēęå \nčøę±½ \nčøę±½ęŗ \nčøę±½ęŗč½¦ \nčøę±½ē«č½¦ \né¦å¤“ \né¢ \nę¹åē \nļ¼č±čēļ¼č\nęØ”ē\nę¢Æå³\nē«ä½å£° \nå¬čÆåØ \nē\nę³/ę”ē¶ē©\nē«¹čč« \nč““ēŗø \néē©ē» \né«č·· \né»č²é±¼ \nę
ę \nę
ęåØ \né«\nē¼ \nč”ē„Ø \néæēč¢\nč
¹éØ\nē³å¤“å»ŗē \nē³é \nē³å±\nē³ē£Ø \nå³å \nåę¢ \nååØ \nēŗ¢ēÆ \nåč½¦ę åæ \nē§č”Ø \nēŗ¢ē»æēÆ \nååØē®± \nåØč室 \nē½/čę°“ę± \nååŗ \nåŗé¢ \né¹³ \né£ę“ \nę“é£äŗ \nēé£ę“éØē \nēå \nęå
\nč·ØéŖ\nčæ껤åØ \nęµ·å³” \nåø¦ \nēØ»č/åøē®”\nčåø½ \nčč \nęŗŖęµ\nč”夓čŗęÆ \nč”夓čŗęÆ家 \nč”č§ \nęµęµŖē\nč”夓é£å \nč·ÆēÆ \nč”åøåŗ \nč”夓ęå½± \nč”ęÆ \nč·Æę \nč”夓å°č“© \nęä¼ø \nę
ę¶ \nē½¢å·„ \nåé\nē»ē»³\nč士ę”\nåø¦å\nę”ēŗ¹ \nę¼«ę„ \nē»ę \nå·„ä½å®¤ \nå½±ę£ęę \nęę\n唫å
ē©å
·åØē©\nęÆē»ē©å
·\né¦
\nę ꔩ \nęäŗŗē \nē¹ę \nä½å” \né£ę ¼ \nęåē¬\nę½č \nę½č形大äøęę²»\nęµ·åŗę°“\néåŗ \nå°é \nå°éē« \nä½é³ē® \nå¤č\nē»é¢é©\nē³ \nē³ē¢ \nēč \nę¹ē³ \nč„æč£
\nå„ęæ\nå¤å¤© \nå¤å¤©åę\n峰锶\nå¤Ŗé³ \nå¤Ŗé³åø½\nę„å
굓 \nåØę„ \nę„ę· \nåę„čµ \nåę„čµē° \nčµč±ē±½\nå¤Ŗé³é \nę“天\nę„åŗ \nę„č½ \né®é³ä¼ \né³å
\nč¶
ēŗ§ē¢ \nč·č½¦ \nč¶
ēŗ§č±é \nč¶
åø \nč¶
åø蓧ę¶\nč¶
ęØ”\nęÆęč
\nå²ęµŖ\nč”Øé¢ \nå²ęµŖęæ \nå²ęµŖč
\nå¤ē§å»ē \nå¤ē§ęęÆ\nēÆē»\nåÆæåø \nåÆæåøå§ \nčåø¦č£¤ \nę¬ę¶ \nåꔄ \nč¶éč½¦\nēå\nēå°¾č¶ \nę²¼ę³½ \n天é¹
\n天é¹
ęøøč \nčæåØč£¤\né²ę±åø¦ \nęÆč”£ \nčæåØ蔫 \nēē\nēŗ¢čÆ \nęøøę³³ \nę³³åø½ \nęøøę³³č
\nęøøę³³ę“ \nęøøę³³ę± \nęåØ\nå¹³č½¬ę”„ \nē§å\nę¼©ę¶” \nå¼å
³ \nč½¬ę¤
\nå \nęé±¼\nč±”å¾ \nåƹē§° \nē¹å¤Ŗęå \nę³Øå°åØ \nē³ęµ \nē³»ē» \ntę¤ \ntę¤ \nå”å·“ęÆē§č¾£ę¤é
±\nčę\nä¹ä¹ēę \nę”é¢ \nę”åø \nå¹³ęæēµč \né¤å
· \nč½¬éč”Ø \nę¦ęŖ\nå¢Øč„æå„ē
ēē±³å· \nč·ę³é \nå¤Ŗę \n尾巓 \nč£ē¼ \nę/ęæ\nčµ·é£ \nčÆ“čÆ/äŗ¤č°/ę¼č®²\nęé¼ \nę£č¤č² \nę©å \nč¶åø¦/ē£åø¦/ē»ē¹ēŗæ\nęęÆÆ \nę²„éē¢ē³č·Æé¢\nč夓 \nēÆ·åø\nęé¦
鄼\nęµč \nå³é \nꦻꦻē±³ \nēŗ¹čŗ« \nēŗ¹čŗ«čŗęÆ家 \né
é¦ \nč¶ \nč¶å
\nč¶čÆä¼\nč¶å \nč¶å£¶ \nč¶å
· \nę \nčåø \nč¶ęÆ \nę°“éø\nå¢éåå½±\nå¢éä»ē» \nē¼ę³Ŗ/ęč£/åē “\nęęÆå \nęęÆ \nę³°čæŖē\nTåå½¢ē©\néå°å¹“ \nēµēŗæę \nåē¦é夓 \nęčæé \nēµč§ \nēµč§ęåęŗ \nēµč§å®¤\nēµč§ę¼ę室 \nęø©åŗ¦ \nåÆŗåŗ \n天å¦ē½ \nē½ē \nē½ēåŗ \nē½ēęÆčµ \nē½ēē½ \nē½ēčæåØå \nē½ēę \nåøēÆ· \né¾čå
°é
\nē»ē«Æ/čŖē«ę„¼\né³å° \nå°å½¢ \nē»ē容åØ \né¢å \nęµčÆ \nęµčÆčµ \nčÆē®”\nęę¬ \nēäæ” \nēŗŗē» \nēŗ¹ē \nęę©č \nęę©čęé¤ \nå§é¢ \nęå§ę¼å\nę²»ē \nęø©åŗ¦č®” \nēę°“ē¶ \nęē¶\nęęø©åØ \nēęØäø \n锶é \näøč„æ\nęč \nč \nå®åŗ§ \néé®ę®æ\nę \nę±ę \né·\né·éØ \nē¾éé¦ \nēå \nč®°å·\nē„Ø \nå®ē„Øäŗ \nę½®ę± \né¢åø¦ \nčč \nē“§ \nē¦\nē·ē å°ęæ \nē¦å±é”¶ \nē·ē å¢ \né” \né”ēŗø \nē®\nęęē±³č \nč½®č \nēŗøå·¾\nē¤é¢å
\nē¤é¢å
ęŗ \nēč \nēę \nå¦ę„ēå°å©\nčč¶¾ \nč±č
\n马ꔶ \n马ꔶåŗ§å \nåå¦å
\näøäŗ¬éå”\nēŖč \nēŖčé
± \nēŖč걤 \nå¢ \né³å\né³å\nå·„å
· \nå·„å
·ē®± \nēå· \nēč \nēē¾ \näæ®åŖęå½¢ēč±å \né
ę\nē«ē¬/å
ęŗ\né¾å·é£ \nēē±³ē²å鄼 \nä¹é¾ \n大ęęč¢ \nå¾č
¾ę± \né¾ē« \nå·Øå“éø \n触ęø \n触å°\nę
č”\nę
ęøø巓士 \nåƼęøø \nęøø客\nę
ęøøęÆē¹ \né¦ę čµ\nęč½¦\nęÆå·¾ \nęÆå·¾ę \nå¤§å¦ \nå”ꔄ \nå°é \nåéå¹æåŗ\nē©å
· \nē©å
·č½¦ \nē©å
·ęŖ \nē©å
·åŗ \nč·é\nęęęŗ \nč“øę \nä¼ ē» \nä¼ ē»ē \näŗ¤é \né„å½¢äŗ¤éč·Æę \näŗ¤éę„å µ \näŗ¤éå µå” \näŗ¤éę åæ \nå°é \né¢åē \nęč½¦ \nē«č½¦ \nē«č½¦ę”„ \nē«č½¦č½¦å¢ \nē«č½¦å
éØ \nē«č½¦č½Øé \nē«č½¦ēŖå£ \nęē» \nč®ē»\nč®ē»éæę¤
\nč®ē»åŗ\nēµč½¦/ęęØč½¦\nč¹¦åŗ \nåå½¢éå\néęåŗ¦ \nę
č” \nęē/ē¢å\nč·ę„ęŗ \nē¾é£\nę \nę ę \nęåŗ \nę č \nę å± \nę ę ¹\nę å¹² \nčÆéŖ \näøč§å½¢ \néäŗŗäø锹\néØč½ \nęÆęµ \nęę³/ē¹ę\näøč½®č½¦ \näæ®åŖ \näøäŗŗē» \näøčę¶ \néæå· \néØé \nå„ęÆ\nå„ęÆ \nēåø¦ \né³é±¼ \nå”č½¦ \nå”č½¦åøęŗ \n굓ē¼ø \nē®”å\nęč¹ \nééé¦ \néęŖé±¼ \nčå \né§é \nę¶”č½® \nē«éø” \nč½¬åØ\nčč\nē»æę¾ē³ \nē®å” \nä¹é¾ \nē ē\nēµč§ę¼å \nēµč§ę \nēµč§å§ \nēµč§čē®ē±»å \nēµč§åäŗŗ \nēµč§čē® \nę
ęÆåå§ \nēµč§å” \nęę”\né»ę\nåčč\néŗ»ēŗæ\nę \nē±»å \né®å
„\nęåęŗ \nå°¤å
éé\nå„„ē¹ę¼ \nä¼ \nå
č”£ \nę°“äø \nē¬č§å
½ \nå¶ę\nå®å® \nå¤§å¦ \nåäø\nååø \nå°æ壶 \nē®\nä½æēØ \nēØå
· \nęē©é“ \nåøå°åØ/ēē©ŗ\nč°· \nééØ\nåøč”鬼\nč“§č½¦\né¦č \nčč£ \nē§ē±»\nč±ē¶/ē¶\néåŗ\nē¢éå”éęå¾ \nē¢éå¾ę \nč¬č\nčå\nč¬čåøåŗ \nę¤č¢« \nč½¦č¾ \né¢ēŗ± \néč \n天é¹
ē» \nčŖåØå®č“§ęŗ \nå°č“©\néé£å\nč”čå± \nč¹ \nčåæ \nå
½å» \nē»éŖäø°åÆē \nå
½å»åå
¬å®¤ \né«ę¶ę”„ \nč§é¢ \nęåęŗ \nēµåęøøę\nå½ååø¦ \nč§é \nå®å¤ \nå«å¢
\nęåŗ\nč¤č\né \nč”čå \nę“å \nē“«ē½å
°č²\nå°ęē“ \nå°ęē“家 \näøęē“ę¼å„č
\nęæęÆ \né®é³ęæ \nä¼ē¹å \nē«å±± \nęē \nęēåŗ \nęēčæåØå \nåæęæč
\nčŖč” \nē§é¹° \nå夫鄼干\nå夫鄼ęŗ\nč“§č½¦\né©¬č½¦č½¦č½® \nč
° \nęå”å \nåęŗ室 \nēå室 \nčµ° \nę„č”\nęę \nęé \nå£ēŗø \nę øę”\nęµ·č±” \nęäŗ \nä»åŗ \nęø©ęē \nč¦åę åæ \nę士 \nåč° \nē£ēŖ \nę“ \nę“č”£ęŗ/å«å\nę“ \nę“č”£ęŗ \né»č \nęµŖč“¹ \nåŗē©å®¹åØ \nęč”Ø\nę°“ \nę°“éø \nę°“ē \nę°“å·å“åØ \n갓껓 \nę°“ęÆ\nēę°“åØ \nę°“ä½ \nč·č±\nę°“äøä¹å\nę°“ē®” \nåę°“åØ \nę»ę°“ęæ\nę°“äøčæåØ \nę°“é¢\nę°“å” \nę°“å½© \nę°“å½©ęå¾ \nę°“å½©ē» \nēåø \nå·å£¶ \nę°“å°å å å¾ē« \nč„æē \né²ę°“å¤å„\nę°“č·Æ\nę³¢ęµŖ\nč” \nę¦åØ \nē©æē\nå¤©ę° \nå¶ē \nē½\nęå夓\nå©ē¤¼ \nē»å©ęę \nå©ē¤¼č±ę \nē»å©čē³ \nę°å©å¤«å¦\nå©ē¤¼čÆ·ę¬\nå©ē¤¼ę“¾åƹ\nå©ēŗ±ē
§ \nå©ē¤¼ęå½±åø \nå©ēŗ±ęå½± \nå©å®“ \nę„ \nęč \néé \nä½éē§¤ \nēę„å·„\näŗ\nč„æé¤ \nč„æé¤å
\nę¹æ\nå§å°\nę½ę°“č”£\nę¹æå°\nę½ę°“ę \né²øé±¼ \né²øé²Ø \nå°éŗ¦ \néŗ¦ē° \nč½¦č½®\nč½®ę¤
\nåč½®ęÆęč½¦ę\nēå„¶ę²¹ \nę
ęåØ\nč”é”»\nåØ士åæ \nåØå\nē½č²\nē½å®« \nē½č”čé
\nē½ęæ \nä¾æéØ\n宽ē\nę„åØ\nåå \nWii \nWiięę\nčé\nč§é©¬\néē«\néč± \néēåØē© \nę³ę \né£ \né£é \né£ēµåŗ\né£åę¶”č½®ęŗ \né£č½¦ \nēŖę·\nēŖå°č±ēē®± \nꩱēŖå±ē¤ŗ \nēŖę” \nēŗ±ēŖ \né ēŖēåŗ§ä½\nēŖå°\néØå®åØ\nę”é£ē»ē \nęé£ē\né
ē¶ \nå·é
åØ\né
ę \né
ēŖ \né
ęÆ \né
ę¶ \nåé
\né
åŗ \nēæ
č\nå¬å¤© \nå¬ē \nå¬å¤©ēę©ęØ \nå¬å£åŗęÆ\nå¬å£čæåØ \nå¬å£é£ę“ \nēµēŗæ\nē“«č¤ \nå·«å©\n儳巫åø½å \nēé
\nē¼ \n儳äŗŗ \nęØ夓\nęéø³éøÆ\nęØå°ęæ \nęØå¢ \nē§ęØē\nęØå \nęå° \nåęØéø \nęØå·„åØ\nē¾ęÆ \nå·„ä½ \nē»ä¹ å” \nå·„ä½å° \nå·„äŗŗ \nå·„ä½åŗę \nč½¦é“ \näøē \nč č« \nę¬ę \n伤å£\nå
\nč£¹čŗ«č£ \nå
č£
ēŗø\nęę\nęč·¤ę\nē±ēŗ¹\nč
åø¦\nå\nä½å®¶\nęå/åčæ¹\nęÆē¬\nååę”\nęøøč\nē¦ē \né¢å\né»č²\nēä¼½ \nēä¼½å« \né
ø儶 \nč½ \nčé» \né幓 \né幓ę
é¦\nčå¤å
\nę马 \nę马ēŗæ \nē¦
ęč±å \nęé¾\nęé¾ \nåµå°ø \nē²½å\nåØē©å\n"
},
{
"path": "ram/data/ram_tag_list_threshold.txt",
"content": "0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.71\n0.75\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.9\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.61\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.7\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.82\n0.8\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.85\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.77\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.89\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.78\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.9\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.9\n0.65\n0.83\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.79\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.89\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.86\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.79\n0.65\n0.63\n0.65\n0.87\n0.8\n0.46\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.9\n0.65\n0.65\n0.9\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.8\n0.65\n0.8\n0.8\n0.8\n0.65\n0.65\n0.84\n0.65\n0.65\n0.79\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.81\n0.65\n0.8\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.87\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.83\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.77\n0.87\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.85\n0.65\n0.68\n0.65\n0.8\n0.65\n0.65\n0.75\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.8\n0.8\n0.8\n0.79\n0.65\n0.85\n0.65\n0.65\n0.65\n0.9\n0.65\n0.89\n0.8\n0.65\n0.65\n0.65\n0.76\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n1\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.9\n0.65\n0.89\n0.7\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.71\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.65\n0.8\n0.65\n0.8\n0.8\n0.9\n0.65\n0.85\n0.8\n0.8\n0.8\n0.9\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.75\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.63\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.88\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.71\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.9\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.71\n0.65\n0.8\n0.76\n0.85\n0.8\n0.65\n0.65\n0.8\n0.65\n0.79\n0.65\n0.75\n0.65\n0.8\n0.65\n0.86\n0.65\n0.65\n0.9\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.73\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.9\n0.65\n0.85\n0.65\n0.65\n0.65\n0.65\n0.8\n0.75\n0.65\n0.65\n0.65\n0.65\n0.8\n0.85\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.77\n0.65\n0.65\n0.65\n0.65\n0.65\n0.86\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.6\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.74\n0.65\n0.65\n0.67\n0.65\n0.65\n0.8\n0.65\n0.65\n0.85\n0.65\n0.8\n0.65\n0.65\n0.84\n0.8\n0.8\n0.8\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.9\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.89\n0.65\n0.65\n0.65\n0.83\n0.65\n0.65\n0.65\n0.65\n0.6\n0.65\n0.8\n0.8\n0.8\n0.65\n0.65\n0.89\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.77\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.87\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.74\n0.65\n0.65\n0.66\n0.89\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.84\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.88\n0.65\n0.65\n0.8\n0.65\n0.65\n0.7\n0.65\n0.65\n0.65\n0.9\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.82\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.9\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.75\n0.65\n0.7\n0.9\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.88\n0.65\n0.65\n1\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.8\n0.8\n0.65\n0.8\n0.65\n0.65\n0.71\n0.65\n0.65\n0.65\n0.79\n0.65\n0.65\n0.65\n0.65\n0.65\n0.89\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.9\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.88\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.82\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.9\n0.65\n0.65\n0.88\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.89\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.87\n0.65\n0.66\n0.65\n0.84\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.84\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.5\n0.65\n0.64\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.81\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.65\n0.84\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.8\n0.65\n0.85\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.73\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.86\n0.65\n0.65\n0.65\n0.65\n0.87\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.8\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.82\n0.8\n0.65\n0.65\n0.65\n0.84\n0.9\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.64\n0.65\n0.65\n0.65\n0.8\n0.8\n0.87\n0.65\n0.65\n0.78\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.9\n0.65\n0.65\n0.8\n0.65\n0.85\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.74\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.88\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.83\n0.89\n0.89\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.86\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.85\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.86\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.87\n0.8\n0.84\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.81\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.7\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.82\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.87\n0.65\n0.9\n0.8\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.7\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.85\n0.65\n0.65\n0.65\n0.65\n0.65\n0.73\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.89\n0.8\n0.65\n0.9\n0.65\n1\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.89\n0.89\n0.65\n0.65\n0.65\n0.8\n0.75\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.8\n0.8\n0.65\n0.65\n0.88\n0.65\n0.8\n0.65\n0.65\n0.8\n0.85\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.9\n0.57\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.8\n0.8\n0.79\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.89\n0.8\n0.65\n0.8\n0.65\n0.8\n0.65\n0.81\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.89\n0.65\n0.65\n0.65\n0.65\n0.65\n0.89\n0.84\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.89\n0.65\n0.8\n0.83\n0.65\n0.65\n0.8\n0.65\n0.65\n0.72\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n1\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.9\n0.65\n0.65\n0.89\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.69\n0.8\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.71\n0.65\n0.65\n0.65\n0.88\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.85\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.87\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.8\n0.9\n0.65\n0.8\n0.8\n0.65\n0.65\n0.8\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.85\n0.65\n0.65\n0.8\n0.65\n0.89\n0.65\n0.65\n0.9\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.86\n0.65\n0.77\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.8\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.89\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.75\n0.8\n0.65\n0.8\n0.88\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.88\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.82\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.83\n0.65\n0.65\n0.92\n0.89\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.75\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.85\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.87\n0.65\n0.79\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.83\n0.8\n0.65\n0.65\n0.8\n0.8\n0.65\n0.7\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.9\n0.8\n0.65\n0.65\n0.65\n0.65\n0.7\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.87\n0.65\n0.65\n0.65\n0.65\n0.8\n0.82\n0.65\n0.8\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n1\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.64\n0.65\n0.65\n0.63\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.76\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.8\n0.65\n0.75\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.87\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.82\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.89\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.9\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.8\n0.65\n0.73\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.86\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.9\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.86\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.86\n0.65\n0.8\n0.8\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.69\n0.65\n0.65\n0.65\n0.65\n0.65\n0.88\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.72\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.9\n0.9\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.45\n0.8\n0.65\n0.88\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.8\n0.51\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.66\n0.65\n0.8\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.81\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.75\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.66\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.8\n0.65\n0.85\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.81\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.89\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.79\n0.75\n0.65\n0.65\n0.8\n0.65\n0.67\n0.8\n0.8\n0.86\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.81\n0.8\n0.65\n0.65\n0.9\n0.65\n0.79\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.77\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.8\n0.65\n0.74\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.6\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.89\n0.8\n0.65\n0.65\n0.88\n0.65\n0.65\n0.65\n0.9\n0.75\n0.65\n0.65\n0.65\n0.8\n0.6\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.84\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.8\n0.8\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.85\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.63\n0.65\n0.65\n0.65\n0.7\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.9\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.84\n0.65\n0.65\n0.8\n0.65\n0.81\n0.8\n0.8\n0.8\n0.82\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.8\n0.65\n0.88\n0.65\n0.8\n0.65\n0.7\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n1\n0.8\n0.8\n0.65\n0.65\n0.65\n0.8\n0.8\n0.8\n0.65\n0.74\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.85\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.9\n0.86\n0.8\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.64\n0.65\n0.65\n0.8\n0.8\n0.65\n0.87\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.87\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.7\n0.65\n0.65\n0.8\n0.65\n0.65\n0.75\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.85\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.71\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.73\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.8\n0.65\n0.86\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.75\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.88\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.81\n0.65\n0.65\n0.8\n0.65\n0.65\n0.9\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.9\n0.65\n0.65\n0.65\n0.65\n0.7\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.77\n0.65\n0.65\n0.65\n0.65\n0.65\n0.85\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.87\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.57\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.76\n1\n0.8\n0.65\n0.65\n0.58\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n1\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.87\n0.8\n0.9\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.87\n0.68\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.99\n0.8\n0.77\n0.65\n0.9\n0.65\n0.65\n0.88\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.88\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.89\n0.65\n0.65\n0.8\n0.8\n0.65\n0.7\n0.65\n0.65\n0.8\n0.9\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.8\n0.65\n0.77\n0.65\n0.65\n0.65\n0.65\n0.79\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.85\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.52\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.86\n0.65\n0.65\n0.8\n0.56\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.72\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.9\n0.65\n0.65\n0.8\n0.65\n0.8\n0.6\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.88\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.89\n0.85\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.87\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.75\n0.65\n0.65\n0.65\n0.65\n0.54\n1\n0.65\n0.65\n0.75\n0.65\n0.75\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.9\n0.62\n0.65\n0.65\n0.65\n0.65\n0.86\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.82\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.9\n0.74\n0.8\n0.65\n0.8\n0.8\n0.7\n0.65\n0.65\n0.65\n0.89\n0.65\n0.65\n0.8\n0.8\n0.8\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.8\n0.8\n0.84\n0.8\n0.65\n0.65\n0.8\n0.75\n0.65\n0.65\n0.65\n0.89\n0.65\n0.65\n0.65\n0.65\n0.82\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.84\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.8\n0.65\n0.7\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.74\n0.65\n0.8\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.65\n0.65\n0.85\n0.65\n0.9\n0.9\n0.65\n0.65\n0.65\n0.63\n0.82\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.7\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.65\n0.74\n0.9\n0.65\n0.8\n0.65\n0.65\n0.58\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.89\n0.75\n0.65\n0.65\n0.8\n0.65\n0.65\n0.88\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.89\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.8\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.87\n0.65\n0.65\n0.65\n0.8\n0.65\n0.64\n0.65\n0.65\n0.65\n0.8\n0.87\n0.65\n0.65\n0.8\n0.9\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.65\n0.89\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.83\n0.65\n0.65\n0.8\n0.65\n0.9\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.78\n0.65\n0.8\n0.65\n0.9\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.9\n0.65\n0.88\n0.8\n0.65\n0.65\n0.65\n0.81\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.77\n0.65\n0.65\n0.65\n0.8\n0.8\n0.8\n0.8\n0.65\n0.65\n0.65\n1\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.85\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.88\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.8\n0.8\n0.65\n0.65\n0.65\n0.65\n0.68\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.89\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.9\n0.65\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.81\n0.65\n0.65\n0.65\n0.8\n0.85\n0.65\n0.77\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.8\n0.8\n0.9\n0.65\n0.65\n0.89\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.8\n0.65\n0.65\n0.65\n0.88\n0.8\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.82\n0.65\n0.8\n0.74\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.85\n0.65\n0.65\n0.85\n0.65\n0.65\n0.65\n0.65\n0.7\n0.7\n0.8\n0.65\n0.65\n0.65\n0.65\n0.87\n0.8\n0.65\n0.65\n0.65\n0.89\n0.85\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.7\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.9\n0.8\n0.8\n0.65\n0.66\n0.57\n0.65\n0.65\n0.65\n0.49\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.65\n0.65\n0.65\n0.8\n0.65\n0.8\n0.8\n0.86\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.89\n0.65\n0.65\n0.65\n0.65\n0.65\n0.65\n0.76\n"
},
{
"path": "ram/data/tag_list.txt",
"content": "tennis\nbear cub\nobservatory\nbicycle\nhillside\njudge\nwatercolor illustration\ngranite\nlobster\nlivery\nstone\nceramic\nranch\ncloth\nsmile\nbuilding\ntattoo\ncricketer\ncheek\npear\nsource\nwinter\nsurface\nspray\nceremony\nmagic\ncurve\ncontainer\nfair\nmedicine\nbaby\ntennis racquet\nornament\nbamboo\nduckling\nsong\nsafari\nteam presentation\ndaffodil\ncross\ntoothpaste\nshield\nfashion model\ncapsule\nmap\ncreek\nglass house\nglass plate\nsiding\ncorner\nwater buffalo\nbison\nfigure skater\ndiploma\ntire\nrace\ncable car\nbrain\ngas stove\nsoap bubble\npalette\nsnowboard\nschool child\ntrench coat\nmonk\nfiber\nkitchen window\nsunglass\ncoffee\nsecurity\nstrawberry\npenguin\ntree root\nloaf\nengagement ring\nlamb\nvector cartoon illustration\nsandwich\nmountain village\nshape\ncharm\nfiction\nknot\ngreenhouse\nsushi\ntext\ndisaster\ntrophy\ngang\nstrap\nsoccer game\ncardinal\ntee\nturtle\nwater surface\ngrassland\ndolphin\nstore\ndirt\niceberg\npergola\nfarmer market\npublicity portrait\ntote bag\nteenage girl\nview mirror\nsession\ncommuter\ndressing room\ntricycle\nchristmas ball\nheadlight\npolice\narmchair\nchart\nyacht\nsaw\nprinter\nrock band\ngingerbread house\ntag\ntable lamp\nhockey game\nslope\nfont\nwicker basket\njewelry\nquarter\nsoftware\nweapon\npin\nworship\npainter\ngoal\nmorning light\nbike\nbaseball bat\nelevator\ncuisine\nsausage\nstunt\nwrestler\nstatue\nlanding\npillar\nwillow tree\nsea wave\nchicken\npeanut\nmuscle\nbob\ntv genre\nbathroom window\nradish\ntextile\npelican\nmarketplace\ncrest\nelevation map\ngift\nparish\ntraffic light\ncampfire\nfog\naward winner\nbeach ball\nmat\nwhite house\nplaster\nmoped\nfootball team\nsolution\nbicyclist\nbit\nplayground\ndarkness\ncake\nmaple leave\nmold\ncracker\nblueberry\nrubble\ncontainer ship\npedestrian bridge\nsnail\nparrot\nform\ncircuit\nhighlight\npickup truck\nkoala\nrain\nsystem\nweather\nraincoat\nsoccer team\nwindshield\nthunderstorm\nmike\nbird house\nbridge\ngrandfather\nrestroom\nanimation\nwilderness\nclown\nbanana\nbrown\nbraid\ndining room\nkindergarten\nlaunch event\npurple\nschool\nstairwell\nbrooch\nmovie poster image\nmountain river\nshelf\nwicket\nheadboard\nbuddha\nflower field\ndugout\ncd\nbald eagle\nlagoon\nseaweed\nagriculture\nemergency service\nmaple tree\nparachute\ncontinent\namusement park\nremote\nbun\ntackle\nhospital\ngarage door\nbirthday party\nfriendship\ngo\nmausoleum\njeep\nraccoon\nstep\nice hockey team\ncigarette\nlace dress\nforest floor\nmall\ncaptain\nmilk\ngolf course\nmeal\npicnic table\nsail\nvolleyball\ncanal\nterrace\ncomputer desk\ncaravan\nhotel\ncheerleader\nnurse\nmuseum\nmarsh\nfox\nplateau\nnight\ntwin\nletter logo\nautumn tree\npowder\nconvention\ncreature\nlighthouse\nshop window\njacket\nstork\ntaxi\ntrade\nblackboard\nolive\nroad sign\nresort\nsnowflake\ncemetery\ntravel\nevening dress\npicnic\ndrink\nwinter morning\nfootball player\nsnack\nboxing glove\ndinner party\nairline\nswing\nport\nwheelbarrow\nbathroom sink\nsweater\nambulance\ngear\noil\nwii controller\narray\nhome office\ncar show\nmixture\nprofession\ntree frog\nsquare\nfacility\ncoral reef\nsea wall\npizza\nexhibit\ndemolition\ntrout\nring\ncoffee shop\nbracelet\nbean\nlip\nfencing\nlandscape\nsitting\npackage\nmetal\nbust\nking\nhair\nwindow seat\nwildlife\ntrunk\ngreenery\nstencil\nfire hydrant\nbridesmaid\nplaza\nalps\ntower bridge\ncrop top\ncrossing\ncinema\npedestrian crossing\nfamily\nshopping cart\nstomach\nchurch building\nscreen door\nskater\nsoccer field\nkettle\nmussel\nraindrop\ncandy cane\nwater lily\nflower girl\ndesert\nenclosure\nchristmas light\nkitchen\ncaterpillar\nplaid\nbath\nbush\nmud\nballet\nknee\nadult\nraft\nsea view\ncactus\noffice chair\noverall\nrim\nscaffolding\npig\ncover\nposter page\nsprinkle\nchandelier\nalgae\ntraffic\nsurfboard\nbook\nfilming\nflash\nmansion\ncamouflage\ntrouser\nticket\nweed\ncab\ntrench\nelephant\nhuddle\nsphere\nchristmas decoration\ncity\nlaunch\ndoll\nchristmas ornament\nfabric\nbikini\nbiplane\nbreakfast\nneighbourhood\nrace track\nfoliage\navocado\nschool bus\nfootwear\nhighway\nocean view\nart vector illustration\nwall clock\ncurtain\nteenager\nkitchen area\nrobot\ntusk\nlounge chair\nbeam\npaddle\ncamel\nlid\nworld map\ncity view\nnewlywed\ncargo ship\nyellow\nexhibition\nbend\nnovel\nwool\nontario\nbread\ncampus\ncoastline\ncutting board\nbooth\ntable top\ncarpet\nbeach chair\nworkout\nstreet food\nfun\ncostumer film designer\ngadget\nartist\nfishing village\nbuilder\nviolinist\niphone\nspider web\ntraffic sign\nruin\nrescue\nclipboard\nseal\nfilm director\npaw\nnursery\nintersection\ntomato sauce\ntaste\npaddy field\nchristmas tree\nwave\nstool\nwatering can\nrug\ndaytime\nsubway station\ncraft\npine forest\nblack\nplanet\nmotif\nchristmas market\nglass window\ncollege\nwheat\ndamage\nrectangle\npicture frame\nchess\nguest room\nstreet corner\nreligion\nseed\npuzzle\nfreeway\nbeauty\nocean\nwatch\nmother\ngarage\nquote\ndj\nsupporter\nhip hop artist\nmuffin\neiffel tower\ncash\nfirefighter\ncauliflower\nbunker\nsled\nmanicure\nshark\nstall\njungle\nfamily home\ntour bus\nchimney\ntouchdown\nroundabout\ncoyote\nstreet scene\ntank\nwedding dress\nmantle\nbedroom window\ncoconut\nchapel\ngoat\nliving space\nrock wall\npolka dot\nrailway\nmandala\nmango\nlesson\nmountain landscape\nteam photo\nbookshelf\nmeter\nbulldog\nevening sun\nstick\ncard\npink\nfish pond\npaint\npill\ncart\npea\nvan\nalbum\nfootball college game\nmountain pass\ndoughnut\nski slope\nmatch\nofficial\nshadow\norgan\ncelebration\ncoin\nlog cabin\nfirework display\npresent\ntwig\nchef\nconfetti\nfootpath\ntour\nponytail\nartwork\nrace car\nclub\nseason\nhose\npencil\naircraft\nrock formation\nwardrobe\nparticipant\npolitician\nengineer\npeace\nfilter\nsailing boat\nwater bottle\nservice dog\npoodle\nloki\nstatesman\nsleeping bag\noutskirt\nclock\nfactory\noak tree\nphysician\ncolor\nroom\nstairway\ncompany\nlady\ngraph\nfaucet\ntablecloth\nsubway train\nchocolate chip cookie\nheadquarters\nscrew\ngoggle\nhalloween\ncity street\nswirl\ncord\nforward\nbone\nbedding\narchway\nwig\nlobby\nmask\nattic\nkitchen table\nskylight\nfire\nexit\noil painting\npassenger\nmeditation\nsalmon\nfedora\nrubber stamp\norange juice\narch\nscientist\nstroll\nmanhattan\nfloat\nbaseball uniform\ncircle\nchurch\ndecker bus\ncompetitor\nzoo\nbasketball team\ntourist\ndaughter\nsilverware\nceiling fan\nbirth\nvase\njack\nmushroom\nspiral\ncage\nlimb\nsalad\nad\ncontrol\nearth\nparty\nbolt\ntractor\nbarley\nwedding photo\nhawk\nwarehouse\nvegetable garden\nchocolate cake\ncabbage\nfloor window\nbaby shower\nmagnifying glass\ntable\nstethoscope\nreading\nmission\ncroissant\ngift box\nrocket\nforest road\ncooking\nsuite\nhill country\nmotorcycle\nbaseball player\nangle\ndrug\nsport association\nchampionship\nfamily portrait\nflorist\nsoftball\negret\noffice\nplywood\njockey\nmosque\nbrunch\nbeanie\noffice building\npattern\ncalendar\nindoor\npepper\nledge\ntrail\nfuel\nlaptop computer\ntennis shoe\ndeck chair\nguitarist\nbarn\nsurgery\ncartoon illustration\nnebula\nrailroad\nmountain goat\ngoose\ncar door\ncheer\nliquid\nhardwood floor\npathway\nacorn\ngull\nairliner\ncouch\nlake house\nspaghetti\npromenade\ncollection\ngarden\nbank\nrobin\ntennis ball\npeony\ngymnast\nlavender\ndeck\ntest\nriverside\nrapper\ndomino\nbride\nmouse\nbasil\nwedding couple\nocean wave\narm\nkitchen floor\ngrove\nfamily member\nbackyard\nraspberry\nforest fire\nofficer\nhibiscus\ncanyon\ncomposer\nsignature\nolive oil\nhibiscus flower\nrose\nvector icon\nsunrise\nhorseback\nmotor scooter\noffice worker\ntradition\ningredient\nwashing machine\nlighting\nbagel\nsailboat\npoliceman\nmare\ngraphic\nhalloween pumpkin\nstock\npilot\neducation\nteam\nbody\nhorse\nkimono\nbazaar\nbag\nrecording studio\nparsley\nentrance\ndenim\nvet\nhorse farm\ncharcoal\narchitecture\nglass vase\npuppy\nestuary\ntelevision show host\ncity bus\nshoulder\nbeast\nbalance\ngolfer\nroadside\ndenim jacket\nstone wall\ncounter top\napp icon\ntoast\nhead coach\nham\nwarrior\ngem\nrefrigerator\nsnowman\nconstruction worker\ncoal\nwebsite\nmorning fog\nmustard\nhuman\nowl\npuppy dog\npiggy bank\nvegetation\npirate\naction film\nmarshmallow\nthanksgiving\nbusiness\ndisease\nsignage\ngreeting\nskate park\ntile\nmouth\nspinach\nvacation\nleader\nshrine\nwalker\nscience fiction film\nbill\nrabbit\nmotor boat\nbar\nradio\nbarge\ntail\nchainsaw\ngallery\nrainbow\npasta\npadlock\nweb\npastry\nink\nreef\nschool uniform\nshawl\ntreasure\npeach\ndinner table\ninjury\nharbor\nwitch\ncar dealership\nlitter\ngesture\ndocumentary\nmarriage\nsea shell\npriest\ndome\nkit\nicon\nseaside\nbucket\nentertainment\nstable\nhat\npuddle\nsock\nshopper\ntechnology\nharbour\norbit\nantler\ntube\nflag waving\ncook\ntight\ncommander\nfarmland\nswitch\nhiker\nwedding ceremony\naward ceremony\nchampion\nchopstick\nfarmhouse\nperformer\nspike\naccident\ncruise ship\npassenger train\nattraction\nentertainer\nrear view\nsidewalk\nparade\nracing\nplane\nritual\npeacock\npocket\nplum\ndrop\ncarrot\nfloor\nsunset\ntroop\narchitect\ncoffee table\ndust\noutline\nleather\ncharity event\nheat\nwhale\nlaundry\ncoconut tree\ncrosswalk\npony\nant\npipe\nstring\ncoat\nangel\nbeef\nchurch tower\ndish\npitch\ncupboard\nthermometer\ndirt field\nfireworks\nminute\ncane\npajama\nflower garden\nautumn\ntrash can\ndachshund\nbanana tree\ntray\nmoose\nroadway\ncarnival\nantenna\npole\ncastle wall\nram\ncattle\nhay\ncookie\nswimmer\nbaseball team\nstrait\nhedge\njet\nfire pit\noctopus\ncalf\ncube\nopera\ncardboard box\ntiara\nkitchen sink\nprairie\nbowl\ngalaxy\nstraw hat\nlinen\nski resort\nstitch\nstreet lamp\nmotorist\nicicle\nstain\nflora\ndrain\nkitchen cabinet\ndecor\nbouquet\npound\ninterior design\nnail polish\nfigurine\ntomb\ndisc\ntwist\nblouse\nribbon\nfigure\nburger\ncork\nsoccer goalkeeper\ntrain bridge\ndrinking water\ndew\nbaker\nstorm cloud\ntarmac\ntv drama\nsponge\nmagnet\nsailor\nentry\nswan\nexercise\nsloth\njewel\nscuba diver\nbite\ncat tree\ntent\ncan\ntennis match\necosystem\npicket fence\npalm\ntrain car\nfrying pan\nrally\ntablet pc\nreindeer\nimage\nwolf\nchin\nconservatory\nflood water\ncityscape\nbeach sand\ncar park\npavement\nfarm field\nswimming\nwinter storm\nstem\npillow\ninning\ngorilla\ndesk\navenue\nfern\nmoney\npearl\ntrain station\nskillet\nnap\nbarber\nlibrary\nfreezer\nlabel\nrainforest\nparking sign\nmirror\nwing\nnoodle\npress room\nsculpture\ntablet\nviewer\nprayer\nmini\nmechanic\nlaugh\nrice field\nhand\nmustache\nmountain road\ncatwalk\nconference\ncape\ninstallation\nmusician\nstream\nmachine\nspeech\ncrocodile\nsoccer match\ntown square\npassport\npost box\npoint\nstone building\nmotorway\nmix\ndentist\nbusinessperson\nhappiness\nboat\nvineyard\ntreadmill\nglass wall\nwater droplet\ncoffee mug\ngraduate\nsunflower\nparliament\nshepherd\nmovie\nwine\norchard\ntulip\nmotherboard\ncup\nbroom\nspot\ndrawing\npolo shirt\ngraduation\nfilm producer\nmoonlight\nglow\nfilm format\nt shirt\nrock face\nsword\nclinic\nfestival day\nmeadow\nstaple\npupil\ntraining ground\nrider\nflower\nfoal\nwharf\nfoot bridge\nshooting\ntop\nmast\npolice car\nrobe\nwedding bouquet\nstop sign\nbirthday cake\nglitter\nbutter\nscooter\ntundra\nsuperhero\npocket watch\ninscription\nyoungster\nfruit tree\nmovie poster\nengine\nfoundation\nmotorcyclist\ntake\nwoman\nantelope\ncountry artist\nroad trip\ntypewriter\ntuxedo\nbrand\npine\nbathroom\nparadise\ntexture\nballoon\ndining table\nhome\ncomputer screen\nactor\nclip\ntv tower\npanorama\nsummit\ncat\nplot\neagle\ndancer\npup\nstudio shot\ntear\nbird bath\nclassroom\nbookstore\ncity wall\ntv programme\nblade\neasel\nbuttercream\nsweet\ndesigner\ndiamond\nhandshake\nherb\ncorn field\nseafront\nconcrete\nstreet artist\ngas\nstamp\nwindow display\npaper\nnote\npint\nquarry\nresearch\nfixture\nmanager\nsoil\nleopard\nboard game\nladder\nstop light\nisland\nramp\nfootball match\nicing\ndrill\ncurrency\nsummer evening\ntopping\npyramid\npomegranate\ncell\nivy\nsquad\nscenery\ncomputer\nlocomotive\nsurf\nmascot\ndune\npath\nduck\ntwilight\nwire\nbow tie\nstrike\ncormorant\ncar wash\ncrane\nmarket\nphilosopher\nalarm clock\ncamera\nbirch\ngreeting card\nplain\nclay\ndonut\nlock\nmoth\nlaboratory\nfan\nviolin\njazz fusion artist\nmountain biker\nterrain\nmagazine\npickup\ncomedy film\nsmartphone\nfilm\nbed\nmicrowave oven\ntournament\nlawn\ncar window\nalligator\nscreen\njetty\nshopping bag\nlandscape view\ncabinetry\nfriendly match\nthing\npetal\nshopping center\ntransport\nballet dancer\nshoreline\nprincess\ncar seat\nparking meter\ngreen\nvodka\nband\nrock\ncostume\nwarning sign\nstrip\nplaque\nwheelchair\nheadband\nginger\ndice\nmedia\nhairdresser\npress\nliving room\nstove\nplayer\ncherry\nworkshop\ncarving\nembroidery\ndoodle\nadventure\nrugby player\nmonument\nbrush\nmarker\nloft\npostcard\ncollage\nball\nprofessor\ndresser\ngig\nfestival\nblackbird\nmakeup artist\nvideo camera\nsticker\npeak\nwildflower\nsanta hat\nrodeo\nwedding photographer\nguy\nstaff\nwaterfall\noperation\ndefender\nfalcon\nhaze\nindividual\ngentleman\ngreyhound\nrocking chair\nrice\ngarbage\nplatter\nchocolate\nsplash\nbusiness suit\ncheetah\nvalley\nmaze\ntrampoline\ngarland\nslalom\nunicorn\ntree stump\npainting\nromance\nfight\nalcohol\nghost\nfondant\nspa\nshutter\ndeath\ndemonstration\ncotton\npier\nflea market\nhistory\nsavannah\nfist\naisle\ncrew\njug\npose\nanchor\nteapot\nboat house\nbusiness team\ntripod\nbee\npebble\nmattress\ncanvas\nhallway\ncampaign\npod\nlake district\narticle\nwhite\nsofa\nhoney\nmarathon\npancake\ntourist attraction\nwedding gown\nbattle\nshelving\nsea\nsheet music\npie\nyarn\nconstruction site\nflyer\ntie\nstar\nlettuce\nmartial artist\ndart\nstraw\nreflection\nconference room\ntemperature\nrugby\nmosquito\nphysicist\nrock climber\ncrash\nbackdrop\ntoilet seat\nsand castle\nwater park\ntoy car\nwaste\nluxury\nhangar\nrv\ntree trunk\nboard\ngold\nproject picture\ncap\ncottage\nrelief\nattire\nmicroscope\nbattery\nroll\nline\nparking garage\ncrystal\nbroadcasting\nbrick wall\nlab\nflooring\nmeeting\n3d cg rendering\ndesktop computer\ncowboy\nsailing ship\njunction\nhairstyle\nhomework\nprofile\nmodel\nflower pot\nstreet light\nsalt lake\nmaple\nspace\nblizzard\nthrow\nzebras\nbrochure\nconstellation\nbeak\nkilt\npond\nblue sky\nsneaker\nsand dune\nmorning sun\nalmond\ngrill\ncurl\nbasketball girl game\nchameleon\ntoilet bowl\nprince\nkeyboard\nqueen\ncomputer monitor\nwriting\ncrown\nbasilica\nkiss\nhouse\nparking\nfootball competition\nshell\nsport equipment\ncomedy\nbaboon\nvendor\nrise building\nwrap\nfood truck\ncat bed\nrickshaw\nflare\nteal\nnectar\neclipse\nvehicle\nsteam locomotive\ngorge\ncow\nchristmas card\ndemonstrator\nmemorial\ntowel\njewellery\ntrain\nfrisbee\nbaseball game\nfur\nafternoon sun\ncommunity\nsparkler\nbandage\nfirework\ndollar\npasture\nvideo\nbus\ntree house\nseashore\nfield\nhamburger\nsouvenir\nhedgehog\nworm\npine cone\nosprey\ndinosaur\nvegetable\njunk\nposter\narmy\nwinger\nbundle\nstage\ngrowth\nwedding party\nservice\nblanket\nruler\neye\ncredit card\ncastle\ndiner\nhut\nelk\nhard rock artist\nnun\ndog breed\nnest\ndrama film\nnumber icon\nwater tank\ngiraffe\naltar\npavilion\ntv personality\nsuv\nstreet vendor\nstreet sign\nditch\ndebris\nfoam\ntakeoff\nspice\nmountain lake\ntea\norchestra\nspacecraft\ncounter\nabbey\nmountain\nhydrangea\nracer\norange tree\ntide\ncowboy hat\nrapid\ntown\nwild\nherd\nvein\ndriveway\njar\nbark\nillustration\nhorror film\ncorn\nstroller\nindustry\nmountain stream\ngym\nneckline\npan\nclient\nspectator\neggplant\ncamper\nfawn\nhoodie\nmeat\nlemonade\nfood market\nslum\ncomic book character\nflower market\nlove\npalace\ngun\nheel\nshopping street\nshooting basketball guard\nfamily photo\nrooftop\nlaundry basket\nairport runway\nhorn\nface mask\nflight\nappetizer\nviolet\ncountry lane\ncement\ninstrument\ntv actor\nspark\ncelebrity\naward\ncountry house\nstanding\nauction\ndate\nengagement\npuck\nadvertisement\nchair\nzebra\ndriftwood\nbumblebee\nmaple leaf\nbonnet\norange\nwater tower\ndoor\nsinger\nfloor plan\ndiscussion\ntheatre\npilgrim\nmug\nbranch\nwindow sill\nbaseball pitcher\nbakery\nlollipop\nbasketball player\ntoilet paper\nchalkboard\ncabin\nsign\nnight sky\ncannon\nfishing net\nsubmarine\nsuit\nfur coat\nwine bottle\nfolder\nstreet art\nsuspension bridge\nevening sky\nbillboard\npostage stamp\nnewspaper\ntransportation\nsurgeon\nlight\npark\nhorizon\nroad\nsand bar\ntrumpet\nlounge\ncloud forest\nbirthday celebration\nbalcony\nanime\nbeehive\numbrella\ngoldfish\nbaseball cap\nwaterhole\nceiling\ncarousel\nbackpack\nplant pot\natmosphere\nsunflower field\nspire\nvision\nwoodpecker\nchip\npool table\nlotus flower\ncone\nhumpback whale\nreservoir\nhunt\npiano\nplate\ndining area\nluggage\nskier\ndance floor\ncrow\nstair\noverpass\nopera house\nbear\njazz artist\nwater\nvessel\ncast\nyard\ncathedral\nbasketball hoop\ngraveyard\nsound\nberry\nonlooker\nfauna\nbirch tree\nretail\nhill\nskeleton\njournalist\nfrost\nbasket\nnail\ndusk\ntrash\ndawn\nclover\nhen\nvolcano\nbasketball coach\nhome decor\ncharge\nhaircut\nsense\nuniversity\nlizard\ndaisy\ntablet computer\ngrass field\nprison\nmetal artist\nbathroom mirror\nwindow frame\nchest\nflavor\npop country artist\nmarket square\nmonkey\nblog\ndeer\nspeech bubble\ndog\nindependence day\ngirl\nboy\ntartan\nfurniture\nappliance\noffice window\nfish boat\nsand box\ntv sitcom\ndrama\nsleigh\ndepression\npaper towel\nbaseball\nprotestor\ngrape\nwedding cake\ninvitation\naccessory\npick\ngrandparent\nracket\ntea plantation\noutdoors\negg\nglass bowl\nsun\norganization\nlion\npanel\nstation\nwallpaper\nhelicopter\nsalt\nvanity\npatio\nlunch\nstreet performer\nmountain range\nsoup\nbacon\npower station\ncantilever bridge\nhummingbird\nshirt\nrope\nhip\nchalk\npendant\nchoir\ntv\nlichen\nrailway bridge\nart gallery\nbartender\nwagon\nbaby elephant\naccordion\nhorseshoe\nbuilding site\nclutch\nharvest\nsavanna\ngeranium\nbusiness woman\npaddock\npatch\nbeech tree\nwar\nsuburbs\nhospital bed\nmotorcycle racer\nmoss\ngravel\ngovernment agency\ndollar bill\nfather\nfjord\nconcert\nnut\nwedding photography\nfinish line\nhome plate\nfood\nnose\nthumb\nvillage\ndining room table\nbumper\nmonster\nblackberry\nlime\nconflict\ngala\nwallet\nwrist\nhug\nmermaid\nlava\nlawyer\nfolk rock artist\narena\nonion\ntoothbrush\nfashion\nperfume\nflip\ntriangle\nwoodland\nmail\ngrasshopper\nstudio\nwood floor\nden\nracquet\ncello\nlemur\nastronaut\nglass table\nblood\ndvd\nplanter\nsilver\nleash\nmaster bedroom\nforest\nbatter\nshoe\nengraving\nopening\nproduct\ntoe\ncocktail\nmallard duck\nbike ride\noasis\nwedding ring\ncinematographer\nholly\nautograph\nfence\nice cube\ncove\npineapple\naurora\nglass bead\nproduce\napartment building\ncob\nminiature\ncockpit\nflashlight\nfrog\nsheep\ngroom\nsteel\nwatermelon\nclip art\npaper plate\nostrich\ncontour\nmural\ncub\npaisley bandanna\nwinery\nturn\nhandle\nsatellite\npost\npork\nchild\nasphalt\ngrocery store\nvulture\ntrolley\nnightclub\nbrick\ntrailer\ncompass\ncereal\ncafe\ncartoon character\nsugar\nfiction book\nglass floor\numpire\nguitar\nhamster\nprotester\nairplane\ngarment\nblazer\nrailway line\nwedding\nshoe box\nparking lot\nconstruction\ngraduation ceremony\ntram\ntelescope\ncopper\npain\nautumn forest\nguest house\npartner\ncrayon\ndip\nboot\ncorridor\ncomputer keyboard\nhockey player\nchicken coop\nbus station\ngathering\nankle\nbunk bed\nwood table\nfootball coach\nmonarch\npharmacy\nlegging\nmannequin\nfemale\ntrain track\nstack\ncanopy\ndesign element\ngrandmother\nsymbol\nbeach hut\nzucchini\nbomb\nbusinessman\nskyscraper\ntongue\ncase\nsparkle\nhighland\nballroom\nprom\nestate\ncustomer\narchipelago\ncheese\ndebate\ncarriage\nbulldozer\npumpkin\nsitting room\ngas station\nwedding reception\ncamp\ndog bed\ntower\nproperty\nriver bed\npop latin artist\nfridge\nwine glass\ncoast\nbeer\ntow truck\nfire truck\nmountain bike\nthigh\nheron\nboat ride\ngondola\nturquoise\nlake\nllama\nkitty\ntin\nwaiting room\ncoffee cup\nsocialite\nguard\ntap\nwaterway\nforehead\nlist\nerosion\nbox\nsea lion\npollen\ndam\nwasp\nsalon\ntennis tournament\nflower box\naquarium\nrain cloud\nclothing store\nlead singer\ncupcake\ntortoise\nlettering\nsport facility\ndance\ndog house\nnature\nfootball\nrooster\nfootballer\nrailway track\ncrowd\nfishing rod\nsilhouette\nwind turbine\nsari\nbus window\ncloud\ncharity\nmedal\nyoga\nevent\nveil\nfashion menswear milan week\nnews\nknife\nprint\nscreen tv\nwalnut\nfungus\nice cream\ncomputer mouse\nplay\ntribe\npicture\nvideo game\nbusiness card\nmusic festival\nrack\nenvelope\nshower\ndirt road\nmine\noyster\nmonarch butterfly\ndude\nfruit salad\npodium\nfork\nlace\ntest match\nboulder\ncricket player\nstaircase\npeninsula\nshopping\npopcorn\noak\nmarket stall\npine tree\nmountaineer\nstudent\ncloset\nhood\nhandstand\ncenterpiece\ninsect\npatient\nmakeover\ntennis player\nsheet\npark bench\napple\norganism\nhook\nturkey\ntangerine\nsibling\nshopping mall\nbird\nscarf\nsmoothie\nnet\ngrass\nnapkin\nray\neyebrow\nlaptop keyboard\nmotorbike\nwoman hand\noven\nbook cover\neaster egg\nmicrowave\nsand\nsnapshot\nsoccer ball\nmakeup\nknight\nbowling ball\nshower curtain\nflame\nlightning\nrunning\npower plant\ncrib\ncartoon\nmoat\nfashion girl\nwedding invitation\nbottle\ncliff\nmonastery\nfile photo\napartment\ncasino\ncream\nsweatshirt\nstorm\ncruise\nteddy bear\nshovel\nwind farm\nwriter\ndock\nprofessional\nhotel room\njob\nmonitor\ndonkey\npass\ninterview\nduchess\nmark\nplank\nbeard\nzombie\ntrio\nchannel\ncricket team\nwindmill\nvest\ndiagram\ncable\nwinter scene\ngolden gate bridge\nbuffalo\nstudio portrait\npagoda\nwhiskey\nfreight train\nkite\nfuture\nsteam train\nphone box\nheadset\nwood\nsnowboarder\npaper bag\nslide\ngrapefruit\nseating\nmorning\nbronze sculpture\ntheatre actor\nstump\njean\nlandmark\njam\nwaist\nwatercolor\nhammock\nlight fixture\nice\nbasin\nbeverage\nshelter\npremiere\nmound\near\nbronze\nsunlight\nstreet\nenergy\nbarn door\nhike\nfleet\nclaw\nbeach\npepperoni\nbin\ntrainer\nbuffet\narchive\ntoddler\nreferee\nbay window\ndove\nproduction company\nevening light\ngate\nfarm\nreed\nfruit stand\nexplorer\nsnow storm\nthrow pillow\nbutton\ndisplay case\nbookcase\nlead\nlipstick\nbasketball court\ncargo\nensemble\npope\nclock tower\nteen\nspeaker\nrat\nlaptop\nski\nmess\nstadium\nferry boat\nbunny\nwaterfront\ndowntown\nsink\npress conference\ndinner\ncondiment\nthread\naudience\ngrid\ncar\nplastic\npeople\nbarbecue\npigeon\nurinal\nseagull\nvolunteer\nhockey\nfir tree\npollution\ntrial\ncollar\narea\nmeeting room\ncircus\nyogurt\norangutan\nviaduct\ncomedian\ndrone\nscissor\npop rock artist\nbiscuit\npanda\nwater feature\nair balloon\nremote control\nwatercolor painting\nshow\nwalk\npost office\nbike path\nrap gangsta artist\nmicrophone\ncrack\nsunset sky\nglass\ntv show\ncartoon style\nstripe\nfoyer\nsignal\ncalligraphy\nbulb\ngardener\ncoffee bean\nspider\ntapestry\ncity skyline\nnecklace\nkitten\ntraveler\nveteran\nfrosting\nfry\ntennis court\ntank top\nbutterfly house\nmist\ndrummer\nwater level\nscale\nbaseball glove\nmusic video performer\nchampagne\ncamping\nclothing\nwater drop\ntelephone box\npen\nmorning mist\nfire engine\nporch\nopening ceremony\nstyle\npalm tree\nfashion show\nuniverse\nscratch\naxe\nottoman\nexplosion\nrib\nboutique\ngame\ncucumber\nfruit\nstone bridge\nnature reserve\ntrack\ntrain window\npunch\ntelephone pole\nvelvet\nsauce\nmoon\ncontrast\nflamingo\nbat\nvending machine\nship\nequestrian\nshade\ncomforter\npallet\nsparrow\nwii\nglaze\ngrocery\nsteeple\nsoccer player\ncontract\nadvertising\nrunner\nchimpanzee\nworld\nseat\nproject\nchihuahua\nbubble\nwillow\npedestal\nsoul hip hop artist\ncurb\ndrawer\nleaf\nbanner\nlaunch party\ncoach\ngovernment\nsnowball\ntoy\nportrait\ndoctor\nwhiteboard\nelectronic\ntiger\ngraffiti\ncolumn\nnightstand\nwhistle\nmaxi dress\nbench\nwetsuit\nbird feeder\nfootball game\nbasketball\nclass\nbathroom door\nstore window\ntext message\nwreath\nstreet view\nbinocular\npet\nfacade\ndrought\nlemon\nnew year\nnight view\nairplane window\nspecie\nrule\njaw\nwheat field\ndiet\npop artist\nhabitat\nscreenshot\nscoreboard\nshore\nmane\nquilt\nski lift\norchid\nturban\nchristmas\nairport\nmarina\nglass door\nglass bottle\nrestaurant\nconductor\nlogo\nsleep\ntape\ntomato\nriver bank\nlilac\ntooth\ntraining\npottery\nshop\nsteam engine\nmason jar\nbase\nprocession\nborder\nshoot\nfootprint\nhotdog\nbull\nstocking\nrecreation\nautomobile model\ndesign\ncountry pop artist\nriver\nretriever\ndepartment store\nauditorium\nsport car\nsupermarket\nbelt\ncricket\nwindow box\ndress shirt\nletter\nresidence\nmegaphone\npant\nwildfire\nbird nest\ncrab\nswimsuit\ncandle\nfuneral\nmill\nnational park\nplant\ncop\npower line\nperch\nblue\nfinger\nferris wheel\nglobe\nskateboard\nhelmet\nmovie theater\nuniform\nhammer\nmaterial\nkid\nwell\nbutterfly\nsideline\nfashion fall show\nplanet earth\nlift\nmale\nsauna\ngray\nflour\nsand sculpture\nprogram\ncabinet\ninfant\nwheel\naircraft model\ndough\ngarlic\nskate\narrow\nwrapping paper\nripple\nlamp\niron\nbanknote\nbeaver\nferry\ncourtyard\nbassist\ncountryside\nsteak\ncomfort\nboxer\nlaundry room\ncampsite\nbrick building\ngolf\nsubway\nheadphone\nfort\nhandbag\ndrum\nflood\nsaddle\nbass\nlabyrinth\nneedle\nsun ray\napp\nmenu\npresident\ncardigan\ndandelion\nwetland\nice hockey player\nnumber\ncity hall\nfishing\nportrait session\npug\nkey\nart print\nminister\nhurdle\nemergency\npainting artist\nflag pole\nevening\npurse\nrecipe\ngolf ball\ncoloring book\nmountain peak\nsenior\nholiday\nbud\ncousin\npantry\nlap\nskin\nflag\ntissue paper\nridge\nwire fence\nsurfer\nclimber\nphotograph\nsewing machine\ncooler\nactress\napple tree\ncancer\nstarfish\nautomobile make\ndumbbell\nbrace\ntunnel\nwindow\npaint artist\ncomposition\nschool student\ncondo\nconvertible\ncushion\nselfie\nterritory\nguide\ntree\ncourt\nshrimp\nstone house\ndress\neyelash\njuice\nbroccoli\nchain\ntourism\nmountain top\nconcept car\nfilm premiere\nlight bulb\ncafeteria\nbadge\nflower bed\ntheater\nroot\nracecar driver\nbasketball boy game\nglove\nskyline\nwall\nglacier\nairport terminal\nbug\ntrim\nrailway station\nbriefcase\nflat\nfountain\nperson\nlane\nasparagus\nart\nlantern\ndishwasher\ndirector\nsnake\nlecture\ngame controller\ntree branch\npub\nbathing suit\nqueue\nbelly\npoppy\nbow\npitcher\nice cream cone\ncave\ncandy\nroad bridge\nhost\ntraffic jam\nearring\nfile\nfoot\nwatermark overlay stamp\nmailbox\nsupercar\nrailing\nbedroom\nseafood\nwaffle\nbronze statue\nplan\nflow\nmarble\nbasketball game\nautomobile\nscene\ncypress tree\nsoldier\nskateboarder\nglass building\ncherry tree\npump\ngrain\nwildebeest\nloop\nframe\nbathtub\nsaxophone\ndiver\nstalk\nlily\nbead\nalley\nflock\nfamily room\nmanufacturing\npointer\nworker\nnavy\npotato\nteacher\nphotography\ndolly\nboardwalk\nwater fountain\nathlete\nside dish\nbay\nice hockey\nphone\nhero\nface\ngold medal\nblind\nswamp\nresearcher\nswim\nmeatball\niguana\nleather jacket\njellyfish\nsite\nsmoke\ntraffic signal\nmelon\nbeetle\ncalculator\nskirt\nplantation\nsculptor\nbarrier\ncatcher\nsecurity guard\nsketch\nawning\nsteering wheel\nmountain view\nbus stop\npool\nleg\nspotlight\napron\nmineral\ninlet\nsleeve\ntorch\nemotion\nmarch\npolice officer\nperformance\nlamp post\nfishing boat\nsummer\npresentation\nsaucer\nsuitcase\nsupermodel\ngoalkeeper\nshrub\nrock artist\ndocument\nbeach house\nman\nblue artist\ncigar\nrailroad track\ngown\nmosaic\nbungalow\nalphabet\nbaseball field\nshed\npedestrian\nrail\nsoap\nkitchen counter\ndessert\ndunk\nblossom\nconversation\nfruit market\nglass jar\nmilitary\nbeer bottle\nphotographer\ntennis racket\ncompetition\nescalator\nbell tower\nstilt\nballerina\ntelevision\nfeather\nfence post\nrear\ndahlia\nred carpet\ntub\nhole\nfortress\npack\ntelephone\ncardboard\ncity park\nplatform\ncollege student\narch bridge\nwind\nblender\nbloom\nice rink\nbirthday\nraven\nfairy\nembankment\nhall\nflower shop\nsuburb\nbarrel\nbiker\nsteam\ndragonfly\nformation\nelectricity\nbusiness people\nsymmetry\nwalkway\nfisherman\ngas mask\nloch\nyouth\nhanger\ndot\nfish\nstreet market\nanimation film\ncrime fiction film\nboar\nemblem\nhalloween costume\nkangaroo\ncouple\nspoon\nsquirrel\nneon sign\nsky\noffice desk\nbeauty salon\nbreakwater\nfashion look\ntoaster\nauthor\nnews conference\noutdoor\ncanoe\ndragon\ntool\nshopping centre\nladybug\nswimming pool\nlandscaping\nski pole\nred\ntruck\nfly\ntemple\nlevel\nsunday\nrailroad bridge\ncar mirror\nlawn mower\nflute\naircraft carrier\nfashion menswear london week\nsunshine\ntile floor\nskull\nfossil\nflower arrangement\ndiaper\nsea turtle\ncherry blossom\nfireman\nshack\nlens\nwaiter\nanimal\nbasement\nsnow\nautumn park\nglass box\nkick\nhead\nanniversary\nvine\nback\npaper lantern\nfish tank\ncellphone\nsilk\ncoral\nnotebook\nphoto\ngazebo\nketchup\ndriver\nfarmer\nbonfire\nchestnut\nphotoshoot\nfootball field\nolive tree\npheasant\nsandal\ntoilet\nfireplace\nmusic\ndeity\nfish market\nfig\nbell\nneck\ngrave\nvilla\ncyclist\ncrate\ngrey\nasphalt road\nsoccer\nhostel\nmunicipality\ncourthouse\nroof\nend table\npot\nsedan\nstructure\nfolk artist\nsport\nsport team\nprotest\nsyringe\nfashion designer\njersey\nheart shape\nkayak\nstare\nsit with\ndirect\nread\nphotograph\nspin\nteach\nlaugh\ncarve\ngrow on\nwarm\nwatch\nstretch\nsmell\ndecorate\nshine\nlight\ndance\nsend\npark\nchase\ncollect\nlead\nkiss\nlead to\nlick\nsmile\ncheer\nsit\npoint\nblock\nrock\ndrop\ncut\nski\nwrap\nlose\nserve\nprovide\nsleep\ndress\nembrace\nburn\npack\nstir\ncreate\ntouch\nwash\nstick\nreveal\nshop\ntrain\npaint\ngroom\nhunt\nbloom\nplay\npay\nbrush\nshoot\nhold\npicture\ncarry\nsip\ncontain\nturn\npour\npitch\ngive\nadd\nblow\nlook in\nshow\nwalk\nilluminate\nkneel\ncover\ndrag\npost\npresent\nfit\noperate\nfish\nrace\nwrite\ndeliver\npeel\npush\nrun\nsit around\nbuy\njump\nwalk on\nattend\nclean\nsell\nride on\nmount\nhost\ndry\nplant\nsing\nrow\nshake\nperch\nride\nfight\nskateboard\nlive\ncall\nsurround\npractice\nplay on\nwork on\nstep\nrelax\nhit\nfall in\nflow\ngreet\nlaunch\nwear\nhang on\ndrive\nsit in\nbreak\nlearn\nfly\nconnect\ndisplay\nlocate\ncompete\ngo for\nsail\nlift\ntoast\nhelp\nrun on\nreflect\npose\nscratch\nframe\ndribble\nherd\nenter\nexit\nplace\ninspect\nbuild\npick\nfill\ngrind\nskate\noffer\nfloat\nsit by\nstand\nrelease\nrest\nsinge\nclimb\ntie\nmark\nlay\nstand around\ncapture\nset\nland\nswinge\nrun in\nkick\nlean\nhead\nsign\napproach\nswim\nclose\ncrash\ncontrol\nfall\nremove\nrepair\nopen\nappear\ntravel\nload\nmiss\ncheck\nsurf\nmoor\nsmoke\ndrink\nboard\nseat\nfeed\nrise\nsit on\nswing\ngrow\nstrike\ndate\nslide\nshare\ngraze\njump in\nlie\nextrude\nroll\nmove\ngather\neat\npull\nrun through\nsqueeze\nlay on\ndraw\nplay with\nwave\nassemble\nperform\nmarch\nscore\nattach\nadjust\nhang\nhug\nsleep on\nthrow\nlive in\ntalk\npet\nwork\nrun with\nsee\nflip\ncatch\ncook\nreceive\ncelebrate\nlook\nclassic\nbridal\nindoor\nindustrial\nteenage\nmini\ngrassy\naged\nlong\nwarm\nlight\nhandsome\nhappy\nthree\npregnant\ncircular\nurban\nsilver\nceramic\n3d\ngreen\nblonde\ngolden\ndark\ntropical\nripe\ndeep\nfat\nmusical\ngiant\nmedical\nmedieval\nbare\nstunning\nbold\ngeographical\nhuge\nplastic\nfoggy\nstormy\ngothic\nbiological\nempty\nclear\nantique\npink\nsteep\nbrown\nstriped\naerial\nrainy\ncool\nflying\ncommercial\npurple\ntrendy\nblank\nhaired\ndead\nwooden\nflat\nhigh\nbeige\npanoramic\nangry\ndozen\nrural\nsolar\nbig\nsmall\nstained\nthick\nmany\nfresh\nclean\nstrong\nabstract\ncrowded\nretro\ndry\ngorgeous\nmartial\nmodern\nblue\ncloudy\nlow\nfour\noutdoor\nsingle\nmuch\nbeautiful\nsnowy\npretty\nnew\nshort\nsunny\nclosed\nrocky\nred\ntwo\ndouble\nmale\ngray\nfive\ncolorful\nautomotive\nvarious\none\nold\nrusty\ntall\nwild\nnarrow\nnatural\nseveral\nfrozen\ntextured\nlush\nyoung\nhot\nmixed\nwhite\nfloat\nquiet\nround\nbright\nreligious\nfemale\nhistorical\nshiny\ntraditional\ntourist\nyellow\nbald\ncoastal\nlovely\nlittle\nbroken\nromantic\nwide\nroyal\nrich\nopen\ncute\nancient\ncold\npolitical\nelderly\ngold\nfull\nrustic\nmetallic\nfloral\nsad\nwet\nfancy\nsenior\ntiny\nstylish\nlarge\nfrosty\norange\ntransparent\nelectronic\nshallow\nscared\narmed\ndirty\nhistoric\nblack\nfew\nwindy\nsome\nsquare\nornamental\nsandy\nthin"
},
{
"path": "ram/models/__init__.py",
"content": "from .ram import ram\nfrom .tag2text import tag2text\n"
},
{
"path": "ram/models/bert.py",
"content": "'''\n * Copyright (c) 2022, salesforce.com, inc.\n * All rights reserved.\n * SPDX-License-Identifier: BSD-3-Clause\n * For full license text, see LICENSE.txt file in the repo root or https://opensource.org/licenses/BSD-3-Clause\n * By Junnan Li\n * Based on huggingface code base\n * https://github.com/huggingface/transformers/blob/v4.15.0/src/transformers/models/bert\n'''\n\nimport math\nimport os\nimport warnings\nfrom dataclasses import dataclass\nfrom typing import Optional, Tuple\n\nimport torch\nfrom torch import Tensor, device, dtype, nn\nimport torch.utils.checkpoint\nfrom torch import nn\nfrom torch.nn import CrossEntropyLoss\nimport torch.nn.functional as F\n\nfrom transformers.activations import ACT2FN\nfrom transformers.file_utils import (\n ModelOutput,\n)\nfrom transformers.modeling_outputs import (\n BaseModelOutputWithPastAndCrossAttentions,\n BaseModelOutputWithPoolingAndCrossAttentions,\n CausalLMOutputWithCrossAttentions,\n MaskedLMOutput,\n MultipleChoiceModelOutput,\n NextSentencePredictorOutput,\n QuestionAnsweringModelOutput,\n SequenceClassifierOutput,\n TokenClassifierOutput,\n)\nfrom transformers.modeling_utils import (\n PreTrainedModel,\n apply_chunking_to_forward,\n find_pruneable_heads_and_indices,\n prune_linear_layer,\n)\nfrom transformers.utils import logging\nfrom transformers.models.bert.configuration_bert import BertConfig\n\n\nlogger = logging.get_logger(__name__)\n\n\nclass BertEmbeddings_nopos(nn.Module):\n \"\"\"Construct the embeddings from word and position embeddings.\"\"\"\n\n def __init__(self, config):\n super().__init__()\n self.word_embeddings = nn.Embedding(config.vocab_size, config.hidden_size, padding_idx=config.pad_token_id)\n # self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.hidden_size)\n\n # self.LayerNorm is not snake-cased to stick with TensorFlow model variable name and be able to load\n # any TensorFlow checkpoint file\n self.LayerNorm = nn.LayerNorm(config.hidden_size, eps=config.layer_norm_eps)\n self.dropout = nn.Dropout(config.hidden_dropout_prob)\n\n # position_ids (1, len position emb) is contiguous in memory and exported when serialized\n # self.register_buffer(\"position_ids\", torch.arange(config.max_position_embeddings).expand((1, -1)))\n # self.position_embedding_type = getattr(config, \"position_embedding_type\", \"absolute\")\n \n self.config = config\n\n def forward(\n self, input_ids=None, position_ids=None, inputs_embeds=None, past_key_values_length=0\n ):\n if input_ids is not None:\n input_shape = input_ids.size()\n else:\n input_shape = inputs_embeds.size()[:-1]\n\n seq_length = input_shape[1]\n\n # if position_ids is None:\n # position_ids = self.position_ids[:, past_key_values_length : seq_length + past_key_values_length]\n\n if inputs_embeds is None:\n inputs_embeds = self.word_embeddings(input_ids)\n\n embeddings = inputs_embeds\n\n # if self.position_embedding_type == \"absolute\":\n # position_embeddings = self.position_embeddings(position_ids)\n # # print('add position_embeddings!!!!')\n # embeddings += position_embeddings\n embeddings = self.LayerNorm(embeddings)\n embeddings = self.dropout(embeddings)\n return embeddings\n\n\n\n\nclass BertEmbeddings(nn.Module):\n \"\"\"Construct the embeddings from word and position embeddings.\"\"\"\n\n def __init__(self, config):\n super().__init__()\n self.word_embeddings = nn.Embedding(config.vocab_size, config.hidden_size, padding_idx=config.pad_token_id)\n self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.hidden_size)\n\n # self.LayerNorm is not snake-cased to stick with TensorFlow model variable name and be able to load\n # any TensorFlow checkpoint file\n self.LayerNorm = nn.LayerNorm(config.hidden_size, eps=config.layer_norm_eps)\n self.dropout = nn.Dropout(config.hidden_dropout_prob)\n\n # position_ids (1, len position emb) is contiguous in memory and exported when serialized\n self.register_buffer(\"position_ids\", torch.arange(config.max_position_embeddings).expand((1, -1)))\n self.position_embedding_type = getattr(config, \"position_embedding_type\", \"absolute\")\n \n self.config = config\n\n def forward(\n self, input_ids=None, position_ids=None, inputs_embeds=None, past_key_values_length=0\n ):\n if input_ids is not None:\n input_shape = input_ids.size()\n else:\n input_shape = inputs_embeds.size()[:-1]\n\n seq_length = input_shape[1]\n\n if position_ids is None:\n position_ids = self.position_ids[:, past_key_values_length : seq_length + past_key_values_length]\n\n if inputs_embeds is None:\n inputs_embeds = self.word_embeddings(input_ids)\n\n embeddings = inputs_embeds\n\n if self.position_embedding_type == \"absolute\":\n position_embeddings = self.position_embeddings(position_ids)\n # print('add position_embeddings!!!!')\n embeddings += position_embeddings\n embeddings = self.LayerNorm(embeddings)\n embeddings = self.dropout(embeddings)\n return embeddings\n\n\nclass BertSelfAttention(nn.Module):\n def __init__(self, config, is_cross_attention):\n super().__init__()\n self.config = config\n if config.hidden_size % config.num_attention_heads != 0 and not hasattr(config, \"embedding_size\"):\n raise ValueError(\n \"The hidden size (%d) is not a multiple of the number of attention \"\n \"heads (%d)\" % (config.hidden_size, config.num_attention_heads)\n )\n \n self.num_attention_heads = config.num_attention_heads\n self.attention_head_size = int(config.hidden_size / config.num_attention_heads)\n self.all_head_size = self.num_attention_heads * self.attention_head_size\n\n self.query = nn.Linear(config.hidden_size, self.all_head_size)\n if is_cross_attention:\n self.key = nn.Linear(config.encoder_width, self.all_head_size)\n self.value = nn.Linear(config.encoder_width, self.all_head_size)\n else:\n self.key = nn.Linear(config.hidden_size, self.all_head_size)\n self.value = nn.Linear(config.hidden_size, self.all_head_size)\n\n self.dropout = nn.Dropout(config.attention_probs_dropout_prob)\n self.position_embedding_type = getattr(config, \"position_embedding_type\", \"absolute\")\n if self.position_embedding_type == \"relative_key\" or self.position_embedding_type == \"relative_key_query\":\n self.max_position_embeddings = config.max_position_embeddings\n self.distance_embedding = nn.Embedding(2 * config.max_position_embeddings - 1, self.attention_head_size)\n self.save_attention = False \n \n def save_attn_gradients(self, attn_gradients):\n self.attn_gradients = attn_gradients\n \n def get_attn_gradients(self):\n return self.attn_gradients\n \n def save_attention_map(self, attention_map):\n self.attention_map = attention_map\n \n def get_attention_map(self):\n return self.attention_map\n \n def transpose_for_scores(self, x):\n new_x_shape = x.size()[:-1] + (self.num_attention_heads, self.attention_head_size)\n x = x.view(*new_x_shape)\n return x.permute(0, 2, 1, 3)\n\n def forward(\n self,\n hidden_states,\n attention_mask=None,\n head_mask=None,\n encoder_hidden_states=None,\n encoder_attention_mask=None,\n past_key_value=None,\n output_attentions=False,\n ):\n mixed_query_layer = self.query(hidden_states)\n\n # If this is instantiated as a cross-attention module, the keys\n # and values come from an encoder; the attention mask needs to be\n # such that the encoder's padding tokens are not attended to.\n is_cross_attention = encoder_hidden_states is not None\n\n if is_cross_attention:\n # print(self.key.weight.shape)\n key_layer = self.transpose_for_scores(self.key(encoder_hidden_states))\n value_layer = self.transpose_for_scores(self.value(encoder_hidden_states))\n attention_mask = encoder_attention_mask\n elif past_key_value is not None:\n key_layer = self.transpose_for_scores(self.key(hidden_states))\n value_layer = self.transpose_for_scores(self.value(hidden_states))\n key_layer = torch.cat([past_key_value[0], key_layer], dim=2)\n value_layer = torch.cat([past_key_value[1], value_layer], dim=2)\n else:\n key_layer = self.transpose_for_scores(self.key(hidden_states))\n value_layer = self.transpose_for_scores(self.value(hidden_states))\n\n query_layer = self.transpose_for_scores(mixed_query_layer)\n\n past_key_value = (key_layer, value_layer)\n\n # compatible with higher versions of transformers \n if key_layer.shape[0] > query_layer.shape[0]:\n key_layer = key_layer[:query_layer.shape[0], :, :, :]\n attention_mask = attention_mask[:query_layer.shape[0], :, :]\n value_layer = value_layer[:query_layer.shape[0], :, :, :]\n\n # Take the dot product between \"query\" and \"key\" to get the raw attention scores.\n attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))\n\n if self.position_embedding_type == \"relative_key\" or self.position_embedding_type == \"relative_key_query\":\n seq_length = hidden_states.size()[1]\n position_ids_l = torch.arange(seq_length, dtype=torch.long, device=hidden_states.device).view(-1, 1)\n position_ids_r = torch.arange(seq_length, dtype=torch.long, device=hidden_states.device).view(1, -1)\n distance = position_ids_l - position_ids_r\n positional_embedding = self.distance_embedding(distance + self.max_position_embeddings - 1)\n positional_embedding = positional_embedding.to(dtype=query_layer.dtype) # fp16 compatibility\n\n if self.position_embedding_type == \"relative_key\":\n relative_position_scores = torch.einsum(\"bhld,lrd->bhlr\", query_layer, positional_embedding)\n attention_scores = attention_scores + relative_position_scores\n elif self.position_embedding_type == \"relative_key_query\":\n relative_position_scores_query = torch.einsum(\"bhld,lrd->bhlr\", query_layer, positional_embedding)\n relative_position_scores_key = torch.einsum(\"bhrd,lrd->bhlr\", key_layer, positional_embedding)\n attention_scores = attention_scores + relative_position_scores_query + relative_position_scores_key\n\n attention_scores = attention_scores / math.sqrt(self.attention_head_size)\n if attention_mask is not None:\n # Apply the attention mask is (precomputed for all layers in BertModel forward() function)\n attention_scores = attention_scores + attention_mask\n\n # Normalize the attention scores to probabilities.\n attention_probs = nn.Softmax(dim=-1)(attention_scores)\n \n if is_cross_attention and self.save_attention:\n self.save_attention_map(attention_probs)\n attention_probs.register_hook(self.save_attn_gradients) \n\n # This is actually dropping out entire tokens to attend to, which might\n # seem a bit unusual, but is taken from the original Transformer paper.\n attention_probs_dropped = self.dropout(attention_probs)\n\n # Mask heads if we want to\n if head_mask is not None:\n attention_probs_dropped = attention_probs_dropped * head_mask\n\n context_layer = torch.matmul(attention_probs_dropped, value_layer)\n\n context_layer = context_layer.permute(0, 2, 1, 3).contiguous()\n new_context_layer_shape = context_layer.size()[:-2] + (self.all_head_size,)\n context_layer = context_layer.view(*new_context_layer_shape)\n\n outputs = (context_layer, attention_probs) if output_attentions else (context_layer,)\n\n outputs = outputs + (past_key_value,)\n return outputs\n\n\nclass BertSelfOutput(nn.Module):\n def __init__(self, config):\n super().__init__()\n self.dense = nn.Linear(config.hidden_size, config.hidden_size)\n self.LayerNorm = nn.LayerNorm(config.hidden_size, eps=config.layer_norm_eps)\n self.dropout = nn.Dropout(config.hidden_dropout_prob)\n\n def forward(self, hidden_states, input_tensor):\n hidden_states = self.dense(hidden_states)\n hidden_states = self.dropout(hidden_states)\n hidden_states = self.LayerNorm(hidden_states + input_tensor)\n return hidden_states\n\n\nclass BertAttention(nn.Module):\n def __init__(self, config, is_cross_attention=False):\n super().__init__()\n self.self = BertSelfAttention(config, is_cross_attention)\n self.output = BertSelfOutput(config)\n self.pruned_heads = set()\n\n def prune_heads(self, heads):\n if len(heads) == 0:\n return\n heads, index = find_pruneable_heads_and_indices(\n heads, self.self.num_attention_heads, self.self.attention_head_size, self.pruned_heads\n )\n\n # Prune linear layers\n self.self.query = prune_linear_layer(self.self.query, index)\n self.self.key = prune_linear_layer(self.self.key, index)\n self.self.value = prune_linear_layer(self.self.value, index)\n self.output.dense = prune_linear_layer(self.output.dense, index, dim=1)\n\n # Update hyper params and store pruned heads\n self.self.num_attention_heads = self.self.num_attention_heads - len(heads)\n self.self.all_head_size = self.self.attention_head_size * self.self.num_attention_heads\n self.pruned_heads = self.pruned_heads.union(heads)\n\n def forward(\n self,\n hidden_states,\n attention_mask=None,\n head_mask=None,\n encoder_hidden_states=None,\n encoder_attention_mask=None,\n past_key_value=None,\n output_attentions=False,\n ):\n self_outputs = self.self(\n hidden_states,\n attention_mask,\n head_mask,\n encoder_hidden_states,\n encoder_attention_mask,\n past_key_value,\n output_attentions,\n )\n attention_output = self.output(self_outputs[0], hidden_states)\n outputs = (attention_output,) + self_outputs[1:] # add attentions if we output them\n return outputs\n\n\nclass BertIntermediate(nn.Module):\n def __init__(self, config):\n super().__init__()\n self.dense = nn.Linear(config.hidden_size, config.intermediate_size)\n if isinstance(config.hidden_act, str):\n self.intermediate_act_fn = ACT2FN[config.hidden_act]\n else:\n self.intermediate_act_fn = config.hidden_act\n\n def forward(self, hidden_states):\n hidden_states = self.dense(hidden_states)\n hidden_states = self.intermediate_act_fn(hidden_states)\n return hidden_states\n\n\nclass BertOutput(nn.Module):\n def __init__(self, config):\n super().__init__()\n self.dense = nn.Linear(config.intermediate_size, config.hidden_size)\n self.LayerNorm = nn.LayerNorm(config.hidden_size, eps=config.layer_norm_eps)\n self.dropout = nn.Dropout(config.hidden_dropout_prob)\n\n def forward(self, hidden_states, input_tensor):\n hidden_states = self.dense(hidden_states)\n hidden_states = self.dropout(hidden_states)\n hidden_states = self.LayerNorm(hidden_states + input_tensor)\n return hidden_states\n\n\nclass BertLayer(nn.Module):\n def __init__(self, config, layer_num):\n super().__init__()\n self.config = config\n self.chunk_size_feed_forward = config.chunk_size_feed_forward\n self.seq_len_dim = 1\n self.attention = BertAttention(config) \n self.layer_num = layer_num \n if self.config.add_cross_attention:\n self.crossattention = BertAttention(config, is_cross_attention=self.config.add_cross_attention)\n self.intermediate = BertIntermediate(config)\n self.output = BertOutput(config)\n\n def forward(\n self,\n hidden_states,\n attention_mask=None,\n head_mask=None,\n encoder_hidden_states=None,\n encoder_attention_mask=None,\n past_key_value=None,\n output_attentions=False,\n mode=None,\n ):\n \n if mode == 'tagging':\n \n assert encoder_hidden_states is not None, \"encoder_hidden_states must be given for cross-attention layers\"\n\n cross_attention_outputs = self.crossattention(\n hidden_states,\n attention_mask,\n head_mask,\n encoder_hidden_states,\n encoder_attention_mask,\n output_attentions=output_attentions,\n )\n attention_output = cross_attention_outputs[0]\n outputs = cross_attention_outputs[1:-1] # add cross attentions if we output attention weights \n\n present_key_value = cross_attention_outputs[-1]\n\n else:\n # decoder uni-directional self-attention cached key/values tuple is at positions 1,2\n self_attn_past_key_value = past_key_value[:2] if past_key_value is not None else None\n self_attention_outputs = self.attention(\n hidden_states,\n attention_mask,\n head_mask,\n output_attentions=output_attentions,\n past_key_value=self_attn_past_key_value,\n )\n attention_output = self_attention_outputs[0]\n\n outputs = self_attention_outputs[1:-1]\n present_key_value = self_attention_outputs[-1]\n\n if mode=='multimodal':\n assert encoder_hidden_states is not None, \"encoder_hidden_states must be given for cross-attention layers\"\n\n cross_attention_outputs = self.crossattention(\n attention_output,\n attention_mask,\n head_mask,\n encoder_hidden_states,\n encoder_attention_mask,\n output_attentions=output_attentions,\n )\n attention_output = cross_attention_outputs[0]\n outputs = outputs + cross_attention_outputs[1:-1] # add cross attentions if we output attention weights \n layer_output = apply_chunking_to_forward(\n self.feed_forward_chunk, self.chunk_size_feed_forward, self.seq_len_dim, attention_output\n )\n outputs = (layer_output,) + outputs\n\n outputs = outputs + (present_key_value,)\n\n return outputs\n\n def feed_forward_chunk(self, attention_output):\n intermediate_output = self.intermediate(attention_output)\n layer_output = self.output(intermediate_output, attention_output)\n return layer_output\n\n\nclass BertEncoder(nn.Module):\n def __init__(self, config):\n super().__init__()\n self.config = config\n self.layer = nn.ModuleList([BertLayer(config,i) for i in range(config.num_hidden_layers)])\n self.gradient_checkpointing = False\n\n def forward(\n self,\n hidden_states,\n attention_mask=None,\n head_mask=None,\n encoder_hidden_states=None,\n encoder_attention_mask=None,\n past_key_values=None,\n use_cache=None,\n output_attentions=False,\n output_hidden_states=False,\n return_dict=True,\n mode='multimodal',\n ):\n all_hidden_states = () if output_hidden_states else None\n all_self_attentions = () if output_attentions else None\n all_cross_attentions = () if output_attentions and self.config.add_cross_attention else None\n\n next_decoder_cache = () if use_cache else None\n \n for i in range(self.config.num_hidden_layers):\n layer_module = self.layer[i]\n if output_hidden_states:\n all_hidden_states = all_hidden_states + (hidden_states,)\n\n layer_head_mask = head_mask[i] if head_mask is not None else None\n past_key_value = past_key_values[i] if past_key_values is not None else None\n\n if self.gradient_checkpointing and self.training:\n\n if use_cache:\n logger.warn(\n \"`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...\"\n )\n use_cache = False\n\n def create_custom_forward(module):\n def custom_forward(*inputs):\n return module(*inputs, past_key_value, output_attentions)\n\n return custom_forward\n\n layer_outputs = torch.utils.checkpoint.checkpoint(\n create_custom_forward(layer_module),\n hidden_states,\n attention_mask,\n layer_head_mask,\n encoder_hidden_states,\n encoder_attention_mask,\n mode=mode,\n )\n else:\n layer_outputs = layer_module(\n hidden_states,\n attention_mask,\n layer_head_mask,\n encoder_hidden_states,\n encoder_attention_mask,\n past_key_value,\n output_attentions,\n mode=mode,\n )\n\n hidden_states = layer_outputs[0]\n if use_cache:\n next_decoder_cache += (layer_outputs[-1],)\n if output_attentions:\n all_self_attentions = all_self_attentions + (layer_outputs[1],)\n\n if output_hidden_states:\n all_hidden_states = all_hidden_states + (hidden_states,)\n\n if not return_dict:\n return tuple(\n v\n for v in [\n hidden_states,\n next_decoder_cache,\n all_hidden_states,\n all_self_attentions,\n all_cross_attentions,\n ]\n if v is not None\n )\n return BaseModelOutputWithPastAndCrossAttentions(\n last_hidden_state=hidden_states,\n past_key_values=next_decoder_cache,\n hidden_states=all_hidden_states,\n attentions=all_self_attentions,\n cross_attentions=all_cross_attentions,\n )\n\n\nclass BertPooler(nn.Module):\n def __init__(self, config):\n super().__init__()\n self.dense = nn.Linear(config.hidden_size, config.hidden_size)\n self.activation = nn.Tanh()\n\n def forward(self, hidden_states):\n # We \"pool\" the model by simply taking the hidden state corresponding\n # to the first token.\n first_token_tensor = hidden_states[:, 0]\n pooled_output = self.dense(first_token_tensor)\n pooled_output = self.activation(pooled_output)\n return pooled_output\n\n\nclass BertPredictionHeadTransform(nn.Module):\n def __init__(self, config):\n super().__init__()\n self.dense = nn.Linear(config.hidden_size, config.hidden_size)\n if isinstance(config.hidden_act, str):\n self.transform_act_fn = ACT2FN[config.hidden_act]\n else:\n self.transform_act_fn = config.hidden_act\n self.LayerNorm = nn.LayerNorm(config.hidden_size, eps=config.layer_norm_eps)\n\n def forward(self, hidden_states):\n hidden_states = self.dense(hidden_states)\n hidden_states = self.transform_act_fn(hidden_states)\n hidden_states = self.LayerNorm(hidden_states)\n return hidden_states\n\n\nclass BertLMPredictionHead(nn.Module):\n def __init__(self, config):\n super().__init__()\n self.transform = BertPredictionHeadTransform(config)\n\n # The output weights are the same as the input embeddings, but there is\n # an output-only bias for each token.\n self.decoder = nn.Linear(config.hidden_size, config.vocab_size, bias=False)\n\n self.bias = nn.Parameter(torch.zeros(config.vocab_size))\n\n # Need a link between the two variables so that the bias is correctly resized with `resize_token_embeddings`\n self.decoder.bias = self.bias\n\n def forward(self, hidden_states):\n hidden_states = self.transform(hidden_states)\n hidden_states = self.decoder(hidden_states)\n return hidden_states\n\n\nclass BertOnlyMLMHead(nn.Module):\n def __init__(self, config):\n super().__init__()\n self.predictions = BertLMPredictionHead(config)\n\n def forward(self, sequence_output):\n prediction_scores = self.predictions(sequence_output)\n return prediction_scores\n\n\nclass BertPreTrainedModel(PreTrainedModel):\n \"\"\"\n An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained\n models.\n \"\"\"\n\n config_class = BertConfig\n base_model_prefix = \"bert\"\n _keys_to_ignore_on_load_missing = [r\"position_ids\"]\n\n def _init_weights(self, module):\n \"\"\" Initialize the weights \"\"\"\n if isinstance(module, (nn.Linear, nn.Embedding)):\n # Slightly different from the TF version which uses truncated_normal for initialization\n # cf https://github.com/pytorch/pytorch/pull/5617\n module.weight.data.normal_(mean=0.0, std=self.config.initializer_range)\n elif isinstance(module, nn.LayerNorm):\n module.bias.data.zero_()\n module.weight.data.fill_(1.0)\n if isinstance(module, nn.Linear) and module.bias is not None:\n module.bias.data.zero_()\n\n\nclass BertModel(BertPreTrainedModel):\n \"\"\"\n The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of\n cross-attention is added between the self-attention layers, following the architecture described in `Attention is\n all you need