[ { "path": ".github/workflows/build_executable.yaml", "content": "name: Build executables\n\non: [push]\n\nenv:\n VERSION: \"5.3\"\n\njobs:\n build-matrix:\n strategy:\n matrix:\n os: [ubuntu-18.04, ubuntu-20.04, ubuntu-22.04, windows-2019, macos-10.15, macos-11, macos-12]\n include:\n - os: windows-2019\n suffix: .exe\n runs-on: ${{ matrix.os }}\n steps:\n - uses: actions/checkout@v2\n - uses: actions/setup-python@v2\n with:\n python-version: '3.9'\n - name: Build\n run: |\n . ./build_scripts/install_dependencies.sh ${{ matrix.os }}\n . ./build_scripts/build.sh ${{ matrix.os }}\n shell: bash\n - name: Release\n uses: softprops/action-gh-release@v1\n # if: startsWith(github.ref, 'refs/tags/')\n with:\n draft: true\n files: | # ./photomosaic-maker-${{ env.VERSION }}-${{ matrix.version }}-x64-build.tar.gz\n ./dist/photomosaic-maker-${{ env.VERSION }}-${{ matrix.os }}-x64${{ matrix.suffix }}\n ./dist/photomosaic-maker-${{ env.VERSION }}-${{ matrix.os }}-x64.tar.gz" }, { "path": ".gitignore", "content": ".vscode\nitchat.pkl\nimg/\ntests/\nbuild/\n*.spec\n__pycache__/\n*test*.sh\n.mypy_cache/\nresult.png\n.pylintrc\ntest.bat\ndist\nvenv" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2017 -T.K.- hanzhi713 -kaiyingshan\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE." }, { "path": "README.md", "content": "\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n
TilesPhotomosaic (Fair tile usage)
Tiles Sorted by RGB sumPhotomosaic (Best-fit)
\r\n\r\n# Photomosaic Maker\r\n\r\n![gui demo](./examples/gui.png)\r\n\r\n- [Distinguishing Features of this Photomosaic Maker](#distinguishing-features-of-this-photomosaic-maker)\r\n- [Getting Started](#getting-started)\r\n - [Using the pre-built binary](#using-the-pre-built-binary)\r\n - [Running Python script directly](#running-python-script-directly)\r\n- [Command line usage](#command-line-usage)\r\n - [Option 1: Sorting](#option-1-sorting)\r\n - [Option 2: Make a photomosaic](#option-2-make-a-photomosaic)\r\n - [Option 2.1: Give a fair chance to each tile](#option-21-give-a-fair-chance-to-each-tile)\r\n - [Option 2.2: Best fit (unfair tile usage)](#option-22-best-fit-unfair-tile-usage)\r\n - [Display salient object only](#display-salient-object-only)\r\n - [Keep transparency](#keep-transparency)\r\n - [Blending Options](#blending-options)\r\n - [Dithering](#dithering)\r\n - [Option 3: Photomosaic Video](#option-3-photomosaic-video)\r\n - [Performance, multiprocessing and GPU acceleration](#performance-multiprocessing-and-gpu-acceleration)\r\n - [Time and space complexity](#time-and-space-complexity)\r\n - [Multiprocessing](#multiprocessing)\r\n - [GPU acceleration](#gpu-acceleration)\r\n - [All command line options](#all-command-line-options)\r\n- [Utility Script: download profile pictures of your WeChat friends](#utility-script-download-profile-pictures-of-your-wechat-friends)\r\n - [Groupchat Members](#groupchat-members)\r\n - [All available profile pictures](#all-available-profile-pictures)\r\n - [Notes](#notes)\r\n- [Credits (Names in alphabetical order)](#credits-names-in-alphabetical-order)\r\n\r\n\r\n## Distinguishing Features of this Photomosaic Maker\r\n\r\nA number of photomosaic makers already exist (like [mosaic](https://github.com/codebox/mosaic) and [Photomosaic-generator](https://github.com/uvipen/Photomosaic-generator)), but this photomosaic maker has the following unique features\r\n\r\n- Can trade off between the fairness of the tiles and quality of the constructed photomosaic\r\n - Can ensure each tile is used exactly N times if desired (N is customizable)\r\n- Supports non square tile size\r\n- Supports photomosaic videos\r\n- Supports maintaining transparency\r\n- Supports dithering\r\n- Supports saliency detection\r\n- Has a graphical user interface\r\n- Optional GPU acceleration\r\n\r\n## Getting Started\r\n\r\nYou can either use our pre-built binaries from [release](https://github.com/hanzhi713/image-collage-maker/releases) or directly run our python script.\r\n\r\n### Using the pre-built binary\r\n\r\n**If you need GPU acceleration or need to make [photomosaic videos](#option-3-photomosaic-video), please refer to the [Command line usage](#command-line-usage)**\r\n\r\nBinaries can be downloaded from [release](https://github.com/hanzhi713/image-collage-maker/releases).\r\n\r\nOn Windows and MacOS, my program may be blocked because it is not signed (signing costs money!). Don't worry as there is no security risk. On MacOS or Linux, after downloading the binary, you may need to add executing permission. Open your terminal, go to the file's directory and type\r\n\r\n```bash\r\nchmod +x ./photomosaic-maker-5.1-macos-x64\r\n```\r\n\r\nThen you can run from terminal as\r\n\r\n```bash\r\n./photomosaic-maker-5.1-macos-x64\r\n```\r\n\r\n### Running Python script directly\r\n\r\nFirst, you need Python >= 3.7 with pip. You can install dependencies by running\r\n\r\n```bash\r\npip install -r requirements.txt\r\n```\r\n\r\nNote: if have problems installing the packages due to missing dependencies, you can use this command (on Linux) to install them one by one. \r\n\r\n```bash\r\ncat requirements.txt | xargs -n 1 pip install\r\n```\r\n\r\nIf you want GPU acceleration, you need to install cupy. Please consult the [cupy documentation](https://docs.cupy.dev/en/stable/install.html).\r\n\r\nThen, you can either use the GUI by running or refer to the commandline usage.\r\n\r\n```bash\r\npython gui.py\r\n```\r\n\r\nIf you see errors like `No module named '_tkinter'`, you may need to install tkinter in your system like this (command will vary depending on the OS)\r\n\r\n```bash\r\nsudo apt-get install python3-tk\r\n```\r\n\r\nFor command line usage and documentation regarding different photmosaic options, please refer to the section below.\r\n\r\n## Command line usage\r\n\r\n> If you do not wish to use the GUI, a command line interface is also available. Make sure that you've installed dependencies in the section above. \r\n\r\n### Option 1: Sorting\r\n\r\n```bash\r\npython make_img.py --path img/zhou --sort bgr_sum --size 50 --out examples/sort-bgr.png\r\n```\r\n\r\n`--size` takes one or two arguments. If only one is specified, it is interpreted as the tile width and tile height will be inferred from the aspect ratios of the tiles provided (this corresponds to the `infer height` option in the GUI). If two are specified, they are interpreted as width and height. \r\n\r\nUse `--ratio w h` to change the aspect ratio, whose default is 16:9. E.g. `--ratio 21 9` specifies the aspect ratio to be 21:9. \r\n\r\n> Note: when the tiles are a bit short to completely fill the grid, white tiles will be added. \r\n\r\nResult:\r\n\r\n\r\n\r\n### Option 2: Make a photomosaic\r\n\r\nTo make a photomosaic, specify the path to the destination image using `--dest_img`\r\n\r\n#### Option 2.1: Give a fair chance to each tile\r\n\r\nThis fitting option ensures that each tile is used for the same amount of times, but is the most computationally and memory intensive option. \r\n\r\n> a few tiles might be used one more time than others. This may happen when the number of tiles is not an integer multiple of the blocks of the destination image. \r\n\r\n```bash\r\npython make_img.py --path img/zhou --dest_img examples/dest.jpg --size 25 --dup 6 --out examples/fair-dup-10.png\r\n```\r\n\r\n`--dup 6` specifies that each tile needs to be used 6 times (i.e. duplicates the set of tiles 6 times). Increase that number if you don't have enough source tiles or you want a better fitting result. This can be a non integer too. For example, `--dup 0.5` means only 50% of the tiles will be used, and `--dup 2.5` means all tiles on average will be used 2.5 times (half of the tiles will be used 2 times and the other half will be used 3 times). \r\n\r\nTo make sure the computation completes within a reasonable amount of time, it is recommended that you use less than 6000 tiles after duplication. Tile number larger than 6000 will probably takes longer than a minute to compute. Note that this recommended limit does **not** apply for the best fit option (see section below). \r\n\r\n| Original | Fitting Result |\r\n| ------------------------------------------- | -------------------------------------------------- |\r\n| | |\r\n\r\n#### Option 2.2: Best fit (unfair tile usage)\r\n\r\nThis fitting option just selects the best subset of tiles you provided to approximate your destination tiles. Each tile in that subset will be used for an arbitrary number of times.\r\n\r\nAdd `--unfair` flag to enable this option. You can also specify `--max_width` to change the width of the grid. The height will be automatically calculated based on the max_width provided. Generally, a larger grid will give a better result. The default value is 80.\r\n\r\n```bash\r\npython make_img.py --path img/zhou --dest_img examples/dest.jpg --size 25 --unfair --max_width 56 --out examples/best-fit.png\r\n```\r\n\r\n| Original | Fitting Result |\r\n| :-----------------------------------------: | :---------------------------------------------: |\r\n| | |\r\n\r\nOptionally, you can specify the `--freq_mul` (frequency multiplier) parameter that trade offs between the fairness of the tiles and quality of the mosaic. \r\n\r\n```bash\r\npython3 make_img.py --path img --out best-fit.png --dest_img img/1.png --size 25 --unfair --freq_mul 1.0\r\n```\r\n\r\nThe larger the `freq_mul`, more tiles will be used to construct the photomosaic, but the quality will deteriorate. The results under different `freq_mul` are shown below. Note that if you need a large `freq_mul`, you will better off by going for the fair tile usage (see section above) instead.\r\n\r\n![](examples/fairness.png)\r\n\r\n#### Display salient object only\r\n\r\nThis option makes photomosaic only for the salient part of the destination image. Rest of the area will be transparent. \r\n\r\nAdd `--salient` flag to enable this option. You can still specify whether each tile is used for the same amount of times with the `--unfair` flag.\r\n\r\nUse `--lower_thresh` to specify the threshold for object detection. The threshold ranges from 0.0 to 1.0; a higher threshold would lead to less object area. The default threshold is 0.5.\r\n\r\n```bash\r\npython make_img.py --path img/zhou --dest_img examples/messi.jpg --size 25 --salient --lower_thresh 0.15 --dup 5 --out examples/messi-fair.png\r\n```\r\n\r\n```bash\r\npython make_img.py --path img/zhou --dest_img examples/messi.jpg --size 25 --salient --lower_thresh 0.15 --unfair --max_width 115 --out examples/messi-unfair.png\r\n```\r\n\r\n| Original | Unfair-Fitting Result | Fair-Fitting Result |\r\n| -------------------------------------------- | --------------------------------------------------- | ------------------------------------------------- |\r\n| | | |\r\n\r\n#### Keep transparency\r\n\r\nIf your destination image has transparent regions, you can add `--transparent` flag to only put tiles for the non transparent part. In this way, the transparent regions are maintained in the resulting photomosaic. Note that this option is not compatible with `--dithering` and `--salient`. \r\n\r\n| Original | Unfair-Fitting Result (`freq_mul=0.5`) | Fair-Fitting Result |\r\n| -------------------------------------------------- | --------------------------------------------------------- | -------------------------------------------------- |\r\n| | | |\r\n\r\n\r\n#### Blending Options\r\n\r\nTo enhance the effect of the photomosaic, you can enable alpha or brightness blending. Use the `--blending` option to select the types of blending and `--blending_level` to change the level of blending. \r\n\r\n```bash\r\n# alpha blending\r\npython make_img.py --path img/zhou --dest_img examples/dest.jpg --size 25 --dup 8 --blending alpha --blending_level 0.25 --out examples/blend-alpha-0.25.png\r\n\r\n# brightness blending\r\npython make_img.py --path img/zhou --dest_img examples/dest.jpg --size 25 --dup 8 --blending brightness --blending_level 0.25 --out examples/blend-brightness-0.25.png\r\n```\r\n\r\n| Fair tile usage, no blending | Alpha blending (25%) | Brightness blending (25%) |\r\n| -------------------------------------------------- | ------------------------------------------------------- | ------------------------------------------------------------ |\r\n| | | |\r\n\r\n#### Dithering\r\n\r\n> See https://en.wikipedia.org/wiki/Dither for a detailed explanation of dithering\r\n\r\nDithering can be used to reduce color banding when there exists a color gradient. To enable dithering, add `--dither` flag. My implementation uses [Floyd–Steinberg dithering](https://en.wikipedia.org/wiki/Floyd%E2%80%93Steinberg_dithering). Note that this option is incompatible with `--transparent` and `--salient`. \r\n\r\n```bash\r\npython make_img.py --path img/zhou --dest_img examples/dest2.jpg --size 10 --unfair --max_width 200 --freq_mul 0.0 --dither --out examples/dither.png\r\n```\r\n\r\n| Original image | Best fit, no dither | Best fit, dither |\r\n| -------------------------------------------- | ------------------------------------------------ | --------------------------------------------- |\r\n| | | |\r\n\r\nWhile dithering works the best when `freq_mul` is set to zero, it can still work and provide some visual differences when `freq_mul > 0`. \r\n\r\n```bash\r\n# dither when freq_mul is 0.1\r\npython make_img.py --path img/zhou --dest_img examples/dest2.jpg --size 10 --unfair --max_width 200 --freq_mul 0.1 --dither --deterministic --out examples/f-dither.png\r\n```\r\n\r\n| Original image | `freq_mul = 0.1`, no dither | `freq_mul = 0.1`, dither |\r\n| -------------------------------------------- | -------------------------------------------------- | ----------------------------------------------- |\r\n| | | |\r\n\r\nNote that dithering is **not supported** in fair mode, when randomization is enabled or when saliency is enabled. Also, dithering is not recommended to use with `--gpu`, or you may experience slow computation compared to CPU processing. \r\n\r\n### Option 3: Photomosaic Video\r\n\r\n![photomosaic-video](examples/v.gif)\r\n\r\nIt is possible to make a photomosaic video simply by repeating the methods listed in Option 2 to every certain frame (specified by `--skip_frame`) of the video. You can pass the path of the video with `--dest_img` and add the `--video` flag to tell the program it is a video. This is much faster than processing the video manually frame by frame (e.g. pass different `dest_img` each time), because a lot of information is cached and can be reused between frames. Example:\r\n\r\n```bash\r\npython make_img.py --path img/catsdogs --dest_img img/2out.mp4 --size 20 --unfair --max_width 100 --freq_mul 2 --out tests/video/frame.png --gpu --video --skip_frame 2\r\n```\r\n\r\nSince photomosaic version 5.2, the output will be uncompressed individual frames rather than a video. This allows users to use custom video encoding format. \r\n\r\nDo note that some options are not supported, and some options are slower than other. Generally, saliency is not recommended to use on videos due to its long computational time and difficulty to tune. Each frame might need its own threshold. \r\n\r\n| Saliency/Fairness | Fair | Unfair |\r\n| ----------------- | ----------- | ------ |\r\n| saliency enabled | Unsupported | Slow |\r\n| saliency disabled | Very slow | Fast |\r\n\r\n### Performance, multiprocessing and GPU acceleration\r\n\r\n#### Time and space complexity\r\n\r\nDifferent photomosaic making options have different computational complexity. The following table shows the time and space complexity of different cases. Here, `n` is the number of tiles (after duplication in fair mode), `m` is the number of pixels in the destination image, and `k` is the number of tiles used in the unfair mode (this is equal to your specified `max_width` multiplied by the aspect ratio of your destination image). \r\n\r\n| Type of photomosaic | Time complexity | Space complexity | GPU acceleration |\r\n| -------------------- | ------------------ | ---------------- | ---------------------------------------------- |\r\n| Fair | `O(nm + n^3)` | `O(nm)` | partial (~10x speed up for the `nm` part only) |\r\n| Unfair, freq_mul > 0 | `O(nm + nk log n)` | `O(m + n)` | full (~5-10x speed up) |\r\n| Unfair, freq_mul = 0 | `O(nm + nk)` | `O(m + n)` | full (~5-10x speed up) |\r\n\r\nTakeaway 1:\r\n\r\nThe high (cubic) computational complexity of the fair mode means that the computation time grows much faster with respect to the number of tiles. It typically takes 30 seconds for 5000 tiles and 5 minutes for 10000 tiles. For large tile count, unless you need strict fair tile usage, you should go for the unfair mode and set freq_mul appropriately. \r\n\r\nTakeaway 2:\r\n\r\nNotice the role of `m` in the complexity. If you have a high-definition destination image (e.g. 8000x6000) and notice the computation time is long, you can first downsample it so the number of pixels (`m`) will be lower. Do note that over downsampling will reduce the quality of the photomosaic. \r\n\r\n#### Multiprocessing\r\n\r\nThe `--num_process` option specifies the number of processes (cpu cores) to use. This defaults to half of you available logical CPUs. However, this only applies to the reading tiles phase and photomosaic video processing. For photomosaic video, if you have a small number of tiles or a large number of available CPU cores, using multiprocessing may be faster than enabling GPU acceleration. \r\n\r\n#### GPU acceleration\r\n\r\nFor command line, GPU acceleration can be enabled with the `--gpu` flag. For GUI (`gui.py`), it will be automatically enabled if you have `cupy` installed. However, note it can only provide the listed speedup if `nm` **is large**, typically **>= 10^10**. Another way to judge whether GPU acceleration could be useful is observe the `Distance matrix size` message from the log. Typically, `Distance matrix size` **>= 100MB** work great on GPU. \r\n\r\n### All command line options\r\n\r\n```python make_img.py -h``` will give you all the available command line options.\r\n\r\n```\r\n$ python make_img.py --help\r\nusage: make_img.py [-h] [--path PATH] [--recursive] [--num_process NUM_PROCESS] [--out OUT] [--size SIZE [SIZE ...]]\r\n [--quiet] [--auto_rotate {-1,0,1}] [--resize_opt {center,stretch,fit}] [--gpu]\r\n [--mem_limit MEM_LIMIT] [--tile_info_out TILE_INFO_OUT] [--ratio RATIO RATIO]\r\n [--sort {none,bgr_sum,av_hue,av_sat,av_lum,rand}] [--rev_row] [--rev_sort] [--dest_img DEST_IMG] \r\n [--colorspace {hsv,hsl,bgr,lab,luv}] [--metric {euclidean,cityblock,chebyshev,cosine}]\r\n [--transparent] [--unfair] [--max_width MAX_WIDTH] [--freq_mul FREQ_MUL] [--dither]\r\n [--deterministic] [--dup DUP] [--salient] [--lower_thresh LOWER_THRESH]\r\n [--blending {alpha,brightness}] [--blending_level BLENDING_LEVEL] [--video]\r\n [--skip_frame SKIP_FRAME] [--exp]\r\n\r\noptional arguments:\r\n -h, --help show this help message and exit\r\n --path PATH Path to the tiles (default: None)\r\n --recursive Whether to read the sub-folders for the specified path (default: False)\r\n --num_process NUM_PROCESS\r\n Number of processes to use for parallelizable operations (default: 8)\r\n --out OUT The filename of the output collage/photomosaic (default: result.png)\r\n --size SIZE [SIZE ...]\r\n Width and height of each tile in pixels in the resulting collage/photomosaic. If two numbers \r\n are specified, they are treated as width and height. If one number is specified, the number \r\n is treated as the width and the height is inferred from the aspect ratios of the images \r\n provided. (default: (50,))\r\n --quiet Do not print progress message to console (default: False)\r\n --auto_rotate {-1,0,1}\r\n Options to auto rotate tiles to best match the specified tile size. 0: do not auto rotate. 1: \r\n attempt to rotate counterclockwise by 90 degrees. -1: attempt to rotate clockwise by 90 \r\n degrees (default: 0)\r\n --resize_opt {center,stretch,fit}\r\n How to resize each tile so they have the desired aspect ratio and size, which can be\r\n specified fully or partially by --size. Center: crop the largest rectangle from the center. \r\n Stretch: stretch the tile. Fit: pad the tiles with white background (default: center)\r\n --gpu Use GPU acceleration. Requires cupy to be installed and a capable GPU. Note that USUALLY this \r\n is useful when you: 1. have a lot of tiles (typically > 10000), and2. are using the unfair \r\n mode, and3. (for photomosaic videos only) only have few cpu coresAlso note: enabling GPU \r\n acceleration will disable multiprocessing on CPU for videos (default: False)\r\n --mem_limit MEM_LIMIT\r\n The APPROXIMATE memory limit in MB when computing a photomosaic in unfair mode. Applicable \r\n both CPU and GPU computing. If you run into memory issues when using GPU, try reduce this \r\n memory limit (default: 4096)\r\n --tile_info_out TILE_INFO_OUT\r\n Path to save the list of tile filenames for the collage/photomosaic. If empty, it will not be \r\n saved. (default: )\r\n --ratio RATIO RATIO Aspect ratio of the output image (default: (16, 9))\r\n --sort {none,bgr_sum,av_hue,av_sat,av_lum,rand}\r\n Sort method to use (default: bgr_sum)\r\n --rev_row Whether to use the S-shaped alignment. (default: False)\r\n --rev_sort Sort in the reverse direction. (default: False)\r\n --dest_img DEST_IMG The path to the destination image that you want to build a photomosaic for (default: )\r\n --colorspace {hsv,hsl,bgr,lab,luv}\r\n The colorspace used to calculate the metric (default: lab)\r\n --metric {euclidean,cityblock,chebyshev,cosine}\r\n Distance metric used when evaluating the distance between two color vectors (default:\r\n euclidean)\r\n --transparent Enable transparency masking. The transparent regions of the destination image will be\r\n maintained in the photomosaicCannot be used together with --salient (default: False)\r\n --unfair Whether to allow each tile to be used different amount of times (unfair tile usage).\r\n (default: False)\r\n --max_width MAX_WIDTH\r\n Maximum width of the collage. This option is only valid if unfair option is enabled (default: \r\n 80)\r\n --freq_mul FREQ_MUL Frequency multiplier to balance tile fairless and mosaic quality. Minimum: 0. More weight \r\n will be put on tile fairness when this number increases. (default: 0.0)\r\n --dither Whether to enabled dithering. You must also specify --deterministic if enabled. (default: \r\n False)\r\n --deterministic Do not randomize the tiles. This option is only valid if unfair option is enabled (default: \r\n False)\r\n --dup DUP If a positive integer: duplicate the set of tiles by how many times. Can be a fraction\r\n (default: 1)\r\n --salient Make photomosaic for salient objects only (default: False)\r\n --lower_thresh LOWER_THRESH\r\n The threshold for saliency detection, between 0.0 (no object area = blank) and 1.0 (maximum \r\n object area = original image) (default: 0.5)\r\n --blending {alpha,brightness}\r\n The types of blending used. alpha: alpha (transparency) blending. Brightness: blending of \r\n brightness (lightness) channel in the HSL colorspace (default: alpha)\r\n --blending_level BLENDING_LEVEL\r\n Level of blending, between 0.0 (no blending) and 1.0 (maximum blending). Default is no\r\n blending (default: 0.0)\r\n --video Make a photomosaic video from dest_img which is assumed to be a video (default: False)\r\n --skip_frame SKIP_FRAME\r\n Make a photomosaic every this number of frames (default: 1)\r\n --exp Do experiments (for testing only) (default: False)\r\n```\r\n\r\n## Utility Script: download profile pictures of your WeChat friends\r\n\r\nIf you have a WeChat account, an utility script `extract_img.py` is provided to download your friends' profile pictures so you can make a photomosaic using them. To use this script, you need to have itchat-uos installed\r\n\r\n```bash\r\npip install itchat-uos\r\n```\r\n\r\nThen, use `--dir` to specify the directory to store the profile pictures of your WeChat friends. \r\n\r\n```bash\r\npython extract_img.py --dir img\r\n```\r\n\r\n### Groupchat Members\r\n\r\nYou can also download the group members' profiles images from a group chat\r\n\r\n```bash\r\npython extract_img.py --dir img --type groupchat --name \"groupchatname\"\r\n```\r\n\r\nYou can download members' profile pictures from all your groupchats if you omit the `--name` argument\r\n\r\n```bash\r\npython extract_img.py --dir img --type groupchat\r\n```\r\n\r\n### All available profile pictures\r\n\r\nYou can download profile pictures from both your friends and members from all your groupchats by specifying `--type all`. \r\n\r\n```bash\r\npython extract_img.py --dir img --type all\r\n```\r\n\r\n### Notes\r\n\r\n1. Due to unknown issues, sometimes some profile pictures are not available, so they will be blank and unusable. The photomosaic maker will automatically ignore them when loading images. \r\n2. When you download a large amount of profile pictures at once, WeChat may block you from downloading more. This will appear as `timeout downloading pics, retrying.... attempt x` in terminal. When this happens, you can terminate the program and run it again a day after. Already downloaded profile pictures will not be downloaded again. \r\n\r\n## Credits (Names in alphabetical order)\r\n\r\nHanzhi Zhou ([hanzhi713](https://github.com/hanzhi713/)): Main algorithm and GUI implementation\r\n\r\nKaiying Shan ([kaiyingshan](https://github.com/kaiyingshan)): Saliency idea and implementation\r\n\r\nXinyue Lin: Idea for the \"Best-fit\"\r\n\r\nYufeng Chi ([T-K](https://github.com/T-K-233/)) : Initial Idea, crawler\r\n" }, { "path": "build_scripts/README.md", "content": "# Build Standalone Executable\r\n\r\nThis folder contains scripts to build standalone executables from the source python files using PyInstaller. [.github/workflows/build_executable.yaml](/.github/workflows/build_executable.yaml) uses these scripts to build standalone executables on GitHub actions.\r\n\r\n## How to use locally\r\n\r\nThese scripts can also run locally with a Bash shell. On Windows, Git Bash can be used. Note that these scripts are meant to be run in the project root. First, you need to setup a Python venv and install all the necessary dependencies. This needs to be done only once. \r\n\r\n```bash\r\n. ./build_scripts/install_dependencies.sh {your-os}\r\n```\r\n\r\nThen, you can run the build script to produce an executable for your platform in the dist folder. Make sure that the Python venv is activated before running this script.\r\n\r\n```bash\r\n. ./build_scripts/build.sh {your-os}\r\n```\r\n\r\n`{your-os}` is one of `macos`, `windows` and `linux`. Due to the way PyInstaller works, you can only build executable for your platform. For example, you cannot build macos and linux executable on windows." }, { "path": "build_scripts/build.sh", "content": "#!/bin/bash\nPLATFORM=$1\nNAME=photomosaic-maker-${VERSION}-${PLATFORM}-x64\n\nrm -rf dist/*\n\nif [[ $PLATFORM == windows* ]]; then\n ./collage/Scripts/activate\n SUFFIX=\".exe\"\nelif [[ $PLATFORM == macos* ]]; then\n source collage/bin/activate\n SUFFIX=\"\"\nelif [[ $PLATFORM == ubuntu* ]]; then\n source collage/bin/activate\n SUFFIX=\"\"\nelse\n echo \"Unsupported platform: \" $PLATFORM\n exit 1\nfi\n\nARGS=\"-y --exclude-module umap --exclude-module matplotlib\"\npyinstaller $ARGS --name \"${NAME}-archive\" gui.py\npyinstaller $ARGS --onefile --name \"$NAME${SUFFIX}\" gui.py\n\npushd dist/$NAME-archive\ntar -czvf ../$NAME.tar.gz .\n# zip -r ../$NAME.zip .\npopd\n\npushd build\ntar -czvf ../$NAME-build.tar.gz .\npopd" }, { "path": "build_scripts/install_dependencies.sh", "content": "#/bin/bash\npython3 -m venv collage\nPLATFORM=$1\nif [[ $PLATFORM == windows* ]]; then\n ./collage/Scripts/activate\nelif [[ $PLATFORM == macos* ]]; then\n source collage/bin/activate\nelif [[ $PLATFORM == ubuntu* ]]; then\n source collage/bin/activate\nelse\n echo \"Unsupported platform: \" $PLATFORM\n exit 1\nfi\npython -m pip install pip==21.2.4\ncat requirements.txt | xargs -n 1 pip install" }, { "path": "examples.sh", "content": "#!/bin/bash\n# This is the script to generate all the examples in the README. \n# If you want to use this script, modify the command and output directory according to your needs.\nCMD=\"python make_img.py --path img/zhou\"\nOUT=examples\n\n$CMD --sort none --size 50 --out $OUT/unsorted.png\n$CMD --sort bgr_sum --size 50 --out $OUT/sort-bgr.png\n$CMD --dest_img examples/dest.jpg --size 25 --dup 8 --out $OUT/fair-dup-10.png\n$CMD --dest_img examples/dest.jpg --size 25 --unfair --max_width 56 --out $OUT/best-fit.png\n\n$CMD --dest_img examples/messi.jpg --size 25 --salient --lower_thresh 0.15 --dup 5 --out $OUT/messi-fair.png\n$CMD --dest_img examples/messi.jpg --size 25 --salient --lower_thresh 0.15 --unfair --max_width 115 --out $OUT/messi-unfair.png\n\n$CMD --dest_img examples/dest-transp.png --size 25 --transparent --dup 5 --out $OUT/transp-fair.png\n$CMD --dest_img examples/dest-transp.png --size 25 --transparent --unfair --max_width 56 --out $OUT/transp-unfair.png\n$CMD --dest_img examples/dest-transp.png --size 25 --transparent --unfair --freq_mul 0.5 --max_width 56 --out $OUT/transp-unfair-freq.png\n\n$CMD --dest_img examples/dest.jpg --size 25 --dup 8 --blending alpha --blending_level 0.25 --out $OUT/blend-alpha-0.25.png\n$CMD --dest_img examples/dest.jpg --size 25 --dup 8 --blending brightness --blending_level 0.25 --out $OUT/blend-brightness-0.25.png\n\n$CMD --dest_img examples/dest2.jpg --size 10 --unfair --max_width 200 --freq_mul 0.0 --out $OUT/dither-no.png\n$CMD --dest_img examples/dest2.jpg --size 10 --unfair --max_width 200 --freq_mul 0.0 --dither --deterministic --out $OUT/dither.png\n$CMD --dest_img examples/dest2.jpg --size 10 --unfair --max_width 200 --freq_mul 0.1 --out $OUT/f-dither-no.png\n$CMD --dest_img examples/dest2.jpg --size 10 --unfair --max_width 200 --freq_mul 0.1 --dither --deterministic --out $OUT/f-dither.png\n\n# $CMD --sort none --exp --size 50 --out $OUT/sort-exp.png\n# $CMD --dest_img examples/dest.jpg --size 25 --exp --unfair --max_width 56" }, { "path": "extract_img.py", "content": "\"\"\"\r\nextract_img.py\r\nget profile pictures from your WeChat friends or group chat members\r\n\"\"\"\r\nimport itchat\r\nimport os\r\nfrom concurrent.futures import ThreadPoolExecutor, TimeoutError\r\nfrom tqdm import tqdm\r\nimport argparse\r\nimport multiprocessing as mp\r\nimport unicodedata\r\nimport re\r\nimport traceback\r\n\r\n\r\ndef slugify(value, allow_unicode=True):\r\n \"\"\"\r\n Taken from https://github.com/django/django/blob/master/django/utils/text.py\r\n Convert to ASCII if 'allow_unicode' is False. Convert spaces or repeated\r\n dashes to single dashes. Remove characters that aren't alphanumerics,\r\n underscores, or hyphens. Convert to lowercase. Also strip leading and\r\n trailing whitespace, dashes, and underscores.\r\n \"\"\"\r\n value = str(value)\r\n if allow_unicode:\r\n value = unicodedata.normalize('NFKC', value)\r\n else:\r\n value = unicodedata.normalize('NFKD', value).encode('ascii', 'ignore').decode('ascii')\r\n value = re.sub(r'[^\\w\\s-]', '', value.lower())\r\n return re.sub(r'[-\\s]+', '-', value).strip('-_')\r\n\r\n\r\ndef download_pic(args):\r\n try:\r\n itchat.get_head_img(**args)\r\n return os.path.getsize(args['picDir'])\r\n except:\r\n traceback.print_exc()\r\n return 0\r\n\r\n\r\ndef get_chatroom_by_name(name, chatrooms):\r\n for chatroom in chatrooms:\r\n if chatroom['NickName'] == name:\r\n return chatroom\r\n\r\n\r\nif __name__ == \"__main__\":\r\n parser = argparse.ArgumentParser()\r\n parser.add_argument(\"--dir\", default=\"img\", type=str,\r\n help=\"Folder to store the downloaded images\")\r\n parser.add_argument(\"--type\", type=str,\r\n choices=[\"self\", \"groupchat\", \"all\"], default=\"self\")\r\n parser.add_argument(\"--name\", type=str, nargs=\"+\",\r\n help=\"Specify the chatroom name if type=chatroom\")\r\n parser.add_argument(\"--list_chatroom\", action=\"store_true\")\r\n\r\n args = parser.parse_args()\r\n download_dir = args.dir\r\n\r\n print(\"Logging in...\")\r\n itchat.auto_login(hotReload=True)\r\n\r\n if args.list_chatroom:\r\n print(\"Getting chatrooms...\")\r\n chatrooms = itchat.get_chatrooms(update=True)\r\n for chatroom in tqdm(chatrooms, desc=\"[Updating Groupchats]\"):\r\n tqdm.write(chatroom['NickName'])\r\n exit()\r\n\r\n download_args = dict()\r\n if args.type == \"self\" or args.type == \"all\":\r\n print(\"Loading contact...\")\r\n for mem in itchat.get_friends(update=True):\r\n download_args[mem['UserName']] = {\r\n \"userName\": mem['UserName'], \r\n \"picDir\": os.path.join(download_dir, slugify(f\"{mem['NickName']}_{mem['RemarkName']}\") + \".jpg\")\r\n }\r\n\r\n if args.type == \"groupchat\" or args.type == \"all\":\r\n print(\"Getting groupchats...\")\r\n chatrooms = itchat.get_chatrooms(update=True)\r\n # Nickname PYQuanPin RemarkPYQuanPin\r\n if not args.name:\r\n print(\"Updating groupchats... this might take a while (several minutes if you have tens of large group chats)\")\r\n itchat.update_chatroom([chatroom['UserName'] for chatroom in chatrooms], True)\r\n chatrooms = itchat.get_chatrooms()\r\n else:\r\n filtered_chatrooms = []\r\n for name in args.name:\r\n chatroom = get_chatroom_by_name(args.name, chatrooms)\r\n assert chatroom is not None, f\"Chatroom \\\"{args.name}\\\" not found\"\r\n filtered_chatrooms.append(chatroom)\r\n \r\n itchat.update_chatroom([chatroom['UserName'] for chatroom in filtered_chatrooms], True)\r\n chatrooms = itchat.get_chatrooms()\r\n chatrooms = [get_chatroom_by_name(name, chatrooms) for name in args.name]\r\n \r\n for chatroom in chatrooms:\r\n cr_name = chatroom['NickName']\r\n for mem in chatroom['MemberList']:\r\n if mem['UserName'] not in download_args:\r\n download_args[mem['UserName']] = {\r\n \"userName\": mem['UserName'],\r\n 'chatroomUserName': chatroom['UserName'],\r\n \"picDir\": os.path.join(download_dir, slugify(f\"{cr_name}_{mem['NickName']}\") + \".jpg\")\r\n }\r\n\r\n if not os.path.isdir(download_dir):\r\n os.mkdir(download_dir)\r\n\r\n download_args = download_args.values()\r\n download_args = [arg for arg in download_args if not os.path.exists(arg['picDir']) or os.path.getsize(arg['picDir']) == 0]\r\n \r\n pool = ThreadPoolExecutor(min(mp.cpu_count(), len(download_args)))\r\n pbar = tqdm(desc=\"[Downloading]\", total=len(download_args))\r\n count = 1\r\n while len(download_args) > 0:\r\n download_args = [arg for arg in download_args if not os.path.exists(arg['picDir']) or os.path.getsize(arg['picDir']) == 0]\r\n try:\r\n for sz in pool.map(download_pic, download_args, timeout=20):\r\n if sz > 0:\r\n pbar.update()\r\n except TimeoutError:\r\n pool.shutdown(False)\r\n pool = ThreadPoolExecutor(min(mp.cpu_count(), len(download_args)))\r\n tqdm.write(f\"timeout downloading pics, retrying.... attempt {count}\")\r\n count += 1\r\n pool.shutdown()" }, { "path": "gui.py", "content": "import tkinter as tk\r\nfrom tkinter import *\r\nfrom tkinter import filedialog, messagebox, colorchooser\r\nfrom tkinter.ttk import *\r\nfrom concurrent.futures import Future, ThreadPoolExecutor\r\nimport multiprocessing as mp\r\nfrom multiprocessing import freeze_support, cpu_count\r\nimport argparse\r\nimport traceback\r\nimport math\r\nimport sys\r\nimport os\r\nimport time\r\nfrom typing import Tuple, Union\r\n\r\nimport cv2\r\nimport numpy as np\r\nimport make_img as mkg\r\nfrom io_utils import SafeText\r\n\r\n\r\ndef limit_wh(w: int, h: int, max_width: int, max_height: int):\r\n if h > max_height:\r\n ratio = max_height / h\r\n h = max_height\r\n w = math.floor(w * ratio)\r\n if w > max_width:\r\n ratio = max_width / w\r\n w = max_width\r\n h = math.floor(h * ratio)\r\n return w, h\r\n\r\n\r\n\"\"\" tk_ToolTip_class101.py\r\ngives a Tkinter widget a tooltip as the mouse is above the widget\r\ntested with Python27 and Python34 by vegaseat 09sep2014\r\nwww.daniweb.com/programming/software-development/code/484591/a-tooltip-class-for-tkinter\r\n\r\nModified to include a delay time by Victor Zaccardo, 25mar16\r\nSource: https://stackoverflow.com/questions/3221956/how-do-i-display-tooltips-in-tkinter\r\n\"\"\"\r\nclass CreateToolTip(object):\r\n \"\"\"\r\n create a tooltip for a given widget\r\n \"\"\"\r\n def __init__(self, widget, text='widget info'):\r\n self.waittime = 500 #miliseconds\r\n self.wraplength = 180 #pixels\r\n self.widget = widget\r\n self.text = text\r\n self.widget.bind(\"\", self.enter)\r\n self.widget.bind(\"\", self.leave)\r\n self.widget.bind(\"\", self.leave)\r\n self.id = None\r\n self.tw = None\r\n\r\n def enter(self, event=None):\r\n self.schedule()\r\n\r\n def leave(self, event=None):\r\n self.unschedule()\r\n self.hidetip()\r\n\r\n def schedule(self):\r\n self.unschedule()\r\n self.id = self.widget.after(self.waittime, self.showtip)\r\n\r\n def unschedule(self):\r\n id = self.id\r\n self.id = None\r\n if id:\r\n self.widget.after_cancel(id)\r\n\r\n def showtip(self, event=None):\r\n x = y = 0\r\n x, y, cx, cy = self.widget.bbox(\"insert\")\r\n x += self.widget.winfo_rootx() + 25\r\n y += self.widget.winfo_rooty() + 20\r\n # creates a toplevel window\r\n self.tw = tk.Toplevel(self.widget)\r\n # Leaves only the label and removes the app window\r\n self.tw.wm_overrideredirect(True)\r\n self.tw.wm_geometry(\"+%d+%d\" % (x, y))\r\n label = tk.Label(self.tw, text=self.text, justify='left',\r\n background=\"#ffffff\", relief='solid', borderwidth=1,\r\n wraplength = self.wraplength)\r\n label.pack(ipadx=1)\r\n\r\n def hidetip(self):\r\n tw = self.tw\r\n self.tw= None\r\n if tw:\r\n tw.destroy()\r\n\r\n\r\nclass LabelWithTooltip(Label):\r\n def __init__(self, *args, **kwargs) -> None:\r\n _tp = kwargs[\"tooltip\"]\r\n del kwargs[\"tooltip\"]\r\n super().__init__(*args, **kwargs)\r\n self._tp = CreateToolTip(self, _tp)\r\n\r\n\r\nclass CheckbuttonWithTooltip(Checkbutton):\r\n def __init__(self, *args, **kwargs) -> None:\r\n _tp = kwargs[\"tooltip\"]\r\n del kwargs[\"tooltip\"]\r\n super().__init__(*args, **kwargs)\r\n self._tp = CreateToolTip(self, _tp)\r\n\r\n\r\nclass RadiobuttonWithTooltip(Radiobutton):\r\n def __init__(self, *args, **kwargs) -> None:\r\n _tp = kwargs[\"tooltip\"]\r\n del kwargs[\"tooltip\"]\r\n super().__init__(*args, **kwargs)\r\n self._tp = CreateToolTip(self, _tp)\r\n\r\n\r\nclass Debounce:\r\n def __init__(self, action) -> None:\r\n self.queue = []\r\n self.action = action\r\n self.pool = ThreadPoolExecutor(1)\r\n \r\n def __call__(self, *args, **kwds):\r\n for f in self.queue:\r\n f.cancel()\r\n self.queue.clear()\r\n self.queue.append(self.pool.submit(self.action))\r\n\r\n\r\nif __name__ == \"__main__\":\r\n freeze_support()\r\n pool = ThreadPoolExecutor(1)\r\n root = Tk()\r\n root.title(\"Photomosaic maker\")\r\n\r\n parser = argparse.ArgumentParser()\r\n parser.add_argument(\"-D\", action=\"store_true\")\r\n parser.add_argument(\"--src\", \"-s\", type=str,\r\n default=os.path.join(os.path.dirname(__file__), \"img\"))\r\n parser.add_argument(\"--collage\", \"-c\", type=str,\r\n default=os.path.join(os.path.dirname(__file__), \"examples\", \"dest.png\"))\r\n cmd_args = parser.parse_args()\r\n init_dir = os.path.expanduser(\"~\")\r\n\r\n # ---------------- initialization -----------------\r\n left_panel = PanedWindow(root)\r\n left_panel.grid(row=0, column=0, sticky=\"NSEW\")\r\n # left_panel.rowconfigure(0, weight=1)\r\n # left_panel.columnconfigure(0, weight=1)\r\n Separator(root, orient=\"vertical\").grid(\r\n row=0, column=1, sticky=\"NSEW\", padx=(5, 5))\r\n right_panel = PanedWindow(root)\r\n right_panel.grid(row=0, column=2, sticky=\"N\")\r\n # ---------------- end initialization -----------------\r\n\r\n # ---------------- left panel's children ----------------\r\n # left panel ROW 0\r\n canvas = Canvas(left_panel, width=800, height=600)\r\n canvas.grid(row=0, column=0)\r\n\r\n # left panel ROW 1\r\n Separator(left_panel, orient=\"horizontal\").grid(row=1, columnspan=2, sticky=\"NSEW\", pady=(5, 5))\r\n\r\n # left panel ROW 2\r\n log_panel = PanedWindow(left_panel)\r\n log_panel.grid(row=2, column=0, columnspan=2, sticky=\"WE\")\r\n log_panel.grid_columnconfigure(0, weight=1)\r\n log_entry = SafeText(log_panel, height=6, bd=0)\r\n mkg.pbar_ncols = log_entry.width\r\n # log_entry.configure(font=(\"\", 10, \"\"))\r\n log_entry.grid(row=1, column=0, sticky=\"NSEW\")\r\n scroll = Scrollbar(log_panel, orient=\"vertical\", command=log_entry.yview)\r\n log_entry.config(yscrollcommand=scroll.set)\r\n scroll.grid(row=1, column=1, sticky=\"NSEW\")\r\n\r\n # used store a reference to the displayed image to we can save it\r\n result_img = None\r\n # ------------------ end left panel ------------------------\r\n\r\n def show_img(img: Union[np.ndarray, Future, Tuple[np.ndarray, str], None], printDone: bool = True) -> None:\r\n \"\"\"\r\n display an image in the canvas and set the global variable result_img to img\r\n \"\"\"\r\n global result_img, result_tile_info\r\n if img is None:\r\n return\r\n if type(img) == Future:\r\n img = img.result()\r\n if type(img) == tuple:\r\n img, result_tile_info = img\r\n result_img = img\r\n width, height = canvas.winfo_width(), canvas.winfo_height()\r\n h, w, _ = img.shape\r\n img = mkg.strip_alpha(img)\r\n if h > height or w > width:\r\n print(\"Resizing image to display it on GUI...\")\r\n w, h = limit_wh(w, h, width, height)\r\n img = cv2.resize(img, (w, h))\r\n _, data = cv2.imencode(\".ppm\", img)\r\n # prevent the image from being garbage-collected\r\n root.preview = PhotoImage(data=data.tobytes())\r\n canvas.delete(\"all\")\r\n canvas.create_image((width - w) // 2, (height - h) // 2, image=root.preview, anchor=NW)\r\n save_button.config(state='enabled')\r\n tile_save_button.config(state='disabled' if result_tile_info is None else 'enabled')\r\n if printDone:\r\n print(\"Done\")\r\n\r\n # --------------------- right panel's children ------------------------\r\n # right panel ROW 0\r\n file_path = StringVar()\r\n file_path.set(\"N/A\")\r\n Label(right_panel, text=\"Path to tiles:\").grid(row=0, columnspan=2, sticky=\"W\", pady=(8, 2))\r\n\r\n # right panel ROW 1\r\n Label(right_panel, textvariable=file_path, wraplength=150).grid(row=1, columnspan=2, sticky=\"W\")\r\n\r\n # right panel ROW 2\r\n opt = StringVar()\r\n opt.set(\"sort\")\r\n\r\n tile_size_panel = PanedWindow(right_panel)\r\n tile_size_panel.grid(row=2, column=0, columnspan=2, sticky=\"W\")\r\n \r\n tile_width = IntVar()\r\n tile_width.set(50)\r\n tile_height = IntVar()\r\n tile_height.set(0)\r\n LabelWithTooltip(tile_size_panel, text=\"Tile size: \", tooltip=mkg.PARAMS.size.help).grid(row=0, column=0, sticky=\"W\")\r\n Entry(tile_size_panel, width=5, textvariable=tile_width).grid(row=0, column=1, sticky=\"W\")\r\n Label(tile_size_panel, text=\"x\", wraplength=150).grid(row=0, column=2, sticky=\"W\")\r\n tile_height_entry = Entry(tile_size_panel, width=5, textvariable=tile_height)\r\n tile_height_entry.grid(row=0, column=3, sticky=\"W\")\r\n\r\n infer_height = BooleanVar()\r\n infer_height.set(True)\r\n def action_infer_height():\r\n if infer_height.get():\r\n tile_height_entry.config(state='disabled')\r\n else:\r\n tile_height_entry.config(state='enabled')\r\n if tile_height.get() == 0:\r\n tile_height.set(tile_width.get())\r\n action_infer_height()\r\n CheckbuttonWithTooltip(tile_size_panel, text=\"Infer height\", variable=infer_height, command=action_infer_height, \r\n tooltip=\"Infer height from width and the aspect ratios of the images provided\").grid(row=1, columnspan=4, sticky=\"W\")\r\n\r\n auto_rotate = IntVar()\r\n auto_rotate.set(0)\r\n LabelWithTooltip(tile_size_panel, text=\"Auto rotate: \", tooltip=mkg.PARAMS.auto_rotate.help).grid(\r\n row=2, column=0, columnspan=2, sticky=\"W\")\r\n OptionMenu(tile_size_panel, auto_rotate, \"\", *mkg.PARAMS.auto_rotate.choices).grid(\r\n row=2, column=2, columnspan=2, sticky=\"W\")\r\n\r\n # right panel ROW 3\r\n resize_opt = StringVar()\r\n resize_opt.set(\"center\")\r\n LabelWithTooltip(right_panel, text=\"Resize option: \", tooltip=mkg.PARAMS.resize_opt.help).grid(\r\n row=3, column=0, sticky=\"W\")\r\n OptionMenu(right_panel, resize_opt, \"\", *mkg.PARAMS.resize_opt.choices).grid(\r\n row=3, column=1, sticky=\"W\")\r\n\r\n # right panel ROW 4\r\n recursive = BooleanVar()\r\n recursive.set(True)\r\n Checkbutton(right_panel, text=\"Read sub-folders\",\r\n variable=recursive).grid(row=4, columnspan=2, sticky=\"W\")\r\n\r\n imgs = None\r\n\r\n def load_img_action():\r\n global imgs\r\n fp = file_path.get()\r\n try:\r\n sizes = [tile_width.get()]\r\n if not infer_height.get():\r\n sizes.append(tile_height.get())\r\n imgs = mkg.read_images(fp, sizes, recursive.get(), mp.Pool(cpu_count() // 2), resize_opt.get(), auto_rotate.get())\r\n shape = imgs[0].shape\r\n if infer_height.get():\r\n tile_height.set(shape[0])\r\n grid = mkg.calc_grid_size(16, 10, len(imgs), shape)\r\n return mkg.make_collage(grid, imgs.copy(), False)\r\n except AssertionError as e:\r\n messagebox.showerror(\"Error\", str(e))\r\n except:\r\n messagebox.showerror(\"Error\", traceback.format_exc())\r\n\r\n \r\n def reload_images():\r\n pool.submit(load_img_action).add_done_callback(show_img)\r\n\r\n\r\n def load_images():\r\n w = tile_width.get()\r\n h = tile_height.get()\r\n if w < 2:\r\n return messagebox.showerror(\"Illegal Argument\", \"Tile width must be greater than 2\")\r\n if not infer_height.get() and h < 2:\r\n return messagebox.showerror(\"Illegal Argument\", \"Tile height must be greater than 2\")\r\n\r\n fp = filedialog.askdirectory(initialdir=file_path.get() if os.path.isdir(file_path.get()) else init_dir, \r\n title=\"Select folder of tiles\")\r\n if len(fp) <= 0 or not os.path.isdir(fp):\r\n return\r\n\r\n def callback(f):\r\n reload_img_button.config(state='enabled')\r\n sort_button.config(state='enabled')\r\n collage_button.config(state='enabled')\r\n show_img(f)\r\n\r\n print(\"Loading tiles from\", fp)\r\n file_path.set(fp)\r\n pool.submit(load_img_action).add_done_callback(callback)\r\n\r\n\r\n # right panel ROW 5\r\n Button(right_panel, text=\"Load tiles\", command=load_images).grid(row=5, column=0, pady=2)\r\n reload_img_button = Button(right_panel, text=\"Reload\", command=reload_images)\r\n reload_img_button.config(state='disabled')\r\n reload_img_button.grid(row=5, column=1, pady=2, padx=(0, 4))\r\n\r\n\r\n def attach_sort():\r\n right_col_opt_panel.grid_remove()\r\n right_sort_opt_panel.grid(row=8, columnspan=2, sticky=\"W\")\r\n\r\n def attach_collage():\r\n right_sort_opt_panel.grid_remove()\r\n right_col_opt_panel.grid(row=8, columnspan=2, sticky=\"W\")\r\n\r\n # right panel ROW 6\r\n Radiobutton(right_panel, text=\"Sort\", value=\"sort\", variable=opt,\r\n state=ACTIVE, command=attach_sort).grid(row=6, column=1, sticky=\"W\")\r\n Radiobutton(right_panel, text=\"Photomosaic\", value=\"collage\", variable=opt,\r\n command=attach_collage).grid(row=6, column=0, sticky=\"W\")\r\n\r\n # right panel ROW 7\r\n Separator(right_panel, orient=\"horizontal\").grid(\r\n row=7, columnspan=2, pady=(6, 3), sticky=\"we\")\r\n\r\n # right panel ROW 8: Dynamically attached\r\n # right sort option panel !!OR!! right collage option panel\r\n right_sort_opt_panel = PanedWindow(right_panel)\r\n right_sort_opt_panel.grid(row=8, column=0, columnspan=2, sticky=\"W\")\r\n\r\n # ------------------------- right sort option panel --------------------------\r\n # right sort option panel ROW 0:\r\n sort_method = StringVar()\r\n sort_method.set(\"bgr_sum\")\r\n LabelWithTooltip(right_sort_opt_panel, text=\"Sort methods:\", tooltip=mkg.PARAMS.sort.help).grid(\r\n row=0, column=0, sticky=\"W\")\r\n OptionMenu(right_sort_opt_panel, sort_method, \"\", *mkg.PARAMS.sort.choices).grid(row=0, column=1)\r\n\r\n # right sort option panel ROW 1:\r\n LabelWithTooltip(right_sort_opt_panel, text=\"Aspect ratio:\", tooltip=mkg.PARAMS.ratio.help).grid(\r\n row=1, column=0, sticky=\"W\")\r\n aspect_ratio_panel = PanedWindow(right_sort_opt_panel)\r\n aspect_ratio_panel.grid(row=1, column=1)\r\n rw = IntVar()\r\n rw.set(16)\r\n rh = IntVar()\r\n rh.set(10)\r\n Entry(aspect_ratio_panel, width=3, textvariable=rw).grid(row=0, column=0)\r\n Label(aspect_ratio_panel, text=\":\").grid(row=0, column=1)\r\n Entry(aspect_ratio_panel, width=3, textvariable=rh).grid(row=0, column=2)\r\n\r\n # right sort option panel ROW 2:\r\n rev_row = BooleanVar()\r\n rev_row.set(False)\r\n rev_sort = BooleanVar()\r\n rev_sort.set(False)\r\n Checkbutton(right_sort_opt_panel, variable=rev_row,\r\n text=\"Reverse consecutive row\").grid(row=2, columnspan=2, sticky=\"W\")\r\n # right sort option panel ROW 3:\r\n Checkbutton(right_sort_opt_panel, variable=rev_sort,\r\n text=\"Reverse sort order\").grid(row=3, columnspan=2, sticky=\"W\")\r\n\r\n def generate_sorted_image():\r\n if imgs is None:\r\n return messagebox.showerror(\"Empty set\", \"Please first load tiles\")\r\n\r\n try:\r\n w, h = rw.get(), rh.get()\r\n assert w > 0, \"Width must be greater than 0\"\r\n assert h > 0, \"Height must be greater than 0\"\r\n except AssertionError as e:\r\n return messagebox.showerror(\"Illegal Argument\", str(e))\r\n\r\n def action():\r\n try:\r\n grid, sorted_imgs = mkg.sort_collage(imgs, (w, h), sort_method.get(), rev_sort.get())\r\n return mkg.make_collage(grid, sorted_imgs, rev_row.get())\r\n except:\r\n messagebox.showerror(\"Error\", traceback.format_exc())\r\n\r\n pool.submit(action).add_done_callback(show_img)\r\n\r\n # right sort option panel ROW 4:\r\n sort_button = Button(right_sort_opt_panel, text=\"Generate sorted image\", command=generate_sorted_image)\r\n sort_button.config(state='disabled')\r\n sort_button.grid(row=4, columnspan=2, pady=5)\r\n # ------------------------ end right sort option panel -----------------------------\r\n\r\n # ------------------------ right collage option panel ------------------------------\r\n # right collage option panel ROW 0:\r\n right_col_opt_panel = PanedWindow(right_panel)\r\n dest_img_path = StringVar()\r\n dest_img_path.set(\"N/A\")\r\n dest_img = None\r\n Label(right_col_opt_panel, text=\"Path to the target image: \").grid(\r\n row=0, columnspan=2, sticky=\"W\", pady=2)\r\n\r\n # right collage option panel ROW 1:\r\n Label(right_col_opt_panel, textvariable=dest_img_path,\r\n wraplength=150).grid(row=1, columnspan=2, sticky=\"W\")\r\n\r\n\r\n def load_dest_img():\r\n global dest_img, result_tile_info\r\n if imgs is None:\r\n return messagebox.showerror(\"Empty set\", \"Please first load tiles\")\r\n\r\n fp = filedialog.askopenfilename(\r\n initialdir=os.path.dirname(dest_img_path.get()) if os.path.isdir(os.path.dirname(dest_img_path.get())) else init_dir, \r\n title=\"Select destination image\",\r\n filetypes=((\"images\", \"*.jpg\"), (\"images\", \"*.png\"), (\"images\", \"*.gif\"), (\"all files\", \"*.*\")))\r\n if fp is not None and len(fp) > 0 and os.path.isfile(fp):\r\n try:\r\n print(\"Destination image loaded from\", fp)\r\n dest_img = mkg.imread(fp, cv2.IMREAD_UNCHANGED)\r\n dest_img_path.set(fp)\r\n result_tile_info = None\r\n show_img(dest_img, False)\r\n transparent.set(dest_img.shape[2] == 4)\r\n is_salient.set(not transparent.get())\r\n except:\r\n messagebox.showerror(\"Error reading file\", traceback.format_exc())\r\n\r\n\r\n # right collage option panel ROW 2:\r\n Button(right_col_opt_panel, text=\"Load destination image\",\r\n command=load_dest_img).grid(row=2, columnspan=2, pady=(3, 2))\r\n\r\n result_collage = None\r\n result_tile_info = None\r\n def change_alpha(_=None, show=True):\r\n if result_collage is not None and dest_img is not None:\r\n if colorization_opt.get() == \"brightness\":\r\n img = mkg.brightness_blend(result_collage, dest_img, 1 - alpha_scale.get() / 100)\r\n else:\r\n img = mkg.alpha_blend(result_collage, dest_img, 1 - alpha_scale.get() / 100)\r\n if show:\r\n show_img(img, False)\r\n return img\r\n \r\n # right collage option panel ROW 3:\r\n LabelWithTooltip(right_col_opt_panel, text=\"Color Blend:\", tooltip=mkg.PARAMS.blending.help).grid(\r\n row=3, column=0, sticky=\"W\", padx=(0, 5))\r\n\r\n change_alpha_debounced = Debounce(change_alpha)\r\n # right collage option panel ROW 4:\r\n colorization_opt = StringVar()\r\n colorization_opt.set(\"brightness\")\r\n RadiobuttonWithTooltip(right_col_opt_panel, text=\"Brightness\", variable=colorization_opt, value=\"brightness\", state=ACTIVE, command=change_alpha_debounced, \r\n tooltip=mkg.PARAMS.blending.help).grid(row=4, column=0, sticky=\"W\")\r\n RadiobuttonWithTooltip(right_col_opt_panel, text=\"Alpha\", variable=colorization_opt, value=\"alpha\", command=change_alpha_debounced,\r\n tooltip=mkg.PARAMS.blending.help).grid(row=4, column=1, sticky=\"W\")\r\n\r\n # right collage option panel ROW 5:\r\n alpha_scale = Scale(right_col_opt_panel, from_=0.0, to=100.0, orient=HORIZONTAL, length=150, command=change_alpha_debounced)\r\n alpha_scale.set(0)\r\n alpha_scale.grid(row=5, columnspan=2, sticky=\"W\")\r\n\r\n # right collage option panel ROW 6:\r\n colorspace = StringVar()\r\n colorspace.set(\"lab\")\r\n LabelWithTooltip(right_col_opt_panel, text=\"Colorspace:\", tooltip=mkg.PARAMS.colorspace.help).grid(row=6, column=0, sticky=\"W\")\r\n OptionMenu(right_col_opt_panel, colorspace, \"\", *mkg.PARAMS.colorspace.choices).grid(row=6, column=1, sticky=\"W\")\r\n\r\n # right collage option panel ROW 7:\r\n dist_metric = StringVar()\r\n dist_metric.set(\"euclidean\")\r\n LabelWithTooltip(right_col_opt_panel, text=\"Metric:\", tooltip=mkg.PARAMS.metric.help).grid(row=7, column=0, sticky=\"W\")\r\n OptionMenu(right_col_opt_panel, dist_metric, \"\", *mkg.PARAMS.metric.choices).grid(row=7, column=1, sticky=\"W\")\r\n\r\n def attach_even():\r\n collage_uneven_panel.grid_remove()\r\n collage_even_panel.grid(row=11, columnspan=2, sticky=\"W\")\r\n\r\n def attach_uneven():\r\n collage_even_panel.grid_remove()\r\n collage_uneven_panel.grid(row=11, columnspan=2, sticky=\"W\")\r\n\r\n # right collage option panel ROW 8:\r\n LabelWithTooltip(right_col_opt_panel, text=\"Fairness of tiles: \", tooltip=mkg.PARAMS.unfair.help).grid(row=8, columnspan=2, sticky=\"W\")\r\n\r\n # right collage option panel ROW 9:\r\n even = StringVar()\r\n even.set(\"even\")\r\n RadiobuttonWithTooltip(right_col_opt_panel, text=\"Fair\", variable=even, value=\"even\", state=ACTIVE, command=attach_even, \r\n tooltip=\"Require all tiles are used the same amount of times (fair tile usage)\").grid(row=9, column=0, sticky=\"W\")\r\n RadiobuttonWithTooltip(right_col_opt_panel, text=\"Unfair\", variable=even, value=\"uneven\", command=attach_uneven,\r\n tooltip=mkg.PARAMS.unfair.help).grid(row=9, column=1, sticky=\"W\")\r\n\r\n # right collage option panel ROW 10:\r\n Separator(right_col_opt_panel, orient=\"horizontal\").grid(row=10, columnspan=2, sticky=\"we\", pady=(5, 5))\r\n\r\n # right collage option panel ROW 11: Dynamically attached\r\n # could EITHER collage even panel OR collage uneven panel\r\n # ----------------------- start collage even panel ------------------------\r\n collage_even_panel = PanedWindow(right_col_opt_panel)\r\n collage_even_panel.grid(row=11, columnspan=2, sticky=\"W\")\r\n\r\n # collage even panel ROW 1\r\n LabelWithTooltip(collage_even_panel, text=\"Duplicates:\", tooltip=mkg.PARAMS.dup.help).grid(row=1, column=0, sticky=\"W\")\r\n dup = DoubleVar()\r\n dup.set(1.0)\r\n Entry(collage_even_panel, textvariable=dup, width=5).grid(row=1, column=1, sticky=\"W\")\r\n # ----------------------- end collage even panel ------------------------\r\n\r\n # ----------------------- start collage uneven panel --------------------\r\n collage_uneven_panel = PanedWindow(right_col_opt_panel)\r\n\r\n # collage uneven panel ROW 0\r\n LabelWithTooltip(collage_uneven_panel, text=\"Max width:\", tooltip=mkg.PARAMS.max_width.help).grid(row=0, column=0, sticky=\"W\")\r\n max_width = IntVar()\r\n max_width.set(80)\r\n Entry(collage_uneven_panel, textvariable=max_width, width=5).grid(row=0, column=1, sticky=\"W\")\r\n\r\n LabelWithTooltip(collage_uneven_panel, text=\"Freq Mul:\", tooltip=mkg.PARAMS.freq_mul.help).grid(row=2, column=0, sticky=\"W\")\r\n freq_mul = DoubleVar()\r\n freq_mul.set(1)\r\n Entry(collage_uneven_panel, textvariable=freq_mul, width=5).grid(row=2, column=1, sticky=\"W\")\r\n\r\n def dither_cb():\r\n if dither.get():\r\n deterministic.set(True)\r\n deterministic_check.config(state='disabled')\r\n is_salient.set(False)\r\n is_salient_check.config(state='disabled')\r\n transparent.set(False)\r\n transparent_check.config(state='disabled')\r\n else:\r\n deterministic_check.config(state='enabled')\r\n is_salient_check.config(state='enabled')\r\n transparent_check.config(state='enabled')\r\n\r\n dither = BooleanVar()\r\n dither.set(False)\r\n dither_check = CheckbuttonWithTooltip(collage_uneven_panel, text=\"Dithering\", variable=dither, command=dither_cb,\r\n tooltip=mkg.PARAMS.dither.help)\r\n dither_check.grid(row=3, columnspan=2, sticky=\"W\")\r\n\r\n deterministic = BooleanVar()\r\n deterministic.set(False)\r\n deterministic_check = CheckbuttonWithTooltip(collage_uneven_panel, text=\"Deterministic\", \r\n variable=deterministic, tooltip=mkg.PARAMS.deterministic.help)\r\n deterministic_check.grid(row=4, columnspan=2, sticky=\"w\")\r\n # ----------------------- end collage uneven panel ----------------------\r\n\r\n def generate_collage():\r\n if imgs is None:\r\n return messagebox.showerror(\"No tiles\", \"Please first load tiles\")\r\n if dest_img is None:\r\n return messagebox.showerror(\"No destination image\", \"Please first load the image that you're trying to fit\")\r\n \r\n try:\r\n if is_salient.get():\r\n lower_thresh = saliency_thresh_scale.get() / 100\r\n assert 0.0 < lower_thresh < 1.0, \"saliency threshold must be between 0 and 1\"\r\n else:\r\n lower_thresh = None\r\n\r\n if even.get() == \"even\":\r\n _dup = mkg.check_dup_valid(dup.get())\r\n \r\n if is_salient.get() or transparent.get():\r\n def action():\r\n return mkg.MosaicFairSalient(\r\n dest_img, imgs, _dup, colorspace.get(), dist_metric.get(), \r\n lower_thresh, transparent.get(), out_wrapper).process_dest_img(dest_img)\r\n else: \r\n def action():\r\n return mkg.MosaicFair(dest_img.shape, imgs, _dup, \r\n colorspace.get(), dist_metric.get()).process_dest_img(dest_img, out_wrapper)\r\n else:\r\n assert max_width.get() > 0, \"Max width must be a positive number\"\r\n assert freq_mul.get() >= 0, \"Max width must be a nonnegative real number\"\r\n\r\n def action():\r\n return mkg.MosaicUnfair(dest_img.shape, imgs, max_width.get(), colorspace.get(), dist_metric.get(), \r\n lower_thresh, freq_mul.get(), not deterministic.get(), dither.get(), transparent.get()).process_dest_img(dest_img)\r\n\r\n def wrapper():\r\n global result_collage\r\n try:\r\n result_collage, tile_info = action()\r\n return change_alpha(show=False), tile_info\r\n except AssertionError as e:\r\n return messagebox.showerror(\"Error\", e)\r\n except:\r\n messagebox.showerror(\"Error\", traceback.format_exc())\r\n\r\n pool.submit(wrapper).add_done_callback(show_img)\r\n\r\n except AssertionError as e:\r\n return messagebox.showerror(\"Error\", e)\r\n except:\r\n return messagebox.showerror(\"Error\", traceback.format_exc())\r\n\r\n\r\n def attach_salient_opt():\r\n if is_salient.get():\r\n salient_opt_panel.grid(row=14, columnspan=2, pady=2, sticky=\"w\")\r\n else:\r\n salient_opt_panel.grid_remove()\r\n\r\n # right collage option panel ROW 12\r\n is_salient = BooleanVar()\r\n is_salient.set(False)\r\n is_salient_check = Checkbutton(right_col_opt_panel, text=\"Salient objects only\",\r\n variable=is_salient, command=attach_salient_opt)\r\n is_salient_check.grid(row=12, columnspan=2, sticky=\"w\")\r\n\r\n def transp_cb():\r\n if transparent.get():\r\n is_salient.set(False)\r\n is_salient_check.config(state='disabled')\r\n dither.set(False)\r\n dither_check.config(state='disabled')\r\n else:\r\n is_salient_check.config(state='enabled')\r\n dither_check.config(state='enabled')\r\n\r\n # right collage option panel ROW 13\r\n transparent = BooleanVar()\r\n transparent.set(False)\r\n transparent_check = CheckbuttonWithTooltip(right_col_opt_panel, text=\"Transparency masking\", \r\n variable=transparent, tooltip=mkg.PARAMS.transparent.help, command=transp_cb)\r\n transparent_check.grid(row=13, columnspan=2, sticky=\"w\")\r\n\r\n def change_thresh():\r\n if dest_img is not None:\r\n lower_thresh = saliency_thresh_scale.get() / 100\r\n assert 0.0 <= lower_thresh <= 1.0\r\n if dest_img.shape[2] == 4:\r\n tmp_dest_img = cv2.cvtColor(dest_img, cv2.COLOR_BGRA2BGR)\r\n else:\r\n tmp_dest_img = dest_img.copy()\r\n _, thresh_map = cv2.saliency.StaticSaliencyFineGrained_create().computeSaliency(tmp_dest_img)\r\n tmp_dest_img[thresh_map < lower_thresh] = 255\r\n show_img(tmp_dest_img, False)\r\n \r\n # right collage option panel ROW 14\r\n changed_thresh_debounced = Debounce(change_thresh)\r\n salient_opt_panel = PanedWindow(right_col_opt_panel)\r\n Label(salient_opt_panel, text=\"Saliency threshold: \").grid(row=0, column=0, sticky=\"w\")\r\n saliency_thresh_scale = Scale(salient_opt_panel, from_=1.0, to=99.0, orient=HORIZONTAL, length=150, command=changed_thresh_debounced)\r\n saliency_thresh_scale.set(50.0)\r\n saliency_thresh_scale.grid(row=1, columnspan=2, sticky=\"W\")\r\n\r\n # right collage option panel ROW 16\r\n collage_button = Button(right_col_opt_panel, text=\" Generate Collage \", command=generate_collage)\r\n collage_button.config(state='disabled')\r\n collage_button.grid(row=16, columnspan=2, pady=(3, 5))\r\n # ------------------------ end right collage option panel --------------------\r\n\r\n # right panel ROW 9:\r\n Separator(right_panel, orient=\"horizontal\").grid(\r\n row=9, columnspan=2, sticky=\"we\", pady=(4, 10))\r\n\r\n save_img_init_dir = init_dir\r\n\r\n def save_img():\r\n global save_img_init_dir\r\n if result_img is None:\r\n messagebox.showerror(\"Error\", \"You don't have any image to save yet!\")\r\n return\r\n \r\n fp = filedialog.asksaveasfilename(initialdir=save_img_init_dir, title=\"Save your collage\",\r\n filetypes=((\"images\", \"*.jpg\"), (\"images\", \"*.png\")),\r\n defaultextension=\".png\", initialfile=\"result.png\")\r\n dir_name = os.path.dirname(fp)\r\n if fp is not None and len(fp) > 0 and os.path.isdir(dir_name):\r\n save_img_init_dir = dir_name\r\n try:\r\n mkg.imwrite(fp, result_img)\r\n print(\"Image saved to\", fp)\r\n except:\r\n messagebox.showerror(\"Error\", traceback.format_exc())\r\n\r\n\r\n def save_tile_info():\r\n global save_img_init_dir\r\n if result_tile_info is None:\r\n messagebox.showerror(\"Error\", \"You need to make a collage/photomosaic first!\")\r\n return\r\n \r\n fp = filedialog.asksaveasfilename(initialdir=save_img_init_dir, title=\"Save your collage\",\r\n filetypes=((\"csv file\", \"*.csv\"), (\"text file\", \"*.txt\")),\r\n defaultextension=\".csv\", initialfile=\"tile_info.csv\")\r\n dir_name = os.path.dirname(fp)\r\n if fp is not None and len(fp) > 0 and os.path.isdir(dir_name):\r\n save_img_init_dir = dir_name\r\n try:\r\n with open(fp, \"w\", encoding=\"utf-8\") as f:\r\n f.write(result_tile_info)\r\n print(\"Tile info saved to\", fp)\r\n except:\r\n messagebox.showerror(\"Error\", traceback.format_exc())\r\n\r\n # right panel ROW 10:\r\n save_button = Button(right_panel, text=\" Save image \", command=save_img)\r\n save_button.config(state='disabled')\r\n save_button.grid(row=10, columnspan=2, pady=(0, 4))\r\n\r\n tile_save_button = Button(right_panel, text=\" Save tile info \", command=save_tile_info)\r\n tile_save_button.config(state='disabled')\r\n tile_save_button.grid(row=11, columnspan=2)\r\n # -------------------------- end right panel -----------------------------------\r\n\r\n # make the window appear at the center\r\n # https://www.reddit.com/r/Python/comments/6m03sh/make_tkinter_window_in_center_of_screen_newbie/\r\n root.update_idletasks()\r\n w = root.winfo_screenwidth()\r\n h = root.winfo_screenheight()\r\n size = tuple(int(pos) for pos in root.geometry().split('+')[0].split('x'))\r\n x = w / 2 - size[0] / 2\r\n y = h / 2 - size[1] / 2 - 10\r\n root.geometry(\"%dx%d+%d+%d\" % (size + (x, y)))\r\n root.update()\r\n\r\n right_panel_width = right_panel.winfo_width()\r\n log_entry_height = left_panel.winfo_height() - canvas.winfo_height()\r\n last_resize_time = time.time()\r\n\r\n def canvas_resize(event):\r\n global last_resize_time\r\n if time.time() - last_resize_time > 0.1 and event.width >= 850 and event.height >= 550:\r\n last_resize_time = time.time()\r\n log_entry.configure(height=6 + math.floor((event.height - 500) / 80))\r\n log_entry.width = log_entry.initial_width + math.floor((event.width - 800) / 10)\r\n mkg.pbar_ncols = log_entry.width\r\n log_entry.update()\r\n canvas.configure(\r\n width=event.width - right_panel_width - 20, \r\n height=event.height - log_entry.winfo_height() - 15)\r\n canvas.update()\r\n show_img(result_img, False)\r\n\r\n root.bind(\"\", canvas_resize)\r\n out_wrapper = log_entry\r\n mkg.enable_gpu(False)\r\n\r\n # mainly for debugging purposes\r\n if cmd_args.D:\r\n file_path.set(cmd_args.src)\r\n print(\"Loading tiles from\", cmd_args.src)\r\n pool.submit(load_img_action).add_done_callback(show_img)\r\n\r\n print(\"Destination image loaded from\", cmd_args.collage)\r\n dest_img = mkg.imread(cmd_args.collage, cv2.IMREAD_UNCHANGED)\r\n show_img(dest_img, False)\r\n dest_img_path.set(cmd_args.collage)\r\n\r\n reload_img_button.config(state='enabled')\r\n sort_button.config(state='enabled')\r\n collage_button.config(state='enabled')\r\n\r\n sys.stdout = out_wrapper\r\n sys.stderr = out_wrapper\r\n root.mainloop()\r\n" }, { "path": "io_utils.py", "content": "import os\nimport io\nimport sys\nimport time\nimport queue\nimport ctypes\nimport platform\nimport tempfile\nimport threading\nfrom tkinter import Text, END\nfrom contextlib import contextmanager\n\nfrom tqdm import tqdm\n\n\n@contextmanager\ndef stdout_redirector(stream: io.TextIOBase):\n \"\"\"\n write stuff from stdout to stream that has a .write method \n\n part of the following function is borrowed from\n 1. https://stackoverflow.com/questions/4675728/redirect-stdout-to-a-file-in-python\n 2. https://gist.github.com/natedileas/8eb31dc03b76183c0211cdde57791005\n \"\"\"\n if platform.system() == \"Windows\":\n if hasattr(sys, 'gettotalrefcount'): # debug build\n libc = ctypes.CDLL('ucrtbased')\n else:\n libc = ctypes.CDLL('api-ms-win-crt-stdio-l1-1-0')\n else:\n libc = ctypes.CDLL(None)\n \n stdout = sys.__stdout__\n stdout_fd = stdout.fileno()\n\n # copy stdout_fd before it is overwritten\n #NOTE: `copied` is inheritable on Windows when duplicating a standard stream\n with os.fdopen(os.dup(stdout_fd), 'wb') as copied, tempfile.TemporaryFile(mode='w+b') as tfile: \n stdout.flush() # flush library buffers that dup2 knows nothing about\n os.dup2(tfile.fileno(), stdout_fd) # $ exec >&to\n\n try:\n flag = True\n\n def poll_once():\n libc.fflush(None)\n stdout.flush()\n tfile.flush()\n tfile.seek(0)\n stream.write(tfile.read().decode(\"utf-8\"))\n tfile.truncate(0)\n\n def poll():\n while flag:\n poll_once()\n time.sleep(0.1)\n poll_once()\n\n poll_thread = threading.Thread(target=poll)\n poll_thread.start()\n\n yield stdout # allow code to be run with the redirected stdout\n finally:\n # restore stdout to its previous value\n # NOTE: dup2 makes stdout_fd inheritable unconditionally\n stdout.flush()\n flag = False\n poll_thread.join()\n os.dup2(copied.fileno(), stdout_fd) # $ exec >&copied\n stream.close()\n\n\nclass JVOutWrapper:\n \"\"\"\n The output wrapper for displaying the progress of J-V algorithm\n \"\"\"\n\n def __init__(self, io_wrapper, ncols):\n self.io_wrapper = io_wrapper\n self.tqdm = None\n self.ncols = ncols\n\n def write(self, lines: str):\n if self.io_wrapper is None:\n return\n for line in lines.split(\"\\n\"):\n if not line.startswith(\"lapjv: \"):\n # self.io_wrapper.write(line + \"\\n\")\n continue\n if line.startswith(\"lapjv: AUGMENT SOLUTION row \"):\n line = line.replace(\" \", \"\")\n slash_idx = line.find(\"/\")\n s_idx = line.find(\"[\")\n e_idx = line.find(\"]\")\n if s_idx > -1 and slash_idx > -1 and e_idx > -1:\n if self.tqdm:\n self.tqdm.n = int(line[s_idx + 1:slash_idx])\n self.tqdm.update(0)\n else:\n self.tqdm = tqdm(file=self.io_wrapper, ncols=self.ncols, total=int(line[slash_idx + 1:e_idx]), desc=\"lapjv: \")\n continue\n if not self.tqdm:\n self.io_wrapper.write(line + \"\\n\")\n\n def flush(self):\n pass\n\n def close(self):\n if self.tqdm:\n self.tqdm.n = self.tqdm.total\n self.tqdm.refresh()\n self.tqdm.close()\n\n\n# adapted from\n# https://stackoverflow.com/questions/16745507/tkinter-how-to-use-threads-to-preventing-main-event-loop-from-freezing\nclass SafeText(Text):\n def __init__(self, master, **options):\n Text.__init__(self, master, **options)\n self.queue = queue.Queue()\n self.encoding = \"utf-8\"\n self.gui = True\n self.initial_width = 85\n self.width = self.initial_width\n self.update_me()\n\n def write(self, line: str):\n self.queue.put(line)\n\n def flush(self):\n pass\n\n # this one run in the main thread\n def update_me(self):\n while not self.queue.empty():\n line = self.queue.get_nowait()\n\n # a naive way to process the \\r control char\n if line.find(\"\\r\") > -1:\n line = line.replace(\"\\r\", \"\")\n row = int(self.index(END).split(\".\")[0])\n self.delete(\"{}.0\".format(row - 1),\n \"{}.{}\".format(row - 1, len(line)))\n self.insert(\"end-1c linestart\", line)\n else:\n self.insert(END, line)\n self.see(\"end-1c\")\n self.update_idletasks()\n self.after(50, self.update_me)\n" }, { "path": "make_img.py", "content": "import os\r\nimport sys\r\nimport time\r\nimport math\r\nimport random\r\nimport argparse\r\nimport itertools\r\nimport traceback\r\nimport multiprocessing as mp\r\nfrom fractions import Fraction\r\nfrom typing import Any, Callable, List, Tuple\r\nfrom collections import defaultdict\r\n\r\nfrom io_utils import stdout_redirector, JVOutWrapper\r\n\r\nimport cv2\r\nimport imagesize\r\nimport numpy as np\r\ncp = np\r\nfrom tqdm import tqdm\r\nfrom lapjv import lapjv\r\n\r\nGrid = Tuple[int, int] # grid size = (width, height)\r\nBackgroundRGB = Tuple[int, int, int]\r\n\r\nif mp.current_process().name != \"MainProcess\":\r\n sys.stdout = open(os.devnull, \"w\")\r\n sys.stderr = sys.stdout\r\n\r\npbar_ncols = None\r\nLIMIT = 2**32\r\n\r\n\r\nclass _PARAMETER:\r\n def __init__(self, type: Any, help: str, default=None, nargs=None, choices: List[Any]=None) -> None:\r\n self.type = type\r\n self.default = default\r\n self.help = help\r\n self.nargs = nargs\r\n self.choices = choices\r\n\r\n# We gather parameters here so they can be reused else where\r\nclass PARAMS:\r\n path = _PARAMETER(help=\"Path to the tiles\", type=str)\r\n recursive = _PARAMETER(type=bool, default=False, help=\"Whether to read the sub-folders for the specified path\")\r\n num_process = _PARAMETER(type=int, default=mp.cpu_count() // 2, help=\"Number of processes to use for parallelizable operations\")\r\n out = _PARAMETER(default=\"result.png\", type=str, help=\"The filename of the output collage/photomosaic\")\r\n size = _PARAMETER(type=int, nargs=\"+\", default=(50,), \r\n help=\"Width and height of each tile in pixels in the resulting collage/photomosaic. \"\r\n \"If two numbers are specified, they are treated as width and height. \"\r\n \"If one number is specified, the number is treated as the width \"\r\n \"and the height is inferred from the aspect ratios of the images provided. \")\r\n quiet = _PARAMETER(type=bool, default=False, help=\"Do not print progress message to console\")\r\n auto_rotate = _PARAMETER(type=int, default=0, choices=[-1, 0, 1],\r\n help=\"Options to auto rotate tiles to best match the specified tile size. 0: do not auto rotate. \"\r\n \"1: attempt to rotate counterclockwise by 90 degrees. -1: attempt to rotate clockwise by 90 degrees\")\r\n resize_opt = _PARAMETER(type=str, default=\"center\", choices=[\"center\", \"stretch\", \"fit\"], \r\n help=\"How to resize each tile so they have the desired aspect ratio and size, \"\r\n \"which can be specified fully or partially by --size. \"\r\n \"Center: crop the largest rectangle from the center. Stretch: stretch the tile. \"\r\n \"Fit: pad the tiles with white background\")\r\n gpu = _PARAMETER(type=bool, default=False, \r\n help=\"Use GPU acceleration. Requires cupy to be installed and a capable GPU. Note that USUALLY this is useful when you: \"\r\n \"1. have a lot of tiles (typically > 10000), and\"\r\n \"2. are using the unfair mode, and\"\r\n \"3. (for photomosaic videos only) only have few cpu cores\"\r\n \"Also note: enabling GPU acceleration will disable multiprocessing on CPU for videos\"\r\n )\r\n mem_limit = _PARAMETER(type=int, default=4096, \r\n help=\"The APPROXIMATE memory limit in MB when computing a photomosaic in unfair mode. Applicable both CPU and GPU computing. \"\r\n \"If you run into memory issues when using GPU, try reduce this memory limit\")\r\n tile_info_out = _PARAMETER(type=str, default=\"\",\r\n help=\"Path to save the list of tile filenames for the collage/photomosaic. If empty, it will not be saved.\")\r\n \r\n # ---------------- sort collage options ------------------\r\n ratio = _PARAMETER(type=int, default=(16, 9), help=\"Aspect ratio of the output image\", nargs=2)\r\n sort = _PARAMETER(type=str, default=\"bgr_sum\", help=\"Sort method to use\", choices=[\r\n \"none\", \"bgr_sum\", \"av_hue\", \"av_sat\", \"av_lum\", \"rand\"\r\n ])\r\n rev_row = _PARAMETER(type=bool, default=False, help=\"Whether to use the S-shaped alignment.\")\r\n rev_sort = _PARAMETER(type=bool, default=False, help=\"Sort in the reverse direction.\")\r\n \r\n # ---------------- photomosaic common options ------------------\r\n dest_img = _PARAMETER(type=str, default=\"\", help=\"The path to the destination image that you want to build a photomosaic for\")\r\n colorspace = _PARAMETER(type=str, default=\"lab\", choices=[\"hsv\", \"hsl\", \"bgr\", \"lab\", \"luv\"], \r\n help=\"The colorspace used to calculate the metric\")\r\n metric = _PARAMETER(type=str, default=\"euclidean\", choices=[\"euclidean\", \"cityblock\", \"chebyshev\", \"cosine\"], \r\n help=\"Distance metric used when evaluating the distance between two color vectors\")\r\n transparent = _PARAMETER(type=bool, default=False, \r\n help=\"Enable transparency masking. The transparent regions of the destination image will be maintained in the photomosaic\"\r\n \"Cannot be used together with --salient\")\r\n \r\n # ---- unfair tile assignment options -----\r\n unfair = _PARAMETER(type=bool, default=False, \r\n help=\"Whether to allow each tile to be used different amount of times (unfair tile usage). \")\r\n max_width = _PARAMETER(type=int, default=80, \r\n help=\"Maximum width of the collage. This option is only valid if unfair option is enabled\") \r\n freq_mul = _PARAMETER(type=float, default=0.0, \r\n help=\"Frequency multiplier to balance tile fairless and mosaic quality. Minimum: 0. \"\r\n \"More weight will be put on tile fairness when this number increases.\")\r\n dither = _PARAMETER(type=bool, default=False, \r\n help=\"Whether to enabled dithering. You must also specify --deterministic if enabled. \")\r\n deterministic = _PARAMETER(type=bool, default=False, \r\n help=\"Do not randomize the tiles. This option is only valid if unfair option is enabled\")\r\n\r\n # --- fair tile assignment options ---\r\n dup = _PARAMETER(type=float, default=1, \r\n help=\"If a positive integer: duplicate the set of tiles by how many times. Can be a fraction\")\r\n\r\n # ---- saliency detection options ---\r\n salient = _PARAMETER(type=bool, default=False, help=\"Make photomosaic for salient objects only\")\r\n lower_thresh = _PARAMETER(type=float, default=0.5, \r\n help=\"The threshold for saliency detection, between 0.0 (no object area = blank) and 1.0 (maximum object area = original image)\")\r\n\r\n # ---- blending options ---\r\n blending = _PARAMETER(type=str, default=\"alpha\", choices=[\"alpha\", \"brightness\"], \r\n help=\"The types of blending used. alpha: alpha (transparency) blending. Brightness: blending of brightness (lightness) channel in the HSL colorspace\")\r\n blending_level = _PARAMETER(type=float, default=0.0, \r\n help=\"Level of blending, between 0.0 (no blending) and 1.0 (maximum blending). Default is no blending\")\r\n\r\n video = _PARAMETER(type=bool, default=False, help=\"Make a photomosaic video from dest_img which is assumed to be a video\")\r\n skip_frame = _PARAMETER(type=int, default=1, help=\"Make a photomosaic every this number of frames\")\r\n\r\n# https://stackoverflow.com/questions/26598109/preserve-custom-attributes-when-pickling-subclass-of-numpy-array\r\nclass InfoArray(np.ndarray):\r\n def __new__(cls, input_array, info=''):\r\n # Input array is an already formed ndarray instance\r\n # We first cast to be our class type\r\n obj = np.asarray(input_array).view(cls)\r\n # add the new attribute to the created instance\r\n obj.info = info\r\n # Finally, we must return the newly created object:\r\n return obj\r\n\r\n def __array_finalize__(self, obj):\r\n # see InfoArray.__array_finalize__ for comments\r\n if obj is None: return\r\n self.info = getattr(obj, 'info', None)\r\n\r\n def __reduce__(self):\r\n # Get the parent's __reduce__ tuple\r\n pickled_state = super(InfoArray, self).__reduce__()\r\n # Create our own tuple to pass to __setstate__\r\n new_state = pickled_state[2] + (self.info,)\r\n # Return a tuple that replaces the parent's __setstate__ tuple with our own\r\n return (pickled_state[0], pickled_state[1], new_state)\r\n\r\n def __setstate__(self, state):\r\n self.info = state[-1] # Set the info attribute\r\n # Call the parent's __setstate__ with the other tuple elements.\r\n super(InfoArray, self).__setstate__(state[0:-1])\r\n\r\n\r\nImgList = List[InfoArray]\r\ncupy_available = False\r\n\r\n\r\ndef fast_sq_euclidean(Asq, Bsq, AB):\r\n AB *= -2\r\n AB += Asq\r\n AB += Bsq\r\n return AB\r\n\r\n\r\ndef fast_cityblock(A, B, axis, out):\r\n Z = A - B\r\n np.abs(Z, out=Z)\r\n return np.sum(Z, axis=axis, out=out)\r\n\r\n\r\ndef fast_chebyshev(A, B, axis, out):\r\n Z = A - B\r\n np.abs(Z, out=Z)\r\n return np.max(Z, axis=axis, out=out)\r\n\r\n\r\ndef to_cpu(X: np.ndarray) -> np.ndarray:\r\n return X.get() if cupy_available else X\r\n\r\n\r\ndef bgr_sum(img: np.ndarray) -> float:\r\n \"\"\"\r\n compute the sum of all RGB values across an image\r\n \"\"\"\r\n return np.sum(img)\r\n\r\n\r\ndef av_hue(img: np.ndarray) -> float:\r\n \"\"\"\r\n compute the average hue of all pixels in HSV color space\r\n \"\"\"\r\n hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\r\n return np.mean(hsv[:, :, 0])\r\n\r\n\r\ndef av_sat(img: np.ndarray) -> float:\r\n \"\"\"\r\n compute the average saturation of all pixels in HSV color space\r\n \"\"\"\r\n hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)\r\n return np.mean(hsv[:, :, 1])\r\n\r\n\r\nlum_coeffs = np.array([0.241, 0.691, 0.068], dtype=np.float32)[np.newaxis, np.newaxis, :]\r\n\r\n\r\ndef av_lum(img) -> float:\r\n \"\"\"\r\n compute the average luminosity\r\n \"\"\"\r\n lum = img * lum_coeffs\r\n np.sqrt(lum, out=lum)\r\n return np.mean(lum)\r\n\r\n\r\ndef rand(img: np.ndarray) -> float:\r\n \"\"\"\r\n generate a random number for each image\r\n \"\"\"\r\n return random.random()\r\n\r\n\r\ndef calc_grid_size(rw: int, rh: int, num_imgs: int, shape: Tuple[int, int, int]) -> Grid:\r\n \"\"\"\r\n :param rw: the width of the target image\r\n :param rh: the height of the target image\r\n :param num_imgs: number of images available\r\n :param shape: the shape of a tile\r\n :return: an optimal grid size\r\n \"\"\"\r\n possible_wh = []\r\n th, tw, _ = shape\r\n for width in range(1, num_imgs):\r\n height = math.ceil(num_imgs / width)\r\n possible_wh.append((width * tw / (th * height), width, height))\r\n dest_ratio = rw / rh\r\n grid = min(possible_wh, key=lambda x: (x[0] - dest_ratio) ** 2)[1:]\r\n print(\"Tile shape:\", (tw, th))\r\n print(\"Calculated grid size based on the aspect ratio of the destination image:\", grid)\r\n print(f\"Collage size will be {grid[0] * tw}x{grid[1] * th}. \")\r\n return grid\r\n\r\n\r\ndef make_collage_helper(grid: Grid, sorted_imgs: ImgList, rev=False, ridx=None, cidx=None, file=None):\r\n grid = grid[::-1]\r\n th, tw, tc = sorted_imgs[0].shape\r\n tile_info = np.full(grid, \"background\", dtype=str)\r\n if ridx is None or cidx is None:\r\n combined_img = np.empty((grid[0] * th, grid[1] * tw, 4), dtype=np.uint8)\r\n # use an array of references to avoid copying individual tiles\r\n tiles = np.array([None] * len(sorted_imgs), dtype=object)\r\n tiles[:] = sorted_imgs\r\n tiles.shape = grid\r\n if rev:\r\n tiles[1::2] = tiles[1::2, ::-1]\r\n \r\n # while we can use a few lines of transpose + reshape to do this, \r\n # we just use an ordinary for loop in order to show the progress\r\n for i, j in tqdm(itertools.product(range(grid[0]), range(grid[1])), \r\n desc=\"[Aligning tiles]\", ncols=pbar_ncols, total=np.prod(grid), file=file):\r\n tile = tiles[i, j]\r\n # if this tile already have an alpha channel, use it\r\n if tile.shape[2] == 4:\r\n combined_img[i * th:(i + 1)*th, j * tw:(j + 1)*tw] = tile\r\n else:\r\n combined_img[i * th:(i + 1)*th, j * tw:(j + 1)*tw, :3] = tile\r\n combined_img[i * th:(i + 1)*th, j * tw:(j + 1)*tw, 3] = 255\r\n tile_info[i, j] = tile.info\r\n else:\r\n assert not rev\r\n combined_img = np.full((grid[0] * th, grid[1] * tw, 4), [255, 255, 255, 0], dtype=np.uint8)\r\n for k in tqdm(range(len(ridx)), desc=\"[Aligning tiles]\", ncols=pbar_ncols, file=file):\r\n i = ridx[k]\r\n j = cidx[k]\r\n combined_img[i * th:(i + 1)*th, j * tw:(j + 1)*tw, :tc] = sorted_imgs[k]\r\n combined_img[i * th:(i + 1)*th, j * tw:(j + 1)*tw, 3] = 255\r\n tile_info[i, j] = sorted_imgs[k].info\r\n return combined_img, f\"Grid dimension: {grid[::-1]}\\n\" + '\\n'.join(tile_info.flatten())\r\n\r\n\r\ndef make_collage(grid: Grid, sorted_imgs: ImgList, rev=False):\r\n \"\"\"\r\n :param grid: grid size\r\n :param sorted_imgs: list of images sorted in correct position\r\n :param rev: whether to have opposite alignment for consecutive rows\r\n :return: a collage\r\n \"\"\"\r\n total = np.prod(grid)\r\n diff = total - len(sorted_imgs)\r\n if diff > 0:\r\n print(f\"Note: {diff} transparent tiles will be added to the grid.\")\r\n sorted_imgs.extend([InfoArray(\r\n np.full((*sorted_imgs[0].shape[:2], 4), [255, 255, 255, 0], dtype=np.uint8), 'background')] * diff)\r\n elif len(sorted_imgs) > total:\r\n print(f\"Note: {len(sorted_imgs) - total} tiles will be dropped from the grid.\")\r\n del sorted_imgs[total:]\r\n return make_collage_helper(grid, sorted_imgs, rev)\r\n\r\n\r\ndef alpha_blend(combined_img: np.ndarray, dest_img: np.ndarray, alpha=0.9):\r\n if alpha == 1.0:\r\n return combined_img\r\n if dest_img.shape[2] == 4:\r\n dest_img = cv2.cvtColor(dest_img, cv2.COLOR_BGRA2BGR)\r\n dest_img = dest_img * np.float32(1 - alpha)\r\n dest_img = cv2.resize(dest_img, combined_img.shape[1::-1])\r\n combined_img = combined_img * np.array([alpha, alpha, alpha, 1], dtype=np.float32).reshape(1, 1, 4)\r\n combined_img[:, :, :3] += dest_img\r\n return combined_img.astype(np.uint8)\r\n\r\n\r\ndef brightness_blend(combined_img: np.ndarray, dest_img: np.ndarray, alpha=0.9):\r\n \"\"\"\r\n blend the 2 imgs in the lightness channel (L in HSL)\r\n \"\"\"\r\n if alpha == 1.0:\r\n return combined_img\r\n if dest_img.shape[2] == 4:\r\n dest_img = cv2.cvtColor(dest_img, cv2.COLOR_BGRA2BGR)\r\n dest_img = cv2.cvtColor(dest_img, cv2.COLOR_BGR2HLS)\r\n dest_l = dest_img[:, :, 1] * np.float32(1 - alpha)\r\n dest_l = cv2.resize(dest_l, combined_img.shape[1::-1])\r\n combined_img_hls = cv2.cvtColor(combined_img[:, :, :3], cv2.COLOR_BGR2HLS)\r\n comb_l = combined_img_hls[:, :, 1] * np.float32(alpha)\r\n comb_l += dest_l\r\n combined_img_hls[:, :, 1] = comb_l\r\n combined_img = combined_img.copy()\r\n combined_img[:, :, :3] = cv2.cvtColor(combined_img_hls, cv2.COLOR_HLS2BGR)\r\n return combined_img\r\n\r\n\r\ndef sort_collage(imgs: ImgList, ratio: Grid, sort_method=\"pca_lab\", rev_sort=False) -> Tuple[Grid, np.ndarray]:\r\n \"\"\"\r\n :param imgs: list of images\r\n :param ratio: The aspect ratio of the collage\r\n :param sort_method:\r\n :param rev_sort: whether to reverse the sorted array\r\n :return: [calculated grid size, sorted image array]\r\n \"\"\"\r\n t = time.time()\r\n grid = calc_grid_size(ratio[0], ratio[1], len(imgs), imgs[0].shape)\r\n\r\n if sort_method == \"none\":\r\n return grid, imgs\r\n\r\n print(\"Sorting images...\") \r\n sort_function = eval(sort_method)\r\n indices = np.array(list(map(sort_function, imgs))).argsort()\r\n if rev_sort:\r\n indices = indices[::-1]\r\n print(\"Time taken: {}s\".format(np.round(time.time() - t, 2)))\r\n return grid, [imgs[i] for i in indices]\r\n\r\n\r\ndef solve_lap(cost_matrix: np.ndarray, v=-1):\r\n if v == -1:\r\n v = sys.__stderr__\r\n \"\"\"\r\n solve the linear sum assignment (LAP) problem with progress info\r\n \"\"\"\r\n print(\"Computing optimal assignment on a {}x{} matrix...\".format(cost_matrix.shape[0], cost_matrix.shape[1]))\r\n wrapper = JVOutWrapper(v, pbar_ncols)\r\n with stdout_redirector(wrapper):\r\n _, cols, cost = lapjv(cost_matrix, verbose=1)\r\n cost = cost[0]\r\n print(\"Total assignment cost:\", cost)\r\n return cols\r\n\r\n\r\ndef solve_lap_greedy(cost_matrix: np.ndarray, v=None):\r\n assert cost_matrix.shape[0] == cost_matrix.shape[1]\r\n\r\n print(\"Computing greedy assignment on a {}x{} matrix...\".format(cost_matrix.shape[0], cost_matrix.shape[1]))\r\n row_idx, col_idx = np.unravel_index(np.argsort(cost_matrix, axis=None), cost_matrix.shape)\r\n cost = 0\r\n row_assigned = np.full(cost_matrix.shape[0], -1, dtype=np.int32)\r\n col_assigned = np.full(cost_matrix.shape[0], -1, dtype=np.int32)\r\n pbar = tqdm(ncols=pbar_ncols, total=cost_matrix.shape[0])\r\n for ridx, cidx in zip(row_idx, col_idx):\r\n if row_assigned[ridx] == -1 and col_assigned[cidx] == -1:\r\n row_assigned[ridx] = cidx\r\n col_assigned[cidx] = ridx\r\n cost += cost_matrix[ridx, cidx]\r\n\r\n pbar.update()\r\n if pbar.n == pbar.total:\r\n break\r\n pbar.close()\r\n print(\"Total assignment cost:\", cost)\r\n return col_assigned\r\n\r\n\r\ndef compute_block_map(thresh_map: np.ndarray, block_width: int, block_height: int, lower_thresh: int):\r\n \"\"\"\r\n Find the indices of the blocks that contain salient pixels according to the thresh_map\r\n\r\n returns [row indices, column indices, resized threshold map] of sizes [(N,), (N,), (W x H)]\r\n \"\"\"\r\n height, width = thresh_map.shape\r\n dst_size = (width - width % block_width, height - height % block_height)\r\n if thresh_map.shape[::-1] != dst_size:\r\n thresh_map = cv2.resize(thresh_map, dst_size)\r\n row_idx, col_idx = np.nonzero(thresh_map.reshape(\r\n dst_size[1] // block_height, block_height, dst_size[0] // block_width, block_width).max(axis=(1, 3)) >= lower_thresh\r\n )\r\n return row_idx, col_idx, thresh_map\r\n\r\n\r\ndef dup_to_meet_total(imgs: ImgList, total: int):\r\n \"\"\"\r\n note that this function modifies imgs in place\r\n \"\"\"\r\n orig_len = len(imgs)\r\n if total < orig_len:\r\n print(f\"{total} tiles will be used 1 time. {orig_len - total}/{orig_len} tiles will not be used. \")\r\n del imgs[total:]\r\n return imgs\r\n \r\n full_count = total // orig_len\r\n remaining = total % orig_len\r\n\r\n imgs *= full_count\r\n if remaining > 0:\r\n print(f\"{orig_len - remaining} tiles will be used {full_count} times. {remaining} tiles will be used {full_count + 1} times. Total tiles: {orig_len}.\")\r\n imgs.extend(imgs[:remaining])\r\n else:\r\n print(f\"Total tiles: {orig_len}. All of them will be used {full_count} times.\")\r\n return imgs\r\n\r\n\r\ndef _cosine(A, B):\r\n return 1 - cp.inner(A / cp.linalg.norm(A, axis=1, keepdims=True), B)\r\n\r\n\r\ndef _euclidean(A, B, BsqT):\r\n Asq = cp.sum(A**2, axis=1, keepdims=True)\r\n return fast_sq_euclidean(Asq, BsqT, A.dot(B.T))\r\n\r\n\r\ndef _other(A, B, dist_func, row_stride):\r\n total = A.shape[0]\r\n dist_mat = cp.empty((total, B.shape[1]), dtype=cp.float32)\r\n i = 0\r\n while i < total - row_stride:\r\n next_i = i + row_stride\r\n dist_func(A[i:next_i, cp.newaxis, :], B, out=dist_mat[i:next_i], axis=2)\r\n i = next_i\r\n if i < total:\r\n dist_func(A[i:, cp.newaxis, :], B, out=dist_mat[i:], axis=2)\r\n return dist_mat\r\n\r\n\r\ndef strip_alpha(dest_img: np.ndarray) -> np.ndarray:\r\n if dest_img.shape[2] == 4:\r\n return cv2.cvtColor(dest_img, cv2.COLOR_BGRA2BGR)\r\n return dest_img\r\n\r\n\r\ndef thresh_map_transp(dest_img: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:\r\n assert dest_img.shape[2] == 4, \"You need an image with transparent background to do this\"\r\n return strip_alpha(dest_img), dest_img[:, :, 3] > 0.0\r\n\r\n\r\nclass CachedCDist:\r\n def __init__(self, metric: str, B: np.ndarray):\r\n \"\"\"\r\n Simple implementation of scipy.spatial.distance.cdist\r\n \"\"\"\r\n if metric == \"cosine\":\r\n self.args = [B / cp.linalg.norm(B, axis=1, keepdims=True)]\r\n self.func = _cosine\r\n elif metric == \"euclidean\":\r\n self.args = [B, cp.sum(B**2, axis=1, keepdims=True).T]\r\n self.func = _euclidean\r\n else:\r\n row_stride = LIMIT // (B.size * 4)\r\n B = B[cp.newaxis]\r\n if metric == \"cityblock\":\r\n self.args = [B, fast_cityblock, row_stride]\r\n elif metric == \"chebyshev\":\r\n self.args = [B, fast_chebyshev, row_stride]\r\n else:\r\n raise ValueError(f\"invalid metric {metric}\")\r\n self.func = _other\r\n\r\n def __call__(self, A: np.ndarray) -> np.ndarray:\r\n return self.func(A, *self.args)\r\n\r\n\r\nclass MosaicCommon:\r\n def __init__(self, imgs: ImgList, colorspace=\"lab\") -> None:\r\n self.imgs = imgs\r\n self.normalize_first = False\r\n if colorspace == \"bgr\":\r\n self.flag = None\r\n elif colorspace == \"hsv\":\r\n self.flag = cv2.COLOR_BGR2HSV\r\n self.normalize_first = True\r\n elif colorspace == \"hsl\":\r\n self.flag = cv2.COLOR_BGR2HLS\r\n self.normalize_first = True\r\n elif colorspace == \"lab\":\r\n self.flag = cv2.COLOR_BGR2LAB\r\n elif colorspace == \"luv\":\r\n self.flag = cv2.COLOR_BGR2LUV\r\n else:\r\n raise ValueError(\"Unknown colorspace \" + colorspace)\r\n\r\n def make_photomosaic(self, assignment: np.ndarray, file=None):\r\n return make_collage_helper(self.grid, [self.imgs[i] for i in assignment], file=file)\r\n\r\n def make_photomosaic_mask(self, assignment: np.ndarray, ridx: np.ndarray, cidx: np.ndarray, file=None):\r\n return make_collage_helper(self.grid, [self.imgs[i] for i in assignment], False, ridx, cidx, file=file)\r\n\r\n def convert_colorspace(self, img: np.ndarray):\r\n if self.flag is None:\r\n return\r\n cv2.cvtColor(img, self.flag, dst=img)\r\n if self.normalize_first:\r\n # for hsv/hsl, h is in range 0~360 while other channels are in range 0~1\r\n # need to normalize\r\n img[:, :, 0] *= 1 / 360.0\r\n \r\n def compute_block_size(self, dest_shape: Tuple[int, int, int], grid: Grid):\r\n self.grid = grid\r\n self.block_height = round(dest_shape[0] / grid[1])\r\n self.block_width = round(dest_shape[1] / grid[0])\r\n th, tw, _ = self.imgs[0].shape\r\n\r\n if self.block_width > tw or self.block_height > th:\r\n m = max(tw / self.block_width, th / self.block_height)\r\n self.block_width = math.floor(self.block_width * m)\r\n self.block_height = math.floor(self.block_height * m)\r\n self.flat_block_size = self.block_width * self.block_height * 3\r\n print(\"Block size:\", (self.block_width, self.block_height))\r\n\r\n self.target_sz = (grid[0] * self.block_width, grid[1] * self.block_height)\r\n print(f\"Resizing dest image from {dest_shape[1]}x{dest_shape[0]} to {self.target_sz[0]}x{self.target_sz[1]}\")\r\n\r\n def imgs_to_flat_blocks(self, metric: str):\r\n img_keys = np.zeros((len(self.imgs), self.block_height, self.block_width, 3), dtype=np.uint8)\r\n for i in range(len(self.imgs)):\r\n cv2.resize(self.imgs[i], (self.block_width, self.block_height), dst=img_keys[i])\r\n img_keys.shape = (-1, self.block_width, 3)\r\n img_keys = img_keys * np.float32(1 / 255.0)\r\n self.convert_colorspace(img_keys)\r\n img_keys.shape = (-1, self.flat_block_size)\r\n img_keys = cp.asarray(img_keys)\r\n self.cdist = CachedCDist(metric, img_keys)\r\n self.img_keys = img_keys\r\n return img_keys\r\n\r\n def dest_to_flat_blocks(self, dest_img: np.ndarray):\r\n dest_img = cv2.resize(dest_img, self.target_sz)\r\n dest_img = dest_img * np.float32(1 / 255.0)\r\n self.convert_colorspace(dest_img)\r\n dest_img = cp.asarray(dest_img)\r\n dest_img.shape = (self.grid[1], self.block_height, self.grid[0], self.block_width, 3)\r\n return dest_img.transpose((0, 2, 1, 3, 4)).reshape(-1, self.flat_block_size)\r\n\r\n def dest_to_flat_blocks_mask(self, dest_img: np.ndarray, ridx: np.ndarray, cidx: np.ndarray):\r\n dest_img = dest_img * np.float32(1 / 255.0)\r\n self.convert_colorspace(dest_img)\r\n dest_img.shape = (self.grid[1], self.block_height, self.grid[0], self.block_width, 3)\r\n dest_img = dest_img[ridx, :, cidx, :, :]\r\n dest_img.shape = (-1, self.flat_block_size)\r\n print(f\"Salient blocks/total blocks = {len(ridx)}/{np.prod(self.grid)}\")\r\n return cp.asarray(dest_img)\r\n\r\n\r\ndef calc_salient_col_even(dest_img: np.ndarray, imgs: ImgList, dup=1, colorspace=\"lab\", \r\n metric=\"euclidean\", lower_thresh=0.5, transparent=False, v=None):\r\n \"\"\"\r\n Compute the optimal assignment between the set of images provided and the set of pixels constitute of salient objects of the\r\n target image, with the restriction that every image should be used the same amount of times\r\n\r\n non salient part of the target image will be transparent\r\n \"\"\"\r\n t = time.time()\r\n print(\"Duplicating {} times\".format(dup))\r\n height, width, _ = dest_img.shape\r\n\r\n # this is just the initial (minimum) grid size\r\n total = round(len(imgs) * dup)\r\n grid = calc_grid_size(width, height, total, imgs[0].shape)\r\n if transparent:\r\n dest_img, orig_thresh_map = thresh_map_transp(dest_img)\r\n orig_thresh_map = orig_thresh_map.astype(np.float32)\r\n lower_thresh = 0.5\r\n else:\r\n dest_img = strip_alpha(dest_img)\r\n _, orig_thresh_map = cv2.saliency.StaticSaliencyFineGrained_create().computeSaliency(dest_img)\r\n \r\n bh_f = height / grid[1]\r\n bw_f = width / grid[0]\r\n # DDA-like algorithm to decrease block size while preserving aspect ratio\r\n if bw_f > bh_f:\r\n bw_delta = 1\r\n bh_delta = bh_f / bw_f\r\n else:\r\n bh_delta = 1\r\n bw_delta = bh_f / bw_f\r\n \r\n imgs = imgs.copy()\r\n while True:\r\n block_width = int(bw_f)\r\n block_height = int(bh_f)\r\n if block_width <= 0 or block_height <= 0:\r\n print(f\"Warning: Salient area is too small to put down all tiles given the duplication factor of {dup}. \"\r\n \"You can try to increase the saliency threshold if this is not desired.\")\r\n block_width = max(block_width, 1)\r\n block_height = max(block_height, 1)\r\n ridx, cidx, thresh_map = compute_block_map(orig_thresh_map, block_width, block_height, lower_thresh)\r\n break\r\n ridx, cidx, thresh_map = compute_block_map(orig_thresh_map, block_width, block_height, lower_thresh)\r\n if len(ridx) >= total:\r\n break\r\n bw_f -= bw_delta\r\n bh_f -= bh_delta\r\n\r\n print(len(ridx), lower_thresh)\r\n dup_to_meet_total(imgs, len(ridx))\r\n\r\n mos = MosaicCommon(imgs, colorspace)\r\n mos.block_width = block_width\r\n mos.block_height = block_height\r\n mos.flat_block_size = block_width * block_height * 3\r\n mos.grid = (thresh_map.shape[1] // block_width, thresh_map.shape[0] // block_height)\r\n\r\n print(\"Block size:\", (block_width, block_height))\r\n print(\"Grid size:\", mos.grid)\r\n\r\n mos.imgs_to_flat_blocks(metric)\r\n dest_img = cv2.resize(dest_img, thresh_map.shape[::-1])\r\n dest_img = mos.dest_to_flat_blocks_mask(dest_img, ridx, cidx)\r\n assignment = solve_lap(to_cpu(mos.cdist(dest_img).T), v)\r\n print(\"Time taken: {}s\".format((np.round(time.time() - t, 2))))\r\n return mos.make_photomosaic_mask(assignment, ridx, cidx)\r\n\r\nclass MosaicFairSalient:\r\n def __init__(self, *args, **kwargs) -> None:\r\n self.args = args\r\n self.kwargs = kwargs\r\n \r\n def process_dest_img(self, dest_img: np.ndarray):\r\n return calc_salient_col_even(dest_img, *self.args[1:], **self.kwargs)\r\n\r\n\r\nclass MosaicFair(MosaicCommon):\r\n def __init__(self, dest_shape: Tuple[int, int, int], imgs: ImgList, dup=1, colorspace=\"lab\", \r\n metric=\"euclidean\", grid=None) -> None:\r\n \"\"\"\r\n Compute the optimal assignment between the set of images provided and the set of pixels of the target image,\r\n with the restriction that every image should be used the same amount of times\r\n \"\"\"\r\n if grid is not None:\r\n print(\"Use the provided grid size:\", grid)\r\n dup = np.prod(grid) // len(imgs) + 1\r\n else:\r\n # Compute the grid size based on the number images that we have\r\n grid = calc_grid_size(dest_shape[1], dest_shape[0], round(len(imgs) * dup), imgs[0].shape)\r\n total = np.prod(grid)\r\n imgs = dup_to_meet_total(imgs.copy(), total)\r\n \r\n if total > 10000:\r\n print(\"Warning: this may take longer than 5 minutes to compute\")\r\n \r\n super().__init__(imgs, colorspace)\r\n self.compute_block_size(dest_shape, grid)\r\n self.imgs_to_flat_blocks(metric)\r\n\r\n def process_dest_img(self, dest_img: np.ndarray, file=None):\r\n dest_img = self.dest_to_flat_blocks(strip_alpha(dest_img))\r\n cols = solve_lap(to_cpu(self.cdist(dest_img).T), file)\r\n return self.make_photomosaic(cols)\r\n\r\n\r\nclass MosaicUnfair(MosaicCommon):\r\n def __init__(self, dest_shape: Tuple[int, int, int], imgs: ImgList, max_width: int, colorspace: str, metric: str, \r\n lower_thresh: float, freq_mul: float, randomize: bool, dither=False, transparent=False) -> None:\r\n # Because we don't have a fixed total amount of images as we can used a single image\r\n # for arbitrary amount of times, we need user to specify the maximum width in order to determine the grid size.\r\n dh, dw, _ = dest_shape\r\n th, tw, _ = imgs[0].shape\r\n grid = (max_width, round(dh * (max_width * tw / dw) / th))\r\n print(\"Calculated grid size based on the aspect ratio of the image provided:\", grid)\r\n print(\"Collage size:\", (grid[0] * tw, grid[1] * th))\r\n\r\n super().__init__(imgs, colorspace)\r\n self.compute_block_size(dest_shape, grid)\r\n img_keys = self.imgs_to_flat_blocks(metric)\r\n\r\n # number of rows in the cost matrix\r\n # note here we compute the cost matrix chunk by chunk to limit memory usage\r\n # a bit like sklearn.metrics.pairwise_distances_chunked\r\n num_rows = int(np.prod(grid))\r\n num_cols = img_keys.shape[0]\r\n print(f\"Distance matrix size: {(num_rows, num_cols)} = {num_rows * num_cols * 4 / 2**20}MB\")\r\n self.row_stride = (LIMIT - (img_keys.size + num_rows * (1 + self.flat_block_size)) * 4) // (num_cols * 4)\r\n if self.row_stride >= num_rows:\r\n print(\"No chunking will be performed on the distance matrix calculation\")\r\n else:\r\n print(f\"Chunk size: {self.row_stride*num_cols* 4 / 2**20}MB | {self.row_stride}/{num_rows}\")\r\n\r\n if freq_mul > 0:\r\n self.row_stride //= 16\r\n self.indices_freq = cp.empty(num_cols, dtype=cp.float32)\r\n self.row_range = cp.arange(0, self.row_stride, dtype=cp.int32)[:, cp.newaxis]\r\n self.temp = cp.arange(0, num_cols, dtype=cp.float32)\r\n else:\r\n self.row_stride //= 4\r\n\r\n self.freq_mul = freq_mul\r\n self.lower_thresh = lower_thresh\r\n self.randomize = randomize\r\n self.transparent = transparent\r\n self.saliency = False\r\n self.dither = dither\r\n _saliency_enabled = lower_thresh is not None\r\n if transparent:\r\n self.lower_thresh = 0.5\r\n if dither:\r\n print(\"Warning: dithering is not supported when transparency masking is on. Dithering will be turned off\")\r\n self.dither = False\r\n if _saliency_enabled:\r\n print(\"Warning: saliency is not supported when transparency masking is on. Saliency will be turned off\")\r\n elif self.dither:\r\n if _saliency_enabled:\r\n print(\"Warning: saliency is not supported when dithering is on. Saliency will be turned off\")\r\n if cp is not np:\r\n print(\"Warning: dithering is typically slower with --gpu enabled\")\r\n if randomize:\r\n print(\"Warning: dithering is not supported when randomization is enabled. Randomization will be turned off.\")\r\n self.randomize = False\r\n elif _saliency_enabled:\r\n self.saliency = cv2.saliency.StaticSaliencyFineGrained_create()\r\n\r\n def process_dest_img(self, dest_img: np.ndarray, file=None):\r\n if self.saliency or self.transparent:\r\n dest_img = cv2.resize(dest_img, self.target_sz)\r\n if self.saliency:\r\n dest_img = strip_alpha(dest_img)\r\n _, thresh_map = self.saliency.computeSaliency(dest_img)\r\n else:\r\n dest_img, thresh_map = thresh_map_transp(dest_img)\r\n ridx, cidx, thresh_map = compute_block_map(thresh_map, self.block_width, self.block_height, self.lower_thresh)\r\n dest_img = self.dest_to_flat_blocks_mask(dest_img, ridx, cidx)\r\n else:\r\n # strip alpha in case a transparent image is passed in but --transparent flag is not enabled\r\n dest_img = self.dest_to_flat_blocks(strip_alpha(dest_img))\r\n\r\n total = dest_img.shape[0]\r\n assignment = cp.empty(total, dtype=cp.int32)\r\n if self.dither:\r\n dest_img.shape = (*self.grid[::-1], -1)\r\n grid_assignment = assignment.reshape(self.grid[::-1])\r\n coeffs = cp.array([0.4375, 0.1875, 0.3125, 0.0625])[..., cp.newaxis]\r\n\r\n pbar = tqdm(desc=\"[Computing assignments]\", total=total, ncols=pbar_ncols, file=file)\r\n i = 0\r\n row_stride = self.row_stride\r\n if self.freq_mul > 0:\r\n _indices = np.arange(0, total, dtype=np.int32)\r\n if self.randomize:\r\n np.random.shuffle(_indices)\r\n dest_img = dest_img[_indices] # reorder the rows of dest img\r\n indices_freq = self.indices_freq\r\n indices_freq.fill(0.0)\r\n freq_mul = self.freq_mul\r\n if self.dither:\r\n for i in range(0, dest_img.shape[0] - 1):\r\n j = 0\r\n dist = self.cdist(dest_img[i, j:j+1])[0]\r\n dist[cp.argsort(dist)] = self.temp\r\n dist += indices_freq\r\n best_i = grid_assignment[i, j] = cp.argmin(dist)\r\n indices_freq[best_i] += freq_mul\r\n pbar.update()\r\n\r\n for j in range(1, dest_img.shape[1] - 1):\r\n block = dest_img[i, j]\r\n dist = self.cdist(block[cp.newaxis])[0]\r\n dist[cp.argsort(dist)] = self.temp\r\n dist += indices_freq\r\n best_i = grid_assignment[i, j] = cp.argmin(dist)\r\n indices_freq[best_i] += freq_mul\r\n\r\n quant_error = (block - self.img_keys[best_i])[cp.newaxis, ...] * coeffs\r\n dest_img[i, j + 1] += quant_error[0]\r\n dest_img[i + 1, j - 1:j + 2] += quant_error[1:]\r\n\r\n pbar.update()\r\n\r\n j += 1\r\n dist = self.cdist(dest_img[i, j:j+1])[0]\r\n dist[cp.argsort(dist)] = self.temp\r\n dist += indices_freq\r\n best_i = grid_assignment[i, j] = cp.argmin(dist)\r\n indices_freq[best_i] += freq_mul\r\n pbar.update()\r\n\r\n # last row\r\n dist_mat = self.cdist(dest_img[-1])\r\n dist_mat[cp.arange(0, dest_img.shape[1], dtype=cp.int32)[:, cp.newaxis], cp.argsort(dist_mat, axis=1)] = self.temp\r\n for j in range(0, dest_img.shape[1]):\r\n row = dist_mat[j, :]\r\n row += indices_freq\r\n idx = cp.argmin(row)\r\n grid_assignment[-1, j] = idx\r\n indices_freq[idx] += freq_mul\r\n pbar.update()\r\n else:\r\n while i < total - row_stride:\r\n dist_mat = self.cdist(dest_img[i:i+row_stride])\r\n dist_mat[self.row_range, cp.argsort(dist_mat, axis=1)] = self.temp\r\n j = 0\r\n while j < row_stride:\r\n row = dist_mat[j, :]\r\n row += indices_freq\r\n idx = cp.argmin(row)\r\n assignment[i] = idx\r\n indices_freq[idx] += freq_mul\r\n i += 1\r\n j += 1\r\n pbar.update()\r\n if i < total:\r\n dist_mat = self.cdist(dest_img[i:])\r\n dist_mat[self.row_range[:total - i], cp.argsort(dist_mat, axis=1)] = self.temp\r\n j = 0\r\n while i < total:\r\n row = dist_mat[j, :]\r\n row += indices_freq\r\n idx = cp.argmin(row)\r\n assignment[i] = idx\r\n indices_freq[idx] += freq_mul\r\n i += 1\r\n j += 1\r\n pbar.update()\r\n assignment[_indices] = assignment.copy()\r\n else:\r\n if self.dither:\r\n for i in range(0, dest_img.shape[0] - 1):\r\n grid_assignment[i, 0] = cp.argmin(self.cdist(dest_img[i, 0:1])[0])\r\n pbar.update()\r\n for j in range(1, dest_img.shape[1] - 1):\r\n block = dest_img[i, j]\r\n dist_mat = self.cdist(block[cp.newaxis])\r\n best_i = cp.argmin(dist_mat[0])\r\n grid_assignment[i, j] = best_i\r\n quant_error = (block - self.img_keys[best_i])[cp.newaxis, ...] * coeffs\r\n dest_img[i, j + 1] += quant_error[0]\r\n dest_img[i + 1, j - 1:j + 2] += quant_error[1:]\r\n pbar.update()\r\n grid_assignment[i, -1] = cp.argmin(self.cdist(dest_img[i, -1:])[0])\r\n pbar.update()\r\n # last row\r\n cp.argmin(self.cdist(dest_img[-1]), axis=1, out=grid_assignment[-1])\r\n pbar.update(dest_img.shape[1])\r\n else:\r\n while i < total - row_stride:\r\n next_i = i + row_stride\r\n dist_mat = self.cdist(dest_img[i:next_i])\r\n cp.argmin(dist_mat, axis=1, out=assignment[i:next_i])\r\n pbar.update(row_stride)\r\n i = next_i\r\n if i < total:\r\n dist_mat = self.cdist(dest_img[i:])\r\n cp.argmin(dist_mat, axis=1, out=assignment[i:])\r\n pbar.update(total - i)\r\n pbar.close()\r\n\r\n assignment = to_cpu(assignment)\r\n if self.saliency or self.transparent:\r\n return self.make_photomosaic_mask(assignment, ridx, cidx, file=file)\r\n return self.make_photomosaic(assignment, file=file)\r\n\r\n\r\ndef imwrite(filename: str, img: np.ndarray) -> None:\r\n ext = os.path.splitext(filename)[1]\r\n result, n = cv2.imencode(ext, img)\r\n assert result, \"Error saving the collage\"\r\n n.tofile(filename)\r\n\r\n\r\ndef save_img(img: np.ndarray, path: str, suffix: str, file=None) -> None:\r\n if len(path) == 0:\r\n path = \"result.png\"\r\n\r\n if len(suffix) == 0:\r\n print(\"Saving to\", path, file=file)\r\n imwrite(path, img)\r\n else:\r\n file_path, ext = os.path.splitext(path)\r\n path = f'{file_path}{suffix}{ext}'\r\n print(\"Saving to\", path, file=file)\r\n imwrite(path, img)\r\n\r\n\r\ndef get_size(img):\r\n try:\r\n w, h = imagesize.get(img)\r\n return int(w), int(h)\r\n except:\r\n return 0, 0\r\n\r\n\r\ndef get_size_slow(filename: str):\r\n img = imread(filename)\r\n if img is None:\r\n return 0, 0\r\n return img.shape[1::-1]\r\n\r\n\r\ndef infer_size(pool, files: List[str], infer_func: Callable[[str], Tuple[int, int]], i_type: str):\r\n sizes = defaultdict(int)\r\n for w, h in tqdm(pool.imap_unordered(infer_func, files, chunksize=64), \r\n total=len(files), desc=f\"[Inferring size ({i_type})]\", ncols=pbar_ncols):\r\n if w == 0 or h == 0: # skip zero size images\r\n continue\r\n sizes[Fraction(w, h)] += 1\r\n sizes = [(args[1], args[0].numerator / args[0].denominator) for args in sizes.items()]\r\n sizes.sort()\r\n return sizes\r\n\r\n\r\ndef read_images(pic_path: str, img_size: List[int], recursive, pool, flag=\"stretch\", auto_rotate=0) -> ImgList:\r\n assert os.path.isdir(pic_path), \"Directory \" + pic_path + \"is non-existent\"\r\n files = []\r\n print(\"Scanning files...\")\r\n for root, _, file_list in os.walk(pic_path):\r\n for f in file_list:\r\n files.append(os.path.join(root, f))\r\n if not recursive:\r\n break\r\n\r\n if len(img_size) == 1:\r\n sizes = infer_size(pool, files, get_size, \"fast\")\r\n if len(sizes) == 0:\r\n print(\"Warning: unable to infer image size through metadata. Will try reading the entire image (slow!)\")\r\n sizes = infer_size(pool, files, get_size_slow, \"slow\")\r\n assert len(sizes) > 0, \"Fail to infer size. All of your images are in an unsupported format!\"\r\n\r\n # print(\"Aspect ratio (width / height, sorted by frequency) statistics:\")\r\n # for freq, ratio in sizes:\r\n # print(f\"{ratio:6.4f}: {freq}\")\r\n\r\n most_freq_ratio = 1 / sizes[-1][1]\r\n img_size = (img_size[0], round(img_size[0] * most_freq_ratio))\r\n print(\"Inferred tile size:\", img_size)\r\n else:\r\n assert len(img_size) == 2\r\n img_size = (img_size[0], img_size[1])\r\n\r\n read_img = read_img_other\r\n if flag == \"center\":\r\n read_img = read_img_center\r\n if flag == \"fit\":\r\n read_img = read_img_fit\r\n result = [\r\n r for r in tqdm(\r\n pool.imap_unordered(\r\n read_img, \r\n zip(files, itertools.repeat(img_size, len(files)), itertools.repeat(auto_rotate, len(files))), \r\n chunksize=32), \r\n total=len(files), desc=\"[Reading files]\", unit=\"file\", ncols=pbar_ncols) \r\n if r is not None\r\n ]\r\n print(f\"Read {len(result)} images. {len(files) - len(result)} files cannot be decoded as images.\")\r\n return result\r\n\r\n\r\ndef imread(filename: str, flag=cv2.IMREAD_COLOR) -> np.ndarray:\r\n \"\"\"\r\n like cv2.imread, but can read images whose path contain unicode characters\r\n \"\"\"\r\n try:\r\n f = np.fromfile(filename, np.uint8)\r\n if not f.size:\r\n return None\r\n return cv2.imdecode(f, flag)\r\n except:\r\n return None\r\n\r\n\r\ndef read_img_center(args: Tuple[str, Tuple[int, int], int]):\r\n # crop the largest rectangle from the center\r\n img_file, img_size, rot = args\r\n img = imread(img_file)\r\n if img is None:\r\n return None\r\n \r\n ratio = img_size[0] / img_size[1]\r\n # rotate the image if possible to preserve more area\r\n h, w, _ = img.shape\r\n if rot != 0 and abs(h / w - ratio) < abs(w / h - ratio):\r\n img = np.rot90(img, k=rot)\r\n w, h = h, w\r\n \r\n cw = round(h * ratio) # cropped width\r\n ch = round(w / ratio) # cropped height\r\n assert cw <= w or ch <= h\r\n cond = cw > w or (ch <= h and (w - cw) * h > (h - ch) * w)\r\n if cond:\r\n img = img.transpose((1, 0, 2))\r\n w, h = h, w\r\n cw = ch\r\n \r\n margin = (w - cw) // 2\r\n add = (w - cw) % 2\r\n img = img[:, margin:w - margin + add, :]\r\n if cond:\r\n img = img.transpose((1, 0, 2))\r\n return InfoArray(cv2.resize(img, img_size), img_file)\r\n\r\n\r\ndef read_img_other(args: Tuple[str, Tuple[int, int], int]):\r\n img_file, img_size, rot = args\r\n img = imread(img_file)\r\n if img is None:\r\n return img\r\n \r\n if rot != 0:\r\n ratio = img_size[0] / img_size[1]\r\n h, w, _ = img.shape\r\n if abs(h / w - ratio) < abs(w / h - ratio):\r\n img = np.rot90(img, k=rot)\r\n return InfoArray(cv2.resize(img, img_size), img_file)\r\n\r\ndef resizeAndPad(img, size, padColor=1.0):\r\n\r\n h, w = img.shape[:2]\r\n sw, sh = size\r\n\r\n # aspect ratio of image\r\n aspect = w / h \r\n saspect = sw / sh\r\n\r\n if (saspect > aspect): # new horizontal image\r\n new_h = sh\r\n new_w = round(new_h * aspect)\r\n pad_horz = (sw - new_w) / 2\r\n pad_left, pad_right = math.floor(pad_horz), math.ceil(pad_horz)\r\n pad_top, pad_bot = 0, 0\r\n elif (saspect < aspect): # new vertical image\r\n new_w = sw\r\n new_h = round(new_w / aspect)\r\n pad_vert = (sh - new_h) / 2\r\n pad_top, pad_bot = math.floor(pad_vert), math.ceil(pad_vert)\r\n pad_left, pad_right = 0, 0\r\n else:\r\n return cv2.resize(img, (sw, sh))\r\n\r\n # set pad color\r\n if len(img.shape) == 3 and not isinstance(padColor, (list, tuple, np.ndarray)): # color image but only one color provided\r\n padColor = [padColor]*3\r\n\r\n # scale and pad\r\n scaled_img = cv2.resize(img, (new_w, new_h))\r\n scaled_img = cv2.copyMakeBorder(scaled_img, pad_top, pad_bot, pad_left, pad_right, borderType=cv2.BORDER_CONSTANT, value=padColor)\r\n\r\n return scaled_img\r\n\r\ndef read_img_fit(args: Tuple[str, Tuple[int, int], int]):\r\n img_file, img_size, rot = args\r\n img = imread(img_file)\r\n if img is None:\r\n return img\r\n \r\n if rot != 0:\r\n ratio = img_size[0] / img_size[1]\r\n h, w, _ = img.shape\r\n if abs(h / w - ratio) < abs(w / h - ratio):\r\n img = np.rot90(img, k=rot)\r\n return InfoArray(resizeAndPad(img, img_size), img_file)\r\n\r\n\r\n# pickleable helper classes for unfair exp\r\nclass _HelperChangeFreq:\r\n def __init__(self, dest_img: np.ndarray, mos: MosaicUnfair) -> None:\r\n self.mos = mos\r\n self.dest_img = dest_img\r\n\r\n def __call__(self, freq) -> Any:\r\n self.mos.freq_mul = freq\r\n return self.mos.process_dest_img(self.dest_img)\r\n\r\nclass _HelperChangeColorspace:\r\n def __init__(self, dest_img, *args) -> None:\r\n self.dest_img = dest_img\r\n self.args = list(args)\r\n\r\n def __call__(self, colorspace) -> Any:\r\n self.args[3] = colorspace\r\n return MosaicUnfair(*self.args).process_dest_img(self.dest_img)\r\n\r\ndef unfair_exp(dest_img: np.ndarray, args, imgs):\r\n import matplotlib.pyplot as plt\r\n all_colorspaces = PARAMS.colorspace.choices\r\n all_freqs = np.zeros(6, dtype=np.float64)\r\n all_freqs[1:] = np.logspace(-2, 2, 5)\r\n\r\n pbar = tqdm(desc=\"[Experimenting]\", total=len(all_freqs) + len(all_colorspaces) + 1, unit=\"exps\")\r\n mos_bgr = MosaicUnfair(dest_img.shape, imgs, args.max_width, \"bgr\", args.metric, None, None, 1.0, not args.deterministic)\r\n mos_fair = MosaicFair(dest_img.shape, imgs, colorspace=\"bgr\", grid=mos_bgr.grid)\r\n change_cp = _HelperChangeColorspace(dest_img, dest_img.shape, imgs, args.max_width, None, args.metric, None, None, 1.0, not args.deterministic)\r\n change_freq = _HelperChangeFreq(dest_img, mos_bgr)\r\n\r\n with mp.Pool(4) as pool:\r\n futures1 = [pool.apply_async(change_cp, (colorspace,)) for colorspace in all_colorspaces]\r\n futures2 = [pool.apply_async(change_freq, (freq,)) for freq in all_freqs]\r\n futures2.append(pool.apply_async(mos_fair.process_dest_img, (dest_img,)))\r\n \r\n def collect_imgs(fname, params, futures, fs):\r\n result_imgs = []\r\n for i in range(len(params)):\r\n result_imgs.append(futures[i].get()[0])\r\n pbar.update()\r\n \r\n plt.figure(figsize=(len(params) * 10, 12))\r\n plt.imshow(cv2.cvtColor(np.hstack(result_imgs), cv2.COLOR_BGR2RGB))\r\n grid_width = result_imgs[0].shape[1]\r\n plt.xticks(np.arange(0, grid_width * len(result_imgs), grid_width) + grid_width / 2, params, fontsize=fs)\r\n plt.yticks([], [])\r\n plt.subplots_adjust(left=0.005, right=0.995)\r\n plt.savefig(f\"{fname}.png\", dpi=100)\r\n # plt.xlabel(xlabel)\r\n\r\n collect_imgs(\"colorspace\", [c.upper() for c in all_colorspaces], futures1, 36)\r\n collect_imgs(\"fairness\", [f\"Frequency multiplier ($\\lambda$) = ${c}$\" for c in all_freqs] + [\"Fair\"], futures2, 20)\r\n pbar.refresh()\r\n # plt.show()\r\n\r\n\r\ndef sort_exp(pool, args, imgs):\r\n n = len(PARAMS.sort.choices)\r\n for sort_method, (grid, sorted_imgs) in zip(\r\n PARAMS.sort.choices, pool.starmap(sort_collage, \r\n zip(itertools.repeat(imgs, n), \r\n itertools.repeat(args.ratio, n), \r\n PARAMS.sort.choices, \r\n itertools.repeat(args.rev_sort, n))\r\n )):\r\n save_img(make_collage(grid, sorted_imgs, args.rev_row)[0], args.out, sort_method)\r\n\r\n\r\ndef frame_generator(ret, frame, dest_video, skip_frame):\r\n i = 0\r\n while ret:\r\n if i % skip_frame == 0:\r\n yield frame\r\n ret, frame = dest_video.read()\r\n i += 1\r\n\r\n\r\nBlendFunc = Callable[[np.ndarray, np.ndarray, int], np.ndarray]\r\n\r\n\r\ndef process_frame(frame: np.ndarray, mos: MosaicUnfair, blend_func: BlendFunc, blending_level: float, file=None):\r\n collage = mos.process_dest_img(frame, file=file)[0]\r\n if blending_level > 0.0:\r\n collage = blend_func(collage, frame, 1.0 - blending_level)\r\n return collage\r\n\r\n\r\ndef frame_process(mos: MosaicUnfair, blend_func: BlendFunc, blending_level: float, path: str, in_q: mp.Queue, out_q: mp.Queue):\r\n \"\"\"\r\n Worker function that receives a frame from in_q, compute photomosaic and put it in out_q\r\n \"\"\"\r\n with open(os.devnull, \"w\") as null:\r\n while True:\r\n i, frame = in_q.get()\r\n if i is None:\r\n break\r\n out = process_frame(frame, mos, blend_func, blending_level, file=null)\r\n save_img(out, path, f\".{i}\", file=null)\r\n out_q.put(i)\r\n\r\n\r\ndef enable_gpu(show_warning=True):\r\n global cupy_available, cp, fast_sq_euclidean, fast_cityblock, fast_chebyshev\r\n try:\r\n import cupy as cp\r\n cupy_available = True\r\n\r\n @cp.fuse\r\n def fast_sq_euclidean(Asq, Bsq, AB):\r\n return Asq + Bsq - 2*AB\r\n\r\n fast_cityblock = cp.ReductionKernel(\r\n 'T x, T y', # input params\r\n 'T z', # output params\r\n 'abs(x - y)', # map\r\n 'a + b', # reduce\r\n 'z = a', # post-reduction map\r\n '0', # identity value\r\n 'fast_cityblock' # kernel name\r\n )\r\n\r\n fast_chebyshev = cp.ReductionKernel(\r\n 'T x, T y', # input params\r\n 'T z', # output params\r\n 'abs(x - y)', # map\r\n 'max(a, b)', # reduce\r\n 'z = a', # post-reduction map\r\n '0', # identity value\r\n 'fast_chebyshev' # kernel name\r\n )\r\n\r\n except ImportError:\r\n if show_warning:\r\n print(\"Warning: GPU acceleration enabled with --gpu but cupy cannot be imported. Make sure that you have cupy properly installed. \")\r\n\r\n\r\ndef check_dup_valid(dup):\r\n assert dup > 0, \"dup must be a positive integer or a real number between 0 and 1\"\r\n return dup\r\n\r\n\r\ndef main(args):\r\n global LIMIT\r\n num_process = max(1, args.num_process)\r\n if args.video and not args.gpu:\r\n LIMIT = (args.mem_limit // num_process) * 2**20\r\n else:\r\n LIMIT = args.mem_limit * 2**20\r\n\r\n if len(args.out) > 0:\r\n folder, file_name = os.path.split(args.out)\r\n if len(folder) > 0:\r\n assert os.path.isdir(folder), \"The output path {} does not exist!\".format(folder)\r\n ext = os.path.splitext(file_name)[-1].lower()\r\n assert ext == \".jpg\" or ext == \".png\" or ext == \".jpeg\", \"The file extension must be .jpg, .jpeg or .png\"\r\n if args.quiet:\r\n sys.stdout = open(os.devnull, \"w\")\r\n \r\n dup = check_dup_valid(args.dup)\r\n if len(args.dest_img) > 0:\r\n assert os.path.isfile(args.dest_img), f\"Non existent destination image {args.dest_img}\" # early check\r\n \r\n with mp.Pool(max(1, num_process)) as pool:\r\n imgs = read_images(args.path, args.size, args.recursive, pool, args.resize_opt, args.auto_rotate)\r\n \r\n if len(args.dest_img) == 0: # sort mode\r\n if args.exp:\r\n sort_exp(pool, args, imgs)\r\n else:\r\n collage, tile_info = make_collage(*sort_collage(imgs, args.ratio, args.sort, args.rev_sort), args.rev_row)\r\n save_img(collage, args.out, \"\")\r\n if args.tile_info_out:\r\n with open(args.tile_info_out, \"w\", encoding=\"utf-8\") as f:\r\n f.write(tile_info)\r\n return\r\n\r\n if args.video:\r\n assert not (args.salient and not args.unfair), \"Sorry, making photomosaic video is unsupported with fair and salient option. \"\r\n assert args.skip_frame >= 1, \"skip frame must be at least 1\"\r\n\r\n # total_frames = count_frames(args.dest_img, args.skip_frame)\r\n dest_video = cv2.VideoCapture(args.dest_img)\r\n ret, frame = dest_video.read()\r\n assert ret, f\"unable to open video {args.dest_img}\"\r\n dest_shape = frame.shape\r\n else:\r\n dest_img = imread(args.dest_img, cv2.IMREAD_UNCHANGED)\r\n if args.transparent:\r\n assert dest_img.shape[2] == 4, \"--transparent flag can only be used for images with transparent background!\"\r\n if dest_img.shape[2] == 4 and not args.transparent:\r\n print(\"Note: alpha channel detected. If you like to perform transparency masking, add the --transparent flag.\")\r\n dest_shape = dest_img.shape\r\n \r\n if args.gpu:\r\n enable_gpu()\r\n\r\n if args.exp:\r\n assert not args.salient\r\n assert args.unfair\r\n unfair_exp(dest_img, args, imgs) \r\n return\r\n \r\n if args.salient or args.transparent:\r\n lower_thresh = None if args.transparent else args.lower_thresh\r\n if args.unfair:\r\n mos = MosaicUnfair(\r\n dest_shape, imgs, args.max_width, args.colorspace, args.metric,\r\n lower_thresh, args.freq_mul, not args.deterministic, args.dither, args.transparent)\r\n else:\r\n mos = MosaicFairSalient(dest_shape, imgs, dup, args.colorspace, args.metric, lower_thresh, args.transparent)\r\n else:\r\n if args.unfair:\r\n mos = MosaicUnfair(\r\n dest_shape, imgs, args.max_width, args.colorspace, args.metric, \r\n None, args.freq_mul, not args.deterministic, args.dither)\r\n else:\r\n mos = MosaicFair(dest_shape, imgs, dup, args.colorspace, args.metric)\r\n \r\n if args.blending == \"alpha\":\r\n blend_func = alpha_blend\r\n else:\r\n blend_func = brightness_blend\r\n\r\n if args.video:\r\n th, tw, _ = mos.imgs[0].shape\r\n res = (tw * mos.grid[0], th * mos.grid[1])\r\n print(\"Photomosaic video resolution:\", res)\r\n frames_gen = frame_generator(ret, frame, dest_video, args.skip_frame)\r\n if args.gpu:\r\n with open(os.devnull, \"w\") as null:\r\n for idx, frame in tqdm(enumerate(frames_gen), desc=\"[Computing frames]\", unit=\"frame\"):\r\n out = process_frame(frame, mos, blend_func, args.blending_level, null)\r\n save_img(out, args.out, f\"_{idx}\", file=null)\r\n else:\r\n in_q = mp.Queue()\r\n out_q = mp.Queue()\r\n processes = []\r\n for i in range(num_process):\r\n p = mp.Process(target=frame_process, args=(mos, blend_func, args.blending_level, args.out, in_q, out_q)) \r\n p.start()\r\n processes.append(p)\r\n\r\n with tqdm(desc=\"[Computing frames]\", unit=\"frame\") as pbar:\r\n for i, frame in enumerate(frames_gen):\r\n in_q.put((i, frame))\r\n if not out_q.empty():\r\n out_q.get()\r\n pbar.update()\r\n\r\n while pbar.n <= i:\r\n out_q.get()\r\n pbar.update()\r\n\r\n for p in processes:\r\n in_q.put((None, None))\r\n\r\n for p in processes:\r\n p.join()\r\n \r\n frames_gen.close()\r\n else:\r\n collage, tile_info = mos.process_dest_img(dest_img)\r\n collage = blend_func(collage, dest_img, 1.0 - args.blending_level)\r\n save_img(collage, args.out, \"\")\r\n if args.tile_info_out:\r\n with open(args.tile_info_out, \"w\", encoding=\"utf-8\") as f:\r\n f.write(tile_info)\r\n\r\n pool.close()\r\n\r\nif __name__ == \"__main__\":\r\n mp.freeze_support()\r\n parser = argparse.ArgumentParser(\r\n formatter_class=argparse.ArgumentDefaultsHelpFormatter\r\n )\r\n for arg_name, data in PARAMS.__dict__.items():\r\n if arg_name.startswith(\"__\"):\r\n continue\r\n \r\n arg_name = \"--\" + arg_name\r\n if data.type == bool:\r\n assert data.default == False\r\n parser.add_argument(arg_name, action=\"store_true\", help=data.help)\r\n continue\r\n \r\n parser.add_argument(arg_name, type=data.type, default=data.default, help=data.help, choices=data.choices, nargs=data.nargs)\r\n parser.add_argument(\"--exp\", action=\"store_true\", help=\"Do experiments (for testing only)\")\r\n main(parser.parse_args())\r\n" }, { "path": "requirements.txt", "content": "opencv-contrib-python==4.5.5.62\nlapjv==1.3.1\npyinstaller==4.8\nimagesize==1.3.0\ntqdm\nwheel" } ]