Repository: hanzhi713/image-collage-maker Branch: master Commit: fbcc26ba9d8c Files: 13 Total size: 131.0 KB Directory structure: gitextract_g30nn7pr/ ├── .github/ │ └── workflows/ │ └── build_executable.yaml ├── .gitignore ├── LICENSE ├── README.md ├── build_scripts/ │ ├── README.md │ ├── build.sh │ └── install_dependencies.sh ├── examples.sh ├── extract_img.py ├── gui.py ├── io_utils.py ├── make_img.py └── requirements.txt ================================================ FILE CONTENTS ================================================ ================================================ FILE: .github/workflows/build_executable.yaml ================================================ name: Build executables on: [push] env: VERSION: "5.3" jobs: build-matrix: strategy: matrix: os: [ubuntu-18.04, ubuntu-20.04, ubuntu-22.04, windows-2019, macos-10.15, macos-11, macos-12] include: - os: windows-2019 suffix: .exe runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v2 - uses: actions/setup-python@v2 with: python-version: '3.9' - name: Build run: | . ./build_scripts/install_dependencies.sh ${{ matrix.os }} . ./build_scripts/build.sh ${{ matrix.os }} shell: bash - name: Release uses: softprops/action-gh-release@v1 # if: startsWith(github.ref, 'refs/tags/') with: draft: true files: | # ./photomosaic-maker-${{ env.VERSION }}-${{ matrix.version }}-x64-build.tar.gz ./dist/photomosaic-maker-${{ env.VERSION }}-${{ matrix.os }}-x64${{ matrix.suffix }} ./dist/photomosaic-maker-${{ env.VERSION }}-${{ matrix.os }}-x64.tar.gz ================================================ FILE: .gitignore ================================================ .vscode itchat.pkl img/ tests/ build/ *.spec __pycache__/ *test*.sh .mypy_cache/ result.png .pylintrc test.bat dist venv ================================================ FILE: LICENSE ================================================ MIT License Copyright (c) 2017 -T.K.- hanzhi713 -kaiyingshan Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: README.md ================================================
Tiles Photomosaic (Fair tile usage)
Tiles Sorted by RGB sum Photomosaic (Best-fit)
# Photomosaic Maker ![gui demo](./examples/gui.png) - [Distinguishing Features of this Photomosaic Maker](#distinguishing-features-of-this-photomosaic-maker) - [Getting Started](#getting-started) - [Using the pre-built binary](#using-the-pre-built-binary) - [Running Python script directly](#running-python-script-directly) - [Command line usage](#command-line-usage) - [Option 1: Sorting](#option-1-sorting) - [Option 2: Make a photomosaic](#option-2-make-a-photomosaic) - [Option 2.1: Give a fair chance to each tile](#option-21-give-a-fair-chance-to-each-tile) - [Option 2.2: Best fit (unfair tile usage)](#option-22-best-fit-unfair-tile-usage) - [Display salient object only](#display-salient-object-only) - [Keep transparency](#keep-transparency) - [Blending Options](#blending-options) - [Dithering](#dithering) - [Option 3: Photomosaic Video](#option-3-photomosaic-video) - [Performance, multiprocessing and GPU acceleration](#performance-multiprocessing-and-gpu-acceleration) - [Time and space complexity](#time-and-space-complexity) - [Multiprocessing](#multiprocessing) - [GPU acceleration](#gpu-acceleration) - [All command line options](#all-command-line-options) - [Utility Script: download profile pictures of your WeChat friends](#utility-script-download-profile-pictures-of-your-wechat-friends) - [Groupchat Members](#groupchat-members) - [All available profile pictures](#all-available-profile-pictures) - [Notes](#notes) - [Credits (Names in alphabetical order)](#credits-names-in-alphabetical-order) ## Distinguishing Features of this Photomosaic Maker A number of photomosaic makers already exist (like [mosaic](https://github.com/codebox/mosaic) and [Photomosaic-generator](https://github.com/uvipen/Photomosaic-generator)), but this photomosaic maker has the following unique features - Can trade off between the fairness of the tiles and quality of the constructed photomosaic - Can ensure each tile is used exactly N times if desired (N is customizable) - Supports non square tile size - Supports photomosaic videos - Supports maintaining transparency - Supports dithering - Supports saliency detection - Has a graphical user interface - Optional GPU acceleration ## Getting Started You can either use our pre-built binaries from [release](https://github.com/hanzhi713/image-collage-maker/releases) or directly run our python script. ### Using the pre-built binary **If you need GPU acceleration or need to make [photomosaic videos](#option-3-photomosaic-video), please refer to the [Command line usage](#command-line-usage)** Binaries can be downloaded from [release](https://github.com/hanzhi713/image-collage-maker/releases). On Windows and MacOS, my program may be blocked because it is not signed (signing costs money!). Don't worry as there is no security risk. On MacOS or Linux, after downloading the binary, you may need to add executing permission. Open your terminal, go to the file's directory and type ```bash chmod +x ./photomosaic-maker-5.1-macos-x64 ``` Then you can run from terminal as ```bash ./photomosaic-maker-5.1-macos-x64 ``` ### Running Python script directly First, you need Python >= 3.7 with pip. You can install dependencies by running ```bash pip install -r requirements.txt ``` Note: if have problems installing the packages due to missing dependencies, you can use this command (on Linux) to install them one by one. ```bash cat requirements.txt | xargs -n 1 pip install ``` If you want GPU acceleration, you need to install cupy. Please consult the [cupy documentation](https://docs.cupy.dev/en/stable/install.html). Then, you can either use the GUI by running or refer to the commandline usage. ```bash python gui.py ``` If you see errors like `No module named '_tkinter'`, you may need to install tkinter in your system like this (command will vary depending on the OS) ```bash sudo apt-get install python3-tk ``` For command line usage and documentation regarding different photmosaic options, please refer to the section below. ## Command line usage > If you do not wish to use the GUI, a command line interface is also available. Make sure that you've installed dependencies in the section above. ### Option 1: Sorting ```bash python make_img.py --path img/zhou --sort bgr_sum --size 50 --out examples/sort-bgr.png ``` `--size` takes one or two arguments. If only one is specified, it is interpreted as the tile width and tile height will be inferred from the aspect ratios of the tiles provided (this corresponds to the `infer height` option in the GUI). If two are specified, they are interpreted as width and height. Use `--ratio w h` to change the aspect ratio, whose default is 16:9. E.g. `--ratio 21 9` specifies the aspect ratio to be 21:9. > Note: when the tiles are a bit short to completely fill the grid, white tiles will be added. Result: ### Option 2: Make a photomosaic To make a photomosaic, specify the path to the destination image using `--dest_img` #### Option 2.1: Give a fair chance to each tile This fitting option ensures that each tile is used for the same amount of times, but is the most computationally and memory intensive option. > a few tiles might be used one more time than others. This may happen when the number of tiles is not an integer multiple of the blocks of the destination image. ```bash python make_img.py --path img/zhou --dest_img examples/dest.jpg --size 25 --dup 6 --out examples/fair-dup-10.png ``` `--dup 6` specifies that each tile needs to be used 6 times (i.e. duplicates the set of tiles 6 times). Increase that number if you don't have enough source tiles or you want a better fitting result. This can be a non integer too. For example, `--dup 0.5` means only 50% of the tiles will be used, and `--dup 2.5` means all tiles on average will be used 2.5 times (half of the tiles will be used 2 times and the other half will be used 3 times). To make sure the computation completes within a reasonable amount of time, it is recommended that you use less than 6000 tiles after duplication. Tile number larger than 6000 will probably takes longer than a minute to compute. Note that this recommended limit does **not** apply for the best fit option (see section below). | Original | Fitting Result | | ------------------------------------------- | -------------------------------------------------- | | | | #### Option 2.2: Best fit (unfair tile usage) This fitting option just selects the best subset of tiles you provided to approximate your destination tiles. Each tile in that subset will be used for an arbitrary number of times. Add `--unfair` flag to enable this option. You can also specify `--max_width` to change the width of the grid. The height will be automatically calculated based on the max_width provided. Generally, a larger grid will give a better result. The default value is 80. ```bash python make_img.py --path img/zhou --dest_img examples/dest.jpg --size 25 --unfair --max_width 56 --out examples/best-fit.png ``` | Original | Fitting Result | | :-----------------------------------------: | :---------------------------------------------: | | | | Optionally, you can specify the `--freq_mul` (frequency multiplier) parameter that trade offs between the fairness of the tiles and quality of the mosaic. ```bash python3 make_img.py --path img --out best-fit.png --dest_img img/1.png --size 25 --unfair --freq_mul 1.0 ``` The larger the `freq_mul`, more tiles will be used to construct the photomosaic, but the quality will deteriorate. The results under different `freq_mul` are shown below. Note that if you need a large `freq_mul`, you will better off by going for the fair tile usage (see section above) instead. ![](examples/fairness.png) #### Display salient object only This option makes photomosaic only for the salient part of the destination image. Rest of the area will be transparent. Add `--salient` flag to enable this option. You can still specify whether each tile is used for the same amount of times with the `--unfair` flag. Use `--lower_thresh` to specify the threshold for object detection. The threshold ranges from 0.0 to 1.0; a higher threshold would lead to less object area. The default threshold is 0.5. ```bash python make_img.py --path img/zhou --dest_img examples/messi.jpg --size 25 --salient --lower_thresh 0.15 --dup 5 --out examples/messi-fair.png ``` ```bash python make_img.py --path img/zhou --dest_img examples/messi.jpg --size 25 --salient --lower_thresh 0.15 --unfair --max_width 115 --out examples/messi-unfair.png ``` | Original | Unfair-Fitting Result | Fair-Fitting Result | | -------------------------------------------- | --------------------------------------------------- | ------------------------------------------------- | | | | | #### Keep transparency If your destination image has transparent regions, you can add `--transparent` flag to only put tiles for the non transparent part. In this way, the transparent regions are maintained in the resulting photomosaic. Note that this option is not compatible with `--dithering` and `--salient`. | Original | Unfair-Fitting Result (`freq_mul=0.5`) | Fair-Fitting Result | | -------------------------------------------------- | --------------------------------------------------------- | -------------------------------------------------- | | | | | #### Blending Options To enhance the effect of the photomosaic, you can enable alpha or brightness blending. Use the `--blending` option to select the types of blending and `--blending_level` to change the level of blending. ```bash # alpha blending python make_img.py --path img/zhou --dest_img examples/dest.jpg --size 25 --dup 8 --blending alpha --blending_level 0.25 --out examples/blend-alpha-0.25.png # brightness blending python make_img.py --path img/zhou --dest_img examples/dest.jpg --size 25 --dup 8 --blending brightness --blending_level 0.25 --out examples/blend-brightness-0.25.png ``` | Fair tile usage, no blending | Alpha blending (25%) | Brightness blending (25%) | | -------------------------------------------------- | ------------------------------------------------------- | ------------------------------------------------------------ | | | | | #### Dithering > See https://en.wikipedia.org/wiki/Dither for a detailed explanation of dithering Dithering can be used to reduce color banding when there exists a color gradient. To enable dithering, add `--dither` flag. My implementation uses [Floyd–Steinberg dithering](https://en.wikipedia.org/wiki/Floyd%E2%80%93Steinberg_dithering). Note that this option is incompatible with `--transparent` and `--salient`. ```bash python make_img.py --path img/zhou --dest_img examples/dest2.jpg --size 10 --unfair --max_width 200 --freq_mul 0.0 --dither --out examples/dither.png ``` | Original image | Best fit, no dither | Best fit, dither | | -------------------------------------------- | ------------------------------------------------ | --------------------------------------------- | | | | | While dithering works the best when `freq_mul` is set to zero, it can still work and provide some visual differences when `freq_mul > 0`. ```bash # dither when freq_mul is 0.1 python make_img.py --path img/zhou --dest_img examples/dest2.jpg --size 10 --unfair --max_width 200 --freq_mul 0.1 --dither --deterministic --out examples/f-dither.png ``` | Original image | `freq_mul = 0.1`, no dither | `freq_mul = 0.1`, dither | | -------------------------------------------- | -------------------------------------------------- | ----------------------------------------------- | | | | | Note that dithering is **not supported** in fair mode, when randomization is enabled or when saliency is enabled. Also, dithering is not recommended to use with `--gpu`, or you may experience slow computation compared to CPU processing. ### Option 3: Photomosaic Video ![photomosaic-video](examples/v.gif) It is possible to make a photomosaic video simply by repeating the methods listed in Option 2 to every certain frame (specified by `--skip_frame`) of the video. You can pass the path of the video with `--dest_img` and add the `--video` flag to tell the program it is a video. This is much faster than processing the video manually frame by frame (e.g. pass different `dest_img` each time), because a lot of information is cached and can be reused between frames. Example: ```bash python make_img.py --path img/catsdogs --dest_img img/2out.mp4 --size 20 --unfair --max_width 100 --freq_mul 2 --out tests/video/frame.png --gpu --video --skip_frame 2 ``` Since photomosaic version 5.2, the output will be uncompressed individual frames rather than a video. This allows users to use custom video encoding format. Do note that some options are not supported, and some options are slower than other. Generally, saliency is not recommended to use on videos due to its long computational time and difficulty to tune. Each frame might need its own threshold. | Saliency/Fairness | Fair | Unfair | | ----------------- | ----------- | ------ | | saliency enabled | Unsupported | Slow | | saliency disabled | Very slow | Fast | ### Performance, multiprocessing and GPU acceleration #### Time and space complexity Different photomosaic making options have different computational complexity. The following table shows the time and space complexity of different cases. Here, `n` is the number of tiles (after duplication in fair mode), `m` is the number of pixels in the destination image, and `k` is the number of tiles used in the unfair mode (this is equal to your specified `max_width` multiplied by the aspect ratio of your destination image). | Type of photomosaic | Time complexity | Space complexity | GPU acceleration | | -------------------- | ------------------ | ---------------- | ---------------------------------------------- | | Fair | `O(nm + n^3)` | `O(nm)` | partial (~10x speed up for the `nm` part only) | | Unfair, freq_mul > 0 | `O(nm + nk log n)` | `O(m + n)` | full (~5-10x speed up) | | Unfair, freq_mul = 0 | `O(nm + nk)` | `O(m + n)` | full (~5-10x speed up) | Takeaway 1: The high (cubic) computational complexity of the fair mode means that the computation time grows much faster with respect to the number of tiles. It typically takes 30 seconds for 5000 tiles and 5 minutes for 10000 tiles. For large tile count, unless you need strict fair tile usage, you should go for the unfair mode and set freq_mul appropriately. Takeaway 2: Notice the role of `m` in the complexity. If you have a high-definition destination image (e.g. 8000x6000) and notice the computation time is long, you can first downsample it so the number of pixels (`m`) will be lower. Do note that over downsampling will reduce the quality of the photomosaic. #### Multiprocessing The `--num_process` option specifies the number of processes (cpu cores) to use. This defaults to half of you available logical CPUs. However, this only applies to the reading tiles phase and photomosaic video processing. For photomosaic video, if you have a small number of tiles or a large number of available CPU cores, using multiprocessing may be faster than enabling GPU acceleration. #### GPU acceleration For command line, GPU acceleration can be enabled with the `--gpu` flag. For GUI (`gui.py`), it will be automatically enabled if you have `cupy` installed. However, note it can only provide the listed speedup if `nm` **is large**, typically **>= 10^10**. Another way to judge whether GPU acceleration could be useful is observe the `Distance matrix size` message from the log. Typically, `Distance matrix size` **>= 100MB** work great on GPU. ### All command line options ```python make_img.py -h``` will give you all the available command line options. ``` $ python make_img.py --help usage: make_img.py [-h] [--path PATH] [--recursive] [--num_process NUM_PROCESS] [--out OUT] [--size SIZE [SIZE ...]] [--quiet] [--auto_rotate {-1,0,1}] [--resize_opt {center,stretch,fit}] [--gpu] [--mem_limit MEM_LIMIT] [--tile_info_out TILE_INFO_OUT] [--ratio RATIO RATIO] [--sort {none,bgr_sum,av_hue,av_sat,av_lum,rand}] [--rev_row] [--rev_sort] [--dest_img DEST_IMG] [--colorspace {hsv,hsl,bgr,lab,luv}] [--metric {euclidean,cityblock,chebyshev,cosine}] [--transparent] [--unfair] [--max_width MAX_WIDTH] [--freq_mul FREQ_MUL] [--dither] [--deterministic] [--dup DUP] [--salient] [--lower_thresh LOWER_THRESH] [--blending {alpha,brightness}] [--blending_level BLENDING_LEVEL] [--video] [--skip_frame SKIP_FRAME] [--exp] optional arguments: -h, --help show this help message and exit --path PATH Path to the tiles (default: None) --recursive Whether to read the sub-folders for the specified path (default: False) --num_process NUM_PROCESS Number of processes to use for parallelizable operations (default: 8) --out OUT The filename of the output collage/photomosaic (default: result.png) --size SIZE [SIZE ...] Width and height of each tile in pixels in the resulting collage/photomosaic. If two numbers are specified, they are treated as width and height. If one number is specified, the number is treated as the width and the height is inferred from the aspect ratios of the images provided. (default: (50,)) --quiet Do not print progress message to console (default: False) --auto_rotate {-1,0,1} Options to auto rotate tiles to best match the specified tile size. 0: do not auto rotate. 1: attempt to rotate counterclockwise by 90 degrees. -1: attempt to rotate clockwise by 90 degrees (default: 0) --resize_opt {center,stretch,fit} How to resize each tile so they have the desired aspect ratio and size, which can be specified fully or partially by --size. Center: crop the largest rectangle from the center. Stretch: stretch the tile. Fit: pad the tiles with white background (default: center) --gpu Use GPU acceleration. Requires cupy to be installed and a capable GPU. Note that USUALLY this is useful when you: 1. have a lot of tiles (typically > 10000), and2. are using the unfair mode, and3. (for photomosaic videos only) only have few cpu coresAlso note: enabling GPU acceleration will disable multiprocessing on CPU for videos (default: False) --mem_limit MEM_LIMIT The APPROXIMATE memory limit in MB when computing a photomosaic in unfair mode. Applicable both CPU and GPU computing. If you run into memory issues when using GPU, try reduce this memory limit (default: 4096) --tile_info_out TILE_INFO_OUT Path to save the list of tile filenames for the collage/photomosaic. If empty, it will not be saved. (default: ) --ratio RATIO RATIO Aspect ratio of the output image (default: (16, 9)) --sort {none,bgr_sum,av_hue,av_sat,av_lum,rand} Sort method to use (default: bgr_sum) --rev_row Whether to use the S-shaped alignment. (default: False) --rev_sort Sort in the reverse direction. (default: False) --dest_img DEST_IMG The path to the destination image that you want to build a photomosaic for (default: ) --colorspace {hsv,hsl,bgr,lab,luv} The colorspace used to calculate the metric (default: lab) --metric {euclidean,cityblock,chebyshev,cosine} Distance metric used when evaluating the distance between two color vectors (default: euclidean) --transparent Enable transparency masking. The transparent regions of the destination image will be maintained in the photomosaicCannot be used together with --salient (default: False) --unfair Whether to allow each tile to be used different amount of times (unfair tile usage). (default: False) --max_width MAX_WIDTH Maximum width of the collage. This option is only valid if unfair option is enabled (default: 80) --freq_mul FREQ_MUL Frequency multiplier to balance tile fairless and mosaic quality. Minimum: 0. More weight will be put on tile fairness when this number increases. (default: 0.0) --dither Whether to enabled dithering. You must also specify --deterministic if enabled. (default: False) --deterministic Do not randomize the tiles. This option is only valid if unfair option is enabled (default: False) --dup DUP If a positive integer: duplicate the set of tiles by how many times. Can be a fraction (default: 1) --salient Make photomosaic for salient objects only (default: False) --lower_thresh LOWER_THRESH The threshold for saliency detection, between 0.0 (no object area = blank) and 1.0 (maximum object area = original image) (default: 0.5) --blending {alpha,brightness} The types of blending used. alpha: alpha (transparency) blending. Brightness: blending of brightness (lightness) channel in the HSL colorspace (default: alpha) --blending_level BLENDING_LEVEL Level of blending, between 0.0 (no blending) and 1.0 (maximum blending). Default is no blending (default: 0.0) --video Make a photomosaic video from dest_img which is assumed to be a video (default: False) --skip_frame SKIP_FRAME Make a photomosaic every this number of frames (default: 1) --exp Do experiments (for testing only) (default: False) ``` ## Utility Script: download profile pictures of your WeChat friends If you have a WeChat account, an utility script `extract_img.py` is provided to download your friends' profile pictures so you can make a photomosaic using them. To use this script, you need to have itchat-uos installed ```bash pip install itchat-uos ``` Then, use `--dir` to specify the directory to store the profile pictures of your WeChat friends. ```bash python extract_img.py --dir img ``` ### Groupchat Members You can also download the group members' profiles images from a group chat ```bash python extract_img.py --dir img --type groupchat --name "groupchatname" ``` You can download members' profile pictures from all your groupchats if you omit the `--name` argument ```bash python extract_img.py --dir img --type groupchat ``` ### All available profile pictures You can download profile pictures from both your friends and members from all your groupchats by specifying `--type all`. ```bash python extract_img.py --dir img --type all ``` ### Notes 1. Due to unknown issues, sometimes some profile pictures are not available, so they will be blank and unusable. The photomosaic maker will automatically ignore them when loading images. 2. When you download a large amount of profile pictures at once, WeChat may block you from downloading more. This will appear as `timeout downloading pics, retrying.... attempt x` in terminal. When this happens, you can terminate the program and run it again a day after. Already downloaded profile pictures will not be downloaded again. ## Credits (Names in alphabetical order) Hanzhi Zhou ([hanzhi713](https://github.com/hanzhi713/)): Main algorithm and GUI implementation Kaiying Shan ([kaiyingshan](https://github.com/kaiyingshan)): Saliency idea and implementation Xinyue Lin: Idea for the "Best-fit" Yufeng Chi ([T-K](https://github.com/T-K-233/)) : Initial Idea, crawler ================================================ FILE: build_scripts/README.md ================================================ # Build Standalone Executable This folder contains scripts to build standalone executables from the source python files using PyInstaller. [.github/workflows/build_executable.yaml](/.github/workflows/build_executable.yaml) uses these scripts to build standalone executables on GitHub actions. ## How to use locally These scripts can also run locally with a Bash shell. On Windows, Git Bash can be used. Note that these scripts are meant to be run in the project root. First, you need to setup a Python venv and install all the necessary dependencies. This needs to be done only once. ```bash . ./build_scripts/install_dependencies.sh {your-os} ``` Then, you can run the build script to produce an executable for your platform in the dist folder. Make sure that the Python venv is activated before running this script. ```bash . ./build_scripts/build.sh {your-os} ``` `{your-os}` is one of `macos`, `windows` and `linux`. Due to the way PyInstaller works, you can only build executable for your platform. For example, you cannot build macos and linux executable on windows. ================================================ FILE: build_scripts/build.sh ================================================ #!/bin/bash PLATFORM=$1 NAME=photomosaic-maker-${VERSION}-${PLATFORM}-x64 rm -rf dist/* if [[ $PLATFORM == windows* ]]; then ./collage/Scripts/activate SUFFIX=".exe" elif [[ $PLATFORM == macos* ]]; then source collage/bin/activate SUFFIX="" elif [[ $PLATFORM == ubuntu* ]]; then source collage/bin/activate SUFFIX="" else echo "Unsupported platform: " $PLATFORM exit 1 fi ARGS="-y --exclude-module umap --exclude-module matplotlib" pyinstaller $ARGS --name "${NAME}-archive" gui.py pyinstaller $ARGS --onefile --name "$NAME${SUFFIX}" gui.py pushd dist/$NAME-archive tar -czvf ../$NAME.tar.gz . # zip -r ../$NAME.zip . popd pushd build tar -czvf ../$NAME-build.tar.gz . popd ================================================ FILE: build_scripts/install_dependencies.sh ================================================ #/bin/bash python3 -m venv collage PLATFORM=$1 if [[ $PLATFORM == windows* ]]; then ./collage/Scripts/activate elif [[ $PLATFORM == macos* ]]; then source collage/bin/activate elif [[ $PLATFORM == ubuntu* ]]; then source collage/bin/activate else echo "Unsupported platform: " $PLATFORM exit 1 fi python -m pip install pip==21.2.4 cat requirements.txt | xargs -n 1 pip install ================================================ FILE: examples.sh ================================================ #!/bin/bash # This is the script to generate all the examples in the README. # If you want to use this script, modify the command and output directory according to your needs. CMD="python make_img.py --path img/zhou" OUT=examples $CMD --sort none --size 50 --out $OUT/unsorted.png $CMD --sort bgr_sum --size 50 --out $OUT/sort-bgr.png $CMD --dest_img examples/dest.jpg --size 25 --dup 8 --out $OUT/fair-dup-10.png $CMD --dest_img examples/dest.jpg --size 25 --unfair --max_width 56 --out $OUT/best-fit.png $CMD --dest_img examples/messi.jpg --size 25 --salient --lower_thresh 0.15 --dup 5 --out $OUT/messi-fair.png $CMD --dest_img examples/messi.jpg --size 25 --salient --lower_thresh 0.15 --unfair --max_width 115 --out $OUT/messi-unfair.png $CMD --dest_img examples/dest-transp.png --size 25 --transparent --dup 5 --out $OUT/transp-fair.png $CMD --dest_img examples/dest-transp.png --size 25 --transparent --unfair --max_width 56 --out $OUT/transp-unfair.png $CMD --dest_img examples/dest-transp.png --size 25 --transparent --unfair --freq_mul 0.5 --max_width 56 --out $OUT/transp-unfair-freq.png $CMD --dest_img examples/dest.jpg --size 25 --dup 8 --blending alpha --blending_level 0.25 --out $OUT/blend-alpha-0.25.png $CMD --dest_img examples/dest.jpg --size 25 --dup 8 --blending brightness --blending_level 0.25 --out $OUT/blend-brightness-0.25.png $CMD --dest_img examples/dest2.jpg --size 10 --unfair --max_width 200 --freq_mul 0.0 --out $OUT/dither-no.png $CMD --dest_img examples/dest2.jpg --size 10 --unfair --max_width 200 --freq_mul 0.0 --dither --deterministic --out $OUT/dither.png $CMD --dest_img examples/dest2.jpg --size 10 --unfair --max_width 200 --freq_mul 0.1 --out $OUT/f-dither-no.png $CMD --dest_img examples/dest2.jpg --size 10 --unfair --max_width 200 --freq_mul 0.1 --dither --deterministic --out $OUT/f-dither.png # $CMD --sort none --exp --size 50 --out $OUT/sort-exp.png # $CMD --dest_img examples/dest.jpg --size 25 --exp --unfair --max_width 56 ================================================ FILE: extract_img.py ================================================ """ extract_img.py get profile pictures from your WeChat friends or group chat members """ import itchat import os from concurrent.futures import ThreadPoolExecutor, TimeoutError from tqdm import tqdm import argparse import multiprocessing as mp import unicodedata import re import traceback def slugify(value, allow_unicode=True): """ Taken from https://github.com/django/django/blob/master/django/utils/text.py Convert to ASCII if 'allow_unicode' is False. Convert spaces or repeated dashes to single dashes. Remove characters that aren't alphanumerics, underscores, or hyphens. Convert to lowercase. Also strip leading and trailing whitespace, dashes, and underscores. """ value = str(value) if allow_unicode: value = unicodedata.normalize('NFKC', value) else: value = unicodedata.normalize('NFKD', value).encode('ascii', 'ignore').decode('ascii') value = re.sub(r'[^\w\s-]', '', value.lower()) return re.sub(r'[-\s]+', '-', value).strip('-_') def download_pic(args): try: itchat.get_head_img(**args) return os.path.getsize(args['picDir']) except: traceback.print_exc() return 0 def get_chatroom_by_name(name, chatrooms): for chatroom in chatrooms: if chatroom['NickName'] == name: return chatroom if __name__ == "__main__": parser = argparse.ArgumentParser() parser.add_argument("--dir", default="img", type=str, help="Folder to store the downloaded images") parser.add_argument("--type", type=str, choices=["self", "groupchat", "all"], default="self") parser.add_argument("--name", type=str, nargs="+", help="Specify the chatroom name if type=chatroom") parser.add_argument("--list_chatroom", action="store_true") args = parser.parse_args() download_dir = args.dir print("Logging in...") itchat.auto_login(hotReload=True) if args.list_chatroom: print("Getting chatrooms...") chatrooms = itchat.get_chatrooms(update=True) for chatroom in tqdm(chatrooms, desc="[Updating Groupchats]"): tqdm.write(chatroom['NickName']) exit() download_args = dict() if args.type == "self" or args.type == "all": print("Loading contact...") for mem in itchat.get_friends(update=True): download_args[mem['UserName']] = { "userName": mem['UserName'], "picDir": os.path.join(download_dir, slugify(f"{mem['NickName']}_{mem['RemarkName']}") + ".jpg") } if args.type == "groupchat" or args.type == "all": print("Getting groupchats...") chatrooms = itchat.get_chatrooms(update=True) # Nickname PYQuanPin RemarkPYQuanPin if not args.name: print("Updating groupchats... this might take a while (several minutes if you have tens of large group chats)") itchat.update_chatroom([chatroom['UserName'] for chatroom in chatrooms], True) chatrooms = itchat.get_chatrooms() else: filtered_chatrooms = [] for name in args.name: chatroom = get_chatroom_by_name(args.name, chatrooms) assert chatroom is not None, f"Chatroom \"{args.name}\" not found" filtered_chatrooms.append(chatroom) itchat.update_chatroom([chatroom['UserName'] for chatroom in filtered_chatrooms], True) chatrooms = itchat.get_chatrooms() chatrooms = [get_chatroom_by_name(name, chatrooms) for name in args.name] for chatroom in chatrooms: cr_name = chatroom['NickName'] for mem in chatroom['MemberList']: if mem['UserName'] not in download_args: download_args[mem['UserName']] = { "userName": mem['UserName'], 'chatroomUserName': chatroom['UserName'], "picDir": os.path.join(download_dir, slugify(f"{cr_name}_{mem['NickName']}") + ".jpg") } if not os.path.isdir(download_dir): os.mkdir(download_dir) download_args = download_args.values() download_args = [arg for arg in download_args if not os.path.exists(arg['picDir']) or os.path.getsize(arg['picDir']) == 0] pool = ThreadPoolExecutor(min(mp.cpu_count(), len(download_args))) pbar = tqdm(desc="[Downloading]", total=len(download_args)) count = 1 while len(download_args) > 0: download_args = [arg for arg in download_args if not os.path.exists(arg['picDir']) or os.path.getsize(arg['picDir']) == 0] try: for sz in pool.map(download_pic, download_args, timeout=20): if sz > 0: pbar.update() except TimeoutError: pool.shutdown(False) pool = ThreadPoolExecutor(min(mp.cpu_count(), len(download_args))) tqdm.write(f"timeout downloading pics, retrying.... attempt {count}") count += 1 pool.shutdown() ================================================ FILE: gui.py ================================================ import tkinter as tk from tkinter import * from tkinter import filedialog, messagebox, colorchooser from tkinter.ttk import * from concurrent.futures import Future, ThreadPoolExecutor import multiprocessing as mp from multiprocessing import freeze_support, cpu_count import argparse import traceback import math import sys import os import time from typing import Tuple, Union import cv2 import numpy as np import make_img as mkg from io_utils import SafeText def limit_wh(w: int, h: int, max_width: int, max_height: int): if h > max_height: ratio = max_height / h h = max_height w = math.floor(w * ratio) if w > max_width: ratio = max_width / w w = max_width h = math.floor(h * ratio) return w, h """ tk_ToolTip_class101.py gives a Tkinter widget a tooltip as the mouse is above the widget tested with Python27 and Python34 by vegaseat 09sep2014 www.daniweb.com/programming/software-development/code/484591/a-tooltip-class-for-tkinter Modified to include a delay time by Victor Zaccardo, 25mar16 Source: https://stackoverflow.com/questions/3221956/how-do-i-display-tooltips-in-tkinter """ class CreateToolTip(object): """ create a tooltip for a given widget """ def __init__(self, widget, text='widget info'): self.waittime = 500 #miliseconds self.wraplength = 180 #pixels self.widget = widget self.text = text self.widget.bind("", self.enter) self.widget.bind("", self.leave) self.widget.bind("", self.leave) self.id = None self.tw = None def enter(self, event=None): self.schedule() def leave(self, event=None): self.unschedule() self.hidetip() def schedule(self): self.unschedule() self.id = self.widget.after(self.waittime, self.showtip) def unschedule(self): id = self.id self.id = None if id: self.widget.after_cancel(id) def showtip(self, event=None): x = y = 0 x, y, cx, cy = self.widget.bbox("insert") x += self.widget.winfo_rootx() + 25 y += self.widget.winfo_rooty() + 20 # creates a toplevel window self.tw = tk.Toplevel(self.widget) # Leaves only the label and removes the app window self.tw.wm_overrideredirect(True) self.tw.wm_geometry("+%d+%d" % (x, y)) label = tk.Label(self.tw, text=self.text, justify='left', background="#ffffff", relief='solid', borderwidth=1, wraplength = self.wraplength) label.pack(ipadx=1) def hidetip(self): tw = self.tw self.tw= None if tw: tw.destroy() class LabelWithTooltip(Label): def __init__(self, *args, **kwargs) -> None: _tp = kwargs["tooltip"] del kwargs["tooltip"] super().__init__(*args, **kwargs) self._tp = CreateToolTip(self, _tp) class CheckbuttonWithTooltip(Checkbutton): def __init__(self, *args, **kwargs) -> None: _tp = kwargs["tooltip"] del kwargs["tooltip"] super().__init__(*args, **kwargs) self._tp = CreateToolTip(self, _tp) class RadiobuttonWithTooltip(Radiobutton): def __init__(self, *args, **kwargs) -> None: _tp = kwargs["tooltip"] del kwargs["tooltip"] super().__init__(*args, **kwargs) self._tp = CreateToolTip(self, _tp) class Debounce: def __init__(self, action) -> None: self.queue = [] self.action = action self.pool = ThreadPoolExecutor(1) def __call__(self, *args, **kwds): for f in self.queue: f.cancel() self.queue.clear() self.queue.append(self.pool.submit(self.action)) if __name__ == "__main__": freeze_support() pool = ThreadPoolExecutor(1) root = Tk() root.title("Photomosaic maker") parser = argparse.ArgumentParser() parser.add_argument("-D", action="store_true") parser.add_argument("--src", "-s", type=str, default=os.path.join(os.path.dirname(__file__), "img")) parser.add_argument("--collage", "-c", type=str, default=os.path.join(os.path.dirname(__file__), "examples", "dest.png")) cmd_args = parser.parse_args() init_dir = os.path.expanduser("~") # ---------------- initialization ----------------- left_panel = PanedWindow(root) left_panel.grid(row=0, column=0, sticky="NSEW") # left_panel.rowconfigure(0, weight=1) # left_panel.columnconfigure(0, weight=1) Separator(root, orient="vertical").grid( row=0, column=1, sticky="NSEW", padx=(5, 5)) right_panel = PanedWindow(root) right_panel.grid(row=0, column=2, sticky="N") # ---------------- end initialization ----------------- # ---------------- left panel's children ---------------- # left panel ROW 0 canvas = Canvas(left_panel, width=800, height=600) canvas.grid(row=0, column=0) # left panel ROW 1 Separator(left_panel, orient="horizontal").grid(row=1, columnspan=2, sticky="NSEW", pady=(5, 5)) # left panel ROW 2 log_panel = PanedWindow(left_panel) log_panel.grid(row=2, column=0, columnspan=2, sticky="WE") log_panel.grid_columnconfigure(0, weight=1) log_entry = SafeText(log_panel, height=6, bd=0) mkg.pbar_ncols = log_entry.width # log_entry.configure(font=("", 10, "")) log_entry.grid(row=1, column=0, sticky="NSEW") scroll = Scrollbar(log_panel, orient="vertical", command=log_entry.yview) log_entry.config(yscrollcommand=scroll.set) scroll.grid(row=1, column=1, sticky="NSEW") # used store a reference to the displayed image to we can save it result_img = None # ------------------ end left panel ------------------------ def show_img(img: Union[np.ndarray, Future, Tuple[np.ndarray, str], None], printDone: bool = True) -> None: """ display an image in the canvas and set the global variable result_img to img """ global result_img, result_tile_info if img is None: return if type(img) == Future: img = img.result() if type(img) == tuple: img, result_tile_info = img result_img = img width, height = canvas.winfo_width(), canvas.winfo_height() h, w, _ = img.shape img = mkg.strip_alpha(img) if h > height or w > width: print("Resizing image to display it on GUI...") w, h = limit_wh(w, h, width, height) img = cv2.resize(img, (w, h)) _, data = cv2.imencode(".ppm", img) # prevent the image from being garbage-collected root.preview = PhotoImage(data=data.tobytes()) canvas.delete("all") canvas.create_image((width - w) // 2, (height - h) // 2, image=root.preview, anchor=NW) save_button.config(state='enabled') tile_save_button.config(state='disabled' if result_tile_info is None else 'enabled') if printDone: print("Done") # --------------------- right panel's children ------------------------ # right panel ROW 0 file_path = StringVar() file_path.set("N/A") Label(right_panel, text="Path to tiles:").grid(row=0, columnspan=2, sticky="W", pady=(8, 2)) # right panel ROW 1 Label(right_panel, textvariable=file_path, wraplength=150).grid(row=1, columnspan=2, sticky="W") # right panel ROW 2 opt = StringVar() opt.set("sort") tile_size_panel = PanedWindow(right_panel) tile_size_panel.grid(row=2, column=0, columnspan=2, sticky="W") tile_width = IntVar() tile_width.set(50) tile_height = IntVar() tile_height.set(0) LabelWithTooltip(tile_size_panel, text="Tile size: ", tooltip=mkg.PARAMS.size.help).grid(row=0, column=0, sticky="W") Entry(tile_size_panel, width=5, textvariable=tile_width).grid(row=0, column=1, sticky="W") Label(tile_size_panel, text="x", wraplength=150).grid(row=0, column=2, sticky="W") tile_height_entry = Entry(tile_size_panel, width=5, textvariable=tile_height) tile_height_entry.grid(row=0, column=3, sticky="W") infer_height = BooleanVar() infer_height.set(True) def action_infer_height(): if infer_height.get(): tile_height_entry.config(state='disabled') else: tile_height_entry.config(state='enabled') if tile_height.get() == 0: tile_height.set(tile_width.get()) action_infer_height() CheckbuttonWithTooltip(tile_size_panel, text="Infer height", variable=infer_height, command=action_infer_height, tooltip="Infer height from width and the aspect ratios of the images provided").grid(row=1, columnspan=4, sticky="W") auto_rotate = IntVar() auto_rotate.set(0) LabelWithTooltip(tile_size_panel, text="Auto rotate: ", tooltip=mkg.PARAMS.auto_rotate.help).grid( row=2, column=0, columnspan=2, sticky="W") OptionMenu(tile_size_panel, auto_rotate, "", *mkg.PARAMS.auto_rotate.choices).grid( row=2, column=2, columnspan=2, sticky="W") # right panel ROW 3 resize_opt = StringVar() resize_opt.set("center") LabelWithTooltip(right_panel, text="Resize option: ", tooltip=mkg.PARAMS.resize_opt.help).grid( row=3, column=0, sticky="W") OptionMenu(right_panel, resize_opt, "", *mkg.PARAMS.resize_opt.choices).grid( row=3, column=1, sticky="W") # right panel ROW 4 recursive = BooleanVar() recursive.set(True) Checkbutton(right_panel, text="Read sub-folders", variable=recursive).grid(row=4, columnspan=2, sticky="W") imgs = None def load_img_action(): global imgs fp = file_path.get() try: sizes = [tile_width.get()] if not infer_height.get(): sizes.append(tile_height.get()) imgs = mkg.read_images(fp, sizes, recursive.get(), mp.Pool(cpu_count() // 2), resize_opt.get(), auto_rotate.get()) shape = imgs[0].shape if infer_height.get(): tile_height.set(shape[0]) grid = mkg.calc_grid_size(16, 10, len(imgs), shape) return mkg.make_collage(grid, imgs.copy(), False) except AssertionError as e: messagebox.showerror("Error", str(e)) except: messagebox.showerror("Error", traceback.format_exc()) def reload_images(): pool.submit(load_img_action).add_done_callback(show_img) def load_images(): w = tile_width.get() h = tile_height.get() if w < 2: return messagebox.showerror("Illegal Argument", "Tile width must be greater than 2") if not infer_height.get() and h < 2: return messagebox.showerror("Illegal Argument", "Tile height must be greater than 2") fp = filedialog.askdirectory(initialdir=file_path.get() if os.path.isdir(file_path.get()) else init_dir, title="Select folder of tiles") if len(fp) <= 0 or not os.path.isdir(fp): return def callback(f): reload_img_button.config(state='enabled') sort_button.config(state='enabled') collage_button.config(state='enabled') show_img(f) print("Loading tiles from", fp) file_path.set(fp) pool.submit(load_img_action).add_done_callback(callback) # right panel ROW 5 Button(right_panel, text="Load tiles", command=load_images).grid(row=5, column=0, pady=2) reload_img_button = Button(right_panel, text="Reload", command=reload_images) reload_img_button.config(state='disabled') reload_img_button.grid(row=5, column=1, pady=2, padx=(0, 4)) def attach_sort(): right_col_opt_panel.grid_remove() right_sort_opt_panel.grid(row=8, columnspan=2, sticky="W") def attach_collage(): right_sort_opt_panel.grid_remove() right_col_opt_panel.grid(row=8, columnspan=2, sticky="W") # right panel ROW 6 Radiobutton(right_panel, text="Sort", value="sort", variable=opt, state=ACTIVE, command=attach_sort).grid(row=6, column=1, sticky="W") Radiobutton(right_panel, text="Photomosaic", value="collage", variable=opt, command=attach_collage).grid(row=6, column=0, sticky="W") # right panel ROW 7 Separator(right_panel, orient="horizontal").grid( row=7, columnspan=2, pady=(6, 3), sticky="we") # right panel ROW 8: Dynamically attached # right sort option panel !!OR!! right collage option panel right_sort_opt_panel = PanedWindow(right_panel) right_sort_opt_panel.grid(row=8, column=0, columnspan=2, sticky="W") # ------------------------- right sort option panel -------------------------- # right sort option panel ROW 0: sort_method = StringVar() sort_method.set("bgr_sum") LabelWithTooltip(right_sort_opt_panel, text="Sort methods:", tooltip=mkg.PARAMS.sort.help).grid( row=0, column=0, sticky="W") OptionMenu(right_sort_opt_panel, sort_method, "", *mkg.PARAMS.sort.choices).grid(row=0, column=1) # right sort option panel ROW 1: LabelWithTooltip(right_sort_opt_panel, text="Aspect ratio:", tooltip=mkg.PARAMS.ratio.help).grid( row=1, column=0, sticky="W") aspect_ratio_panel = PanedWindow(right_sort_opt_panel) aspect_ratio_panel.grid(row=1, column=1) rw = IntVar() rw.set(16) rh = IntVar() rh.set(10) Entry(aspect_ratio_panel, width=3, textvariable=rw).grid(row=0, column=0) Label(aspect_ratio_panel, text=":").grid(row=0, column=1) Entry(aspect_ratio_panel, width=3, textvariable=rh).grid(row=0, column=2) # right sort option panel ROW 2: rev_row = BooleanVar() rev_row.set(False) rev_sort = BooleanVar() rev_sort.set(False) Checkbutton(right_sort_opt_panel, variable=rev_row, text="Reverse consecutive row").grid(row=2, columnspan=2, sticky="W") # right sort option panel ROW 3: Checkbutton(right_sort_opt_panel, variable=rev_sort, text="Reverse sort order").grid(row=3, columnspan=2, sticky="W") def generate_sorted_image(): if imgs is None: return messagebox.showerror("Empty set", "Please first load tiles") try: w, h = rw.get(), rh.get() assert w > 0, "Width must be greater than 0" assert h > 0, "Height must be greater than 0" except AssertionError as e: return messagebox.showerror("Illegal Argument", str(e)) def action(): try: grid, sorted_imgs = mkg.sort_collage(imgs, (w, h), sort_method.get(), rev_sort.get()) return mkg.make_collage(grid, sorted_imgs, rev_row.get()) except: messagebox.showerror("Error", traceback.format_exc()) pool.submit(action).add_done_callback(show_img) # right sort option panel ROW 4: sort_button = Button(right_sort_opt_panel, text="Generate sorted image", command=generate_sorted_image) sort_button.config(state='disabled') sort_button.grid(row=4, columnspan=2, pady=5) # ------------------------ end right sort option panel ----------------------------- # ------------------------ right collage option panel ------------------------------ # right collage option panel ROW 0: right_col_opt_panel = PanedWindow(right_panel) dest_img_path = StringVar() dest_img_path.set("N/A") dest_img = None Label(right_col_opt_panel, text="Path to the target image: ").grid( row=0, columnspan=2, sticky="W", pady=2) # right collage option panel ROW 1: Label(right_col_opt_panel, textvariable=dest_img_path, wraplength=150).grid(row=1, columnspan=2, sticky="W") def load_dest_img(): global dest_img, result_tile_info if imgs is None: return messagebox.showerror("Empty set", "Please first load tiles") fp = filedialog.askopenfilename( initialdir=os.path.dirname(dest_img_path.get()) if os.path.isdir(os.path.dirname(dest_img_path.get())) else init_dir, title="Select destination image", filetypes=(("images", "*.jpg"), ("images", "*.png"), ("images", "*.gif"), ("all files", "*.*"))) if fp is not None and len(fp) > 0 and os.path.isfile(fp): try: print("Destination image loaded from", fp) dest_img = mkg.imread(fp, cv2.IMREAD_UNCHANGED) dest_img_path.set(fp) result_tile_info = None show_img(dest_img, False) transparent.set(dest_img.shape[2] == 4) is_salient.set(not transparent.get()) except: messagebox.showerror("Error reading file", traceback.format_exc()) # right collage option panel ROW 2: Button(right_col_opt_panel, text="Load destination image", command=load_dest_img).grid(row=2, columnspan=2, pady=(3, 2)) result_collage = None result_tile_info = None def change_alpha(_=None, show=True): if result_collage is not None and dest_img is not None: if colorization_opt.get() == "brightness": img = mkg.brightness_blend(result_collage, dest_img, 1 - alpha_scale.get() / 100) else: img = mkg.alpha_blend(result_collage, dest_img, 1 - alpha_scale.get() / 100) if show: show_img(img, False) return img # right collage option panel ROW 3: LabelWithTooltip(right_col_opt_panel, text="Color Blend:", tooltip=mkg.PARAMS.blending.help).grid( row=3, column=0, sticky="W", padx=(0, 5)) change_alpha_debounced = Debounce(change_alpha) # right collage option panel ROW 4: colorization_opt = StringVar() colorization_opt.set("brightness") RadiobuttonWithTooltip(right_col_opt_panel, text="Brightness", variable=colorization_opt, value="brightness", state=ACTIVE, command=change_alpha_debounced, tooltip=mkg.PARAMS.blending.help).grid(row=4, column=0, sticky="W") RadiobuttonWithTooltip(right_col_opt_panel, text="Alpha", variable=colorization_opt, value="alpha", command=change_alpha_debounced, tooltip=mkg.PARAMS.blending.help).grid(row=4, column=1, sticky="W") # right collage option panel ROW 5: alpha_scale = Scale(right_col_opt_panel, from_=0.0, to=100.0, orient=HORIZONTAL, length=150, command=change_alpha_debounced) alpha_scale.set(0) alpha_scale.grid(row=5, columnspan=2, sticky="W") # right collage option panel ROW 6: colorspace = StringVar() colorspace.set("lab") LabelWithTooltip(right_col_opt_panel, text="Colorspace:", tooltip=mkg.PARAMS.colorspace.help).grid(row=6, column=0, sticky="W") OptionMenu(right_col_opt_panel, colorspace, "", *mkg.PARAMS.colorspace.choices).grid(row=6, column=1, sticky="W") # right collage option panel ROW 7: dist_metric = StringVar() dist_metric.set("euclidean") LabelWithTooltip(right_col_opt_panel, text="Metric:", tooltip=mkg.PARAMS.metric.help).grid(row=7, column=0, sticky="W") OptionMenu(right_col_opt_panel, dist_metric, "", *mkg.PARAMS.metric.choices).grid(row=7, column=1, sticky="W") def attach_even(): collage_uneven_panel.grid_remove() collage_even_panel.grid(row=11, columnspan=2, sticky="W") def attach_uneven(): collage_even_panel.grid_remove() collage_uneven_panel.grid(row=11, columnspan=2, sticky="W") # right collage option panel ROW 8: LabelWithTooltip(right_col_opt_panel, text="Fairness of tiles: ", tooltip=mkg.PARAMS.unfair.help).grid(row=8, columnspan=2, sticky="W") # right collage option panel ROW 9: even = StringVar() even.set("even") RadiobuttonWithTooltip(right_col_opt_panel, text="Fair", variable=even, value="even", state=ACTIVE, command=attach_even, tooltip="Require all tiles are used the same amount of times (fair tile usage)").grid(row=9, column=0, sticky="W") RadiobuttonWithTooltip(right_col_opt_panel, text="Unfair", variable=even, value="uneven", command=attach_uneven, tooltip=mkg.PARAMS.unfair.help).grid(row=9, column=1, sticky="W") # right collage option panel ROW 10: Separator(right_col_opt_panel, orient="horizontal").grid(row=10, columnspan=2, sticky="we", pady=(5, 5)) # right collage option panel ROW 11: Dynamically attached # could EITHER collage even panel OR collage uneven panel # ----------------------- start collage even panel ------------------------ collage_even_panel = PanedWindow(right_col_opt_panel) collage_even_panel.grid(row=11, columnspan=2, sticky="W") # collage even panel ROW 1 LabelWithTooltip(collage_even_panel, text="Duplicates:", tooltip=mkg.PARAMS.dup.help).grid(row=1, column=0, sticky="W") dup = DoubleVar() dup.set(1.0) Entry(collage_even_panel, textvariable=dup, width=5).grid(row=1, column=1, sticky="W") # ----------------------- end collage even panel ------------------------ # ----------------------- start collage uneven panel -------------------- collage_uneven_panel = PanedWindow(right_col_opt_panel) # collage uneven panel ROW 0 LabelWithTooltip(collage_uneven_panel, text="Max width:", tooltip=mkg.PARAMS.max_width.help).grid(row=0, column=0, sticky="W") max_width = IntVar() max_width.set(80) Entry(collage_uneven_panel, textvariable=max_width, width=5).grid(row=0, column=1, sticky="W") LabelWithTooltip(collage_uneven_panel, text="Freq Mul:", tooltip=mkg.PARAMS.freq_mul.help).grid(row=2, column=0, sticky="W") freq_mul = DoubleVar() freq_mul.set(1) Entry(collage_uneven_panel, textvariable=freq_mul, width=5).grid(row=2, column=1, sticky="W") def dither_cb(): if dither.get(): deterministic.set(True) deterministic_check.config(state='disabled') is_salient.set(False) is_salient_check.config(state='disabled') transparent.set(False) transparent_check.config(state='disabled') else: deterministic_check.config(state='enabled') is_salient_check.config(state='enabled') transparent_check.config(state='enabled') dither = BooleanVar() dither.set(False) dither_check = CheckbuttonWithTooltip(collage_uneven_panel, text="Dithering", variable=dither, command=dither_cb, tooltip=mkg.PARAMS.dither.help) dither_check.grid(row=3, columnspan=2, sticky="W") deterministic = BooleanVar() deterministic.set(False) deterministic_check = CheckbuttonWithTooltip(collage_uneven_panel, text="Deterministic", variable=deterministic, tooltip=mkg.PARAMS.deterministic.help) deterministic_check.grid(row=4, columnspan=2, sticky="w") # ----------------------- end collage uneven panel ---------------------- def generate_collage(): if imgs is None: return messagebox.showerror("No tiles", "Please first load tiles") if dest_img is None: return messagebox.showerror("No destination image", "Please first load the image that you're trying to fit") try: if is_salient.get(): lower_thresh = saliency_thresh_scale.get() / 100 assert 0.0 < lower_thresh < 1.0, "saliency threshold must be between 0 and 1" else: lower_thresh = None if even.get() == "even": _dup = mkg.check_dup_valid(dup.get()) if is_salient.get() or transparent.get(): def action(): return mkg.MosaicFairSalient( dest_img, imgs, _dup, colorspace.get(), dist_metric.get(), lower_thresh, transparent.get(), out_wrapper).process_dest_img(dest_img) else: def action(): return mkg.MosaicFair(dest_img.shape, imgs, _dup, colorspace.get(), dist_metric.get()).process_dest_img(dest_img, out_wrapper) else: assert max_width.get() > 0, "Max width must be a positive number" assert freq_mul.get() >= 0, "Max width must be a nonnegative real number" def action(): return mkg.MosaicUnfair(dest_img.shape, imgs, max_width.get(), colorspace.get(), dist_metric.get(), lower_thresh, freq_mul.get(), not deterministic.get(), dither.get(), transparent.get()).process_dest_img(dest_img) def wrapper(): global result_collage try: result_collage, tile_info = action() return change_alpha(show=False), tile_info except AssertionError as e: return messagebox.showerror("Error", e) except: messagebox.showerror("Error", traceback.format_exc()) pool.submit(wrapper).add_done_callback(show_img) except AssertionError as e: return messagebox.showerror("Error", e) except: return messagebox.showerror("Error", traceback.format_exc()) def attach_salient_opt(): if is_salient.get(): salient_opt_panel.grid(row=14, columnspan=2, pady=2, sticky="w") else: salient_opt_panel.grid_remove() # right collage option panel ROW 12 is_salient = BooleanVar() is_salient.set(False) is_salient_check = Checkbutton(right_col_opt_panel, text="Salient objects only", variable=is_salient, command=attach_salient_opt) is_salient_check.grid(row=12, columnspan=2, sticky="w") def transp_cb(): if transparent.get(): is_salient.set(False) is_salient_check.config(state='disabled') dither.set(False) dither_check.config(state='disabled') else: is_salient_check.config(state='enabled') dither_check.config(state='enabled') # right collage option panel ROW 13 transparent = BooleanVar() transparent.set(False) transparent_check = CheckbuttonWithTooltip(right_col_opt_panel, text="Transparency masking", variable=transparent, tooltip=mkg.PARAMS.transparent.help, command=transp_cb) transparent_check.grid(row=13, columnspan=2, sticky="w") def change_thresh(): if dest_img is not None: lower_thresh = saliency_thresh_scale.get() / 100 assert 0.0 <= lower_thresh <= 1.0 if dest_img.shape[2] == 4: tmp_dest_img = cv2.cvtColor(dest_img, cv2.COLOR_BGRA2BGR) else: tmp_dest_img = dest_img.copy() _, thresh_map = cv2.saliency.StaticSaliencyFineGrained_create().computeSaliency(tmp_dest_img) tmp_dest_img[thresh_map < lower_thresh] = 255 show_img(tmp_dest_img, False) # right collage option panel ROW 14 changed_thresh_debounced = Debounce(change_thresh) salient_opt_panel = PanedWindow(right_col_opt_panel) Label(salient_opt_panel, text="Saliency threshold: ").grid(row=0, column=0, sticky="w") saliency_thresh_scale = Scale(salient_opt_panel, from_=1.0, to=99.0, orient=HORIZONTAL, length=150, command=changed_thresh_debounced) saliency_thresh_scale.set(50.0) saliency_thresh_scale.grid(row=1, columnspan=2, sticky="W") # right collage option panel ROW 16 collage_button = Button(right_col_opt_panel, text=" Generate Collage ", command=generate_collage) collage_button.config(state='disabled') collage_button.grid(row=16, columnspan=2, pady=(3, 5)) # ------------------------ end right collage option panel -------------------- # right panel ROW 9: Separator(right_panel, orient="horizontal").grid( row=9, columnspan=2, sticky="we", pady=(4, 10)) save_img_init_dir = init_dir def save_img(): global save_img_init_dir if result_img is None: messagebox.showerror("Error", "You don't have any image to save yet!") return fp = filedialog.asksaveasfilename(initialdir=save_img_init_dir, title="Save your collage", filetypes=(("images", "*.jpg"), ("images", "*.png")), defaultextension=".png", initialfile="result.png") dir_name = os.path.dirname(fp) if fp is not None and len(fp) > 0 and os.path.isdir(dir_name): save_img_init_dir = dir_name try: mkg.imwrite(fp, result_img) print("Image saved to", fp) except: messagebox.showerror("Error", traceback.format_exc()) def save_tile_info(): global save_img_init_dir if result_tile_info is None: messagebox.showerror("Error", "You need to make a collage/photomosaic first!") return fp = filedialog.asksaveasfilename(initialdir=save_img_init_dir, title="Save your collage", filetypes=(("csv file", "*.csv"), ("text file", "*.txt")), defaultextension=".csv", initialfile="tile_info.csv") dir_name = os.path.dirname(fp) if fp is not None and len(fp) > 0 and os.path.isdir(dir_name): save_img_init_dir = dir_name try: with open(fp, "w", encoding="utf-8") as f: f.write(result_tile_info) print("Tile info saved to", fp) except: messagebox.showerror("Error", traceback.format_exc()) # right panel ROW 10: save_button = Button(right_panel, text=" Save image ", command=save_img) save_button.config(state='disabled') save_button.grid(row=10, columnspan=2, pady=(0, 4)) tile_save_button = Button(right_panel, text=" Save tile info ", command=save_tile_info) tile_save_button.config(state='disabled') tile_save_button.grid(row=11, columnspan=2) # -------------------------- end right panel ----------------------------------- # make the window appear at the center # https://www.reddit.com/r/Python/comments/6m03sh/make_tkinter_window_in_center_of_screen_newbie/ root.update_idletasks() w = root.winfo_screenwidth() h = root.winfo_screenheight() size = tuple(int(pos) for pos in root.geometry().split('+')[0].split('x')) x = w / 2 - size[0] / 2 y = h / 2 - size[1] / 2 - 10 root.geometry("%dx%d+%d+%d" % (size + (x, y))) root.update() right_panel_width = right_panel.winfo_width() log_entry_height = left_panel.winfo_height() - canvas.winfo_height() last_resize_time = time.time() def canvas_resize(event): global last_resize_time if time.time() - last_resize_time > 0.1 and event.width >= 850 and event.height >= 550: last_resize_time = time.time() log_entry.configure(height=6 + math.floor((event.height - 500) / 80)) log_entry.width = log_entry.initial_width + math.floor((event.width - 800) / 10) mkg.pbar_ncols = log_entry.width log_entry.update() canvas.configure( width=event.width - right_panel_width - 20, height=event.height - log_entry.winfo_height() - 15) canvas.update() show_img(result_img, False) root.bind("", canvas_resize) out_wrapper = log_entry mkg.enable_gpu(False) # mainly for debugging purposes if cmd_args.D: file_path.set(cmd_args.src) print("Loading tiles from", cmd_args.src) pool.submit(load_img_action).add_done_callback(show_img) print("Destination image loaded from", cmd_args.collage) dest_img = mkg.imread(cmd_args.collage, cv2.IMREAD_UNCHANGED) show_img(dest_img, False) dest_img_path.set(cmd_args.collage) reload_img_button.config(state='enabled') sort_button.config(state='enabled') collage_button.config(state='enabled') sys.stdout = out_wrapper sys.stderr = out_wrapper root.mainloop() ================================================ FILE: io_utils.py ================================================ import os import io import sys import time import queue import ctypes import platform import tempfile import threading from tkinter import Text, END from contextlib import contextmanager from tqdm import tqdm @contextmanager def stdout_redirector(stream: io.TextIOBase): """ write stuff from stdout to stream that has a .write method part of the following function is borrowed from 1. https://stackoverflow.com/questions/4675728/redirect-stdout-to-a-file-in-python 2. https://gist.github.com/natedileas/8eb31dc03b76183c0211cdde57791005 """ if platform.system() == "Windows": if hasattr(sys, 'gettotalrefcount'): # debug build libc = ctypes.CDLL('ucrtbased') else: libc = ctypes.CDLL('api-ms-win-crt-stdio-l1-1-0') else: libc = ctypes.CDLL(None) stdout = sys.__stdout__ stdout_fd = stdout.fileno() # copy stdout_fd before it is overwritten #NOTE: `copied` is inheritable on Windows when duplicating a standard stream with os.fdopen(os.dup(stdout_fd), 'wb') as copied, tempfile.TemporaryFile(mode='w+b') as tfile: stdout.flush() # flush library buffers that dup2 knows nothing about os.dup2(tfile.fileno(), stdout_fd) # $ exec >&to try: flag = True def poll_once(): libc.fflush(None) stdout.flush() tfile.flush() tfile.seek(0) stream.write(tfile.read().decode("utf-8")) tfile.truncate(0) def poll(): while flag: poll_once() time.sleep(0.1) poll_once() poll_thread = threading.Thread(target=poll) poll_thread.start() yield stdout # allow code to be run with the redirected stdout finally: # restore stdout to its previous value # NOTE: dup2 makes stdout_fd inheritable unconditionally stdout.flush() flag = False poll_thread.join() os.dup2(copied.fileno(), stdout_fd) # $ exec >&copied stream.close() class JVOutWrapper: """ The output wrapper for displaying the progress of J-V algorithm """ def __init__(self, io_wrapper, ncols): self.io_wrapper = io_wrapper self.tqdm = None self.ncols = ncols def write(self, lines: str): if self.io_wrapper is None: return for line in lines.split("\n"): if not line.startswith("lapjv: "): # self.io_wrapper.write(line + "\n") continue if line.startswith("lapjv: AUGMENT SOLUTION row "): line = line.replace(" ", "") slash_idx = line.find("/") s_idx = line.find("[") e_idx = line.find("]") if s_idx > -1 and slash_idx > -1 and e_idx > -1: if self.tqdm: self.tqdm.n = int(line[s_idx + 1:slash_idx]) self.tqdm.update(0) else: self.tqdm = tqdm(file=self.io_wrapper, ncols=self.ncols, total=int(line[slash_idx + 1:e_idx]), desc="lapjv: ") continue if not self.tqdm: self.io_wrapper.write(line + "\n") def flush(self): pass def close(self): if self.tqdm: self.tqdm.n = self.tqdm.total self.tqdm.refresh() self.tqdm.close() # adapted from # https://stackoverflow.com/questions/16745507/tkinter-how-to-use-threads-to-preventing-main-event-loop-from-freezing class SafeText(Text): def __init__(self, master, **options): Text.__init__(self, master, **options) self.queue = queue.Queue() self.encoding = "utf-8" self.gui = True self.initial_width = 85 self.width = self.initial_width self.update_me() def write(self, line: str): self.queue.put(line) def flush(self): pass # this one run in the main thread def update_me(self): while not self.queue.empty(): line = self.queue.get_nowait() # a naive way to process the \r control char if line.find("\r") > -1: line = line.replace("\r", "") row = int(self.index(END).split(".")[0]) self.delete("{}.0".format(row - 1), "{}.{}".format(row - 1, len(line))) self.insert("end-1c linestart", line) else: self.insert(END, line) self.see("end-1c") self.update_idletasks() self.after(50, self.update_me) ================================================ FILE: make_img.py ================================================ import os import sys import time import math import random import argparse import itertools import traceback import multiprocessing as mp from fractions import Fraction from typing import Any, Callable, List, Tuple from collections import defaultdict from io_utils import stdout_redirector, JVOutWrapper import cv2 import imagesize import numpy as np cp = np from tqdm import tqdm from lapjv import lapjv Grid = Tuple[int, int] # grid size = (width, height) BackgroundRGB = Tuple[int, int, int] if mp.current_process().name != "MainProcess": sys.stdout = open(os.devnull, "w") sys.stderr = sys.stdout pbar_ncols = None LIMIT = 2**32 class _PARAMETER: def __init__(self, type: Any, help: str, default=None, nargs=None, choices: List[Any]=None) -> None: self.type = type self.default = default self.help = help self.nargs = nargs self.choices = choices # We gather parameters here so they can be reused else where class PARAMS: path = _PARAMETER(help="Path to the tiles", type=str) recursive = _PARAMETER(type=bool, default=False, help="Whether to read the sub-folders for the specified path") num_process = _PARAMETER(type=int, default=mp.cpu_count() // 2, help="Number of processes to use for parallelizable operations") out = _PARAMETER(default="result.png", type=str, help="The filename of the output collage/photomosaic") size = _PARAMETER(type=int, nargs="+", default=(50,), help="Width and height of each tile in pixels in the resulting collage/photomosaic. " "If two numbers are specified, they are treated as width and height. " "If one number is specified, the number is treated as the width " "and the height is inferred from the aspect ratios of the images provided. ") quiet = _PARAMETER(type=bool, default=False, help="Do not print progress message to console") auto_rotate = _PARAMETER(type=int, default=0, choices=[-1, 0, 1], help="Options to auto rotate tiles to best match the specified tile size. 0: do not auto rotate. " "1: attempt to rotate counterclockwise by 90 degrees. -1: attempt to rotate clockwise by 90 degrees") resize_opt = _PARAMETER(type=str, default="center", choices=["center", "stretch", "fit"], help="How to resize each tile so they have the desired aspect ratio and size, " "which can be specified fully or partially by --size. " "Center: crop the largest rectangle from the center. Stretch: stretch the tile. " "Fit: pad the tiles with white background") gpu = _PARAMETER(type=bool, default=False, help="Use GPU acceleration. Requires cupy to be installed and a capable GPU. Note that USUALLY this is useful when you: " "1. have a lot of tiles (typically > 10000), and" "2. are using the unfair mode, and" "3. (for photomosaic videos only) only have few cpu cores" "Also note: enabling GPU acceleration will disable multiprocessing on CPU for videos" ) mem_limit = _PARAMETER(type=int, default=4096, help="The APPROXIMATE memory limit in MB when computing a photomosaic in unfair mode. Applicable both CPU and GPU computing. " "If you run into memory issues when using GPU, try reduce this memory limit") tile_info_out = _PARAMETER(type=str, default="", help="Path to save the list of tile filenames for the collage/photomosaic. If empty, it will not be saved.") # ---------------- sort collage options ------------------ ratio = _PARAMETER(type=int, default=(16, 9), help="Aspect ratio of the output image", nargs=2) sort = _PARAMETER(type=str, default="bgr_sum", help="Sort method to use", choices=[ "none", "bgr_sum", "av_hue", "av_sat", "av_lum", "rand" ]) rev_row = _PARAMETER(type=bool, default=False, help="Whether to use the S-shaped alignment.") rev_sort = _PARAMETER(type=bool, default=False, help="Sort in the reverse direction.") # ---------------- photomosaic common options ------------------ dest_img = _PARAMETER(type=str, default="", help="The path to the destination image that you want to build a photomosaic for") colorspace = _PARAMETER(type=str, default="lab", choices=["hsv", "hsl", "bgr", "lab", "luv"], help="The colorspace used to calculate the metric") metric = _PARAMETER(type=str, default="euclidean", choices=["euclidean", "cityblock", "chebyshev", "cosine"], help="Distance metric used when evaluating the distance between two color vectors") transparent = _PARAMETER(type=bool, default=False, help="Enable transparency masking. The transparent regions of the destination image will be maintained in the photomosaic" "Cannot be used together with --salient") # ---- unfair tile assignment options ----- unfair = _PARAMETER(type=bool, default=False, help="Whether to allow each tile to be used different amount of times (unfair tile usage). ") max_width = _PARAMETER(type=int, default=80, help="Maximum width of the collage. This option is only valid if unfair option is enabled") freq_mul = _PARAMETER(type=float, default=0.0, help="Frequency multiplier to balance tile fairless and mosaic quality. Minimum: 0. " "More weight will be put on tile fairness when this number increases.") dither = _PARAMETER(type=bool, default=False, help="Whether to enabled dithering. You must also specify --deterministic if enabled. ") deterministic = _PARAMETER(type=bool, default=False, help="Do not randomize the tiles. This option is only valid if unfair option is enabled") # --- fair tile assignment options --- dup = _PARAMETER(type=float, default=1, help="If a positive integer: duplicate the set of tiles by how many times. Can be a fraction") # ---- saliency detection options --- salient = _PARAMETER(type=bool, default=False, help="Make photomosaic for salient objects only") lower_thresh = _PARAMETER(type=float, default=0.5, help="The threshold for saliency detection, between 0.0 (no object area = blank) and 1.0 (maximum object area = original image)") # ---- blending options --- blending = _PARAMETER(type=str, default="alpha", choices=["alpha", "brightness"], help="The types of blending used. alpha: alpha (transparency) blending. Brightness: blending of brightness (lightness) channel in the HSL colorspace") blending_level = _PARAMETER(type=float, default=0.0, help="Level of blending, between 0.0 (no blending) and 1.0 (maximum blending). Default is no blending") video = _PARAMETER(type=bool, default=False, help="Make a photomosaic video from dest_img which is assumed to be a video") skip_frame = _PARAMETER(type=int, default=1, help="Make a photomosaic every this number of frames") # https://stackoverflow.com/questions/26598109/preserve-custom-attributes-when-pickling-subclass-of-numpy-array class InfoArray(np.ndarray): def __new__(cls, input_array, info=''): # Input array is an already formed ndarray instance # We first cast to be our class type obj = np.asarray(input_array).view(cls) # add the new attribute to the created instance obj.info = info # Finally, we must return the newly created object: return obj def __array_finalize__(self, obj): # see InfoArray.__array_finalize__ for comments if obj is None: return self.info = getattr(obj, 'info', None) def __reduce__(self): # Get the parent's __reduce__ tuple pickled_state = super(InfoArray, self).__reduce__() # Create our own tuple to pass to __setstate__ new_state = pickled_state[2] + (self.info,) # Return a tuple that replaces the parent's __setstate__ tuple with our own return (pickled_state[0], pickled_state[1], new_state) def __setstate__(self, state): self.info = state[-1] # Set the info attribute # Call the parent's __setstate__ with the other tuple elements. super(InfoArray, self).__setstate__(state[0:-1]) ImgList = List[InfoArray] cupy_available = False def fast_sq_euclidean(Asq, Bsq, AB): AB *= -2 AB += Asq AB += Bsq return AB def fast_cityblock(A, B, axis, out): Z = A - B np.abs(Z, out=Z) return np.sum(Z, axis=axis, out=out) def fast_chebyshev(A, B, axis, out): Z = A - B np.abs(Z, out=Z) return np.max(Z, axis=axis, out=out) def to_cpu(X: np.ndarray) -> np.ndarray: return X.get() if cupy_available else X def bgr_sum(img: np.ndarray) -> float: """ compute the sum of all RGB values across an image """ return np.sum(img) def av_hue(img: np.ndarray) -> float: """ compute the average hue of all pixels in HSV color space """ hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV) return np.mean(hsv[:, :, 0]) def av_sat(img: np.ndarray) -> float: """ compute the average saturation of all pixels in HSV color space """ hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV) return np.mean(hsv[:, :, 1]) lum_coeffs = np.array([0.241, 0.691, 0.068], dtype=np.float32)[np.newaxis, np.newaxis, :] def av_lum(img) -> float: """ compute the average luminosity """ lum = img * lum_coeffs np.sqrt(lum, out=lum) return np.mean(lum) def rand(img: np.ndarray) -> float: """ generate a random number for each image """ return random.random() def calc_grid_size(rw: int, rh: int, num_imgs: int, shape: Tuple[int, int, int]) -> Grid: """ :param rw: the width of the target image :param rh: the height of the target image :param num_imgs: number of images available :param shape: the shape of a tile :return: an optimal grid size """ possible_wh = [] th, tw, _ = shape for width in range(1, num_imgs): height = math.ceil(num_imgs / width) possible_wh.append((width * tw / (th * height), width, height)) dest_ratio = rw / rh grid = min(possible_wh, key=lambda x: (x[0] - dest_ratio) ** 2)[1:] print("Tile shape:", (tw, th)) print("Calculated grid size based on the aspect ratio of the destination image:", grid) print(f"Collage size will be {grid[0] * tw}x{grid[1] * th}. ") return grid def make_collage_helper(grid: Grid, sorted_imgs: ImgList, rev=False, ridx=None, cidx=None, file=None): grid = grid[::-1] th, tw, tc = sorted_imgs[0].shape tile_info = np.full(grid, "background", dtype=str) if ridx is None or cidx is None: combined_img = np.empty((grid[0] * th, grid[1] * tw, 4), dtype=np.uint8) # use an array of references to avoid copying individual tiles tiles = np.array([None] * len(sorted_imgs), dtype=object) tiles[:] = sorted_imgs tiles.shape = grid if rev: tiles[1::2] = tiles[1::2, ::-1] # while we can use a few lines of transpose + reshape to do this, # we just use an ordinary for loop in order to show the progress for i, j in tqdm(itertools.product(range(grid[0]), range(grid[1])), desc="[Aligning tiles]", ncols=pbar_ncols, total=np.prod(grid), file=file): tile = tiles[i, j] # if this tile already have an alpha channel, use it if tile.shape[2] == 4: combined_img[i * th:(i + 1)*th, j * tw:(j + 1)*tw] = tile else: combined_img[i * th:(i + 1)*th, j * tw:(j + 1)*tw, :3] = tile combined_img[i * th:(i + 1)*th, j * tw:(j + 1)*tw, 3] = 255 tile_info[i, j] = tile.info else: assert not rev combined_img = np.full((grid[0] * th, grid[1] * tw, 4), [255, 255, 255, 0], dtype=np.uint8) for k in tqdm(range(len(ridx)), desc="[Aligning tiles]", ncols=pbar_ncols, file=file): i = ridx[k] j = cidx[k] combined_img[i * th:(i + 1)*th, j * tw:(j + 1)*tw, :tc] = sorted_imgs[k] combined_img[i * th:(i + 1)*th, j * tw:(j + 1)*tw, 3] = 255 tile_info[i, j] = sorted_imgs[k].info return combined_img, f"Grid dimension: {grid[::-1]}\n" + '\n'.join(tile_info.flatten()) def make_collage(grid: Grid, sorted_imgs: ImgList, rev=False): """ :param grid: grid size :param sorted_imgs: list of images sorted in correct position :param rev: whether to have opposite alignment for consecutive rows :return: a collage """ total = np.prod(grid) diff = total - len(sorted_imgs) if diff > 0: print(f"Note: {diff} transparent tiles will be added to the grid.") sorted_imgs.extend([InfoArray( np.full((*sorted_imgs[0].shape[:2], 4), [255, 255, 255, 0], dtype=np.uint8), 'background')] * diff) elif len(sorted_imgs) > total: print(f"Note: {len(sorted_imgs) - total} tiles will be dropped from the grid.") del sorted_imgs[total:] return make_collage_helper(grid, sorted_imgs, rev) def alpha_blend(combined_img: np.ndarray, dest_img: np.ndarray, alpha=0.9): if alpha == 1.0: return combined_img if dest_img.shape[2] == 4: dest_img = cv2.cvtColor(dest_img, cv2.COLOR_BGRA2BGR) dest_img = dest_img * np.float32(1 - alpha) dest_img = cv2.resize(dest_img, combined_img.shape[1::-1]) combined_img = combined_img * np.array([alpha, alpha, alpha, 1], dtype=np.float32).reshape(1, 1, 4) combined_img[:, :, :3] += dest_img return combined_img.astype(np.uint8) def brightness_blend(combined_img: np.ndarray, dest_img: np.ndarray, alpha=0.9): """ blend the 2 imgs in the lightness channel (L in HSL) """ if alpha == 1.0: return combined_img if dest_img.shape[2] == 4: dest_img = cv2.cvtColor(dest_img, cv2.COLOR_BGRA2BGR) dest_img = cv2.cvtColor(dest_img, cv2.COLOR_BGR2HLS) dest_l = dest_img[:, :, 1] * np.float32(1 - alpha) dest_l = cv2.resize(dest_l, combined_img.shape[1::-1]) combined_img_hls = cv2.cvtColor(combined_img[:, :, :3], cv2.COLOR_BGR2HLS) comb_l = combined_img_hls[:, :, 1] * np.float32(alpha) comb_l += dest_l combined_img_hls[:, :, 1] = comb_l combined_img = combined_img.copy() combined_img[:, :, :3] = cv2.cvtColor(combined_img_hls, cv2.COLOR_HLS2BGR) return combined_img def sort_collage(imgs: ImgList, ratio: Grid, sort_method="pca_lab", rev_sort=False) -> Tuple[Grid, np.ndarray]: """ :param imgs: list of images :param ratio: The aspect ratio of the collage :param sort_method: :param rev_sort: whether to reverse the sorted array :return: [calculated grid size, sorted image array] """ t = time.time() grid = calc_grid_size(ratio[0], ratio[1], len(imgs), imgs[0].shape) if sort_method == "none": return grid, imgs print("Sorting images...") sort_function = eval(sort_method) indices = np.array(list(map(sort_function, imgs))).argsort() if rev_sort: indices = indices[::-1] print("Time taken: {}s".format(np.round(time.time() - t, 2))) return grid, [imgs[i] for i in indices] def solve_lap(cost_matrix: np.ndarray, v=-1): if v == -1: v = sys.__stderr__ """ solve the linear sum assignment (LAP) problem with progress info """ print("Computing optimal assignment on a {}x{} matrix...".format(cost_matrix.shape[0], cost_matrix.shape[1])) wrapper = JVOutWrapper(v, pbar_ncols) with stdout_redirector(wrapper): _, cols, cost = lapjv(cost_matrix, verbose=1) cost = cost[0] print("Total assignment cost:", cost) return cols def solve_lap_greedy(cost_matrix: np.ndarray, v=None): assert cost_matrix.shape[0] == cost_matrix.shape[1] print("Computing greedy assignment on a {}x{} matrix...".format(cost_matrix.shape[0], cost_matrix.shape[1])) row_idx, col_idx = np.unravel_index(np.argsort(cost_matrix, axis=None), cost_matrix.shape) cost = 0 row_assigned = np.full(cost_matrix.shape[0], -1, dtype=np.int32) col_assigned = np.full(cost_matrix.shape[0], -1, dtype=np.int32) pbar = tqdm(ncols=pbar_ncols, total=cost_matrix.shape[0]) for ridx, cidx in zip(row_idx, col_idx): if row_assigned[ridx] == -1 and col_assigned[cidx] == -1: row_assigned[ridx] = cidx col_assigned[cidx] = ridx cost += cost_matrix[ridx, cidx] pbar.update() if pbar.n == pbar.total: break pbar.close() print("Total assignment cost:", cost) return col_assigned def compute_block_map(thresh_map: np.ndarray, block_width: int, block_height: int, lower_thresh: int): """ Find the indices of the blocks that contain salient pixels according to the thresh_map returns [row indices, column indices, resized threshold map] of sizes [(N,), (N,), (W x H)] """ height, width = thresh_map.shape dst_size = (width - width % block_width, height - height % block_height) if thresh_map.shape[::-1] != dst_size: thresh_map = cv2.resize(thresh_map, dst_size) row_idx, col_idx = np.nonzero(thresh_map.reshape( dst_size[1] // block_height, block_height, dst_size[0] // block_width, block_width).max(axis=(1, 3)) >= lower_thresh ) return row_idx, col_idx, thresh_map def dup_to_meet_total(imgs: ImgList, total: int): """ note that this function modifies imgs in place """ orig_len = len(imgs) if total < orig_len: print(f"{total} tiles will be used 1 time. {orig_len - total}/{orig_len} tiles will not be used. ") del imgs[total:] return imgs full_count = total // orig_len remaining = total % orig_len imgs *= full_count if remaining > 0: print(f"{orig_len - remaining} tiles will be used {full_count} times. {remaining} tiles will be used {full_count + 1} times. Total tiles: {orig_len}.") imgs.extend(imgs[:remaining]) else: print(f"Total tiles: {orig_len}. All of them will be used {full_count} times.") return imgs def _cosine(A, B): return 1 - cp.inner(A / cp.linalg.norm(A, axis=1, keepdims=True), B) def _euclidean(A, B, BsqT): Asq = cp.sum(A**2, axis=1, keepdims=True) return fast_sq_euclidean(Asq, BsqT, A.dot(B.T)) def _other(A, B, dist_func, row_stride): total = A.shape[0] dist_mat = cp.empty((total, B.shape[1]), dtype=cp.float32) i = 0 while i < total - row_stride: next_i = i + row_stride dist_func(A[i:next_i, cp.newaxis, :], B, out=dist_mat[i:next_i], axis=2) i = next_i if i < total: dist_func(A[i:, cp.newaxis, :], B, out=dist_mat[i:], axis=2) return dist_mat def strip_alpha(dest_img: np.ndarray) -> np.ndarray: if dest_img.shape[2] == 4: return cv2.cvtColor(dest_img, cv2.COLOR_BGRA2BGR) return dest_img def thresh_map_transp(dest_img: np.ndarray) -> Tuple[np.ndarray, np.ndarray]: assert dest_img.shape[2] == 4, "You need an image with transparent background to do this" return strip_alpha(dest_img), dest_img[:, :, 3] > 0.0 class CachedCDist: def __init__(self, metric: str, B: np.ndarray): """ Simple implementation of scipy.spatial.distance.cdist """ if metric == "cosine": self.args = [B / cp.linalg.norm(B, axis=1, keepdims=True)] self.func = _cosine elif metric == "euclidean": self.args = [B, cp.sum(B**2, axis=1, keepdims=True).T] self.func = _euclidean else: row_stride = LIMIT // (B.size * 4) B = B[cp.newaxis] if metric == "cityblock": self.args = [B, fast_cityblock, row_stride] elif metric == "chebyshev": self.args = [B, fast_chebyshev, row_stride] else: raise ValueError(f"invalid metric {metric}") self.func = _other def __call__(self, A: np.ndarray) -> np.ndarray: return self.func(A, *self.args) class MosaicCommon: def __init__(self, imgs: ImgList, colorspace="lab") -> None: self.imgs = imgs self.normalize_first = False if colorspace == "bgr": self.flag = None elif colorspace == "hsv": self.flag = cv2.COLOR_BGR2HSV self.normalize_first = True elif colorspace == "hsl": self.flag = cv2.COLOR_BGR2HLS self.normalize_first = True elif colorspace == "lab": self.flag = cv2.COLOR_BGR2LAB elif colorspace == "luv": self.flag = cv2.COLOR_BGR2LUV else: raise ValueError("Unknown colorspace " + colorspace) def make_photomosaic(self, assignment: np.ndarray, file=None): return make_collage_helper(self.grid, [self.imgs[i] for i in assignment], file=file) def make_photomosaic_mask(self, assignment: np.ndarray, ridx: np.ndarray, cidx: np.ndarray, file=None): return make_collage_helper(self.grid, [self.imgs[i] for i in assignment], False, ridx, cidx, file=file) def convert_colorspace(self, img: np.ndarray): if self.flag is None: return cv2.cvtColor(img, self.flag, dst=img) if self.normalize_first: # for hsv/hsl, h is in range 0~360 while other channels are in range 0~1 # need to normalize img[:, :, 0] *= 1 / 360.0 def compute_block_size(self, dest_shape: Tuple[int, int, int], grid: Grid): self.grid = grid self.block_height = round(dest_shape[0] / grid[1]) self.block_width = round(dest_shape[1] / grid[0]) th, tw, _ = self.imgs[0].shape if self.block_width > tw or self.block_height > th: m = max(tw / self.block_width, th / self.block_height) self.block_width = math.floor(self.block_width * m) self.block_height = math.floor(self.block_height * m) self.flat_block_size = self.block_width * self.block_height * 3 print("Block size:", (self.block_width, self.block_height)) self.target_sz = (grid[0] * self.block_width, grid[1] * self.block_height) print(f"Resizing dest image from {dest_shape[1]}x{dest_shape[0]} to {self.target_sz[0]}x{self.target_sz[1]}") def imgs_to_flat_blocks(self, metric: str): img_keys = np.zeros((len(self.imgs), self.block_height, self.block_width, 3), dtype=np.uint8) for i in range(len(self.imgs)): cv2.resize(self.imgs[i], (self.block_width, self.block_height), dst=img_keys[i]) img_keys.shape = (-1, self.block_width, 3) img_keys = img_keys * np.float32(1 / 255.0) self.convert_colorspace(img_keys) img_keys.shape = (-1, self.flat_block_size) img_keys = cp.asarray(img_keys) self.cdist = CachedCDist(metric, img_keys) self.img_keys = img_keys return img_keys def dest_to_flat_blocks(self, dest_img: np.ndarray): dest_img = cv2.resize(dest_img, self.target_sz) dest_img = dest_img * np.float32(1 / 255.0) self.convert_colorspace(dest_img) dest_img = cp.asarray(dest_img) dest_img.shape = (self.grid[1], self.block_height, self.grid[0], self.block_width, 3) return dest_img.transpose((0, 2, 1, 3, 4)).reshape(-1, self.flat_block_size) def dest_to_flat_blocks_mask(self, dest_img: np.ndarray, ridx: np.ndarray, cidx: np.ndarray): dest_img = dest_img * np.float32(1 / 255.0) self.convert_colorspace(dest_img) dest_img.shape = (self.grid[1], self.block_height, self.grid[0], self.block_width, 3) dest_img = dest_img[ridx, :, cidx, :, :] dest_img.shape = (-1, self.flat_block_size) print(f"Salient blocks/total blocks = {len(ridx)}/{np.prod(self.grid)}") return cp.asarray(dest_img) def calc_salient_col_even(dest_img: np.ndarray, imgs: ImgList, dup=1, colorspace="lab", metric="euclidean", lower_thresh=0.5, transparent=False, v=None): """ Compute the optimal assignment between the set of images provided and the set of pixels constitute of salient objects of the target image, with the restriction that every image should be used the same amount of times non salient part of the target image will be transparent """ t = time.time() print("Duplicating {} times".format(dup)) height, width, _ = dest_img.shape # this is just the initial (minimum) grid size total = round(len(imgs) * dup) grid = calc_grid_size(width, height, total, imgs[0].shape) if transparent: dest_img, orig_thresh_map = thresh_map_transp(dest_img) orig_thresh_map = orig_thresh_map.astype(np.float32) lower_thresh = 0.5 else: dest_img = strip_alpha(dest_img) _, orig_thresh_map = cv2.saliency.StaticSaliencyFineGrained_create().computeSaliency(dest_img) bh_f = height / grid[1] bw_f = width / grid[0] # DDA-like algorithm to decrease block size while preserving aspect ratio if bw_f > bh_f: bw_delta = 1 bh_delta = bh_f / bw_f else: bh_delta = 1 bw_delta = bh_f / bw_f imgs = imgs.copy() while True: block_width = int(bw_f) block_height = int(bh_f) if block_width <= 0 or block_height <= 0: print(f"Warning: Salient area is too small to put down all tiles given the duplication factor of {dup}. " "You can try to increase the saliency threshold if this is not desired.") block_width = max(block_width, 1) block_height = max(block_height, 1) ridx, cidx, thresh_map = compute_block_map(orig_thresh_map, block_width, block_height, lower_thresh) break ridx, cidx, thresh_map = compute_block_map(orig_thresh_map, block_width, block_height, lower_thresh) if len(ridx) >= total: break bw_f -= bw_delta bh_f -= bh_delta print(len(ridx), lower_thresh) dup_to_meet_total(imgs, len(ridx)) mos = MosaicCommon(imgs, colorspace) mos.block_width = block_width mos.block_height = block_height mos.flat_block_size = block_width * block_height * 3 mos.grid = (thresh_map.shape[1] // block_width, thresh_map.shape[0] // block_height) print("Block size:", (block_width, block_height)) print("Grid size:", mos.grid) mos.imgs_to_flat_blocks(metric) dest_img = cv2.resize(dest_img, thresh_map.shape[::-1]) dest_img = mos.dest_to_flat_blocks_mask(dest_img, ridx, cidx) assignment = solve_lap(to_cpu(mos.cdist(dest_img).T), v) print("Time taken: {}s".format((np.round(time.time() - t, 2)))) return mos.make_photomosaic_mask(assignment, ridx, cidx) class MosaicFairSalient: def __init__(self, *args, **kwargs) -> None: self.args = args self.kwargs = kwargs def process_dest_img(self, dest_img: np.ndarray): return calc_salient_col_even(dest_img, *self.args[1:], **self.kwargs) class MosaicFair(MosaicCommon): def __init__(self, dest_shape: Tuple[int, int, int], imgs: ImgList, dup=1, colorspace="lab", metric="euclidean", grid=None) -> None: """ Compute the optimal assignment between the set of images provided and the set of pixels of the target image, with the restriction that every image should be used the same amount of times """ if grid is not None: print("Use the provided grid size:", grid) dup = np.prod(grid) // len(imgs) + 1 else: # Compute the grid size based on the number images that we have grid = calc_grid_size(dest_shape[1], dest_shape[0], round(len(imgs) * dup), imgs[0].shape) total = np.prod(grid) imgs = dup_to_meet_total(imgs.copy(), total) if total > 10000: print("Warning: this may take longer than 5 minutes to compute") super().__init__(imgs, colorspace) self.compute_block_size(dest_shape, grid) self.imgs_to_flat_blocks(metric) def process_dest_img(self, dest_img: np.ndarray, file=None): dest_img = self.dest_to_flat_blocks(strip_alpha(dest_img)) cols = solve_lap(to_cpu(self.cdist(dest_img).T), file) return self.make_photomosaic(cols) class MosaicUnfair(MosaicCommon): def __init__(self, dest_shape: Tuple[int, int, int], imgs: ImgList, max_width: int, colorspace: str, metric: str, lower_thresh: float, freq_mul: float, randomize: bool, dither=False, transparent=False) -> None: # Because we don't have a fixed total amount of images as we can used a single image # for arbitrary amount of times, we need user to specify the maximum width in order to determine the grid size. dh, dw, _ = dest_shape th, tw, _ = imgs[0].shape grid = (max_width, round(dh * (max_width * tw / dw) / th)) print("Calculated grid size based on the aspect ratio of the image provided:", grid) print("Collage size:", (grid[0] * tw, grid[1] * th)) super().__init__(imgs, colorspace) self.compute_block_size(dest_shape, grid) img_keys = self.imgs_to_flat_blocks(metric) # number of rows in the cost matrix # note here we compute the cost matrix chunk by chunk to limit memory usage # a bit like sklearn.metrics.pairwise_distances_chunked num_rows = int(np.prod(grid)) num_cols = img_keys.shape[0] print(f"Distance matrix size: {(num_rows, num_cols)} = {num_rows * num_cols * 4 / 2**20}MB") self.row_stride = (LIMIT - (img_keys.size + num_rows * (1 + self.flat_block_size)) * 4) // (num_cols * 4) if self.row_stride >= num_rows: print("No chunking will be performed on the distance matrix calculation") else: print(f"Chunk size: {self.row_stride*num_cols* 4 / 2**20}MB | {self.row_stride}/{num_rows}") if freq_mul > 0: self.row_stride //= 16 self.indices_freq = cp.empty(num_cols, dtype=cp.float32) self.row_range = cp.arange(0, self.row_stride, dtype=cp.int32)[:, cp.newaxis] self.temp = cp.arange(0, num_cols, dtype=cp.float32) else: self.row_stride //= 4 self.freq_mul = freq_mul self.lower_thresh = lower_thresh self.randomize = randomize self.transparent = transparent self.saliency = False self.dither = dither _saliency_enabled = lower_thresh is not None if transparent: self.lower_thresh = 0.5 if dither: print("Warning: dithering is not supported when transparency masking is on. Dithering will be turned off") self.dither = False if _saliency_enabled: print("Warning: saliency is not supported when transparency masking is on. Saliency will be turned off") elif self.dither: if _saliency_enabled: print("Warning: saliency is not supported when dithering is on. Saliency will be turned off") if cp is not np: print("Warning: dithering is typically slower with --gpu enabled") if randomize: print("Warning: dithering is not supported when randomization is enabled. Randomization will be turned off.") self.randomize = False elif _saliency_enabled: self.saliency = cv2.saliency.StaticSaliencyFineGrained_create() def process_dest_img(self, dest_img: np.ndarray, file=None): if self.saliency or self.transparent: dest_img = cv2.resize(dest_img, self.target_sz) if self.saliency: dest_img = strip_alpha(dest_img) _, thresh_map = self.saliency.computeSaliency(dest_img) else: dest_img, thresh_map = thresh_map_transp(dest_img) ridx, cidx, thresh_map = compute_block_map(thresh_map, self.block_width, self.block_height, self.lower_thresh) dest_img = self.dest_to_flat_blocks_mask(dest_img, ridx, cidx) else: # strip alpha in case a transparent image is passed in but --transparent flag is not enabled dest_img = self.dest_to_flat_blocks(strip_alpha(dest_img)) total = dest_img.shape[0] assignment = cp.empty(total, dtype=cp.int32) if self.dither: dest_img.shape = (*self.grid[::-1], -1) grid_assignment = assignment.reshape(self.grid[::-1]) coeffs = cp.array([0.4375, 0.1875, 0.3125, 0.0625])[..., cp.newaxis] pbar = tqdm(desc="[Computing assignments]", total=total, ncols=pbar_ncols, file=file) i = 0 row_stride = self.row_stride if self.freq_mul > 0: _indices = np.arange(0, total, dtype=np.int32) if self.randomize: np.random.shuffle(_indices) dest_img = dest_img[_indices] # reorder the rows of dest img indices_freq = self.indices_freq indices_freq.fill(0.0) freq_mul = self.freq_mul if self.dither: for i in range(0, dest_img.shape[0] - 1): j = 0 dist = self.cdist(dest_img[i, j:j+1])[0] dist[cp.argsort(dist)] = self.temp dist += indices_freq best_i = grid_assignment[i, j] = cp.argmin(dist) indices_freq[best_i] += freq_mul pbar.update() for j in range(1, dest_img.shape[1] - 1): block = dest_img[i, j] dist = self.cdist(block[cp.newaxis])[0] dist[cp.argsort(dist)] = self.temp dist += indices_freq best_i = grid_assignment[i, j] = cp.argmin(dist) indices_freq[best_i] += freq_mul quant_error = (block - self.img_keys[best_i])[cp.newaxis, ...] * coeffs dest_img[i, j + 1] += quant_error[0] dest_img[i + 1, j - 1:j + 2] += quant_error[1:] pbar.update() j += 1 dist = self.cdist(dest_img[i, j:j+1])[0] dist[cp.argsort(dist)] = self.temp dist += indices_freq best_i = grid_assignment[i, j] = cp.argmin(dist) indices_freq[best_i] += freq_mul pbar.update() # last row dist_mat = self.cdist(dest_img[-1]) dist_mat[cp.arange(0, dest_img.shape[1], dtype=cp.int32)[:, cp.newaxis], cp.argsort(dist_mat, axis=1)] = self.temp for j in range(0, dest_img.shape[1]): row = dist_mat[j, :] row += indices_freq idx = cp.argmin(row) grid_assignment[-1, j] = idx indices_freq[idx] += freq_mul pbar.update() else: while i < total - row_stride: dist_mat = self.cdist(dest_img[i:i+row_stride]) dist_mat[self.row_range, cp.argsort(dist_mat, axis=1)] = self.temp j = 0 while j < row_stride: row = dist_mat[j, :] row += indices_freq idx = cp.argmin(row) assignment[i] = idx indices_freq[idx] += freq_mul i += 1 j += 1 pbar.update() if i < total: dist_mat = self.cdist(dest_img[i:]) dist_mat[self.row_range[:total - i], cp.argsort(dist_mat, axis=1)] = self.temp j = 0 while i < total: row = dist_mat[j, :] row += indices_freq idx = cp.argmin(row) assignment[i] = idx indices_freq[idx] += freq_mul i += 1 j += 1 pbar.update() assignment[_indices] = assignment.copy() else: if self.dither: for i in range(0, dest_img.shape[0] - 1): grid_assignment[i, 0] = cp.argmin(self.cdist(dest_img[i, 0:1])[0]) pbar.update() for j in range(1, dest_img.shape[1] - 1): block = dest_img[i, j] dist_mat = self.cdist(block[cp.newaxis]) best_i = cp.argmin(dist_mat[0]) grid_assignment[i, j] = best_i quant_error = (block - self.img_keys[best_i])[cp.newaxis, ...] * coeffs dest_img[i, j + 1] += quant_error[0] dest_img[i + 1, j - 1:j + 2] += quant_error[1:] pbar.update() grid_assignment[i, -1] = cp.argmin(self.cdist(dest_img[i, -1:])[0]) pbar.update() # last row cp.argmin(self.cdist(dest_img[-1]), axis=1, out=grid_assignment[-1]) pbar.update(dest_img.shape[1]) else: while i < total - row_stride: next_i = i + row_stride dist_mat = self.cdist(dest_img[i:next_i]) cp.argmin(dist_mat, axis=1, out=assignment[i:next_i]) pbar.update(row_stride) i = next_i if i < total: dist_mat = self.cdist(dest_img[i:]) cp.argmin(dist_mat, axis=1, out=assignment[i:]) pbar.update(total - i) pbar.close() assignment = to_cpu(assignment) if self.saliency or self.transparent: return self.make_photomosaic_mask(assignment, ridx, cidx, file=file) return self.make_photomosaic(assignment, file=file) def imwrite(filename: str, img: np.ndarray) -> None: ext = os.path.splitext(filename)[1] result, n = cv2.imencode(ext, img) assert result, "Error saving the collage" n.tofile(filename) def save_img(img: np.ndarray, path: str, suffix: str, file=None) -> None: if len(path) == 0: path = "result.png" if len(suffix) == 0: print("Saving to", path, file=file) imwrite(path, img) else: file_path, ext = os.path.splitext(path) path = f'{file_path}{suffix}{ext}' print("Saving to", path, file=file) imwrite(path, img) def get_size(img): try: w, h = imagesize.get(img) return int(w), int(h) except: return 0, 0 def get_size_slow(filename: str): img = imread(filename) if img is None: return 0, 0 return img.shape[1::-1] def infer_size(pool, files: List[str], infer_func: Callable[[str], Tuple[int, int]], i_type: str): sizes = defaultdict(int) for w, h in tqdm(pool.imap_unordered(infer_func, files, chunksize=64), total=len(files), desc=f"[Inferring size ({i_type})]", ncols=pbar_ncols): if w == 0 or h == 0: # skip zero size images continue sizes[Fraction(w, h)] += 1 sizes = [(args[1], args[0].numerator / args[0].denominator) for args in sizes.items()] sizes.sort() return sizes def read_images(pic_path: str, img_size: List[int], recursive, pool, flag="stretch", auto_rotate=0) -> ImgList: assert os.path.isdir(pic_path), "Directory " + pic_path + "is non-existent" files = [] print("Scanning files...") for root, _, file_list in os.walk(pic_path): for f in file_list: files.append(os.path.join(root, f)) if not recursive: break if len(img_size) == 1: sizes = infer_size(pool, files, get_size, "fast") if len(sizes) == 0: print("Warning: unable to infer image size through metadata. Will try reading the entire image (slow!)") sizes = infer_size(pool, files, get_size_slow, "slow") assert len(sizes) > 0, "Fail to infer size. All of your images are in an unsupported format!" # print("Aspect ratio (width / height, sorted by frequency) statistics:") # for freq, ratio in sizes: # print(f"{ratio:6.4f}: {freq}") most_freq_ratio = 1 / sizes[-1][1] img_size = (img_size[0], round(img_size[0] * most_freq_ratio)) print("Inferred tile size:", img_size) else: assert len(img_size) == 2 img_size = (img_size[0], img_size[1]) read_img = read_img_other if flag == "center": read_img = read_img_center if flag == "fit": read_img = read_img_fit result = [ r for r in tqdm( pool.imap_unordered( read_img, zip(files, itertools.repeat(img_size, len(files)), itertools.repeat(auto_rotate, len(files))), chunksize=32), total=len(files), desc="[Reading files]", unit="file", ncols=pbar_ncols) if r is not None ] print(f"Read {len(result)} images. {len(files) - len(result)} files cannot be decoded as images.") return result def imread(filename: str, flag=cv2.IMREAD_COLOR) -> np.ndarray: """ like cv2.imread, but can read images whose path contain unicode characters """ try: f = np.fromfile(filename, np.uint8) if not f.size: return None return cv2.imdecode(f, flag) except: return None def read_img_center(args: Tuple[str, Tuple[int, int], int]): # crop the largest rectangle from the center img_file, img_size, rot = args img = imread(img_file) if img is None: return None ratio = img_size[0] / img_size[1] # rotate the image if possible to preserve more area h, w, _ = img.shape if rot != 0 and abs(h / w - ratio) < abs(w / h - ratio): img = np.rot90(img, k=rot) w, h = h, w cw = round(h * ratio) # cropped width ch = round(w / ratio) # cropped height assert cw <= w or ch <= h cond = cw > w or (ch <= h and (w - cw) * h > (h - ch) * w) if cond: img = img.transpose((1, 0, 2)) w, h = h, w cw = ch margin = (w - cw) // 2 add = (w - cw) % 2 img = img[:, margin:w - margin + add, :] if cond: img = img.transpose((1, 0, 2)) return InfoArray(cv2.resize(img, img_size), img_file) def read_img_other(args: Tuple[str, Tuple[int, int], int]): img_file, img_size, rot = args img = imread(img_file) if img is None: return img if rot != 0: ratio = img_size[0] / img_size[1] h, w, _ = img.shape if abs(h / w - ratio) < abs(w / h - ratio): img = np.rot90(img, k=rot) return InfoArray(cv2.resize(img, img_size), img_file) def resizeAndPad(img, size, padColor=1.0): h, w = img.shape[:2] sw, sh = size # aspect ratio of image aspect = w / h saspect = sw / sh if (saspect > aspect): # new horizontal image new_h = sh new_w = round(new_h * aspect) pad_horz = (sw - new_w) / 2 pad_left, pad_right = math.floor(pad_horz), math.ceil(pad_horz) pad_top, pad_bot = 0, 0 elif (saspect < aspect): # new vertical image new_w = sw new_h = round(new_w / aspect) pad_vert = (sh - new_h) / 2 pad_top, pad_bot = math.floor(pad_vert), math.ceil(pad_vert) pad_left, pad_right = 0, 0 else: return cv2.resize(img, (sw, sh)) # set pad color if len(img.shape) == 3 and not isinstance(padColor, (list, tuple, np.ndarray)): # color image but only one color provided padColor = [padColor]*3 # scale and pad scaled_img = cv2.resize(img, (new_w, new_h)) scaled_img = cv2.copyMakeBorder(scaled_img, pad_top, pad_bot, pad_left, pad_right, borderType=cv2.BORDER_CONSTANT, value=padColor) return scaled_img def read_img_fit(args: Tuple[str, Tuple[int, int], int]): img_file, img_size, rot = args img = imread(img_file) if img is None: return img if rot != 0: ratio = img_size[0] / img_size[1] h, w, _ = img.shape if abs(h / w - ratio) < abs(w / h - ratio): img = np.rot90(img, k=rot) return InfoArray(resizeAndPad(img, img_size), img_file) # pickleable helper classes for unfair exp class _HelperChangeFreq: def __init__(self, dest_img: np.ndarray, mos: MosaicUnfair) -> None: self.mos = mos self.dest_img = dest_img def __call__(self, freq) -> Any: self.mos.freq_mul = freq return self.mos.process_dest_img(self.dest_img) class _HelperChangeColorspace: def __init__(self, dest_img, *args) -> None: self.dest_img = dest_img self.args = list(args) def __call__(self, colorspace) -> Any: self.args[3] = colorspace return MosaicUnfair(*self.args).process_dest_img(self.dest_img) def unfair_exp(dest_img: np.ndarray, args, imgs): import matplotlib.pyplot as plt all_colorspaces = PARAMS.colorspace.choices all_freqs = np.zeros(6, dtype=np.float64) all_freqs[1:] = np.logspace(-2, 2, 5) pbar = tqdm(desc="[Experimenting]", total=len(all_freqs) + len(all_colorspaces) + 1, unit="exps") mos_bgr = MosaicUnfair(dest_img.shape, imgs, args.max_width, "bgr", args.metric, None, None, 1.0, not args.deterministic) mos_fair = MosaicFair(dest_img.shape, imgs, colorspace="bgr", grid=mos_bgr.grid) change_cp = _HelperChangeColorspace(dest_img, dest_img.shape, imgs, args.max_width, None, args.metric, None, None, 1.0, not args.deterministic) change_freq = _HelperChangeFreq(dest_img, mos_bgr) with mp.Pool(4) as pool: futures1 = [pool.apply_async(change_cp, (colorspace,)) for colorspace in all_colorspaces] futures2 = [pool.apply_async(change_freq, (freq,)) for freq in all_freqs] futures2.append(pool.apply_async(mos_fair.process_dest_img, (dest_img,))) def collect_imgs(fname, params, futures, fs): result_imgs = [] for i in range(len(params)): result_imgs.append(futures[i].get()[0]) pbar.update() plt.figure(figsize=(len(params) * 10, 12)) plt.imshow(cv2.cvtColor(np.hstack(result_imgs), cv2.COLOR_BGR2RGB)) grid_width = result_imgs[0].shape[1] plt.xticks(np.arange(0, grid_width * len(result_imgs), grid_width) + grid_width / 2, params, fontsize=fs) plt.yticks([], []) plt.subplots_adjust(left=0.005, right=0.995) plt.savefig(f"{fname}.png", dpi=100) # plt.xlabel(xlabel) collect_imgs("colorspace", [c.upper() for c in all_colorspaces], futures1, 36) collect_imgs("fairness", [f"Frequency multiplier ($\lambda$) = ${c}$" for c in all_freqs] + ["Fair"], futures2, 20) pbar.refresh() # plt.show() def sort_exp(pool, args, imgs): n = len(PARAMS.sort.choices) for sort_method, (grid, sorted_imgs) in zip( PARAMS.sort.choices, pool.starmap(sort_collage, zip(itertools.repeat(imgs, n), itertools.repeat(args.ratio, n), PARAMS.sort.choices, itertools.repeat(args.rev_sort, n)) )): save_img(make_collage(grid, sorted_imgs, args.rev_row)[0], args.out, sort_method) def frame_generator(ret, frame, dest_video, skip_frame): i = 0 while ret: if i % skip_frame == 0: yield frame ret, frame = dest_video.read() i += 1 BlendFunc = Callable[[np.ndarray, np.ndarray, int], np.ndarray] def process_frame(frame: np.ndarray, mos: MosaicUnfair, blend_func: BlendFunc, blending_level: float, file=None): collage = mos.process_dest_img(frame, file=file)[0] if blending_level > 0.0: collage = blend_func(collage, frame, 1.0 - blending_level) return collage def frame_process(mos: MosaicUnfair, blend_func: BlendFunc, blending_level: float, path: str, in_q: mp.Queue, out_q: mp.Queue): """ Worker function that receives a frame from in_q, compute photomosaic and put it in out_q """ with open(os.devnull, "w") as null: while True: i, frame = in_q.get() if i is None: break out = process_frame(frame, mos, blend_func, blending_level, file=null) save_img(out, path, f".{i}", file=null) out_q.put(i) def enable_gpu(show_warning=True): global cupy_available, cp, fast_sq_euclidean, fast_cityblock, fast_chebyshev try: import cupy as cp cupy_available = True @cp.fuse def fast_sq_euclidean(Asq, Bsq, AB): return Asq + Bsq - 2*AB fast_cityblock = cp.ReductionKernel( 'T x, T y', # input params 'T z', # output params 'abs(x - y)', # map 'a + b', # reduce 'z = a', # post-reduction map '0', # identity value 'fast_cityblock' # kernel name ) fast_chebyshev = cp.ReductionKernel( 'T x, T y', # input params 'T z', # output params 'abs(x - y)', # map 'max(a, b)', # reduce 'z = a', # post-reduction map '0', # identity value 'fast_chebyshev' # kernel name ) except ImportError: if show_warning: print("Warning: GPU acceleration enabled with --gpu but cupy cannot be imported. Make sure that you have cupy properly installed. ") def check_dup_valid(dup): assert dup > 0, "dup must be a positive integer or a real number between 0 and 1" return dup def main(args): global LIMIT num_process = max(1, args.num_process) if args.video and not args.gpu: LIMIT = (args.mem_limit // num_process) * 2**20 else: LIMIT = args.mem_limit * 2**20 if len(args.out) > 0: folder, file_name = os.path.split(args.out) if len(folder) > 0: assert os.path.isdir(folder), "The output path {} does not exist!".format(folder) ext = os.path.splitext(file_name)[-1].lower() assert ext == ".jpg" or ext == ".png" or ext == ".jpeg", "The file extension must be .jpg, .jpeg or .png" if args.quiet: sys.stdout = open(os.devnull, "w") dup = check_dup_valid(args.dup) if len(args.dest_img) > 0: assert os.path.isfile(args.dest_img), f"Non existent destination image {args.dest_img}" # early check with mp.Pool(max(1, num_process)) as pool: imgs = read_images(args.path, args.size, args.recursive, pool, args.resize_opt, args.auto_rotate) if len(args.dest_img) == 0: # sort mode if args.exp: sort_exp(pool, args, imgs) else: collage, tile_info = make_collage(*sort_collage(imgs, args.ratio, args.sort, args.rev_sort), args.rev_row) save_img(collage, args.out, "") if args.tile_info_out: with open(args.tile_info_out, "w", encoding="utf-8") as f: f.write(tile_info) return if args.video: assert not (args.salient and not args.unfair), "Sorry, making photomosaic video is unsupported with fair and salient option. " assert args.skip_frame >= 1, "skip frame must be at least 1" # total_frames = count_frames(args.dest_img, args.skip_frame) dest_video = cv2.VideoCapture(args.dest_img) ret, frame = dest_video.read() assert ret, f"unable to open video {args.dest_img}" dest_shape = frame.shape else: dest_img = imread(args.dest_img, cv2.IMREAD_UNCHANGED) if args.transparent: assert dest_img.shape[2] == 4, "--transparent flag can only be used for images with transparent background!" if dest_img.shape[2] == 4 and not args.transparent: print("Note: alpha channel detected. If you like to perform transparency masking, add the --transparent flag.") dest_shape = dest_img.shape if args.gpu: enable_gpu() if args.exp: assert not args.salient assert args.unfair unfair_exp(dest_img, args, imgs) return if args.salient or args.transparent: lower_thresh = None if args.transparent else args.lower_thresh if args.unfair: mos = MosaicUnfair( dest_shape, imgs, args.max_width, args.colorspace, args.metric, lower_thresh, args.freq_mul, not args.deterministic, args.dither, args.transparent) else: mos = MosaicFairSalient(dest_shape, imgs, dup, args.colorspace, args.metric, lower_thresh, args.transparent) else: if args.unfair: mos = MosaicUnfair( dest_shape, imgs, args.max_width, args.colorspace, args.metric, None, args.freq_mul, not args.deterministic, args.dither) else: mos = MosaicFair(dest_shape, imgs, dup, args.colorspace, args.metric) if args.blending == "alpha": blend_func = alpha_blend else: blend_func = brightness_blend if args.video: th, tw, _ = mos.imgs[0].shape res = (tw * mos.grid[0], th * mos.grid[1]) print("Photomosaic video resolution:", res) frames_gen = frame_generator(ret, frame, dest_video, args.skip_frame) if args.gpu: with open(os.devnull, "w") as null: for idx, frame in tqdm(enumerate(frames_gen), desc="[Computing frames]", unit="frame"): out = process_frame(frame, mos, blend_func, args.blending_level, null) save_img(out, args.out, f"_{idx}", file=null) else: in_q = mp.Queue() out_q = mp.Queue() processes = [] for i in range(num_process): p = mp.Process(target=frame_process, args=(mos, blend_func, args.blending_level, args.out, in_q, out_q)) p.start() processes.append(p) with tqdm(desc="[Computing frames]", unit="frame") as pbar: for i, frame in enumerate(frames_gen): in_q.put((i, frame)) if not out_q.empty(): out_q.get() pbar.update() while pbar.n <= i: out_q.get() pbar.update() for p in processes: in_q.put((None, None)) for p in processes: p.join() frames_gen.close() else: collage, tile_info = mos.process_dest_img(dest_img) collage = blend_func(collage, dest_img, 1.0 - args.blending_level) save_img(collage, args.out, "") if args.tile_info_out: with open(args.tile_info_out, "w", encoding="utf-8") as f: f.write(tile_info) pool.close() if __name__ == "__main__": mp.freeze_support() parser = argparse.ArgumentParser( formatter_class=argparse.ArgumentDefaultsHelpFormatter ) for arg_name, data in PARAMS.__dict__.items(): if arg_name.startswith("__"): continue arg_name = "--" + arg_name if data.type == bool: assert data.default == False parser.add_argument(arg_name, action="store_true", help=data.help) continue parser.add_argument(arg_name, type=data.type, default=data.default, help=data.help, choices=data.choices, nargs=data.nargs) parser.add_argument("--exp", action="store_true", help="Do experiments (for testing only)") main(parser.parse_args()) ================================================ FILE: requirements.txt ================================================ opencv-contrib-python==4.5.5.62 lapjv==1.3.1 pyinstaller==4.8 imagesize==1.3.0 tqdm wheel