Repository: technopagan/adept-jpg-compressor
Branch: master
Commit: 364e7d9375db
Files: 7
Total size: 57.4 KB
Directory structure:
gitextract_4klx5xfu/
├── LICENSE
├── README.md
├── adept-jpeg.sh
├── adept.sh
├── batch-compress.sh
├── man/
│ └── man1/
│ └── adept-jpeg.sh.1
└── unittests/
└── tests_adept.bats
================================================
FILE CONTENTS
================================================
================================================
FILE: LICENSE
================================================
This software is published under the BSD licence 3.0
Copyright (c) 2013-2019, Tobias Baldauf All rights reserved.
Mail: kontakt@tobias-baldauf.de
Web: http://who.tobias.is/
Twitter: @tbaldauf
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of the author nor the names of contributors may be
used to endorse or promote products derived from this software without
specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
================================================
FILE: README.md
================================================
Adept - the adaptive JPG Compressor
====================
## Quick Start
* Please install [a MSS saliency algorithm binary](http://github.com/technopagan/mss-saliency), [ImageMagick](http://www.imagemagick.org/), [MozJPEG](https://github.com/mozilla/mozjpeg) and [JPEGrescan](https://github.com/kud/jpegrescan)
* Make MozJPEG available as "mozjpeg" in your $PATH, e.g. via symlink
* Fetch a copy of [adept](https://raw.github.com/technopagan/adept-jpg-compressor/master/adept.sh) and place it somewhere you deem a good place for 3rd party shellscripts, e.g. "/usr/local/bin". Make sure the location is in the $PATH of the user(s) who will run adept and ensure that the script is executable (chmod -x).
* Symlink it as "adept" for your own convenience
* Now you can run "adept /path/to/image.jpg" to compress JPEGs far more effectively!
## Introduction
When compressing JPEG images, the same compression level is used on the entire image. However, most JPEG images contain homogeneous and heterogeneous areas, which are varyingly well-suited for compression. Compressing heterogeneous areas in JPEGs to reduce filesize causes [compression artefacts](https://en.wikipedia.org/wiki/Compression_artifact) due to the lossy nature of JPEG compression.
This script adaptively alters the compression level for areas within JPEGs (per-block) to achieve optimized file size while maintaining a decent visual quality. This script achieves a significantly reduced file size compared to standard tools such as cjpeg while maintaining good visual quality, as can be measured via SSIM. This is good news for the [Web Performance](https://twitter.com/search?q=%23WebPerf&src=typd) and thus Web Developer community to achieve a great user experience on websites.
## Example
Best quality/size:
```
$ convert -verbose -quality 100 images/lena.png images/lena.q100.jpg
images/lena.png PNG 512x512 512x512+0+0 8-bit DirectClass 475KB 0.010u 0:00.010
images/lena.png=>images/lena.q100.jpg PNG 512x512 512x512+0+0 8-bit DirectClass 401KB 0.030u 0:00.039
$ ./adept-jpeg.sh images/lena.q100.jpg
./adept-jpeg.sh options: inherit, 69, autodetect, _adept_compress_imagemagick
404266 images/lena.q100.jpg
size=8 512 512
tilecount=64x64
salient=1.04084
salient=74.321
salient=48.2852
salient=36.5063
threshold=43
slice to ram... ok.
estimate content complexity and compress... 4096 tile ok.
final image... ok.
200860 images/lena.q100_adept_compress_imagemagick.jpg (50%)
$ dssim -o images/lena.q100_adept_compress_imagemagick.c.png images/lena.png images/lena.q100_adept_compress_imagemagick.jpg
0.00236562 lena.q100_adept_compress_imagemagick.jpg
```
   
PNG -> JPEG Q=100 -> ADEPT JPEG -> Compare (DSSIM)
## Contributors
In alphabetical order:
* [Andy Davies](http://twitter.com/andydavies)
* [Gregor Fabritius](http://twitter.com/grefab)
* [Neil Jedrzejewski](http://www.wunderboy.org/about.php)
* [Alessandro Lenzen](http://twitter.com/adelnorsz)
* [Claus Meteling](http://www.xing.com/profile/Claus_Meteling)
* [André Roaldseth](http://twitter.com/androa)
* [Christian Schäfer](http://twitter.com/derSchepp)
* [Yoav Weiss](http://twitter.com/yoavweiss)
## Licence
This software is published under the BSD licence 3.0
Copyright (c) 2015, Tobias Baldauf
All rights reserved.
Mail: [kontakt@tobias-baldauf.de](mailto:kontakt@tobias-baldauf.de)
Web: [who.tobias.is](http://who.tobias.is/)
Twitter: [@tbaldauf](http://twitter.com/tbaldauf)
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
* Neither the name of the author nor the names of contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
================================================
FILE: adept-jpeg.sh
================================================
#!/usr/bin/env bash
###############################################################################
#
# Bash script to automate adaptive JPEG compression using common CLI tools
#
# Usage: bash adept.sh /path/to/image.jpg
#
###############################################################################
#
# Brief overview of the mode of operation:
#
# The input JPG gets sliced into tiles, sized as a multiple of 8 due to the
# nature of JPG compression. The image is also run through a saliency
# detection algorithm and its resulting output further reduced to a
# 2-color black+white PNG.
#
# This bi-color PNG is ideal to measure tiles' gray channel mean value and use
# it as a single integer indicator to judge its perceivable complexity.
#
# Areas with low complexity contents are then exposed to heavier compression.
# At reassemlby, this leads to savings in image bytesize while maintaining
# good visual quality because no compression artefacts occur in areas of
# high-complexity or sharp contrasts.
#
###############################################################################
# Tools that need to be pre-installed:
#
# * Maximum Symmetric Surround Saliency Algorithm Binary
# http://github.com/technopagan/mss-saliency
#
# * ImageMagick >= v.6.6
#
# * MozJPEG
# http://github.com/mozilla/mozjpeg
# Expects Mozjpeg to be available under 'mozjpeg', e.g. via symlink
#
# * JPEGRescan Perl Script for lossless JPG compression
# http://github.com/kud/jpegrescan
#
# Note: Additonal tools are required to run Adept, such as "bc",
# "find", "mv", "rm" and Bash 3.x. As all of these tools are provided by lsbcore, core-utils
# or similar default packages, we can expect them to be always available.
#
###############################################################################
#
# This software is published under the BSD licence 3.0
#
# Copyright (c) 2013-2015, Tobias Baldauf
# All rights reserved.
#
# Mail: kontakt@tobias-baldauf.de
# Web: http://who.tobias.is/
# Twitter: @tbaldauf
#
# Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
#
# * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
#
# * Neither the name of the author nor the names of contributors may be used to endorse or promote products derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
###############################################################################
HELP="adept-jpeg.sh is a Bash script to automate adaptive JPEG compression using common CLI tools.
USAGE:
$0 [options] path/to/file.jpg
OPTIONS:
-h Displays this help message.
-c string Name compressor: imagemagick (default), cjpeg, jpegoptim, jpge, mozcjpeg.
-q 0..100 Compression rate of areas considered as important, default value is 'inherit'.
-Q 0..100 Compression rate of areas deemed suitable for high compression, default is 69.
-s suffix This will append 'suffix' to filenames. Default value is '_adept_compress'.
-t integer tile size, accepted values are 8,16,32,64,128 and 256. The smaller the value, the better results
but might take signifficantly more time to process. The default value is 'autodetect'.
"
###############################################################################
# USER CONFIGURABLE PARAMETERS
###############################################################################
# Default JPG quality setting, either inherited or defined as an integer of 0-100
# Default: inherit
DEFAULTCOMPRESSIONRATE="inherit"
# JPEG quality setting for areas of the image deemed suitable for high compression in an integer of 0-100
# Default: 69
HIGHCOMPRESSIONRATE="69"
# Suffix string to attach to the output JPG filename, e.g. '_adept_compress'
# If deliberatly set empty (''), the input JPG will be replaced with the new compressed JPG
OUTPUTFILESUFFIXBEGIN="_adept_compress"
# Square dimensions for all temporary tiles. Tile size heavily influences compression efficiency at the cost of runtime performance
# E.g. a tile size of 8 yields maximum compression results while taking several minutes of runtime
# If you chose to manually adjust tile size, only use multiples of 8 (8/16/32/64/128/256)
# Default: autodetect
TILESIZE="autodetect"
# Jpeg compressor
# Default: imagemagick
JPEGCNAME="imagemagick"
###############################################################################
# READ COMMAND LINE USER PARAMETERS
###############################################################################
while getopts ":c:q:Q:s:t:h" opt
do
case "$opt" in
c) JPEGCNAME=$OPTARG
;;
q) DEFAULTCOMPRESSIONRATE=$OPTARG
;;
Q) HIGHCOMPRESSIONRATE=$OPTARG
;;
s) OUTPUTFILESUFFIXBEGIN=$OPTARG
;;
t) TILESIZE=$OPTARG
;;
h) echo "${HELP}"
exit 0
;;
\?) echo "Invalid option: -$OPTARG" >&2
exit 1
;;
:) echo "Option -$OPTARG requires an argument." >&2
exit 1
;;
esac
done
if [ -z "$JPEGCNAME" ]
then
JPEGCNAME="imagemagick"
elif [ "$JPEGCNAME" != "cjpeg" -a "$JPEGCNAME" != "jpegoptim" -a "$JPEGCNAME" != "jpge" -a "$JPEGCNAME" != "mozcjpeg" ]
then
JPEGCNAME="imagemagick"
fi
OUTPUTFILESUFFIX="${OUTPUTFILESUFFIXBEGIN}_${JPEGCNAME}"
echo "$0 options: $DEFAULTCOMPRESSIONRATE, $HIGHCOMPRESSIONRATE, $TILESIZE, $OUTPUTFILESUFFIX"
shift "$(($OPTIND - 1))"
###############################################################################
# RUNTIME VARIABLES (usually do not require tuning by user)
###############################################################################
# Accept the jpg filename as a parameter
FILE="$1"
if [ -z "$FILE" ]
then
echo "${HELP}"
exit 0
fi
# Retrieve clean filename without extension
CLEANFILENAME=${FILE%.jp*g}
# Retrieve only the file extension
FILEEXTENSION=${FILE##*.}
# Retrieve clean path directory without filename
CLEANPATH="${FILE%/*}"
# If the JPEG is in the same direcctory as Adept, empty the path variable
# Or if it is set, make sure the path has a trailing slash
if [ "$CLEANPATH" == "$FILE" ]; then
CLEANPATH=""
else
CLEANPATH="$CLEANPATH/"
fi
# Storage location for all temporary files during runtime
# Use locations like /dev/shm (/run/shm/ in Ubuntu) to save files in Shared Memory Space (RAM) to avoid disk i/o troubles
TILESTORAGEPATH="/dev/shm/"
# Check if the directory for temporary image storage during runtime actually exists (honoring symlinks)
# In case it does not, fall back to using "/tmp/" because it is very likely available on all Unix systems
if [ ! -d "$TILESTORAGEPATH" ]; then
TILESTORAGEPATH="/tmp/"
fi
# Set locales to C (raw uninterpreted byte sequence)
# to avoid Illegal byte sequence errors and invalid number errors
export LANG=C LC_NUMERIC=C LC_COLLATE=C
###############################################################################
# MAIN PROGRAM
###############################################################################
prepwork () {
find_tool IDENTIFY_COMMAND identify
find_tool CONVERT_COMMAND convert
find_tool MONTAGE_COMMAND montage
if [ "${JPEGCNAME}" = "cjpeg" ]
then
find_tool JPEGCOMPRESSION_COMMAND cjpeg
find_tool JPEGDECOMPRESSION_COMMAND djpeg
elif [ "${JPEGCNAME}" = "imagemagick" ]
then
find_tool MOGRIFY_COMMAND mogrify
elif [ "${JPEGCNAME}" = "jpegoptim" ]
then
find_tool JPEGOPTIM_COMMAND jpegoptim
elif [ "${JPEGCNAME}" = "jpge" ]
then
find_tool JPEGCOMPRESSION_COMMAND jpge
elif [ "${JPEGCNAME}" = "mozcjpeg" ]
then
find_tool JPEGCOMPRESSION_COMMAND mozcjpeg
fi
find_tool JPEGRESCAN_COMMAND jpegrescan
find_tool SALIENCYDETECTOR_COMMAND SaliencyDetector
validate_image VALIDJPEG "${FILE}"
}
main () {
FILESIZE="$(stat -c %s ${FILE})"
echo "${FILESIZE} ${FILE}"
find_image_dimension IMAGEWIDTH "${FILE}" 'w'
find_image_dimension IMAGEHEIGHT "${FILE}" 'h'
optimize_tile_size TILESIZE ${TILESIZE} ${IMAGEWIDTH} ${IMAGEHEIGHT}
echo "size=${TILESIZE} ${IMAGEWIDTH} ${IMAGEHEIGHT}"
calculate_tile_count TILEROWS ${IMAGEHEIGHT} ${TILESIZE}
calculate_tile_count TILECOLUMNS ${IMAGEWIDTH} ${TILESIZE}
echo "tilecount=${TILEROWS}x${TILECOLUMNS}"
optimize_salient_regions_amount BLACKWHITETHRESHOLD "${FILE}"
echo "threshold=${BLACKWHITETHRESHOLD}"
${SALIENCYDETECTOR_COMMAND} -q -L0 -U${BLACKWHITETHRESHOLD} "${FILE}" "${TILESTORAGEPATH}${CLEANFILENAME##*/}_saliency_bw.png"
slice_image_to_ram "${FILE}" ${TILESIZE} ${TILESTORAGEPATH}
estimate_content_complexity_and_compress
reassemble_tiles_into_final_image "${FILESIZE}"
}
###############################################################################
# FUNCTIONS
###############################################################################
floatToInt() {
printf "%.0f\n" "$@"
}
# Find the proper handle for the required commandline tool
# This function can take an optional third parameter when being called to manually define the path to the CLI tool
function find_tool () {
# Define local variables to work with
local __result=$1
local __tool=$2
local __customtoolpath=$3
# Array of possible tool locations: name, name as ALL-CAPS, /usr/bin/name, /usr/local/bin/name and custom path
local __possibletoollocations=(${__tool} /usr/bin/${__tool} /usr/local/bin/${__tool} ${__customtoolpath})
# For each possible tool location, test if its actually available there
for i in "${__possibletoollocations[@]}"; do
local __commandlinetool=$(type -p $i)
# If 'type -p' returned something, we now have our proper handle
if [ "$__commandlinetool" ]; then
break
fi
done
# In case none of the given inputs works, apologize & quit
if [ ! "$__commandlinetool" ]; then
echo "Unable to find ${__tool}. Please ensure that it is installed. If necessary, set its CLI path+name in the find_tool function call and then retry."
exit 1
fi
# Return the result
eval $__result="'${__commandlinetool}'"
}
# Validate that we are working on an actual intact JPEG image before launch
function validate_image () {
# Define local variables to work with
local __result=$1
local __imagetovalidate=$2
# If the script is called without an input file, explain how to use it
# We don't "exit 1" here anymore because our unit tests source the script
# and would abort if "exit 1" was called
if [ ! -f "$__imagetovalidate" ]; then
local __validationresult=0
echo "Missing input JPEG. Usage: $0 /path/to/jpeg/image.jpg"
else
# Use IM identify to read the file magic of the input file to validate it's a JPEG
local __filemagic=$(${IDENTIFY_COMMAND} -format %m "$__imagetovalidate")
if [ "$__filemagic" == "JPEG" ] ; then
# Set a switch that it is ok to work on the input file, launching the main funtion
local __validationresult=1
fi
fi
# Return the result
eval $__result="'${__validationresult}'"
}
# Read width (%w) or height (%h) of the input image via IM identify
function find_image_dimension () {
# Define local variables to work with
local __result=$1
local __imagetomeasure=$2
local __dimensiontomeasure=$3
# Read the width or height of the input image into a global variable
local __imagedimension=$(${IDENTIFY_COMMAND} -format '%'${__dimensiontomeasure} ${__imagetomeasure})
# Return the result
eval $__result="'${__imagedimension}'"
}
# Tile size is the no.1 performance bottleneck for Adept, so it is important we pick an optimal tile size for the input image dimensions
# Also, the number of tiles to be recombined affects compression efficiency and salient areas within an image tend to have similar dimensional
# relations to total image size, so it makes sense to change tile size accordingly
function optimize_tile_size () {
# Define local variables to work with
local __result=$1
local __optimaltilesize=$2
local __currentimagewidth=$3
local __currentimageheight=$4
# The default "autodetect" setting causes Adept to find a suitable tile size according to image dimensions
if [ "$TILESIZE" == "autodetect" ] ; then
# Pick the smaller of the two dimensions of the image as the decisive integer for tile size
local __decisivedimension=${__currentimageheight}
if (( $IMAGEWIDTH < $__decisivedimension )); then
__decisivedimension=${__currentimagewidth}
fi
# For a series of sensible steps, change the tile size accordingly
if (( $__decisivedimension <= 512 )); then
__optimaltilesize="8"
elif (( $__decisivedimension >= 513 )) && (( $__decisivedimension <= 1024 )); then
__optimaltilesize="16"
elif (( $__decisivedimension >= 1025 )); then
__optimaltilesize="32"
else
__optimaltilesize="8"
fi
# In case the user has changed the configuration from "autodetect" to a custom setting, respect & return this instead
else
__optimaltilesize=${TILESIZE}
fi
# Return the result
eval $__result="'${__optimaltilesize}'"
}
function optimize_salient_regions_amount () {
# Define local variables to work with
local __result=$1
local __imagetomeasure=$2
local __lower_bound="0"
local __upper_bound="100"
local __current_threshold=$(( $__upper_bound/2 ))
local __mean_graychannel="0"
# Run the saliency detector function to retrieve the Median gray channel
calculate_salient_regions_amount __mean_graychannel "${__imagetomeasure}" ${__upper_bound}
__mean_graychannel=$(floatToInt $__mean_graychannel)
# If we didn't hit the sweet spot on our initial run, keep homing in on the ideal threshold value using binary search
while ( (( $__mean_graychannel > 40 )) || (( $__mean_graychannel < 20 )) ) && (( $__lower_bound < $__upper_bound-1 )); do
# If the Median is too low, reduce the upper threshold value to get more white pixels
if (( $__mean_graychannel < 20 )); then
__upper_bound=${__current_threshold}
# Else if the Median is too high, raise the threshold to get fewer white pixels
elif (( $__mean_graychannel > 40 )); then
__lower_bound=${__current_threshold}
fi
# Calculate the new middle threshold
__current_threshold=$(( ($__upper_bound-$__lower_bound)/2+$__lower_bound ))
# Rerun the saliency detector with a better estimated threshold value
calculate_salient_regions_amount __mean_graychannel "${__imagetomeasure}" ${__current_threshold}
__mean_graychannel=$(floatToInt $__mean_graychannel)
done
# Return result
eval $__result="'${__current_threshold}'"
}
# Measure the black/white median of a saliency mapped image to use it as an indicator for successfull saliency mapped contents
function calculate_salient_regions_amount () {
# Define local variables to work with
local __result=$1
local __imagetomeasure=$2
local __threshold=$3
# Use the MSS Saliency Detector with custom thresholds to generate a black+white salient map of an input image
# Then use the gray channel's mean as a single indicator to judge how much of the image's contents have been marked as salient
local __salient_amount=$(${SALIENCYDETECTOR_COMMAND} -q -L0 -U${__threshold} "${__imagetomeasure}" "png:-" | ${IDENTIFY_COMMAND} -channel Gray -format "%[fx:255*mean]" -)
echo "salient=${__salient_amount}"
# Return result
eval $__result="'${__salient_amount}'"
}
# Slice the input image into equally sized tiles
function slice_image_to_ram () {
# Define local variables to work with
local __filetoprocess=$1
local __currenttilesize=$2
local __currenttilestoragepath=$3
echo -n "slice to ram..."
# If $DEFAULTCOMPRESSIONRATE is set to "inherit", discover the input JPG quality
if [ "$DEFAULTCOMPRESSIONRATE" == "inherit" ] ; then
DEFAULTCOMPRESSIONRATE=$(${IDENTIFY_COMMAND} -format "%Q" ${__filetoprocess})
fi
echo " ok."
${CONVERT_COMMAND} "$__filetoprocess" -strip -quality "${DEFAULTCOMPRESSIONRATE}" -define jpeg:dct-method=default -crop "${__currenttilesize}"x"${__currenttilesize}" +repage +adjoin "${__currenttilestoragepath}tile_tmp_%06d_${CLEANFILENAME##*/}.${FILEEXTENSION}"
}
# For each tile, test if it is suitable for higher compression and if so, proceed
function estimate_content_complexity_and_compress () {
# Set up a counter so we keep track of the full name of the current temporary tile to work on
local __currenttilecount=0
# Let's create a walker that iterates over the sobeled and b/w reduced full size image
# This way, the edge detection happens only in memory and does not need additional tiles to be created on the filesystem
# The walker inputs X+Y coordinates and only analyses a single tile's size on that spot within the image
# The exception to this being when we are close to the image's end and we have to reduce tile size to whatever is left vertically or horizontally
echo -n "estimate content complexity and compress..."
for((y=0;y<$TILEROWS;y++)) ; do
for((x=0;x<$TILECOLUMNS;x++)) ; do
# Reset tile dimensions for each run because we need to check them anew each time
local __currenttileheight=${TILESIZE}
local __currenttilewidth=${TILESIZE}
# Prepend leading zeros to the counter so the integer matches the numbers handed out to the filename by ImageMagick
__currenttilecount=$(printf "%06d" $__currenttilecount);
# If we are nearing the end of the image height, reduce tile size to whatever is left vertically
if (( $y + 1 == $TILEROWS )) && (( $TILEROWS * $__currenttileheight > $IMAGEHEIGHT )); then
__currenttileheight=$(( (($y+1)*$TILESIZE) - $IMAGEHEIGHT ))
__currenttilerow=$(( $y+1 ))
fi
# And if we are nearing the end of the image width, reduce tile size to whatever is left horizontally
if (( $x + 1 == $TILECOLUMNS )) && (( $TILECOLUMNS * $__currenttilewidth > $IMAGEWIDTH )); then
__currenttilewidth=$(( (($x+1)*$TILESIZE) - $IMAGEWIDTH ))
__currenttilecolumn=$(( $x+1 ))
fi
# Run identify on the 2-color limited palette PNG8 to retrieve the mean for the gray channel
# In this case we are using coordinates and dynamic tile sizes according to the walker logic we have created in order to dynamically view a specific image area without creating actual tiles for it
# The result will be a decimal number (or zero) by which we can judge the visible object complexity in the current tile
local __currentbwmedian=$(identify -size "${IMAGEWIDTH}"x"${IMAGEHEIGHT}" -channel Gray -format "%[fx:255*mean]" "${TILESTORAGEPATH}${CLEANFILENAME##*/}_saliency_bw.png["${__currenttilewidth}"x"${__currenttileheight}"+$(echo $((${x}*${__currenttilewidth})))+$(echo $((${y}*${__currenttileheight})))]")
# If the gray channel median is below a defined threshold, the visible area in the current tile is very likely simple & rather monotonous and can safely be exposed to a higher compression rate
# Untouched JPGs simply stay at the defined default quality setting ($DEFAULTCOMPRESSIONRATE)
if (( $(echo "$__currentbwmedian < 0.825" | bc) )); then
# We experimented with blurring/smoothing of tiles here to enhance JPEG compression, but results were insignificant
if [ "${JPEGCNAME}" = "cjpeg" ]
then
${JPEGDECOMPRESSION_COMMAND} "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"."${FILEEXTENSION}" | ${JPEGCOMPRESSION_COMMAND} -progressive -optimize -quality ${HIGHCOMPRESSIONRATE} -outfile "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"_cjpeg."${FILEEXTENSION}"
mv "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"_cjpeg."${FILEEXTENSION}" "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"."${FILEEXTENSION}"
elif [ "${JPEGCNAME}" = "imagemagick" ]
then
${MOGRIFY_COMMAND} -quality ${HIGHCOMPRESSIONRATE} "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"."${FILEEXTENSION}"
elif [ "${JPEGCNAME}" = "jpegoptim" ]
then
${JPEGOPTIM_COMMAND} --max=${HIGHCOMPRESSIONRATE} --strip-all --strip-iptc --strip-icc "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"."${FILEEXTENSION}" >/dev/null 2>/dev/null
elif [ "${JPEGCNAME}" = "jpge" ]
then
${JPEGCOMPRESSION_COMMAND} "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"."${FILEEXTENSION}" "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"_jpge."${FILEEXTENSION}" ${HIGHCOMPRESSIONRATE} > /dev/null
mv "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"_jpge."${FILEEXTENSION}" "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"."${FILEEXTENSION}"
elif [ "${JPEGCNAME}" = "mozcjpeg" ]
then
${JPEGCOMPRESSION_COMMAND} -progressive -optimize -quality ${HIGHCOMPRESSIONRATE} -outfile "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"_mozjpeg."${FILEEXTENSION}" "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"."${FILEEXTENSION}"
mv "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"_mozjpeg."${FILEEXTENSION}" "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"."${FILEEXTENSION}"
fi
fi
# As the last action within the loop, increment the counter for the processed tile number. Use Base10 because with the padding of leading zeros, Bash would interprete the integer as Base8 per default.
__currenttilecount=$(( 10#$__currenttilecount + 1 ))
done
done
echo " ${__currenttilecount} tile ok."
}
# For the reassembly of the image, we need the count of rows and columns of tiles that were created
function calculate_tile_count () {
# Define local variables to work with
local __result=$1
local __currentimagedimension=$2
local __currenttilesize=$3
# Make use of Bash's behaviour of rounding down to see if we're tilecount = integer + 1
local __tilecountroundeddown=$(( $__currentimagedimension / $__currenttilesize ))
# Check if we need to +1 our integer because the decimal is larger than the integer
if (( $__currenttilesize * $__tilecountroundeddown < $__currentimagedimension )); then
local __tilecount=$(( $__tilecountroundeddown + 1 ))
else
local __tilecount=${__tilecountroundeddown}
fi
# Return result
eval $__result="'${__tilecount}'"
}
# Now that we know the number of rows+columns, we use montage to recombine the now partially compressed tiles into a new coherant JPEG image
function reassemble_tiles_into_final_image () {
local __filesize="$1"
echo -n "final image..."
# Use montage to reassemble the individual, partially optimized tiles into a new consistent JPEG image
FILEOUT="${CLEANPATH}${CLEANFILENAME##*/}${OUTPUTFILESUFFIX}.${FILEEXTENSION}"
${MONTAGE_COMMAND} -quiet -strip -quality "${DEFAULTCOMPRESSIONRATE}" -mode concatenate -tile "${TILECOLUMNS}x${TILEROWS}" $(find "${TILESTORAGEPATH}" -maxdepth 1 -type f -name "tile_tmp_*_${CLEANFILENAME##*/}.${FILEEXTENSION}" | sort) "$FILEOUT" >/dev/null 2>/dev/null
# During montage reassembly, the resulting image received bytes of padding due to the way the JPEG compression algorithm works on tiles not sized as a multiple of 8
# So we run jpegrescan on the final image to losslessly remove this padding and make the output JPG progressive
${JPEGRESCAN_COMMAND} -q -s -i "${FILEOUT}" "${FILEOUT}"
# Cleanup temporary files
rm ${TILESTORAGEPATH}${CLEANFILENAME##*/}_saliency_bw.png
# We are using find to circumvent issues on Kernel based shell limitations when iterating over a large number of files with rm
find "${TILESTORAGEPATH}" -maxdepth 1 -type f -name "tile_tmp_*_${CLEANFILENAME##*/}.${FILEEXTENSION}" -exec rm {} \;
echo " ok."
FILEOUTSIZE="$(stat -c %s ${FILEOUT})"
let "PERCENT=(${FILEOUTSIZE} * 100 + ${__filesize} / 2)/ ${__filesize}"
echo "${FILEOUTSIZE} ${FILEOUT} (${PERCENT}%)"
}
# Initiate preparatory checks
prepwork
# If the preparations worked, launch the main program
if (( VALIDJPEG )); then
main
fi
###############################################################################
# EOF
###############################################################################
================================================
FILE: adept.sh
================================================
#!/usr/bin/env bash
###############################################################################
#
# Bash script to automate adaptive JPEG compression using common CLI tools
#
# Usage: bash adept.sh /path/to/image.jpg
#
###############################################################################
#
# Brief overview of the mode of operation:
#
# The input JPG gets sliced into tiles, sized as a multiple of 8 due to the
# nature of JPG compression. The image is also run through a saliency
# detection algorithm and its resulting output further reduced to a
# 2-color black+white PNG.
#
# This bi-color PNG is ideal to measure tiles' gray channel mean value and use
# it as a single integer indicator to judge its perceivable complexity.
#
# Areas with low complexity contents are then exposed to heavier compression.
# At reassemlby, this leads to savings in image bytesize while maintaining
# good visual quality because no compression artefacts occur in areas of
# high-complexity or sharp contrasts.
#
###############################################################################
# Tools that need to be pre-installed:
#
# * Maximum Symmetric Surround Saliency Algorithm Binary
# http://github.com/technopagan/mss-saliency
#
# * ImageMagick >= v.6.6
#
# * MozJPEG
# http://github.com/mozilla/mozjpeg
# Expects Mozjpeg to be available under 'mozjpeg', e.g. via symlink
#
# * JPEGRescan Perl Script for lossless JPG compression
# http://github.com/kud/jpegrescan
#
# Note: Additonal tools are required to run Adept, such as "bc",
# "find", "mv", "rm" and Bash 3.x. As all of these tools are provided by lsbcore, core-utils
# or similar default packages, we can expect them to be always available.
#
###############################################################################
#
# This software is published under the BSD licence 3.0
#
# Copyright (c) 2013-2015, Tobias Baldauf
# All rights reserved.
#
# Mail: kontakt@tobias-baldauf.de
# Web: http://who.tobias.is/
# Twitter: @tbaldauf
#
# Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
#
# * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
#
# * Neither the name of the author nor the names of contributors may be used to endorse or promote products derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
###############################################################################
###############################################################################
# USER CONFIGURABLE PARAMETERS
###############################################################################
# Default JPG quality setting, either inherited or defined as an integer of 0-100
# Default: inherit
DEFAULTCOMPRESSIONRATE="inherit"
# JPEG quality setting for areas of the image deemed suitable for high compression in an integer of 0-100
# Default: 69
HIGHCOMPRESSIONRATE="69"
# Suffix string to attach to the output JPG filename, e.g. '_adept_compress'
# If deliberatly set empty (''), the input JPG will be replaced with the new compressed JPG
OUTPUTFILESUFFIX="_adept_compress"
###############################################################################
# RUNTIME VARIABLES (usually do not require tuning by user)
###############################################################################
# Accept the jpg filename as a parameter
FILE="$1"
# Retrieve clean filename without extension
CLEANFILENAME=${FILE%.jp*g}
# Retrieve only the file extension
FILEEXTENSION=${FILE##*.}
# Retrieve clean path directory without filename
CLEANPATH="${FILE%/*}"
# If the JPEG is in the same direcctory as Adept, empty the path variable
# Or if it is set, make sure the path has a trailing slash
if [ "$CLEANPATH" == "$FILE" ]; then
CLEANPATH=""
else
CLEANPATH="$CLEANPATH/"
fi
# Storage location for all temporary files during runtime
# Use locations like /dev/shm (/run/shm/ in Ubuntu) to save files in Shared Memory Space (RAM) to avoid disk i/o troubles
TILESTORAGEPATH="/dev/shm/"
# Check if the directory for temporary image storage during runtime actually exists (honoring symlinks)
# In case it does not, fall back to using "/tmp/" because it is very likely available on all Unix systems
if [ ! -d "$TILESTORAGEPATH" ]; then
TILESTORAGEPATH="/tmp/"
fi
# Square dimensions for all temporary tiles. Tile size heavily influences compression efficiency at the cost of runtime performance
# E.g. a tile size of 8 yields maximum compression results while taking several minutes of runtime
# If you chose to manually adjust tile size, only use multiples of 8 (8/16/32/64/128/256)
# Default: autodetect
TILESIZE="autodetect"
# Set locales to C (raw uninterpreted byte sequence)
# to avoid Illegal byte sequence errors and invalid number errors
export LANG=C LC_NUMERIC=C LC_COLLATE=C
###############################################################################
# COMMAND LINE OPTIONS
###############################################################################
# Allow user to specify -c, -h or -o on the command line for the compression rate, high compression rate, and file suffix
usage() {
echo "Usage: $0 [options] /path/to/jpeg/image.jpg
Options (and defaults):
-c INT Default compression rate ($DEFAULTCOMPRESSIONRATE)
-h INT High compression rate ($HIGHCOMPRESSIONRATE)
-o SUFF Output suffix ($OUTPUTFILESUFFIX)
"
exit 1
}
while getopts "c:h:o:" optionName; do
case "$optionName" in
c) DEFAULTCOMPRESSIONRATE="$OPTARG";;
h) HIGHCOMPRESSIONRATE="$OPTARG";;
o) OUTPUTFILESUFFIX="$OPTARG";;
\?) usage;;
esac
done
shift `expr $OPTIND - 1`
[ -z "$1" ] && usage
###############################################################################
# MAIN PROGRAM
###############################################################################
prepwork () {
find_tool IDENTIFY_COMMAND identify
find_tool CONVERT_COMMAND convert
find_tool MONTAGE_COMMAND montage
find_tool JPEGCOMPRESSION_COMMAND mozjpeg
find_tool JPEGRESCAN_COMMAND jpegrescan
find_tool SALIENCYDETECTOR_COMMAND SaliencyDetector
validate_image VALIDJPEG "${FILE}"
}
main () {
find_image_dimension IMAGEWIDTH "${FILE}" 'w'
find_image_dimension IMAGEHEIGHT "${FILE}" 'h'
optimize_tile_size TILESIZE ${TILESIZE} ${IMAGEWIDTH} ${IMAGEHEIGHT}
calculate_tile_count TILEROWS ${IMAGEHEIGHT} ${TILESIZE}
calculate_tile_count TILECOLUMNS ${IMAGEWIDTH} ${TILESIZE}
optimize_salient_regions_amount BLACKWHITETHRESHOLD "${FILE}"
${SALIENCYDETECTOR_COMMAND} -q -L0 -U${BLACKWHITETHRESHOLD} "${FILE}" "${TILESTORAGEPATH}${CLEANFILENAME##*/}_saliency_bw.png"
slice_image_to_ram "${FILE}" ${TILESIZE} ${TILESTORAGEPATH}
estimate_content_complexity_and_compress
reassemble_tiles_into_final_image
}
###############################################################################
# FUNCTIONS
###############################################################################
floatToInt() {
printf "%.0f\n" "$@"
}
# Find the proper handle for the required commandline tool
# This function can take an optional third parameter when being called to manually define the path to the CLI tool
function find_tool () {
# Define local variables to work with
local __result=$1
local __tool=$2
local __customtoolpath=$3
# Array of possible tool locations: name, name as ALL-CAPS, /usr/bin/name, /usr/local/bin/name and custom path
local __possibletoollocations=(${__tool} /usr/bin/${__tool} /usr/local/bin/${__tool} ${__customtoolpath})
# For each possible tool location, test if its actually available there
for i in "${__possibletoollocations[@]}"; do
local __commandlinetool=$(type -p $i)
# If 'type -p' returned something, we now have our proper handle
if [ "$__commandlinetool" ]; then
break
fi
done
# In case none of the given inputs works, apologize & quit
if [ ! "$__commandlinetool" ]; then
echo "Unable to find ${__tool}. Please ensure that it is installed. If necessary, set its CLI path+name in the find_tool function call and then retry."
exit 1
fi
# Return the result
eval $__result="'${__commandlinetool}'"
}
# Validate that we are working on an actual intact JPEG image before launch
function validate_image () {
# Define local variables to work with
local __result=$1
local __imagetovalidate=$2
# If the script is called without an input file, explain how to use it
# We don't "exit 1" here anymore because our unit tests source the script
# and would abort if "exit 1" was called
if [ ! -f "$__imagetovalidate" ]; then
local __validationresult=0
echo "Missing input JPEG. Usage: $0 /path/to/jpeg/image.jpg"
else
# Use IM identify to read the file magic of the input file to validate it's a JPEG
local __filemagic=$(${IDENTIFY_COMMAND} -format %m "$__imagetovalidate")
if [ "$__filemagic" == "JPEG" ] ; then
# Set a switch that it is ok to work on the input file, launching the main funtion
local __validationresult=1
fi
fi
# Return the result
eval $__result="'${__validationresult}'"
}
# Read width (%w) or height (%h) of the input image via IM identify
function find_image_dimension () {
# Define local variables to work with
local __result=$1
local __imagetomeasure=$2
local __dimensiontomeasure=$3
# Read the width or height of the input image into a global variable
local __imagedimension=$(${IDENTIFY_COMMAND} -format '%'${__dimensiontomeasure} ${__imagetomeasure})
# Return the result
eval $__result="'${__imagedimension}'"
}
# Tile size is the no.1 performance bottleneck for Adept, so it is important we pick an optimal tile size for the input image dimensions
# Also, the number of tiles to be recombined affects compression efficiency and salient areas within an image tend to have similar dimensional
# relations to total image size, so it makes sense to change tile size accordingly
function optimize_tile_size () {
# Define local variables to work with
local __result=$1
local __optimaltilesize=$2
local __currentimagewidth=$3
local __currentimageheight=$4
# The default "autodetect" setting causes Adept to find a suitable tile size according to image dimensions
if [ "$TILESIZE" == "autodetect" ] ; then
# Pick the smaller of the two dimensions of the image as the decisive integer for tile size
local __decisivedimension=${__currentimageheight}
if (( $IMAGEWIDTH < $__decisivedimension )); then
__decisivedimension=${__currentimagewidth}
fi
# For a series of sensible steps, change the tile size accordingly
if (( $__decisivedimension <= 512 )); then
__optimaltilesize="8"
elif (( $__decisivedimension >= 513 )) && (( $__decisivedimension <= 1024 )); then
__optimaltilesize="16"
elif (( $__decisivedimension >= 1025 )); then
__optimaltilesize="32"
else
__optimaltilesize="8"
fi
# In case the user has changed the configuration from "autodetect" to a custom setting, respect & return this instead
else
__optimaltilesize=${TILESIZE}
fi
# Return the result
eval $__result="'${__optimaltilesize}'"
}
function optimize_salient_regions_amount () {
# Define local variables to work with
local __result=$1
local __imagetomeasure=$2
local __lower_bound="0"
local __upper_bound="100"
local __current_threshold=$(( $__upper_bound/2 ))
local __mean_graychannel="0"
# Run the saliency detector function to retrieve the Median gray channel
calculate_salient_regions_amount __mean_graychannel "${__imagetomeasure}" ${__upper_bound}
__mean_graychannel=$(floatToInt $__mean_graychannel)
# If we didn't hit the sweet spot on our initial run, keep homing in on the ideal threshold value using binary search
while ( (( $__mean_graychannel > 40 )) || (( $__mean_graychannel < 20 )) ) && (( $__lower_bound < $__upper_bound-1 )); do
# If the Median is too low, reduce the upper threshold value to get more white pixels
if (( $__mean_graychannel < 20 )); then
__upper_bound=${__current_threshold}
# Else if the Median is too high, raise the threshold to get fewer white pixels
elif (( $__mean_graychannel > 40 )); then
__lower_bound=${__current_threshold}
fi
# Calculate the new middle threshold
__current_threshold=$(( ($__upper_bound-$__lower_bound)/2+$__lower_bound ))
# Rerun the saliency detector with a better estimated threshold value
calculate_salient_regions_amount __mean_graychannel "${__imagetomeasure}" ${__current_threshold}
__mean_graychannel=$(floatToInt $__mean_graychannel)
done
# Return result
eval $__result="'${__current_threshold}'"
}
# Measure the black/white median of a saliency mapped image to use it as an indicator for successfull saliency mapped contents
function calculate_salient_regions_amount () {
# Define local variables to work with
local __result=$1
local __imagetomeasure=$2
local __threshold=$3
# Use the MSS Saliency Detector with custom thresholds to generate a black+white salient map of an input image
# Then use the gray channel's mean as a single indicator to judge how much of the image's contents have been marked as salient
local __salient_amount=$(${SALIENCYDETECTOR_COMMAND} -q -L0 -U${__threshold} "${__imagetomeasure}" "png:-" | ${IDENTIFY_COMMAND} -channel Gray -format "%[fx:255*mean]" -)
# Return result
eval $__result="'${__salient_amount}'"
}
# Slice the input image into equally sized tiles
function slice_image_to_ram () {
# Define local variables to work with
local __filetoprocess=$1
local __currenttilesize=$2
local __currenttilestoragepath=$3
# If $DEFAULTCOMPRESSIONRATE is set to "inherit", discover the input JPG quality
if [ "$DEFAULTCOMPRESSIONRATE" == "inherit" ] ; then
DEFAULTCOMPRESSIONRATE=$(${IDENTIFY_COMMAND} -format "%Q" ${__filetoprocess})
fi
${CONVERT_COMMAND} "$__filetoprocess" -strip -quality "${DEFAULTCOMPRESSIONRATE}" -define jpeg:dct-method=default -crop "${__currenttilesize}"x"${__currenttilesize}" +repage +adjoin "${__currenttilestoragepath}tile_tmp_%06d_${CLEANFILENAME##*/}.${FILEEXTENSION}"
}
# For each tile, test if it is suitable for higher compression and if so, proceed
function estimate_content_complexity_and_compress () {
# Set up a counter so we keep track of the full name of the current temporary tile to work on
local __currenttilecount=0
# Let's create a walker that iterates over the sobeled and b/w reduced full size image
# This way, the edge detection happens only in memory and does not need additional tiles to be created on the filesystem
# The walker inputs X+Y coordinates and only analyses a single tile's size on that spot within the image
# The exception to this being when we are close to the image's end and we have to reduce tile size to whatever is left vertically or horizontally
for((y=0;y<$TILEROWS;y++)) ; do
for((x=0;x<$TILECOLUMNS;x++)) ; do
# Reset tile dimensions for each run because we need to check them anew each time
local __currenttileheight=${TILESIZE}
local __currenttilewidth=${TILESIZE}
# Prepend leading zeros to the counter so the integer matches the numbers handed out to the filename by ImageMagick
__currenttilecount=$(printf "%06d" $__currenttilecount);
# If we are nearing the end of the image height, reduce tile size to whatever is left vertically
if (( $y + 1 == $TILEROWS )) && (( $TILEROWS * $__currenttileheight > $IMAGEHEIGHT )); then
__currenttileheight=$(( (($y+1)*$TILESIZE) - $IMAGEHEIGHT ))
__currenttilerow=$(( $y+1 ))
fi
# And if we are nearing the end of the image width, reduce tile size to whatever is left horizontally
if (( $x + 1 == $TILECOLUMNS )) && (( $TILECOLUMNS * $__currenttilewidth > $IMAGEWIDTH )); then
__currenttilewidth=$(( (($x+1)*$TILESIZE) - $IMAGEWIDTH ))
__currenttilecolumn=$(( $x+1 ))
fi
# Run identify on the 2-color limited palette PNG8 to retrieve the mean for the gray channel
# In this case we are using coordinates and dynamic tile sizes according to the walker logic we have created in order to dynamically view a specific image area without creating actual tiles for it
# The result will be a decimal number (or zero) by which we can judge the visible object complexity in the current tile
local __currentbwmedian=$(identify -size "${IMAGEWIDTH}"x"${IMAGEHEIGHT}" -channel Gray -format "%[fx:255*mean]" "${TILESTORAGEPATH}${CLEANFILENAME##*/}_saliency_bw.png["${__currenttilewidth}"x"${__currenttileheight}"+$(echo $((${x}*${__currenttilewidth})))+$(echo $((${y}*${__currenttileheight})))]")
# If the gray channel median is below a defined threshold, the visible area in the current tile is very likely simple & rather monotonous and can safely be exposed to a higher compression rate
# Untouched JPGs simply stay at the defined default quality setting ($DEFAULTCOMPRESSIONRATE)
if (( $(echo "$__currentbwmedian < 0.825" | bc) )); then
# We experimented with blurring/smoothing of tiles here to enhance JPEG compression, but results were insignificant
${JPEGCOMPRESSION_COMMAND} -progressive -optimize -quality ${HIGHCOMPRESSIONRATE} -outfile "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"_mozjpeg."${FILEEXTENSION}" "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"."${FILEEXTENSION}"
mv "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"_mozjpeg."${FILEEXTENSION}" "${TILESTORAGEPATH}"tile_tmp_"${__currenttilecount}"_"${CLEANFILENAME##*/}"."${FILEEXTENSION}"
fi
# As the last action within the loop, increment the counter for the processed tile number. Use Base10 because with the padding of leading zeros, Bash would interprete the integer as Base8 per default.
__currenttilecount=$(( 10#$__currenttilecount + 1 ))
done
done
}
# For the reassembly of the image, we need the count of rows and columns of tiles that were created
function calculate_tile_count () {
# Define local variables to work with
local __result=$1
local __currentimagedimension=$2
local __currenttilesize=$3
# Make use of Bash's behaviour of rounding down to see if we're tilecount = integer + 1
local __tilecountroundeddown=$(( $__currentimagedimension / $__currenttilesize ))
# Check if we need to +1 our integer because the decimal is larger than the integer
if (( $__currenttilesize * $__tilecountroundeddown < $__currentimagedimension )); then
local __tilecount=$(( $__tilecountroundeddown + 1 ))
else
local __tilecount=${__tilecountroundeddown}
fi
# Return result
eval $__result="'${__tilecount}'"
}
# Now that we know the number of rows+columns, we use montage to recombine the now partially compressed tiles into a new coherant JPEG image
function reassemble_tiles_into_final_image () {
# Use montage to reassemble the individual, partially optimized tiles into a new consistent JPEG image
${MONTAGE_COMMAND} -quiet -strip -quality "${DEFAULTCOMPRESSIONRATE}" -mode concatenate -tile "${TILECOLUMNS}x${TILEROWS}" $(find "${TILESTORAGEPATH}" -maxdepth 1 -type f -name "tile_tmp_*_${CLEANFILENAME##*/}.${FILEEXTENSION}" | sort) "${CLEANPATH}${CLEANFILENAME##*/}${OUTPUTFILESUFFIX}".${FILEEXTENSION} >/dev/null 2>/dev/null
# During montage reassembly, the resulting image received bytes of padding due to the way the JPEG compression algorithm works on tiles not sized as a multiple of 8
# So we run jpegrescan on the final image to losslessly remove this padding and make the output JPG progressive
${JPEGRESCAN_COMMAND} -q -s -i "${CLEANPATH}${CLEANFILENAME##*/}${OUTPUTFILESUFFIX}".${FILEEXTENSION} "${CLEANPATH}${CLEANFILENAME##*/}${OUTPUTFILESUFFIX}".${FILEEXTENSION}
# Cleanup temporary files
rm ${TILESTORAGEPATH}${CLEANFILENAME##*/}_saliency_bw.png
# We are using find to circumvent issues on Kernel based shell limitations when iterating over a large number of files with rm
find "${TILESTORAGEPATH}" -maxdepth 1 -type f -name "tile_tmp_*_${CLEANFILENAME##*/}.${FILEEXTENSION}" -exec rm {} \;
}
# Initiate preparatory checks
prepwork
# If the preparations worked, launch the main program
if (( VALIDJPEG )); then
main
fi
###############################################################################
# EOF
###############################################################################
================================================
FILE: batch-compress.sh
================================================
#!/bin/env bash
ADEPT="jpeg-adept.sh"
if ! which "$ADEPT" > /dev/null; then echo Please install $ADEPT in your path.; fi
usage() {
echo "Usage: $0 [Options] PATH
Recursively compresses all the *.jpe?g files in PATH,
saving the originals.
Options (and defaults):
-d max-depth (no max directory depth)
-P parallel (1)
-n max-count (no max # of filesinfinite)
-x remove original images
-D turn on debug mode
The originals are saved as '_adept_save.jpg' files.
The flag file '_adept.flag' is created, so this program
can be run automatically. Removal of this file will
cause recompression of the compressed image.
-x is unsafe. It's strongly recommended to actually look at
the results for a while before removing the originals. Also
JPEG compression is lossy. You've been warned.
"
}
debug() {
if [ $debug ]; then echo "$@" 1>&2; fi
}
compress() {
local fil="$1"
local bn="${fil%.*}"
if [ "$fil" -nt "${bn}_adept_save.jpg" ]; then
# newer than both?
if [ "$fil" -nt "${bn}_adept_compress.jpg" ]; then
if [ "$fil" -nt "${bn}_adept.flag" ]; then
# this is a new file, so compress it
echo + jpeg-adept.sh \"$fil\" 1>&2
jpeg-adept.sh "$fil"
rm -f "${bn}_adept.flag"
fi
fi
# ok, we just compressed it
if [ "${bn}_adept_compress.jpg" -nt "${bn}.jpg" ]; then
# check to see if it's smaller
local var1=$(stat -c%s "$fil")
local var2=$(stat -c%s "${bn}_adept_compress.jpg")
debug "$fil: $var1, compressed: $var2"
if [ $var2 -lt $var1 ]; then
# save original, and
if [ $rmorig ]; then
debug "Destroying original '$fil'"
else
mv "$fil" "${bn}_adept_save.jpg"
fi
mv -f "${bn}_adept_compress.jpg" "$fil"
echo "ok"> "${bn}_adept.flag"
else
echo "toobig"> "${bn}_adept.flag"
unlink "${bn}_adept_compress.jpg"
fi
fi
fi
}
debug=""
unset count pll depth fil rmorig
while getopts "Dxd:f:r:n:P:" optionName; do
case "$optionName" in
P) pll="$OPTARG";;
d) depth="$OPTARG";;
D) debug=1;;
n) count="$OPTARG";;
f) fil="$OPTARG";;
x) rmorig=1;;
\?) usage;;
esac
done
unset xarg targ
if [ $debug ]; then
# turn on xtrace, xargs verbose, and pass-through -D
set -o xtrace
xarg="$xarg --verbose"
targ="$targ -D"
fi
if [ $rmorig ]; then
targ="$targ -x"
fi
if [ $depth ]; then
# turn into a 'find' arg
depth=" -maxdepth $depth"
fi
if [ $pll ]; then
# run commands in parallel
xarg="$xarg -P $pll"
fi
shift `expr $OPTIND - 1`
path="$1"
# inject head command
unset head
[ -n "$count" ] && head="head -n $count |"
if [ "$fil" ]; then
compress "$fil"
elif [ "$path" ]; then
# image stream
find "$path" $depth \( -name '*.jpg' -or -name '*.jpeg' \) -and -not -name '*_adept_compress.jpg' -and -not -name '*_adept_save.jpg' $farg | eval $head xargs $xarg -n 1 "$0" $targ -f
else
usage
exit 1
fi
================================================
FILE: man/man1/adept-jpeg.sh.1
================================================
.TH Adept 1 "21 Dec 2018" "0.0.70" "User Manual"
.SH NAME
adept-jpeg.sh \- the adaptive JPG Compressor
.SH DESCRIPTION
This script adaptively alters the compression level for areas within JPEGs (per-block)
to achieve optimized file size while maintaining a decent visual quality.
This script achieves a significantly reduced file size compared to standard tools
such as cjpeg while maintaining good visual quality, as can be measured via SSIM.
This is good news for the and thus Web Developer community to achieve
a great user experience on websites.
.SH SYNOPSIS
.B adept-jpeg.sh [ OPTIONS ] /path/to/image.jpeg
.SH OPTIONS
.TP
.B \-h
Displays help message.
.TP
.B \-c string
Name compressor: imagemagick (default), cjpeg, jpegoptim, jpge, mozcjpeg.
.TP
.B \-q 0..100
Compression rate of areas considered as important, default value is 'inherit'.
.TP
.B \-Q 0..100
Compression rate of areas deemed suitable for high compression, default is 69.
.TP
.B \-s suffix
This will append 'suffix' to filenames. Default value is '_adept_compress[_compressor]'.
.TP
.B \-t integer
tile size, accepted values are 8,16,32,64,128 and 256. The smaller the value, the better results but might take signifficantly more time to process. The default value is 'autodetect'.
.SH COMPRESSOR
.TP
.B cjpeg
Standard jpeg library.
.TP
.B imagemagick
Standard jpeg library using imagemagick.
.TP
.B jpegoptim
Standard jpeg library using jpegoptim.
.TP
.B jpge
Research JPEG encoder.
.TP
.B mozcjpeg
Mozjpeg library. A JPEG codec that provides increased compression for JPEG images.
.SH EXAMPLE
adept-jpeg.sh sample.jpg
.SH SEE ALSO
.BR imagemagick (1),
.BR djpeg (1),
.BR cjpeg (1),
.BR jpegoptim (1),
.BR jpegrescan (1),
.BR jpge (1),
.BR mozcjpeg (1),
.SH AUTHOR
Copyright (c) 2015, Tobias Baldauf.
All rights reserved.
================================================
FILE: unittests/tests_adept.bats
================================================
#!/usr/bin/env bats
source "${BATS_TEST_DIRNAME}/../adept.sh" >/dev/null 2>/dev/null
@test "Find Tools" {
run find_tool IDENTIFY_COMMAND identify
[ $status -eq 0 ]
}
# Make sure that we actually look for tools in other directories than $PATH
@test "Find Tools with path resolving" {
run find_tool IDENTIFY_COMMAND nonexistingcommand identify
[ $status -eq 0 ]
}
@test "Validate Input JPEG" {
validate_image VALIDJPEG "test.jpg"
result=${VALIDJPEG}
[ "$result" -eq 1 ]
}
@test "Read Image Dimension" {
find_image_dimension IMAGEWIDTH "test.jpg" 'w'
result=${IMAGEWIDTH}
[ "$result" -eq 512 ]
}
@test "Optimize Tile Size" {
optimize_tile_size TILESIZE 'autodetect' 513 513
result=${TILESIZE}
[ "$result" -eq 16 ]
}
@test "Slice Image into Tiles" {
CLEANFILENAME='test'
FILEEXTENSION='jpg'
slice_image_to_ram "test.jpg" 32 "$BATS_TMPDIR/"
TILESARRAY=($(find "$BATS_TMPDIR/" -maxdepth 1 -iregex ".*$CLEANFILENAME.jpe*g"))
result=${#TILESARRAY[@]}
[ "$result" -eq 256 ]
rm -f "$BATS_TMPDIR/.*.jpe*g"
}
@test "Calculate Tile Count for Reassembly" {
calculate_tile_count TILEROWS 512 32
result=${TILEROWS}
[ "$result" -eq 16 ]
}
# NEEDS IMPLEMENTING: optimize_salient_regions_amount
# NEEDS REFACTORING: reassemble_tiles_into_final_image
# NEEDS REFACTORING: estimate_tile_content_complexity_and_compress
# Uses globals ${TILESTORAGEPATH} and ${FILEEXTENSION} etc. Replace with locals.
# Currently has two steps: recompile + recompress. This should be two seperate functions.
gitextract_4klx5xfu/
├── LICENSE
├── README.md
├── adept-jpeg.sh
├── adept.sh
├── batch-compress.sh
├── man/
│ └── man1/
│ └── adept-jpeg.sh.1
└── unittests/
└── tests_adept.bats
Condensed preview — 7 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (62K chars).
[
{
"path": "LICENSE",
"chars": 1614,
"preview": "This software is published under the BSD licence 3.0\n\nCopyright (c) 2013-2019, Tobias Baldauf All rights reserved.\n\nMail"
},
{
"path": "README.md",
"chars": 5036,
"preview": "Adept - the adaptive JPG Compressor\n====================\n\n## Quick Start\n\n* Please install [a MSS saliency algorithm bin"
},
{
"path": "adept-jpeg.sh",
"chars": 24823,
"preview": "#!/usr/bin/env bash\n\n###############################################################################\n#\n# Bash script to "
},
{
"path": "adept.sh",
"chars": 21213,
"preview": "#!/usr/bin/env bash\n\n###############################################################################\n#\n# Bash script to "
},
{
"path": "batch-compress.sh",
"chars": 2815,
"preview": "#!/bin/env bash\n\nADEPT=\"jpeg-adept.sh\"\n\nif ! which \"$ADEPT\" > /dev/null; then echo Please install $ADEPT in your path.; "
},
{
"path": "man/man1/adept-jpeg.sh.1",
"chars": 1796,
"preview": ".TH Adept 1 \"21 Dec 2018\" \"0.0.70\" \"User Manual\"\n.SH NAME\nadept-jpeg.sh \\- the adaptive JPG Compressor\n.SH DESCRIPTION\nT"
},
{
"path": "unittests/tests_adept.bats",
"chars": 1524,
"preview": "#!/usr/bin/env bats\n\nsource \"${BATS_TEST_DIRNAME}/../adept.sh\" >/dev/null 2>/dev/null\n\n@test \"Find Tools\" {\n run find_t"
}
]
About this extraction
This page contains the full source code of the technopagan/adept-jpg-compressor GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 7 files (57.4 KB), approximately 15.2k tokens. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.