Full Code of ccampbell/html-muncher for AI

Repository: ccampbell/html-muncher
Branch: master
Commit: 86c425b0a3db
Files: 16
Total size: 51.6 KB

Directory structure:
gitextract_o1tbrn8m/

├── .gitignore
├── README
├── demo/
│   ├── css/
│   │   ├── stylesheet1.css
│   │   └── stylesheet2.css
│   ├── js/
│   │   └── test.js
│   ├── single-file/
│   │   └── view-with-inline-styles.html
│   └── views/
│       ├── view1.html
│       └── view2.html
├── munch
├── muncher/
│   ├── __init__.py
│   ├── config.py
│   ├── muncher.py
│   ├── sizetracker.py
│   ├── util.py
│   └── varfactory.py
└── setup.py

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
*.pyc
*.opt.html
*.opt.js
*.opt.css
demo/css_opt
demo/js_opt
demo/views_opt


================================================
FILE: README
================================================
--------------
  ABOUT
--------------

HTML Muncher is a Python utility that rewrites CSS, HTML, and JavaScript files in order to save precious bytes and obfuscate your code

if your stylesheet starts out looking like this:

.file2 #special {
    font-size: 1.5em;
    color: #F737FF;
}

.file2 #special2 {
    letter-spacing: 0;
}

.box {
    border: 2px solid #aaa;
    -webkit-border-radius: 5px;
    background: #eee;
    padding: 5px;
}

it will be rewritten as

.a #a {
    font-size: 1.5em;
    color: #F737FF;
}

.a #b {
    letter-spacing: 0;
}

.b {
    border: 2px solid #aaa;
    -webkit-border-radius: 5px;
    background: #eee;
    padding: 5px;
}


--------------
 INSTALLATION
--------------

easy_install http://htmlmuncher.com/htmlmuncher.egg

OR:

download the source from http://github.com/ccampbell/html-muncher
cd html-muncher
python setup.py install


--------------
 USAGE
--------------
http://htmlmuncher.com/#usage

OR:

munch --help


--------------
 EXAMPLES
--------------

to update a bunch of stylesheets and views:
munch --css demo/css --html demo/views

to update a single file with inline styles/javascript:
munch --html demo/single-file/view-with-inline-styles.html

you can also select specific files:
munch --css file1.css,file2.css --html view1.html,view2.html

or you can mix and match files and directories
munch --css /my/css/directory,global.css --html /view/directory1,/view/directory2,/view/directory3,template.html


================================================
FILE: demo/css/stylesheet1.css
================================================
body {
    font-family: helvetica;
}

h1 {
    text-align: center;
    font-size: 2em;
}

.red {
    color: red;
}

.blue {
    color: blue;
}

.green {
    color: green;
}

.box.purple {
    color: purple;
}

.underline {
    text-decoration: underline;
}

#special {
    color: orange;
    font-style: italic;
}

.italic {
    font-style: italic;
}

#special2 {
    letter-spacing: .5em;
    color: grey;
}

#new_id, #special, #special2 {
    font-size: 1em;
}

================================================
FILE: demo/css/stylesheet2.css
================================================
.file2 #special {
    font-size: 1.5em;
    color: #F737FF;
}

.file2 #special2 {
    letter-spacing: 0;
}

.box {
    border: 2px solid #aaa;
    -webkit-border-radius: 5px;
    background: #eee;
    padding: 5px;
}

================================================
FILE: demo/js/test.js
================================================
$ = {
    qs: function(query) {
        return document.querySelector(query);
    }
};

window.onload = function()
{
    $.qs("#special").innerHTML = "new text for this paragraph";
    document.getElementById("special").innerHTML = "change it again";
    var italic = document.getElementsByClassName('italic');

    // mootools
    var test = $('test');
    if (test.hasClass("dont_know")) {
        test.removeClass("dont_know");
        test.addClass('now_i_know');
    }

    var class_thing = $('class_thing');
    class_thing.addClass(test, "whatever");
    class_thing.removeClass(test, "whatever");
    $.qs(".dont_know", class_thing).value;
    var cool = $.qs("#one_id.class_thing", test);
    var another_weird_thing = $.qs(".class1.class2 #another_id");
    $.qs(".selector1 > .selector2 .selector3");
    var test = document.querySelector(".selector1");
}


================================================
FILE: demo/single-file/view-with-inline-styles.html
================================================
<html>
<head>
    <title>view with inline styles</title>
    <style type="text/css">
        .content {
            padding: 25px 0px 50px 0px;
            overflow: hidden;
        }

        a {
            color: #FF5F1E;
        }

        .content, #footer {
            width: 800px;
        }

        ul.menu {
            width: 200px;
            float: left;
        }

        #main {
            width: 500px;
            float: left;
        }

        ul {
            padding: 0px;
            margin: 0px;
            list-style: none;
        }

        ul li {
            margin-top: 20px;
        }

        ul li.first_child {
            margin-top: 0px;
        }

        .box {
            width: 500px;
            border: 1px solid #aaa;
            padding: 5px;
            -moz-border-radius: 5px;
            -webkit-border-radius: 5px;
            margin-bottom: 15px;
            background: #fff url("../test.jpg");
        }

        .green {
            background: #ACFFB9;
        }

        .blue {
            background: #C5D7FF;
        }

        #footer ul {
            text-align: center;
        }

        #footer ul li {
            display: inline;
            margin-left: 20px;
        }

        #footer ul li.first_child {
            margin-left: 0px;
        }

        p.special span {
            font-weight: bold;
        }
    </style>
</head>
<body>
    <h1>test page with inline styles</h1>

    <div class="content">
        <ul class="menu">
            <li class="first_child"><a href="#">link 1</a></li>
            <li class="unused_class"><a href="#">link 2</a></li>
            <li><a href="#">link 3</a></li>
            <li><a href="#">link 4</a></li>
            <li><a href="#">link 5</a></li>
        </ul>

        <div id="main">
            <div class="box green">
                <p>this is a green box</p>
            </div>

            <div class="box blue">
                <p>this is a blue box</p>
            </div>

            <p class="special">this is <span>paragraph</span></p>
        </div>
    </div>

    <div id="footer">
        <ul>
            <li class="first_child"><a href="#">footer link 1</a></li>
            <li><a href="#">footer link 2</a></li>
            <li><a href="#">footer link 3</a></li>
            <li><a href="#">footer link 4</a></li>
        </ul>
    </div>
    <script type="text/javascript">
        $('.special').hide();
        $("#main .box").hide();
    </script>
</body>
</html>

================================================
FILE: demo/views/view1.html
================================================
<html>
<head>
    <title>test page 1 changed</title>
    <link href="../css/stylesheet1.css" rel="stylesheet" type="text/css" />
</head>
<body>
    <h1>this is the first test page</h1>
    <p class="red">Assertively leverage existing scalable growth strategies with revolutionary value. Distinctively recaptiualize top-line models rather than leveraged e-commerce. Quickly engineer orthogonal e-markets via holistic human capital.</p>
    <p class="box purple">this is a test with two classes</p>
    <p class="blue">Appropriately incubate collaborative imperatives after team building networks. Uniquely enhance sticky e-services for value-added solutions. Seamlessly parallel task value-added quality vectors before state of the art methods of empowerment.</p>
    <p class="green underline">Interactively strategize plug-and-play platforms whereas efficient infrastructures. Synergistically negotiate user-centric metrics rather than worldwide quality vectors. Holisticly deliver bleeding-edge leadership vis-a-vis world-class architectures.</p>
    <p id="special">Professionally iterate multifunctional systems before optimal materials. Efficiently reintermediate wireless total linkage with distributed portals. Globally exploit resource-leveling human capital without economically sound infomediaries.</p>
    <p id="special2" class="italic">Intrinsicly streamline user-centric users before visionary scenarios. Dynamically repurpose seamless channels via multifunctional process improvements. Professionally synthesize exceptional infrastructures without premier vortals.</p>
    <script type="text/javascript" src="../js/test.js"></script>
</body>
</html>

================================================
FILE: demo/views/view2.html
================================================
<html>
<head>
    <title>test page 2</title>
    <link href="../css/stylesheet1.css" rel="stylesheet" type="text/css" />
    <link href="../css/stylesheet2.css" rel="stylesheet" type="text/css" />
</head>
<body class="file2">
    <h1>this is the second test page</h1>
    <p class="red">Assertively leverage existing scalable growth strategies with revolutionary value. Distinctively recaptiualize top-line models rather than leveraged e-commerce. Quickly engineer orthogonal e-markets via holistic human capital.</p>
    <p class="box blue">Appropriately incubate collaborative imperatives after team building networks. Uniquely enhance sticky e-services for value-added solutions. Seamlessly parallel task value-added quality vectors before state of the art methods of empowerment.</p>
    <p id="special">Professionally iterate multifunctional systems before optimal materials. Efficiently reintermediate wireless total linkage with distributed portals. Globally exploit resource-leveling human capital without economically sound infomediaries.</p>
    <p id="special2" class="italic">Intrinsicly streamline user-centric users before visionary scenarios. Dynamically repurpose seamless channels via multifunctional process improvements. Professionally synthesize exceptional infrastructures without premier vortals.</p>
</body>
</html>


================================================
FILE: munch
================================================
#!/usr/bin/env python

# Copyright 2011 Craig Campbell
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import sys
from muncher.config import Config
from muncher.muncher import Muncher

config = Config()
config.processArgs()
muncher = Muncher(config)
muncher.run()


================================================
FILE: muncher/__init__.py
================================================


================================================
FILE: muncher/config.py
================================================
#!/usr/bin/env python
# Copyright 2011 Craig Campbell
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import sys, getopt
from muncher import Muncher

class Config(object):
    """configuration object for handling all config options for html-muncher"""
    def __init__(self):
        """config object constructor

        Returns:
        void

        """
        self.css = []
        self.views = []
        self.js = []
        self.ignore = []
        self.class_selectors = ["getElementsByClassName", "hasClass", "addClass", "removeClass"]
        self.id_selectors = ["getElementById"]
        self.custom_selectors = ["document.querySelector"]
        self.framework = None
        self.view_extension = "html"
        self.js_manifest = None
        self.show_savings = False
        self.compress_html = False
        self.rewrite_constants = False
        self.verbose = False

    def getArgCount(self):
        """gets the count of how many arguments are present

        Returns:
        int

        """
        return len(sys.argv)

    def setIgnore(self, value):
        """sets what classes and ids we should ignore and not shorten

        Arguments:
        value -- comma separated list of classes or ids

        Returns:
        void

        """
        for name in value.split(","):
            self.ignore.append(name)

    def setCustomSelectors(self, value):
        for value in value.split(","):
            self.custom_selectors.append(value.lstrip("."))

    def addClassSelectors(self, value):
        for value in value.split(","):
            self.class_selectors.append(value)

    def addIdSelectors(self, value):
        for value in value.split(","):
            self.id_selectors.append(value)

    def setCssFiles(self, value):
        for value in value.split(","):
            self.css.append(value.rstrip("/"))

    def setViewFiles(self, value):
        for value in value.split(","):
            self.views.append(value.rstrip("/"))

    def setJsFiles(self, value):
        for value in value.split(","):
            self.js.append(value.rstrip("/"))

    def setFramework(self, name):
        self.framework = name.lower()
        if self.framework == "jquery":
            self.custom_selectors.append("$")
            self.custom_selectors.append("jQuery")
        elif self.framework == "mootools":
            self.id_selectors.append("$")
            self.custom_selectors.append("getElement")

    def processArgs(self):
        """processes arguments passed in via command line and sets config settings accordingly

        Returns:
        void

        """
        try:
            opts, args = getopt.getopt(sys.argv[1:], "", ["css=", "views=", "html=", "js=", "help", "view-ext=", "ignore=", "framework=", "selectors=", "class-selectors=", "id-selectors=", "compress-html", "show-savings", "verbose", "js-manifest=", "rewrite-constants"])
        except:
            Muncher.showUsage()

        views_set = False

        for key, value in opts:
            if key == "--help":
                Muncher.showUsage()
            elif key == "--css":
                self.setCssFiles(value)
            elif key == "--views" or key == "--html":
                views_set = True
                self.setViewFiles(value)
            elif key == "--js":
                self.setJsFiles(value)
            elif key == "--ignore":
                self.setIgnore(value)
            elif key == "--view-ext":
                self.view_extension = value
            elif key == "--framework":
                self.setFramework(value)
            elif key == "--selectors":
                self.setCustomSelectors(value)
            elif key == "--class-selectors":
                self.addClassSelectors(value)
            elif key == "--id-selectors":
                self.addIdSelectors(value)
            elif key == "--compress-html":
                self.compress_html = True
            elif key == "--show-savings":
                self.show_savings = True
            elif key == "--verbose":
                self.verbose = True
            elif key == "--js-manifest":
                self.js_manifest = value
            elif key == "--rewrite-constants":
                self.rewrite_constants = True

        # you have to at least have a view
        if views_set is False:
            Muncher.showUsage()


================================================
FILE: muncher/muncher.py
================================================
#!/usr/bin/env python
# Copyright 2011 Craig Campbell
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import sys, re, glob, os
from operator import itemgetter
from util import Util
from varfactory import VarFactory
from sizetracker import SizeTracker

class Muncher(object):
    def __init__(self, config):
        """constructor

        Returns:
        void

        """
        self.id_counter = {}
        self.class_counter = {}
        self.id_map = {}
        self.class_map = {}
        self.config = config

    @staticmethod
    def showUsage():
        """shows usage information for this script"""
        print "\n---------------------------------"
        print " html-muncher"
        print "---------------------------------"

        print "\n" + '\033[91m' + "USAGE:" + '\033[0m'
        print "munch --css file1.css,/path/to/css1,file2.css,file3.css --html /path/to/views1,file1.html,/path/to/views2/,file3.html --js main.js,/path/to/js"
        print "\n" + '\033[91m' + "REQUIRED ARGUMENTS:" + '\033[0m'
        print "--html {path/to/views}       html files to rewrite (comma separated list of directories and files)"
        print "\n" + '\033[91m' + "OPTIONAL ARGUMENTS:" + '\033[0m'
        print "--css {path/to/css}          css files to rewrite (comma separated list of directories and files)"
        print ""
        print "--js {path/to/js}            js files to rewrite (comma separated list of directories and files)"
        print ""
        print "--view-ext {extension}       sets the extension to look for in the view directory (defaults to html)"
        print ""
        print "--ignore {classes,ids}       comma separated list of classes or ids to ignore when rewriting css (ie .sick_class,#sweet_id)"
        print ""
        print "--compress-html              strips new line characters to compress html files specified with --html"
        print "                             be careful when using this becuase it has not been thoroughly tested"
        print ""
        print "--framework                  name of js framework to use for selectors (currently only jquery or mootools)"
        print ""
        print "--selectors                  comma separated custom selectors using css selectors"
        print "                             for example if you have $.qs(\"#test .div\") this param would be qs"
        print ""
        print "--id-selectors               comma separated id selectors with strings"
        print "                             for example if you are using .addId(\"test\") this param would be addId"
        print ""
        print "--class-selectors            comma separated class selectors with strings"
        print "                             for example if you have selectClass(\"my_class\") this param would be selectClass"
        print ""
        print "--js-manifest                path to a js file containing class name/id constants"
        print ""
        print "--rewrite-constants          when using a manifest file this will take any constants with values as strings"
        print "                             and rewrite the values to be numbers"
        print ""
        print "--show-savings               will output how many bytes were saved by munching"
        print ""
        print "--verbose                    output more information while the script runs"
        print ""
        print "--help                       shows this menu\n"
        sys.exit(2)

    def run(self):
        """runs the optimizer and does all the magic

        Returns:
        void

        """
        self.output("searching for classes and ids...", False)

        if self.config.js_manifest is not None:
            self.outputJsWarnings()

        self.processCss()
        self.processViews()

        if self.config.js_manifest is None:
            self.processJs()
        else:
            self.processJsManifest()

        self.output("mapping classes and ids to new names...", False)
        # maps all classes and ids found to shorter names
        self.processMaps()

        # optimize everything
        self.output("munching css files...", False)
        self.optimizeFiles(self.config.css, self.optimizeCss)

        self.output("munching html files...", False)
        self.optimizeFiles(self.config.views, self.optimizeHtml, self.config.view_extension, self.config.compress_html)

        self.output("munching js files...", False)

        if self.config.js_manifest is None:
            self.optimizeFiles(self.config.js, self.optimizeJavascript)
        else:
            self.optimizeJsManifest()

        self.output("done", False)

        if self.config.show_savings:
            self.output(SizeTracker.savings(), False)

    def outputJsWarnings(self):
        pass

    def output(self, text, verbose_only = True):
        """outputs text during the script run

        Arguments:
        text -- string of text to output
        verbose_only -- should we only show this in verbose mode?

        Returns:
        void

        """
        if verbose_only and not self.config.verbose:
            return

        print text

    def processCssDirectory(self, file):
        """processes a directory of css files

        Arguments:
        file -- path to directory

        Returns:
        void

        """
        if ".svn" in file:
            return

        for dir_file in Util.getFilesFromDir(file):
            if Util.isDir(dir_file):
                self.processCssDirectory(dir_file)
                continue

            self.processCssFile(dir_file)

    def processCss(self):
        """gets all css files from config and processes them to see what to replace

        Returns:
        void

        """
        files = self.config.css
        for file in files:
            if not Util.isDir(file):
                self.processCssFile(file)
                continue
            self.processCssDirectory(file)

    def processViewDirectory(self, file):
        """processes a directory of view files

        Arguments:
        file -- path to directory

        Returns:
        void

        """
        if ".svn" in file:
            return

        for dir_file in Util.getFilesFromDir(file):
            if Util.isDir(dir_file):
                self.processViewDirectory(dir_file)
                continue

            self.processView(dir_file)

    def processViews(self):
        """processes all view files

        Returns:
        void

        """
        files = self.config.views
        for file in files:
            if not Util.isDir(file):
                self.processView(file)
                continue
            self.processViewDirectory(file)

    def processJsDirectory(self, file):
        """processes a directory of js files

        Arguments:
        file -- path to directory

        Returns:
        void

        """
        if ".svn" in file:
            return

        for dir_file in Util.getFilesFromDir(file):
            if Util.isDir(dir_file):
                self.processJsDirectory(dir_file)
                continue
            self.processJsFile(dir_file)

    def processJs(self):
        """gets all js files from config and processes them to see what to replace

        Returns:
        void

        """
        files = self.config.js
        for file in files:
            if not Util.isDir(file):
                self.processJsFile(file)
                continue
            self.processJsDirectory(file)

    def processView(self, file):
        """processes a single view file

        Arguments:
        file -- path to directory

        """
        self.processCssFile(file, True)
        self.processJsFile(file, True)

    def processCssFile(self, path, inline = False):
        """processes a single css file to find all classes and ids to replace

        Arguments:
        path -- path to css file to process

        Returns:
        void

        """
        contents = Util.fileGetContents(path)
        if inline is True:
            blocks = self.getCssBlocks(contents)
            contents = ""
            for block in blocks:
                contents = contents + block

        ids_found = re.findall(r'((?<!\:\s)(?<!\:)#\w+)(\.|\{|,|\s|#)', contents, re.DOTALL)
        classes_found = re.findall(r'(?!\.[0-9])\.\w+', contents)
        self.addIds(ids_found)
        self.addClasses(classes_found)

    def processJsFile(self, path, inline = False):
        """processes a single js file to find all classes and ids to replace

        Arguments:
        path -- path to css file to process

        Returns:
        void

        """
        contents = Util.fileGetContents(path)
        if inline is True:
            blocks = self.getJsBlocks(contents)
            contents = ""
            for block in blocks:
                contents = contents + block

        selectors = self.getJsSelectors(contents, self.config)
        for selector in selectors:
            if selector[0] in self.config.id_selectors:
                if ',' in selector[2]:
                    id_to_add = re.search(r'(\'|\")(.*?)(\'|\")', selector[2])
                    if id_to_add is None:
                        continue

                    if not id_to_add.group(2):
                        continue

                    self.addId("#" + id_to_add.group(2))

                # if this is something like document.getElementById(variable) don't add it
                if not '\'' in selector[2] and not '"' in selector[2]:
                    continue

                self.addId("#" + selector[2].strip("\"").strip("'"))
                continue

            if selector[0] in self.config.class_selectors:
                class_to_add = re.search(r'(\'|\")(.*?)(\'|\")', selector[2])
                if class_to_add is None:
                    continue

                if not class_to_add.group(2):
                    continue

                self.addClass("." + class_to_add.group(2))
                continue

            if selector[0] in self.config.custom_selectors:
                matches = re.findall(r'((#|\.)[a-zA-Z0-9_]*)', selector[2])
                for match in matches:
                    if match[1] == "#":
                        self.addId(match[0])
                        continue

                    self.addClass(match[0])

    def processJsManifest(self):
        contents = Util.fileGetContents(self.config.js_manifest)
        ids = re.findall(r'\s+?(var\s)?\${1}([A-Z0-9_]+)\s?=\s?[\'|\"](.*?)[\'|\"][,|;]', contents)
        classes = re.findall(r'\s+?(var\s)?\${2}([A-Z0-9_]+)\s?=\s?[\'|\"](.*?)[\'|\"][,|;]', contents)

        self.manifest_ids = {}
        self.manifest_classes = {}

        for id in ids:
            self.addId("#" + id[2])
            self.manifest_ids[id[1]] = id[2]

        for manifest_class in classes:
            self.addClass("." + manifest_class[2])
            self.manifest_classes[manifest_class[1]] = manifest_class[2]

    def optimizeJsManifest(self):
        contents = Util.fileGetContents(self.config.js_manifest)

        for key, value in self.manifest_ids.items():
            if "#" + value in self.id_map:
                contents = re.sub(r'((?<!\$)\${1}[A-Z0-9_]+\s?=\s?[\'|\"])(' + value + ')([\'|\"][,|;])', r'\1' + self.id_map["#" + value].replace("#", "") + r'\3', contents)

        for key, value in self.manifest_classes.items():
            if "." + value in self.class_map:
                contents = re.sub(r'(\${2}[A-Z0-9_]+\s?=\s?[\'|\"])(' + value + ')([\'|\"][,|;])', r'\1' + self.class_map["." + value].replace(".", "") + r'\3', contents)

        if self.config.rewrite_constants:
            constants = re.findall(r'(\s+?(var\s)?([A-Z0-9_]+)\s?=\s?[\'|\"](.*?)[\'|\"][,|;])', contents)
            new_constants = {}
            i = 0
            for constant in constants:
                # underscore variables are ignored
                if constant[2][0] == "_":
                    continue

                i += 1
                new_constant = re.sub(r'=(.*)([,|;])','= ' + str(i) + r'\2', constant[0])
                contents = contents.replace(constant[0], new_constant)

        new_manifest = Util.prependExtension("opt", self.config.js_manifest)
        Util.filePutContents(new_manifest, contents)

        if self.config.show_savings:
            SizeTracker.trackFile(self.config.js_manifest, new_manifest)

    def processMaps(self):
        """loops through classes and ids to process to determine shorter names to use for them
        and creates a dictionary with these mappings

        Returns:
        void

        """
        # reverse sort so we can figure out the biggest savings
        classes = self.class_counter.items()
        classes.sort(key = itemgetter(1), reverse=True)

        for class_name, savings in classes:
            small_class = "." + VarFactory.getNext("class")

            # adblock extensions may block class "ad" so we should never generate it
            # also if the generated class already exists as a class to be processed
            # we can't use it or bad things will happen
            while small_class == ".ad" or Util.keyInTupleList(small_class, classes):
                small_class = "." + VarFactory.getNext("class")

            self.class_map[class_name] = small_class

        ids = self.id_counter.items()
        ids.sort(key = itemgetter(1), reverse=True)

        for id, savings in ids:
            small_id = "#" + VarFactory.getNext("id")

            # same holds true for ids as classes
            while small_id == "#ad" or Util.keyInTupleList(small_id, ids):
                small_id = "#" + VarFactory.getNext("id")

            self.id_map[id] = small_id

    def incrementIdCounter(self, name):
        """called for every time an id is added to increment the bytes we will save

        Arguments:
        name -- string of id

        Returns:
        void

        """
        length = len(name)

        if not name in self.id_counter:
            self.id_counter[name] = length
            return

        self.id_counter[name] += length

    def incrementClassCounter(self, name):
        """called for every time a class is added to increment the bytes we will save

        Arguments:
        name -- string of class

        Returns:
        void

        """
        length = len(name)

        if not name in self.class_counter:
            self.class_counter[name] = length
            return

        self.class_counter[name] += length

    def incrementCounter(self, name):
        """called everytime a class or id is added

        Arguments:
        name -- string of class or id name

        Returns:
        void

        """
        if name[0] == "#":
            return self.incrementIdCounter(name)

        return self.incrementClassCounter(name)

    def addId(self, id):
        """adds a single id to the master list of ids

        Arguments:
        id -- single id to add

        Returns:
        void

        """
        if id in self.config.ignore or id is '#':
            return

        # skip $ ids from manifest
        if self.config.js_manifest is not None and id[1] == '$':
            return

        self.incrementCounter(id)

    def addIds(self, ids):
        """adds a list of ids to the master id list to replace

        Arguments:
        ids -- list of ids to add

        Returns:
        void

        """
        for id in ids:
            self.addId(id[0])

    def addClass(self, class_name):
        """adds a single class to the master list of classes

        Arguments:
        class_name -- single class to add

        Returns:
        void

        """
        if class_name in self.config.ignore or class_name is '.':
            return

        # skip $$ class names from manifest
        if self.config.js_manifest is not None and class_name[1:2] == '$$':
            return

        self.incrementCounter(class_name)

    def addClasses(self, classes):
        """adds a list of classes to the master class list to replace

        Arguments:
        classes -- list of classes to add

        Returns:
        void

        """
        for class_name in classes:
            self.addClass(class_name)

    def optimizeFiles(self, paths, callback, extension = "", minimize = False):
        """loops through a bunch of files and directories, runs them through a callback, then saves them to disk

        Arguments:
        paths -- array of files and directories
        callback -- function to process each file with

        Returns:
        void

        """
        for file in paths:
            if not Util.isDir(file):
                self.optimizeFile(file, callback, minimize)
                continue

            self.optimizeDirectory(file, callback, extension, minimize)

    def optimizeFile(self, file, callback, minimize = False, new_path = None, prepend = "opt"):
        """optimizes a single file

        Arguments:
        file -- path to file
        callback -- function to run the file through
        minimize -- whether or not we should minimize the file contents (html)
        prepend -- what extension to prepend

        Returns:
        void

        """
        content = callback(file)
        if new_path is None:
            new_path = Util.prependExtension(prepend, file)
        if minimize is True:
            self.output("minimizing " + file)
            content = self.minimize(content)
        self.output("optimizing " + file + " to " + new_path)
        Util.filePutContents(new_path, content)

        if self.config.show_savings:
            SizeTracker.trackFile(file, new_path)

    def prepareDirectory(self, path):
        if ".svn" in path:
            return True

        if Util.isDir(path):
            return False

        Util.unlinkDir(path)
        self.output("creating directory " + path)
        os.mkdir(path)
        return False

    def optimizeDirectory(self, path, callback, extension = "", minimize = False):
        """optimizes a directory

        Arguments:
        path -- path to directory
        callback -- function to run the file through
        extension -- extension to search for in the directory
        minimize -- whether or not we should minimize the file contents (html)

        Returns:
        void

        """
        directory = path + "_opt"
        skip = self.prepareDirectory(directory)
        if skip is True:
            return

        for dir_file in Util.getFilesFromDir(path, extension):
            if Util.isDir(dir_file):
                self.optimizeSubdirectory(dir_file, callback, directory, extension, minimize)
                continue

            new_path = directory + "/" + Util.getFileName(dir_file)
            self.optimizeFile(dir_file, callback, minimize, new_path)

    def optimizeSubdirectory(self, path, callback, new_path, extension = "", minimize = False):
        """optimizes a subdirectory within a directory being optimized

        Arguments:
        path -- path to directory
        callback -- function to run the file through
        new_path -- path to optimized parent directory
        extension -- extension to search for in the directory
        minimize -- whether or not we should minimize the file contents (html)

        Returns:
        void

        """
        subdir_path = new_path + "/" + path.split("/").pop()
        skip = self.prepareDirectory(subdir_path)
        if skip is True:
            return

        for dir_file in Util.getFilesFromDir(path, extension):
            if Util.isDir(dir_file):
                self.optimizeSubdirectory(dir_file, callback, subdir_path, extension, minimize)
                continue

            new_file_path = subdir_path + "/" + Util.getFileName(dir_file)
            self.optimizeFile(dir_file, callback, minimize, new_file_path)

    def minimize(self, content):
        content = re.sub(r'\n', '', content)
        content = re.sub(r'\s\s+', '', content)
        content = re.sub(r'(<!--(?!\[if)(.*?)-->)', '', content, re.MULTILINE)
        return content

    def optimizeCss(self, path):
        """replaces classes and ids with new values in a css file

        Arguments:
        path -- string path to css file to optimize

        Returns:
        string

        """
        css = Util.fileGetContents(path)
        return self.replaceCss(css)

    def optimizeHtml(self, path):
        """replaces classes and ids with new values in an html file

        Uses:
        Muncher.replaceHtml

        Arguments:
        path -- string path to file to optimize

        Returns:
        string

        """
        html = Util.fileGetContents(path)
        html = self.replaceHtml(html)
        html = self.optimizeCssBlocks(html)
        html = self.optimizeJavascriptBlocks(html)

        return html

    def replaceHtml(self, html):
        """replaces classes and ids with new values in an html file

        Arguments:
        html -- contents to replace

        Returns:
        string

        """
        html = self.replaceHtmlIds(html)
        html = self.replaceHtmlClasses(html)
        return html

    def replaceHtmlIds(self, html):
        """replaces any instances of ids in html markup

        Arguments:
        html -- contents of file to replaces ids in

        Returns:
        string

        """
        for key, value in self.id_map.items():
            key = key[1:]
            value = value[1:]
            html = html.replace("id=\"" + key + "\"", "id=\"" + value + "\"")

        return html

    def replaceClassBlock(self, class_block, key, value):
        """replaces a class string with the new class name

        Arguments:
        class_block -- string from what would be found within class="{class_block}"
        key -- current class
        value -- new class

        Returns:
        string

        """
        key_length = len(key)
        classes = class_block.split(" ")
        i = 0
        for class_name in classes:
            if class_name == key:
                classes[i] = value

            # allows support for things like a.class_name as one of the js selectors
            elif key[0] in (".", "#") and class_name[-key_length:] == key:
                classes[i] = class_name.replace(key, value)
            i = i + 1

        return " ".join(classes)

    def replaceHtmlClasses(self, html):
        """replaces any instances of classes in html markup

        Arguments:
        html -- contents of file to replace classes in

        Returns:
        string

        """
        for key, value in self.class_map.items():
            key = key[1:]
            value = value[1:]
            class_blocks = re.findall(r'class\=((\'|\")(.*?)(\'|\"))', html)
            for class_block in class_blocks:
                new_block = self.replaceClassBlock(class_block[2], key, value)
                html = html.replace("class=" + class_block[0], "class=" + class_block[1] + new_block + class_block[3])

        return html

    def optimizeCssBlocks(self, html):
        """rewrites css blocks that are part of an html file

        Arguments:
        html -- contents of file we are replacing

        Returns:
        string

        """
        result_css = ""
        matches = self.getCssBlocks(html)
        for match in matches:
            match = self.replaceCss(match)
            result_css = result_css + match

        if len(matches):
            return html.replace(matches[0], result_css)

        return html

    @staticmethod
    def getCssBlocks(html):
        """searches a file and returns all css blocks <style type="text/css"></style>

        Arguments:
        html -- contents of file we are replacing

        Returns:
        list

        """
        return re.compile(r'\<style.*?\>(.*)\<\/style\>', re.DOTALL).findall(html)

    def replaceCss(self, css):
        """single call to handle replacing ids and classes

        Arguments:
        css -- contents of file to replace

        Returns:
        string

        """
        css = self.replaceCssFromDictionary(self.class_map, css)
        css = self.replaceCssFromDictionary(self.id_map, css)
        return css

    def replaceCssFromDictionary(self, dictionary, css):
        """replaces any instances of classes and ids based on a dictionary

        Arguments:
        dictionary -- map of classes or ids to replace
        css -- contents of css to replace

        Returns:
        string

        """
        # this really should be done better
        for key, value in dictionary.items():
            css = css.replace(key + "{", value + "{")
            css = css.replace(key + " {", value + " {")
            css = css.replace(key + "#", value + "#")
            css = css.replace(key + " #", value + " #")
            css = css.replace(key + ".", value + ".")
            css = css.replace(key + " .", value + " .")
            css = css.replace(key + ",", value + ",")
            css = css.replace(key + " ", value + " ")
            css = css.replace(key + ":", value + ":")
            # if key == ".svg":
                # print "replacing " + key + " with " + value

        return css

    def optimizeJavascriptBlocks(self, html):
        """rewrites javascript blocks that are part of an html file

        Arguments:
        html -- contents of file we are replacing

        Returns:
        string

        """
        matches = self.getJsBlocks(html)

        for match in matches:
            new_js = match
            if self.config.compress_html:
                matches = re.findall(r'((:?)\/\/.*?\n|\/\*.*?\*\/)', new_js, re.DOTALL)
                for single_match in matches:
                    if single_match[1] == ':':
                        continue
                    new_js = new_js.replace(single_match[0], '');
            new_js = self.replaceJavascript(new_js)
            html = html.replace(match, new_js)

        return html

    @staticmethod
    def getJsBlocks(html):
        """searches a file and returns all javascript blocks: <script type="text/javascript"></script>

        Arguments:
        html -- contents of file we are replacing

        Returns:
        list

        """
        return re.compile(r'\<script(?! src).*?\>(.*?)\<\/script\>', re.DOTALL).findall(html)

    def optimizeJavascript(self, path):
        """optimizes javascript for a specific file

        Arguments:
        path -- path to js file on disk that we are optimizing

        Returns:
        string -- contents to replace file with

        """
        js = Util.fileGetContents(path)
        return self.replaceJavascript(js)

    def replaceJavascript(self, js):
        """single call to handle replacing ids and classes

        Arguments:
        js -- contents of file to replace

        Returns:
        string

        """
        js = self.replaceJsFromDictionary(self.id_map, js)
        js = self.replaceJsFromDictionary(self.class_map, js)
        return js

    @staticmethod
    def getJsSelectors(js, config):
        """finds all js selectors within a js block

        Arguments:
        js -- contents of js file to search

        Returns:
        list

        """
        valid_selectors = "|".join(config.custom_selectors) + "|" + "|".join(config.id_selectors) + "|" + "|".join(config.class_selectors)
        valid_selectors = valid_selectors.replace('$', '\$')
        return re.findall(r'(' + valid_selectors + ')(\(([^<>]*?)\))', js, re.DOTALL)

    def replaceJsFromDictionary(self, dictionary, js):
        """replaces any instances of classes and ids based on a dictionary

        Arguments:
        dictionary -- map of classes or ids to replace
        js -- contents of javascript to replace

        Returns:
        string

        """
        for key, value in dictionary.items():
            blocks = self.getJsSelectors(js, self.config)
            for block in blocks:
                if key[0] == "#" and block[0] in self.config.class_selectors:
                    continue

                if key[0] == "." and block[0] in self.config.id_selectors:
                    continue

                old_selector = block[0] + block[1]

                # custom selectors
                if block[0] in self.config.custom_selectors:
                    new_selector = old_selector.replace(key + ".", value + ".")
                    new_selector = new_selector.replace(key + " ", value + " ")
                    new_selector = new_selector.replace(key + "\"", value + "\"")
                    new_selector = new_selector.replace(key + "\'", value + "\'")
                else:
                    new_selector = old_selector.replace("'" + key[1:] + "'", "'" + value[1:] + "'")
                    new_selector = new_selector.replace("\"" + key[1:] + "\"", "\"" + value[1:] + "\"")

                js = js.replace(old_selector, new_selector)

        return js


================================================
FILE: muncher/sizetracker.py
================================================
#!/usr/bin/env python
# Copyright 2011 Craig Campbell
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import sys, os, gzip
from util import Util

class SizeTracker(object):
    original_size = 0
    original_size_gzip = 0
    new_size = 0
    new_size_gzip = 0

    @staticmethod
    def addSize(path, new = False):

        # gzip the file to get that size
        gzip_path = path + '.gz'
        f_in = open(path, 'rb')
        f_out = gzip.open(gzip_path, 'wb')
        f_out.writelines(f_in)
        f_out.close()
        f_in.close()

        size = os.path.getsize(path)
        gzip_size = os.path.getsize(gzip_path)

        if new is False:
            SizeTracker.original_size += size
            SizeTracker.original_size_gzip += gzip_size
        else:
            SizeTracker.new_size += size
            SizeTracker.new_size_gzip += gzip_size

        Util.unlink(gzip_path)

    @staticmethod
    def trackFile(path, new_path):
        SizeTracker.addSize(path)
        SizeTracker.addSize(new_path, True)

    @staticmethod
    def getSize(bytes):
        if bytes < 1024:
            return str(bytes) + " bytes"

        kb = float(bytes) / 1024
        kb = round(kb, 2)
        return str(kb) + " KB"

    @staticmethod
    def savings():
        percent = 100 - (float(SizeTracker.new_size) / float(SizeTracker.original_size)) * 100
        gzip_percent = 100 - (float(SizeTracker.new_size_gzip) / float(SizeTracker.original_size_gzip)) * 100

        string = "\noriginal size:   " + SizeTracker.getSize(SizeTracker.original_size) + " (" + SizeTracker.getSize(SizeTracker.original_size_gzip) + " gzipped)"
        string += "\nmunched size:    " + SizeTracker.getSize(SizeTracker.new_size) + " (" + SizeTracker.getSize(SizeTracker.new_size_gzip) + " gzipped)"
        string += "\n                 saved " + str(round(percent, 2)) + "% off the original size (" + str(round(gzip_percent, 2)) + "% off the gzipped size)\n"
        return string


================================================
FILE: muncher/util.py
================================================
#!/usr/bin/env python
# Copyright 2011 Craig Campbell
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import os, shutil, glob

class Util:
    """collection of various utility functions"""
    @staticmethod
    def fileExists(path):
        """determines if a file exists

        Arguments:
        path -- path to file on disk

        Returns:
        bool

        """
        return os.path.isfile(path)

    @staticmethod
    def isDir(path):
        """determines if a path is a directory

        Arguments:
        path -- path on disk

        Returns:
        bool

        """
        return os.path.isdir(path)

    @staticmethod
    def getFilesFromDir(path, extension = ""):
        path = path + "/*"

        if not extension == "":
            path = path + "." + extension.lstrip(".")

        return glob.glob(path)

    @staticmethod
    def dump(obj):
        """displays an object as a string for debugging

        Arguments:
        obj -- object

        Returns:
        string

        """
        for attr in dir(obj):
            print "obj.%s = %s" % (attr, getattr(obj, attr))

    @staticmethod
    def getExtension(path):
        """gets the extension from a file

        Arguments:
        path -- string of the file name

        Returns:
        string

        """
        return path.split(".").pop()

    @staticmethod
    def prependExtension(ext, path):
        current_ext = Util.getExtension(path)
        return path.replace("." + current_ext, "." + ext + "." + current_ext)

    @staticmethod
    def getBasePath(path):
        """gets the base directory one level up from the current path

        Arguments:
        path -- path to file or directory

        Returns:
        string

        """
        bits = path.split("/")
        last_bit = bits.pop()
        return "/".join(bits)
        # return "/".join(bits).rstrip(last_bit)

    @staticmethod
    def getFileName(path):
        return path.replace(Util.getBasePath(path), "").lstrip("/")

    @staticmethod
    def unlink(path):
        """deletes a file on disk

        Arguments:
        path -- path to file on disk

        Returns:
        void

        """
        if Util.fileExists(path):
            os.unlink(path)

    @staticmethod
    def unlinkDir(path):
        """removes an entire directory on disk

        Arguments:
        path -- path to directory to remove

        Returns:
        void

        """
        try:
            shutil.rmtree(path)
        except:
            pass

    @staticmethod
    def fileGetContents(path):
        """gets the contents of a file

        Arguments:
        path -- path to file on disk

        Returns:
        string

        """
        if not Util.fileExists(path):
            print "file does not exist at path " + path
            print "skipping"
        file = open(path, "r")
        contents = file.read()
        file.close()
        return contents

    @staticmethod
    def filePutContents(path, contents):
        """puts contents into a file

        Arguments:
        path -- path to file to write to
        contents -- contents to put into file

        Returns:
        void

        """
        file = open(path, "w")
        file.write(contents)
        file.close()

    @staticmethod
    def keyInTupleList(key, tuple_list):
        """checks a list of tuples for the given key"""
        for tuple in tuple_list:
            if tuple[0] == key:
                return True
        return False


================================================
FILE: muncher/varfactory.py
================================================
#!/usr/bin/env python
# Copyright 2011 Craig Campbell
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import math

class VarFactory:
    """class to keep multiple counters and turn numeric counters into alphabetical ones"""
    types = {}
    letters = map(chr, range(97, 123))

    @staticmethod
    def getNext(type):
        """gets the next letter name based on counter name

        Arguments:
        type -- name of counter we want the next value for

        Returns:
        string

        """
        i = VarFactory.getVersion(type)
        return VarFactory.getSmallName(i)

    @staticmethod
    def getVersion(type):
        """gets the next number in the counter for this type

        Arguments:
        type -- name of counter we are incrementing

        Resturns:
        int

        """
        if not type in VarFactory.types:
            VarFactory.types[type] = 0
            return 0

        VarFactory.types[type] += 1

        return VarFactory.types[type]

    @staticmethod
    def getSmallName(index):
        """gets a letter index based on the numeric index

        Arguments:
        index -- the number you are looking for

        Returns:
        string

        """
        # total number of combinations for this index size
        combinations = 0
        letters = 0
        while (combinations + (((letters - 1) * 26) - 1) < index):
            letters += 1
            combinations = int(math.pow(len(VarFactory.letters), letters))

        if (index > 701):
            raise Exception("until my math skillz get better we can only support 702 possibilities!")

        a = int(index) + 1

        if a < 27:
            return chr(a + 96)

        b = 0
        while a > 26:
            b += 1
            a = a - 26

        b = chr(b + 96)
        a = chr(a + 96)
        return b + a


================================================
FILE: setup.py
================================================
#!/usr/bin/env python

from setuptools import setup

setup(name='htmlmuncher',
    version='1.0',
    description='Utility that rewrites CSS, HTML, and JavaScript files in order to save bytes and obfuscate your code.',
    author='Craig Campbell',
    author_email='iamcraigcampbell@gmail.com',
    url='http://htmlmuncher.com',
    packages=['muncher'],
    scripts=['munch']
)