[ { "path": ".github/workflows/main.yml", "content": "name: CI\non: [push]\njobs:\n build:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v1\n - uses: actions/setup-python@v1\n with:\n python-version: '3.6'\n architecture: 'x64'\n - name: Install the library\n run: |\n pip install nbdev jupyter\n pip install -e .\n - name: Read all notebooks\n run: |\n nbdev_read_nbs\n - name: Check if all notebooks are cleaned\n run: |\n echo \"Check we are starting with clean git checkout\"\n if [ -n \"$(git status -uno -s)\" ]; then echo \"git status is not clean\"; false; fi\n echo \"Trying to strip out notebooks\"\n nbdev_clean_nbs\n echo \"Check that strip out was unnecessary\"\n git status -s # display the status to see which nbs need cleaning up\n if [ -n \"$(git status -uno -s)\" ]; then echo -e \"!!! Detected unstripped out notebooks\\n!!!Remember to run nbdev_install_git_hooks\"; false; fi\n - name: Check if there is no diff library/notebooks\n run: |\n if [ -n \"$(nbdev_diff_nbs)\" ]; then echo -e \"!!! Detected difference between the notebooks and the library\"; false; fi\n - name: Run tests\n run: |\n nbdev_test_nbs\n" }, { "path": ".gitignore", "content": ".last_checked\n.idea/\n# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n.gitconfig\n.gitattributes\n\n# Distribution / packaging\n.Python\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\n*.egg-info/\n.installed.cfg\n*.egg\nMANIFEST\n\n# PyInstaller\n# Usually these files are written by a python script from a template\n# before PyInstaller builds the exe, so as to inject date/other infos into it.\n*.manifest\n*.spec\n\n# Installer logs\npip-log.txt\npip-delete-this-directory.txt\n\n# Unit test / coverage reports\nhtmlcov/\n.tox/\n.coverage\n.coverage.*\n.cache\nnosetests.xml\ncoverage.xml\n*.cover\n.hypothesis/\n.pytest_cache/\n\n# Translations\n*.mo\n*.pot\n\n# Django stuff:\n*.log\nlocal_settings.py\ndb.sqlite3\n\n# Flask stuff:\ninstance/\n.webassets-cache\n\n# Scrapy stuff:\n.scrapy\n\n# Sphinx documentation\ndocs/_build/\n\n# PyBuilder\ntarget/\n\n# Jupyter Notebook\n.ipynb_checkpoints\n\n# pyenv\n.python-version\n\n# celery beat schedule file\ncelerybeat-schedule\n\n# SageMath parsed files\n*.sage.py\n\n# Environments\n.env\n.venv\nenv/\nvenv/\nENV/\nenv.bak/\nvenv.bak/\n\n# Spyder project settings\n.spyderproject\n.spyproject\n\n# Rope project settings\n.ropeproject\n\n# mkdocs documentation\n/site\n\n# mypy\n.mypy_cache/\n" }, { "path": "CONTRIBUTING.md", "content": "# How to contribute\n\n## How to get started\n\nBefore anything else, please install the git hooks that run automatic scripts during each commit and merge to strip the notebooks of superfluous metadata (and avoid merge conflicts). After cloning the repository, run the following command inside it:\n```\nnbdev_install_git_hooks\n```\n\n## Did you find a bug?\n\n* Ensure the bug was not already reported by searching on GitHub under Issues.\n* If you're unable to find an open issue addressing the problem, open a new one. Be sure to include a title and clear description, as much relevant information as possible, and a code sample or an executable test case demonstrating the expected behavior that is not occurring.\n* Be sure to add the complete error messages.\n\n#### Did you write a patch that fixes a bug?\n\n* Open a new GitHub pull request with the patch.\n* Ensure that your PR includes a test that fails without your patch, and pass with it.\n* Ensure the PR description clearly describes the problem and solution. Include the relevant issue number if applicable.\n\n## PR submission guidelines\n\n* Keep each PR focused. While it's more convenient, do not combine several unrelated fixes together. Create as many branches as needing to keep each PR focused.\n* Do not mix style changes/fixes with \"functional\" changes. It's very difficult to review such PRs and it most likely get rejected.\n* Do not add/remove vertical whitespace. Preserve the original style of the file you edit as much as you can.\n* Do not turn an already submitted PR into your development playground. If after you submitted PR, you discovered that more work is needed - close the PR, do the required work and then submit a new PR. Otherwise each of your commits requires attention from maintainers of the project.\n* If, however, you submitted a PR and received a request for changes, you should proceed with commits inside that PR, so that the maintainer can see the incremental fixes and won't need to review the whole PR again. In the exception case where you realize it'll take many many commits to complete the requests, then it's probably best to close the PR, do the work and then submit it again. Use common sense where you'd choose one way over another.\n\n## Do you want to contribute to the documentation?\n\n* Docs are automatically created from the notebooks in the nbs folder.\n\n" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) [2019] [Joakim Rishaug]\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n\n" }, { "path": "Makefile", "content": "SRC = $(wildcard lib/*.ipynb)\n\nall: \"online_triplet_loss\" docs\n\n\"online_triplet_loss\": $(SRC)\n\tnbdev_build_lib\n\ttouch \"online_triplet_loss\"\n\ndocs_serve: docs\n\tcd docs && bundle exec jekyll serve\n\ndocs: $(SRC)\n\tnbdev_build_docs\n\ttouch docs\n\ntest:\n\tnbdev_test_nbs\n\nrelease: pypi\n\tnbdev_bump_version\n\npypi: dist\n\ttwine upload --repository pypi dist/*\n\ndist: clean\n\tpython setup.py sdist bdist_wheel\n\nclean:\n\trm -rf dist" }, { "path": "README.md", "content": "# online_triplet_loss\n> PyTorch conversion of the excellent post on the same topic in Tensorflow. Simply an implementation of a triple loss with online mining of candidate triplets used in semi-supervised learning.\n\n\n## Install\n\n`pip install online_triplet_loss`\n\nThen import with:\n`from online_triplet_loss.losses import *`\n\nPS: Requires Pytorch version 1.1.0 or above to use.\n\n## How to use\n\nIn these examples I use a really large margin, since the embedding space is so small. A more realistic margins seems to be between `0.1 and 2.0`\n\n```\nfrom torch import nn\nimport torch\n\nmodel = nn.Embedding(10, 10)\n```\n\n```\n#from online_triplet_loss.losses import *\nlabels = torch.randint(high=10, size=(5,)) # our five labels\n\nembeddings = model(labels)\nprint('Labels:', labels)\nprint('Embeddings:', embeddings)\nloss = batch_hard_triplet_loss(labels, embeddings, margin=100)\nprint('Loss:', loss)\nloss.backward()\n```\n\n Labels: tensor([6, 1, 3, 6, 6])\n Embeddings: tensor([[-1.1335, 0.3364, -3.0174, -0.8732, -0.9301, 1.3619, 0.3746, 0.0457,\n 0.0180, -0.4500],\n [ 1.0757, -0.8420, -0.7630, -0.0746, 1.1545, 0.4017, 0.5587, 1.7947,\n 0.1992, -2.2288],\n [ 0.2646, 1.2383, 0.1949, 0.5743, -0.8460, -0.9929, -2.0350, 0.2095,\n 0.2129, -0.4855],\n [-1.1335, 0.3364, -3.0174, -0.8732, -0.9301, 1.3619, 0.3746, 0.0457,\n 0.0180, -0.4500],\n [-1.1335, 0.3364, -3.0174, -0.8732, -0.9301, 1.3619, 0.3746, 0.0457,\n 0.0180, -0.4500]], grad_fn=)\n Loss: tensor(95.1271, grad_fn=)\n\n\n```\n#from online_triplet_loss.losses import *\nembeddings = model(labels)\nprint('Labels:', labels)\nprint('Embeddings:', embeddings)\nloss, fraction_pos = batch_all_triplet_loss(labels, embeddings, squared=False, margin=100)\nprint('Loss:', loss)\nloss.backward()\n```\n\n Labels: tensor([6, 1, 3, 6, 6])\n Embeddings: tensor([[-1.1335, 0.3364, -3.0174, -0.8732, -0.9301, 1.3619, 0.3746, 0.0457,\n 0.0180, -0.4500],\n [ 1.0757, -0.8420, -0.7630, -0.0746, 1.1545, 0.4017, 0.5587, 1.7947,\n 0.1992, -2.2288],\n [ 0.2646, 1.2383, 0.1949, 0.5743, -0.8460, -0.9929, -2.0350, 0.2095,\n 0.2129, -0.4855],\n [-1.1335, 0.3364, -3.0174, -0.8732, -0.9301, 1.3619, 0.3746, 0.0457,\n 0.0180, -0.4500],\n [-1.1335, 0.3364, -3.0174, -0.8732, -0.9301, 1.3619, 0.3746, 0.0457,\n 0.0180, -0.4500]], grad_fn=)\n tensor(94.9947, grad_fn=) tensor(1.)\n Loss: tensor(94.9947, grad_fn=)\n\n\n## References\n* [Triplet Loss and Online Triplet Mining in Tensorflow](https://github.com/omoindrot/tensorflow-triplet-loss)\n* [Facenet paper](https://arxiv.org/abs/1503.03832)\n* [adambielski's nice implementation](https://github.com/adambielski/siamese-triplet) (unfortunately context switches between CPU / GPU)\n" }, { "path": "docs/.gitignore", "content": "_site/\n" }, { "path": "docs/Gemfile", "content": "source \"https://rubygems.org\"\n\ngem 'github-pages', group: :jekyll_plugins\n\n\n# Added at 2019-11-25 10:11:40 -0800 by jhoward:\ngem \"jekyll\", \"~> 3.7\"\n" }, { "path": "docs/_config.yml", "content": "repository: \"NegatioN/OnlineMiningTripletLoss\"\noutput: web\ntopnav_title: \"online_triplet_loss\"\nsite_title: \"online_triplet_loss\"\ncompany_name: \"Joakim Rishaug\"\ndescription: \"Online mining triplet losses for Pytorch\"\n# Set to false to disable KaTeX math\nuse_math: true\n# Add Google analytics id if you have one and want to use it here\ngoogle_analytics:\n# See http://nbdev.fast.ai/search for help with adding Search\ngoogle_search:\n\nhost: 127.0.0.1\n# the preview server used. Leave as is.\nport: 4000\n# the port where the preview is rendered.\n\nexclude:\n - .idea/\n - .gitignore\n - vendor\n \nexclude: [vendor]\n\nhighlighter: rouge\nmarkdown: kramdown\nkramdown:\n input: GFM\n auto_ids: true\n hard_wrap: false\n syntax_highlighter: rouge\n\ncollections:\n tooltips:\n output: false\n\ndefaults:\n -\n scope:\n path: \"\"\n type: \"pages\"\n values:\n layout: \"page\"\n comments: true\n search: true\n sidebar: home_sidebar\n topnav: topnav\n -\n scope:\n path: \"\"\n type: \"tooltips\"\n values:\n layout: \"page\"\n comments: true\n search: true\n tooltip: true\n\nsidebars:\n- home_sidebar\npermalink: pretty\n\ntheme: jekyll-theme-cayman\nbaseurl: /OnlineMiningTripletLoss/" }, { "path": "docs/_data/alerts.yml", "content": "tip: '
Tip: '\nnote: '
Note: '\nimportant: '
Important: '\nwarning: '
Warning: '\nend: '
'\n\ncallout_danger: '
'\ncallout_default: '
'\ncallout_primary: '
'\ncallout_success: '
'\ncallout_info: '
'\ncallout_warning: '
'\n\nhr_faded: '
'\nhr_shaded: '
'" }, { "path": "docs/_data/definitions.yml", "content": "\n" }, { "path": "docs/_data/glossary.yml", "content": "\n" }, { "path": "docs/_data/sidebars/home_sidebar.yml", "content": "\n#################################################\n### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###\n#################################################\n# Instead edit ../../sidebar.json\nentries:\n- folders:\n - folderitems:\n - output: web,pdf\n title: Overview\n url: /\n - output: web,pdf\n title: Losses\n url: /triplet_loss\n output: web\n title: online_triplet_loss\n output: web\n title: Sidebar\n" }, { "path": "docs/_data/tags.yml", "content": "allowed-tags:\n - getting_started\n - navigation\n" }, { "path": "docs/_data/terms.yml", "content": "" }, { "path": "docs/_data/topnav.yml", "content": "topnav:\n- title: Topnav\n items:\n - title: GitHub\n external_url: https://github.com/\"NegatioN\"/\"online_triplet_loss\"\n\n#Topnav dropdowns\ntopnav_dropdowns:\n- title: Topnav dropdowns\n folders:" }, { "path": "docs/_includes/archive.html", "content": "---\nlayout: default\ntype: archive\n---\n\n
\n

{{ page.title }}

\n
\n
\n\n{{ content }}\n
\n\n \n\n" }, { "path": "docs/_includes/callout.html", "content": "
{{include.content}}
\n" }, { "path": "docs/_includes/footer.html", "content": "
\n
\n
\n

\"Company

\n ©{{ site.time | date: \"%Y\" }} {{site.company_name}}. All rights reserved.
\n{% if page.last_updated %}Page last updated: {{page.last_updated}}
{% endif %} Site last generated: {{ site.time | date: \"%b %-d, %Y\" }}
\n
\n
\n
\n" }, { "path": "docs/_includes/google_analytics.html", "content": "\n\n{% if site.google_analytics %}\n\n\n{% endif %}" }, { "path": "docs/_includes/head.html", "content": "\n\n\n\n\n{{ page.title }} | {{ site.site_title }}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n{% if site.use_math %}\n\n\n\n\n{% endif %}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n" }, { "path": "docs/_includes/head_print.html", "content": "\n\n\n\n\n{% if page.homepage == true %} {{site.homepage_title}} {% elsif page.title %}{{ page.title }}{% endif %} | {{ site.site_title }}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n" }, { "path": "docs/_includes/image.html", "content": "
{% if {{include.url}} %}{% endif %}\"{{include.alt}}\"{% if {{include.url}} %}{% endif %}{% if {{include.caption}} %}
{{include.caption}}
{% endif %}
\n" }, { "path": "docs/_includes/important.html", "content": "
Important: {{include.content}}
" }, { "path": "docs/_includes/initialize_shuffle.html", "content": "\n\n\n\n \n\n \n\n\n\n\n" }, { "path": "docs/_includes/inline_image.html", "content": "\"{{include.alt}}\"\n" }, { "path": "docs/_includes/links.html", "content": "{% comment %}Get links from each sidebar, as listed in the _config.yml file under sidebars{% endcomment %}\n\n{% for sidebar in site.sidebars %}\n{% for entry in site.data.sidebars[sidebar].entries %}\n{% for folder in entry.folders %}\n{% for folderitem in folder.folderitems %}\n{% if folderitem.url contains \"html#\" %}\n[{{folderitem.url | remove: \"/\" }}]: {{folderitem.url | remove: \"/\"}}\n{% else %}\n[{{folderitem.url | remove: \"/\" | remove: \".html\"}}]: {{folderitem.url | remove: \"/\"}}\n{% endif %}\n{% for subfolders in folderitem.subfolders %}\n{% for subfolderitem in subfolders.subfolderitems %}\n[{{subfolderitem.url | remove: \"/\" | remove: \".html\"}}]: {{subfolderitem.url | remove: \"/\"}}\n{% endfor %}\n{% endfor %}\n{% endfor %}\n{% endfor %}\n{% endfor %}\n{% endfor %}\n\n\n{% comment %} Get links from topnav {% endcomment %}\n\n{% for entry in site.data.topnav.topnav %}\n{% for item in entry.items %}\n{% if item.external_url == null %}\n[{{item.url | remove: \"/\" | remove: \".html\"}}]: {{item.url | remove: \"/\"}}\n{% endif %}\n{% endfor %}\n{% endfor %}\n\n{% comment %}Get links from topnav dropdowns {% endcomment %}\n\n{% for entry in site.data.topnav.topnav_dropdowns %}\n{% for folder in entry.folders %}\n{% for folderitem in folder.folderitems %}\n{% if folderitem.external_url == null %}\n[{{folderitem.url | remove: \"/\" | remove: \".html\"}}]: {{folderitem.url | remove: \"/\"}}\n{% endif %}\n{% endfor %}\n{% endfor %}\n{% endfor %}\n\n" }, { "path": "docs/_includes/note.html", "content": "
Note: {{include.content}}
\n" }, { "path": "docs/_includes/search_google_custom.html", "content": "\n\n
\n \n
\n\n" }, { "path": "docs/_includes/search_simple_jekyll.html", "content": "
\n \n
    \n
    \n\n\n" }, { "path": "docs/_includes/sidebar.html", "content": "{% assign sidebar = site.data.sidebars[page.sidebar].entries %}\n{% assign pageurl = page.url | remove: \".html\" %}\n\n
      \n
    • {{sidebar[0].product}} {{sidebar[0].version}}
      • \n {% for entry in sidebar %}\n {% for folder in entry.folders %}\n {% if folder.output contains \"web\" %}\n
    • \n {{ folder.title }}\n \n
      • \n {% endif %}\n {% endfor %}\n {% endfor %}\n \n
    \n\n\n\n" }, { "path": "docs/_includes/tip.html", "content": "
    Tip: {{include.content}}
    " }, { "path": "docs/_includes/toc.html", "content": "\n\n\n\n
    \n" }, { "path": "docs/_includes/topnav.html", "content": "\n\n" }, { "path": "docs/_includes/warning.html", "content": "
    Warning: {{include.content}}
    " }, { "path": "docs/_layouts/default.html", "content": "\n\n\n {% include head.html %}\n \n \n \n {% if page.datatable == true %}\n \n \n \n \n \n {% endif %}\n\n\n\n{% include topnav.html %}\n\n
    \n
    \n \n
    \n {% assign content_col_size = \"col-md-12\" %}\n {% unless page.hide_sidebar %}\n \n
    \n {% include sidebar.html %}\n
    \n {% assign content_col_size = \"col-md-9\" %}\n {% endunless %}\n\n \n
    \n {{content}}\n
    \n \n
    \n\n
    \n\n
    \n\n\n{% if site.google_analytics %}\n{% include google_analytics.html %}\n{% endif %}\n\n" }, { "path": "docs/_layouts/default_print.html", "content": "\n\n\n\n {% include head_print.html %}\n\n\n\n\n\n\n\n
    \n \n
    \n\n {{content}}\n
    \n\n
    \n\n\n\n\n\n" }, { "path": "docs/_layouts/none.html", "content": "---\n---\n{{content}}" }, { "path": "docs/_layouts/page.html", "content": "---\nlayout: default\n---\n\n
    \n \n

    {{ page.title }}

    \n
    \n\n{% if page.simple_map == true %}\n\n\n\n{% include custom/{{page.map_name}}.html %}\n\n{% elsif page.complex_map == true %}\n\n\n\n{% include custom/{{page.map_name}}.html %}\n\n{% endif %}\n\n
    \n\n {% if page.summary %}\n
    {{page.summary}}
    \n {% endif %}\n\n {% unless page.toc == false %}\n {% include toc.html %}\n {% endunless %}\n\n\n {% if site.github_editme_path %}\n\n Edit me\n\n {% endif %}\n\n {{content}}\n\n
    \n {% if page.tags != null %}\n Tags: \n {% assign projectTags = site.data.tags.allowed-tags %}\n {% for tag in page.tags %}\n {% if projectTags contains tag %}\n {{page.tagName}}{{tag}}\n {% endif %}\n {% endfor %}\n {% endif %}\n
    \n\n
    \n\n{{site.data.alerts.hr_shaded}}\n\n{% include footer.html %}\n" }, { "path": "docs/_layouts/page_print.html", "content": "---\nlayout: default_print\ncomments: true\n---\n
    \n

    {{ page.title }}

    \n
    \n\n
    \n\n {% if page.summary %}\n
    {{page.summary}}
    \n {% endif %}\n {{ content }}\n
    \n" }, { "path": "docs/css/boxshadowproperties.css", "content": "/* box-shadow fonts return errors with prince, so extracting here to put in web output only */\n\n#search-demo-container ul#results-container {\n box-shadow: 2px 3px 2px #dedede;\n}\n\n\nhr.shaded {\n box-shadow: inset 0 6px 6px -6px rgba(0,0,0,0.5);\n}\n\n.videoThumbs img {\n box-shadow: 2px 2px 1px #f0f0f0;\n}\n\n.box {\n box-shadow: 2px 2px 4px #dedede;\n}\n\n@media (max-width: 1200px) {\n .navbar-collapse {\n box-shadow: inset 0 1px 0 rgba(255,255,255,0.1);\n }\n}\n" }, { "path": "docs/css/customstyles.css", "content": ".anchor-link {\n display: none;\n}\n\nbody {\n font-size:15px;\n}\n\n.bs-callout {\n padding: 20px;\n margin: 20px 0;\n border: 1px solid #eee;\n border-left-width: 5px;\n border-radius: 3px;\n}\n.bs-callout h4 {\n margin-top: 0;\n margin-bottom: 5px;\n}\n.bs-callout p:last-child {\n margin-bottom: 0;\n}\n.bs-callout code {\n border-radius: 3px;\n}\n.bs-callout+.bs-callout {\n margin-top: -5px;\n}\n.bs-callout-default {\n border-left-color: #777;\n}\n.bs-callout-default h4 {\n color: #777;\n}\n.bs-callout-primary {\n border-left-color: #428bca;\n}\n.bs-callout-primary h4 {\n color: #428bca;\n}\n.bs-callout-success {\n border-left-color: #5cb85c;\n}\n.bs-callout-success h4 {\n color: #5cb85c;\n}\n.bs-callout-danger {\n border-left-color: #d9534f;\n}\n.bs-callout-danger h4 {\n color: #d9534f;\n}\n.bs-callout-warning {\n border-left-color: #f0ad4e;\n}\n.bs-callout-warning h4 {\n color: #f0ad4e;\n}\n.bs-callout-info {\n border-left-color: #5bc0de;\n}\n.bs-callout-info h4 {\n color: #5bc0de;\n}\n\n\n.gi-2x{font-size: 2em;}\n.gi-3x{font-size: 3em;}\n.gi-4x{font-size: 4em;}\n.gi-5x{font-size: 5em;}\n\n\n.breadcrumb > .active {color: #777 !important;}\n\n/* make room for the nav bar */\nh1[id]\n/*,h2[id],\nh3[id],\nh4[id],\nh5[id],\nh6[id],\ndt[id]*/\n{\npadding-top: 60px;\nmargin-top: -40px\n}\n\n.output_html a{\n font-family: Menlo, Monaco, Consolas, \"Courier New\", monospace;\n margin: 25px 0px;\n display: block;\n padding: 9.5px;\n font-size: 13px;\n line-height: 1.42857143;\n color: #333;\n word-break: break-all;\n word-wrap: break-word;\n background-color: #F5F5F5;\n border: 1px solid #FFA500;\n border-radius: 4px;\n white-space: pre-wrap;\n box-sizing: border-box;\n overflow: auto;\n}\n\n.col-md-1, .col-md-10, .col-md-11, .col-md-12, .col-md-2, .col-md-3, .col-md-4, .col-md-5, .col-md-6, .col-md-7, .col-md-8, .col-md-9 {\n float: left;\n}\n\n.col-md-9 {\n width: 75%;\n}\n\n/* From: https://www.w3schools.com/howto/tryit.asp?filename=tryhow_css_download_button' */\n.cstm_btn a{\n background-color: DodgerBlue;\n border: none;\n color: white;\n padding: 12px 30px;\n cursor: pointer;\n font-size: 20px;\n box-sizing: 15px;\n position: absolute;\n}\n\n/* Darker background on mouse-over */\n.cstm_btn a:hover {\n background-color: RoyalBlue;\n}\n\n.container #notebook-container{\n width: 100%;\n box-sizing: border-box;\n }\n\n.post-content img {\n margin: 12px 0px 3px 0px;\n width: auto;\n height: auto;\n max-width: 100%;\n max-height: 100%;\n}\n\n.post-content ol li, .post-content ul li {\n margin: 10px 0px;\n}\n\n.pageSummary {\n font-size:13px;\n display:block;\n margin-bottom:15px;\n padding-left:20px;\n}\n\n.post-summary {\n margin-bottom:12px;\n}\n\n.bs-example{\n margin: 20px;\n}\n\n.breadcrumb li {\n color: gray;\n}\n\ntable {\n background-color: transparent;\n}\ncaption {\n padding-top: 8px;\n padding-bottom: 8px;\n color: #777;\n text-align: left;\n}\nth {\n text-align: left;\n}\ntable {\n max-width: 90%;\n margin-bottom: 20px;\n border: 1px solid #dedede;\n}\n\ntable > thead > tr > th,\ntable > tbody > tr > th,\ntable > tfoot > tr > th,\ntable > thead > tr > td,\ntable > tbody > tr > td,\ntable > tfoot > tr > td {\n padding: 8px;\n line-height: 1.42857143;\n vertical-align: top;\n border-top: 1px solid #ddd;\n}\ntable > thead > tr > th {\n vertical-align: bottom;\n border-bottom: 2px solid #ddd;\n text-transform: none;\n background-color: #777;\n color: white;\n text-align: left;\n}\ntable > caption + thead > tr:first-child > th,\ntable > colgroup + thead > tr:first-child > th,\ntable > thead:first-child > tr:first-child > th,\ntable > caption + thead > tr:first-child > td,\ntable > colgroup + thead > tr:first-child > td,\ntable > thead:first-child > tr:first-child > td {\n border-top: 0;\n}\n\ntable > tbody > tr:nth-of-type(odd) {\n background-color: #f9f9f9;\n}\n\ntable col[class*=\"col-\"] {\n position: static;\n display: table-column;\n float: none;\n}\ntable td[class*=\"col-\"],\ntable th[class*=\"col-\"] {\n position: static;\n display: table-cell;\n float: none;\n}\n\ntable tr td {\n hyphens: auto;\n}\n\n\np.external a {\n text-align:right;\n font-size:12px;\n color: #0088cc;\n display:inline;\n}\n\n#definition-box-container div a.active {\n font-weight: bold;\n}\np.post-meta {font-size: 80%; color: #777;}\n\n.entry-date{font-size:14px;font-size:0.875rem;line-height:1.71429;margin-bottom:0;text-transform:uppercase;}\n\n/* search area */\n#search-demo-container ul#results-container {\n list-style: none;\n font-size: 12px;\n background-color: white;\n position: absolute;\n top: 40px; /* if you change anything about the nav, you'll prob. need to reset the top and left values here.*/\n left: 20px;\n z-index: -1;\n width:223px;\n border-left: 1px solid #dedede;\n}\n\n\nul#results-container a {\n background-color: transparent;\n}\n\nul#results-container a:hover {\n color: black;\n}\n\n\n#search-demo-container a:hover {\n color: black;\n}\n#search-input {\n padding: .5em;\n margin-left:20px;\n width:20em;\n font-size: 0.8em;\n -webkit-box-sizing: border-box;\n -moz-box-sizing: border-box;\n box-sizing: border-box;\n margin-top:10px;\n}\n/* end search */\n\n.filter-options {\n margin-bottom: 20px;\n}\n.filter-options button {\n margin: 3px;\n}\n\na.source_link {\n float:right;\n font-size:15px;\n font-weight:normal;\n}\n\ndiv#toc ul {\n font-size: 90%;\n background-color: whitesmoke;\n padding: 5px;\n border-radius: 5px;\n max-width: 450px;\n color: gray;\n list-style-type: none;\n}\n\n/* when using relative fonts like 0.9em or 90%, remember to then set the nested items to 1em or 100%, otherwise each level will get smaller and smaller and less readable */\ndiv#toc ul li {\n margin: 8px 0px 8px 22px;\n font-size: 100%;\n list-style-type: none;\n}\n\ndiv#toc ul li ul {\n font-size: 100%;\n padding-left:30px;\n padding-top: 0px;\n padding-bottom: 0px;\n list-style-type: none;\n}\n\ndiv#toc ul li ul li.hide_content::before {\n content: none;\n}\n\ndiv#toc >ul::before {\n content: \"Table of Contents\";\n font-weight: 500;\n color: #555;\n text-align:center;\n margin-left:auto;\n margin-right:auto;\n width:70px;\n padding-top:150px;\n padding-bottom:40px;\n padding-left:10px;\n}\n\nli.dropdownActive a {\n font-weight: bold;\n}\n\n\n.post-content a.fa-rss {\n color: orange;\n}\n\n\n.navbar-inverse .navbar-nav > li > a {\n background-color: transparent;\n margin-top:10px;\n}\n\n.post-content .rssfeedLink {\n color: #248EC2;\n}\n\nfooter {\n font-size: smaller;\n}\n\n/* FAQ page */\n#accordion .panel-heading {\n font-size: 12px;\n}\n\na.accordion-toggle, a.accordion-collapsed {\n font-size: 14px;\n text-decoration: none;\n}\n\n/* navgoco sidebar styles (customized) */\n.nav, .nav ul, .nav li {\n list-style: none;\n}\n\n.nav ul {\n padding: 0;\n /*margin: 0 0 0 18px;*/\n margin:0px;\n}\n\n.nav {\n /* padding: 4px;*/\n padding:0px;\n margin: 0px;\n}\n\n.nav > li {\n margin: 1px 0;\n}\n\n.nav > li li {\n margin: 2px 0;\n}\n\n.nav a {\n color: #333;\n display: block;\n outline: none;\n /*-webkit-border-radius: 4px;\n -moz-border-radius: 4px;\n border-radius: 4px;*/\n text-decoration: none;\n}\n\n.nav li > a > span {\n float: right;\n font-size: 19px;\n font-weight: bolder;\n}\n\n\n.nav li > a > span:after {\n content: '\\25be';\n}\n.nav li.active > a > span:after {\n content: '\\25b4';\n}\n\n.nav a:hover, .nav li.active > a {\n background-color: #8D8D8D;\n color: #f5f5f5;\n}\n\n.nav > li.active > a {\nbackground-color: #347DBE;\n}\n\n.nav li a {\n font-size: 12px;\n line-height: 18px;\n padding: 2px 10px;\n background-color: #f1f1f1;\n}\n\n.nav > li > a {\n font-size: 14px;\n line-height: 20px;\n padding: 4px 10px;\n}\n\nul#mysidebar {\n border-radius:0px;\n}\n\n.nav ul li ul li a {\n padding-left:40px;\n}\n\n.nav li.thirdlevel > a {\n color: #248EC2;\n font-weight:bold;\n padding-left:20px;\n background-color: whitesmoke !important;\n}\n\n\n.nav ul li a {\n background-color: #FAFAFA;\n}\n\n.nav li a {\n padding-right:10px;\n}\n\n.nav li a:hover {\n background-color: #8D8D8D;\n}\n\n.nav ul li a {\n border-top:1px solid whitesmoke;\n padding-left:10px;\n}\n/* end sidebar */\n\n.navbar-inverse .navbar-nav > .active > a, .navbar-inverse .navbar-nav > .active > a:hover, .navbar-inverse .navbar-nav > .active > a:focus {\n border-radius:5px;\n}\n\n.navbar-inverse .navbar-nav>.open>a, .navbar-inverse .navbar-nav>.open>a:focus, .navbar-inverse .navbar-nav>.open>a:hover {\n border-radius: 5px;\n}\n\nspan.projectTitle {\n font-family: Helvetica;\n font-weight: bold;\n}\n\n.footer {\n text-align: right;\n}\n\n.footerMeta {\n background-color: whitesmoke;\n padding: 10px;\n max-width: 250px;\n border-radius: 5px;\n margin-top: 50px;\n font-style:italic;\n font-size:12px;\n}\n\nimg.screenshotSmall {\n max-width: 300px;\n}\n\n\ndl dt p {\n margin-left:20px;\n}\n\n\ndl dd {\n margin-top:10px;\n margin-bottom:10px;\n}\n\ndl.dl-horizontal dd {\n padding-top: 20px;\n}\n\nfigcaption {\n\n padding-bottom:12px;\n padding-top:6px;\n max-width: 90%;\n margin-bottom:20px;\n font-style: italic;\n color: gray;\n\n}\n\n.testing {\n color: orange;\n}\n\n.preference {\n color: red;\n}\n\n\ntable.dataTable thead {\n background-color: #444;\n}\ntable td {\n hyphens: auto;\n}\n\nsection table tr.success {\n background-color: #dff0d8 !important;\n}\n\ntable tr.info {\n background-color: #d9edf7 !important;\n}\n\nsection table tr.warning, table tr.testing, table tr.testing > td.sorting_1 {\n background-color: #fcf8e3 !important;\n}\nsection table tr.danger, table tr.preference, table tr.preference > td.sorting_1 {\n background-color: #f2dede !important;\n}\n\n.orange {\n color: orange;\n}\n\ntable.profile thead tr th {\n background-color: #248ec2;\n}\n\ntable.request thead tr th {\n background-color: #ED1951;\n}\n\n.audienceLabel {\n margin: 10px;\n float: right;\n border:1px solid #dedede;\n padding:7px;\n}\n\n.prefaceAudienceLabel {\n color: gray;\n text-align: center;\n margin:5px;\n}\nspan.myLabel {\n padding-left:10px;\n padding-right:10px;\n}\n\nbutton.cursorNorm {\n cursor: default;\n}\n\na.dropdown-toggle, .navbar-inverse .navbar-nav > li > a {\n margin-left: 10px;\n}\n\nhr.faded {\n border: 0;\n height: 1px;\n background-image: -webkit-linear-gradient(left, rgba(0,0,0,0), rgba(0,0,0,0.75), rgba(0,0,0,0));\n background-image: -moz-linear-gradient(left, rgba(0,0,0,0), rgba(0,0,0,0.75), rgba(0,0,0,0));\n background-image: -ms-linear-gradient(left, rgba(0,0,0,0), rgba(0,0,0,0.75), rgba(0,0,0,0));\n background-image: -o-linear-gradient(left, rgba(0,0,0,0), rgba(0,0,0,0.75), rgba(0,0,0,0));\n}\n\nhr.shaded {\n height: 12px;\n border: 0;\n margin-top: 70px;\n background: white;\n width: 100%;\n margin-bottom: 10px;\n}\n\n.fa-6x{font-size:900%;}\n.fa-7x{font-size:1100%;}\n.fa-8x{font-size:1300%;}\n.fa-9x{font-size:1500%;}\n.fa-10x{font-size:1700%;}\n\ni.border {\n padding: 10px 20px;\n background-color: whitesmoke;\n}\n\na[data-toggle] {\n color: #248EC2;\n}\n\n.summary {\n font-size:120%;\n color: #808080;\n margin:20px 0px 20px 0px;\n border-left: 5px solid #ED1951;\n padding-left: 10px;\n\n}\n\na.fa.fa-envelope-o.mailto {\n font-weight: 600;\n}\n\nh3 {color: #ED1951; font-weight:normal; font-size:130%;}\nh4 {color: #000000; font-weight:normal; font-size:120%; font-weight:bold;}\n\n.alert, .callout {\n overflow: hidden;\n}\n\n.nav-tabs > li.active > a, .nav-tabs > li.active > a:hover, .nav-tabs > li.active > a:focus {\n background-color: #248ec2;\n color: white;\n}\n\nol li ol li {list-style-type: lower-alpha;}\nol li ul li {list-style-type: disc;}\n\nli img {clear:both; }\n\ndiv#toc ul li ul li {\n list-style-type: none;\n margin: 5px 0px 0px 0px;\n}\n\n.tab-content {\n padding: 15px;\n background-color: #FAFAFA;\n}\n\nspan.tagTitle {font-weight: 500;}\n\nli.activeSeries {\n font-weight: bold;\n}\n\n.seriesContext .dropdown-menu li.active {\n font-weight: bold;\n margin-left: 43px;\n font-size:18px;\n}\n\n.alert-warning {\n color: #444;\n}\n\ndiv.alert code, h2 code {\n background-color: transparent !important;\n}\n/* without this, the links in these notes aren't visible.*/\n.alert a {\n text-decoration: underline;\n}\n\ndiv.tags {padding: 10px 5px;}\n\n.tabLabel {\n font-weight: normal;\n}\n\nhr {\n background: #999;\n margin: 30px 0px;\n width: 90%;\n margin-left: auto;\n margin-right: auto;\n}\n\nbutton.cursorNorm {\n cursor: pointer;\n}\n\nh2 {\n font-size:24px;\n line-height:29px;\n border-top: 3px solid blue;\n padding-top: 5px;\n}\n\nspan.otherProgrammingLanguages {\n font-style: normal;\n}\n\na[data-toggle=\"tooltip\"] {\n color: #649345;\n font-style: italic;\n cursor: default;\n}\n\n.seriesNext, .seriesContext {\n margin-top: 15px;\n margin-bottom: 15px;\n}\n\n.seriescontext ol li {\n list-style-type: upper-roman;\n}\n\nol.series li {\n list-style-type: decimal;\n margin-left: 40px;\n padding-left: 0px;\n}\n\n.siteTagline {\n font-size: 200%;\n font-weight: bold;\n color: silver;\n font-family: monospace;\n text-align: center;\n line-height: 10px;\n margin: 20px 0px;\n display: block;\n}\n\n.versionTagline {\n text-align: center;\n margin-bottom: 20px;\n font-family: courier;\n color: silver;\n color: #444;\n display:block;\n}\n\n/* not sure if using this ...*/\n.navbar-inverse .navbar-collapse, .navbar-inverse .navbar-form {\n border-color: #248ec2 !important;\n}\n\n#mysidebar .nav ul {\n background-color: #FAFAFA;\n}\n.nav ul.series li {\n list-style: decimal;\n font-size:12px;\n}\n\n.nav ul.series li a:hover {\n background-color: gray;\n}\n.nav ul.series {\n padding-left: 30px;\n}\n\n.nav ul.series {\n background-color: #FAFAFA;\n}\n\n/*\na.dropdown-toggle.otherProgLangs {\n color: #f7e68f !important;\n}\n*/\n\nspan.muted {color: #666;}\n\ntable code {background-color: transparent;}\n\n.highlight .err {\n color: #a61717;\n background-color: transparent !important;\n}\n\ntable p {\n margin-top: 12px;\n margin-bottom: 12px;\n}\n\npre, table code {\n white-space: pre-wrap; /* css-3 */\n white-space: -moz-pre-wrap; /* Mozilla, since 1999 */\n white-space: -pre-wrap; /* Opera 4-6 */\n white-space: -o-pre-wrap; /* Opera 7 */\n word-wrap: break-word; /* Internet Explorer 5.5+ */\n}\n\npre {\n margin: 25px 0px;\n}\n\n#json-box-container pre {\n margin: 0px;\n}\n\n.video-js {\n margin: 30px 0px;\n}\n\nvideo {\n display: block;\n margin: 30px 0px;\n border: 1px solid #c0c0c0;\n}\n\n\np.required, p.dataType {display: block; color: #c0c0c0; font-size: 80%; margin-left:4px;}\n\ndd {margin-left:20px;}\n\n.post-content img.inline {\n margin:0px;\n margin-bottom:6px;\n}\n.panel-heading {\n font-weight: bold;\n}\n\n.note code, .alert code, .warning code, div#toc code, h2 code, h3 code, h4 code {\n color: inherit;\n padding: 0px;\n}\n\n.alert {\n margin-bottom:10px;\n margin-top:10px;\n}\n\na.accordion-toggle {\n font-style: normal;\n}\n\nspan.red {\n color: red;\n font-family: Monaco, Menlo, Consolas, \"Courier New\", monospace;\n}\n\nh3.codeExplanation {\n font-size:18px;\n font-style:normal;\n color: black;\n line-height: 24px;\n}\n\nspan.soft {\n color: #c0c0c0;\n}\n\n.githubEditButton {\n margin-bottom:7px;\n}\n\n.endpoint {\n padding: 15px;\n background-color: #f0f0f0;\n font-family: courier;\n font-size: 110%;\n margin: 20px 0px;\n color: #444;\n}\n\n.parameter {\n font-family: courier;\n color: red !important;\n}\n\n.formBoundary {\n border: 1px solid gray;\n padding: 15px;\n margin: 15px 0px;\n background-color: whitesmoke;\n}\n\n@media (max-width: 767px) {\n .navbar-inverse .navbar-nav .open .dropdown-menu > li > a {\n color: #444;\n }\n}\n\n@media (max-width: 990px) {\n #mysidebar {\n position: relative;\n }\n .col-md-9 {\n width: 95%;\n }\n}\n\n\n@media (min-width: 1000px) {\n\n ul#mysidebar {\n width: 225px;\n }\n}\n\n@media (max-width: 900px) {\n\n ul#mysidebar {\n max-width: 100%;\n }\n}\n\n.col-md-9 img {\n max-width: 100%;\n max-height: 100%;\n}\n\n\n.post-content img {\n margin: 12px 0px 3px 0px;\n width: auto;\n height: auto;\n max-width: 100%;\n max-height: 100%;\n}\n.col-md-9 img {\n max-width: 100%;\n max-height: 100%;\n}\n\n\n.post-content img {\n margin: 12px 0px 3px 0px;\n width: auto;\n height: auto;\n max-width: 100%;\n max-height: 100%;\n}\n\n.videoThumbs img {\n float: left;\n margin:15px 15px 15px 0px;\n border: 1px solid #dedede;\n}\n\n\n@media only screen and (min-width: 900px), only screen and (min-device-width: 900px) {\n .col-md-9 img {\n max-width: 990px;\n max-height: 700px;\n }\n}\n\n*:hover > .anchorjs-link {\n transition: color .25s linear;\n text-decoration: none;\n}\n\n.kbCaption {\n color: white;\n background-color: #444;\n padding:10px;\n}\n\n/* Strip the outbound icon when this class is present */\na[href].noCrossRef::after,\na.no_icon:after\n {\n content:\"\" !important;\n padding-left: 0;\n}\n\n.btn-default {\n margin-bottom: 10px;\n}\n\n/* algolia search */\n\n.search {\n text-align: left;\n}\n.search input {\n font-size: 20px;\n width: 300px;\n}\n.results {\n margin: auto;\n text-align: left;\n}\n.results ul {\n list-style-type: none;\n padding: 0;\n}\n\n/* algolia */\n\ndiv.results {\n position: absolute;\n background-color: white;\n width: 100%;\n}\n\n.post-meta {\n font-size: 14px;\n color: #828282;\n}\n\n.post-link {\n font-size: 22px;\n}\n\n.post-list p {\n margin: 10px 0px;\n}\n\ntime {\n margin-right: 10px;\n}\n\np.post-meta time {\n margin-right: 0px;\n}\n\nspan.label.label-default {\n background-color: gray;\n}\n\nspan.label.label-primary {\n background-color: #f0ad4e;\n}\n.col-lg-12 .nav li a {background-color: white}\n\n\n.nav li.active > a.subfoldersTitle {\n background-color: whitesmoke;\n font-weight: bold;\n color: black;\n }\n\ncode {\n color: #1d1f5b;\n background-color: #f3f3f3;\n}\n\na code {\n color: #0082C2;\n}\n\ntable th code {\n color: white;\n}\n\nol li ul li ol li {\n list-style: decimal;\n}\n\nol li ul li ol li ul li{\n list-style: disc;\n}\n\n.post-content table th {\n vertical-align: top;\n}\n\ntable thead th code.highlighter-rouge {\n background-color: transparent;\n}\n\n\n.box {\n padding: 10px;\n border: 1px solid #888;\n width: 100px;\n height: 80px;\n background-color: #f5f5f5;\n font-family: Arial;\n font-size: 12px;\n hyphens: auto;\n float: left;\n font-size: 12px;\n}\n\n.box:hover {\n background-color: #f0f0f0;\n}\n\n#userMap {\n overflow-x: auto;\n overflow-y: auto;\n padding: 20px;\n min-width: 770px;\n}\n\n#userMap .active {\n background-color: #d6f5d6;\n border:1px solid #555;\n font-weight: bold;\n}\n\nh2.userMapTitle {\n font-family: Arial;\n}\n\n#userMap a:hover {\n text-decoration: none;\n }\n\ndiv.arrow {\n max-width: 50px;\n margin-left: 15px;\n margin-right: 15px;\n font-size: 20px;\n}\n\ndiv.content {\n max-width: 110px\n}\n\n#userMap div.arrow, #userMap div.content {\n float: left;\n}\n\n.clearfix {\n clear: both;\n}\n\n\n#userMap div.arrow {\n position: relative;\n top: 30px;\n}\n\n.box1 {\n margin-left:0px;\n}\n\nbutton.btn.btn-default.btn-lg.modalButton1 {\n margin-left: -20px;\n}\n\ndiv.box.box1 {\n margin-left: -20px;\n}\n\n#userMap .btn-lg {\n width: 100px;\n height: 80px;\n\n}\n\n#userMap .complexArrow {\n font-size: 22px;\n margin: 0px 10px;\n}\n\n\n#userMap .btn-lg .active {\n background-color: #d6f5d6;\n}\n\n#userMap .btn-lg {\n white-space: pre-wrap; /* css-3 */\n white-space: -moz-pre-wrap; /* Mozilla, since 1999 */\n white-space: -pre-wrap; /* Opera 4-6 */\n white-space: -o-pre-wrap; /* Opera 7 */\n word-wrap: break-word; /* Internet Explorer 5.5+ */\n font-size: 14px;\n }\n\n/*\n * Let's target IE to respect aspect ratios and sizes for img tags containing SVG files\n *\n * [1] IE9\n * [2] IE10+\n */\n/* 1 */\n.ie9 img[src$=\".svg\"] {\n width: 100%;\n}\n/* 2 */\n@media screen and (-ms-high-contrast: active), (-ms-high-contrast: none) {\n img[src$=\".svg\"] {\n width: 100%;\n }\n}\n\nh4.panel-title {\n padding-top: 0px;\n margin-top: 0px;\n}\n\n/*set navbar breakpoint so that it converts to hamburger earlier */\n\n@media (max-width: 1200px) {\n .navbar-header {\n float: none;\n }\n .navbar-left,.navbar-right {\n float: none !important;\n }\n .navbar-toggle {\n display: block;\n }\n .navbar-collapse {\n border-top: 1px solid transparent;\n }\n .navbar-fixed-top {\n top: 0;\n border-width: 0 0 1px;\n }\n .navbar-collapse.collapse {\n display: none!important;\n }\n .navbar-nav {\n float: none!important;\n margin-top: 7.5px;\n }\n .navbar-nav>li {\n float: none;\n }\n .navbar-nav>li>a {\n padding-top: 10px;\n padding-bottom: 10px;\n }\n .collapse.in{\n display:block !important;\n }\n}\n\n.input_area pre {\n padding-bottom: 3;\n margin-top: 10px;\n background-color: #f5f1e0;\n border: 0px;\n}\n.output_area pre {\n padding-top: 3;\n margin-top: -25px;\n margin-bottom: 10px;\n border: 0px;\n padding-left: 2em;\n}\n.pytest_card {\n padding: 10px 1em 4px 1em;\n background-color: #eef2ff;\n border-radius: 5px;\n margin-bottom: 20px;\n}\n.pytest_card .close {\n color: #000\n}\nblockquote {\n font-family: Menlo,Monaco,Consolas,\"Courier New\",monospace;\n}\nblockquote code {\n background-color: white;\n}\nh4 code {\n background-color: white;\n}\n\n/* Google Custom Search styling */\n\n#gcs-search-container {\n width: 340px !important; /* ToDo: define width without using absolute px value */\n display: inline-block !important;\n}\n\n.gsc-search-box.gsc-search-box-tools table {\n margin-bottom: 0px !important;\n}\n.gsc-input, .gsc-search-button {\n padding: 0;\n}\n.gsc-control-cse {\n padding: 6px !important;\n}\ninput.gsc-input {\n font-size: 12px !important;\n}\n" }, { "path": "docs/css/modern-business.css", "content": "/*!\n * Start Bootstrap - Modern Business HTML Template (http://startbootstrap.com)\n * Code licensed under the Apache License v2.0.\n * For details, see http://www.apache.org/licenses/LICENSE-2.0.\n */\n\n/* Global Styles */\n\nhtml,\nbody {\n height: 100%;\n}\n\n.img-portfolio {\n margin-bottom: 30px;\n}\n\n.img-hover:hover {\n opacity: 0.8;\n}\n\n/* Home Page Carousel */\n\nheader.carousel {\n height: 50%;\n}\n\nheader.carousel .item,\nheader.carousel .item.active,\nheader.carousel .carousel-inner {\n height: 100%;\n}\n\nheader.carousel .fill {\n width: 100%;\n height: 100%;\n background-position: center;\n background-size: cover;\n}\n\n/* 404 Page Styles */\n\n.error-404 {\n font-size: 100px;\n}\n\n/* Pricing Page Styles */\n\n.price {\n display: block;\n font-size: 50px;\n line-height: 50px;\n}\n\n.price sup {\n top: -20px;\n left: 2px;\n font-size: 20px;\n}\n\n.period {\n display: block;\n font-style: italic;\n}\n\n/* Footer Styles */\n\nfooter {\n margin: 50px 0;\n}\n\n/* Responsive Styles */\n\n@media(max-width:991px) {\n .client-img,\n .img-related {\n margin-bottom: 30px;\n }\n}\n\n@media(max-width:767px) {\n .img-portfolio {\n margin-bottom: 15px;\n }\n\n header.carousel .carousel {\n height: 70%;\n }\n}\n" }, { "path": "docs/css/printstyles.css", "content": "\n/*body.print .container {max-width: 650px;}*/\n\nbody {\n font-size:14px;\n}\n.nav ul li a {border-top:0px; background-color:transparent; color: #808080; }\n#navig a[href] {color: #595959 !important;}\ntable .table {max-width:650px;}\n\n#navig li.sectionHead {font-weight: bold; font-size: 18px; color: #595959 !important; }\n#navig li {font-weight: normal; }\n\n#navig a[href]::after { content: leader(\".\") target-counter(attr(href), page); }\n\na[href]::after {\n content: \" (page \" target-counter(attr(href), page) \")\"\n}\n\na[href^=\"http:\"]::after, a[href^=\"https:\"]::after {\n content: \"\";\n}\n\na[href] {\n color: blue !important;\n}\na[href*=\"mailto\"]::after, a[data-toggle=\"tooltip\"]::after, a[href].noCrossRef::after {\n content: \"\";\n}\n\n\n@page {\n margin: 60pt 90pt 60pt 90pt;\n font-family: sans-serif;\n font-style:none;\n color: gray;\n\n}\n\n.printTitle {\n line-height:30pt;\n font-size:27pt;\n font-weight: bold;\n letter-spacing: -.5px;\n margin-bottom:25px;\n}\n\n.printSubtitle {\n font-size: 19pt;\n color: #cccccc !important;\n font-family: \"Grotesque MT Light\";\n line-height: 22pt;\n letter-spacing: -.5px;\n margin-bottom:20px;\n}\n.printTitleArea hr {\n color: #999999 !important;\n height: 2px;\n width: 100%;\n}\n\n.printTitleImage {\n max-width:300px;\n margin-bottom:200px;\n}\n\n\n.printTitleImage {\n max-width: 250px;\n}\n\n#navig {\n /*page-break-before: always;*/\n}\n\n.copyrightBoilerplate {\n page-break-before:always;\n font-size:14px;\n}\n\n.lastGeneratedDate {\n font-style: italic;\n font-size:14px;\n color: gray;\n}\n\n.alert a {\n text-decoration: none !important;\n}\n\n\nbody.title { page: title }\n\n@page title {\n @top-left {\n content: \" \";\n }\n @top-right {\n content: \" \"\n }\n @bottom-right {\n content: \" \";\n }\n @bottom-left {\n content: \" \";\n }\n}\n\nbody.frontmatter { page: frontmatter }\nbody.frontmatter {counter-reset: page 1}\n\n\n@page frontmatter {\n @top-left {\n content: prince-script(guideName);\n }\n @top-right {\n content: prince-script(datestamp);\n }\n @bottom-right {\n content: counter(page, lower-roman);\n }\n @bottom-left {\n content: \"youremail@domain.com\"; }\n}\n\nbody.first_page {counter-reset: page 1}\n\nh1 { string-set: doctitle content() }\n\n@page {\n @top-left {\n content: string(doctitle);\n font-size: 11px;\n font-style: italic;\n }\n @top-right {\n content: prince-script(datestamp);\n font-size: 11px;\n }\n\n @bottom-right {\n content: \"Page \" counter(page);\n font-size: 11px;\n }\n @bottom-left {\n content: prince-script(guideName);\n font-size: 11px;\n }\n}\n.alert {\n background-color: #fafafa !important;\n border-color: #dedede !important;\n color: black;\n}\n\npre {\n background-color: #fafafa;\n}\n" }, { "path": "docs/css/syntax.css", "content": ".highlight { background: #ffffff; }\n.highlight .c { color: #999988; font-style: italic } /* Comment */\n.highlight .err { color: #a61717; background-color: #e3d2d2 } /* Error */\n.highlight .k { font-weight: bold } /* Keyword */\n.highlight .o { font-weight: bold } /* Operator */\n.highlight .cm { color: #999988; font-style: italic } /* Comment.Multiline */\n.highlight .cp { color: #999999; font-weight: bold } /* Comment.Preproc */\n.highlight .c1 { color: #999988; font-style: italic } /* Comment.Single */\n.highlight .cs { color: #999999; font-weight: bold; font-style: italic } /* Comment.Special */\n.highlight .gd { color: #000000; background-color: #ffdddd } /* Generic.Deleted */\n.highlight .gd .x { color: #000000; background-color: #ffaaaa } /* Generic.Deleted.Specific */\n.highlight .ge { font-style: italic } /* Generic.Emph */\n.highlight .gr { color: #aa0000 } /* Generic.Error */\n.highlight .gh { color: #999999 } /* Generic.Heading */\n.highlight .gi { color: #000000; background-color: #ddffdd } /* Generic.Inserted */\n.highlight .gi .x { color: #000000; background-color: #aaffaa } /* Generic.Inserted.Specific */\n.highlight .go { color: #888888 } /* Generic.Output */\n.highlight .gp { color: #555555 } /* Generic.Prompt */\n.highlight .gs { font-weight: bold } /* Generic.Strong */\n.highlight .gu { color: #aaaaaa } /* Generic.Subheading */\n.highlight .gt { color: #aa0000 } /* Generic.Traceback */\n.highlight .kc { font-weight: bold } /* Keyword.Constant */\n.highlight .kd { font-weight: bold } /* Keyword.Declaration */\n.highlight .kp { font-weight: bold } /* Keyword.Pseudo */\n.highlight .kr { font-weight: bold } /* Keyword.Reserved */\n.highlight .kt { color: #445588; font-weight: bold } /* Keyword.Type */\n.highlight .m { color: #009999 } /* Literal.Number */\n.highlight .s { color: #d14 } /* Literal.String */\n.highlight .na { color: #008080 } /* Name.Attribute */\n.highlight .nb { color: #0086B3 } /* Name.Builtin */\n.highlight .nc { color: #445588; font-weight: bold } /* Name.Class */\n.highlight .no { color: #008080 } /* Name.Constant */\n.highlight .ni { color: #800080 } /* Name.Entity */\n.highlight .ne { color: #990000; font-weight: bold } /* Name.Exception */\n.highlight .nf { color: #990000; font-weight: bold } /* Name.Function */\n.highlight .nn { color: #555555 } /* Name.Namespace */\n.highlight .nt { color: #000080 } /* Name.Tag */\n.highlight .nv { color: #008080 } /* Name.Variable */\n.highlight .ow { font-weight: bold } /* Operator.Word */\n.highlight .w { color: #bbbbbb } /* Text.Whitespace */\n.highlight .mf { color: #009999 } /* Literal.Number.Float */\n.highlight .mh { color: #009999 } /* Literal.Number.Hex */\n.highlight .mi { color: #009999 } /* Literal.Number.Integer */\n.highlight .mo { color: #009999 } /* Literal.Number.Oct */\n.highlight .sb { color: #d14 } /* Literal.String.Backtick */\n.highlight .sc { color: #d14 } /* Literal.String.Char */\n.highlight .sd { color: #d14 } /* Literal.String.Doc */\n.highlight .s2 { color: #d14 } /* Literal.String.Double */\n.highlight .se { color: #d14 } /* Literal.String.Escape */\n.highlight .sh { color: #d14 } /* Literal.String.Heredoc */\n.highlight .si { color: #d14 } /* Literal.String.Interpol */\n.highlight .sx { color: #d14 } /* Literal.String.Other */\n.highlight .sr { color: #009926 } /* Literal.String.Regex */\n.highlight .s1 { color: #d14 } /* Literal.String.Single */\n.highlight .ss { color: #990073 } /* Literal.String.Symbol */\n.highlight .bp { color: #999999 } /* Name.Builtin.Pseudo */\n.highlight .vc { color: #008080 } /* Name.Variable.Class */\n.highlight .vg { color: #008080 } /* Name.Variable.Global */\n.highlight .vi { color: #008080 } /* Name.Variable.Instance */\n.highlight .il { color: #009999 } /* Literal.Number.Integer.Long */" }, { "path": "docs/css/theme-blue.css", "content": ".summary {\n color: #808080;\n border-left: 5px solid #ED1951;\n font-size:16px;\n}\n\n\nh3 {color: #000000; }\nh4 {color: #000000; }\n\n.nav-tabs > li.active > a, .nav-tabs > li.active > a:hover, .nav-tabs > li.active > a:focus {\n background-color: #248ec2;\n color: white;\n}\n\n.nav > li.active > a {\n background-color: #347DBE;\n}\n\n.nav > li > a:hover {\n background-color: #248ec2;\n}\n\ndiv.navbar-collapse .dropdown-menu > li > a:hover {\n background-color: #347DBE;\n}\n\n.nav li.thirdlevel > a {\n background-color: #FAFAFA !important;\n color: #248EC2;\n font-weight: bold;\n}\n\na[data-toggle=\"tooltip\"] {\n color: #649345;\n font-style: italic;\n cursor: default;\n}\n\n.navbar-inverse {\n background-color: #347DBE;\n border-color: #015CAE;\n}\n.navbar-inverse .navbar-nav>li>a, .navbar-inverse .navbar-brand {\n color: white;\n}\n\n.navbar-inverse .navbar-nav>li>a:hover, a.fa.fa-home.fa-lg.navbar-brand:hover {\n color: #f0f0f0;\n}\n\na.navbar-brand:hover {\n color: #f0f0f0;\n}\n\n.navbar-inverse .navbar-nav > .open > a, .navbar-inverse .navbar-nav > .open > a:hover, .navbar-inverse .navbar-nav > .open > a:focus {\n color: #015CAE;\n}\n\n.navbar-inverse .navbar-nav > .open > a, .navbar-inverse .navbar-nav > .open > a:hover, .navbar-inverse .navbar-nav > .open > a:focus {\n background-color: #015CAE;\n color: #ffffff;\n}\n\n.navbar-inverse .navbar-collapse, .navbar-inverse .navbar-form {\n border-color: #248ec2 !important;\n}\n\n.btn-primary {\n color: #ffffff;\n background-color: #347DBE;\n border-color: #347DBE;\n}\n\n.navbar-inverse .navbar-nav > .active > a, .navbar-inverse .navbar-nav > .active > a:hover, .navbar-inverse .navbar-nav > .active > a:focus {\n background-color: #347DBE;\n}\n\n.btn-primary:hover,\n.btn-primary:focus,\n.btn-primary:active,\n.btn-primary.active,\n.open .dropdown-toggle.btn-primary {\n background-color: #248ec2;\n border-color: #347DBE;\n}\n\n.printTitle {\n color: #015CAE !important;\n}\n\nbody.print h1 {color: #015CAE !important; font-size:28px !important;}\nbody.print h2 {color: #595959 !important; font-size:20px !important;}\nbody.print h3 {color: #E50E51 !important; font-size:14px !important;}\nbody.print h4 {color: #679DCE !important; font-size:14px; font-style: italic !important;}\n\n.anchorjs-link:hover {\n color: #216f9b;\n}\n\ndiv.sidebarTitle {\n color: #015CAE;\n}\n\nli.sidebarTitle {\n margin-top:20px;\n font-weight:normal;\n font-size:130%;\n color: #ED1951;\n margin-bottom:10px;\n margin-left: 5px;\n\n}\n\n.navbar-inverse .navbar-toggle:focus, .navbar-inverse .navbar-toggle:hover {\n background-color: #015CAE;\n}\n\n.navbar-inverse .navbar-toggle {\n border-color: #015CAE;\n}\n" }, { "path": "docs/css/theme-green.css", "content": ".summary {\n color: #808080;\n border-left: 5px solid #E50E51;\n font-size:16px;\n}\n\n\nh3 {color: #E50E51; }\nh4 {color: #808080; }\n\n.nav-tabs > li.active > a, .nav-tabs > li.active > a:hover, .nav-tabs > li.active > a:focus {\n background-color: #248ec2;\n color: white;\n}\n\n.nav > li.active > a {\n background-color: #72ac4a;\n}\n\n.nav > li > a:hover {\n background-color: #72ac4a;\n}\n\ndiv.navbar-collapse .dropdown-menu > li > a:hover {\n background-color: #72ac4a;\n}\n\n.navbar-inverse .navbar-nav>li>a, .navbar-inverse .navbar-brand {\n color: white;\n}\n\n.navbar-inverse .navbar-nav>li>a:hover, a.fa.fa-home.fa-lg.navbar-brand:hover {\n color: #f0f0f0;\n}\n\n.nav li.thirdlevel > a {\n background-color: #FAFAFA !important;\n color: #72ac4a;\n font-weight: bold;\n}\n\na[data-toggle=\"tooltip\"] {\n color: #649345;\n font-style: italic;\n cursor: default;\n}\n\n.navbar-inverse {\n background-color: #72ac4a;\n border-color: #5b893c;\n}\n\n.navbar-inverse .navbar-nav > .open > a, .navbar-inverse .navbar-nav > .open > a:hover, .navbar-inverse .navbar-nav > .open > a:focus {\n color: #5b893c;\n}\n\n.navbar-inverse .navbar-nav > .open > a, .navbar-inverse .navbar-nav > .open > a:hover, .navbar-inverse .navbar-nav > .open > a:focus {\n background-color: #5b893c;\n color: #ffffff;\n}\n\n/* not sure if using this ...*/\n.navbar-inverse .navbar-collapse, .navbar-inverse .navbar-form {\n border-color: #72ac4a !important;\n}\n\n.btn-primary {\n color: #ffffff;\n background-color: #5b893c;\n border-color: #5b893c;\n}\n\n.btn-primary:hover,\n.btn-primary:focus,\n.btn-primary:active,\n.btn-primary.active,\n.open .dropdown-toggle.btn-primary {\n background-color: #72ac4a;\n border-color: #5b893c;\n}\n\n.printTitle {\n color: #5b893c !important;\n}\n\nbody.print h1 {color: #5b893c !important; font-size:28px;}\nbody.print h2 {color: #595959 !important; font-size:24px;}\nbody.print h3 {color: #E50E51 !important; font-size:14px;}\nbody.print h4 {color: #679DCE !important; font-size:14px; font-style: italic;}\n\n.anchorjs-link:hover {\n color: #4f7233;\n}\n\ndiv.sidebarTitle {\n color: #E50E51;\n}\n\nli.sidebarTitle {\n margin-top:20px;\n font-weight:normal;\n font-size:130%;\n color: #ED1951;\n margin-bottom:10px;\n margin-left: 5px;\n}\n\n.navbar-inverse .navbar-toggle:focus, .navbar-inverse .navbar-toggle:hover {\n background-color: #E50E51;\n}\n" }, { "path": "docs/feed.xml", "content": "---\nsearch: exclude\nlayout: none\n---\n\n\n\n \n {{ site.title | xml_escape }}\n {{ site.description | xml_escape }}\n {{ site.url }}/\n \n {{ site.time | date_to_rfc822 }}\n {{ site.time | date_to_rfc822 }}\n Jekyll v{{ jekyll.version }}\n {% for post in site.posts limit:10 %}\n \n {{ post.title | xml_escape }}\n {{ post.content | xml_escape }}\n {{ post.date | date_to_rfc822 }}\n {{ post.url | prepend: site.url }}\n {{ post.url | prepend: site.url }}\n {% for tag in post.tags %}\n {{ tag | xml_escape }}\n {% endfor %}\n {% for tag in page.tags %}\n {{ cat | xml_escape }}\n {% endfor %}\n \n {% endfor %}\n \n\n" }, { "path": "docs/index.html", "content": "---\n\ntitle: online_triplet_loss\n\nkeywords: fastai\nsidebar: home_sidebar\n\nsummary: \"PyTorch conversion of the excellent post on the same topic in Tensorflow. Simply an implementation of a triple loss with online mining of candidate triplets used in semi-supervised learning.\"\ndescription: \"PyTorch conversion of the excellent post on the same topic in Tensorflow. Simply an implementation of a triple loss with online mining of candidate triplets used in semi-supervised learning.\"\n---\n\n\n
    \n \n {% raw %}\n \n
    \n\n
    \n {% endraw %}\n\n
    \n
    \n

    Install

    \n
    \n
    \n
    \n
    \n
    \n

    pip install online_triplet_loss

    \n

    Then import with:\nfrom online_triplet_loss.losses import *

    \n

    PS: Requires Pytorch version 1.1.0 or above to use.

    \n\n
    \n
    \n
    \n
    \n
    \n

    How to use

    In these examples I use a really large margin, since the embedding space is so small. A more realistic margins seems to be between 0.1 and 2.0

    \n\n
    \n
    \n
    \n {% raw %}\n \n
    \n
    \n\n
    \n
    \n
    from torch import nn\nimport torch\n\nmodel = nn.Embedding(10, 10)\n
    \n\n
    \n
    \n
    \n\n
    \n {% endraw %}\n\n {% raw %}\n \n
    \n
    \n\n
    \n
    \n
    #from online_triplet_loss.losses import *\nlabels = torch.randint(high=10, size=(5,)) # our five labels\n\nembeddings = model(labels)\nprint('Labels:', labels)\nprint('Embeddings:', embeddings)\nloss = batch_hard_triplet_loss(labels, embeddings, margin=100)\nprint('Loss:', loss)\nloss.backward()\n
    \n\n
    \n
    \n
    \n\n
    \n
    \n\n
    \n\n
    \n
    Labels: tensor([6, 1, 3, 6, 6])\nEmbeddings: tensor([[-1.1335,  0.3364, -3.0174, -0.8732, -0.9301,  1.3619,  0.3746,  0.0457,\n          0.0180, -0.4500],\n        [ 1.0757, -0.8420, -0.7630, -0.0746,  1.1545,  0.4017,  0.5587,  1.7947,\n          0.1992, -2.2288],\n        [ 0.2646,  1.2383,  0.1949,  0.5743, -0.8460, -0.9929, -2.0350,  0.2095,\n          0.2129, -0.4855],\n        [-1.1335,  0.3364, -3.0174, -0.8732, -0.9301,  1.3619,  0.3746,  0.0457,\n          0.0180, -0.4500],\n        [-1.1335,  0.3364, -3.0174, -0.8732, -0.9301,  1.3619,  0.3746,  0.0457,\n          0.0180, -0.4500]], grad_fn=<EmbeddingBackward>)\nLoss: tensor(95.1271, grad_fn=<MeanBackward0>)\n
    \n
    \n
    \n\n
    \n
    \n\n
    \n {% endraw %}\n\n {% raw %}\n \n
    \n
    \n\n
    \n
    \n
    #from online_triplet_loss.losses import *\nembeddings = model(labels)\nprint('Labels:', labels)\nprint('Embeddings:', embeddings)\nloss, fraction_pos = batch_all_triplet_loss(labels, embeddings, squared=False, margin=100)\nprint('Loss:', loss)\nloss.backward()\n
    \n\n
    \n
    \n
    \n\n
    \n
    \n\n
    \n\n
    \n
    Labels: tensor([6, 1, 3, 6, 6])\nEmbeddings: tensor([[-1.1335,  0.3364, -3.0174, -0.8732, -0.9301,  1.3619,  0.3746,  0.0457,\n          0.0180, -0.4500],\n        [ 1.0757, -0.8420, -0.7630, -0.0746,  1.1545,  0.4017,  0.5587,  1.7947,\n          0.1992, -2.2288],\n        [ 0.2646,  1.2383,  0.1949,  0.5743, -0.8460, -0.9929, -2.0350,  0.2095,\n          0.2129, -0.4855],\n        [-1.1335,  0.3364, -3.0174, -0.8732, -0.9301,  1.3619,  0.3746,  0.0457,\n          0.0180, -0.4500],\n        [-1.1335,  0.3364, -3.0174, -0.8732, -0.9301,  1.3619,  0.3746,  0.0457,\n          0.0180, -0.4500]], grad_fn=<EmbeddingBackward>)\ntensor(94.9947, grad_fn=<DivBackward0>) tensor(1.)\nLoss: tensor(94.9947, grad_fn=<DivBackward0>)\n
    \n
    \n
    \n\n
    \n
    \n\n
    \n {% endraw %}\n\n
    \n
    \n

    References

    \n\n
    \n
    \n
    \n
    \n \n\n" }, { "path": "docs/js/customscripts.js", "content": "$('#mysidebar').height($(\".nav\").height());\n\n\n$( document ).ready(function() {\n\n //this script says, if the height of the viewport is greater than 800px, then insert affix class, which makes the nav bar float in a fixed\n // position as your scroll. if you have a lot of nav items, this height may not work for you.\n var h = $(window).height();\n //console.log (h);\n if (h > 800) {\n $( \"#mysidebar\" ).attr(\"class\", \"nav affix\");\n }\n // activate tooltips. although this is a bootstrap js function, it must be activated this way in your theme.\n $('[data-toggle=\"tooltip\"]').tooltip({\n placement : 'top'\n });\n\n /**\n * AnchorJS\n */\n anchors.add('h2,h3,h4,h5');\n\n});\n\n// needed for nav tabs on pages. See Formatting > Nav tabs for more details.\n// script from http://stackoverflow.com/questions/10523433/how-do-i-keep-the-current-tab-active-with-twitter-bootstrap-after-a-page-reload\n$(function() {\n var json, tabsState;\n $('a[data-toggle=\"pill\"], a[data-toggle=\"tab\"]').on('shown.bs.tab', function(e) {\n var href, json, parentId, tabsState;\n\n tabsState = localStorage.getItem(\"tabs-state\");\n json = JSON.parse(tabsState || \"{}\");\n parentId = $(e.target).parents(\"ul.nav.nav-pills, ul.nav.nav-tabs\").attr(\"id\");\n href = $(e.target).attr('href');\n json[parentId] = href;\n\n return localStorage.setItem(\"tabs-state\", JSON.stringify(json));\n });\n\n tabsState = localStorage.getItem(\"tabs-state\");\n json = JSON.parse(tabsState || \"{}\");\n\n $.each(json, function(containerId, href) {\n return $(\"#\" + containerId + \" a[href=\" + href + \"]\").tab('show');\n });\n\n $(\"ul.nav.nav-pills, ul.nav.nav-tabs\").each(function() {\n var $this = $(this);\n if (!json[$this.attr(\"id\")]) {\n return $this.find(\"a[data-toggle=tab]:first, a[data-toggle=pill]:first\").tab(\"show\");\n }\n });\n});\n" }, { "path": "docs/js/jekyll-search.js", "content": "!function e(t,n,r){function s(o,u){if(!n[o]){if(!t[o]){var a=\"function\"==typeof require&&require;if(!u&&a)return a(o,!0);if(i)return i(o,!0);throw new Error(\"Cannot find module '\"+o+\"'\")}var f=n[o]={exports:{}};t[o][0].call(f.exports,function(e){var n=t[o][1][e];return s(n?n:e)},f,f.exports,e,t,n,r)}return n[o].exports}for(var i=\"function\"==typeof require&&require,o=0;o=0}var self=this;self.matches=function(string,crit){return\"string\"!=typeof string?!1:(string=string.trim(),doMatch(string,crit))}}module.exports=new LiteralSearchStrategy},{}],4:[function(require,module){module.exports=function(){function findMatches(store,crit,strategy){for(var data=store.get(),i=0;i{title}',noResultsText:\"No results found\",limit:10,fuzzy:!1};self.init=function(_opt){validateOptions(_opt),assignOptions(_opt),isJSON(opt.dataSource)?initWithJSON(opt.dataSource):initWithURL(opt.dataSource)}}var Searcher=require(\"./Searcher\"),Templater=require(\"./Templater\"),Store=require(\"./Store\"),JSONLoader=require(\"./JSONLoader\"),searcher=new Searcher,templater=new Templater,store=new Store,jsonLoader=new JSONLoader;window.SimpleJekyllSearch=new SimpleJekyllSearch}(window,document)},{\"./JSONLoader\":1,\"./Searcher\":4,\"./Store\":5,\"./Templater\":6}]},{},[7]);\n" }, { "path": "docs/js/toc.js", "content": "// https://github.com/ghiculescu/jekyll-table-of-contents\n// this library modified by fastai to:\n// - update the location.href with the correct anchor when a toc item is clicked on\n(function($){\n $.fn.toc = function(options) {\n var defaults = {\n noBackToTopLinks: false,\n title: '',\n minimumHeaders: 3,\n headers: 'h1, h2, h3, h4',\n listType: 'ol', // values: [ol|ul]\n showEffect: 'show', // values: [show|slideDown|fadeIn|none]\n showSpeed: 'slow' // set to 0 to deactivate effect\n },\n settings = $.extend(defaults, options);\n\n var headers = $(settings.headers).filter(function() {\n // get all headers with an ID\n var previousSiblingName = $(this).prev().attr( \"name\" );\n if (!this.id && previousSiblingName) {\n this.id = $(this).attr( \"id\", previousSiblingName.replace(/\\./g, \"-\") );\n }\n return this.id;\n }), output = $(this);\n if (!headers.length || headers.length < settings.minimumHeaders || !output.length) {\n return;\n }\n\n if (0 === settings.showSpeed) {\n settings.showEffect = 'none';\n }\n\n var render = {\n show: function() { output.hide().html(html).show(settings.showSpeed); },\n slideDown: function() { output.hide().html(html).slideDown(settings.showSpeed); },\n fadeIn: function() { output.hide().html(html).fadeIn(settings.showSpeed); },\n none: function() { output.html(html); }\n };\n\n var get_level = function(ele) { return parseInt(ele.nodeName.replace(\"H\", \"\"), 10); }\n var highest_level = headers.map(function(_, ele) { return get_level(ele); }).get().sort()[0];\n //var return_to_top = '';\n // other nice icons that can be used instead: glyphicon-upload glyphicon-hand-up glyphicon-chevron-up glyphicon-menu-up glyphicon-triangle-top\n var level = get_level(headers[0]),\n this_level,\n html = settings.title + \" <\"+settings.listType+\">\";\n headers.on('click', function() {\n if (!settings.noBackToTopLinks) {\n var pos = $(window).scrollTop();\n window.location.hash = this.id;\n $(window).scrollTop(pos);\n }\n })\n .addClass('clickable-header')\n .each(function(_, header) {\n base_url = window.location.href;\n base_url = base_url.replace(/#.*$/, \"\");\n this_level = get_level(header);\n //if (!settings.noBackToTopLinks && this_level > 1) {\n // $(header).addClass('top-level-header').before(return_to_top);\n //}\n txt = header.textContent.split('ΒΆ')[0].split(/\\[(test|source)\\]/)[0];\n if (!txt) {return;}\n if (this_level === level) // same level as before; same indenting\n html += \"
  • \" + txt + \"\";\n else if (this_level <= level){ // higher level than before; end parent ol\n for(i = this_level; i < level; i++) {\n html += \"
    • \"\n }\n html += \"
  • \" + txt + \"\";\n }\n else if (this_level > level) { // lower level than before; expand the previous to contain a ol\n for(i = this_level; i > level; i--) {\n html += \"<\"+settings.listType+\">\"+((i-level == 2) ? \"
  • \" : \"
  • \")\n }\n html += \"\" + txt + \"\";\n }\n level = this_level; // update for the next one\n });\n html += \"\";\n if (!settings.noBackToTopLinks) {\n $(document).on('click', '.back-to-top', function() {\n $(window).scrollTop(0);\n window.location.hash = '';\n });\n }\n\n render[settings.showEffect]();\n };\n})(jQuery);\n" }, { "path": "docs/licenses/LICENSE", "content": "/* This license pertains to the docs template, except for the Navgoco jQuery component. */\n\nThe MIT License (MIT)\n\nOriginal theme: Copyright (c) 2016 Tom Johnson\nModifications: Copyright (c) 2017 onwards fast.ai, Inc\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "docs/licenses/LICENSE-BSD-NAVGOCO.txt", "content": "/* This license pertains to the Navgoco jQuery component used for the sidebar. */\n\nCopyright (c) 2013, Christodoulos Tsoulloftas, http://www.komposta.net\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without modification,\nare permitted provided that the following conditions are met:\n\n * Redistributions of source code must retain the above copyright notice,\n this list of conditions and the following disclaimer.\n * Redistributions in binary form must reproduce the above copyright notice,\n this list of conditions and the following disclaimer in the documentation\n and/or other materials provided with the distribution.\n * Neither the name of the nor the names of its\n contributors may be used to endorse or promote products derived from this\n software without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND\n ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED\nWARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.\nIN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,\nINDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,\nBUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,\nDATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF\nLIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE\nOR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED\nOF THE POSSIBILITY OF SUCH DAMAGE." }, { "path": "docs/sidebar.json", "content": "{\n \"online_triplet_loss\": {\n \"Overview\": \"/\",\n \"Losses\": \"/triplet_loss\"\n }\n}" }, { "path": "docs/sitemap.xml", "content": "---\nlayout: none\nsearch: exclude\n---\n\n\n\n {% for post in site.posts %}\n {% unless post.search == \"exclude\" %}\n \n {{site.url}}{{post.url}}\n \n {% endunless %}\n {% endfor %}\n\n\n {% for page in site.pages %}\n {% unless page.search == \"exclude\" %}\n \n {{site.url}}{{ page.url}}\n \n {% endunless %}\n {% endfor %}\n" }, { "path": "docs/tooltips.json", "content": "---\nlayout: null\nsearch: exclude\n---\n\n{\n\"entries\":\n[\n{% for page in site.tooltips %}\n{\n\"doc_id\": \"{{ page.doc_id }}\",\n\"body\": \"{{ page.content | strip_newlines | replace: '\\', '\\\\\\\\' | replace: '\"', '\\\\\"' }}\"\n} {% unless forloop.last %},{% endunless %}\n{% endfor %}\n]\n}\n\n\n\n" }, { "path": "docs/triplet_loss.html", "content": "---\n\ntitle: Losses\n\nkeywords: fastai\nsidebar: home_sidebar\n\nsummary: \"Where all of the losses are situated.\"\ndescription: \"Where all of the losses are situated.\"\n---\n\n\n
    \n \n {% raw %}\n \n
    \n\n
    \n {% endraw %}\n\n {% raw %}\n \n
    \n\n
    \n {% endraw %}\n\n {% raw %}\n \n
    \n\n
    \n {% endraw %}\n\n {% raw %}\n \n
    \n\n
    \n
    \n\n
    \n\n\n
    \n

    batch_hard_triplet_loss[source]

    batch_hard_triplet_loss(labels, embeddings, margin, squared=False)

    \n
    \n

    Build the triplet loss over a batch of embeddings.

    \n

    For each anchor, we get the hardest positive and hardest negative to form a triplet.

    \n

    Args:\n labels: labels of the batch, of size (batch_size,)\n embeddings: tensor of shape (batch_size, embed_dim)\n margin: margin for triplet loss\n squared: Boolean. If true, output is the pairwise squared euclidean distance matrix.\n If false, output is the pairwise euclidean distance matrix.

    \n

    Returns:\n triplet_loss: scalar tensor containing the triplet loss

    \n\n
    \n\n
    \n\n
    \n
    \n\n
    \n {% endraw %}\n\n {% raw %}\n \n
    \n\n
    \n {% endraw %}\n\n {% raw %}\n \n
    \n\n
    \n
    \n\n
    \n\n\n
    \n

    batch_all_triplet_loss[source]

    batch_all_triplet_loss(labels, embeddings, margin, squared=False)

    \n
    \n

    Build the triplet loss over a batch of embeddings.

    \n

    We generate all the valid triplets and average the loss over the positive ones.

    \n

    Args:\n labels: labels of the batch, of size (batch_size,)\n embeddings: tensor of shape (batch_size, embed_dim)\n margin: margin for triplet loss\n squared: Boolean. If true, output is the pairwise squared euclidean distance matrix.\n If false, output is the pairwise euclidean distance matrix.

    \n

    Returns:\n triplet_loss: scalar tensor containing the triplet loss

    \n\n
    \n\n
    \n\n
    \n
    \n\n
    \n {% endraw %}\n\n
    \n \n\n" }, { "path": "index.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"#hide\\n\",\n \"from online_triplet_loss.losses import *\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# online_triplet_loss\\n\",\n \"\\n\",\n \"> PyTorch conversion of the excellent post on the [same topic in Tensorflow](https://omoindrot.github.io/triplet-loss). Simply an implementation of a triple loss with online mining of candidate triplets used in semi-supervised learning.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Install\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"`pip install online_triplet_loss`\\n\",\n \"\\n\",\n \"Then import with:\\n\",\n \"`from online_triplet_loss.losses import *`\\n\",\n \"\\n\",\n \"PS: Requires Pytorch version 1.1.0 or above to use.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## How to use\\n\",\n \"\\n\",\n \"In these examples I use a really large margin, since the embedding space is so small. A more realistic margins seems to be between `0.1 and 2.0`\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from torch import nn\\n\",\n \"import torch\\n\",\n \"\\n\",\n \"model = nn.Embedding(10, 10)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": \"Labels: tensor([6, 1, 3, 6, 6])\\nEmbeddings: tensor([[-1.1335, 0.3364, -3.0174, -0.8732, -0.9301, 1.3619, 0.3746, 0.0457,\\n 0.0180, -0.4500],\\n [ 1.0757, -0.8420, -0.7630, -0.0746, 1.1545, 0.4017, 0.5587, 1.7947,\\n 0.1992, -2.2288],\\n [ 0.2646, 1.2383, 0.1949, 0.5743, -0.8460, -0.9929, -2.0350, 0.2095,\\n 0.2129, -0.4855],\\n [-1.1335, 0.3364, -3.0174, -0.8732, -0.9301, 1.3619, 0.3746, 0.0457,\\n 0.0180, -0.4500],\\n [-1.1335, 0.3364, -3.0174, -0.8732, -0.9301, 1.3619, 0.3746, 0.0457,\\n 0.0180, -0.4500]], grad_fn=)\\nLoss: tensor(95.1271, grad_fn=)\\n\"\n }\n ],\n \"source\": [\n \"#from online_triplet_loss.losses import *\\n\",\n \"labels = torch.randint(high=10, size=(5,)) # our five labels\\n\",\n \"\\n\",\n \"embeddings = model(labels)\\n\",\n \"print('Labels:', labels)\\n\",\n \"print('Embeddings:', embeddings)\\n\",\n \"loss = batch_hard_triplet_loss(labels, embeddings, margin=100)\\n\",\n \"print('Loss:', loss)\\n\",\n \"loss.backward()\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": \"Labels: tensor([6, 1, 3, 6, 6])\\nEmbeddings: tensor([[-1.1335, 0.3364, -3.0174, -0.8732, -0.9301, 1.3619, 0.3746, 0.0457,\\n 0.0180, -0.4500],\\n [ 1.0757, -0.8420, -0.7630, -0.0746, 1.1545, 0.4017, 0.5587, 1.7947,\\n 0.1992, -2.2288],\\n [ 0.2646, 1.2383, 0.1949, 0.5743, -0.8460, -0.9929, -2.0350, 0.2095,\\n 0.2129, -0.4855],\\n [-1.1335, 0.3364, -3.0174, -0.8732, -0.9301, 1.3619, 0.3746, 0.0457,\\n 0.0180, -0.4500],\\n [-1.1335, 0.3364, -3.0174, -0.8732, -0.9301, 1.3619, 0.3746, 0.0457,\\n 0.0180, -0.4500]], grad_fn=)\\ntensor(94.9947, grad_fn=) tensor(1.)\\nLoss: tensor(94.9947, grad_fn=)\\n\"\n }\n ],\n \"source\": [\n \"#from online_triplet_loss.losses import *\\n\",\n \"embeddings = model(labels)\\n\",\n \"print('Labels:', labels)\\n\",\n \"print('Embeddings:', embeddings)\\n\",\n \"loss, fraction_pos = batch_all_triplet_loss(labels, embeddings, squared=False, margin=100)\\n\",\n \"print('Loss:', loss)\\n\",\n \"loss.backward()\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"## References\\n\",\n \"* [Triplet Loss and Online Triplet Mining in Tensorflow](https://github.com/omoindrot/tensorflow-triplet-loss)\\n\",\n \"* [Facenet paper](https://arxiv.org/abs/1503.03832)\\n\",\n \"* [adambielski's nice implementation](https://github.com/adambielski/siamese-triplet) (unfortunately context switches between CPU / GPU)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": []\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 2\n}\n" }, { "path": "online_triplet_loss/__init__.py", "content": "__version__ = \"0.0.6\"\n" }, { "path": "online_triplet_loss/_nbdev.py", "content": "# AUTOGENERATED BY NBDEV! DO NOT EDIT!\n\n__all__ = [\"index\", \"modules\", \"custom_doc_links\", \"git_url\"]\n\nindex = {\"batch_hard_triplet_loss\": \"triplet_loss.ipynb\",\n \"batch_all_triplet_loss\": \"triplet_loss.ipynb\"}\n\nmodules = [\"losses.py\"]\n\ndoc_url = \"https://NegatioN.github.io/OnlineMiningTripletLoss/\"\n\ngit_url = \"https://github.com/NegatioN/OnlineMiningTripletLoss/tree/master/\"\n\ndef custom_doc_links(name): return None\n" }, { "path": "online_triplet_loss/losses.py", "content": "# AUTOGENERATED! DO NOT EDIT! File to edit: triplet_loss.ipynb (unless otherwise specified).\n\n__all__ = ['batch_hard_triplet_loss', 'batch_all_triplet_loss']\n\n# Cell\nimport torch\nimport torch.nn.functional as F\ndef _pairwise_distances(embeddings, squared=False):\n \"\"\"Compute the 2D matrix of distances between all the embeddings.\n\n Args:\n embeddings: tensor of shape (batch_size, embed_dim)\n squared: Boolean. If true, output is the pairwise squared euclidean distance matrix.\n If false, output is the pairwise euclidean distance matrix.\n\n Returns:\n pairwise_distances: tensor of shape (batch_size, batch_size)\n \"\"\"\n dot_product = torch.matmul(embeddings, embeddings.t())\n\n # Get squared L2 norm for each embedding. We can just take the diagonal of `dot_product`.\n # This also provides more numerical stability (the diagonal of the result will be exactly 0).\n # shape (batch_size,)\n square_norm = torch.diag(dot_product)\n\n # Compute the pairwise distance matrix as we have:\n # ||a - b||^2 = ||a||^2 - 2 + ||b||^2\n # shape (batch_size, batch_size)\n distances = square_norm.unsqueeze(0) - 2.0 * dot_product + square_norm.unsqueeze(1)\n\n # Because of computation errors, some distances might be negative so we put everything >= 0.0\n distances[distances < 0] = 0\n\n if not squared:\n # Because the gradient of sqrt is infinite when distances == 0.0 (ex: on the diagonal)\n # we need to add a small epsilon where distances == 0.0\n mask = distances.eq(0).float()\n distances = distances + mask * 1e-16\n\n distances = (1.0 -mask) * torch.sqrt(distances)\n\n return distances\n\ndef _get_triplet_mask(labels):\n \"\"\"Return a 3D mask where mask[a, p, n] is True iff the triplet (a, p, n) is valid.\n A triplet (i, j, k) is valid if:\n - i, j, k are distinct\n - labels[i] == labels[j] and labels[i] != labels[k]\n Args:\n labels: tf.int32 `Tensor` with shape [batch_size]\n \"\"\"\n # Check that i, j and k are distinct\n indices_equal = torch.eye(labels.size(0), device=labels.device).bool()\n indices_not_equal = ~indices_equal\n i_not_equal_j = indices_not_equal.unsqueeze(2)\n i_not_equal_k = indices_not_equal.unsqueeze(1)\n j_not_equal_k = indices_not_equal.unsqueeze(0)\n\n distinct_indices = (i_not_equal_j & i_not_equal_k) & j_not_equal_k\n\n\n label_equal = labels.unsqueeze(0) == labels.unsqueeze(1)\n i_equal_j = label_equal.unsqueeze(2)\n i_equal_k = label_equal.unsqueeze(1)\n\n valid_labels = ~i_equal_k & i_equal_j\n\n return valid_labels & distinct_indices\n\n\ndef _get_anchor_positive_triplet_mask(labels):\n \"\"\"Return a 2D mask where mask[a, p] is True iff a and p are distinct and have same label.\n Args:\n labels: tf.int32 `Tensor` with shape [batch_size]\n Returns:\n mask: tf.bool `Tensor` with shape [batch_size, batch_size]\n \"\"\"\n # Check that i and j are distinct\n indices_equal = torch.eye(labels.size(0), device=labels.device).bool()\n indices_not_equal = ~indices_equal\n\n # Check if labels[i] == labels[j]\n # Uses broadcasting where the 1st argument has shape (1, batch_size) and the 2nd (batch_size, 1)\n labels_equal = labels.unsqueeze(0) == labels.unsqueeze(1)\n\n return labels_equal & indices_not_equal\n\n\ndef _get_anchor_negative_triplet_mask(labels):\n \"\"\"Return a 2D mask where mask[a, n] is True iff a and n have distinct labels.\n Args:\n labels: tf.int32 `Tensor` with shape [batch_size]\n Returns:\n mask: tf.bool `Tensor` with shape [batch_size, batch_size]\n \"\"\"\n # Check if labels[i] != labels[k]\n # Uses broadcasting where the 1st argument has shape (1, batch_size) and the 2nd (batch_size, 1)\n\n return ~(labels.unsqueeze(0) == labels.unsqueeze(1))\n\n\n# Cell\ndef batch_hard_triplet_loss(labels, embeddings, margin, squared=False):\n \"\"\"Build the triplet loss over a batch of embeddings.\n\n For each anchor, we get the hardest positive and hardest negative to form a triplet.\n\n Args:\n labels: labels of the batch, of size (batch_size,)\n embeddings: tensor of shape (batch_size, embed_dim)\n margin: margin for triplet loss\n squared: Boolean. If true, output is the pairwise squared euclidean distance matrix.\n If false, output is the pairwise euclidean distance matrix.\n\n Returns:\n triplet_loss: scalar tensor containing the triplet loss\n \"\"\"\n # Get the pairwise distance matrix\n pairwise_dist = _pairwise_distances(embeddings, squared=squared)\n\n # For each anchor, get the hardest positive\n # First, we need to get a mask for every valid positive (they should have same label)\n mask_anchor_positive = _get_anchor_positive_triplet_mask(labels).float()\n\n # We put to 0 any element where (a, p) is not valid (valid if a != p and label(a) == label(p))\n anchor_positive_dist = mask_anchor_positive * pairwise_dist\n\n # shape (batch_size, 1)\n hardest_positive_dist, _ = anchor_positive_dist.max(1, keepdim=True)\n\n # For each anchor, get the hardest negative\n # First, we need to get a mask for every valid negative (they should have different labels)\n mask_anchor_negative = _get_anchor_negative_triplet_mask(labels).float()\n\n # We add the maximum value in each row to the invalid negatives (label(a) == label(n))\n max_anchor_negative_dist, _ = pairwise_dist.max(1, keepdim=True)\n anchor_negative_dist = pairwise_dist + max_anchor_negative_dist * (1.0 - mask_anchor_negative)\n\n # shape (batch_size,)\n hardest_negative_dist, _ = anchor_negative_dist.min(1, keepdim=True)\n\n # Combine biggest d(a, p) and smallest d(a, n) into final triplet loss\n tl = hardest_positive_dist - hardest_negative_dist + margin\n tl = F.relu(tl)\n triplet_loss = tl.mean()\n\n return triplet_loss\n\n# Cell\ndef batch_all_triplet_loss(labels, embeddings, margin, squared=False):\n \"\"\"Build the triplet loss over a batch of embeddings.\n\n We generate all the valid triplets and average the loss over the positive ones.\n\n Args:\n labels: labels of the batch, of size (batch_size,)\n embeddings: tensor of shape (batch_size, embed_dim)\n margin: margin for triplet loss\n squared: Boolean. If true, output is the pairwise squared euclidean distance matrix.\n If false, output is the pairwise euclidean distance matrix.\n\n Returns:\n triplet_loss: scalar tensor containing the triplet loss\n \"\"\"\n # Get the pairwise distance matrix\n pairwise_dist = _pairwise_distances(embeddings, squared=squared)\n\n anchor_positive_dist = pairwise_dist.unsqueeze(2)\n anchor_negative_dist = pairwise_dist.unsqueeze(1)\n\n # Compute a 3D tensor of size (batch_size, batch_size, batch_size)\n # triplet_loss[i, j, k] will contain the triplet loss of anchor=i, positive=j, negative=k\n # Uses broadcasting where the 1st argument has shape (batch_size, batch_size, 1)\n # and the 2nd (batch_size, 1, batch_size)\n triplet_loss = anchor_positive_dist - anchor_negative_dist + margin\n\n\n\n # Put to zero the invalid triplets\n # (where label(a) != label(p) or label(n) == label(a) or a == p)\n mask = _get_triplet_mask(labels)\n triplet_loss = mask.float() * triplet_loss\n\n # Remove negative losses (i.e. the easy triplets)\n triplet_loss = F.relu(triplet_loss)\n\n # Count number of positive triplets (where triplet_loss > 0)\n valid_triplets = triplet_loss[triplet_loss > 1e-16]\n num_positive_triplets = valid_triplets.size(0)\n num_valid_triplets = mask.sum()\n\n fraction_positive_triplets = num_positive_triplets / (num_valid_triplets.float() + 1e-16)\n\n # Get final mean triplet loss over the positive valid triplets\n triplet_loss = triplet_loss.sum() / (num_positive_triplets + 1e-16)\n\n return triplet_loss, fraction_positive_triplets\n" }, { "path": "settings.ini", "content": "[DEFAULT]\nlib_name = online_triplet_loss\nuser = NegatioN\ndescription = \"Online mining triplet losses for Pytorch\"\nkeywords = pytorch loss online triplet mining\nauthor = \"Joakim Rishaug\"\nauthor_email = \"joakimrishaug@notmyrealemail.com\"\ncopyright = \"Joakim Rishaug\"\nbranch = master\nversion = 0.0.6\nmin_python = 3.6\naudience = Developers\nlanguage = English\ncustom_sidebar = False\nlicense = apache2\nstatus = 2\nnbs_path = .\ndoc_path = docs\nrequirements = numpy>=1.15.4 torch>=1.1.0\nrepo_name = OnlineMiningTripletLoss\ndoc_host = https://NegatioN.github.io\ndoc_baseurl = /OnlineMiningTripletLoss/\ngit_url = https://github.com/NegatioN/OnlineMiningTripletLoss/tree/master/\nlib_path = online_triplet_loss\ntitle = OnlineMiningTripletLoss\nhost = github\n\n" }, { "path": "setup.py", "content": "from packaging.version import parse\nfrom configparser import ConfigParser\nimport setuptools\nassert parse(setuptools.__version__)>=parse('36.2')\n\n# note: all settings are in settings.ini; edit there, not here\nconfig = ConfigParser(delimiters=['='])\nconfig.read('settings.ini')\ncfg = config['DEFAULT']\n\ncfg_keys = 'version description keywords author author_email'.split()\nexpected = cfg_keys + \"lib_name user branch license status min_python audience language\".split()\nfor o in expected: assert o in cfg, \"missing expected setting: {}\".format(o)\nsetup_cfg = {o:cfg[o] for o in cfg_keys}\n\nlicenses = {\n 'apache2': ('Apache Software License 2.0','OSI Approved :: Apache Software License'),\n}\nstatuses = [ '1 - Planning', '2 - Pre-Alpha', '3 - Alpha',\n '4 - Beta', '5 - Production/Stable', '6 - Mature', '7 - Inactive' ]\npy_versions = '2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8'.split()\n\nrequirements = cfg.get('requirements','').split()\nlic = licenses[cfg['license']]\nmin_python = cfg['min_python']\n\nsetuptools.setup(\n name = cfg['lib_name'],\n license = lic[0],\n classifiers = [\n 'Development Status :: ' + statuses[int(cfg['status'])],\n 'Intended Audience :: ' + cfg['audience'].title(),\n 'License :: ' + lic[1],\n 'Natural Language :: ' + cfg['language'].title(),\n ] + ['Programming Language :: Python :: '+o for o in py_versions[py_versions.index(min_python):]],\n url = 'https://github.com/{}/{}'.format(cfg['user'],cfg['repo_name']),\n packages = setuptools.find_packages(),\n include_package_data = True,\n install_requires = requirements,\n python_requires = '>=' + cfg['min_python'],\n long_description = open('README.md').read(),\n long_description_content_type = 'text/markdown',\n zip_safe = False,\n entry_points = { 'console_scripts': cfg.get('console_scripts','').split() },\n **setup_cfg)\n\n" }, { "path": "triplet_loss.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# default_exp losses\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Losses\\n\",\n \"> Where all of the losses are situated.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"#hide\\n\",\n \"from nbdev.showdoc import *\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"#export\\n\",\n \"import torch\\n\",\n \"import torch.nn.functional as F\\n\",\n \"def _pairwise_distances(embeddings, squared=False):\\n\",\n \" \\\"\\\"\\\"Compute the 2D matrix of distances between all the embeddings.\\n\",\n \"\\n\",\n \" Args:\\n\",\n \" embeddings: tensor of shape (batch_size, embed_dim)\\n\",\n \" squared: Boolean. If true, output is the pairwise squared euclidean distance matrix.\\n\",\n \" If false, output is the pairwise euclidean distance matrix.\\n\",\n \"\\n\",\n \" Returns:\\n\",\n \" pairwise_distances: tensor of shape (batch_size, batch_size)\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" dot_product = torch.matmul(embeddings, embeddings.t())\\n\",\n \"\\n\",\n \" # Get squared L2 norm for each embedding. We can just take the diagonal of `dot_product`.\\n\",\n \" # This also provides more numerical stability (the diagonal of the result will be exactly 0).\\n\",\n \" # shape (batch_size,)\\n\",\n \" square_norm = torch.diag(dot_product)\\n\",\n \"\\n\",\n \" # Compute the pairwise distance matrix as we have:\\n\",\n \" # ||a - b||^2 = ||a||^2 - 2 + ||b||^2\\n\",\n \" # shape (batch_size, batch_size)\\n\",\n \" distances = square_norm.unsqueeze(0) - 2.0 * dot_product + square_norm.unsqueeze(1)\\n\",\n \"\\n\",\n \" # Because of computation errors, some distances might be negative so we put everything >= 0.0\\n\",\n \" distances[distances < 0] = 0\\n\",\n \"\\n\",\n \" if not squared:\\n\",\n \" # Because the gradient of sqrt is infinite when distances == 0.0 (ex: on the diagonal)\\n\",\n \" # we need to add a small epsilon where distances == 0.0\\n\",\n \" mask = distances.eq(0).float()\\n\",\n \" distances = distances + mask * 1e-16\\n\",\n \"\\n\",\n \" distances = (1.0 -mask) * torch.sqrt(distances)\\n\",\n \"\\n\",\n \" return distances\\n\",\n \"\\n\",\n \"def _get_triplet_mask(labels):\\n\",\n \" \\\"\\\"\\\"Return a 3D mask where mask[a, p, n] is True iff the triplet (a, p, n) is valid.\\n\",\n \" A triplet (i, j, k) is valid if:\\n\",\n \" - i, j, k are distinct\\n\",\n \" - labels[i] == labels[j] and labels[i] != labels[k]\\n\",\n \" Args:\\n\",\n \" labels: tf.int32 `Tensor` with shape [batch_size]\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # Check that i, j and k are distinct\\n\",\n \" indices_equal = torch.eye(labels.size(0), device=labels.device).bool()\\n\",\n \" indices_not_equal = ~indices_equal\\n\",\n \" i_not_equal_j = indices_not_equal.unsqueeze(2)\\n\",\n \" i_not_equal_k = indices_not_equal.unsqueeze(1)\\n\",\n \" j_not_equal_k = indices_not_equal.unsqueeze(0)\\n\",\n \"\\n\",\n \" distinct_indices = (i_not_equal_j & i_not_equal_k) & j_not_equal_k\\n\",\n \"\\n\",\n \"\\n\",\n \" label_equal = labels.unsqueeze(0) == labels.unsqueeze(1)\\n\",\n \" i_equal_j = label_equal.unsqueeze(2)\\n\",\n \" i_equal_k = label_equal.unsqueeze(1)\\n\",\n \"\\n\",\n \" valid_labels = ~i_equal_k & i_equal_j\\n\",\n \"\\n\",\n \" return valid_labels & distinct_indices\\n\",\n \"\\n\",\n \"\\n\",\n \"def _get_anchor_positive_triplet_mask(labels):\\n\",\n \" \\\"\\\"\\\"Return a 2D mask where mask[a, p] is True iff a and p are distinct and have same label.\\n\",\n \" Args:\\n\",\n \" labels: tf.int32 `Tensor` with shape [batch_size]\\n\",\n \" Returns:\\n\",\n \" mask: tf.bool `Tensor` with shape [batch_size, batch_size]\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # Check that i and j are distinct\\n\",\n \" indices_equal = torch.eye(labels.size(0), device=labels.device).bool()\\n\",\n \" indices_not_equal = ~indices_equal\\n\",\n \"\\n\",\n \" # Check if labels[i] == labels[j]\\n\",\n \" # Uses broadcasting where the 1st argument has shape (1, batch_size) and the 2nd (batch_size, 1)\\n\",\n \" labels_equal = labels.unsqueeze(0) == labels.unsqueeze(1)\\n\",\n \"\\n\",\n \" return labels_equal & indices_not_equal\\n\",\n \"\\n\",\n \"\\n\",\n \"def _get_anchor_negative_triplet_mask(labels):\\n\",\n \" \\\"\\\"\\\"Return a 2D mask where mask[a, n] is True iff a and n have distinct labels.\\n\",\n \" Args:\\n\",\n \" labels: tf.int32 `Tensor` with shape [batch_size]\\n\",\n \" Returns:\\n\",\n \" mask: tf.bool `Tensor` with shape [batch_size, batch_size]\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # Check if labels[i] != labels[k]\\n\",\n \" # Uses broadcasting where the 1st argument has shape (1, batch_size) and the 2nd (batch_size, 1)\\n\",\n \"\\n\",\n \" return ~(labels.unsqueeze(0) == labels.unsqueeze(1))\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"#hide\\n\",\n \"import numpy as np\\n\",\n \"\\n\",\n \"# Test-suite from https://github.com/omoindrot/tensorflow-triplet-loss/blob/master/model/tests/test_triplet_loss.py\\n\",\n \"# Skipped the `test_gradients_pairwise_distances()` test since it's trivial to see if your model loss turns NaN\\n\",\n \"# and porting it proved more difficult than expected.\\n\",\n \"\\n\",\n \"def pairwise_distance_np(feature, squared=False):\\n\",\n \" \\\"\\\"\\\"Computes the pairwise distance matrix in numpy.\\n\",\n \" Args:\\n\",\n \" feature: 2-D numpy array of size [number of data, feature dimension]\\n\",\n \" squared: Boolean. If true, output is the pairwise squared euclidean\\n\",\n \" distance matrix; else, output is the pairwise euclidean distance matrix.\\n\",\n \" Returns:\\n\",\n \" pairwise_distances: 2-D numpy array of size\\n\",\n \" [number of data, number of data].\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" triu = np.triu_indices(feature.shape[0], 1)\\n\",\n \" upper_tri_pdists = np.linalg.norm(feature[triu[1]] - feature[triu[0]], axis=1)\\n\",\n \" if squared:\\n\",\n \" upper_tri_pdists **= 2.\\n\",\n \" num_data = feature.shape[0]\\n\",\n \" pairwise_distances = np.zeros((num_data, num_data))\\n\",\n \" pairwise_distances[np.triu_indices(num_data, 1)] = upper_tri_pdists\\n\",\n \" # Make symmetrical.\\n\",\n \" pairwise_distances = pairwise_distances + pairwise_distances.T - np.diag(\\n\",\n \" pairwise_distances.diagonal())\\n\",\n \" return pairwise_distances\\n\",\n \"\\n\",\n \"def test_pairwise_distances():\\n\",\n \" \\\"\\\"\\\"Test the pairwise distances function.\\\"\\\"\\\"\\n\",\n \" num_data = 64\\n\",\n \" feat_dim = 6\\n\",\n \" np.random.seed(42)\\n\",\n \"\\n\",\n \" embeddings = np.random.randn(num_data, feat_dim).astype(np.float32)\\n\",\n \" embeddings[1] = embeddings[0] # to get distance 0\\n\",\n \"\\n\",\n \" for squared in [True, False]:\\n\",\n \" res_np = pairwise_distance_np(embeddings, squared=squared)\\n\",\n \" res_pt = _pairwise_distances(torch.from_numpy(embeddings), squared=squared)\\n\",\n \" assert np.allclose(res_np, res_pt)\\n\",\n \"\\n\",\n \"def test_pairwise_distances_are_positive():\\n\",\n \" \\\"\\\"\\\"Test that the pairwise distances are always positive.\\n\",\n \" Use a tricky case where numerical errors are common.\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" num_data = 64\\n\",\n \" feat_dim = 6\\n\",\n \" np.random.seed(42)\\n\",\n \"\\n\",\n \" # Create embeddings very close to each other in [1.0 - 2e-7, 1.0 + 2e-7]\\n\",\n \" # This will encourage errors in the computation\\n\",\n \" embeddings = 1.0 + 2e-7 * np.random.randn(num_data, feat_dim).astype(np.float32)\\n\",\n \" embeddings[1] = embeddings[0] # to get distance 0\\n\",\n \"\\n\",\n \" for squared in [True, False]:\\n\",\n \" res_tf = _pairwise_distances(torch.from_numpy(embeddings), squared=squared)\\n\",\n \" assert res_tf[res_tf < 0].sum() == 0\\n\",\n \"\\n\",\n \"\\n\",\n \"def test_triplet_mask():\\n\",\n \" \\\"\\\"\\\"Test function _get_triplet_mask.\\\"\\\"\\\"\\n\",\n \" num_data = 64\\n\",\n \" num_classes = 10\\n\",\n \" np.random.seed(42)\\n\",\n \"\\n\",\n \" labels = np.random.randint(0, num_classes, size=(num_data)).astype(np.float32)\\n\",\n \"\\n\",\n \" mask_np = np.zeros((num_data, num_data, num_data))\\n\",\n \" for i in range(num_data):\\n\",\n \" for j in range(num_data):\\n\",\n \" for k in range(num_data):\\n\",\n \" distinct = (i != j and i != k and j != k)\\n\",\n \" valid = (labels[i] == labels[j]) and (labels[i] != labels[k])\\n\",\n \" mask_np[i, j, k] = (distinct and valid)\\n\",\n \"\\n\",\n \" mask_tf_val = _get_triplet_mask(torch.from_numpy(labels))\\n\",\n \" assert np.allclose(mask_np, mask_tf_val)\\n\",\n \"\\n\",\n \"def test_anchor_positive_triplet_mask():\\n\",\n \" \\\"\\\"\\\"Test function _get_anchor_positive_triplet_mask.\\\"\\\"\\\"\\n\",\n \" num_data = 64\\n\",\n \" num_classes = 10\\n\",\n \" np.random.seed(42)\\n\",\n \"\\n\",\n \" labels = np.random.randint(0, num_classes, size=(num_data)).astype(np.float32)\\n\",\n \"\\n\",\n \" mask_np = np.zeros((num_data, num_data))\\n\",\n \" for i in range(num_data):\\n\",\n \" for j in range(num_data):\\n\",\n \" distinct = (i != j)\\n\",\n \" valid = labels[i] == labels[j]\\n\",\n \" mask_np[i, j] = (distinct and valid)\\n\",\n \"\\n\",\n \" mask_tf_val = _get_anchor_positive_triplet_mask(torch.from_numpy(labels))\\n\",\n \"\\n\",\n \" assert np.allclose(mask_np, mask_tf_val)\\n\",\n \"\\n\",\n \"def test_anchor_negative_triplet_mask():\\n\",\n \" \\\"\\\"\\\"Test function _get_anchor_negative_triplet_mask.\\\"\\\"\\\"\\n\",\n \" num_data = 64\\n\",\n \" num_classes = 10\\n\",\n \" np.random.seed(42)\\n\",\n \"\\n\",\n \" labels = np.random.randint(0, num_classes, size=(num_data)).astype(np.float32)\\n\",\n \"\\n\",\n \" mask_np = np.zeros((num_data, num_data))\\n\",\n \" for i in range(num_data):\\n\",\n \" for k in range(num_data):\\n\",\n \" distinct = (i != k)\\n\",\n \" valid = (labels[i] != labels[k])\\n\",\n \" mask_np[i, k] = (distinct and valid)\\n\",\n \"\\n\",\n \" mask_tf_val = _get_anchor_negative_triplet_mask(torch.from_numpy(labels))\\n\",\n \"\\n\",\n \" assert np.allclose(mask_np, mask_tf_val)\\n\",\n \"\\n\",\n \"test_pairwise_distances()\\n\",\n \"test_pairwise_distances_are_positive()\\n\",\n \"test_triplet_mask()\\n\",\n \"test_anchor_positive_triplet_mask()\\n\",\n \"test_anchor_negative_triplet_mask()\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"#export\\n\",\n \"def batch_hard_triplet_loss(labels, embeddings, margin, squared=False):\\n\",\n \" \\\"\\\"\\\"Build the triplet loss over a batch of embeddings.\\n\",\n \"\\n\",\n \" For each anchor, we get the hardest positive and hardest negative to form a triplet.\\n\",\n \"\\n\",\n \" Args:\\n\",\n \" labels: labels of the batch, of size (batch_size,)\\n\",\n \" embeddings: tensor of shape (batch_size, embed_dim)\\n\",\n \" margin: margin for triplet loss\\n\",\n \" squared: Boolean. If true, output is the pairwise squared euclidean distance matrix.\\n\",\n \" If false, output is the pairwise euclidean distance matrix.\\n\",\n \"\\n\",\n \" Returns:\\n\",\n \" triplet_loss: scalar tensor containing the triplet loss\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # Get the pairwise distance matrix\\n\",\n \" pairwise_dist = _pairwise_distances(embeddings, squared=squared)\\n\",\n \"\\n\",\n \" # For each anchor, get the hardest positive\\n\",\n \" # First, we need to get a mask for every valid positive (they should have same label)\\n\",\n \" mask_anchor_positive = _get_anchor_positive_triplet_mask(labels).float()\\n\",\n \"\\n\",\n \" # We put to 0 any element where (a, p) is not valid (valid if a != p and label(a) == label(p))\\n\",\n \" anchor_positive_dist = mask_anchor_positive * pairwise_dist\\n\",\n \"\\n\",\n \" # shape (batch_size, 1)\\n\",\n \" hardest_positive_dist, _ = anchor_positive_dist.max(1, keepdim=True)\\n\",\n \"\\n\",\n \" # For each anchor, get the hardest negative\\n\",\n \" # First, we need to get a mask for every valid negative (they should have different labels)\\n\",\n \" mask_anchor_negative = _get_anchor_negative_triplet_mask(labels).float()\\n\",\n \"\\n\",\n \" # We add the maximum value in each row to the invalid negatives (label(a) == label(n))\\n\",\n \" max_anchor_negative_dist, _ = pairwise_dist.max(1, keepdim=True)\\n\",\n \" anchor_negative_dist = pairwise_dist + max_anchor_negative_dist * (1.0 - mask_anchor_negative)\\n\",\n \"\\n\",\n \" # shape (batch_size,)\\n\",\n \" hardest_negative_dist, _ = anchor_negative_dist.min(1, keepdim=True)\\n\",\n \"\\n\",\n \" # Combine biggest d(a, p) and smallest d(a, n) into final triplet loss\\n\",\n \" tl = hardest_positive_dist - hardest_negative_dist + margin\\n\",\n \" tl = F.relu(tl)\\n\",\n \" triplet_loss = tl.mean()\\n\",\n \"\\n\",\n \" return triplet_loss\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"#export\\n\",\n \"def batch_all_triplet_loss(labels, embeddings, margin, squared=False):\\n\",\n \" \\\"\\\"\\\"Build the triplet loss over a batch of embeddings.\\n\",\n \"\\n\",\n \" We generate all the valid triplets and average the loss over the positive ones.\\n\",\n \"\\n\",\n \" Args:\\n\",\n \" labels: labels of the batch, of size (batch_size,)\\n\",\n \" embeddings: tensor of shape (batch_size, embed_dim)\\n\",\n \" margin: margin for triplet loss\\n\",\n \" squared: Boolean. If true, output is the pairwise squared euclidean distance matrix.\\n\",\n \" If false, output is the pairwise euclidean distance matrix.\\n\",\n \"\\n\",\n \" Returns:\\n\",\n \" triplet_loss: scalar tensor containing the triplet loss\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # Get the pairwise distance matrix\\n\",\n \" pairwise_dist = _pairwise_distances(embeddings, squared=squared)\\n\",\n \"\\n\",\n \" anchor_positive_dist = pairwise_dist.unsqueeze(2)\\n\",\n \" anchor_negative_dist = pairwise_dist.unsqueeze(1)\\n\",\n \"\\n\",\n \" # Compute a 3D tensor of size (batch_size, batch_size, batch_size)\\n\",\n \" # triplet_loss[i, j, k] will contain the triplet loss of anchor=i, positive=j, negative=k\\n\",\n \" # Uses broadcasting where the 1st argument has shape (batch_size, batch_size, 1)\\n\",\n \" # and the 2nd (batch_size, 1, batch_size)\\n\",\n \" triplet_loss = anchor_positive_dist - anchor_negative_dist + margin\\n\",\n \"\\n\",\n \"\\n\",\n \"\\n\",\n \" # Put to zero the invalid triplets\\n\",\n \" # (where label(a) != label(p) or label(n) == label(a) or a == p)\\n\",\n \" mask = _get_triplet_mask(labels)\\n\",\n \" triplet_loss = mask.float() * triplet_loss\\n\",\n \"\\n\",\n \" # Remove negative losses (i.e. the easy triplets)\\n\",\n \" triplet_loss = F.relu(triplet_loss)\\n\",\n \"\\n\",\n \" # Count number of positive triplets (where triplet_loss > 0)\\n\",\n \" valid_triplets = triplet_loss[triplet_loss > 1e-16]\\n\",\n \" num_positive_triplets = valid_triplets.size(0)\\n\",\n \" num_valid_triplets = mask.sum()\\n\",\n \"\\n\",\n \" fraction_positive_triplets = num_positive_triplets / (num_valid_triplets.float() + 1e-16)\\n\",\n \"\\n\",\n \" # Get final mean triplet loss over the positive valid triplets\\n\",\n \" triplet_loss = triplet_loss.sum() / (num_positive_triplets + 1e-16)\\n\",\n \"\\n\",\n \" return triplet_loss, fraction_positive_triplets\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"tensor(0.) tensor(0.)\\n\",\n \"tensor(0.) tensor(0.)\\n\",\n \"tensor(0.7060) tensor(0.7155)\\n\",\n \"tensor(0.3978) tensor(0.8276)\\n\"\n ]\n }\n ],\n \"source\": [\n \"#hide\\n\",\n \"def test_simple_batch_all_triplet_loss():\\n\",\n \" \\\"\\\"\\\"Test the triplet loss with batch all triplet mining in a simple case.\\n\",\n \" There is just one class in this super simple edge case, and we want to make sure that\\n\",\n \" the loss is 0.\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" num_data = 10\\n\",\n \" feat_dim = 6\\n\",\n \" margin = 0.2\\n\",\n \" num_classes = 1\\n\",\n \" np.random.seed(42)\\n\",\n \"\\n\",\n \" embeddings = np.random.rand(num_data, feat_dim).astype(np.float32)\\n\",\n \" labels = np.random.randint(0, num_classes, size=(num_data)).astype(np.float32)\\n\",\n \" labels, embeddings = torch.from_numpy(labels), torch.from_numpy(embeddings)\\n\",\n \"\\n\",\n \" for squared in [True, False]:\\n\",\n \" loss_np = 0.0\\n\",\n \"\\n\",\n \" # Compute the loss in TF.\\n\",\n \" loss_tf_val, fraction_val = batch_all_triplet_loss(labels, embeddings, margin, squared=squared)\\n\",\n \" print(loss_tf_val, fraction_val)\\n\",\n \" assert np.allclose(loss_np, loss_tf_val)\\n\",\n \" assert np.allclose(fraction_val, 0.0)\\n\",\n \"\\n\",\n \"\\n\",\n \"def test_batch_all_triplet_loss():\\n\",\n \" \\\"\\\"\\\"Test the triplet loss with batch all triplet mining\\\"\\\"\\\"\\n\",\n \" num_data = 10\\n\",\n \" feat_dim = 6\\n\",\n \" margin = 0.2\\n\",\n \" num_classes = 5\\n\",\n \" np.random.seed(42)\\n\",\n \"\\n\",\n \" embeddings = np.random.rand(num_data, feat_dim).astype(np.float32)\\n\",\n \" labels = np.random.randint(0, num_classes, size=(num_data)).astype(np.float32)\\n\",\n \"\\n\",\n \" for squared in [True, False]:\\n\",\n \" pdist_matrix = pairwise_distance_np(embeddings, squared=squared)\\n\",\n \"\\n\",\n \" loss_np = 0.0\\n\",\n \" num_positives = 0.0\\n\",\n \" num_valid = 0.0\\n\",\n \" for i in range(num_data):\\n\",\n \" for j in range(num_data):\\n\",\n \" for k in range(num_data):\\n\",\n \" distinct = (i != j and i != k and j != k)\\n\",\n \" valid = (labels[i] == labels[j]) and (labels[i] != labels[k])\\n\",\n \" if distinct and valid:\\n\",\n \" num_valid += 1.0\\n\",\n \"\\n\",\n \" pos_distance = pdist_matrix[i][j]\\n\",\n \" neg_distance = pdist_matrix[i][k]\\n\",\n \"\\n\",\n \" loss = np.maximum(0.0, pos_distance - neg_distance + margin)\\n\",\n \" loss_np += loss\\n\",\n \"\\n\",\n \" num_positives += (loss > 0)\\n\",\n \"\\n\",\n \" loss_np /= num_positives\\n\",\n \"\\n\",\n \" # Compute the loss in TF.\\n\",\n \" loss_tf_val, fraction_val = batch_all_triplet_loss(torch.from_numpy(labels), torch.from_numpy(embeddings), margin, squared=squared)\\n\",\n \" print(loss_tf_val, fraction_val)\\n\",\n \" assert np.allclose(loss_np, loss_tf_val)\\n\",\n \" assert np.allclose(num_positives / num_valid, fraction_val)\\n\",\n \"\\n\",\n \"def test_batch_hard_triplet_loss():\\n\",\n \" \\\"\\\"\\\"Test the triplet loss with batch hard triplet mining\\\"\\\"\\\"\\n\",\n \" num_data = 50\\n\",\n \" feat_dim = 6\\n\",\n \" margin = 0.2\\n\",\n \" num_classes = 5\\n\",\n \" np.random.seed(42)\\n\",\n \"\\n\",\n \" embeddings = np.random.rand(num_data, feat_dim).astype(np.float32)\\n\",\n \" labels = np.random.randint(0, num_classes, size=(num_data)).astype(np.float32)\\n\",\n \"\\n\",\n \" for squared in [True, False]:\\n\",\n \" pdist_matrix = pairwise_distance_np(embeddings, squared=squared)\\n\",\n \"\\n\",\n \" loss_np = 0.0\\n\",\n \" for i in range(num_data):\\n\",\n \" # Select the hardest positive\\n\",\n \" max_pos_dist = np.max(pdist_matrix[i][labels == labels[i]])\\n\",\n \"\\n\",\n \" # Select the hardest negative\\n\",\n \" min_neg_dist = np.min(pdist_matrix[i][labels != labels[i]])\\n\",\n \"\\n\",\n \" loss = np.maximum(0.0, max_pos_dist - min_neg_dist + margin)\\n\",\n \" loss_np += loss\\n\",\n \"\\n\",\n \" loss_np /= num_data\\n\",\n \"\\n\",\n \" # Compute the loss in TF.\\n\",\n \" loss_tf_val = batch_hard_triplet_loss(torch.from_numpy(labels), torch.from_numpy(embeddings), margin, squared=squared)\\n\",\n \" assert np.allclose(loss_np, loss_tf_val)\\n\",\n \"\\n\",\n \"test_simple_batch_all_triplet_loss()\\n\",\n \"test_batch_all_triplet_loss()\\n\",\n \"test_batch_hard_triplet_loss()\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": []\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" } ]