[ { "path": ".dockerignore", "content": ".git\n.github\nscripts\ndocs" }, { "path": ".github/README-exec/onnx.readme.md", "content": "# CLIPOnnxEncoder\n\n**CLIPOnnxEncoder** is the executor implemented in [CLIP-as-service](https://github.com/jina-ai/clip-as-service). \nThe various `CLIP` models implemented in the [OpenAI](https://github.com/openai/CLIP) and [OpenCLIP](https://github.com/mlfoundations/open_clip) are supported with ONNX runtime (๐Ÿš€ **3x** speed up). \nThe introduction of the CLIP model [can be found here](https://openai.com/blog/clip/).\n\n- ๐Ÿ”€ **Automatic**: Auto-detect image and text documents depending on their content.\n- โšก **Efficiency**: Faster CLIP model inference on CPU and GPU via ONNX runtime. \n- ๐Ÿ“ˆ **Observability**: Monitoring the serving via Prometheus and Grafana (see [Usage Guide](https://docs.jina.ai/how-to/monitoring/#deploying-locally)).\n\n\n## Model support\n\n `ViT-B-32::openai` is used as the default model. To use specific pretrained models provided by `open_clip`, please use `::` to separate model name and pretrained weight name, e.g. `ViT-B-32::laion2b_e16`. Please also note that **different models give different sizes of output dimensions**.\n\n| Model | ONNX | Output dimension | \n|---------------------------------------|------|------------------|\n| RN50 | โœ… | 1024 | \n| RN101 | โœ… | 512 | \n| RN50x4 | โœ… | 640 |\n| RN50x16 | โœ… | 768 |\n| RN50x64 | โœ… | 1024 |\n| ViT-B-32 | โœ… | 512 |\n| ViT-B-16 | โœ… | 512 |\n| ViT-B-16-plus-240 | โœ… | 640 |\n| ViT-L-14 | โœ… | 768 |\n| ViT-L-14-336 | โœ… | 768 |\n| ViT-H-14 | โœ… | 1024 |\n| ViT-g-14 | โœ… | 1024 |\n| M-CLIP/XLM_Roberta-Large-Vit-B-32 | โœ… | 512 |\n| M-CLIP/XLM-Roberta-Large-Vit-L-14 | โœ… | 768 |\n| M-CLIP/XLM-Roberta-Large-Vit-B-16Plus | โœ… | 640 |\n| M-CLIP/LABSE-Vit-L-14 | โœ… | 768 |\n\nโœ… = First class support \n\nFull list of open_clip models and weights can be found [here](https://github.com/mlfoundations/open_clip#pretrained-model-interface).\n\n```{note}\nFor model definition with `-quickgelu` postfix, please use non `-quickgelu` model name.\n```\n\n## Usage\n\n### Use in Jina Flow \n\n- **via Docker image (recommended)**\n\n```python\nfrom jina import Flow\nfrom docarray import Document\nimport numpy as np\n\nf = Flow().add(\n uses='jinahub+docker://CLIPOnnxEncoder',\n)\n```\n\n- **via source code**\n\n```python\nfrom jina import Flow\nfrom docarray import Document\nimport numpy as np\n\nf = Flow().add(\n uses='jinahub://CLIPOnnxEncoder',\n)\n```\n\nYou can set the following parameters via `with`:\n\n| Parameter | Description |\n|-----------|-------------------------------------------------------------------------------------------------------------------------------|\n| `name` | Model weights, default is `ViT-B/32`. Support all OpenAI released pretrained models. |\n| `num_worker_preprocess` | The number of CPU workers for image & text prerpocessing, default 4. | \n| `minibatch_size` | The size of a minibatch for CPU preprocessing and GPU encoding, default 16. Reduce the size of it if you encounter OOM on GPU. |\n| `device` | `cuda` or `cpu`. Default is `None` means auto-detect. |\n\n### Encoding\n\nEncoding here means getting the fixed-length vector representation of a sentence or image.\n\n```python\nfrom jina import Flow\nfrom docarray import Document, DocumentArray\n\nda = DocumentArray(\n [\n Document(text='she smiled, with pain'),\n Document(uri='apple.png'),\n Document(uri='apple.png').load_uri_to_image_tensor(),\n Document(blob=open('apple.png', 'rb').read()),\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n Document(\n uri='data:image/gif;base64,R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o/XBs/fNwfjZ0frl3/zy7////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7'\n ),\n ]\n)\n\nf = Flow().add(\n uses='jinahub+docker://CLIPOnnxEncoder',\n)\nwith f:\n f.post(on='/', inputs=da)\n da.summary()\n```\n\nFrom the output, you will see all the text and image docs have `embedding` attached.\n\n```text\nโ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Documents Summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ\nโ”‚ โ”‚\nโ”‚ Length 6 โ”‚\nโ”‚ Homogenous Documents False โ”‚\nโ”‚ 4 Documents have attributes ('id', 'mime_type', 'uri', 'embedding') โ”‚\nโ”‚ 1 Document has attributes ('id', 'mime_type', 'text', 'embedding') โ”‚\nโ”‚ 1 Document has attributes ('id', 'embedding') โ”‚\nโ”‚ โ”‚\nโ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ\nโ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Attributes Summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ\nโ”‚ โ”‚\nโ”‚ Attribute Data type #Unique values Has empty value โ”‚\nโ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚\nโ”‚ embedding ('ndarray',) 6 False โ”‚\nโ”‚ id ('str',) 6 False โ”‚\nโ”‚ mime_type ('str',) 5 False โ”‚\nโ”‚ text ('str',) 2 False โ”‚\nโ”‚ uri ('str',) 4 False โ”‚\nโ”‚ โ”‚\nโ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ\n```\n\n๐Ÿ‘‰ Access the embedding playground in **CLIP-as-service** [doc](https://clip-as-service.jina.ai/playground/embedding), type sentence or image URL and see **live embedding**!\n\n### Ranking\n\nOne can also rank cross-modal matches via `/rank` endpoint. \nFirst construct a *cross-modal* Document where the root contains an image and `.matches` contain sentences to rerank. \n\n```python\nfrom docarray import Document\n\nd = Document(\n uri='rerank.png',\n matches=[\n Document(text=f'a photo of a {p}')\n for p in (\n 'control room',\n 'lecture room',\n 'conference room',\n 'podium indoor',\n 'television studio',\n )\n ],\n)\n```\n\nThen send the request via `/rank` endpoint:\n\n```python\nf = Flow().add(\n uses='jinahub+docker://CLIPOnnxEncoder',\n)\nwith f:\n r = f.post(on='/rank', inputs=[d])\n print(r['@m', ['text', 'scores__clip_score__value']])\n```\n\nFinally, in the return you can observe the matches are re-ranked according to `.scores['clip_score']`:\n\n```bash\n[['a photo of a television studio', 'a photo of a conference room', 'a photo of a lecture room', 'a photo of a control room', 'a photo of a podium indoor'], \n[0.9920725226402283, 0.006038925610482693, 0.0009973491542041302, 0.00078492151806131, 0.00010626466246321797]]\n```\n\nOne can also construct `text-to-image` rerank as below:\n\n```python\nfrom docarray import Document\n\nd = Document(\n text='a photo of conference room',\n matches=[\n Document(uri='https://picsum.photos/300'),\n Document(uri='https://picsum.photos/id/331/50'),\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n ],\n)\n```\n\n๐Ÿ‘‰ Access the ranking playground in **CLIP-as-service** [doc](https://clip-as-service.jina.ai/playground/reasoning/). Just input the reasoning texts as prompts, the server will rank the prompts and return sorted prompts with scores." }, { "path": ".github/README-exec/torch.readme.md", "content": "# CLIPTorchEncoder\n\n**CLIPTorchEncoder** is the executor implemented in [CLIP-as-service](https://github.com/jina-ai/clip-as-service). \nThe various `CLIP` models implemented in the [OpenAI](https://github.com/openai/CLIP), [OpenCLIP](https://github.com/mlfoundations/open_clip), and [MultilingualCLIP](https://github.com/FreddeFrallan/Multilingual-CLIP) are supported with PyTorch runtime.\nThe introduction of the CLIP model [can be found here](https://openai.com/blog/clip/).\n\n- ๐Ÿ”€ **Automatic**: Auto-detect image and text documents depending on their content.\n- โšก **Efficiency**: Faster CLIP model inference on CPU and GPU via leveraging the best practices. \n- ๐Ÿ“ˆ **Observability**: Monitoring the serving via Prometheus and Grafana (see [Usage Guide](https://docs.jina.ai/how-to/monitoring/#deploying-locally)).\n\nWith advances of ONNX runtime, you can use `CLIPOnnxEncoder` (see [link](https://cloud.jina.ai/executor/2a7auwg2)) instead to achieve **3x** model inference speed up. \n\n## Model support\n\n`ViT-B-32::openai` is used as the default model. To use specific pretrained models provided by `open_clip`, please use `::` to separate model name and pretrained weight name, e.g. `ViT-B-32::laion2b_e16`. Please also note that **different models give different sizes of output dimensions**.\n\n| Model | PyTorch | Output dimension | \n|---------------------------------------|---------|------------------|\n| RN50 | โœ… | 1024 | \n| RN101 | โœ… | 512 | \n| RN50x4 | โœ… | 640 |\n| RN50x16 | โœ… | 768 |\n| RN50x64 | โœ… | 1024 |\n| ViT-B-32 | โœ… | 512 |\n| ViT-B-16 | โœ… | 512 |\n| ViT-B-16-plus-240 | โœ… | 640 |\n| ViT-L-14 | โœ… | 768 |\n| ViT-L-14-336 | โœ… | 768 |\n| ViT-H-14 | โœ… | 1024 |\n| ViT-g-14 | โœ… | 1024 |\n| M-CLIP/XLM_Roberta-Large-Vit-B-32 | โœ… | 512 |\n| M-CLIP/XLM-Roberta-Large-Vit-L-14 | โœ… | 768 |\n| M-CLIP/XLM-Roberta-Large-Vit-B-16Plus | โœ… | 640 |\n| M-CLIP/LABSE-Vit-L-14 | โœ… | 768 |\n\nโœ… = First class support\n\n\nFull list of open_clip models and weights can be found [here](https://github.com/mlfoundations/open_clip#pretrained-model-interface).\n\n```{note}\nFor model definition with `-quickgelu` postfix, please use non `-quickgelu` model name.\n```\n\n\n## Usage\n\n### Use in Jina Flow \n\n- **via Docker image (recommended)**\n\n```python\nfrom jina import Flow\nfrom docarray import Document\nimport numpy as np\n\nf = Flow().add(\n uses='jinahub+docker://CLIPTorchEncoder',\n)\n```\n\n- **via source code**\n\n```python\nfrom jina import Flow\nfrom docarray import Document\nimport numpy as np\n\nf = Flow().add(\n uses='jinahub://CLIPTorchEncoder',\n)\n```\n\nYou can set the following parameters via `with`:\n\n| Parameter | Description |\n|-------------------------|--------------------------------------------------------------------------------------------------------------------------------|\n| `name` | Model weights, default is `ViT-B/32`. Support all OpenAI released pretrained models. |\n| `num_worker_preprocess` | The number of CPU workers for image & text prerpocessing, default 4. | \n| `minibatch_size` | The size of a minibatch for CPU preprocessing and GPU encoding, default 32. Reduce the size of it if you encounter OOM on GPU. |\n| `device` | `cuda` or `cpu`. Default is `None` means auto-detect. |\n| `jit` | If to enable Torchscript JIT, default is `False`. |\n\n### Encoding\n\nEncoding here means getting the fixed-length vector representation of a sentence or image.\n\n```python\nfrom jina import Flow\nfrom docarray import Document, DocumentArray\n\nda = DocumentArray(\n [\n Document(text='she smiled, with pain'),\n Document(uri='apple.png'),\n Document(uri='apple.png').load_uri_to_image_tensor(),\n Document(blob=open('apple.png', 'rb').read()),\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n Document(\n uri='data:image/gif;base64,R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o/XBs/fNwfjZ0frl3/zy7////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7'\n ),\n ]\n)\n\nf = Flow().add(\n uses='jinahub+docker://CLIPTorchEncoder',\n)\nwith f:\n f.post(on='/', inputs=da)\n da.summary()\n```\n\nFrom the output, you will see all the text and image docs have `embedding` attached.\n\n```text\nโ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Documents Summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ\nโ”‚ โ”‚\nโ”‚ Length 6 โ”‚\nโ”‚ Homogenous Documents False โ”‚\nโ”‚ 4 Documents have attributes ('id', 'mime_type', 'uri', 'embedding') โ”‚\nโ”‚ 1 Document has attributes ('id', 'mime_type', 'text', 'embedding') โ”‚\nโ”‚ 1 Document has attributes ('id', 'embedding') โ”‚\nโ”‚ โ”‚\nโ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ\nโ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Attributes Summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ\nโ”‚ โ”‚\nโ”‚ Attribute Data type #Unique values Has empty value โ”‚\nโ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚\nโ”‚ embedding ('ndarray',) 6 False โ”‚\nโ”‚ id ('str',) 6 False โ”‚\nโ”‚ mime_type ('str',) 5 False โ”‚\nโ”‚ text ('str',) 2 False โ”‚\nโ”‚ uri ('str',) 4 False โ”‚\nโ”‚ โ”‚\nโ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ\n```\n\n๐Ÿ‘‰ Access the embedding playground in **CLIP-as-service** [doc](https://clip-as-service.jina.ai/playground/embedding), type sentence or image URL and see **live embedding**!\n\n### Ranking\n\nOne can also rank cross-modal matches via `/rank` endpoint. \nFirst construct a *cross-modal* Document where the root contains an image and `.matches` contain sentences to rerank. \n\n```python\nfrom docarray import Document\n\nd = Document(\n uri='rerank.png',\n matches=[\n Document(text=f'a photo of a {p}')\n for p in (\n 'control room',\n 'lecture room',\n 'conference room',\n 'podium indoor',\n 'television studio',\n )\n ],\n)\n```\n\nThen send the request via `/rank` endpoint:\n\n```python\nf = Flow().add(\n uses='jinahub+docker://CLIPTorchEncoder',\n)\nwith f:\n r = f.post(on='/rank', inputs=[d])\n print(r['@m', ['text', 'scores__clip_score__value']])\n```\n\nFinally, you can observe the matches are re-ranked based on `.scores['clip_score']`:\n\n```bash\n[['a photo of a television studio', 'a photo of a conference room', 'a photo of a lecture room', 'a photo of a control room', 'a photo of a podium indoor'], \n[0.9920725226402283, 0.006038925610482693, 0.0009973491542041302, 0.00078492151806131, 0.00010626466246321797]]\n```\n\nOne can also construct `text-to-image` rerank as below:\n\n```python\nfrom docarray import Document\n\nd = Document(\n text='a photo of conference room',\n matches=[\n Document(uri='https://picsum.photos/300'),\n Document(uri='https://picsum.photos/id/331/50'),\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n ],\n)\n```\n\n๐Ÿ‘‰ Access the ranking playground in **CLIP-as-service** [doc](https://clip-as-service.jina.ai/playground/reasoning/). Just input the reasoning texts as prompts, the server will rank the prompts and return sorted prompts with scores." }, { "path": ".github/codecov.yml", "content": "codecov:\n # https://docs.codecov.io/docs/comparing-commits\n allow_coverage_offsets: true\ncoverage:\n status:\n project:\n default:\n informational: true\n target: auto # auto compares coverage to the previous base commit\n comment:\n layout: \"reach, diff, flags, files\"\n behavior: default\n require_changes: false # if true: only post the comment if coverage changes\n branches: # branch names that can post comment\n - \"main\"\n" }, { "path": ".github/labeler.yml", "content": "# Add 'label1' to any changes within 'example' folder or any subfolders\narea/docs:\n - docs/**/*\n - ./*.md\n\narea/testing:\n - tests/**/*\n\narea/setup:\n - setup.py\n - requirements.txt\n - MANIFEST.in\n\narea/housekeeping:\n - .github/**/*\n - ./.gitignore\n - ./*.yaml\n - ./*.yml\n\narea/cicd:\n - .github/workflows/**/*\n\narea/docker:\n - Dockerfiles/**/*\n - ./.dockerignore\n\narea/script:\n - script/**/*\n\ncomponent/client:\n - client/**/*\n\ncomponent/server:\n - server/**/*\n\n" }, { "path": ".github/release-template.ejs", "content": "<% var groupCommits = [\n{\n name: 'breaking',\n show: true,\n list: []\n}, {\n name: 'feat',\n show: true,\n list: []\n}, {\n name: 'perf',\n show: true,\n list: []\n}, {\n name: 'fix',\n show: true,\n list: []\n}, {\n name: 'refactor',\n show: true,\n list: []\n}, {\n name: 'docs',\n show: true,\n list: []\n}, {\n name: 'test',\n show: true,\n list: []\n}, {\n name: 'other',\n show: true,\n list: []\n}\n]\n\nvar all_titles = {};\nvar all_commiters = {};\nvar commitHref = \"https://github.com/jina-ai/clip-as-service/commit/\"\n\ncommits.forEach(function (commit) {\n\n var result = (commit.title).match(/^(\\w*)(\\((.*)\\))?\\: (.*)$/);\n\n var type = result && result[1];\n var scope = result && result[3];\n var title = result && result[4];\n var committer = commit.authorName\n\n if (!(committer in all_commiters)) {\n all_commiters[committer] = 1\n }\n\n if (!(title in all_titles)) {\n all_titles[title] = 1\n if( title != null && (title.indexOf('๐Ÿ’ฅ')>-1 || title.indexOf(':boom:')>-1) ){\n groupCommits.find(item => item.name === 'breaking').list.push({\n type: type,\n scope: scope,\n title: title,\n commit: commit\n })\n } else if(type == 'fix' || type == 'fixed'){\n groupCommits.find(item => item.name === 'fix').list.push({\n type: type,\n scope: scope,\n title: title,\n commit: commit\n })\n } else if(type == 'perf' || type == 'performance'){\n groupCommits.find(item => item.name === 'perf').list.push({\n type: type,\n scope: scope,\n title: title,\n commit: commit\n })\n } else if(type == 'feat' || type == 'feature'){\n groupCommits.find(item => item.name === 'feat').list.push({\n type: type,\n scope: scope,\n title: title,\n commit: commit\n })\n } else if(type == 'refactor'){\n groupCommits.find(item => item.name === 'refactor').list.push({\n type: type,\n scope: scope,\n title: title,\n commit: commit\n })\n } else if(type == 'docs' || type == 'doc'){\n groupCommits.find(item => item.name === 'docs').list.push({\n type: type,\n scope: scope,\n title: title,\n commit: commit\n })\n } else if(type == 'test' || type == 'tests' || type == 'ci'){\n groupCommits.find(item => item.name === 'test').list.push({\n type: type,\n scope: scope,\n title: title,\n commit: commit\n })\n } else {\n groupCommits.find(item => item.name === 'other').list.push({\n type: type,\n scope: scope,\n title: title,\n commit: commit\n })\n }\n }\n\n\n});\n\n\nvar listCommits = function(list, key){\n\nlist.forEach(function (ct) {\n\n var type = ct.type;\n var scope = ct.scope;\n var title = '';\n var commit = ct.commit;\n\n if(type){\n if(key != 'other'){\n title = (scope? '__'+scope+'__: ':'') + ct.title;\n }else{\n title = '__' + type + (scope? '('+scope+')':'') + '__ : ' + ct.title;\n }\n }else{\n title = commit.title;\n }\n%> - <% if(typeof commitHref === 'undefined' || commitHref === '') { %>[```<%=commit.sha1.slice(0,8)%>```]<% } else { %>[[```<%=commit.sha1.slice(0,8)%>```](<%=commitHref%><%=commit.sha1%>)]<%}%> __-__ <%=title%> (*<%= commit.authorName %>*)\n<% })} %>\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n<% Object.keys(all_commiters).forEach(function (key) { %> <%= key %>, <% }) %> ๐Ÿ™‡\n\n<%\n for(var i of groupCommits){\n if(i.list.length == 0) continue;\n\nif (i.name === 'breaking' && i.show) { %>\n### ๐Ÿ’ฅ Breaking changes\n\n<%\t} else if (i.name === 'fix' && i.show) { %>\n### ๐Ÿž Bug fixes\n\n<%\t} else if( i.name === 'feat' && i.show) { %>\n### ๐Ÿ†• New Features\n\n<%\t} else if(i.name === 'perf' && i.show) { %>\n### โšก Performance Improvements\n\n<%\t} else if(i.name === 'refactor' && i.show) { %>\n### ๐Ÿงผ Code Refactoring\n\n<%\t} else if(i.name === 'docs' && i.show) { %>\n### ๐Ÿ“— Documentation\n\n<%\t} else if(i.name === 'test' && i.show) { %>\n### ๐Ÿ Unit Test and CICD\n\n<%\t} else if (i.name === 'other' && i.show) { %>\n### ๐Ÿน Other Improvements\n\n<%\t}\n i.show && listCommits(i.list, i);\n} %>" }, { "path": ".github/workflows/cd.yml", "content": "name: CD\n\non:\n push:\n branches:\n - main\n\n\njobs:\n prep-testbed:\n if: |\n !startsWith(github.event.head_commit.message, 'chore') &&\n !startsWith(github.event.head_commit.message, 'build: hotfix') &&\n !endsWith(github.event.head_commit.message, 'reformatted by jina-dev-bot')\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v2\n - id: set-matrix\n run: |\n sudo apt-get install jq\n echo \"::set-output name=matrix::$(bash scripts/get-all-test-paths.sh)\"\n outputs:\n matrix: ${{ steps.set-matrix.outputs.matrix }}\n\n core-test:\n needs: prep-testbed\n runs-on: ubuntu-latest\n strategy:\n fail-fast: false\n matrix:\n python-version: [3.7]\n test-path: ${{fromJson(needs.prep-testbed.outputs.matrix)}}\n steps:\n - uses: actions/checkout@v2\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v2\n with:\n python-version: ${{ matrix.python-version }}\n - name: Prepare enviroment\n run: |\n python -m pip install --upgrade pip\n python -m pip install wheel\n pip install --no-cache-dir \"client/[test]\"\n pip install --no-cache-dir \"server/[onnx]\"\n pip install --no-cache-dir \"server/[transformers]\"\n pip install --no-cache-dir \"server/[search]\"\n pip install --no-cache-dir \"server/[cn_clip]\"\n - name: Test\n id: test\n run: |\n pytest --suppress-no-test-exit-code --cov=clip_client --cov=clip_server --cov-report=xml \\\n -v -s -m \"not gpu\" ${{ matrix.test-path }}\n echo \"::set-output name=codecov_flag::cas\"\n timeout-minutes: 30\n - name: Check codecov file\n id: check_files\n uses: andstor/file-existence-action@v1\n with:\n files: \"coverage.xml\"\n - name: Upload coverage from test to Codecov\n uses: codecov/codecov-action@v2\n if: steps.check_files.outputs.files_exists == 'true' && ${{ matrix.python-version }} == '3.7'\n with:\n file: coverage.xml\n flags: ${{ steps.test.outputs.codecov_flag }}\n fail_ci_if_error: false\n token: ${{ secrets.CODECOV_TOKEN }} # not required for public repos\n\n gpu-test:\n needs: prep-testbed\n runs-on: [self-hosted, x64, gpu, linux]\n strategy:\n fail-fast: false\n matrix:\n python-version: [ 3.7 ]\n steps:\n - uses: actions/checkout@v2\n with:\n # For coverage builds fetch the whole history\n fetch-depth: 0\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v2\n with:\n python-version: ${{ matrix.python-version }}\n - name: Prepare enviroment\n run: |\n python -m pip install --upgrade pip\n python -m pip install wheel pytest pytest-cov nvidia-pyindex\n pip install -e \"client/[test]\"\n pip install -e \"server/[tensorrt]\"\n - name: Test\n id: test\n run: |\n pytest --suppress-no-test-exit-code --cov=clip_client --cov=clip_server --cov-report=xml \\\n -v -s -m \"gpu\" ./tests/test_tensorrt.py\n echo \"::set-output name=codecov_flag::cas\"\n timeout-minutes: 30\n env:\n # fix re-initialized torch runtime error on cuda device\n JINA_MP_START_METHOD: spawn\n - name: Check codecov file\n id: check_files\n uses: andstor/file-existence-action@v1\n with:\n files: \"coverage.xml\"\n - name: Upload coverage from test to Codecov\n uses: codecov/codecov-action@v3\n if: steps.check_files.outputs.files_exists == 'true' && ${{ matrix.python-version }} == '3.7'\n with:\n file: coverage.xml\n name: gpu-related-codecov\n flags: ${{ steps.test.outputs.codecov_flag }}\n fail_ci_if_error: false\n token: ${{ secrets.CODECOV_TOKEN }} # not required for public repos\n\n prerelease:\n needs: [core-test, gpu-test]\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v2\n with:\n fetch-depth: 100\n - name: Pre-release (.devN)\n run: |\n git fetch --depth=1 origin +refs/tags/*:refs/tags/*\n pip install twine wheel\n ./scripts/release.sh\n env:\n TWINE_USERNAME: ${{ secrets.TWINE_USERNAME }}\n TWINE_PASSWORD: ${{ secrets.TWINE_PASSWORD }}\n - name: Pre-release docker (.devN)\n uses: benc-uk/workflow-dispatch@v1\n with:\n workflow: Manual Docker Build\n inputs: '{ \"release_token\": \"${{ env.release_token }}\", \"triggered_by\": \"CD\"}'\n token: ${{ secrets.JINA_DEV_BOT }}\n env:\n release_token: ${{ secrets.CAS_RELEASE_TOKEN }}\n - uses: benc-uk/workflow-dispatch@v1\n with:\n workflow: Manual CAS Docker Build\n inputs: '{ \"release_token\": \"${{ env.release_token }}\", \"triggered_by\": \"CD\"}'\n token: ${{ secrets.JINA_DEV_BOT }}\n env:\n release_token: ${{ secrets.CAS_RELEASE_TOKEN }}\n - name: Pre-release hub (.devN)\n uses: benc-uk/workflow-dispatch@v1\n with:\n workflow: Manual Hub Push\n inputs: '{ \"release_token\": \"${{ env.release_token }}\", \"triggered_by\": \"CD\"}'\n token: ${{ secrets.JINA_DEV_BOT }}\n env:\n release_token: ${{ secrets.CAS_RELEASE_TOKEN }}\n" }, { "path": ".github/workflows/ci.yml", "content": "name: CI\n\non:\n pull_request:\n\n\njobs:\n commit-lint:\n runs-on: ubuntu-latest\n steps:\n - name: find the prev warning if exist\n uses: peter-evans/find-comment@v1\n id: fc\n with:\n issue-number: ${{ github.event.pull_request.number }}\n comment-author: \"github-actions[bot]\"\n body-includes: \"bad commit message\"\n - name: Delete comment if exist\n if: ${{ steps.fc.outputs.comment-id != 0 }}\n uses: actions/github-script@v3\n with:\n github-token: ${{ secrets.GITHUB_TOKEN }}\n script: |\n github.issues.deleteComment({\n owner: context.repo.owner,\n repo: context.repo.repo,\n comment_id: ${{ steps.fc.outputs.comment-id }},\n })\n - uses: actions/checkout@v2.5.0\n with:\n fetch-depth: 0\n - run: 'echo \"module.exports = {extends: [''@commitlint/config-conventional'']}\" > commitlint.config.js'\n - uses: wagoid/commitlint-github-action@v4\n env:\n GITHUB_TOKEN: \"${{ secrets.GITHUB_TOKEN }}\"\n - name: if lint failed\n if: ${{ failure() }}\n uses: peter-evans/create-or-update-comment@v1\n with:\n issue-number: ${{ github.event.pull_request.number }}\n body: |\n Thanks for your contribution :heart:\n :broken_heart: Unfortunately, this PR has one or more **bad commit messages**, it can not be merged. To fix this problem, please refer to:\n - [Commit Message Guideline for the First Time Contributor](https://github.com/jina-ai/jina/issues/553)\n - [Contributing Guideline](https://github.com/jina-ai/jina/blob/master/CONTRIBUTING.md)\n This message will be deleted automatically when the commit messages get fixed.\n reaction-type: \"eyes\"\n\n lint-flake-8:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v2\n - name: Set up Python 3.7\n uses: actions/setup-python@v2\n with:\n python-version: 3.7\n - name: Lint with flake8\n run: |\n pip install flake8\n # stop the build if there are Python syntax errors or undefined names\n flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics --exclude .git,__pycache__,docs/source/conf.py,old,build,dist,tests/,jina/resources/\n # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide\n flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics --exclude .git,__pycache__,docs/source/conf.py,old,build,dist,tests/,jina/resources/\n\n check-black:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v2\n with:\n fetch-depth: 0\n - name: Set up Python 3.7\n uses: actions/setup-python@v2\n with:\n python-version: 3.7\n - id: file_changes\n uses: Ana06/get-changed-files@v1.2\n - name: check black\n run: ./scripts/black.sh\n env:\n CHANGED_FILES: ${{ steps.file_changes.outputs.added_modified }}\n\n prep-testbed:\n runs-on: ubuntu-latest\n needs: [lint-flake-8, check-black]\n steps:\n - uses: actions/checkout@v2\n - id: set-matrix\n run: |\n sudo apt-get install jq\n echo \"::set-output name=matrix::$(bash scripts/get-all-test-paths.sh)\"\n outputs:\n matrix: ${{ steps.set-matrix.outputs.matrix }}\n\n core-test:\n needs: prep-testbed\n runs-on: ubuntu-latest\n strategy:\n fail-fast: false\n matrix:\n python-version: [3.7]\n test-path: ${{fromJson(needs.prep-testbed.outputs.matrix)}}\n steps:\n - uses: actions/checkout@v2\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v2\n with:\n python-version: ${{ matrix.python-version }}\n - name: Prepare enviroment\n run: |\n python -m pip install --upgrade pip\n python -m pip install wheel pytest pytest-cov\n pip install --no-cache-dir \"client/[test]\"\n pip install --no-cache-dir \"server/[onnx]\"\n pip install --no-cache-dir \"server/[transformers]\"\n pip install --no-cache-dir \"server/[search]\"\n pip install --no-cache-dir \"server/[cn_clip]\"\n - name: Test\n id: test\n run: |\n pytest --suppress-no-test-exit-code --cov=clip_client --cov=clip_server --cov-report=xml \\\n -v -s ${{ matrix.test-path }}\n echo \"::set-output name=codecov_flag::cas\"\n timeout-minutes: 30\n - name: Check codecov file\n id: check_files\n uses: andstor/file-existence-action@v1\n with:\n files: \"coverage.xml\"\n - name: Upload coverage from test to Codecov\n uses: codecov/codecov-action@v3\n if: steps.check_files.outputs.files_exists == 'true' && ${{ matrix.python-version }} == '3.7'\n with:\n file: coverage.xml\n name: ${{ matrix.test-path }}-codecov\n flags: ${{ steps.test.outputs.codecov_flag }}\n fail_ci_if_error: false\n token: ${{ secrets.CODECOV_TOKEN }} # not required for public repos\n\n trt-gpu-test:\n needs: prep-testbed\n runs-on: [self-hosted, x64, gpu, linux]\n strategy:\n fail-fast: false\n matrix:\n python-version: [ 3.7 ]\n steps:\n - uses: actions/checkout@v2\n with:\n # For coverage builds fetch the whole history\n fetch-depth: 0\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v2\n with:\n python-version: ${{ matrix.python-version }}\n - name: Prepare enviroment\n run: |\n python -m pip install pip==23.0.1\n python -m pip install wheel pytest pytest-cov nvidia-pyindex\n pip install -e \"client/[test]\"\n pip install -e \"server/[tensorrt]\"\n pip install -e \"server/[onnx]\"\n pip install -e \"server/[transformers]\"\n {\n pip install -e \"server/[flash-attn]\"\n } || {\n echo \"flash attention was not installed.\"\n }\n pip install --no-cache-dir \"server/[cn_clip]\"\n - name: Test\n id: test\n run: |\n pytest --suppress-no-test-exit-code --cov=clip_client --cov=clip_server --cov-report=xml \\\n -v -s -m \"gpu\" ./tests/test_tensorrt.py\n pytest --suppress-no-test-exit-code --cov=clip_client --cov=clip_server --cov-report=xml \\\n -v -s -m \"gpu\" ./tests/test_simple.py\n echo \"::set-output name=codecov_flag::cas\"\n timeout-minutes: 30\n env:\n # fix re-initialized torch runtime error on cuda device\n JINA_MP_START_METHOD: spawn\n - name: Check codecov file\n id: check_files\n uses: andstor/file-existence-action@v1\n with:\n files: \"coverage.xml\"\n - name: Upload coverage from test to Codecov\n uses: codecov/codecov-action@v3\n if: steps.check_files.outputs.files_exists == 'true' && ${{ matrix.python-version }} == '3.7'\n with:\n file: coverage.xml\n name: gpu-related-codecov\n flags: ${{ steps.test.outputs.codecov_flag }}\n fail_ci_if_error: false\n token: ${{ secrets.CODECOV_TOKEN }} # not required for public repos\n\n gpu-model-test:\n needs: prep-testbed\n runs-on: [ self-hosted, x64, gpu, linux ]\n strategy:\n fail-fast: false\n matrix:\n python-version: [ 3.7 ]\n steps:\n - uses: actions/checkout@v2\n with:\n # For coverage builds fetch the whole history\n fetch-depth: 0\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v2\n with:\n python-version: ${{ matrix.python-version }}\n - name: Prepare enviroment\n run: |\n python -m pip install pip==23.0.1\n python -m pip install wheel pytest pytest-cov nvidia-pyindex\n pip install -e \"client/[test]\"\n pip install -e \"server/[onnx]\"\n pip install -e \"server/[transformers]\"\n {\n pip install -e \"server/[flash-attn]\"\n } || {\n echo \"flash attention was not installed.\"\n }\n pip install --no-cache-dir \"server/[cn_clip]\"\n - name: Test\n id: test\n run: |\n pytest --suppress-no-test-exit-code --cov=clip_client --cov=clip_server --cov-report=xml \\\n -v -s -m \"gpu\" ./tests/test_model.py\n echo \"::set-output name=codecov_flag::cas\"\n timeout-minutes: 30\n env:\n # fix re-initialized torch runtime error on cuda device\n JINA_MP_START_METHOD: spawn\n - name: Check codecov file\n id: check_files\n uses: andstor/file-existence-action@v1\n with:\n files: \"coverage.xml\"\n - name: Upload coverage from test to Codecov\n uses: codecov/codecov-action@v3\n if: steps.check_files.outputs.files_exists == 'true' && ${{ matrix.python-version }} == '3.7'\n with:\n file: coverage.xml\n name: gpu-related-codecov\n flags: ${{ steps.test.outputs.codecov_flag }}\n fail_ci_if_error: false\n token: ${{ secrets.CODECOV_TOKEN }} # not required for public repos\n\n # just for blocking the merge until all parallel core-test are successful\n success-all-test:\n needs: [commit-lint, core-test, trt-gpu-test, gpu-model-test]\n if: always()\n runs-on: ubuntu-latest\n steps:\n - uses: technote-space/workflow-conclusion-action@v2\n - name: Check Failure\n if: env.WORKFLOW_CONCLUSION == 'failure'\n run: exit 1\n - name: Success\n if: ${{ success() }}\n run: echo \"All Done\"\n" }, { "path": ".github/workflows/force-docker-build-cas.yml", "content": "name: Manual CAS Docker Build\n\non:\n workflow_dispatch:\n inputs:\n release_token:\n description: 'Your release token'\n required: true\n triggered_by:\n description: 'CD | TAG | MANUAL'\n required: false\n default: MANUAL\n\njobs:\n token-check:\n runs-on: ubuntu-latest\n steps:\n - run: echo \"success!\"\n if: \"${{ github.event.inputs.release_token }} == ${{ env.release_token }}\"\n env:\n release_token: ${{ secrets.CAS_RELEASE_TOKEN }}\n\n regular-release:\n needs: token-check\n runs-on: ubuntu-latest\n strategy:\n fail-fast: false\n matrix:\n pip_tag: [ \"\", \"onnx\", \"tensorrt\"] # default: \"\" = core\n steps:\n - uses: actions/checkout@v2\n - name: Set envs and versions\n run: |\n VCS_REF=${{ github.ref }}\n echo \"VCS_REF=$VCS_REF\" >> $GITHUB_ENV\n echo \"Will build $VCS_REF\"\n echo \"BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')\" >> $GITHUB_ENV\n\n if [[ \"${{ matrix.pip_tag }}\" == \"perf\" ]]; then\n echo \"JINA_PIP_INSTALL_PERF=1\" >> $GITHUB_ENV\n fi\n\n if [[ \"${{ matrix.pip_tag }}\" == \"\" ]]; then\n echo \"JINA_PIP_INSTALL_CORE=1\" >> $GITHUB_ENV\n fi\n\n JINA_VERSION=$(sed -n '/^__version__/p' ./server/clip_server/__init__.py | cut -d \\' -f2)\n V_JINA_VERSION=v${JINA_VERSION}\n JINA_MINOR_VERSION=${JINA_VERSION%.*}\n JINA_MAJOR_VERSION=${JINA_MINOR_VERSION%.*}\n\n PY_TAG=${{matrix.py_version}}\n if [ -n \"${PY_TAG}\" ]; then\n PY_TAG=-py${PY_TAG//./}\n fi\n\n PIP_TAG=${{ matrix.pip_tag }}\n if [ -n \"${PIP_TAG}\" ]; then\n PIP_TAG=-${PIP_TAG}\n fi\n\n if [[ \"${{ github.event.inputs.triggered_by }}\" == \"CD\" ]]; then\n\n if [[ \"${{ matrix.py_version }}\" == \"$DEFAULT_PY_VERSION\" ]]; then\n echo \"TAG_ALIAS=\\\n jinaai/clip-server:master${PY_TAG}${PIP_TAG}, \\\n jinaai/clip-server:master${PIP_TAG}\" \\\n >> $GITHUB_ENV\n else\n # on every CD\n echo \"TAG_ALIAS=\\\n jinaai/clip-server:master${PY_TAG}${PIP_TAG}\" \\\n >> $GITHUB_ENV\n fi\n\n elif [[ \"${{ github.event.inputs.triggered_by }}\" == \"TAG\" ]]; then\n # on every tag release\n\n if [[ \"${{ matrix.py_version }}\" == \"$DEFAULT_PY_VERSION\" ]]; then\n echo \"TAG_ALIAS=\\\n jinaai/clip-server:latest${PY_TAG}${PIP_TAG}, \\\n jinaai/clip-server:${JINA_VERSION}${PY_TAG}${PIP_TAG}, \\\n jinaai/clip-server:${JINA_MINOR_VERSION}${PY_TAG}${PIP_TAG}, \\\n jinaai/clip-server:${JINA_MAJOR_VERSION}${PY_TAG}${PIP_TAG}, \\\n jinaai/clip-server:latest${PIP_TAG}, \\\n jinaai/clip-server:${JINA_VERSION}${PIP_TAG}, \\\n jinaai/clip-server:${JINA_MINOR_VERSION}${PIP_TAG}, \\\n jinaai/clip-server:${JINA_MAJOR_VERSION}${PIP_TAG} \\\n \" >> $GITHUB_ENV\n else\n echo \"TAG_ALIAS=\\\n jinaai/clip-server:latest${PY_TAG}${PIP_TAG}, \\\n jinaai/clip-server:${JINA_VERSION}${PY_TAG}${PIP_TAG}, \\\n jinaai/clip-server:${JINA_MINOR_VERSION}${PY_TAG}${PIP_TAG}, \\\n jinaai/clip-server:${JINA_MAJOR_VERSION}${PY_TAG}${PIP_TAG} \\\n \" >> $GITHUB_ENV\n fi\n elif [[ \"${{ github.event.inputs.triggered_by }}\" == \"MANUAL\" ]]; then\n # on every manual release\n if [[ \"${{ matrix.py_version }}\" == \"$DEFAULT_PY_VERSION\" ]]; then\n echo \"TAG_ALIAS=\\\n jinaai/clip-server:${JINA_VERSION}${PIP_TAG}, \\\n jinaai/clip-server:${JINA_VERSION}${PY_TAG}${PIP_TAG} \\\n \" >> $GITHUB_ENV\n else\n echo \"TAG_ALIAS=\\\n jinaai/clip-server:${JINA_VERSION}${PY_TAG}${PIP_TAG} \\\n \" >> $GITHUB_ENV\n fi\n else\n echo \"Bad triggered_by: ${{ github.event.inputs.triggered_by }}!\"\n exit 1\n fi\n\n echo \"JINA_VERSION=${JINA_VERSION}\" >> $GITHUB_ENV\n\n - name: Set up Docker Buildx\n id: buildx\n uses: docker/setup-buildx-action@v1\n with:\n install: true\n - name: Login to DockerHub\n uses: docker/login-action@v1\n with:\n username: ${{ secrets.DOCKERHUB_DEVBOT_USER }}\n password: ${{ secrets.DOCKERHUB_DEVBOT_TOKEN }}\n - run: |\n # https://github.com/docker/buildx/issues/464#issuecomment-741507760\n # https://github.com/kubernetes-sigs/azuredisk-csi-driver/pull/808/files\n docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64\n docker run --rm --privileged tonistiigi/binfmt --install all\n - name: Build and push\n uses: docker/build-push-action@v2\n with:\n context: .\n file: Dockerfiles/server.Dockerfile\n platforms: linux/amd64\n push: true\n tags: ${{env.TAG_ALIAS}}\n build-args: |\n BUILD_DATE=${{env.BUILD_DATE}}\n JINA_VERSION=${{env.JINA_VERSION}}\n VCS_REF=${{env.VCS_REF}}\n PIP_INSTALL_CORE=${{env.JINA_PIP_INSTALL_CORE}}\n PIP_INSTALL_PERF=${{env.JINA_PIP_INSTALL_PERF}}\n PIP_TAG=${{matrix.pip_tag}}\n" }, { "path": ".github/workflows/force-docker-build.yml", "content": "name: Manual Docker Build\n\non:\n workflow_dispatch:\n inputs:\n release_token:\n description: 'Your release token'\n required: true\n triggered_by:\n description: 'CD | TAG | MANUAL'\n required: false\n default: MANUAL\n\n\njobs:\n token-check:\n runs-on: ubuntu-latest\n steps:\n - run: echo \"success!\"\n if: \"${{ github.event.inputs.release_token }} == ${{ env.release_token }}\"\n env:\n release_token: ${{ secrets.CAS_RELEASE_TOKEN }}\n\n docker-release:\n needs: token-check\n runs-on: ubuntu-latest\n strategy:\n fail-fast: false\n matrix:\n pip_tag: [\"\", \"onnx\", \"tensorrt\"] # default: \"\" = torch\n engine_tag: [\"\", \"cuda\"] # default: \"\" = cpu\n steps:\n - uses: actions/checkout@v2\n - name: Set envs and versions\n run: |\n VCS_REF=${{ github.ref }}\n echo \"VCS_REF=$VCS_REF\" >> $GITHUB_ENV\n echo \"Will build $VCS_REF\"\n echo \"BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ')\" >> $GITHUB_ENV\n echo \"BUILD_TARGET=clip_executor\" >> $GITHUB_ENV\n\n CAS_VERSION=$(sed -n '/^__version__/p' ./server/clip_server/__init__.py | cut -d \\' -f2)\n V_CAS_VERSION=v${CAS_VERSION}\n CAS_MINOR_VERSION=${CAS_VERSION%.*}\n CAS_MAJOR_VERSION=${CAS_MINOR_VERSION%.*}\n \n ENGINE_TAG=${{matrix.engine_tag}}\n if [ -n \"${ENGINE_TAG}\" ]; then\n ENGINE_TAG=-${ENGINE_TAG//./}\n fi\n\n PIP_TAG=${{ matrix.pip_tag }}\n BACKEND_TAG=torch\n if [ -n \"${PIP_TAG}\" ]; then\n BACKEND_TAG=${PIP_TAG}\n PIP_TAG=-${PIP_TAG}\n fi\n\n if [[ \"${{ github.event.inputs.triggered_by }}\" == \"CD\" ]]; then\n # on every CD release\n echo \"TAG_ALIAS=\\\n jinaai/clip_executor:master${PIP_TAG}${ENGINE_TAG}\" \\\n >> $GITHUB_ENV\n\n elif [[ \"${{ github.event.inputs.triggered_by }}\" == \"TAG\" ]]; then\n # on every tag release\n echo \"TAG_ALIAS=\\\n jinaai/clip_executor:latest${PIP_TAG}${ENGINE_TAG}, \\\n jinaai/clip_executor:${CAS_VERSION}${PIP_TAG}${ENGINE_TAG}, \\\n jinaai/clip_executor:${CAS_MINOR_VERSION}${PIP_TAG}${ENGINE_TAG} \\\n \" >> $GITHUB_ENV\n \n elif [[ \"${{ github.event.inputs.triggered_by }}\" == \"MANUAL\" ]]; then\n # on every manual release\n echo \"TAG_ALIAS=\\\n jinaai/clip_executor:${CAS_VERSION}${PIP_TAG}${ENGINE_TAG} \\\n \" >> $GITHUB_ENV\n else\n echo \"Bad triggered_by: ${{ github.event.inputs.triggered_by }}!\"\n exit 1\n fi\n\n echo \"CAS_VERSION=${CAS_VERSION}\" >> $GITHUB_ENV\n echo \"BACKEND_TAG=${BACKEND_TAG}\" >> $GITHUB_ENV\n\n - name: Set up Docker Buildx\n id: buildx\n uses: docker/setup-buildx-action@v2\n with:\n install: true\n - name: Login to DockerHub\n uses: docker/login-action@v2\n with:\n username: ${{ secrets.DOCKERHUB_DEVBOT_USER }}\n password: ${{ secrets.DOCKERHUB_DEVBOT_TOKEN }}\n - run: |\n # https://github.com/docker/buildx/issues/464#issuecomment-741507760\n # https://github.com/kubernetes-sigs/azuredisk-csi-driver/pull/808/files\n docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64\n docker run --rm --privileged tonistiigi/binfmt --install all\n - name: CPU Build and push\n id: base_docker_build\n if: ${{ matrix.engine_tag == '' && matrix.pip_tag != 'tensorrt' }}\n uses: docker/build-push-action@v2\n with:\n context: server\n file: Dockerfiles/base.Dockerfile\n platforms: linux/amd64\n cache-from: type=registry,ref=jinaai/clip_executor:latest\n cache-to: type=inline\n push: true\n tags: ${{env.TAG_ALIAS}}\n build-args: |\n BUILD_DATE=${{env.BUILD_DATE}}\n CAS_VERSION=${{env.CAS_VERSION}}\n VCS_REF=${{env.VCS_REF}}\n BACKEND_TAG=${{env.BACKEND_TAG}}\n - name: CUDA Build and push\n id: cuda_docker_build\n if: ${{ matrix.engine_tag == 'cuda' }}\n uses: docker/build-push-action@v2\n with:\n context: server\n file: Dockerfiles/cuda.Dockerfile\n platforms: linux/amd64\n cache-from: type=registry,ref=jinaai/clip_executor:latest-cuda\n cache-to: type=inline\n push: true\n tags: ${{env.TAG_ALIAS}}\n build-args: |\n BUILD_DATE=${{env.BUILD_DATE}}\n CAS_VERSION=${{env.CAS_VERSION}}\n VCS_REF=${{env.VCS_REF}}\n BACKEND_TAG=${{env.BACKEND_TAG}}\n" }, { "path": ".github/workflows/force-docs-build.yml", "content": "name: Manual Docs Build\n\non:\n workflow_dispatch:\n inputs:\n release_token:\n description: 'Your release token'\n required: true\n triggered_by:\n description: 'CD | TAG | MANUAL'\n required: false\n default: MANUAL\n\njobs:\n token-check:\n runs-on: ubuntu-latest\n steps:\n - run: echo \"success!\"\n if: \"${{ github.event.inputs.release_token }} == ${{ env.release_token }}\"\n env:\n release_token: ${{ secrets.CAS_RELEASE_TOKEN }}\n\n release-docs:\n needs: token-check\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v2\n with:\n fetch-depth: 0\n - uses: actions/setup-python@v2\n with:\n python-version: 3.7\n - name: Build doc and push to gh-pages\n run: |\n git config --local user.email \"dev-bot@jina.ai\"\n git config --local user.name \"Jina Dev Bot\"\n pip install --no-cache-dir client/\n pip install --no-cache-dir server/\n mkdir gen-html\n cd docs\n pip install -r requirements.txt\n pip install --pre -U furo\n bash makedoc.sh\n cd ./_build/dirhtml/\n cp -r ./ ../../../gen-html\n cd - # back to ./docs\n cd ..\n git checkout -f gh-pages\n git rm -rf ./docs\n mkdir -p docs\n cd gen-html\n cp -r ./ ../docs\n cd ../docs\n ls -la\n touch .nojekyll\n cp 404/index.html 404.html\n sed -i 's/href=\"\\.\\./href=\"/' 404.html # fix asset urls that needs to be updated in 404.html\n echo clip-as-service.jina.ai > CNAME\n cd ..\n git add docs\n git status\n git commit -m \"chore(docs): update docs due to ${{github.event_name}} on ${{github.repository}}\"\n git push --force origin gh-pages" }, { "path": ".github/workflows/force-hub-push.yml", "content": "name: Manual Hub Push\n\non:\n workflow_dispatch:\n inputs:\n release_token:\n description: 'Your release token'\n required: true\n triggered_by:\n description: 'CD | TAG | MANUAL'\n required: false\n default: MANUAL\n\n#on:\n# pull_request:\n\njobs:\n token-check:\n runs-on: ubuntu-latest\n steps:\n - run: echo \"success!\"\n if: \"${{ github.event.inputs.release_token }} == ${{ env.release_token }}\"\n env:\n release_token: ${{ secrets.CAS_RELEASE_TOKEN }}\n\n hub-release:\n needs: token-check\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v2\n - name: Set envs and versions\n run: |\n VCS_REF=${{ github.ref }}\n echo \"VCS_REF=$VCS_REF\" >> $GITHUB_ENV\n echo \"Will push $VCS_REF\"\n\n CAS_VERSION=$(sed -n '/^__version__/p' ./server/clip_server/__init__.py | cut -d \\' -f2)\n V_CAS_VERSION=v${CAS_VERSION}\n CAS_MINOR_VERSION=${CAS_VERSION%.*}\n CAS_MAJOR_VERSION=${CAS_MINOR_VERSION%.*}\n\n if [[ \"${{ github.event.inputs.triggered_by }}\" == \"CD\" ]]; then\n # on every CD release\n echo \"TAG_ALIAS=\\\n -t latest \\\n \" >> $GITHUB_ENV\n echo \"GPU_TAG_ALIAS=\\\n -t latest-gpu \\\n \" >> $GITHUB_ENV\n\n elif [[ \"${{ github.event.inputs.triggered_by }}\" == \"TAG\" ]]; then\n # on every tag release\n echo \"TAG_ALIAS=\\\n -t latest \\\n -t ${CAS_VERSION} \\\n -t ${CAS_MINOR_VERSION} \\\n \" >> $GITHUB_ENV\n echo \"GPU_TAG_ALIAS=\\\n -t latest-gpu \\\n -t ${CAS_VERSION}-gpu \\\n -t ${CAS_MINOR_VERSION}-gpu \\\n \" >> $GITHUB_ENV\n\n elif [[ \"${{ github.event.inputs.triggered_by }}\" == \"MANUAL\" ]]; then\n # on every manual release\n echo \"TAG_ALIAS=\\\n -t ${CAS_VERSION} \\\n \" >> $GITHUB_ENV\n echo \"GPU_TAG_ALIAS=\\\n -t ${CAS_VERSION}-gpu \\\n \" >> $GITHUB_ENV\n else\n echo \"TAG_ALIAS=\\\n -t latest \\\n \" >> $GITHUB_ENV\n echo \"GPU_TAG_ALIAS=\\\n -t latest-gpu \\\n \" >> $GITHUB_ENV\n fi\n\n echo \"CAS_VERSION=${CAS_VERSION}\" >> $GITHUB_ENV\n\n - name: Prepare enviroment\n run: |\n python -m pip install --upgrade jina yq\n\n - name: Push Torch Executor\n id: push_torch_executor\n run: |\n # FIX the import issue\n echo -e \"\\\n __version__ = '$CAS_VERSION'\n from .executors.clip_torch import CLIPEncoder\\n\\\n \" > server/clip_server/__init__.py\n \n echo -e \"\\\n jtype: CLIPEncoder\\n\\\n metas:\\n\\\n py_modules:\\n\\\n - clip_server/__init__.py\\n\\\n \" > server/config.yml\n \n echo -e \"\\\n manifest_version: 1\\n\\\n name: CLIPTorchEncoder\\n\\\n description: Embed images and sentences into fixed-length vectors with CLIP\\n\\\n url: https://github.com/jina-ai/clip-as-service\\n\\\n keywords: [clip, clip-model, clip-as-service, pytorch]\\n\\\n \" > server/manifest.yml\n \n python scripts/get-requirements.py \"\" server/requirements.txt\n \n cp .github/README-exec/torch.readme.md server/README.md\n \n exec_name=`yq -r .name server/manifest.yml`\n echo executor name is $exec_name\n\n cp Dockerfiles/base.Dockerfile server/Dockerfile\n JINA_AUTH_TOKEN=${{secrets.JINAHUB_TOKEN}} jina hub push --force $exec_name --secret ${{secrets.TORCH_EXEC_SECRET}} server ${{env.TAG_ALIAS}}\n\n cp Dockerfiles/cuda.Dockerfile server/Dockerfile\n JINA_AUTH_TOKEN=${{secrets.JINAHUB_TOKEN}} jina hub push --force $exec_name --secret ${{secrets.TORCH_EXEC_SECRET}} server ${{env.GPU_TAG_ALIAS}}\n\n - name: Push Onnx Executor\n id: push_onnx_executor\n run: |\n # FIX the import issue\n echo -e \"\\\n __version__ = '$CAS_VERSION'\n from .executors.clip_onnx import CLIPEncoder\\n\\\n \" > server/clip_server/__init__.py\n \n echo -e \"\\\n jtype: CLIPEncoder\\n\\\n metas:\\n\\\n py_modules:\\n\\\n - clip_server/__init__.py\\n\\\n \" > server/config.yml\n \n echo -e \"\\\n manifest_version: 1\\n\\\n name: CLIPOnnxEncoder\\n\\\n description: Embed images and sentences into fixed-length vectors with CLIP\\n\\\n url: https://github.com/jina-ai/clip-as-service\\n\\\n keywords: [clip, clip-model, clip-as-service, onnx, onnx-runtime]\\n\\\n \" > server/manifest.yml\n \n python scripts/get-requirements.py onnx server/requirements.txt\n \n cp .github/README-exec/onnx.readme.md server/README.md\n \n exec_name=`yq -r .name server/manifest.yml`\n echo executor name is $exec_name\n \n cp Dockerfiles/base.Dockerfile server/Dockerfile\n sed -i 's/ARG BACKEND_TAG=torch/ARG BACKEND_TAG=onnx/g' server/Dockerfile \n JINA_AUTH_TOKEN=${{secrets.JINAHUB_TOKEN}} jina hub push --force $exec_name --secret ${{secrets.ONNX_EXEC_SECRET}} server ${{env.TAG_ALIAS}}\n \n cp Dockerfiles/cuda.Dockerfile server/Dockerfile\n sed -i 's/ARG BACKEND_TAG=torch/ARG BACKEND_TAG=onnx/g' server/Dockerfile\n JINA_AUTH_TOKEN=${{secrets.JINAHUB_TOKEN}} jina hub push --force $exec_name --secret ${{secrets.ONNX_EXEC_SECRET}} server ${{env.GPU_TAG_ALIAS}}\n\n - name: Push TensorRT Executor\n id: push_tensorrt_executor\n run: |\n # FIX the import issue\n echo -e \"\\\n __version__ = '$CAS_VERSION'\n from .executors.clip_tensorrt import CLIPEncoder\\n\\\n \" > server/clip_server/__init__.py\n \n echo -e \"\\\n jtype: CLIPEncoder\\n\\\n metas:\\n\\\n py_modules:\\n\\\n - clip_server/__init__.py\\n\\\n \" > server/config.yml\n \n echo -e \"\\\n manifest_version: 1\\n\\\n name: CLIPTensorRTEncoder\\n\\\n description: Embed images and sentences into fixed-length vectors with CLIP\\n\\\n url: https://github.com/jina-ai/clip-as-service\\n\\\n keywords: [clip, clip-model, clip-as-service, onnx, tensorrt]\\n\\\n \" > server/manifest.yml\n \n python scripts/get-requirements.py tensorrt server/requirements.txt\n \n cp Dockerfiles/tensorrt.Dockerfile server/Dockerfile\n \n exec_name=`yq -r .name server/manifest.yml`\n echo executor name is $exec_name\n \n # FIXME: disable uploading at debugging\n # JINA_AUTH_TOKEN=${{secrets.JINAHUB_TOKEN}} jina hub push --force $exec_name --secret ${{secrets.TENSORRT_EXEC_SECRET}} server ${{env.TAG_ALIAS}}\n" }, { "path": ".github/workflows/force-release.yml", "content": "name: Manual Release\n\non:\n workflow_dispatch:\n inputs:\n release_token:\n description: 'Your release token'\n required: true\n release_reason:\n description: 'Short reason for this manual release'\n required: true\n\njobs:\n token-check:\n runs-on: ubuntu-latest\n steps:\n - run: echo \"success!\"\n if: \"${{ github.event.inputs.release_token }} == ${{ env.release_token }}\"\n env:\n release_token: ${{ secrets.CAS_RELEASE_TOKEN }}\n\n regular-release:\n needs: token-check\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v2\n with:\n token: ${{ secrets.JINA_DEV_BOT }}\n fetch-depth: 100 # means max contribute history is limited to 100 lines\n# submodules: true\n - uses: actions/setup-python@v2\n with:\n python-version: 3.7\n - run: |\n git fetch --depth=1 origin +refs/tags/*:refs/tags/*\n npm install git-release-notes\n pip install twine wheel\n ./scripts/release.sh final \"${{ github.event.inputs.release_reason }}\" \"${{github.actor}}\"\n env:\n TWINE_USERNAME: ${{ secrets.TWINE_USERNAME }}\n TWINE_PASSWORD: ${{ secrets.TWINE_PASSWORD }}\n - if: failure()\n run: echo \"nothing to release\"\n - name: bumping master version\n uses: ad-m/github-push-action@v0.6.0\n with:\n github_token: ${{ secrets.JINA_DEV_BOT }}\n tags: true\n branch: main\n" }, { "path": ".github/workflows/label-pr.yml", "content": "name: PR\n\non:\n pull_request:\n\n\njobs:\n assign-label-to-pr:\n runs-on: ubuntu-latest\n if: ${{ !github.event.pull_request.head.repo.fork }}\n steps:\n - uses: codelytv/pr-size-labeler@v1\n with:\n GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n xs_max_size: '10'\n s_max_size: '100'\n m_max_size: '500'\n l_max_size: '1000'\n fail_if_xl: 'false'\n - uses: actions/labeler@v3\n with:\n repo-token: \"${{ secrets.GITHUB_TOKEN }}\"\n - id: docs_updated\n if: contains( github.event.pull_request.labels.*.name, 'area/docs')\n run: echo '::set-output name=docs::true'\n outputs:\n docs: ${{ steps.docs_updated.outputs.docs }}\n\n deploy-to-netlify:\n runs-on: ubuntu-latest\n needs: [assign-label-to-pr]\n if: ${{ needs.assign-label-to-pr.outputs.docs == 'true' }}\n steps:\n - run: |\n echo \"BRANCH_NAME=${{ github.head_ref }}\" >> $GITHUB_ENV\n - uses: actions/checkout@v2\n with:\n repository: jina-ai/clip-as-service\n ref: ${{ env.BRANCH_NAME }}\n - uses: actions/setup-python@v2\n with:\n python-version: 3.7\n - uses: actions/setup-node@v2\n with:\n node-version: '14'\n - name: Build and Deploy\n run: |\n npm i -g netlify-cli\n python -m pip install --upgrade pip\n pip install -r requirements.txt\n git fetch origin\n export NUM_RELEASES=2 # only 2 last tags to save build time\n bash makedoc.sh development\n netlify deploy --dir=_build/dirhtml --alias=\"ft-${{ env.BRANCH_NAME }}\" --message=\"Deploying docs to ${{ env.BRANCH_NAME }} branch\"\n env:\n NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN1 }}\n NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}\n working-directory: docs\n - name: Find the prev comment if exists\n uses: peter-evans/find-comment@v1\n id: fc\n with:\n issue-number: ${{ github.event.pull_request.number }}\n comment-author: 'github-actions[bot]'\n body-includes: 'Docs are deployed'\n - name: Delete comment if exists\n if: ${{ steps.fc.outputs.comment-id != 0 && !github.event.pull_request.head.repo.fork }}\n uses: actions/github-script@v3\n with:\n github-token: ${{ secrets.GITHUB_TOKEN }}\n script: |\n github.issues.deleteComment({\n owner: context.repo.owner,\n repo: context.repo.repo,\n comment_id: ${{ steps.fc.outputs.comment-id }},\n })\n - name: Add or update comment\n uses: peter-evans/create-or-update-comment@v1\n with:\n issue-number: ${{ github.event.pull_request.number }}\n body: |\n :memo: Docs are deployed on https://ft-${{ env.BRANCH_NAME }}--jina-docs.netlify.app :tada: \n" }, { "path": ".github/workflows/tag.yml", "content": "name: Release CD\n\non:\n push:\n tags:\n - \"v*\" # push to version tags trigger the build\n\njobs:\n update-doc:\n runs-on: ubuntu-latest\n steps:\n - uses: benc-uk/workflow-dispatch@v1\n with:\n workflow: Manual Docs Build\n token: ${{ secrets.JINA_DEV_BOT }}\n inputs: '{ \"release_token\": \"${{ env.release_token }}\", \"triggered_by\": \"TAG\"}'\n env:\n release_token: ${{ secrets.CAS_RELEASE_TOKEN }}\n\n update-docker:\n needs: update-doc\n runs-on: ubuntu-latest\n steps:\n - name: CAS Docker Build\n uses: benc-uk/workflow-dispatch@v1\n with:\n workflow: Manual CAS Docker Build\n inputs: '{ \"release_token\": \"${{ env.release_token }}\", \"triggered_by\": \"TAG\"}'\n token: ${{ secrets.JINA_DEV_BOT }}\n env:\n release_token: ${{ secrets.CAS_RELEASE_TOKEN }}\n - name: Helm Executor Build\n uses: benc-uk/workflow-dispatch@v1\n with:\n workflow: Manual Docker Build\n inputs: '{ \"release_token\": \"${{ env.release_token }}\", \"triggered_by\": \"TAG\"}'\n token: ${{ secrets.JINA_DEV_BOT }}\n env:\n release_token: ${{ secrets.CAS_RELEASE_TOKEN }}\n - name: Hub Executor Build\n uses: benc-uk/workflow-dispatch@v1\n with:\n workflow: Manual Hub Push\n inputs: '{ \"release_token\": \"${{ env.release_token }}\", \"triggered_by\": \"TAG\"}'\n token: ${{ secrets.JINA_DEV_BOT }}\n env:\n release_token: ${{ secrets.CAS_RELEASE_TOKEN }}\n\n create-release:\n needs: update-doc\n runs-on: ubuntu-latest\n steps:\n - name: Checkout code\n uses: actions/checkout@v2\n with:\n ref: 'main'\n - uses: actions/setup-python@v2\n with:\n python-version: 3.7\n - run: |\n python scripts/get-last-release-note.py\n - name: Create Release\n id: create_release\n uses: actions/create-release@v1\n env:\n GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # This token is provided by Actions, you do not need to create your own token\n with:\n tag_name: ${{ github.ref }}\n release_name: ๐Ÿ’ซ Patch ${{ github.ref }}\n body_path: 'tmp.md'\n draft: false\n prerelease: false\n" }, { "path": ".gitignore", "content": "# Initially taken from Github's Python gitignore file\n\n# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# Distribution / packaging\n.Python\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\n*.egg-info/\n.installed.cfg\n*.egg\nMANIFEST\n\n# PyInstaller\n# Usually these files are written by a python script from a template\n# before PyInstaller builds the exe, so as to inject date/other infos into it.\n*.manifest\n*.spec\n\n# Installer logs\npip-log.txt\npip-delete-this-directory.txt\n\n# Unit test / coverage reports\nhtmlcov/\n.tox/\n.nox/\n.coverage\n.coverage.*\n.cache\nnosetests.xml\ncoverage.xml\n*.cover\n.hypothesis/\n.pytest_cache/\n\n# Translations\n*.mo\n*.pot\n\n# Django stuff:\n*.log\nlocal_settings.py\ndb.sqlite3\n\n# Flask stuff:\ninstance/\n.webassets-cache\n\n# Scrapy stuff:\n.scrapy\n\n# Sphinx documentation\ndocs/_build/\ndocs/api/\n\n# PyBuilder\ntarget/\n\n# Jupyter Notebook\n.ipynb_checkpoints\n\n# IPython\nprofile_default/\nipython_config.py\n\n# pyenv\ndocs/.python-version\n\n# celery beat schedule file\ncelerybeat-schedule\n\n# SageMath parsed files\n*.sage.py\n\n# Environments\n.venv\nenv/\nvenv/\nENV/\nenv.bak/\nvenv.bak/\n\n# Spyder project settings\n.spyderproject\n.spyproject\n\n# Rope project settings\n.ropeproject\n\n# mkdocs documentation\n/site\n\n# mypy\n.mypy_cache/\n.dmypy.json\ndmypy.json\n\n# Pyre type checker\n.pyre/\n.idea/\ntoy*.py\n.DS_Store\npost/\ntoy*.ipynb\ndata/\n*.c\n.nes_cache\ntoy*.yml\n*.tmp\n\nshell/jina-wizard.sh\n/junit/\n/tests/junit/\n/docs/chapters/proto/docs.md\n/tests/.pytest-kind\n\n# IntelliJ IDEA\n*.iml\n.idea\n\n# VSCode\n.vscode\n\n# test with config in resources\ntests/integration/crud/simple/simple_indexer/\n\n# latency tracking\nlatency\nMyIndexer/\nMyMemMap/\noriginal/\noutput/\n\n# kubernetes testing\n.pytest-kind\n.kube" }, { "path": ".pre-commit-config.yaml", "content": "repos:\n- repo: https://github.com/ambv/black\n rev: 22.3.0\n hooks:\n - id: black\n types: [python]\n exclude: ^(docs/|server/clip_server/resources/)\n args:\n - -S\n- repo: https://github.com/asottile/blacken-docs\n rev: v1.12.1\n hooks:\n - id: blacken-docs\n args:\n - -S" }, { "path": "CHANGELOG.md", "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n## Release Note (`0.0.3`)\n\n> Release time: 2022-03-23 21:42:16\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Dmitry Kan, varshaneya, Ilya Usvyatsky, Nicko van Someren, George Gkotsis, Jhang, Changrui Zhang, DomHudson, Filip Bednรกrik, ๐Ÿ™‡\n\n\n### ๐Ÿน Other Improvements\n\n - [[```378d82b5```](https://github.com/jina-ai/clip-as-service/commit/378d82b5d20a627a7a32239ebd9b47cbd12a5f7a)] __-__ fix setup and release script (*Han Xiao*)\n - [[```372de00f```](https://github.com/jina-ai/clip-as-service/commit/372de00f286565750cab51d6e33ca4d9471a2934)] __-__ fix workflow yaml config (*Han Xiao*)\n - [[```11822f60```](https://github.com/jina-ai/clip-as-service/commit/11822f6050096a4ed1ca7bb9ee3c082deec56fb4)] __-__ fix image (*Han Xiao*)\n - [[```78a6a8b9```](https://github.com/jina-ai/clip-as-service/commit/78a6a8b9de9cab353be63311facce670212de08d)] __-__ first commit (*Han Xiao*)\n - [[```f5e42383```](https://github.com/jina-ai/clip-as-service/commit/f5e4238397ab76a1539bb6f22d5735f563ff187e)] __-__ update readme (*Han Xiao*)\n - [[```c4790fbe```](https://github.com/jina-ai/clip-as-service/commit/c4790fbef902e8d3509d1229788cb81abcdc33d6)] __-__ modified the port 8001->8081 to match Vue.js demo (*Dmitry Kan*)\n - [[```749c8e45```](https://github.com/jina-ai/clip-as-service/commit/749c8e45d301967dcaa8744ef846190ac86d2932)] __-__ update readme header (*Han Xiao*)\n\n\n## Release Note (`0.0.4`)\n\n> Release time: 2022-03-23 21:45:59\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿน Other Improvements\n\n - [[```f8936108```](https://github.com/jina-ai/clip-as-service/commit/f89361085bbe2715fd0d1c2d769389e0a46dc860)] __-__ fix setup and release script (*Han Xiao*)\n - [[```d2e4cfbf```](https://github.com/jina-ai/clip-as-service/commit/d2e4cfbf1977db1cf0fee433600b48b0c3626312)] __-__ __version__: the next version will be 0.0.4 (*Jina Dev Bot*)\n\n\n## Release Note (`0.0.5`)\n\n> Release time: 2022-03-23 22:09:18\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿน Other Improvements\n\n - [[```7ed4643c```](https://github.com/jina-ai/clip-as-service/commit/7ed4643cc471a6433ee6cc8699617ae61978bcdb)] __-__ fix doc setup (*Han Xiao*)\n - [[```fe09c32c```](https://github.com/jina-ai/clip-as-service/commit/fe09c32c8629b20ea9f677b7097556e33091876f)] __-__ __version__: the next version will be 0.0.5 (*Jina Dev Bot*)\n - [[```f8936108```](https://github.com/jina-ai/clip-as-service/commit/f89361085bbe2715fd0d1c2d769389e0a46dc860)] __-__ fix setup and release script (*Han Xiao*)\n\n\n## Release Note (`0.0.6`)\n\n> Release time: 2022-03-23 22:42:28\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿน Other Improvements\n\n - [[```f7044fb2```](https://github.com/jina-ai/clip-as-service/commit/f7044fb2f9c81f5dbfb7fec7c12c6a3b0dd54fa6)] __-__ fix doc setup (*Han Xiao*)\n - [[```c04eb30e```](https://github.com/jina-ai/clip-as-service/commit/c04eb30e8e09c89b1dd7061d0cb044286a1c5fc2)] __-__ __version__: the next version will be 0.0.6 (*Jina Dev Bot*)\n\n\n## Release Note (`0.0.7`)\n\n> Release time: 2022-03-24 07:04:50\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿน Other Improvements\n\n - [[```e8aa643a```](https://github.com/jina-ai/clip-as-service/commit/e8aa643a802fca900f2111407093107f22f08917)] __-__ update docs and license (*Han Xiao*)\n - [[```7245f67a```](https://github.com/jina-ai/clip-as-service/commit/7245f67adf76e600762d8dbdbaee747764c0677c)] __-__ __version__: the next version will be 0.0.7 (*Jina Dev Bot*)\n - [[```f7044fb2```](https://github.com/jina-ai/clip-as-service/commit/f7044fb2f9c81f5dbfb7fec7c12c6a3b0dd54fa6)] __-__ fix doc setup (*Han Xiao*)\n\n\n## Release Note (`0.1.0`)\n\n> Release time: 2022-03-24 08:19:14\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ“— Documentation\n\n - [[```fa9d50c2```](https://github.com/jina-ai/clip-as-service/commit/fa9d50c2395ea1e1e60a2a0fc9c0105df472bcdb)] __-__ fix readme (#656) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```1a2a2af9```](https://github.com/jina-ai/clip-as-service/commit/1a2a2af932717cc0da12667453cd00c58e6ee443)] __-__ bump version (*Han Xiao*)\n - [[```44f4e52e```](https://github.com/jina-ai/clip-as-service/commit/44f4e52eba72d903883a827e115fb8dead69111f)] __-__ update docstring (*Han Xiao*)\n - [[```12bf98aa```](https://github.com/jina-ai/clip-as-service/commit/12bf98aa5b098f9017af186642990d9f854c54d7)] __-__ __version__: the next version will be 0.0.8 (*Jina Dev Bot*)\n - [[```e8aa643a```](https://github.com/jina-ai/clip-as-service/commit/e8aa643a802fca900f2111407093107f22f08917)] __-__ update docs and license (*Han Xiao*)\n\n\n## Release Note (`0.1.1`)\n\n> Release time: 2022-03-24 09:03:13\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Wang Bo, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```dd4cb3c3```](https://github.com/jina-ai/clip-as-service/commit/dd4cb3c3e142a8b8f702c4eee57a251aab7a10d5)] __-__ url description and keywords in setup (#657) (*Wang Bo*)\n\n### ๐Ÿน Other Improvements\n\n - [[```1b679cdc```](https://github.com/jina-ai/clip-as-service/commit/1b679cdc8c657e872833f1344f19ed7d6bb57b0a)] __-__ fix banner (*Han Xiao*)\n - [[```9e0a1058```](https://github.com/jina-ai/clip-as-service/commit/9e0a1058b77f33508a3dbdffabaacf3bb0c53cb4)] __-__ __version__: the next version will be 0.1.1 (*Jina Dev Bot*)\n - [[```1a2a2af9```](https://github.com/jina-ai/clip-as-service/commit/1a2a2af932717cc0da12667453cd00c58e6ee443)] __-__ bump version (*Han Xiao*)\n\n\n## Release Note (`0.1.2`)\n\n> Release time: 2022-03-24 10:57:53\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Alex Cureton-Griffiths, Wang Bo, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿงผ Code Refactoring\n\n - [[```715e8ba9```](https://github.com/jina-ai/clip-as-service/commit/715e8ba90faf1c58daed608cd02340404484a018)] __-__ remove unused main unify keywords (#658) (*Wang Bo*)\n\n### ๐Ÿ“— Documentation\n\n - [[```ff16ce1d```](https://github.com/jina-ai/clip-as-service/commit/ff16ce1db88bbac14cb19c05266b1286c224e65d)] __-__ __readme__: polish (#660) (*Alex Cureton-Griffiths*)\n\n### ๐Ÿน Other Improvements\n\n - [[```0ec1fac2```](https://github.com/jina-ai/clip-as-service/commit/0ec1fac2a4f0c02a0cc704da9b2361e1d22108fa)] __-__ fix setup deps (*Han Xiao*)\n - [[```59b154a7```](https://github.com/jina-ai/clip-as-service/commit/59b154a7358cfdf1449fab8a2855599b657ddf24)] __-__ __version__: the next version will be 0.1.2 (*Jina Dev Bot*)\n - [[```1b679cdc```](https://github.com/jina-ai/clip-as-service/commit/1b679cdc8c657e872833f1344f19ed7d6bb57b0a)] __-__ fix banner (*Han Xiao*)\n\n\n## Release Note (`0.1.3`)\n\n> Release time: 2022-03-24 13:03:30\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Wang Bo, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿงผ Code Refactoring\n\n - [[```ae4d0bac```](https://github.com/jina-ai/clip-as-service/commit/ae4d0bacbfadb57e1cb1de8a5f6adca5426e62d3)] __-__ remove inference model pytorch from onnx (#661) (*Wang Bo*)\n\n### ๐Ÿน Other Improvements\n\n - [[```dece9dd0```](https://github.com/jina-ai/clip-as-service/commit/dece9dd0695bc418fc13fe9ed95ef52ba9f8f19d)] __-__ fix setup file (*Han Xiao*)\n - [[```3d04e695```](https://github.com/jina-ai/clip-as-service/commit/3d04e69511098490b1e56e15640a6378df89ff04)] __-__ __version__: the next version will be 0.1.3 (*Jina Dev Bot*)\n - [[```0ec1fac2```](https://github.com/jina-ai/clip-as-service/commit/0ec1fac2a4f0c02a0cc704da9b2361e1d22108fa)] __-__ fix setup deps (*Han Xiao*)\n\n\n## Release Note (`0.1.5`)\n\n> Release time: 2022-03-24 19:17:54\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿน Other Improvements\n\n - [[```989a706a```](https://github.com/jina-ai/clip-as-service/commit/989a706aa53271d9018ce48d9241e37887572025)] __-__ hide top-level setup (*Han Xiao*)\n - [[```f07e0f57```](https://github.com/jina-ai/clip-as-service/commit/f07e0f57ab795688100d07c09564fe3680031a83)] __-__ __version__: the next version will be 0.1.4 (*Jina Dev Bot*)\n - [[```dece9dd0```](https://github.com/jina-ai/clip-as-service/commit/dece9dd0695bc418fc13fe9ed95ef52ba9f8f19d)] __-__ fix setup file (*Han Xiao*)\n\n\n## Release Note (`0.1.6`)\n\n> Release time: 2022-03-29 07:02:26\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, felix-wang, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```b4624dd4```](https://github.com/jina-ai/clip-as-service/commit/b4624dd408e0ee3908f57b7e5e598e5631de3d95)] __-__ __client__: raise value when embedding is empty (#666) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```16f8c403```](https://github.com/jina-ai/clip-as-service/commit/16f8c403a274961c3acd1ecddbef823dcea488b2)] __-__ fix typo (#664) (*felix-wang*)\n - [[```da1dd85c```](https://github.com/jina-ai/clip-as-service/commit/da1dd85cba6bc72c4ff83fd0c00c2651eef56075)] __-__ hide top-level setup (*Han Xiao*)\n - [[```fe22c3f2```](https://github.com/jina-ai/clip-as-service/commit/fe22c3f21ee180c7f74e1a996572b0eb04a36a6d)] __-__ __version__: the next version will be 0.1.6 (*Jina Dev Bot*)\n\n\n## Release Note (`0.1.7`)\n\n> Release time: 2022-03-30 11:40:37\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Roshan Jossy, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```d56b1463```](https://github.com/jina-ai/clip-as-service/commit/d56b146392c16d7f917b3a6baeba9d6b77121249)] __-__ __client__: more comprehensive progressbar (#667) (*Han Xiao*)\n\n### ๐Ÿž Bug fixes\n\n - [[```b4624dd4```](https://github.com/jina-ai/clip-as-service/commit/b4624dd408e0ee3908f57b7e5e598e5631de3d95)] __-__ __client__: raise value when embedding is empty (#666) (*Han Xiao*)\n\n### ๐Ÿ“— Documentation\n\n - [[```cfaba711```](https://github.com/jina-ai/clip-as-service/commit/cfaba7119c334d87a5f7860cc4df65a52f19c750)] __-__ __tracking__: add scarf tracking (#665) (*Roshan Jossy*)\n\n### ๐Ÿน Other Improvements\n\n - [[```9e276744```](https://github.com/jina-ai/clip-as-service/commit/9e27674447d041c21078082478acc234d5c0e3f7)] __-__ fix readme (*Han Xiao*)\n - [[```1c2de8da```](https://github.com/jina-ai/clip-as-service/commit/1c2de8da0ac3cfa7fa7304f6c55b4979747f22a5)] __-__ __version__: the next version will be 0.1.7 (*Jina Dev Bot*)\n\n\n## Release Note (`0.1.8`)\n\n> Release time: 2022-03-30 14:30:36\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```d56b1463```](https://github.com/jina-ai/clip-as-service/commit/d56b146392c16d7f917b3a6baeba9d6b77121249)] __-__ __client__: more comprehensive progressbar (#667) (*Han Xiao*)\n\n### ๐Ÿงผ Code Refactoring\n\n - [[```065d6a91```](https://github.com/jina-ai/clip-as-service/commit/065d6a910e44a31718463a51ba0e3c29ca926d1c)] __-__ __client__: use docarray pbar (#668) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```dd61bdce```](https://github.com/jina-ai/clip-as-service/commit/dd61bdce6f6e908a6a583106d1c51ca5725b1ad4)] __-__ update readme (*Han Xiao*)\n - [[```be5fff81```](https://github.com/jina-ai/clip-as-service/commit/be5fff8129628d069e8d2992705f5f8b4681e040)] __-__ __version__: the next version will be 0.1.8 (*Jina Dev Bot*)\n\n\n## Release Note (`0.1.9`)\n\n> Release time: 2022-03-30 23:20:09\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### โšก Performance Improvements\n\n - [[```88123432```](https://github.com/jina-ai/clip-as-service/commit/8812343211701fd93b75dab9f5b86bd5bc9e6819)] __-__ __server__: use map_batch to overlap cpu gpu (#669) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```41b93773```](https://github.com/jina-ai/clip-as-service/commit/41b937732a4f241e22ef9089071e9c7b611a6674)] __-__ fix readme (*Han Xiao*)\n - [[```da3227d3```](https://github.com/jina-ai/clip-as-service/commit/da3227d3b3df984a2816dbea283c209020b0815a)] __-__ update readme (*Han Xiao*)\n - [[```431d4635```](https://github.com/jina-ai/clip-as-service/commit/431d46353c3382ff954fc05bc8397f3e560a7ac7)] __-__ __version__: the next version will be 0.1.9 (*Jina Dev Bot*)\n\n\n## Release Note (`0.1.10`)\n\n> Release time: 2022-03-31 10:31:26\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### โšก Performance Improvements\n\n - [[```962a1b5c```](https://github.com/jina-ai/clip-as-service/commit/962a1b5ce92249d98e4850e0af003d63b95a3f9d)] __-__ __server__: reuse the preprocessing pool (#670) (*Han Xiao*)\n - [[```88123432```](https://github.com/jina-ai/clip-as-service/commit/8812343211701fd93b75dab9f5b86bd5bc9e6819)] __-__ __server__: use map_batch to overlap cpu gpu (#669) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```f0dfc34a```](https://github.com/jina-ai/clip-as-service/commit/f0dfc34adca3c3d7b9879ac518735dd929074e80)] __-__ __version__: the next version will be 0.1.10 (*Jina Dev Bot*)\n\n\n## Release Note (`0.1.11`)\n\n> Release time: 2022-04-01 15:46:07\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### โšก Performance Improvements\n\n - [[```962a1b5c```](https://github.com/jina-ai/clip-as-service/commit/962a1b5ce92249d98e4850e0af003d63b95a3f9d)] __-__ __server__: reuse the preprocessing pool (#670) (*Han Xiao*)\n\n### ๐Ÿ“— Documentation\n\n - [[```257f0393```](https://github.com/jina-ai/clip-as-service/commit/257f03931d28a0d9020a3381495a41df1185fd9c)] __-__ add http endpoint explain (#671) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```2e9e212f```](https://github.com/jina-ai/clip-as-service/commit/2e9e212f0d9501c7d55d0fd8dd82c180a067ca89)] __-__ add demo server (*Han Xiao*)\n - [[```99133924```](https://github.com/jina-ai/clip-as-service/commit/991339248bb3d7e9b6a741f2cf456f7f8bade154)] __-__ __version__: the next version will be 0.1.11 (*Jina Dev Bot*)\n\n\n## Release Note (`0.1.12`)\n\n> Release time: 2022-04-07 02:20:52\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, samsja, Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```ffc4bdc4```](https://github.com/jina-ai/clip-as-service/commit/ffc4bdc4e2414fa0aff67bb5cbaffd4077a7c8f4)] __-__ gitignore (#673) (*samsja*)\n\n### ๐Ÿž Bug fixes\n\n - [[```aeb64c08```](https://github.com/jina-ai/clip-as-service/commit/aeb64c082d4819ab9330ba65f57f13ba2a868268)] __-__ ignore onnxruntime-gpu on macos (#675) (*felix-wang*)\n\n### ๐Ÿ“— Documentation\n\n - [[```257f0393```](https://github.com/jina-ai/clip-as-service/commit/257f03931d28a0d9020a3381495a41df1185fd9c)] __-__ add http endpoint explain (#671) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```e2b2ae8b```](https://github.com/jina-ai/clip-as-service/commit/e2b2ae8bb96ce8602f9808c01e9a1712b7cdf7ac)] __-__ update readme (*Han Xiao*)\n - [[```d7aa1615```](https://github.com/jina-ai/clip-as-service/commit/d7aa161503ed72e662f8efe477af702e8759375e)] __-__ __version__: the next version will be 0.1.12 (*Jina Dev Bot*)\n\n\n## Release Note (`0.1.13`)\n\n> Release time: 2022-04-11 08:02:20\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, felix-wang, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```8b800eea```](https://github.com/jina-ai/clip-as-service/commit/8b800eea5d40d02417a4368fcc38ac58dc2d651d)] __-__ __server__: allow client sending tensor document (#678) (*Han Xiao*)\n\n### ๐Ÿž Bug fixes\n\n - [[```aeb64c08```](https://github.com/jina-ai/clip-as-service/commit/aeb64c082d4819ab9330ba65f57f13ba2a868268)] __-__ ignore onnxruntime-gpu on macos (#675) (*felix-wang*)\n\n### ๐Ÿ“— Documentation\n\n - [[```b6f9d849```](https://github.com/jina-ai/clip-as-service/commit/b6f9d849e5693d6c40744feebd5d309dbeed1cb3)] __-__ __server__: docs document tensor (#679) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```fa42dc50```](https://github.com/jina-ai/clip-as-service/commit/fa42dc50f6c766c60fe246802ecc8c15e37fbdf4)] __-__ update docs (*Han Xiao*)\n - [[```c91fa4d1```](https://github.com/jina-ai/clip-as-service/commit/c91fa4d16fd01ba8cc571041919201bbb1a76e31)] __-__ __version__: the next version will be 0.1.13 (*Jina Dev Bot*)\n\n\n## Release Note (`0.1.14`)\n\n> Release time: 2022-04-14 02:39:16\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, Jina Dev Bot, Han Xiao, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```8286eeed```](https://github.com/jina-ai/clip-as-service/commit/8286eeed13e65b7414e7bff0ace689daac103101)] __-__ tensor input document (#681) (*felix-wang*)\n\n### ๐Ÿ“— Documentation\n\n - [[```b6f9d849```](https://github.com/jina-ai/clip-as-service/commit/b6f9d849e5693d6c40744feebd5d309dbeed1cb3)] __-__ __server__: docs document tensor (#679) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```ef6ea254```](https://github.com/jina-ai/clip-as-service/commit/ef6ea254b48d2675364184c3cfcb787819e8433a)] __-__ __version__: the next version will be 0.1.14 (*Jina Dev Bot*)\n\n\n## Release Note (`0.1.15`)\n\n> Release time: 2022-04-18 04:07:21\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### โšก Performance Improvements\n\n - [[```10d53eb4```](https://github.com/jina-ai/clip-as-service/commit/10d53eb4c4b84c95412b5f904de705cc01d7ba89)] __-__ scalable benchmark (#680) (*felix-wang*)\n\n### ๐Ÿž Bug fixes\n\n - [[```8286eeed```](https://github.com/jina-ai/clip-as-service/commit/8286eeed13e65b7414e7bff0ace689daac103101)] __-__ tensor input document (#681) (*felix-wang*)\n\n### ๐Ÿน Other Improvements\n\n - [[```fb229ae8```](https://github.com/jina-ai/clip-as-service/commit/fb229ae81f708d00f01af2d791de6eb67174e1c7)] __-__ add jcloud logo (*Han Xiao*)\n - [[```a3891eed```](https://github.com/jina-ai/clip-as-service/commit/a3891eedfa88e92f4a730f94b897da79ec28848d)] __-__ fix readme (*Han Xiao*)\n - [[```bfd04706```](https://github.com/jina-ai/clip-as-service/commit/bfd047061d7465877503caa3f11f77065718cbc4)] __-__ __version__: the next version will be 0.1.15 (*Jina Dev Bot*)\n\n\n## Release Note (`0.2.0`)\n\n> Release time: 2022-04-18 04:27:56\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n numb3r3, Jina Dev Bot, felix-wang, ๐Ÿ™‡\n\n\n### โšก Performance Improvements\n\n - [[```10d53eb4```](https://github.com/jina-ai/clip-as-service/commit/10d53eb4c4b84c95412b5f904de705cc01d7ba89)] __-__ scalable benchmark (#680) (*felix-wang*)\n\n### ๐Ÿน Other Improvements\n\n - [[```67226f5c```](https://github.com/jina-ai/clip-as-service/commit/67226f5c8ec7d652f5d47ed0ab21dc8d14fc49c8)] __-__ bump version (*numb3r3*)\n - [[```5dc64878```](https://github.com/jina-ai/clip-as-service/commit/5dc6487819483ee680cf6e4c6a5fd1c0c27e2b54)] __-__ __version__: the next version will be 0.1.16 (*Jina Dev Bot*)\n\n\n## Release Note (`0.2.1`)\n\n> Release time: 2022-04-21 15:32:35\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, Han Xiao, Jina Dev Bot, numb3r3, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```2558d738```](https://github.com/jina-ai/clip-as-service/commit/2558d7388a6ebdbfa99e489d9d005d9e1c22c33a)] __-__ pass extra_search path (#687) (*felix-wang*)\n - [[```71e9ebc8```](https://github.com/jina-ai/clip-as-service/commit/71e9ebc89a27b9ed64def3b6968b78c4ea1c3256)] __-__ remove process backend (#685) (*felix-wang*)\n - [[```65ad956d```](https://github.com/jina-ai/clip-as-service/commit/65ad956dab7e4f011568ac14357d636312dd4f5e)] __-__ use one iteration step (#683) (*felix-wang*)\n\n### ๐Ÿน Other Improvements\n\n - [[```e3d4e918```](https://github.com/jina-ai/clip-as-service/commit/e3d4e9181b645dfebd23620e153d86c563d38c9b)] __-__ update readme (*Han Xiao*)\n - [[```ec3a700c```](https://github.com/jina-ai/clip-as-service/commit/ec3a700c632dba97c01529c0049c383d68636aa8)] __-__ update docs (#684) (*felix-wang*)\n - [[```11487cee```](https://github.com/jina-ai/clip-as-service/commit/11487ceec184fbcf93c0c1debedbf22d097d9254)] __-__ __version__: the next version will be 0.2.1 (*Jina Dev Bot*)\n - [[```67226f5c```](https://github.com/jina-ai/clip-as-service/commit/67226f5c8ec7d652f5d47ed0ab21dc8d14fc49c8)] __-__ bump version (*numb3r3*)\n\n\n## Release Note (`0.2.2`)\n\n> Release time: 2022-04-24 07:28:34\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```3bd74641```](https://github.com/jina-ai/clip-as-service/commit/3bd74641e52a48d9b1bac15554c8e42050d47094)] __-__ download with resume (#689) (*felix-wang*)\n - [[```2558d738```](https://github.com/jina-ai/clip-as-service/commit/2558d7388a6ebdbfa99e489d9d005d9e1c22c33a)] __-__ pass extra_search path (#687) (*felix-wang*)\n\n### ๐Ÿน Other Improvements\n\n - [[```6eadbfba```](https://github.com/jina-ai/clip-as-service/commit/6eadbfbac798c72b0319071b3f9c8480f97155d8)] __-__ __version__: the next version will be 0.2.2 (*Jina Dev Bot*)\n\n\n## Release Note (`0.2.3`)\n\n> Release time: 2022-04-25 11:42:51\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, felix-wang, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```0ebc4c03```](https://github.com/jina-ai/clip-as-service/commit/0ebc4c0363aa182d2992bbc11bbcf676aeaf69df)] __-__ __server__: add rank endpoint (#694) (*Han Xiao*)\n\n### ๐Ÿž Bug fixes\n\n - [[```3bd74641```](https://github.com/jina-ai/clip-as-service/commit/3bd74641e52a48d9b1bac15554c8e42050d47094)] __-__ download with resume (#689) (*felix-wang*)\n\n### ๐Ÿน Other Improvements\n\n - [[```22cfffaf```](https://github.com/jina-ai/clip-as-service/commit/22cfffaffbd73d1d15afd87d08e2bb62bb2acbdc)] __-__ __version__: the next version will be 0.2.3 (*Jina Dev Bot*)\n\n\n## Release Note (`0.3.0`)\n\n> Release time: 2022-04-25 15:13:21\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```b7270862```](https://github.com/jina-ai/clip-as-service/commit/b7270862e353b73531cd8bff4735d537c905a312)] __-__ __client__: add rank endpoint (#695) (*Han Xiao*)\n - [[```0ebc4c03```](https://github.com/jina-ai/clip-as-service/commit/0ebc4c0363aa182d2992bbc11bbcf676aeaf69df)] __-__ __server__: add rank endpoint (#694) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```8600286c```](https://github.com/jina-ai/clip-as-service/commit/8600286cf53755c127cf258af918b6bdf3e86691)] __-__ update readme (*Han Xiao*)\n - [[```5e1dd607```](https://github.com/jina-ai/clip-as-service/commit/5e1dd607e47a94265f48cbb2a70406c5057b86fa)] __-__ __version__: the next version will be 0.2.4 (*Jina Dev Bot*)\n\n\n## Release Note (`0.3.1`)\n\n> Release time: 2022-04-26 08:03:08\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```ca5f3021```](https://github.com/jina-ai/clip-as-service/commit/ca5f30211d87fc324e9cfffb3fcc89682a233ba8)] __-__ __helper__: add version check for client and server (#696) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```234650f4```](https://github.com/jina-ai/clip-as-service/commit/234650f48a1e59cd274748f98ae84f6648b811af)] __-__ __version__: the next version will be 0.3.1 (*Jina Dev Bot*)\n - [[```8600286c```](https://github.com/jina-ai/clip-as-service/commit/8600286cf53755c127cf258af918b6bdf3e86691)] __-__ update readme (*Han Xiao*)\n\n\n## Release Note (`0.3.2`)\n\n> Release time: 2022-04-26 09:16:04\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```f5ba35ab```](https://github.com/jina-ai/clip-as-service/commit/f5ba35abf37a5d140f2e8491cb0dcab13c25869f)] __-__ __helper__: add version check for client and server (*Han Xiao*)\n - [[```ca5f3021```](https://github.com/jina-ai/clip-as-service/commit/ca5f30211d87fc324e9cfffb3fcc89682a233ba8)] __-__ __helper__: add version check for client and server (#696) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```27ffd856```](https://github.com/jina-ai/clip-as-service/commit/27ffd85623407033de72ad259a725848b3412822)] __-__ __version__: the next version will be 0.3.2 (*Jina Dev Bot*)\n\n\n## Release Note (`0.3.3`)\n\n> Release time: 2022-04-26 09:38:17\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```076d6537```](https://github.com/jina-ai/clip-as-service/commit/076d65378b3653dc9f1213f398d1d70824e67513)] __-__ __helper__: add version check for client and server (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```9bcbb1f9```](https://github.com/jina-ai/clip-as-service/commit/9bcbb1f9147aa3dda857a5a130b531e88f27baf2)] __-__ __version__: the next version will be 0.3.3 (*Jina Dev Bot*)\n\n\n## Release Note (`0.3.4`)\n\n> Release time: 2022-04-30 15:17:02\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```076d6537```](https://github.com/jina-ai/clip-as-service/commit/076d65378b3653dc9f1213f398d1d70824e67513)] __-__ __helper__: add version check for client and server (*Han Xiao*)\n\n### ๐Ÿž Bug fixes\n\n - [[```8ac2e9bb```](https://github.com/jina-ai/clip-as-service/commit/8ac2e9bb68b96d1421f7e2ae6b01cec95aad3183)] __-__ __torch__: fix oom in rerank endpoint (#699) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```dd508167```](https://github.com/jina-ai/clip-as-service/commit/dd5081672718a12e7aeede0400432e8f6a01a744)] __-__ __version__: the next version will be 0.3.4 (*Jina Dev Bot*)\n\n\n## Release Note (`0.3.5`)\n\n> Release time: 2022-04-30 18:55:10\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```8ac2e9bb```](https://github.com/jina-ai/clip-as-service/commit/8ac2e9bb68b96d1421f7e2ae6b01cec95aad3183)] __-__ __torch__: fix oom in rerank endpoint (#699) (*Han Xiao*)\n\n### ๐Ÿงผ Code Refactoring\n\n - [[```050c34e0```](https://github.com/jina-ai/clip-as-service/commit/050c34e0906f9593aee054ac37a0a830478a3a3b)] __-__ use packaging instead of distutil (#700) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```d2c2c872```](https://github.com/jina-ai/clip-as-service/commit/d2c2c8729e0872b0c5e299916ae8cf58be7ec516)] __-__ __version__: the next version will be 0.3.5 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.0`)\n\n> Release time: 2022-04-30 20:25:29\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```33efcb00```](https://github.com/jina-ai/clip-as-service/commit/33efcb00414f6cbf5f860bafe7cc183773e08241)] __-__ add async rerank (#701) (*Han Xiao*)\n - [[```12d33c49```](https://github.com/jina-ai/clip-as-service/commit/12d33c49d5ac4d55a7351a442335f25218661dc8)] __-__ add async rerank (*Han Xiao*)\n\n### ๐Ÿงผ Code Refactoring\n\n - [[```050c34e0```](https://github.com/jina-ai/clip-as-service/commit/050c34e0906f9593aee054ac37a0a830478a3a3b)] __-__ use packaging instead of distutil (#700) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```20e66b95```](https://github.com/jina-ai/clip-as-service/commit/20e66b953af17480e062a8e84719b5a6823ba648)] __-__ __version__: the next version will be 0.3.6 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.1`)\n\n> Release time: 2022-05-04 17:38:48\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```f66b145b```](https://github.com/jina-ai/clip-as-service/commit/f66b145be9a19b64e6404665fce8ff5c14b9552b)] __-__ add ranker endpoint for all backends (#707) (*felix-wang*)\n - [[```f7b9af40```](https://github.com/jina-ai/clip-as-service/commit/f7b9af40c3bb693ca70faddfcbc49d07d287df62)] __-__ add tensorrt support (#688) (*felix-wang*)\n - [[```33efcb00```](https://github.com/jina-ai/clip-as-service/commit/33efcb00414f6cbf5f860bafe7cc183773e08241)] __-__ add async rerank (#701) (*Han Xiao*)\n\n### ๐Ÿž Bug fixes\n\n - [[```618dbdb2```](https://github.com/jina-ai/clip-as-service/commit/618dbdb2cfe7a765c3748708bf6b16560175ca51)] __-__ cd workflow (#706) (*felix-wang*)\n\n### ๐Ÿน Other Improvements\n\n - [[```3f34d46d```](https://github.com/jina-ai/clip-as-service/commit/3f34d46d662998ae39f7bfe50aface9a9582bb0c)] __-__ __docs__: add cas async usage to readme (*Han Xiao*)\n - [[```0f941660```](https://github.com/jina-ai/clip-as-service/commit/0f941660a78879a8eea5846ecb4f7c2dabf0f34c)] __-__ __version__: the next version will be 0.4.1 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.2`)\n\n> Release time: 2022-05-09 05:32:39\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```f66b145b```](https://github.com/jina-ai/clip-as-service/commit/f66b145be9a19b64e6404665fce8ff5c14b9552b)] __-__ add ranker endpoint for all backends (#707) (*felix-wang*)\n\n### ๐Ÿž Bug fixes\n\n - [[```835eb13f```](https://github.com/jina-ai/clip-as-service/commit/835eb13fcb84d126b6a35e18b2fe0ef9d8b835b7)] __-__ use cosine as the rank score (#708) (*felix-wang*)\n\n### ๐Ÿน Other Improvements\n\n - [[```706fa624```](https://github.com/jina-ai/clip-as-service/commit/706fa624cb567857e6bc024f52c93a10ab410651)] __-__ __docs__: update readme (*Han Xiao*)\n - [[```7fd04d2d```](https://github.com/jina-ai/clip-as-service/commit/7fd04d2d65dc033ccdec3b970046692816a84880)] __-__ __docs__: add cas async usage to readme (*Han Xiao*)\n - [[```90bb4c5c```](https://github.com/jina-ai/clip-as-service/commit/90bb4c5c70c5e51a1c29bf5aed2906d284c077f7)] __-__ __version__: the next version will be 0.4.2 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.3`)\n\n> Release time: 2022-05-09 10:23:15\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, Roshan Jossy, Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```bb520d14```](https://github.com/jina-ai/clip-as-service/commit/bb520d14b6c5172fce9a971b51c4125b60418119)] __-__ keep logit_scale on same device (#710) (*felix-wang*)\n - [[```835eb13f```](https://github.com/jina-ai/clip-as-service/commit/835eb13fcb84d126b6a35e18b2fe0ef9d8b835b7)] __-__ use cosine as the rank score (#708) (*felix-wang*)\n\n### ๐Ÿ“— Documentation\n\n - [[```da87d13a```](https://github.com/jina-ai/clip-as-service/commit/da87d13a753cdb394fe24a00af7e02f0dcc5fa00)] __-__ __tracking__: update external links' source (#711) (*Roshan Jossy*)\n\n### ๐Ÿน Other Improvements\n\n - [[```099e2218```](https://github.com/jina-ai/clip-as-service/commit/099e2218a47bacb73afed2a7739b58f4e56c7c68)] __-__ __docs__: update readme (*Han Xiao*)\n - [[```ce5806d3```](https://github.com/jina-ai/clip-as-service/commit/ce5806d33ccad1ef83106900b44da3e85eb0ce64)] __-__ update index html (#709) (*felix-wang*)\n - [[```a1651079```](https://github.com/jina-ai/clip-as-service/commit/a16510799b648beb29bd422cdddc1e5dee5db061)] __-__ __version__: the next version will be 0.4.3 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.4`)\n\n> Release time: 2022-05-11 12:00:46\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, felix-wang, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```edf0d862```](https://github.com/jina-ai/clip-as-service/commit/edf0d862228fb69ed7aa25e6ac71a322494c3c29)] __-__ add dockerfiles and cd workflow (#712) (*felix-wang*)\n\n### ๐Ÿž Bug fixes\n\n - [[```bb520d14```](https://github.com/jina-ai/clip-as-service/commit/bb520d14b6c5172fce9a971b51c4125b60418119)] __-__ keep logit_scale on same device (#710) (*felix-wang*)\n\n### ๐Ÿงผ Code Refactoring\n\n - [[```59c06986```](https://github.com/jina-ai/clip-as-service/commit/59c06986387b59c0f536af08d0f6abed71cd7a41)] __-__ __server__: remove redundant logics of rank (#715) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```72d69c75```](https://github.com/jina-ai/clip-as-service/commit/72d69c75be20cc8126f9c406cb621d349b87ba3c)] __-__ __docs__: update readme (*Han Xiao*)\n - [[```f898c8ce```](https://github.com/jina-ai/clip-as-service/commit/f898c8ce73ea3884037b900ca45fda4a477efdce)] __-__ __version__: the next version will be 0.4.4 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.5`)\n\n> Release time: 2022-05-11 12:10:29\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```6ed4c484```](https://github.com/jina-ai/clip-as-service/commit/6ed4c484346e16846e0076d7bf99388c858581be)] __-__ convert distance to score (*Han Xiao*)\n\n### ๐Ÿงผ Code Refactoring\n\n - [[```59c06986```](https://github.com/jina-ai/clip-as-service/commit/59c06986387b59c0f536af08d0f6abed71cd7a41)] __-__ __server__: remove redundant logics of rank (#715) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```d565d31f```](https://github.com/jina-ai/clip-as-service/commit/d565d31f80e4a289529477159eba07161c6c6066)] __-__ __version__: the next version will be 0.4.5 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.6`)\n\n> Release time: 2022-05-11 15:10:52\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### โšก Performance Improvements\n\n - [[```cda93fdd```](https://github.com/jina-ai/clip-as-service/commit/cda93fdd648a64f16bd9f194079e3e9220629af2)] __-__ __server__: use await gather in rank function (*Han Xiao*)\n\n### ๐Ÿž Bug fixes\n\n - [[```6ed4c484```](https://github.com/jina-ai/clip-as-service/commit/6ed4c484346e16846e0076d7bf99388c858581be)] __-__ convert distance to score (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```06fcd07b```](https://github.com/jina-ai/clip-as-service/commit/06fcd07bcf208753de15f051339fd48a0e8186f9)] __-__ __version__: the next version will be 0.4.6 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.7`)\n\n> Release time: 2022-05-11 16:25:08\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### โšก Performance Improvements\n\n - [[```72f1bc4a```](https://github.com/jina-ai/clip-as-service/commit/72f1bc4af0bc6ad01e645a889fd8ee505f986b42)] __-__ __server__: use await gather in rank function (#716) (*Han Xiao*)\n - [[```cda93fdd```](https://github.com/jina-ai/clip-as-service/commit/cda93fdd648a64f16bd9f194079e3e9220629af2)] __-__ __server__: use await gather in rank function (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```66b14fc6```](https://github.com/jina-ai/clip-as-service/commit/66b14fc6f7e8abd3322d0a9e8e0bdfe942d187d3)] __-__ __version__: the next version will be 0.4.7 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.8`)\n\n> Release time: 2022-05-13 09:24:42\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, numb3r3, felix-wang, Jina Dev Bot, ๐Ÿ™‡\n\n\n### โšก Performance Improvements\n\n - [[```72f1bc4a```](https://github.com/jina-ai/clip-as-service/commit/72f1bc4af0bc6ad01e645a889fd8ee505f986b42)] __-__ __server__: use await gather in rank function (#716) (*Han Xiao*)\n\n### ๐Ÿž Bug fixes\n\n - [[```65991a3f```](https://github.com/jina-ai/clip-as-service/commit/65991a3f9126b19c99f21e44cd3dd4227cbe80c7)] __-__ __client__: fix https args to tls (#722) (*Han Xiao*)\n - [[```1002a913```](https://github.com/jina-ai/clip-as-service/commit/1002a9132120dbf52a0dd4700c740692c959a422)] __-__ docker release cd (#717) (*felix-wang*)\n - [[```71d2c867```](https://github.com/jina-ai/clip-as-service/commit/71d2c867b5d45c8dfe42872b7e5697793be79f4b)] __-__ docker build push (#714) (*felix-wang*)\n\n### ๐Ÿ Unit Test and CICD\n\n - [[```38043676```](https://github.com/jina-ai/clip-as-service/commit/3804367632cccdc1f7fa1f5fc998f3530e1ee05c)] __-__ fix force release (*numb3r3*)\n\n### ๐Ÿน Other Improvements\n\n - [[```0da311e4```](https://github.com/jina-ai/clip-as-service/commit/0da311e4113b0afb60cb16779d86c48399c86f24)] __-__ __docs__: change http to https (*Han Xiao*)\n - [[```741ad796```](https://github.com/jina-ai/clip-as-service/commit/741ad796be93df808a634a41105f2860751afc2d)] __-__ __docs__: add playground (*Han Xiao*)\n - [[```a2b6d337```](https://github.com/jina-ai/clip-as-service/commit/a2b6d33738f3cea55e429db21691534c62c08320)] __-__ __version__: the next version will be 0.4.8 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.9`)\n\n> Release time: 2022-05-23 15:13:23\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, numb3r3, felix-wang, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```a7311fbf```](https://github.com/jina-ai/clip-as-service/commit/a7311fbf6ae6e988b78924fea239a9c8236dd8ff)] __-__ __server__: recover original contents of the input da (#726) (*Han Xiao*)\n - [[```42ef75b1```](https://github.com/jina-ai/clip-as-service/commit/42ef75b185369c1ffd64f8ea5a7b0fca8c8656b2)] __-__ __server__: remove embeddings to save bandwidth (*Han Xiao*)\n - [[```2d2da147```](https://github.com/jina-ai/clip-as-service/commit/2d2da147c3781233dc3812e2e7dfebbcbeb5f20e)] __-__ docker push cd (*numb3r3*)\n - [[```994635fa```](https://github.com/jina-ai/clip-as-service/commit/994635fabc84293b05adff123263a0d276812202)] __-__ k8s dockerize (#725) (*felix-wang*)\n - [[```d12c5115```](https://github.com/jina-ai/clip-as-service/commit/d12c5115c946497b4ddf03a759a63c4040bcf8c7)] __-__ docker file (#719) (*felix-wang*)\n - [[```65991a3f```](https://github.com/jina-ai/clip-as-service/commit/65991a3f9126b19c99f21e44cd3dd4227cbe80c7)] __-__ __client__: fix https args to tls (#722) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```b6adcf8b```](https://github.com/jina-ai/clip-as-service/commit/b6adcf8be6a087d446e833478a0ac05a7900c24b)] __-__ __docs__: add multi gpu setting (*Han Xiao*)\n - [[```3d8c552a```](https://github.com/jina-ai/clip-as-service/commit/3d8c552a721a52fb374f73fcc9725f7d06e2383f)] __-__ __version__: the next version will be 0.4.9 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.10`)\n\n> Release time: 2022-05-24 07:46:48\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```0054b47c```](https://github.com/jina-ai/clip-as-service/commit/0054b47cf12043b0bf493424ec22defa9448a9be)] __-__ __server__: fix content assignment (#727) (*Han Xiao*)\n - [[```a7311fbf```](https://github.com/jina-ai/clip-as-service/commit/a7311fbf6ae6e988b78924fea239a9c8236dd8ff)] __-__ __server__: recover original contents of the input da (#726) (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```926621bc```](https://github.com/jina-ai/clip-as-service/commit/926621bc97972694caff79700c5b70031a2677c1)] __-__ __version__: the next version will be 0.4.10 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.11`)\n\n> Release time: 2022-05-27 07:44:46\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n samsja, Shubham Goel, Han Xiao, Ziniu Yu, Roshan Jossy, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```60a986a0```](https://github.com/jina-ai/clip-as-service/commit/60a986a07921b2374fdc64dccde7e1ec1e728cdb)] __-__ add monitoring (#674) (*samsja*)\n\n### ๐Ÿž Bug fixes\n\n - [[```59f48e60```](https://github.com/jina-ai/clip-as-service/commit/59f48e60a286344f66f37aea46b0b1b148ca90f4)] __-__ windows file name conflict (#729) (*Ziniu Yu*)\n - [[```0054b47c```](https://github.com/jina-ai/clip-as-service/commit/0054b47cf12043b0bf493424ec22defa9448a9be)] __-__ __server__: fix content assignment (#727) (*Han Xiao*)\n\n### ๐Ÿ“— Documentation\n\n - [[```2f3a2077```](https://github.com/jina-ai/clip-as-service/commit/2f3a207734c829e6baa94dfa5715dc8d5c3f12de)] __-__ __tracking__: remove utm source in links (#728) (*Roshan Jossy*)\n\n### ๐Ÿน Other Improvements\n\n - [[```c7c96251```](https://github.com/jina-ai/clip-as-service/commit/c7c9625163d97ca3a6ad2b845309bad9e34e5d87)] __-__ Corrected replicas indentation in server.md (#731) (*Shubham Goel*)\n - [[```8d112275```](https://github.com/jina-ai/clip-as-service/commit/8d1122754dc53c75bce75e313ee74796bd9614e7)] __-__ fix docs (*Han Xiao*)\n - [[```7323d99e```](https://github.com/jina-ai/clip-as-service/commit/7323d99edd7814ece9ed8f5c0adce949047c987d)] __-__ __version__: the next version will be 0.4.11 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.12`)\n\n> Release time: 2022-06-01 08:28:41\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, Ziniu Yu, Jina Dev Bot, samsja, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```60a986a0```](https://github.com/jina-ai/clip-as-service/commit/60a986a07921b2374fdc64dccde7e1ec1e728cdb)] __-__ add monitoring (#674) (*samsja*)\n\n### ๐Ÿž Bug fixes\n\n - [[```bb8c4ce0```](https://github.com/jina-ai/clip-as-service/commit/bb8c4ce01de76d2be63444a517e90b530422110e)] __-__ better monitoring (#738) (*felix-wang*)\n - [[```751cf9de```](https://github.com/jina-ai/clip-as-service/commit/751cf9de0d8bb727c44ecaf7950a3658b868399d)] __-__ does not require port (#735) (*Ziniu Yu*)\n\n### ๐Ÿ“— Documentation\n\n - [[```5e06667a```](https://github.com/jina-ai/clip-as-service/commit/5e06667ac9afef335b98b72a58ba0d28985d9b18)] __-__ update monitoring feature (#737) (*felix-wang*)\n\n### ๐Ÿน Other Improvements\n\n - [[```b523c624```](https://github.com/jina-ai/clip-as-service/commit/b523c62468dd6088095b00b5335160c57a1cb25e)] __-__ __version__: the next version will be 0.4.12 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.13`)\n\n> Release time: 2022-06-09 04:42:07\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, Ziniu Yu, Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```d675148b```](https://github.com/jina-ai/clip-as-service/commit/d675148b4305338e9d17d449e42ab7c142896c06)] __-__ add clip_hg executor (#740) (*Ziniu Yu*)\n\n### ๐Ÿงผ Code Refactoring\n\n - [[```5eb5d7e8```](https://github.com/jina-ai/clip-as-service/commit/5eb5d7e8ed6f924c6560bb850edb08bbd809ff09)] __-__ monitor (#743) (*felix-wang*)\n\n### ๐Ÿ“— Documentation\n\n - [[```130108c1```](https://github.com/jina-ai/clip-as-service/commit/130108c1aaf993bebc8215527c47d98c7e2169c5)] __-__ add JCloud deployment docs (#739) (*Ziniu Yu*)\n - [[```5e06667a```](https://github.com/jina-ai/clip-as-service/commit/5e06667ac9afef335b98b72a58ba0d28985d9b18)] __-__ update monitoring feature (#737) (*felix-wang*)\n\n### ๐Ÿน Other Improvements\n\n - [[```4b88e992```](https://github.com/jina-ai/clip-as-service/commit/4b88e99263a29903312f52bae01465b44b7a0cce)] __-__ fix docs (*Han Xiao*)\n - [[```b130d645```](https://github.com/jina-ai/clip-as-service/commit/b130d645409b044df6f1e0bcda78b42e79cb98d9)] __-__ add grafana dashboard (#741) (*felix-wang*)\n - [[```12ede839```](https://github.com/jina-ai/clip-as-service/commit/12ede83996f62af8c549a1d6621ae1dd32b7de7d)] __-__ __version__: the next version will be 0.4.13 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.14`)\n\n> Release time: 2022-06-09 13:39:46\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```752202f8```](https://github.com/jina-ai/clip-as-service/commit/752202f8b730d0ab8785a703fc719dacfbe2993b)] __-__ monitor documentation (#745) (*felix-wang*)\n\n### ๐Ÿงผ Code Refactoring\n\n - [[```5eb5d7e8```](https://github.com/jina-ai/clip-as-service/commit/5eb5d7e8ed6f924c6560bb850edb08bbd809ff09)] __-__ monitor (#743) (*felix-wang*)\n\n### ๐Ÿน Other Improvements\n\n - [[```06097f20```](https://github.com/jina-ai/clip-as-service/commit/06097f2098190b5a8a40fc82354b642730e617e0)] __-__ __version__: the next version will be 0.4.14 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.15`)\n\n> Release time: 2022-06-13 13:06:16\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, felix-wang, Ziniu Yu, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```e022bd46```](https://github.com/jina-ai/clip-as-service/commit/e022bd46c8c1773620f635148cb999e23ff7167e)] __-__ add traversal paths (#750) (*felix-wang*)\n - [[```4fe5a1b1```](https://github.com/jina-ai/clip-as-service/commit/4fe5a1b1dc9672be98638ba57023936b5ed69a6c)] __-__ add traversal paths (#748) (*felix-wang*)\n\n### ๐Ÿž Bug fixes\n\n - [[```752202f8```](https://github.com/jina-ai/clip-as-service/commit/752202f8b730d0ab8785a703fc719dacfbe2993b)] __-__ monitor documentation (#745) (*felix-wang*)\n\n### ๐Ÿน Other Improvements\n\n - [[```dab8341e```](https://github.com/jina-ai/clip-as-service/commit/dab8341e9ffb0716eb8c17477534ef91f19d8c5d)] __-__ add cas on colab section (*Han Xiao*)\n - [[```29bd68a4```](https://github.com/jina-ai/clip-as-service/commit/29bd68a4bc1f17c34016d45901858c65c8cf5623)] __-__ add replicas field in all yamls (*Han Xiao*)\n - [[```d5be8c2f```](https://github.com/jina-ai/clip-as-service/commit/d5be8c2f85e47fcafa7587f3e75b76fbc42300e5)] __-__ Revert "feat: add traversal paths (#748)" (#749) (*Han Xiao*)\n - [[```7f2d8fe8```](https://github.com/jina-ai/clip-as-service/commit/7f2d8fe88643ae71e5d8b38547faa32570886e46)] __-__ update links in docs (#747) (*Ziniu Yu*)\n - [[```52a8b0a6```](https://github.com/jina-ai/clip-as-service/commit/52a8b0a6c62204d37556f31fd79fd1ee621b45e3)] __-__ __version__: the next version will be 0.4.15 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.16`)\n\n> Release time: 2022-06-14 08:52:07\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, Ziniu Yu, Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```eca1e700```](https://github.com/jina-ai/clip-as-service/commit/eca1e700493d59f475714aa5f49ccd33247cb983)] __-__ add integerate test for client (#753) (*felix-wang*)\n - [[```b5c339fe```](https://github.com/jina-ai/clip-as-service/commit/b5c339feda8ed89538b92d59624a781a8725f304)] __-__ fix client concurrent issue (#752) (*Ziniu Yu*)\n\n### ๐Ÿน Other Improvements\n\n - [[```e5ab22f5```](https://github.com/jina-ai/clip-as-service/commit/e5ab22f58ea8888aef0ead6902d2412301e9e5fc)] __-__ update slack (*Han Xiao*)\n - [[```5503becb```](https://github.com/jina-ai/clip-as-service/commit/5503becb5308ca34062bc611a4a431815de5383c)] __-__ fix docs (*Han Xiao*)\n - [[```909cdb11```](https://github.com/jina-ai/clip-as-service/commit/909cdb110ff27f811e5ed718bb99291f454af03a)] __-__ add cas on colab section (*Han Xiao*)\n - [[```3d3ef936```](https://github.com/jina-ai/clip-as-service/commit/3d3ef9363c9a322660fe84e94a2e610a24be0f0e)] __-__ __version__: the next version will be 0.4.16 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.17`)\n\n> Release time: 2022-06-20 10:56:12\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Ziniu Yu, numb3r3, felix-wang, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```03541dd7```](https://github.com/jina-ai/clip-as-service/commit/03541dd765849ec453b501c83dbf4071b317bce1)] __-__ add cas server dockerfile (#757) (*Han Xiao*)\n - [[```4d069a84```](https://github.com/jina-ai/clip-as-service/commit/4d069a84ac0414059acce322f00815bf0cd12536)] __-__ upload torch executor (#723) (*Ziniu Yu*)\n\n### ๐Ÿž Bug fixes\n\n - [[```eca1e700```](https://github.com/jina-ai/clip-as-service/commit/eca1e700493d59f475714aa5f49ccd33247cb983)] __-__ add integerate test for client (#753) (*felix-wang*)\n\n### ๐Ÿ“— Documentation\n\n - [[```7c2faae2```](https://github.com/jina-ai/clip-as-service/commit/7c2faae270e276bfc36f4c51e4abe101194f1799)] __-__ update jcloud docs (#754) (*Ziniu Yu*)\n - [[```9d872f2e```](https://github.com/jina-ai/clip-as-service/commit/9d872f2e20e53e988a6d13dff191a42fa6e7e0d2)] __-__ add disk usage / memory usage benchmark table (#751) (*Ziniu Yu*)\n\n### ๐Ÿน Other Improvements\n\n - [[```9e469bf7```](https://github.com/jina-ai/clip-as-service/commit/9e469bf70f4cd314353bde9c1ca8dfbda45fa532)] __-__ fix readme (*Han Xiao*)\n - [[```4c4e74b2```](https://github.com/jina-ai/clip-as-service/commit/4c4e74b2d5ebcede408d49b6455e1a13293edf95)] __-__ upload executor in cd workflow (*numb3r3*)\n - [[```96923f12```](https://github.com/jina-ai/clip-as-service/commit/96923f12a33e2c30dc55dc993648d9758f96a132)] __-__ fix docker cd (#755) (*felix-wang*)\n - [[```1869e61f```](https://github.com/jina-ai/clip-as-service/commit/1869e61f3e0c46a7322abc42be6983e951d5806d)] __-__ add visual reasoning to docs (*Han Xiao*)\n - [[```2083f097```](https://github.com/jina-ai/clip-as-service/commit/2083f0970985a2260a7b6fbbaaaa8b1210036765)] __-__ __version__: the next version will be 0.4.17 (*Jina Dev Bot*)\n\n\n## Release Note (`0.4.18`)\n\n> Release time: 2022-06-20 11:21:16\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿน Other Improvements\n\n - [[```a0c2661b```](https://github.com/jina-ai/clip-as-service/commit/a0c2661bc4764a74ff8737744b0d47fac4c1a5e9)] __-__ fix tag docker build job (*Han Xiao*)\n - [[```23f738ec```](https://github.com/jina-ai/clip-as-service/commit/23f738ecabebf906d001f83481f8cd10b89f5fb0)] __-__ __version__: the next version will be 0.4.18 (*Jina Dev Bot*)\n - [[```9e469bf7```](https://github.com/jina-ai/clip-as-service/commit/9e469bf70f4cd314353bde9c1ca8dfbda45fa532)] __-__ fix readme (*Han Xiao*)\n\n\n## Release Note (`0.4.19`)\n\n> Release time: 2022-06-20 16:32:32\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```6902d2df```](https://github.com/jina-ai/clip-as-service/commit/6902d2dffc04b57e7a49f308139b27775a2193fa)] __-__ read config from stdin to allow pipe (#758) (*Han Xiao*)\n\n### ๐Ÿ“— Documentation\n\n - [[```6e054db8```](https://github.com/jina-ai/clip-as-service/commit/6e054db893fcff4a2fe6c86073dd049e1c13f954)] __-__ read config from stdin to allow pipe (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```4a298d4f```](https://github.com/jina-ai/clip-as-service/commit/4a298d4f9fcbe342855f234d59c8e920e6918659)] __-__ add docker image docs (*Han Xiao*)\n - [[```1e931e8b```](https://github.com/jina-ai/clip-as-service/commit/1e931e8b2d2d8e5429c69e25df95ab15cb84ab66)] __-__ __version__: the next version will be 0.4.19 (*Jina Dev Bot*)\n - [[```a0c2661b```](https://github.com/jina-ai/clip-as-service/commit/a0c2661bc4764a74ff8737744b0d47fac4c1a5e9)] __-__ fix tag docker build job (*Han Xiao*)\n\n\n## Release Note (`0.4.20`)\n\n> Release time: 2022-06-21 15:45:06\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Han Xiao, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```79e85eed```](https://github.com/jina-ai/clip-as-service/commit/79e85eed7c89f31c16399bfcc1bb098f0ae5c920)] __-__ miscalling clip_server in clip_client (*Han Xiao*)\n\n### ๐Ÿ“— Documentation\n\n - [[```6e054db8```](https://github.com/jina-ai/clip-as-service/commit/6e054db893fcff4a2fe6c86073dd049e1c13f954)] __-__ read config from stdin to allow pipe (*Han Xiao*)\n\n### ๐Ÿน Other Improvements\n\n - [[```c3e75133```](https://github.com/jina-ai/clip-as-service/commit/c3e751336722b415aa88992794119f32b7ddee77)] __-__ __version__: the next version will be 0.4.20 (*Jina Dev Bot*)\n\n\n## Release Note (`0.5.0`)\n\n> Release time: 2022-08-03 05:13:06\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n numb3r3, Ziniu Yu, Alex Shan, felix-wang, Sha Zhou, Jina Dev Bot, Han Xiao, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```3402b1d1```](https://github.com/jina-ai/clip-as-service/commit/3402b1d1726120d8ed39ae561e441695f24ddeb3)] __-__ replace traversal_paths with access_paths (#791) (*Ziniu Yu*)\n - [[```87928a7b```](https://github.com/jina-ai/clip-as-service/commit/87928a7b8be9e8a4fce4d2352e82975252db162b)] __-__ update onnx models and md5 (#785) (*Ziniu Yu*)\n - [[```8bd83896```](https://github.com/jina-ai/clip-as-service/commit/8bd838964b7975c9c1a2394c0ae681507ed5dc18)] __-__ support onnx backend for openclip (#781) (*felix-wang*)\n - [[```f043b4d9```](https://github.com/jina-ai/clip-as-service/commit/f043b4d934a9454b5db32e7ea7331307506a1a6f)] __-__ update openclip loader (#782) (*Alex Shan*)\n - [[```fa62d8e9```](https://github.com/jina-ai/clip-as-service/commit/fa62d8e93baf2579b2934cc0ed8daca12c144d7d)] __-__ support openclip&mclip models + refactor model loader (#774) (*Alex Shan*)\n - [[```32b11cd6```](https://github.com/jina-ai/clip-as-service/commit/32b11cd64bb76bca5075fbcbc84b9334952c236c)] __-__ allow model selection in client (#775) (*Ziniu Yu*)\n - [[```0ff4e252```](https://github.com/jina-ai/clip-as-service/commit/0ff4e2526394e0fa86266668f1162f4a6b922bd8)] __-__ allow credential in client (#765) (*Ziniu Yu*)\n - [[```ee7da10d```](https://github.com/jina-ai/clip-as-service/commit/ee7da10d1f56a130e6f9a85d5fb3518b80e5df0d)] __-__ support custom onnx file and update model signatures (#761) (*Ziniu Yu*)\n - [[```ed1b92d1```](https://github.com/jina-ai/clip-as-service/commit/ed1b92d1896cc0c12733b51bd1bd83040676f505)] __-__ __docs__: add qabot (#759) (*Sha Zhou*)\n\n### ๐Ÿž Bug fixes\n\n - [[```e48a7a38```](https://github.com/jina-ai/clip-as-service/commit/e48a7a38ac01fe0db47a7898ae1401f25394402f)] __-__ change onnx and trt default model name to ViT-B-32::openai (#793) (*Ziniu Yu*)\n - [[```8b8082a9```](https://github.com/jina-ai/clip-as-service/commit/8b8082a939f67f7ea01cc9f55ebce9c5368ebe1a)] __-__ mclip cuda device (#792) (*felix-wang*)\n - [[```8681b88e```](https://github.com/jina-ai/clip-as-service/commit/8681b88eb3a7806c1286eaefff3bd8a8ab28ff03)] __-__ fp16 inference (#790) (*felix-wang*)\n - [[```ab00c2ae```](https://github.com/jina-ai/clip-as-service/commit/ab00c2ae4067678b8f9c8351244867257031f3c2)] __-__ upgrade jina (#788) (*felix-wang*)\n - [[```1db43b48```](https://github.com/jina-ai/clip-as-service/commit/1db43b485b0fe368eb3949ddc052b5dd8002c279)] __-__ no allow client to change server batch size (#787) (*Ziniu Yu*)\n - [[```58772079```](https://github.com/jina-ai/clip-as-service/commit/5877207924c088739644873d6cf654aabb1f7134)] __-__ add models and md5 (#783) (*Ziniu Yu*)\n - [[```7c8285bb```](https://github.com/jina-ai/clip-as-service/commit/7c8285bbf7eb5d757cba1f85b56e6528be66396b)] __-__ async progress bar does not display (#779) (*Ziniu Yu*)\n - [[```79e85eed```](https://github.com/jina-ai/clip-as-service/commit/79e85eed7c89f31c16399bfcc1bb098f0ae5c920)] __-__ miscalling clip_server in clip_client (*Han Xiao*)\n\n### ๐Ÿ“— Documentation\n\n - [[```c67a7f59```](https://github.com/jina-ai/clip-as-service/commit/c67a7f59c25760e32a611b330fd9ff5959aa1e4b)] __-__ add model support (#784) (*Alex Shan*)\n - [[```bc6b72e6```](https://github.com/jina-ai/clip-as-service/commit/bc6b72e65cce999ad7b09ecb93b25b07ff8f4de1)] __-__ add finetuner docs (#771) (*Ziniu Yu*)\n - [[```2b78b12e```](https://github.com/jina-ai/clip-as-service/commit/2b78b12e3aa527b386eac4ee7eed74e580eadbf6)] __-__ improve model support (#768) (*Ziniu Yu*)\n\n### ๐Ÿน Other Improvements\n\n - [[```b00963c4```](https://github.com/jina-ai/clip-as-service/commit/b00963c45983dfdac6d05258b03298de5ad1edf6)] __-__ bump version to 0.5.0 (*numb3r3*)\n - [[```c458dd65```](https://github.com/jina-ai/clip-as-service/commit/c458dd6579d6e3125028ad4cb2b88f9f481b4686)] __-__ remove clip_hg (#786) (*Ziniu Yu*)\n - [[```ca03dca3```](https://github.com/jina-ai/clip-as-service/commit/ca03dca369d2e7ed55d2f2a339fa9b4e9f41667d)] __-__ fix markdown-table extention (#772) (*felix-wang*)\n - [[```7b19bffe```](https://github.com/jina-ai/clip-as-service/commit/7b19bffecb739a74a524544472aa3ad07dff2f2a)] __-__ __version__: the next version will be 0.4.21 (*Jina Dev Bot*)\n\n\n## Release Note (`0.5.1`)\n\n> Release time: 2022-08-08 05:11:18\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Ziniu Yu, Jina Dev Bot, numb3r3, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```65032f02```](https://github.com/jina-ai/clip-as-service/commit/65032f02db30671f7a2a6ca78e371588ae98ab2b)] __-__ encode text first when both text and uri are presented (#795) (*Ziniu Yu*)\n\n### ๐Ÿ“— Documentation\n\n - [[```7c6708fa```](https://github.com/jina-ai/clip-as-service/commit/7c6708fa8a592b5ce306f1ab2f1af1504148484a)] __-__ update hub readme (#794) (*Ziniu Yu*)\n\n### ๐Ÿน Other Improvements\n\n - [[```a7c4f490```](https://github.com/jina-ai/clip-as-service/commit/a7c4f4903df5736bcf9e85d82bb83497d850bc4d)] __-__ __version__: the next version will be 0.5.1 (*Jina Dev Bot*)\n - [[```b00963c4```](https://github.com/jina-ai/clip-as-service/commit/b00963c45983dfdac6d05258b03298de5ad1edf6)] __-__ bump version to 0.5.0 (*numb3r3*)\n\n\n## Release Note (`0.6.0`)\n\n> Release time: 2022-08-30 04:19:21\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n numb3r3, Ziniu Yu, felix-wang, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```3c43eed3```](https://github.com/jina-ai/clip-as-service/commit/3c43eed38afe2ff84c8b06368f4301afcd332cf5)] __-__ do not send blob from server when it is loaded in client (#804) (*Ziniu Yu*)\n - [[```f852dfc8```](https://github.com/jina-ai/clip-as-service/commit/f852dfc876caa7b98552b5c707d4c85babc46393)] __-__ add warning if input is too large (#796) (*Ziniu Yu*)\n - [[```65032f02```](https://github.com/jina-ai/clip-as-service/commit/65032f02db30671f7a2a6ca78e371588ae98ab2b)] __-__ encode text first when both text and uri are presented (#795) (*Ziniu Yu*)\n\n### ๐Ÿž Bug fixes\n\n - [[```bb2c142b```](https://github.com/jina-ai/clip-as-service/commit/bb2c142b8899075c00db3b08e506fb970fee1478)] __-__ cast dtype for fp16 (#801) (*felix-wang*)\n\n### ๐Ÿ“— Documentation\n\n - [[```a5893c70```](https://github.com/jina-ai/clip-as-service/commit/a5893c70531830f236d38fde5a880a9a2556474f)] __-__ update jcloud gpu usage (#809) (*Ziniu Yu*)\n - [[```b4fb0dd2```](https://github.com/jina-ai/clip-as-service/commit/b4fb0dd2823b6218da4395989c6b011cf3de1a38)] __-__ fix hub table typo (#803) (*Ziniu Yu*)\n\n### ๐Ÿน Other Improvements\n\n - [[```2a80235c```](https://github.com/jina-ai/clip-as-service/commit/2a80235c0aa16eefdc6703989fc6da670cbd5c89)] __-__ bump version to 0.6.0 (*numb3r3*)\n - [[```59b9f771```](https://github.com/jina-ai/clip-as-service/commit/59b9f7716df9a325fb6e707d086ca6f2612da975)] __-__ update protobuf version (#810) (*Ziniu Yu*)\n - [[```89205f06```](https://github.com/jina-ai/clip-as-service/commit/89205f06d1b740952e79c512d6b0ef6f8db18300)] __-__ update executor docstring (#806) (*Ziniu Yu*)\n - [[```25c91e21```](https://github.com/jina-ai/clip-as-service/commit/25c91e21ee8de9e2cd1766d2c6c319f6e5609e80)] __-__ __version__: the next version will be 0.5.2 (*Jina Dev Bot*)\n\n\n## Release Note (`0.6.1`)\n\n> Release time: 2022-08-30 13:57:32\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n felix-wang, Jina Dev Bot, numb3r3, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```ea239685```](https://github.com/jina-ai/clip-as-service/commit/ea239685bff56372aeadaeb3050f5c2ccc37175f)] __-__ grpc meta auth (#811) (*felix-wang*)\n\n### ๐Ÿน Other Improvements\n\n - [[```83a8120c```](https://github.com/jina-ai/clip-as-service/commit/83a8120c22c76cf34f0d2e5966c368031e0fe9b4)] __-__ __version__: the next version will be 0.6.1 (*Jina Dev Bot*)\n - [[```2a80235c```](https://github.com/jina-ai/clip-as-service/commit/2a80235c0aa16eefdc6703989fc6da670cbd5c89)] __-__ bump version to 0.6.0 (*numb3r3*)\n\n\n## Release Note (`0.6.2`)\n\n> Release time: 2022-09-01 04:16:27\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Ziniu Yu, Jina Dev Bot, felix-wang, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```ea239685```](https://github.com/jina-ai/clip-as-service/commit/ea239685bff56372aeadaeb3050f5c2ccc37175f)] __-__ grpc meta auth (#811) (*felix-wang*)\n\n### ๐Ÿ“— Documentation\n\n - [[```4461d2e9```](https://github.com/jina-ai/clip-as-service/commit/4461d2e9ab07c01669237b220cd24cd6f95e30e8)] __-__ update model support table (#813) (*Ziniu Yu*)\n\n### ๐Ÿน Other Improvements\n\n - [[```f7ee26a1```](https://github.com/jina-ai/clip-as-service/commit/f7ee26a17d47c1de0efc1122ccb40d3b22d217a8)] __-__ improve model not found error msg (#812) (*Ziniu Yu*)\n - [[```f1c0057d```](https://github.com/jina-ai/clip-as-service/commit/f1c0057d7e1c51953303bbf7b3743e19a9c300ab)] __-__ __version__: the next version will be 0.6.2 (*Jina Dev Bot*)\n\n\n## Release Note (`0.7.0`)\n\n> Release time: 2022-09-13 13:47:54\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n numb3r3, felix-wang, Jie Fu, Ziniu Yu, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```a07a5218```](https://github.com/jina-ai/clip-as-service/commit/a07a52182d02b3cab1135235c9aee8e1af4f280c)] __-__ support clip retrieval (#816) (*felix-wang*)\n\n### ๐Ÿž Bug fixes\n\n - [[```213ecc28```](https://github.com/jina-ai/clip-as-service/commit/213ecc28afa20bbb0984efd4ab28dd08443e9369)] __-__ always return docarray as search result (#821) (*felix-wang*)\n - [[```eca57745```](https://github.com/jina-ai/clip-as-service/commit/eca577455a0d378cc4d9974ef3109f2d2e74c1b3)] __-__ __readme__: use new demo server (#819) (*felix-wang*)\n\n### ๐Ÿ“— Documentation\n\n - [[```8d9725fb```](https://github.com/jina-ai/clip-as-service/commit/8d9725fb874d94944cb1129ca2ccc8293c52dc90)] __-__ update clip search (#820) (*felix-wang*)\n - [[```fa7e5776```](https://github.com/jina-ai/clip-as-service/commit/fa7e577606d68e65a0e7952048c64d2b3a28e231)] __-__ docs for retrieval (#808) (*Jie Fu*)\n - [[```47144c23```](https://github.com/jina-ai/clip-as-service/commit/47144c23fd6b10f9aed0dfc4a2e37f83bc33f284)] __-__ enable horizontal scrolling in wide tables (#818) (*Ziniu Yu*)\n\n### ๐Ÿน Other Improvements\n\n - [[```53636cea```](https://github.com/jina-ai/clip-as-service/commit/53636cea63bf8063bcfd744aae4577df8e0eab2e)] __-__ bump version to 0.7.0 (*numb3r3*)\n - [[```eda4aa8e```](https://github.com/jina-ai/clip-as-service/commit/eda4aa8e958bbbd83dddcd5932622bcf041f3918)] __-__ __version__: the next version will be 0.6.3 (*Jina Dev Bot*)\n - [[```f7ee26a1```](https://github.com/jina-ai/clip-as-service/commit/f7ee26a17d47c1de0efc1122ccb40d3b22d217a8)] __-__ improve model not found error msg (#812) (*Ziniu Yu*)\n\n\n## Release Note (`0.8.0`)\n\n> Release time: 2022-10-12 08:11:40\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n numb3r3, Jie Fu, Ziniu Yu, felix-wang, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```2ba8a4fe```](https://github.com/jina-ai/clip-as-service/commit/2ba8a4fe71f26faa5e92d62df04edb616389f6bd)] __-__ support large ONNX model files (#828) (*Ziniu Yu*)\n - [[```09d15485```](https://github.com/jina-ai/clip-as-service/commit/09d15485d50c51a77cb57380f4b848b41764a1b6)] __-__ support B/32, L/14, H/14, and g/14 trained on LAION-2B (#825) (*Ziniu Yu*)\n - [[```c690c247```](https://github.com/jina-ai/clip-as-service/commit/c690c247946017d178d9340d8c951342c0321943)] __-__ drop image content to boost latency (#824) (*felix-wang*)\n - [[```bcce9900```](https://github.com/jina-ai/clip-as-service/commit/bcce990032abfd618cea408ab3f0fb4e352789ae)] __-__ in-place result in clip_client; preserve output order by uid (#815) (*Ziniu Yu*)\n\n### ๐Ÿ“— Documentation\n\n - [[```87fdc548```](https://github.com/jina-ai/clip-as-service/commit/87fdc5489c5b33b76e28dd1c0b54017a51dd4abe)] __-__ add memory profile (#841) (*Jie Fu*)\n - [[```7ee58c8b```](https://github.com/jina-ai/clip-as-service/commit/7ee58c8b2751f949790983f223209ad1d2261fca)] __-__ clip benchmark on zeroshot classification and retrieval tasks (#832) (*Ziniu Yu*)\n\n### ๐Ÿน Other Improvements\n\n - [[```920b3107```](https://github.com/jina-ai/clip-as-service/commit/920b31070f54b1b6af4d4e58e7db351a576e0783)] __-__ bump version to 0.8.0 (*numb3r3*)\n - [[```54e99786```](https://github.com/jina-ai/clip-as-service/commit/54e99786ea07b9ad109f593890e3b4945d39b768)] __-__ add description for retrieval playground (#834) (*Jie Fu*)\n - [[```a26a883f```](https://github.com/jina-ai/clip-as-service/commit/a26a883fa15a47243450c9cebbfc7f472e6cfa04)] __-__ use open clip naming convention for model names (#836) (*Ziniu Yu*)\n - [[```f40513d5```](https://github.com/jina-ai/clip-as-service/commit/f40513d57c0c3f7e466160f41547c970618af85a)] __-__ fix docs website template (#833) (*Ziniu Yu*)\n - [[```d520ebb8```](https://github.com/jina-ai/clip-as-service/commit/d520ebb835e2814f7696148a0dcabbbf8bdadc76)] __-__ remove unused md (*numb3r3*)\n - [[```2c3c61f9```](https://github.com/jina-ai/clip-as-service/commit/2c3c61f9d6f5a351f235dbad45879f0c7c4fd986)] __-__ __version__: the next version will be 0.7.1 (*Jina Dev Bot*)\n - [[```53636cea```](https://github.com/jina-ai/clip-as-service/commit/53636cea63bf8063bcfd744aae4577df8e0eab2e)] __-__ bump version to 0.7.0 (*numb3r3*)\n\n\n## Release Note (`0.8.1`)\n\n> Release time: 2022-11-15 11:15:48\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n YangXiuyu, Ziniu Yu, felix-wang, Jie Fu, Jina Dev Bot, numb3r3, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```e4717a35```](https://github.com/jina-ai/clip-as-service/commit/e4717a35f850e6a2cd8b4d8b4c994fad30fd5c72)] __-__ Integrate flash attention (#853) (*YangXiuyu*)\n - [[```4fcbf68a```](https://github.com/jina-ai/clip-as-service/commit/4fcbf68a883cb3143e47738df4c8044dfec2a131)] __-__ allow custom callback in clip_client (#849) (*Ziniu Yu*)\n\n### ๐Ÿž Bug fixes\n\n - [[```71086227```](https://github.com/jina-ai/clip-as-service/commit/710862279bdef342983bd7944f413d8ee54f9603)] __-__ increase timeout ready for executor docker images (#854) (*Ziniu Yu*)\n - [[```f96ce543```](https://github.com/jina-ai/clip-as-service/commit/f96ce5433dc1ec473ae89e22f01520b93abc6071)] __-__ install transformers for executor docker images (#851) (*Ziniu Yu*)\n\n### ๐Ÿ“— Documentation\n\n - [[```9aa8c224```](https://github.com/jina-ai/clip-as-service/commit/9aa8c224f93c4a7b52fecac8fe8a18832ce98814)] __-__ add tips for client parallelism usage (#846) (*Ziniu Yu*)\n - [[```8776784d```](https://github.com/jina-ai/clip-as-service/commit/8776784d2cf2b0bfce44724db10c18dbda7acb77)] __-__ add instructions for using clip server hosted by jina (#848) (*Ziniu Yu*)\n - [[```d91da50c```](https://github.com/jina-ai/clip-as-service/commit/d91da50cc86942623dbee2cdb6b31350d9ce6a8e)] __-__ move benchmark conclusion to beginning (#847) (*Ziniu Yu*)\n - [[```baf94b5f```](https://github.com/jina-ai/clip-as-service/commit/baf94b5f70b9c18cfe2c0fea3e284fe30e4ca093)] __-__ update finetuner docs (#843) (*Jie Fu*)\n\n### ๐Ÿน Other Improvements\n\n - [[```d2ecec60```](https://github.com/jina-ai/clip-as-service/commit/d2ecec60e9be4235518d19b0e2f2342fa5401dfc)] __-__ allow test to pass even if commit name is not good (#856) (*Ziniu Yu*)\n - [[```ebfa494c```](https://github.com/jina-ai/clip-as-service/commit/ebfa494c9218a848e0bc49a552dabecda1373dbb)] __-__ replace clip server address in docs (#857) (*Ziniu Yu*)\n - [[```fe112ea5```](https://github.com/jina-ai/clip-as-service/commit/fe112ea5ec8dd9de8fd842633b17dcb9079c79a4)] __-__ change hub url from hub.jina.ai to cloud.jina.ai (#845) (*Ziniu Yu*)\n - [[```ae05624d```](https://github.com/jina-ai/clip-as-service/commit/ae05624d68bf8c3fbcefc1d07b0adabbe1cad422)] __-__ use new free service in playground (#844) (*felix-wang*)\n - [[```6cdc3e21```](https://github.com/jina-ai/clip-as-service/commit/6cdc3e21bb6e0b0476b94e40cfa88a475d4a5f7d)] __-__ __version__: the next version will be 0.8.1 (*Jina Dev Bot*)\n - [[```920b3107```](https://github.com/jina-ai/clip-as-service/commit/920b31070f54b1b6af4d4e58e7db351a576e0783)] __-__ bump version to 0.8.0 (*numb3r3*)\n\n\n## Release Note (`0.8.2`)\n\n> Release time: 2023-04-19 08:23:45\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Ziniu Yu, Yang Ruiyi, YangXiuyu, Jie Fu, zawabest, Girish Chandrashekar, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿ†• New Features\n\n - [[```cce3b05a```](https://github.com/jina-ai/clip-as-service/commit/cce3b05a1cfa23db129e8a7077e75e75f5da73c6)] __-__ set prefetch in client for traffic control (#897) (*Ziniu Yu*)\n - [[```dabbe8bc```](https://github.com/jina-ai/clip-as-service/commit/dabbe8bc3ef633e4460e1be3f1c06792fe08f00c)] __-__ add cn clip model (#888) (*Yang Ruiyi*)\n - [[```1fe3a5a0```](https://github.com/jina-ai/clip-as-service/commit/1fe3a5a01123dcfea8a7981fc5aea212d42c1299)] __-__ add fp16 inference support (torch/onnx) (#871) (*YangXiuyu*)\n - [[```1eebdd7f```](https://github.com/jina-ai/clip-as-service/commit/1eebdd7f489abb8e694226d5c5c29b011eab229a)] __-__ add custom tracing spans with jina>=3.12.0 (#861) (*Girish Chandrashekar*)\n - [[```f2515394```](https://github.com/jina-ai/clip-as-service/commit/f25153942464bb9230158af33c324cdb0b8b70a4)] __-__ add three new open clip roberta base models (#860) (*YangXiuyu*)\n - [[```e4717a35```](https://github.com/jina-ai/clip-as-service/commit/e4717a35f850e6a2cd8b4d8b4c994fad30fd5c72)] __-__ Integrate flash attention (#853) (*YangXiuyu*)\n\n### ๐Ÿž Bug fixes\n\n - [[```280b925e```](https://github.com/jina-ai/clip-as-service/commit/280b925e16ab5605a124d412f66ff56caa492553)] __-__ fix docarray at v1 (#911) (*Ziniu Yu*)\n - [[```35733a0b```](https://github.com/jina-ai/clip-as-service/commit/35733a0ba7fe6d9ae64d2d4d657d6ded2df3a6d1)] __-__ replace transform ndarray with transform blob (#910) (*Ziniu Yu*)\n - [[```d70f2382```](https://github.com/jina-ai/clip-as-service/commit/d70f238220f76593fb9b14e43e50f9a9d2cecd8a)] __-__ onnx package conflict during setup (#894) (*Ziniu Yu*)\n - [[```8a576c58```](https://github.com/jina-ai/clip-as-service/commit/8a576c585756e6526b1fe4a526858252d096535a)] __-__ install pytorch cu116 for server docker image (#882) (*Ziniu Yu*)\n - [[```0b293ec8```](https://github.com/jina-ai/clip-as-service/commit/0b293ec834e80f7335aa625d683904594373a607)] __-__ dynamic convert onnx model to fp16 during start session (#876) (*YangXiuyu*)\n - [[```fd16e5ab```](https://github.com/jina-ai/clip-as-service/commit/fd16e5abef94e274572d40912f12baeffece8696)] __-__ check dtype when loading models (#872) (*Ziniu Yu*)\n - [[```67f551ca```](https://github.com/jina-ai/clip-as-service/commit/67f551ca46c2bcf8c8598d6749544bd335da8bdb)] __-__ torchvision version to avoid compatibility issue (#866) (*Jie Fu*)\n - [[```0223e6fa```](https://github.com/jina-ai/clip-as-service/commit/0223e6fa071534bfc1a3b2010dd7065623afd540)] __-__ add pip installable flash attention (#863) (*YangXiuyu*)\n\n### ๐Ÿ“— Documentation\n\n - [[```1888ef65```](https://github.com/jina-ai/clip-as-service/commit/1888ef65f20a94b38f318696e663d447c7cb1dc6)] __-__ fix broken link in client doc (#909) (*Ziniu Yu*)\n - [[```f4eed3bc```](https://github.com/jina-ai/clip-as-service/commit/f4eed3bcbf5757571365159582d09f22c0ca8ed2)] __-__ add link and intro to inference api (#900) (*Ziniu Yu*)\n - [[```702fff88```](https://github.com/jina-ai/clip-as-service/commit/702fff88fc8070138b6eee517d9bb6167da0e87f)] __-__ default model suggestion (#874) (*Jie Fu*)\n\n### ๐Ÿน Other Improvements\n\n - [[```19b4fa51```](https://github.com/jina-ai/clip-as-service/commit/19b4fa51f7534b38a8ca236f05483602e44c0536)] __-__ remove docsqa html (#899) (*Ziniu Yu*)\n - [[```aa07d257```](https://github.com/jina-ai/clip-as-service/commit/aa07d2577fd27df03ccfff409ee00420071c41af)] __-__ remove docsqa (#898) (*Ziniu Yu*)\n - [[```f3421f7c```](https://github.com/jina-ai/clip-as-service/commit/f3421f7c1decbbdd3a5e1f1038666479c8fe60f6)] __-__ bump open-clip-torch to v2.8.0 (#883) (*Ziniu Yu*)\n - [[```c7af9f71```](https://github.com/jina-ai/clip-as-service/commit/c7af9f718550600973c6880de442619228f655e8)] __-__ fix configuration file for the search flow doc (#869) (*zawabest*)\n - [[```53cd0630```](https://github.com/jina-ai/clip-as-service/commit/53cd06301efde97e6e59a2b143323ccd5f5f2565)] __-__ hide changelog in docs (#864) (*Ziniu Yu*)\n - [[```9bb7d1f4```](https://github.com/jina-ai/clip-as-service/commit/9bb7d1f47d19e15e844108dec5f84cabcce7975d)] __-__ __version__: the next version will be 0.8.2 (*Jina Dev Bot*)\n\n\n## Release Note (`0.8.3`)\n\n> Release time: 2023-12-20 04:13:18\n\n\n\n๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular,\n Zihao Jing, Han Xiao, Nick de Silva, Ziniu Yu, Jina Dev Bot, ๐Ÿ™‡\n\n\n### ๐Ÿž Bug fixes\n\n - [[```280b925e```](https://github.com/jina-ai/clip-as-service/commit/280b925e16ab5605a124d412f66ff56caa492553)] __-__ fix docarray at v1 (#911) (*Ziniu Yu*)\n\n### ๐Ÿ“— Documentation\n\n - [[```ca2b25b7```](https://github.com/jina-ai/clip-as-service/commit/ca2b25b7564bc9b18ae38b93f0134e1f9aa0cee7)] __-__ remove jina self-hosted parts (#942) (*Zihao Jing*)\n - [[```6e418fe6```](https://github.com/jina-ai/clip-as-service/commit/6e418fe69c10dbac155e02267828d922a5601691)] __-__ replace free service docs with inference docs (#918) (*Ziniu Yu*)\n\n### ๐Ÿน Other Improvements\n\n - [[```d4e7a30b```](https://github.com/jina-ai/clip-as-service/commit/d4e7a30b755b2d314f89181fcc42624a1224b9ae)] __-__ Update README.md (*Han Xiao*)\n - [[```679de4e3```](https://github.com/jina-ai/clip-as-service/commit/679de4e3c9cb02b712f58540f6a3dd2e32d8e5e9)] __-__ change slack link to discord (*Han Xiao*)\n - [[```02abdc7b```](https://github.com/jina-ai/clip-as-service/commit/02abdc7b68214bedc181d9ef4be1c093ee60c609)] __-__ __version__: the next version will be 0.8.3 (*Jina Dev Bot*)\n\n" }, { "path": "Dockerfiles/base.Dockerfile", "content": "# !!! An ARG declared before a FROM is outside of a build stage, so it canโ€™t be used in any instruction after a FROM\nARG JINA_VERSION=3.11.0\n\nFROM jinaai/jina:${JINA_VERSION}-py38-standard\n\nARG BACKEND_TAG=torch\n\n# constant, wont invalidate cache\nLABEL org.opencontainers.image.vendor=\"Jina AI Limited\" \\\n org.opencontainers.image.licenses=\"Apache 2.0\" \\\n org.opencontainers.image.title=\"CLIP-as-Service\" \\\n org.opencontainers.image.description=\"Embed images and sentences into fixed-length vectors with CLIP\" \\\n org.opencontainers.image.authors=\"hello@jina.ai\" \\\n org.opencontainers.image.url=\"clip-as-service\" \\\n org.opencontainers.image.documentation=\"https://clip-as-service.jina.ai/\"\n\nRUN pip3 install --no-cache-dir torch torchvision torchaudio transformers --extra-index-url https://download.pytorch.org/whl/cpu\n\n# copy will almost always invalid the cache\nCOPY . /cas/\n\nWORKDIR /cas\n\nRUN if [ \"${BACKEND_TAG}\" != \"torch\" ]; then python3 -m pip install --no-cache-dir \"./[${BACKEND_TAG}]\" ; fi \\\n && python3 -m pip install --no-cache-dir .\n\nRUN echo \"\\\njtype: CLIPEncoder\\n\\\nmetas:\\n\\\n py_modules:\\n\\\n - clip_server.executors.clip_$BACKEND_TAG\\n\\\n\" > /tmp/config.yml\n\n\nENTRYPOINT [\"jina\", \"executor\", \"--uses\", \"/tmp/config.yml\", \"--timeout-ready\", \"3000000\"]\n" }, { "path": "Dockerfiles/cuda.Dockerfile", "content": "ARG CUDA_VERSION=11.4.2\n\nFROM nvcr.io/nvidia/cuda:${CUDA_VERSION}-cudnn8-runtime-ubuntu20.04\nENV DEBIAN_FRONTEND=noninteractive\n\nARG JINA_VERSION=3.11.0\nARG BACKEND_TAG=torch\n\n# constant, wont invalidate cache\nLABEL org.opencontainers.image.vendor=\"Jina AI Limited\" \\\n org.opencontainers.image.licenses=\"Apache 2.0\" \\\n org.opencontainers.image.title=\"CLIP-as-Service\" \\\n org.opencontainers.image.description=\"Embed images and sentences into fixed-length vectors with CLIP\" \\\n org.opencontainers.image.authors=\"hello@jina.ai\" \\\n org.opencontainers.image.url=\"clip-as-service\" \\\n org.opencontainers.image.documentation=\"https://clip-as-service.jina.ai/\"\n\nRUN apt-get update && apt-get install -y --no-install-recommends \\\n python3-setuptools python3-wheel python3-pip \\\n && apt-get clean && rm -rf /var/lib/apt/lists/*;\n\nRUN python3 -m pip install --default-timeout=1000 --no-cache-dir torch torchvision torchaudio nvidia-pyindex transformers --extra-index-url https://download.pytorch.org/whl/cu113\nRUN python3 -m pip install --default-timeout=1000 --no-cache-dir \"jina[standard]==${JINA_VERSION}\"\n\n# copy will almost always invalid the cache\nCOPY . /cas/\n\nWORKDIR /cas\n\nRUN if [ \"${BACKEND_TAG}\" != \"torch\" ]; then python3 -m pip install --no-cache-dir \"./[${BACKEND_TAG}]\" ; fi \\\n && python3 -m pip install --no-cache-dir .\n\nRUN echo \"\\\njtype: CLIPEncoder\\n\\\nmetas:\\n\\\n py_modules:\\n\\\n - clip_server.executors.clip_$BACKEND_TAG\\n\\\n\" > /tmp/config.yml\n\nENTRYPOINT [\"jina\", \"executor\", \"--uses\", \"/tmp/config.yml\", \"--timeout-ready\", \"3000000\"]\n\n\n\n\n\n" }, { "path": "Dockerfiles/server.Dockerfile", "content": "ARG CUDA_VERSION=11.6.0\n\nFROM nvidia/cuda:${CUDA_VERSION}-devel-ubuntu20.04\n\nARG CAS_NAME=cas\nWORKDIR /${CAS_NAME}\n\nENV PIP_NO_CACHE_DIR=1 \\\n PIP_DISABLE_PIP_VERSION_CHECK=1\n\n# constant, wont invalidate cache\nLABEL org.opencontainers.image.vendor=\"Jina AI Limited\" \\\n org.opencontainers.image.licenses=\"Apache 2.0\" \\\n org.opencontainers.image.title=\"CLIP-as-Service\" \\\n org.opencontainers.image.description=\"Embed images and sentences into fixed-length vectors with CLIP\" \\\n org.opencontainers.image.authors=\"hello@jina.ai\" \\\n org.opencontainers.image.url=\"clip-as-service\" \\\n org.opencontainers.image.documentation=\"https://clip-as-service.jina.ai/\"\n\n\nRUN apt-get update \\\n && apt-get install -y --no-install-recommends python3 python3-pip wget \\\n && ln -sf python3 /usr/bin/python \\\n && ln -sf pip3 /usr/bin/pip \\\n && pip install --upgrade pip \\\n && pip install wheel setuptools nvidia-pyindex \\\n && pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116\n\nCOPY server ./server\n# given by builder\nARG PIP_TAG\nRUN pip install --default-timeout=1000 --compile ./server/ \\\n && if [ -n \"${PIP_TAG}\" ]; then pip install --default-timeout=1000 --compile \"./server[${PIP_TAG}]\" ; fi\n\nENV LD_LIBRARY_PATH=/usr/local/cuda/lib64\n\nARG USER_ID=1000\nARG GROUP_ID=1000\nARG USER_NAME=${CAS_NAME}\nARG GROUP_NAME=${CAS_NAME}\n\nRUN groupadd -g ${GROUP_ID} ${USER_NAME} &&\\\n useradd -l -u ${USER_ID} -g ${USER_NAME} ${GROUP_NAME} &&\\\n mkdir /home/${USER_NAME} &&\\\n chown ${USER_NAME}:${GROUP_NAME} /home/${USER_NAME} &&\\\n chown -R ${USER_NAME}:${GROUP_NAME} /${CAS_NAME}/\n\nUSER ${USER_NAME}\n\nENTRYPOINT [\"python\", \"-m\", \"clip_server\"]" }, { "path": "Dockerfiles/tensorrt.Dockerfile", "content": "# Dockerfile to run Clip-as-Service with TensorRT, CUDA integration\n\nARG TENSORRT_VERSION=22.04\n\nFROM nvcr.io/nvidia/tensorrt:${TENSORRT_VERSION}-py3\n\nARG JINA_VERSION=3.7.0\nARG BACKEND_TAG=tensorrt\n\n# constant, wont invalidate cache\nLABEL org.opencontainers.image.vendor=\"Jina AI Limited\" \\\n org.opencontainers.image.licenses=\"Apache 2.0\" \\\n org.opencontainers.image.title=\"CLIP-as-Service\" \\\n org.opencontainers.image.description=\"Embed images and sentences into fixed-length vectors with CLIP\" \\\n org.opencontainers.image.authors=\"hello@jina.ai\" \\\n org.opencontainers.image.url=\"clip-as-service\" \\\n org.opencontainers.image.documentation=\"https://clip-as-service.jina.ai/\"\n\nRUN pip3 install --default-timeout=1000 --no-cache-dir torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113\nRUN pip3 -m pip install --default-timeout=1000 --no-cache-dir \"jina[standard]==${JINA_VERSION}\"\n\n# copy will almost always invalid the cache\nCOPY . /cas/\nWORKDIR /cas\n\nRUN python3 -m pip install --no-cache-dir \"./[$BACKEND_TAG]\"\n\n\nRUN echo \"\\\njtype: CLIPEncoder\\n\\\nmetas:\\n\\\n py_modules:\\n\\\n - clip_server.executors.clip_$BACKEND_TAG\\n\\\n\" > /tmp/config.yml\n\n\nENTRYPOINT [\"jina\", \"executor\", \"--uses\", \"/tmp/config.yml\"]\n\n" }, { "path": "LICENSE", "content": "Copyright 2020-2022 Jina AI Limited. All rights reserved.\n\nThe following two files are licensed under MIT License via https://github.com/mlfoundations/open_clip Copyright (c) 2021, OpenCLIP\n server/clip_server/model/model.py\n server/clip_server/model/simple_tokenizer.py\n\n\n Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n Copyright 2020-2022 Jina AI Limited\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n" }, { "path": "README.md", "content": "

\n\"CLIP-as-service\n


\n

\n\n\n

\n\"PyPI\"\n\n\"Codecov\n\"Host\n

\n\n\n\nCLIP-as-service is a low-latency high-scalability service for embedding images and text. It can be easily integrated as a microservice into neural search solutions.\n\nโšก **Fast**: Serve CLIP models with TensorRT, ONNX runtime and PyTorch w/o JIT with 800QPS[*]. Non-blocking duplex streaming on requests and responses, designed for large data and long-running tasks. \n\n๐Ÿซ **Elastic**: Horizontally scale up and down multiple CLIP models on single GPU, with automatic load balancing.\n\n๐Ÿฅ **Easy-to-use**: No learning curve, minimalist design on client and server. Intuitive and consistent API for image and sentence embedding. \n\n๐Ÿ‘’ **Modern**: Async client support. Easily switch between gRPC, HTTP, WebSocket protocols with TLS and compression.\n\n๐Ÿฑ **Integration**: Smooth integration with neural search ecosystem including [Jina](https://github.com/jina-ai/jina) and [DocArray](https://github.com/jina-ai/docarray). Build cross-modal and multi-modal solutions in no time. \n\n[*] with default config (single replica, PyTorch no JIT) on GeForce RTX 3090. \n\n\n\n### Text & image embedding\n\n\n\n\n\n\n\n\n\n\n
via HTTPS ๐Ÿ” via gRPC ๐Ÿ”โšกโšก
\n\n```bash\ncurl \\\n-X POST https://-http.wolf.jina.ai/post \\\n-H 'Content-Type: application/json' \\\n-H 'Authorization: ' \\\n-d '{\"data\":[{\"text\": \"First do it\"}, \n {\"text\": \"then do it right\"}, \n {\"text\": \"then do it better\"}, \n {\"uri\": \"https://picsum.photos/200\"}], \n \"execEndpoint\":\"/\"}'\n```\n\n\n\n```python\n# pip install clip-client\nfrom clip_client import Client\n\nc = Client(\n 'grpcs://-grpc.wolf.jina.ai',\n credential={'Authorization': ''},\n)\n\nr = c.encode(\n [\n 'First do it',\n 'then do it right',\n 'then do it better',\n 'https://picsum.photos/200',\n ]\n)\nprint(r)\n```\n
\n\n### Visual reasoning\n\nThere are four basic visual reasoning skills: object recognition, object counting, color recognition, and spatial relation understanding. Let's try some:\n\n> You need to install [`jq` (a JSON processor)](https://stedolan.github.io/jq/) to prettify the results.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Image via HTTPS ๐Ÿ”
\n\n\n\n```bash\ncurl \\\n-X POST https://-http.wolf.jina.ai/post \\\n-H 'Content-Type: application/json' \\\n-H 'Authorization: ' \\\n-d '{\"data\":[{\"uri\": \"https://picsum.photos/id/1/300/300\",\n\"matches\": [{\"text\": \"there is a woman in the photo\"},\n {\"text\": \"there is a man in the photo\"}]}],\n \"execEndpoint\":\"/rank\"}' \\\n| jq \".data[].matches[] | (.text, .scores.clip_score.value)\"\n```\n\ngives:\n\n```\n\"there is a woman in the photo\"\n0.626907229423523\n\"there is a man in the photo\"\n0.37309277057647705\n```\n\n
\n\n\n\n```bash\ncurl \\\n-X POST https://-http.wolf.jina.ai/post \\\n-H 'Content-Type: application/json' \\\n-H 'Authorization: ' \\\n-d '{\"data\":[{\"uri\": \"https://picsum.photos/id/133/300/300\",\n\"matches\": [\n{\"text\": \"the blue car is on the left, the red car is on the right\"},\n{\"text\": \"the blue car is on the right, the red car is on the left\"},\n{\"text\": \"the blue car is on top of the red car\"},\n{\"text\": \"the blue car is below the red car\"}]}],\n\"execEndpoint\":\"/rank\"}' \\\n| jq \".data[].matches[] | (.text, .scores.clip_score.value)\"\n```\n\ngives:\n```\n\"the blue car is on the left, the red car is on the right\"\n0.5232442617416382\n\"the blue car is on the right, the red car is on the left\"\n0.32878655195236206\n\"the blue car is below the red car\"\n0.11064132302999496\n\"the blue car is on top of the red car\"\n0.03732786327600479\n```\n\n
\n\n\n\n```bash\ncurl \\\n-X POST https://-http.wolf.jina.ai/post \\\n-H 'Content-Type: application/json' \\\n-H 'Authorization: ' \\\n-d '{\"data\":[{\"uri\": \"https://picsum.photos/id/102/300/300\",\n\"matches\": [{\"text\": \"this is a photo of one berry\"},\n {\"text\": \"this is a photo of two berries\"},\n {\"text\": \"this is a photo of three berries\"},\n {\"text\": \"this is a photo of four berries\"},\n {\"text\": \"this is a photo of five berries\"},\n {\"text\": \"this is a photo of six berries\"}]}],\n \"execEndpoint\":\"/rank\"}' \\\n| jq \".data[].matches[] | (.text, .scores.clip_score.value)\"\n```\n\ngives:\n```\n\"this is a photo of three berries\"\n0.48507222533226013\n\"this is a photo of four berries\"\n0.2377079576253891\n\"this is a photo of one berry\"\n0.11304923892021179\n\"this is a photo of five berries\"\n0.0731358453631401\n\"this is a photo of two berries\"\n0.05045759305357933\n\"this is a photo of six berries\"\n0.04057715833187103\n```\n\n
\n\n\n## [Documentation](https://clip-as-service.jina.ai)\n\n## Install\n\nCLIP-as-service consists of two Python packages `clip-server` and `clip-client` that can be installed _independently_. Both require Python 3.7+. \n\n### Install server\n\n\n\n\n\n\n\n\n\n\n\n\n
Pytorch Runtime โšก ONNX Runtime โšกโšก TensorRT Runtime โšกโšกโšก
\n\n```bash\npip install clip-server\n```\n\n\n\n```bash\npip install \"clip-server[onnx]\"\n```\n\n\n\n```bash\npip install nvidia-pyindex \npip install \"clip-server[tensorrt]\"\n```\n
\n\nYou can also [host the server on Google Colab](https://clip-as-service.jina.ai/hosting/colab/), leveraging its free GPU/TPU.\n\n### Install client\n\n```bash\npip install clip-client\n```\n\n### Quick check\n\nYou can run a simple connectivity check after install.\n\n\n\n\n \n \n\n\n\n\n\n\n\n\n\n\n\n\n
C/S Command Expect output
\nServer\n \n\n```bash\npython -m clip_server\n```\n \n\n\n\"Expected\n\n
\nClient\n \n\n```python\nfrom clip_client import Client\n\nc = Client('grpc://0.0.0.0:23456')\nc.profile()\n```\n \n\n\n\"Expected\n\n
\n\n\nYou can change `0.0.0.0` to the intranet or public IP address to test the connectivity over private and public network. \n\n\n## Get Started\n\n### Basic usage\n\n1. Start the server: `python -m clip_server`. Remember its address and port.\n2. Create a client:\n ```python\n from clip_client import Client\n \n c = Client('grpc://0.0.0.0:51000')\n ```\n3. To get sentence embedding:\n ```python \n r = c.encode(['First do it', 'then do it right', 'then do it better'])\n \n print(r.shape) # [3, 512] \n ```\n4. To get image embedding:\n ```python \n r = c.encode(['apple.png', # local image \n 'https://clip-as-service.jina.ai/_static/favicon.png', # remote image\n 'data:image/gif;base64,R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o/XBs/fNwfjZ0frl3/zy7////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7']) # in image URI\n \n print(r.shape) # [3, 512]\n ```\n\nMore comprehensive server and client user guides can be found in the [docs](https://clip-as-service.jina.ai/).\n\n### Text-to-image cross-modal search in 10 lines\n\nLet's build a text-to-image search using CLIP-as-service. Namely, a user can input a sentence and the program returns matching images. We'll use the [Totally Looks Like](https://sites.google.com/view/totally-looks-like-dataset) dataset and [DocArray](https://github.com/jina-ai/docarray) package. Note that DocArray is included within `clip-client` as an upstream dependency, so you don't need to install it separately.\n\n#### Load images\n\nFirst we load images. You can simply pull them from Jina Cloud:\n\n```python\nfrom docarray import DocumentArray\n\nda = DocumentArray.pull('ttl-original', show_progress=True, local_cache=True)\n```\n\n
\nor download TTL dataset, unzip, load manually\n\nAlternatively, you can go to [Totally Looks Like](https://sites.google.com/view/totally-looks-like-dataset) official website, unzip and load images:\n\n```python\nfrom docarray import DocumentArray\n\nda = DocumentArray.from_files(['left/*.jpg', 'right/*.jpg'])\n```\n\n
\n\nThe dataset contains 12,032 images, so it may take a while to pull. Once done, you can visualize it and get the first taste of those images:\n\n```python\nda.plot_image_sprites()\n```\n\n

\n\"Visualization\n

\n\n#### Encode images\n\nStart the server with `python -m clip_server`. Let's say it's at `0.0.0.0:51000` with `GRPC` protocol (you will get this information after running the server).\n\nCreate a Python client script:\n\n```python\nfrom clip_client import Client\n\nc = Client(server='grpc://0.0.0.0:51000')\n\nda = c.encode(da, show_progress=True)\n```\n\nDepending on your GPU and client-server network, it may take a while to embed 12K images. In my case, it took about two minutes.\n\n
\nDownload the pre-encoded dataset\n\nIf you're impatient or don't have a GPU, waiting can be Hell. In this case, you can simply pull our pre-encoded image dataset:\n\n```python\nfrom docarray import DocumentArray\n\nda = DocumentArray.pull('ttl-embedding', show_progress=True, local_cache=True)\n```\n\n
\n\n#### Search via sentence \n\nLet's build a simple prompt to allow a user to type sentence:\n\n```python\nwhile True:\n vec = c.encode([input('sentence> ')])\n r = da.find(query=vec, limit=9)\n r[0].plot_image_sprites()\n```\n\n#### Showcase\n\nNow you can input arbitrary English sentences and view the top-9 matching images. Search is fast and instinctive. Let's have some fun:\n\n\n\n \n \n\n\n\n\n\n\n\n
\"a happy potato\" \"a super evil AI\" \"a guy enjoying his burger\"
\n\n

\n\"Visualization\n

\n\n		\n\n

\n\"Visualization\n

\n\n		\n\n

\n\"Visualization\n

\n\n
\n\n\n\n\n \n \n\n\n\n\n\n\n\n
\"professor cat is very serious\" \"an ego engineer lives with parent\" \"there will be no tomorrow so lets eat unhealthy\"
\n\n

\n\"Visualization\n

\n\n		\n\n

\n\"Visualization\n

\n\n		\n\n

\n\"Visualization\n

\n\n
\n\nLet's save the embedding result for our next example: \n\n```python\nda.save_binary('ttl-image')\n```\n\n### Image-to-text cross-modal search in 10 Lines\n\nWe can also switch the input and output of the last program to achieve image-to-text search. Precisely, given a query image find the sentence that best describes the image.\n\nLet's use all sentences from the book \"Pride and Prejudice\". \n\n```python\nfrom docarray import Document, DocumentArray\n\nd = Document(uri='https://www.gutenberg.org/files/1342/1342-0.txt').load_uri_to_text()\nda = DocumentArray(\n Document(text=s.strip()) for s in d.text.replace('\\r\\n', '').split('.') if s.strip()\n)\n```\n\nLet's look at what we got:\n\n```python\nda.summary()\n```\n\n```text\n Documents Summary \n \n Length 6403 \n Homogenous Documents True \n Common Attributes ('id', 'text') \n \n Attributes Summary \n \n Attribute Data type #Unique values Has empty value \n โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ \n id ('str',) 6403 False \n text ('str',) 6030 False \n```\n\n#### Encode sentences\n\nNow encode these 6,403 sentences, it may take 10 seconds or less depending on your GPU and network: \n\n```python\nfrom clip_client import Client\n\nc = Client('grpc://0.0.0.0:51000')\n\nr = c.encode(da, show_progress=True)\n```\n\n
\nDownload the pre-encoded dataset\n\nAgain, for people who are impatient or don't have a GPU, we have prepared a pre-encoded text dataset:\n\n```python\nfrom docarray import DocumentArray\n\nda = DocumentArray.pull('ttl-textual', show_progress=True, local_cache=True)\n```\n\n
\n\n#### Search via image\n\nLet's load our previously stored image embedding, randomly sample 10 image Documents, then find top-1 nearest neighbour of each.\n\n```python\nfrom docarray import DocumentArray\n\nimg_da = DocumentArray.load_binary('ttl-image')\n\nfor d in img_da.sample(10):\n print(da.find(d.embedding, limit=1)[0].text)\n```\n\n#### Showcase\n\nFun time! Note, unlike the previous example, here the input is an image and the sentence is the output. All sentences come from the book \"Pride and Prejudice\". \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n

\n\"Visualization\n

\n\n\n		\n\n

\n\"Visualization\n

\n\n		\n\n

\n\"Visualization\n

\n\n		\n\n

\n\"Visualization\n

\n\n		\n\n

\n\"Visualization\n

\n\n
Besides, there was truth in his looksGardiner smiledwhatโ€™s his nameBy tea time, however, the dose had been enough, and MrYou do not look well
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\n

\n\"Visualization\n

\n\n\n		\n\n

\n\"Visualization\n

\n\n		\n\n

\n\"Visualization\n

\n\n		\n\n

\n\"Visualization\n

\n\n		\n\n

\n\"Visualization\n

\n\n
โ€œA gamester!โ€ she criedIf you mention my name at the Bell, you will be attended toNever mind Miss Lizzyโ€™s hairElizabeth will soon be the wife of MrI saw them the night before last
\n\n\n\n### Rank image-text matches via CLIP model\n\nFrom `0.3.0` CLIP-as-service adds a new `/rank` endpoint that re-ranks cross-modal matches according to their joint likelihood in CLIP model. For example, given an image Document with some predefined sentence matches as below:\n\n```python\nfrom clip_client import Client\nfrom docarray import Document\n\nc = Client(server='grpc://0.0.0.0:51000')\nr = c.rank(\n [\n Document(\n uri='.github/README-img/rerank.png',\n matches=[\n Document(text=f'a photo of a {p}')\n for p in (\n 'control room',\n 'lecture room',\n 'conference room',\n 'podium indoor',\n 'television studio',\n )\n ],\n )\n ]\n)\n\nprint(r['@m', ['text', 'scores__clip_score__value']])\n```\n\n```text\n[['a photo of a television studio', 'a photo of a conference room', 'a photo of a lecture room', 'a photo of a control room', 'a photo of a podium indoor'], \n[0.9920725226402283, 0.006038925610482693, 0.0009973491542041302, 0.00078492151806131, 0.00010626466246321797]]\n```\n\nOne can see now `a photo of a television studio` is ranked to the top with `clip_score` score at `0.992`. In practice, one can use this endpoint to re-rank the matching result from another search system, for improving the cross-modal search quality.\n\n\n\n\n\n\n
\n\"Rerank\n\n\"Rerank\n
\n\n### Rank text-image matches via CLIP model\n\nIn the [DALLยทE Flow](https://github.com/jina-ai/dalle-flow) project, CLIP is called for ranking the generated results from DALLยทE. [It has an Executor wrapped on top of `clip-client`](https://github.com/jina-ai/dalle-flow/blob/main/executors/rerank/executor.py), which calls `.arank()` - the async version of `.rank()`:\n\n```python\nfrom clip_client import Client\nfrom jina import Executor, requests, DocumentArray\n\n\nclass ReRank(Executor):\n def __init__(self, clip_server: str, **kwargs):\n super().__init__(**kwargs)\n self._client = Client(server=clip_server)\n\n @requests(on='/')\n async def rerank(self, docs: DocumentArray, **kwargs):\n return await self._client.arank(docs)\n```\n\n

\n\"CLIP-as-service\n

\n\nIntrigued? That's only scratching the surface of what CLIP-as-service is capable of. [Read our docs to learn more](https://clip-as-service.jina.ai).\n\n\n## Support\n\n- Join our [Discord community](https://discord.jina.ai) and chat with other community members about ideas.\n- Watch our [Engineering All Hands](https://youtube.com/playlist?list=PL3UBBWOUVhFYRUa_gpYYKBqEAkO4sxmne) to learn Jina's new features and stay up-to-date with the latest AI techniques.\n- Subscribe to the latest video tutorials on our [YouTube channel](https://youtube.com/c/jina-ai)\n\n## Join Us\n\nCLIP-as-service is backed by [Jina AI](https://jina.ai) and licensed under [Apache-2.0](./LICENSE). [We are actively hiring](https://jobs.jina.ai) AI engineers, solution engineers to build the next neural search ecosystem in open-source.\n\n\n" }, { "path": "client/clip_client/__init__.py", "content": "__version__ = '0.8.4'\n\nimport os\n\nfrom clip_client.client import Client\n\nif 'NO_VERSION_CHECK' not in os.environ:\n from clip_client.helper import is_latest_version\n\n is_latest_version(github_repo='clip-as-service')\n" }, { "path": "client/clip_client/client.py", "content": "import mimetypes\nimport os\nimport time\nimport warnings\nfrom typing import (\n overload,\n TYPE_CHECKING,\n Optional,\n Union,\n Iterator,\n Generator,\n Iterable,\n Dict,\n)\nfrom urllib.parse import urlparse\nfrom functools import partial\nfrom docarray import DocumentArray\n\nif TYPE_CHECKING:\n import numpy as np\n from docarray import Document\n from jina.clients.base import CallbackFnType\n\n\nclass Client:\n def __init__(self, server: str, credential: dict = {}, **kwargs):\n \"\"\"Create a Clip client object that connects to the Clip server.\n Server scheme is in the format of ``scheme://netloc:port``, where\n - scheme: one of grpc, websocket, http, grpcs, websockets, https\n - netloc: the server ip address or hostname\n - port: the public port of the server\n :param server: the server URI\n :param credential: the credential for authentication ``{'Authentication': ''}``\n \"\"\"\n try:\n r = urlparse(server)\n _port = r.port\n self._scheme = r.scheme\n except:\n raise ValueError(f'{server} is not a valid scheme')\n\n _tls = False\n if self._scheme in ('grpcs', 'https', 'wss'):\n self._scheme = self._scheme[:-1]\n _tls = True\n\n if self._scheme == 'ws':\n self._scheme = 'websocket' # temp fix for the core\n if credential:\n warnings.warn(\n 'Credential is not supported for websocket, please use grpc or http'\n )\n\n if self._scheme in ('grpc', 'http', 'websocket'):\n _kwargs = dict(host=r.hostname, port=_port, protocol=self._scheme, tls=_tls)\n\n from jina import Client\n\n self._client = Client(**_kwargs)\n self._async_client = Client(**_kwargs, asyncio=True)\n else:\n raise ValueError(f'{server} is not a valid scheme')\n\n self._authorization = credential.get(\n 'Authorization', os.environ.get('CLIP_AUTH_TOKEN')\n )\n\n def profile(self, content: Optional[str] = '') -> Dict[str, float]:\n \"\"\"Profiling a single query's roundtrip including network and computation latency. Results is summarized in a table.\n :param content: the content to be sent for profiling. By default it sends an empty Document\n that helps you understand the network latency.\n :return: the latency report in a dict.\n \"\"\"\n st = time.perf_counter()\n r = self._client.post(\n '/', self._iter_doc([content], DocumentArray()), return_responses=True\n )\n ed = (time.perf_counter() - st) * 1000\n route = r[0].routes\n gateway_time = (\n route[0].end_time.ToMilliseconds() - route[0].start_time.ToMilliseconds()\n )\n clip_time = (\n route[1].end_time.ToMilliseconds() - route[1].start_time.ToMilliseconds()\n )\n network_time = ed - gateway_time\n server_network = gateway_time - clip_time\n\n from rich.table import Table\n\n def make_table(_title, _time, _percent):\n table = Table(show_header=False, box=None)\n table.add_row(\n _title, f'[b]{_time:.0f}[/b]ms', f'[dim]{_percent * 100:.0f}%[/dim]'\n )\n return table\n\n from rich.tree import Tree\n\n t = Tree(make_table('Roundtrip', ed, 1))\n t.add(make_table('Client-server network', network_time, network_time / ed))\n t2 = t.add(make_table('Server', gateway_time, gateway_time / ed))\n t2.add(\n make_table(\n 'Gateway-CLIP network', server_network, server_network / gateway_time\n )\n )\n t2.add(make_table('CLIP model', clip_time, clip_time / gateway_time))\n\n from rich import print\n\n print(t)\n\n return {\n 'Roundtrip': ed,\n 'Client-server network': network_time,\n 'Server': gateway_time,\n 'Gateway-CLIP network': server_network,\n 'CLIP model': clip_time,\n }\n\n def _update_pbar(self, response, func: Optional['CallbackFnType'] = None):\n from rich import filesize\n\n r = response.data.docs\n if not self._pbar._tasks[self._r_task].started:\n self._pbar.start_task(self._r_task)\n self._pbar.update(\n self._r_task,\n advance=len(r),\n total_size=str(\n filesize.decimal(int(os.environ.get('JINA_GRPC_RECV_BYTES', '0')))\n ),\n )\n if func is not None:\n func(response)\n\n def _prepare_streaming(self, disable, total):\n if total is None:\n total = 500\n warnings.warn(\n 'The length of the input is unknown, the progressbar would not be accurate.'\n )\n elif total > 500:\n warnings.warn(\n 'Please ensure all the inputs are valid, otherwise the request will be aborted.'\n )\n\n from docarray.array.mixins.io.pbar import get_pbar\n\n self._pbar = get_pbar(disable)\n\n os.environ['JINA_GRPC_SEND_BYTES'] = '0'\n os.environ['JINA_GRPC_RECV_BYTES'] = '0'\n\n self._r_task = self._pbar.add_task(\n ':arrow_down: Progress', total=total, total_size=0, start=False\n )\n\n @staticmethod\n def _gather_result(\n response, results: 'DocumentArray', attribute: Optional[str] = None\n ):\n r = response.data.docs\n if attribute:\n results[r[:, 'id']][:, attribute] = r[:, attribute]\n\n def _iter_doc(\n self, content, results: Optional['DocumentArray'] = None\n ) -> Generator['Document', None, None]:\n from docarray import Document\n\n for c in content:\n if isinstance(c, str):\n _mime = mimetypes.guess_type(c)[0]\n if _mime and _mime.startswith('image'):\n d = Document(\n uri=c,\n ).load_uri_to_blob()\n else:\n d = Document(text=c)\n elif isinstance(c, Document):\n if c.content_type in ('text', 'blob'):\n d = c\n elif not c.blob and c.uri:\n c.load_uri_to_blob()\n d = c\n elif c.tensor is not None:\n d = c\n else:\n raise TypeError(f'unsupported input type {c!r} {c.content_type}')\n else:\n raise TypeError(f'unsupported input type {c!r}')\n\n if results is not None:\n results.append(d)\n yield d\n\n def _get_post_payload(\n self, content, results: Optional['DocumentArray'] = None, **kwargs\n ):\n payload = dict(\n inputs=self._iter_doc(content, results),\n request_size=kwargs.get('batch_size', 8),\n total_docs=len(content) if hasattr(content, '__len__') else None,\n )\n\n if self._scheme == 'grpc' and self._authorization:\n payload.update(metadata=(('authorization', self._authorization),))\n elif self._scheme == 'http' and self._authorization:\n payload.update(headers={'Authorization': self._authorization})\n return payload\n\n @staticmethod\n def _unboxed_result(results: Optional['DocumentArray'] = None, unbox: bool = False):\n if results is not None:\n if results.embeddings is None:\n raise ValueError(\n 'Empty embedding returned from the server. '\n 'This often due to a mis-config of the server, '\n 'restarting the server or changing the serving port number often solves the problem'\n )\n return results.embeddings if unbox else results\n\n @overload\n def encode(\n self,\n content: Iterable[str],\n *,\n batch_size: Optional[int] = None,\n show_progress: bool = False,\n parameters: Optional[dict] = None,\n on_done: Optional['CallbackFnType'] = None,\n on_error: Optional['CallbackFnType'] = None,\n on_always: Optional['CallbackFnType'] = None,\n prefetch: int = 100,\n ) -> 'np.ndarray':\n \"\"\"Encode images and texts into embeddings where the input is an iterable of raw strings.\n Each image and text must be represented as a string. The following strings are acceptable:\n - local image filepath, will be considered as an image\n - remote image http/https, will be considered as an image\n - a dataURI, will be considered as an image\n - plain text, will be considered as a sentence\n :param content: an iterator of image URIs or sentences, each element is an image or a text sentence as a string.\n :param batch_size: the number of elements in each request when sending ``content``\n :param show_progress: if set, show a progress bar\n :param parameters: the parameters for the encoding, you can specify the model to use when you have multiple models\n :param on_done: the callback function executed while streaming, after successful completion of each request.\n It takes the response ``DataRequest`` as the only argument\n :param on_error: the callback function executed while streaming, after failed completion of each request.\n It takes the response ``DataRequest`` as the only argument\n :param on_always: the callback function executed while streaming, after completion of each request.\n It takes the response ``DataRequest`` as the only argument\n :param prefetch: the number of in-flight batches made by the post() method. Use a lower value for expensive\n operations, and a higher value for faster response times\n :return: the embedding in a numpy ndarray with shape ``[N, D]``. ``N`` is in the same length of ``content``\n \"\"\"\n ...\n\n @overload\n def encode(\n self,\n content: Union['DocumentArray', Iterable['Document']],\n *,\n batch_size: Optional[int] = None,\n show_progress: bool = False,\n parameters: Optional[dict] = None,\n on_done: Optional['CallbackFnType'] = None,\n on_error: Optional['CallbackFnType'] = None,\n on_always: Optional['CallbackFnType'] = None,\n prefetch: int = 100,\n ) -> 'DocumentArray':\n \"\"\"Encode images and texts into embeddings where the input is an iterable of :class:`docarray.Document`.\n :param content: an iterable of :class:`docarray.Document`, each Document must be filled with `.uri`, `.text` or `.blob`.\n :param batch_size: the number of elements in each request when sending ``content``\n :param show_progress: if set, show a progress bar\n :param parameters: the parameters for the encoding, you can specify the model to use when you have multiple models\n :param on_done: the callback function executed while streaming, after successful completion of each request.\n It takes the response ``DataRequest`` as the only argument\n :param on_error: the callback function executed while streaming, after failed completion of each request.\n It takes the response ``DataRequest`` as the only argument\n :param on_always: the callback function executed while streaming, after completion of each request.\n It takes the response ``DataRequest`` as the only argument\n :param prefetch: the number of in-flight batches made by the post() method. Use a lower value for expensive\n operations, and a higher value for faster response times\n :return: the embedding in a numpy ndarray with shape ``[N, D]``. ``N`` is in the same length of ``content``\n \"\"\"\n ...\n\n def encode(self, content, **kwargs):\n if isinstance(content, str):\n raise TypeError(\n f'Content must be an Iterable of [str, Document], try `.encode([\"{content}\"])` instead'\n )\n if hasattr(content, '__len__') and len(content) == 0:\n return DocumentArray() if isinstance(content, DocumentArray) else []\n\n self._prepare_streaming(\n not kwargs.get('show_progress'),\n total=len(content) if hasattr(content, '__len__') else None,\n )\n on_done = kwargs.pop('on_done', None)\n on_error = kwargs.pop('on_error', None)\n on_always = kwargs.pop('on_always', None)\n prefetch = kwargs.pop('prefetch', 100)\n results = DocumentArray() if not on_done and not on_always else None\n if not on_done:\n on_done = partial(\n self._gather_result, results=results, attribute='embedding'\n )\n\n with self._pbar:\n parameters = kwargs.pop('parameters', {})\n parameters['drop_image_content'] = parameters.get(\n 'drop_image_content', True\n )\n model_name = parameters.pop('model_name', '') if parameters else ''\n\n self._client.post(\n on=f'/encode/{model_name}'.rstrip('/'),\n **self._get_post_payload(content, results, **kwargs),\n on_done=on_done,\n on_error=on_error,\n on_always=partial(self._update_pbar, func=on_always),\n parameters=parameters,\n prefetch=prefetch,\n )\n\n unbox = hasattr(content, '__len__') and isinstance(content[0], str)\n return self._unboxed_result(results, unbox)\n\n @overload\n async def aencode(\n self,\n content: Iterator[str],\n *,\n batch_size: Optional[int] = None,\n show_progress: bool = False,\n parameters: Optional[dict] = None,\n on_done: Optional['CallbackFnType'] = None,\n on_error: Optional['CallbackFnType'] = None,\n on_always: Optional['CallbackFnType'] = None,\n prefetch: int = 100,\n ) -> 'np.ndarray':\n ...\n\n @overload\n async def aencode(\n self,\n content: Union['DocumentArray', Iterable['Document']],\n *,\n batch_size: Optional[int] = None,\n show_progress: bool = False,\n parameters: Optional[dict] = None,\n on_done: Optional['CallbackFnType'] = None,\n on_error: Optional['CallbackFnType'] = None,\n on_always: Optional['CallbackFnType'] = None,\n prefetch: int = 100,\n ) -> 'DocumentArray':\n ...\n\n async def aencode(self, content, **kwargs):\n if isinstance(content, str):\n raise TypeError(\n f'Content must be an Iterable of [str, Document], try `.aencode([\"{content}\"])` instead'\n )\n if hasattr(content, '__len__') and len(content) == 0:\n return DocumentArray() if isinstance(content, DocumentArray) else []\n\n self._prepare_streaming(\n not kwargs.get('show_progress'),\n total=len(content) if hasattr(content, '__len__') else None,\n )\n on_done = kwargs.pop('on_done', None)\n on_error = kwargs.pop('on_error', None)\n on_always = kwargs.pop('on_always', None)\n prefetch = kwargs.pop('prefetch', 100)\n results = DocumentArray() if not on_done and not on_always else None\n if not on_done:\n on_done = partial(\n self._gather_result, results=results, attribute='embedding'\n )\n\n with self._pbar:\n parameters = kwargs.pop('parameters', {})\n parameters['drop_image_content'] = parameters.get(\n 'drop_image_content', True\n )\n model_name = parameters.get('model_name', '') if parameters else ''\n\n async for _ in self._async_client.post(\n on=f'/encode/{model_name}'.rstrip('/'),\n **self._get_post_payload(content, results, **kwargs),\n on_done=on_done,\n on_error=on_error,\n on_always=partial(self._update_pbar, func=on_always),\n parameters=parameters,\n prefetch=prefetch,\n ):\n continue\n\n unbox = hasattr(content, '__len__') and isinstance(content[0], str)\n return self._unboxed_result(results, unbox)\n\n def _iter_rank_docs(\n self, content, results: Optional['DocumentArray'] = None, source='matches'\n ) -> Generator['Document', None, None]:\n from docarray import Document\n\n for c in content:\n if isinstance(c, Document):\n d = self._prepare_rank_doc(c, source)\n else:\n raise TypeError(f'Unsupported input type {c!r}')\n if results is not None:\n results.append(d)\n yield d\n\n def _get_rank_payload(\n self, content, results: Optional['DocumentArray'] = None, **kwargs\n ):\n payload = dict(\n inputs=self._iter_rank_docs(\n content, results, source=kwargs.get('source', 'matches')\n ),\n request_size=kwargs.get('batch_size', 8),\n total_docs=len(content) if hasattr(content, '__len__') else None,\n )\n if self._scheme == 'grpc' and self._authorization:\n payload.update(metadata=(('authorization', self._authorization),))\n elif self._scheme == 'http' and self._authorization:\n payload.update(headers={'Authorization': self._authorization})\n return payload\n\n @staticmethod\n def _prepare_single_doc(d: 'Document'):\n if d.content_type in ('text', 'blob'):\n return d\n elif not d.blob and d.uri:\n d.load_uri_to_blob()\n return d\n elif d.tensor is not None:\n return d\n else:\n raise TypeError(f'Unsupported input type {d!r} {d.content_type}')\n\n @staticmethod\n def _prepare_rank_doc(d: 'Document', _source: str = 'matches'):\n _get = lambda d: getattr(d, _source)\n if not _get(d):\n raise ValueError(f'`.rank()` requires every doc to have `.{_source}`')\n d = Client._prepare_single_doc(d)\n setattr(d, _source, [Client._prepare_single_doc(c) for c in _get(d)])\n return d\n\n def rank(\n self, docs: Union['DocumentArray', Iterable['Document']], **kwargs\n ) -> 'DocumentArray':\n \"\"\"Rank image-text matches according to the server CLIP model.\n Given a Document with nested matches, where the root is image/text and the matches is in another modality, i.e.\n text/image; this method ranks the matches according to the CLIP model.\n Each match now has a new score inside ``clip_score`` and matches are sorted descendingly according to this score.\n More details can be found in: https://github.com/openai/CLIP#usage\n\n :param docs: the input Documents\n :return: the ranked Documents in a DocumentArray.\n \"\"\"\n if isinstance(docs, str):\n raise TypeError(f'Content must be an Iterable of [Document]')\n\n self._prepare_streaming(\n not kwargs.get('show_progress'),\n total=len(docs) if hasattr(docs, '__len__') else None,\n )\n\n on_done = kwargs.pop('on_done', None)\n on_error = kwargs.pop('on_error', None)\n on_always = kwargs.pop('on_always', None)\n prefetch = kwargs.pop('prefetch', 100)\n results = DocumentArray() if not on_done and not on_always else None\n if not on_done:\n on_done = partial(self._gather_result, results=results, attribute='matches')\n\n with self._pbar:\n parameters = kwargs.pop('parameters', {})\n parameters['drop_image_content'] = parameters.get(\n 'drop_image_content', True\n )\n model_name = parameters.get('model_name', '') if parameters else ''\n\n self._client.post(\n on=f'/rank/{model_name}'.rstrip('/'),\n **self._get_rank_payload(docs, results, **kwargs),\n on_done=on_done,\n on_error=on_error,\n on_always=partial(self._update_pbar, func=on_always),\n parameters=parameters,\n prefetch=prefetch,\n )\n\n return results\n\n async def arank(\n self, docs: Union['DocumentArray', Iterable['Document']], **kwargs\n ) -> 'DocumentArray':\n if isinstance(docs, str):\n raise TypeError(f'Content must be an Iterable of [Document]')\n\n self._prepare_streaming(\n not kwargs.get('show_progress'),\n total=len(docs) if hasattr(docs, '__len__') else None,\n )\n on_done = kwargs.pop('on_done', None)\n on_error = kwargs.pop('on_error', None)\n on_always = kwargs.pop('on_always', None)\n prefetch = kwargs.pop('prefetch', 100)\n results = DocumentArray() if not on_done and not on_always else None\n if not on_done:\n on_done = partial(self._gather_result, results=results, attribute='matches')\n\n with self._pbar:\n parameters = kwargs.pop('parameters', {})\n parameters['drop_image_content'] = parameters.get(\n 'drop_image_content', True\n )\n model_name = parameters.get('model_name', '') if parameters else ''\n\n async for _ in self._async_client.post(\n on=f'/rank/{model_name}'.rstrip('/'),\n **self._get_rank_payload(docs, results, **kwargs),\n on_done=on_done,\n on_error=on_error,\n on_always=partial(self._update_pbar, func=on_always),\n parameters=parameters,\n prefetch=prefetch,\n ):\n continue\n\n return results\n\n @overload\n def index(\n self,\n content: Iterable[str],\n *,\n batch_size: Optional[int] = None,\n show_progress: bool = False,\n parameters: Optional[Dict] = None,\n on_done: Optional['CallbackFnType'] = None,\n on_error: Optional['CallbackFnType'] = None,\n on_always: Optional['CallbackFnType'] = None,\n prefetch: int = 100,\n ):\n \"\"\"Index the images or texts where their embeddings are computed by the server CLIP model.\n\n Each image and text must be represented as a string. The following strings are acceptable:\n - local image filepath, will be considered as an image\n - remote image http/https, will be considered as an image\n - a dataURI, will be considered as an image\n - plain text, will be considered as a sentence\n :param content: an iterator of image URIs or sentences, each element is an image or a text sentence as a string.\n :param batch_size: the number of elements in each request when sending ``content``\n :param show_progress: if set, show a progress bar\n :param parameters: the parameters for the indexing, you can specify the model to use when you have multiple models\n :param on_done: the callback function executed while streaming, after successful completion of each request.\n It takes the response ``DataRequest`` as the only argument\n :param on_error: the callback function executed while streaming, after an error occurs in each request.\n It takes the response ``DataRequest`` as the only argument\n :param on_always: the callback function executed while streaming, after each request is completed.\n It takes the response ``DataRequest`` as the only argument\n :param prefetch: the number of in-flight batches made by the post() method. Use a lower value for expensive\n operations, and a higher value for faster response times\n :return: the embedding in a numpy ndarray with shape ``[N, D]``. ``N`` is in the same length of ``content``\n \"\"\"\n ...\n\n @overload\n def index(\n self,\n content: Union['DocumentArray', Iterable['Document']],\n *,\n batch_size: Optional[int] = None,\n show_progress: bool = False,\n parameters: Optional[dict] = None,\n on_done: Optional['CallbackFnType'] = None,\n on_error: Optional['CallbackFnType'] = None,\n on_always: Optional['CallbackFnType'] = None,\n prefetch: int = 100,\n ) -> 'DocumentArray':\n \"\"\"Index the images or texts where their embeddings are computed by the server CLIP model.\n\n :param content: an iterable of :class:`docarray.Document`, each Document must be filled with `.uri`, `.text` or `.blob`.\n :param batch_size: the number of elements in each request when sending ``content``\n :param show_progress: if set, show a progress bar\n :param parameters: the parameters for the indexing, you can specify the model to use when you have multiple models\n :param on_done: the callback function executed while streaming, after successful completion of each request.\n It takes the response ``DataRequest`` as the only argument\n :param on_error: the callback function executed while streaming, after an error occurs in each request.\n It takes the response ``DataRequest`` as the only argument\n :param on_always: the callback function executed while streaming, after each request is completed.\n It takes the response ``DataRequest`` as the only argument\n :param prefetch: the number of in-flight batches made by the post() method. Use a lower value for expensive\n operations, and a higher value for faster response times\n :return: the embedding in a numpy ndarray with shape ``[N, D]``. ``N`` is in the same length of ``content``\n \"\"\"\n ...\n\n def index(self, content, **kwargs):\n if isinstance(content, str):\n raise TypeError(\n f'content must be an Iterable of [str, Document], try `.index([\"{content}\"])` instead'\n )\n\n self._prepare_streaming(\n not kwargs.get('show_progress'),\n total=len(content) if hasattr(content, '__len__') else None,\n )\n on_done = kwargs.pop('on_done', None)\n on_error = kwargs.pop('on_error', None)\n on_always = kwargs.pop('on_always', None)\n prefetch = kwargs.pop('prefetch', 100)\n results = DocumentArray() if not on_done and not on_always else None\n if not on_done:\n on_done = partial(\n self._gather_result, results=results, attribute='embedding'\n )\n\n with self._pbar:\n parameters = kwargs.pop('parameters', {})\n parameters['drop_image_content'] = parameters.get(\n 'drop_image_content', True\n )\n\n self._client.post(\n on='/index',\n **self._get_post_payload(content, results, **kwargs),\n on_done=on_done,\n on_error=on_error,\n on_always=partial(self._update_pbar, func=on_always),\n parameters=parameters,\n prefetch=prefetch,\n )\n\n return results\n\n @overload\n async def aindex(\n self,\n content: Iterator[str],\n *,\n batch_size: Optional[int] = None,\n show_progress: bool = False,\n parameters: Optional[Dict] = None,\n on_done: Optional['CallbackFnType'] = None,\n on_error: Optional['CallbackFnType'] = None,\n on_always: Optional['CallbackFnType'] = None,\n prefetch: int = 100,\n ):\n ...\n\n @overload\n async def aindex(\n self,\n content: Union['DocumentArray', Iterable['Document']],\n *,\n batch_size: Optional[int] = None,\n show_progress: bool = False,\n parameters: Optional[dict] = None,\n on_done: Optional['CallbackFnType'] = None,\n on_error: Optional['CallbackFnType'] = None,\n on_always: Optional['CallbackFnType'] = None,\n prefetch: int = 100,\n ):\n ...\n\n async def aindex(self, content, **kwargs):\n if isinstance(content, str):\n raise TypeError(\n f'content must be an Iterable of [str, Document], try `.aindex([\"{content}\"])` instead'\n )\n\n self._prepare_streaming(\n not kwargs.get('show_progress'),\n total=len(content) if hasattr(content, '__len__') else None,\n )\n on_done = kwargs.pop('on_done', None)\n on_error = kwargs.pop('on_error', None)\n on_always = kwargs.pop('on_always', None)\n prefetch = kwargs.pop('prefetch', 100)\n results = DocumentArray() if not on_done and not on_always else None\n if not on_done:\n on_done = partial(\n self._gather_result, results=results, attribute='embedding'\n )\n\n with self._pbar:\n parameters = kwargs.pop('parameters', {})\n parameters['drop_image_content'] = parameters.get(\n 'drop_image_content', True\n )\n\n async for _ in self._async_client.post(\n on='/index',\n **self._get_post_payload(content, results, **kwargs),\n on_done=on_done,\n on_error=on_error,\n on_always=partial(self._update_pbar, func=on_always),\n parameters=parameters,\n prefetch=prefetch,\n ):\n continue\n\n return results\n\n @overload\n def search(\n self,\n content: Iterable[str],\n *,\n limit: int = 10,\n batch_size: Optional[int] = None,\n show_progress: bool = False,\n parameters: Optional[Dict] = None,\n on_done: Optional['CallbackFnType'] = None,\n on_error: Optional['CallbackFnType'] = None,\n on_always: Optional['CallbackFnType'] = None,\n prefetch: int = 100,\n ) -> 'DocumentArray':\n \"\"\"Search for top k results for given query string or ``Document``.\n\n If the input is a string, will use this string as query. If the input is a ``Document``,\n will use this ``Document`` as query.\n\n :param content: list of queries.\n :param limit: the number of results to return.\n :param batch_size: the number of elements in each request when sending ``content``.\n :param show_progress: if set, show a progress bar.\n :param parameters: parameters passed to search function.\n :param on_done: the callback function executed while streaming, after successful completion of each request.\n It takes the response ``DataRequest`` as the only argument\n :param on_error: the callback function executed while streaming, after an error occurs in each request.\n It takes the response ``DataRequest`` as the only argument\n :param on_always: the callback function executed while streaming, after each request is completed.\n It takes the response ``DataRequest`` as the only argument\n :param prefetch: the number of in-flight batches made by the post() method. Use a lower value for expensive\n operations, and a higher value for faster response times\n \"\"\"\n ...\n\n @overload\n def search(\n self,\n content: Union['DocumentArray', Iterable['Document']],\n *,\n limit: int = 10,\n batch_size: Optional[int] = None,\n show_progress: bool = False,\n parameters: Optional[dict] = None,\n on_done: Optional['CallbackFnType'] = None,\n on_error: Optional['CallbackFnType'] = None,\n on_always: Optional['CallbackFnType'] = None,\n prefetch: int = 100,\n ) -> 'DocumentArray':\n \"\"\"Search for top k results for given query string or ``Document``.\n\n If the input is a string, will use this string as query. If the input is a ``Document``,\n will use this ``Document`` as query.\n\n :param content: list of queries.\n :param limit: the number of results to return.\n :param batch_size: the number of elements in each request when sending ``content``.\n :param show_progress: if set, show a progress bar.\n :param parameters: parameters passed to search function.\n :param on_done: the callback function executed while streaming, after successful completion of each request.\n It takes the response ``DataRequest`` as the only argument\n :param on_error: the callback function executed while streaming, after an error occurs in each request.\n It takes the response ``DataRequest`` as the only argument\n :param on_always: the callback function executed while streaming, after each request is completed.\n It takes the response ``DataRequest`` as the only argument\n :param prefetch: the number of in-flight batches made by the post() method. Use a lower value for expensive\n operations, and a higher value for faster response times\n \"\"\"\n ...\n\n def search(self, content, limit: int = 10, **kwargs) -> 'DocumentArray':\n if isinstance(content, str):\n raise TypeError(\n f'content must be an Iterable of [str, Document], try `.search([\"{content}\"])` instead'\n )\n\n self._prepare_streaming(\n not kwargs.get('show_progress'),\n total=len(content) if hasattr(content, '__len__') else None,\n )\n on_done = kwargs.pop('on_done', None)\n on_error = kwargs.pop('on_error', None)\n on_always = kwargs.pop('on_always', None)\n prefetch = kwargs.pop('prefetch', 100)\n results = DocumentArray() if not on_done and not on_always else None\n if not on_done:\n on_done = partial(self._gather_result, results=results, attribute='matches')\n\n with self._pbar:\n parameters = kwargs.pop('parameters', {})\n parameters['limit'] = limit\n parameters['drop_image_content'] = parameters.get(\n 'drop_image_content', True\n )\n\n self._client.post(\n on='/search',\n **self._get_post_payload(content, results, **kwargs),\n on_done=on_done,\n on_error=on_error,\n on_always=partial(self._update_pbar, func=on_always),\n parameters=parameters,\n prefetch=prefetch,\n )\n\n return results\n\n @overload\n async def asearch(\n self,\n content: Iterator[str],\n *,\n limit: int = 10,\n batch_size: Optional[int] = None,\n show_progress: bool = False,\n parameters: Optional[Dict] = None,\n on_done: Optional['CallbackFnType'] = None,\n on_error: Optional['CallbackFnType'] = None,\n on_always: Optional['CallbackFnType'] = None,\n prefetch: int = 100,\n ):\n ...\n\n @overload\n async def asearch(\n self,\n content: Union['DocumentArray', Iterable['Document']],\n *,\n limit: int = 10,\n batch_size: Optional[int] = None,\n show_progress: bool = False,\n parameters: Optional[dict] = None,\n on_done: Optional['CallbackFnType'] = None,\n on_error: Optional['CallbackFnType'] = None,\n on_always: Optional['CallbackFnType'] = None,\n prefetch: int = 100,\n ):\n ...\n\n async def asearch(self, content, limit: int = 10, **kwargs):\n if isinstance(content, str):\n raise TypeError(\n f'content must be an Iterable of [str, Document], try `.asearch([\"{content}\"])` instead'\n )\n\n self._prepare_streaming(\n not kwargs.get('show_progress'),\n total=len(content) if hasattr(content, '__len__') else None,\n )\n on_done = kwargs.pop('on_done', None)\n on_error = kwargs.pop('on_error', None)\n on_always = kwargs.pop('on_always', None)\n prefetch = kwargs.pop('prefetch', 100)\n results = DocumentArray() if not on_done and not on_always else None\n if not on_done:\n on_done = partial(self._gather_result, results=results, attribute='matches')\n\n with self._pbar:\n parameters = kwargs.pop('parameters', {})\n parameters['limit'] = limit\n parameters['drop_image_content'] = parameters.get(\n 'drop_image_content', True\n )\n\n async for _ in self._async_client.post(\n on='/search',\n **self._get_post_payload(content, results, **kwargs),\n on_done=on_done,\n on_error=on_error,\n on_always=partial(self._update_pbar, func=on_always),\n parameters=parameters,\n prefetch=prefetch,\n ):\n continue\n\n return results\n" }, { "path": "client/clip_client/helper.py", "content": "import json\nimport sys\nimport threading\nfrom packaging.version import Version\nfrom urllib.request import Request, urlopen\n\nimport pkg_resources\nfrom rich import print\nfrom rich.panel import Panel\n\n\ndef _version_check(package: str = None, github_repo: str = None):\n try:\n\n if not package:\n package = vars(sys.modules[__name__])['__package__']\n if not github_repo:\n github_repo = package\n\n cur_ver = Version(pkg_resources.get_distribution(package).version)\n req = Request(\n f'https://pypi.python.org/pypi/{package}/json',\n headers={'User-Agent': 'Mozilla/5.0'},\n )\n with urlopen(\n req, timeout=1\n ) as resp: # 'with' is important to close the resource after use\n j = json.load(resp)\n releases = j.get('releases', {})\n latest_release_ver = max(\n Version(v) for v in releases.keys() if '.dev' not in v\n )\n if cur_ver < latest_release_ver:\n print(\n Panel(\n f'You are using [b]{package} {cur_ver}[/b], but [bold green]{latest_release_ver}[/] is available. '\n f'You may upgrade it via [b]pip install -U {package}[/b]. [link=https://github.com/jina-ai/{github_repo}/releases]Read Changelog here[/link].',\n title=':new: New version available!',\n width=50,\n )\n )\n except Exception:\n # no network, too slow, PyPi is down\n pass\n\n\ndef is_latest_version(package: str = None, github_repo: str = None) -> None:\n \"\"\"Check if there is a latest version from Pypi, set env `NO_VERSION_CHECK` to disable it.\n\n :param package: package name if none auto-detected\n :param github_repo: repo name that contains CHANGELOG if none then the same as package name\n \"\"\"\n\n threading.Thread(target=_version_check, args=(package, github_repo)).start()\n" }, { "path": "client/setup.py", "content": "import sys\nfrom os import path\n\nfrom setuptools import find_packages\nfrom setuptools import setup\n\nif sys.version_info < (3, 7, 0):\n raise OSError(f'CLIP-as-service requires Python >=3.7, but yours is {sys.version}')\n\ntry:\n pkg_name = 'clip-client'\n libinfo_py = path.join(\n path.dirname(__file__), pkg_name.replace('-', '_'), '__init__.py'\n )\n libinfo_content = open(libinfo_py, 'r', encoding='utf8').readlines()\n version_line = [l.strip() for l in libinfo_content if l.startswith('__version__')][\n 0\n ]\n exec(version_line) # gives __version__\nexcept FileNotFoundError as ex:\n __version__ = '0.0.0'\n\ntry:\n with open('../README.md', encoding='utf8') as fp:\n _long_description = fp.read()\nexcept FileNotFoundError:\n _long_description = ''\n\nsetup(\n name=pkg_name,\n packages=find_packages(),\n version=__version__,\n include_package_data=True,\n description='Embed images and sentences into fixed-length vectors via CLIP',\n author='Jina AI',\n author_email='hello@jina.ai',\n license='Apache 2.0',\n url='https://github.com/jina-ai/clip-as-service',\n download_url='https://github.com/jina-ai/clip-as-service/tags',\n long_description=_long_description,\n long_description_content_type='text/markdown',\n zip_safe=False,\n setup_requires=['setuptools>=18.0', 'wheel'],\n install_requires=[\n 'jina>=3.12.0',\n 'docarray[common]>=0.19.0,<0.30.0',\n 'packaging',\n ],\n extras_require={\n 'test': [\n 'pytest',\n 'pytest-timeout',\n 'pytest-mock',\n 'pytest-asyncio',\n 'pytest-cov',\n 'pytest-repeat',\n 'pytest-reraise',\n 'mock',\n 'pytest-custom_exit_code',\n 'black',\n ],\n },\n classifiers=[\n 'Development Status :: 5 - Production/Stable',\n 'Intended Audience :: Developers',\n 'Intended Audience :: Education',\n 'Intended Audience :: Science/Research',\n 'Programming Language :: Python :: 3.7',\n 'Programming Language :: Python :: 3.8',\n 'Programming Language :: Python :: 3.9',\n 'Programming Language :: Python :: 3.10',\n 'Programming Language :: Unix Shell',\n 'Environment :: Console',\n 'License :: OSI Approved :: Apache Software License',\n 'Operating System :: OS Independent',\n 'Topic :: Database :: Database Engines/Servers',\n 'Topic :: Scientific/Engineering :: Artificial Intelligence',\n 'Topic :: Internet :: WWW/HTTP :: Indexing/Search',\n 'Topic :: Scientific/Engineering :: Image Recognition',\n 'Topic :: Multimedia :: Video',\n 'Topic :: Scientific/Engineering',\n 'Topic :: Scientific/Engineering :: Mathematics',\n 'Topic :: Software Development',\n 'Topic :: Software Development :: Libraries',\n 'Topic :: Software Development :: Libraries :: Python Modules',\n ],\n project_urls={\n 'Documentation': 'https://clip-as-service.jina.ai',\n 'Source': 'https://github.com/jina-ai/clip-as-service/',\n 'Tracker': 'https://github.com/jina-ai/clip-as-service/issues',\n },\n keywords='jina openai clip deep-learning cross-modal multi-modal neural-search',\n)\n" }, { "path": "docs/Makefile", "content": "# Minimal makefile for Sphinx documentation\n# Used only for local building\n\n# You can set these variables from the command line.\nSPHINXOPTS =\nSPHINXBUILD = sphinx-build\nSOURCEDIR = .\nBUILDDIR = _build\n\n# Put it first so that \"make\" without argument is like \"make help\".\nhelp:\n\t@$(SPHINXBUILD) -M help \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)\n\n.PHONY: help Makefile\n\n# Catch-all target: route all unknown targets to Sphinx using the new\n# \"make mode\" option. $(O) is meant as a shortcut for $(SPHINXOPTS).\n%: Makefile\n\t@$(SPHINXBUILD) -M $@ \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)" }, { "path": "docs/_static/cas-grafana.json", "content": "{\n \"__inputs\": [\n {\n \"name\": \"DS_PROMETHEUS\",\n \"label\": \"Prometheus\",\n \"description\": \"\",\n \"type\": \"datasource\",\n \"pluginId\": \"prometheus\",\n \"pluginName\": \"Prometheus\"\n }\n ],\n \"__elements\": [],\n \"__requires\": [\n {\n \"type\": \"grafana\",\n \"id\": \"grafana\",\n \"name\": \"Grafana\",\n \"version\": \"8.5.3\"\n },\n {\n \"type\": \"panel\",\n \"id\": \"piechart\",\n \"name\": \"Pie chart\",\n \"version\": \"\"\n },\n {\n \"type\": \"datasource\",\n \"id\": \"prometheus\",\n \"name\": \"Prometheus\",\n \"version\": \"1.0.0\"\n },\n {\n \"type\": \"panel\",\n \"id\": \"stat\",\n \"name\": \"Stat\",\n \"version\": \"\"\n },\n {\n \"type\": \"panel\",\n \"id\": \"timeseries\",\n \"name\": \"Time series\",\n \"version\": \"\"\n }\n ],\n \"annotations\": {\n \"list\": [\n {\n \"builtIn\": 1,\n \"datasource\": {\n \"type\": \"datasource\",\n \"uid\": \"grafana\"\n },\n \"enable\": true,\n \"hide\": true,\n \"iconColor\": \"rgba(0, 211, 255, 1)\",\n \"name\": \"Annotations & Alerts\",\n \"target\": {\n \"limit\": 100,\n \"matchAny\": false,\n \"tags\": [],\n \"type\": \"dashboard\"\n },\n \"type\": \"dashboard\"\n }\n ]\n },\n \"description\": \"The datashboard for CLIP-as-service\",\n \"editable\": true,\n \"fiscalYearStartMonth\": 0,\n \"graphTooltip\": 0,\n \"id\": null,\n \"iteration\": 1654148217937,\n \"links\": [],\n \"liveNow\": false,\n \"panels\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"viz\": false\n }\n },\n \"mappings\": [],\n \"unit\": \"s\"\n },\n \"overrides\": [\n {\n \"__systemRef\": \"hideSeriesFrom\",\n \"matcher\": {\n \"id\": \"byNames\",\n \"options\": {\n \"mode\": \"exclude\",\n \"names\": [\n \"gateway overhead\",\n \"gateway/worker network\",\n \"processing-\",\n \"preproc text\"\n ],\n \"prefix\": \"All except:\",\n \"readOnly\": true\n }\n },\n \"properties\": [\n {\n \"id\": \"custom.hideFrom\",\n \"value\": {\n \"legend\": false,\n \"tooltip\": false,\n \"viz\": true\n }\n }\n ]\n }\n ]\n },\n \"gridPos\": {\n \"h\": 16,\n \"w\": 13,\n \"x\": 0,\n \"y\": 0\n },\n \"id\": 41,\n \"options\": {\n \"displayLabels\": [\n \"name\"\n ],\n \"legend\": {\n \"displayMode\": \"table\",\n \"placement\": \"right\",\n \"values\": [\n \"value\",\n \"percent\"\n ]\n },\n \"pieType\": \"pie\",\n \"reduceOptions\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"fields\": \"\",\n \"values\": false\n },\n \"tooltip\": {\n \"mode\": \"single\",\n \"sort\": \"none\"\n }\n },\n \"pluginVersion\": \"8.4.4\",\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"exemplar\": true,\n \"expr\": \"jina_receiving_request_seconds_sum / jina_receiving_request_seconds_count\",\n \"hide\": false,\n \"interval\": \"\",\n \"legendFormat\": \"receiving-{{job}}\",\n \"refId\": \"A\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"exemplar\": true,\n \"expr\": \"jina_sending_request_seconds_sum / jina_sending_request_seconds_count\",\n \"hide\": false,\n \"interval\": \"\",\n \"legendFormat\": \"sending-{{job}}\",\n \"refId\": \"D\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"exemplar\": true,\n \"expr\": \"jina_preprocess_texts_seconds_sum / jina_preprocess_texts_seconds_count\",\n \"hide\": false,\n \"interval\": \"\",\n \"legendFormat\": \"preproc text\",\n \"refId\": \"B\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"editorMode\": \"code\",\n \"exemplar\": true,\n \"expr\": \"jina_encode_texts_seconds_sum / jina_encode_texts_seconds_count\",\n \"hide\": false,\n \"interval\": \"\",\n \"legendFormat\": \"encode text\",\n \"range\": true,\n \"refId\": \"C\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"editorMode\": \"code\",\n \"exemplar\": true,\n \"expr\": \"jina_process_request_seconds_sum / jina_process_request_seconds_count\",\n \"hide\": false,\n \"interval\": \"\",\n \"legendFormat\": \"processing-encode\",\n \"range\": true,\n \"refId\": \"E\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"editorMode\": \"code\",\n \"expr\": \"jina_preprocess_images_seconds_sum / jina_preprocess_images_seconds_count\",\n \"hide\": false,\n \"legendFormat\": \"preproc image\",\n \"range\": true,\n \"refId\": \"F\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"editorMode\": \"code\",\n \"expr\": \"jina_encode_images_seconds_sum / jina_encode_images_seconds_count\",\n \"hide\": false,\n \"legendFormat\": \"encode image\",\n \"range\": true,\n \"refId\": \"G\"\n }\n ],\n \"title\": \"life cycle of a request\",\n \"transformations\": [\n {\n \"id\": \"calculateField\",\n \"options\": {\n \"alias\": \"gateway overhead\",\n \"binary\": {\n \"left\": \"receiving-gateway\",\n \"operator\": \"-\",\n \"reducer\": \"sum\",\n \"right\": \"sending-gateway\"\n },\n \"mode\": \"binary\",\n \"reduce\": {\n \"reducer\": \"sum\"\n }\n }\n },\n {\n \"id\": \"calculateField\",\n \"options\": {\n \"alias\": \"worker-overhead\",\n \"binary\": {\n \"left\": \"receiving-exec\",\n \"operator\": \"-\",\n \"reducer\": \"sum\",\n \"right\": \"processing-encode\"\n },\n \"mode\": \"binary\",\n \"reduce\": {\n \"reducer\": \"sum\"\n }\n }\n },\n {\n \"id\": \"calculateField\",\n \"options\": {\n \"alias\": \"text-model-inference\",\n \"binary\": {\n \"left\": \"processing-encode\",\n \"operator\": \"-\",\n \"reducer\": \"sum\",\n \"right\": \"preproc text\"\n },\n \"mode\": \"binary\",\n \"reduce\": {\n \"reducer\": \"sum\"\n }\n }\n },\n {\n \"id\": \"calculateField\",\n \"options\": {\n \"alias\": \"gateway/worker network\",\n \"binary\": {\n \"left\": \"sending-gateway\",\n \"operator\": \"-\",\n \"reducer\": \"sum\",\n \"right\": \"receiving-exec\"\n },\n \"mode\": \"binary\",\n \"reduce\": {\n \"reducer\": \"sum\"\n }\n }\n },\n {\n \"id\": \"calculateField\",\n \"options\": {\n \"alias\": \"visual-model-inference\",\n \"binary\": {\n \"left\": \"processing-encode\",\n \"reducer\": \"sum\",\n \"right\": \"preproc image\"\n },\n \"mode\": \"binary\",\n \"reduce\": {\n \"reducer\": \"sum\"\n }\n }\n }\n ],\n \"type\": \"piechart\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n }\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 6,\n \"x\": 15,\n \"y\": 0\n },\n \"id\": 32,\n \"options\": {\n \"colorMode\": \"value\",\n \"graphMode\": \"area\",\n \"justifyMode\": \"auto\",\n \"orientation\": \"auto\",\n \"reduceOptions\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"fields\": \"\",\n \"values\": false\n },\n \"textMode\": \"auto\"\n },\n \"pluginVersion\": \"8.5.3\",\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"exemplar\": true,\n \"expr\": \"jina_receiving_request_seconds_count{runtime_name=~\\\"gateway.*\\\"}\",\n \"instant\": false,\n \"interval\": \"\",\n \"intervalFactor\": 1,\n \"legendFormat\": \"\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Number of Request processed \",\n \"type\": \"stat\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 0,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"viz\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"auto\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"s\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 15,\n \"x\": 0,\n \"y\": 16\n },\n \"id\": 39,\n \"options\": {\n \"legend\": {\n \"calcs\": [],\n \"displayMode\": \"list\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"single\",\n \"sort\": \"none\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"exemplar\": true,\n \"expr\": \"jina_receiving_request_seconds_sum / jina_receiving_request_seconds_count\",\n \"interval\": \"\",\n \"legendFormat\": \"{{runtime_name}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"jina_receiving_request_seconds_sum\",\n \"type\": \"timeseries\"\n },\n {\n \"collapsed\": false,\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"PBFA97CFB590B2093\"\n },\n \"gridPos\": {\n \"h\": 1,\n \"w\": 24,\n \"x\": 0,\n \"y\": 24\n },\n \"id\": 4,\n \"panels\": [],\n \"repeat\": \"Executor\",\n \"title\": \"$Executor\",\n \"type\": \"row\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\"\n }\n ]\n }\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 5,\n \"w\": 8,\n \"x\": 0,\n \"y\": 25\n },\n \"id\": 2,\n \"options\": {\n \"colorMode\": \"value\",\n \"graphMode\": \"area\",\n \"justifyMode\": \"auto\",\n \"orientation\": \"auto\",\n \"reduceOptions\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"fields\": \"\",\n \"values\": false\n },\n \"textMode\": \"auto\"\n },\n \"pluginVersion\": \"8.5.3\",\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"exemplar\": true,\n \"expr\": \"jina_document_processed_total{runtime_name=\\\"$Executor\\\"}\",\n \"instant\": false,\n \"interval\": \"\",\n \"intervalFactor\": 1,\n \"legendFormat\": \"{{executor_endpoint}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Number of Documents processed per endpoint\",\n \"type\": \"stat\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\"\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n }\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 5,\n \"w\": 8,\n \"x\": 8,\n \"y\": 25\n },\n \"id\": 7,\n \"options\": {\n \"colorMode\": \"value\",\n \"graphMode\": \"area\",\n \"justifyMode\": \"auto\",\n \"orientation\": \"auto\",\n \"reduceOptions\": {\n \"calcs\": [\n \"lastNotNull\"\n ],\n \"fields\": \"\",\n \"values\": false\n },\n \"textMode\": \"auto\"\n },\n \"pluginVersion\": \"8.5.3\",\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"exemplar\": true,\n \"expr\": \"jina_process_request_seconds_count{runtime_name=\\\"$Executor\\\"}\",\n \"instant\": false,\n \"interval\": \"\",\n \"intervalFactor\": 1,\n \"legendFormat\": \"{{executor_endpoint}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Number of requests per endpoint\",\n \"type\": \"stat\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 0,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"viz\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"auto\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\"\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"s\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 6,\n \"w\": 18,\n \"x\": 0,\n \"y\": 30\n },\n \"id\": 12,\n \"options\": {\n \"legend\": {\n \"calcs\": [],\n \"displayMode\": \"list\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"single\",\n \"sort\": \"none\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"exemplar\": true,\n \"expr\": \"jina_process_request_seconds_sum{runtime_name=\\\"$Executor\\\"} / jina_process_request_seconds_count{runtime_name=\\\"$Executor\\\"}\",\n \"interval\": \"\",\n \"legendFormat\": \"{{executor_endpoint}}-process\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Time spend calling the Executor method link the to endpoint\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 0,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"viz\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"auto\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\"\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"s\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 6,\n \"w\": 18,\n \"x\": 0,\n \"y\": 36\n },\n \"id\": 17,\n \"options\": {\n \"legend\": {\n \"calcs\": [],\n \"displayMode\": \"list\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"single\",\n \"sort\": \"none\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"exemplar\": true,\n \"expr\": \"jina_receiving_request_seconds_sum{runtime_name=\\\"$Executor\\\"} / jina_receiving_request_seconds_count{runtime_name=\\\"$Executor\\\"}\",\n \"interval\": \"\",\n \"legendFormat\": \"{{executor_endpoint}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Time spend calling between receiving and responding \",\n \"type\": \"timeseries\"\n }\n ],\n \"refresh\": \"\",\n \"schemaVersion\": 36,\n \"style\": \"dark\",\n \"tags\": [\n \"clip\",\n \"jina\"\n ],\n \"templating\": {\n \"list\": [\n {\n \"current\": {},\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"definition\": \"label_values(jina_document_processed_created,executor_endpoint)\\n\",\n \"description\": \"\",\n \"hide\": 0,\n \"includeAll\": true,\n \"multi\": true,\n \"name\": \"Endpoint\",\n \"options\": [],\n \"query\": {\n \"query\": \"label_values(jina_document_processed_created,executor_endpoint)\\n\",\n \"refId\": \"StandardVariableQuery\"\n },\n \"refresh\": 1,\n \"regex\": \"\",\n \"skipUrlSync\": false,\n \"sort\": 0,\n \"type\": \"query\"\n },\n {\n \"current\": {},\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"${DS_PROMETHEUS}\"\n },\n \"definition\": \"label_values(jina_document_processed_created,runtime_name)\\n\",\n \"description\": \"\",\n \"hide\": 0,\n \"includeAll\": true,\n \"multi\": true,\n \"name\": \"Executor\",\n \"options\": [],\n \"query\": {\n \"query\": \"label_values(jina_document_processed_created,runtime_name)\\n\",\n \"refId\": \"StandardVariableQuery\"\n },\n \"refresh\": 1,\n \"regex\": \"\",\n \"skipUrlSync\": false,\n \"sort\": 0,\n \"type\": \"query\"\n },\n {\n \"current\": {\n \"selected\": false,\n \"text\": \"Prometheus\",\n \"value\": \"Prometheus\"\n },\n \"hide\": 0,\n \"includeAll\": false,\n \"multi\": false,\n \"name\": \"datasource\",\n \"options\": [],\n \"query\": \"prometheus\",\n \"queryValue\": \"\",\n \"refresh\": 1,\n \"regex\": \"\",\n \"skipUrlSync\": false,\n \"type\": \"datasource\"\n }\n ]\n },\n \"time\": {\n \"from\": \"now-5m\",\n \"to\": \"now\"\n },\n \"timepicker\": {},\n \"timezone\": \"\",\n \"title\": \"clip-as-service\",\n \"uid\": \"e_4RtOlnz\",\n \"version\": 3,\n \"weekStart\": \"\"\n}" }, { "path": "docs/_static/demo-embed.html", "content": "\n\n\n
\n\n\n \n\n \n \n \n \n\n \n \n \n \n\n \n \n \n \n\n
Input a sentence (or an image URL)
Click an image to select\n
\n \n
\n
Upload image from local
\n\n
\n\n
\n

Done in {{elapsed}}ms\n \n {{query}}\n

\n \n
\n\n
\n {{value.toString().charAt(3)}}\n
\n\n
\n\n \n \n
\n\n
\n\n\n
\n\n\n\n" }, { "path": "docs/_static/demo-text-rank.html", "content": "\n\n\n
\n \n \n \n \n \n\n \n \n \n \n\n \n \n \n \n\n
Input an image URL
Click an image to select\n
\n \n
\n
Upload image from local
\n\n

\n \n \n

\n
  • \n \n
    • \n\n

    \n

    \n\n
    \n

    Done in {{elapsed}}ms\n \n \n {{query}}\n \n

    \n
    \n\n
    \n

    \n

    \n Showing reasoning results (score in softmax):\n \n \n \n \n \n
    \n {{item.text}}\n {{(item.scores['clip_score'].value * 100).toFixed(2)}}
    \n

    \n
    \n\n\n\n" }, { "path": "docs/_static/main.css", "content": "html.loaded-in-iframe #announcement,\nhtml.loaded-in-iframe #sidebar-drawer,\nhtml.loaded-in-iframe footer,\nhtml.loaded-in-iframe #toc-drawer {\n display: none!important;\n}\n\nhtml.loaded-in-iframe .page .main {\n justify-content: center;\n}\n\n.sidebar-logo {\n max-width: 70%;\n}\n\n\ntable.docutils {\n border: thin;\n}\n\ntable.docutils td, table.docutils th {\n padding: 1rem 1rem;\n}\n\n.highlight {\n background: #f5f5f5;\n}\n\nh1, h2, h3 {\n margin-top: 3rem;\n}\n\n.highlight-console .highlight {\n background: #00232b !important;\n color: whitesmoke;\n}\n\n.highlight-text .highlight {\n background: #00232b !important;\n color: whitesmoke;\n}\n\n.highlight-json .highlight {\n background: #00232b !important;\n color: whitesmoke;\n}\n\n.highlight-shell .highlight {\n background: #00232b !important;\n color: whitesmoke;\n}\n\n.highlight-bash .highlight {\n background: #00232b !important;\n color: whitesmoke;\n}\n\n.tab-set > input:checked + label {\n border-color: var(--tabs--label-text--active);\n}\n\n.tab-set > input:checked + label:hover {\n border-color: var(--tabs--label-text--active);\n}\n\n\ntable code {\n background: var(--color-inline-code-background);\n border: 1px solid var(--color-background-border);\n border-radius: .2em;\n font-size: var(--font-size--small--2);\n padding: .1em .2em;\n}\n\n.related-information {\n justify-content: space-between;\n}\n\n.social-btn {\n margin: 0 .3em;\n}\n\n.social-btn:hover {\n opacity: .5;\n}\n\n.social-btns {\n display: inline-block;\n}\n\n.announcement {\n background-color: var(--color-brand-primary);\n color: var(--color-background-primary) !important;\n}\n\n.announcement a {\n color: inherit;\n text-decoration: none;\n}\n\n.announcement a:hover {\n color: inherit;\n text-decoration: underline;\n}\n\n.sidebar-ecosys-logo {\n width: 1.2em;\n margin-right: .5em;\n vertical-align: middle\n}\n\n\nbody[data-theme=\"dark\"] .only-dark-line {\n display: inline-block !important;\n}\n\nbody[data-theme=\"dark\"] .only-light-line {\n display: none !important;\n}\n\nbody[data-theme=\"light\"] .only-light-line {\n display: inline-block !important;\n}\n\nbody[data-theme=\"light\"] .only-dark-line {\n display: none !important;\n}\n\nbody[data-theme=\"auto\"] .only-light-line {\n display: inline-block !important;\n}\n\nbody[data-theme=\"auto\"] .only-dark-line {\n display: none !important;\n}\n\n.color-gradient-card {\n background: linear-gradient(270deg, #22c1c3, #fdbb2d);\n background-size: 200% 200%;\n\n -webkit-animation: AnimationName 30s ease infinite;\n -moz-animation: AnimationName 30s ease infinite;\n animation: AnimationName 30s ease infinite;\n}\n\n@-webkit-keyframes AnimationName {\n 0%{background-position:0% 50%}\n 50%{background-position:100% 50%}\n 100%{background-position:0% 50%}\n}\n@-moz-keyframes AnimationName {\n 0%{background-position:0% 50%}\n 50%{background-position:100% 50%}\n 100%{background-position:0% 50%}\n}\n@keyframes AnimationName {\n 0%{background-position:0% 50%}\n 50%{background-position:100% 50%}\n 100%{background-position:0% 50%}\n}\n\n.version-select {\n font-size: .7em;\n border-radius: 5px;\n cursor: pointer;\n background-color: #fff;\n background-image: linear-gradient(to top, #f9f9f9, #fff 33%);\n border-color: var(--color-background-border);\n height: 1.8em;\n line-height: 1.8em;\n outline: none;\n text-align: center;\n max-width: 7em;\n color: var(--color-foreground-muted);\n}" }, { "path": "docs/_templates/page.html", "content": "{% extends \"base.html\" %}\n\n{% block body -%}\n{{ super() }}\n{% include \"partials/icons.html\" %}\n\n\n\n\n\n\n{% if theme_announcement -%}\n
    \n \n
    \n{%- endif %}\n\n
    \n
    \n
    \n \n
    \n
    \n \n
    {{ docstitle if docstitle else project }}
    \n     \n
    \n
    \n
    \n \n
    \n \n
    \n
    \n \n
    \n
    \n
    \n \n \n \n \n {% trans %}Back to top{% endtrans %}\n \n
    \n {#- Edit this page, on GitHub -#}\n {%- if READTHEDOCS and conf_py_path and page_source_suffix and github_user != \"None\" and github_repo\n != \"None\" and github_version %}\n
    \n \n \n \n \n \n \n {{ _(\"Edit this page\") }}\n \n
    \n {% endif %}\n {#- Theme toggle -#}\n
    \n \n
    \n \n
    \n
    \n {% block content %}{{ body }}{% endblock %}\n
    \n
    \n \n
    \n \n
    \n
    \n\n{%- endblock %}" }, { "path": "docs/_templates/sidebar/brand.html", "content": "\n {% block brand_content %}\n {%- if logo_url %}\n
    \n \"Logo\"\n
    \n {%- endif %}\n {%- if theme_light_logo and theme_dark_logo %}\n
    \n \n \n
    \n {%- endif %}\n {% if not theme_sidebar_hide_name %}\n {{ docstitle if docstitle else project }}\n {%- endif %}\n {% endblock brand_content %}\n    \n
    \n Star\n {% if versions %}\n \n {% endif %}\n
    \n" }, { "path": "docs/_templates/sidebar/navigation.html", "content": "
    \n {{ furo_navigation_tree }}\n

    Ecosystem

    \n \n
    \n" }, { "path": "docs/changelog/index.md", "content": "# Changelog\n\nCLIP-as-service follows semantic versioning. However, before the project reach 1.0.0, any breaking change will only bump the minor version. An automated release note is [generated on every release](https://github.com/jina-ai/clip-as-service/releases). The release note includes features, bugs, refactorings etc. \n\nThis chapter only tracks the most important breaking changes and explain the rationale behind them.\n\n## 0.4.0: rename `rerank` concept to `rank`\n\n\"Reranking\" is a new feature introduced since 0.3.3. This feature allows user to rank and score `document.matches` in a cross-modal way. From 0.4.0, this feature as well as all related functions will refer it simply as \"rank\".\n\n## 0.2.0: improve the service scalability with replicas\n\nThis change is mainly intended to improve the inference performance with replicas.\n\nHere is the short benchmark summary of the improvement (`replicas=4`):\n\n| batch_size | before | after |\n|-------------|--------|---------|\n| 1 | 23.74 | 18.89 |\n| 8 | 58.88 | 30.38 |\n| 16 | 14.96 | 91.86 |\n| 32 | 14.78 | 101.75 |\n" }, { "path": "docs/conf.py", "content": "import os\nimport re\nimport sys\nfrom os import path\n\nsys.path.insert(0, path.abspath('..'))\n\nproject = 'CLIP-as-service'\nslug = re.sub(r'\\W+', '-', project.lower())\nauthor = 'Jina AI'\ncopyright = 'Jina AI Limited. All rights reserved.'\nsource_suffix = ['.rst', '.md']\nmaster_doc = 'index'\nlanguage = 'en'\nrepo_dir = '../'\n\ntry:\n if 'CAS_VERSION' not in os.environ:\n libinfo_py = path.join(repo_dir, 'client/clip_client', '__init__.py')\n libinfo_content = open(libinfo_py, 'r').readlines()\n version_line = [\n l.strip() for l in libinfo_content if l.startswith('__version__')\n ][0]\n exec(version_line)\n else:\n __version__ = os.environ['CAS_VERSION']\nexcept FileNotFoundError:\n __version__ = '0.0.0'\n\nversion = __version__\nrelease = __version__\n\ntemplates_path = ['_templates']\nexclude_patterns = [\n '_build',\n 'Thumbs.db',\n '.DS_Store',\n 'tests',\n 'page_templates',\n '.github',\n]\npygments_style = 'rainbow_dash'\nhtml_theme = 'furo'\n\nbase_url = '/'\nhtml_baseurl = 'https://clip-as-service.jina.ai'\nsitemap_url_scheme = '{link}'\nsitemap_locales = [None]\nsitemap_filename = \"sitemap.xml\"\n\nhtml_theme_options = {\n 'light_logo': 'logo-light.svg',\n 'dark_logo': 'logo-dark.svg',\n \"sidebar_hide_name\": True,\n \"light_css_variables\": {\n \"color-brand-primary\": \"#009191\",\n \"color-brand-content\": \"#009191\",\n },\n \"dark_css_variables\": {\n \"color-brand-primary\": \"#FBCB67\",\n \"color-brand-content\": \"#FBCB67\",\n },\n # PLEASE DO NOT DELETE the empty line between `start-announce` and `end-announce`\n # PLEASE DO NOT DELETE `start-announce`/ `end-announce` it is used for our dev bot to inject announcement from GH\n # start-announce\n # end-announce\n}\n\nhtml_static_path = ['_static']\nhtml_extra_path = ['html_extra']\nhtml_css_files = [\n 'main.css',\n 'https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0-beta2/css/all.min.css',\n]\nhtml_js_files = [\n 'https://cdn.jsdelivr.net/npm/vue@2/dist/vue.min.js',\n]\nhtmlhelp_basename = slug\nhtml_show_sourcelink = False\nhtml_favicon = '_static/favicon.png'\n\nintersphinx_mapping = {'docarray': ('https://docarray.jina.ai/', None), 'finetuner': ('https://finetuner.jina.ai/', None)}\n\nlatex_documents = [(master_doc, f'{slug}.tex', project, author, 'manual')]\nman_pages = [(master_doc, slug, project, [author], 1)]\ntexinfo_documents = [\n (master_doc, slug, project, author, slug, project, 'Miscellaneous')\n]\nepub_title = project\nepub_exclude_files = ['search.html']\n\n# -- Extension configuration -------------------------------------------------\n\nextensions = [\n 'sphinx.ext.autodoc',\n 'sphinx_autodoc_typehints',\n 'sphinx.ext.viewcode',\n 'sphinx.ext.coverage',\n 'sphinxcontrib.apidoc',\n 'sphinxarg.ext',\n 'sphinx_copybutton',\n 'sphinx_sitemap',\n 'sphinx.ext.intersphinx',\n 'sphinxext.opengraph',\n 'notfound.extension',\n 'myst_parser',\n 'sphinx_design',\n 'sphinx_inline_tabs',\n]\n\nmyst_enable_extensions = ['colon_fence', 'substitution', 'deflist']\n\n# -- Custom 404 page\n\n# sphinx-notfound-page\n# https://github.com/readthedocs/sphinx-notfound-page\nnotfound_context = {\n 'title': 'Page Not Found',\n 'body': '''\n

    Page Not Found

    \n

    Oops, we couldn't find that page.

    \n

    You can try \"asking our docs\" on the right corner of the page to find answer.

    \n

    Otherwise, please create a Github issue and one of our team will respond.

    \n\n''',\n}\nnotfound_no_urls_prefix = True\n\napidoc_module_dir = '../client'\napidoc_output_dir = 'api'\napidoc_excluded_paths = ['tests', 'legacy', 'hub', 'toy*', 'setup.py']\napidoc_separate_modules = True\napidoc_extra_args = ['-t', 'template/']\nautodoc_member_order = 'bysource'\nautodoc_mock_imports = ['argparse', 'numpy', 'np', 'tensorflow', 'torch', 'scipy']\nautoclass_content = 'both'\nset_type_checking_flag = False\nhtml_last_updated_fmt = ''\nnitpicky = True\nnitpick_ignore = [('py:class', 'type')]\nlinkcheck_ignore = [\n # Avoid link check on local uri\n 'http://0.0.0.0:*',\n 'pods/encode.yml',\n 'https://github.com/jina-ai/clip-as-service/commit/*',\n '.github/*',\n 'extra-requirements.txt',\n 'fastentrypoints.py' '../../101',\n '../../102',\n 'http://www.twinsun.com/tz/tz-link.htm', # Broken link from pytz library\n 'https://urllib3.readthedocs.io/en/latest/contrib.html#google-app-engine', # Broken link from urllib3 library\n 'https://linuxize.com/post/how-to-add-swap-space-on-ubuntu-20-04/',\n # This link works but gets 403 error on linkcheck\n]\nlinkcheck_timeout = 20\nlinkcheck_retries = 2\nlinkcheck_anchors = False\n\nogp_site_url = 'https://clip-as-service.jina.ai/'\nogp_image = 'https://clip-as-service.jina.ai/_static/banner.png'\nogp_use_first_image = True\nogp_description_length = 300\nogp_type = 'website'\nogp_site_name = f'CLIP-as-service {os.environ.get(\"SPHINX_MULTIVERSION_VERSION\", version)} Documentation'\n\nogp_custom_meta_tags = [\n '',\n '',\n '',\n '',\n '',\n '''\n\n\n\n\n\n ''',\n]\n\n\ndef add_server_address(app):\n # This makes variable `server_address` available to docbot.js\n server_address = app.config['server_address']\n js_text = \"var server_address = '%s';\" % server_address\n app.add_js_file(None, body=js_text)\n\n\ndef setup(app):\n from sphinx.domains.python import PyField\n from sphinx.util.docfields import Field\n from sphinx.locale import _\n\n app.add_object_type(\n 'confval',\n 'confval',\n objname='configuration value',\n indextemplate='pair: %s; configuration value',\n doc_field_types=[\n PyField(\n 'type',\n label=_('Type'),\n has_arg=False,\n names=('type',),\n bodyrolename='class',\n ),\n Field(\n 'default',\n label=_('Default'),\n has_arg=False,\n names=('default',),\n ),\n ],\n )\n" }, { "path": "docs/hosting/by-jina.md", "content": "# Hosted by Jina AI\n\n```{include} ../../README.md\n:start-after: \n:end-before: \n```\n\nIn today's dynamic business environment, enterprises face a multitude of challenges that require advanced solutions to \nmaintain a competitive edge. \nFrom managing vast amounts of unstructured data to delivering personalized customer experiences, businesses need \nefficient tools to tackle these obstacles. \nMachine learning (ML) has emerged as a powerful tool for automating repetitive tasks, processing data effectively, and \ngenerating valuable insights from multimedia content. \nJina AI's Inference offers a comprehensive solution to streamline access to curated, state-of-the-art ML models, \neliminating traditional roadblocks such as costly and time-consuming MLOps steps and the distinction between public and \ncustom neural network models.\n\n## Getting started\n\nTo access the fastest and most performant CLIP models, [Jina AI's Inference](https://cloud.jina.ai/user/inference) is \nthe go-to choice. \nFollow the steps below to get started:\n\n1. Sign up for a free account at [Jina AI Cloud](https://cloud.jina.ai).\n2. Once you have created an account, navigate to the Inference tab to create a new CLIP model.\n3. The model can be accessed either through an HTTP endpoint or a gRPC endpoint.\n\n## Obtaining a Personal Access Token\n\nBefore you begin using [Jina AI's Inference](https://cloud.jina.ai/user/inference), ensure that you have obtained a \npersonal access token (PAT) from the [Jina AI Cloud](https://cloud.jina.ai) or through the command-line interface (CLI). \nUse the following guide to create a new PAT:\n\n1. Access the [Jina AI Cloud](https://cloud.jina.ai) and log in to your account.\n2. Navigate to the [**Access token**](https://cloud.jina.ai/settings/tokens) section in the **Settings** tab, or alternatively, create a PAT via the CLI using the command:\n\n```bash\njina auth token create -e \n```\n\n## Installing the Inference Client\n\nTo interact with the model created in Inference, you will need to install the `inference-client` Python package. \nFollow the steps below to install the package using pip:\n\n```bash\npip install inference-client\n```\n\n## Interacting with the Model\n\nOnce you have your personal access token and the model name listed in the Inference detail page, you can start \ninteracting with the model using the `inference-client` Python package. \nFollow the example code snippet below:\n\n```python\nfrom inference_client import Client\n\nclient = Client(token='')\n\nmodel = client.get_model('')\n```\n\nThe CLIP models offer the following functionalities:\n\n1. Encoding: Users can encode data by calling the `model.encode` method. For detailed instructions on using this method, refer to the [Encode documentation](https://jina.readme.io/docs/encode).\n2. Ranking: Users can perform ranking by calling the `model.rank` method. Refer to the [Rank documentation](https://jina.readme.io/docs/rank) for detailed instructions on using this method.\n\nFor further details on usage and information about other tasks and models supported in Inference, as well as how to use \n`curl` to interact with the model, please consult the [Inference documentation](https://jina.readme.io/docs/inference).\n" }, { "path": "docs/hosting/cas-on-colab.ipynb", "content": "{\n \"nbformat\": 4,\n \"nbformat_minor\": 0,\n \"metadata\": {\n \"colab\": {\n \"name\": \"cas-on-colab.ipynb\",\n \"provenance\": [],\n \"collapsed_sections\": []\n },\n \"kernelspec\": {\n \"name\": \"python3\",\n \"display_name\": \"Python 3\"\n },\n \"language_info\": {\n \"name\": \"python\"\n },\n \"accelerator\": \"GPU\"\n },\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"# Hosting CLIP-as-service on Google Colab with TPU/GPU support\\n\",\n \"\\n\",\n \"This tutorial guides you on how to implement the following architecture:\\n\",\n \"\\n\",\n \"[![](https://mermaid.ink/img/pako:eNp1kEFrwzAMhf-K0bkh99xGVwpjh9Ctp7oMxVYTM8cOttwy2v732fMGgzFd9Hjvk0C6gvKaoIMx4DKJ5510IldMQzW23o-WxNpbHMRBlXasSCmF8S1SOFNommbb79vXfl9TcrqKh0OBlDXk-Cgydht3_bqdmJf2QkP06p349mtTHXvCM0YVzMJfMwX_C6kU7L8xrGCmMKPR-bprcSTwRDNJ6LLUdMJkWYJ094ymRSPTRhv2AboT2kgrwMT-5cMp6Dgk-oEeDebfzN_U_RP7v2yd)](https://mermaid.live/edit#pako:eNp1kEFrwzAMhf-K0bkh99xGVwpjh9Ctp7oMxVYTM8cOttwy2v732fMGgzFd9Hjvk0C6gvKaoIMx4DKJ5510IldMQzW23o-WxNpbHMRBlXasSCmF8S1SOFNommbb79vXfl9TcrqKh0OBlDXk-Cgydht3_bqdmJf2QkP06p349mtTHXvCM0YVzMJfMwX_C6kU7L8xrGCmMKPR-bprcSTwRDNJ6LLUdMJkWYJ094ymRSPTRhv2AboT2kgrwMT-5cMp6Dgk-oEeDebfzN_U_RP7v2yd)\\n\",\n \"\\n\",\n \"CLIP-as-service is powered by Jina, [there is another tutorial showing you how to host Jina service on Colab in general](https://colab.research.google.com/github/jina-ai/jina/blob/master/docs/Using_Jina_on_Colab.ipynb). Highly recommended!\\n\",\n \"\\n\",\n \"\\n\",\n \"## 1. Change runtime type\\n\",\n \"\\n\",\n \"Go to menu `Runtime -> Change run time type -> GPU/TPU`\\n\",\n \"\\n\",\n \"\\n\",\n \"## 2. Install Packages\\n\",\n \"\\n\",\n \"As we will run the client locally, we only need to install `clip_server` package on Colab.\\n\",\n \"\\n\",\n \"\\n\",\n \"**โš ๏ธ You will be asked to \\\"Restart Runtime\\\" after this step, please click the button and restart the runtime.**\"\n ],\n \"metadata\": {\n \"id\": \"lbUpcvs1p1CF\",\n \"pycharm\": {\n \"name\": \"#%% md\\n\"\n }\n }\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {\n \"id\": \"MRrB2If6kDfX\",\n \"pycharm\": {\n \"name\": \"#%%\\n\"\n }\n },\n \"outputs\": [],\n \"source\": [\n \"!pip install clip_server pyngrok\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"## 3. Config Flow YAML\\n\",\n \"\\n\",\n \"\\n\",\n \"Unlike classic entrypoint from CLI, here we need to start the Flow in Python. Let's load use Pytorch backend and write a Flow YAML. Note that we need to load the torch Python file from `clip_server` installation, hence you see `cas_path` below. More available options [can be found here](https://github.com/jina-ai/clip-as-service/tree/main/server/clip_server/executors).\"\n ],\n \"metadata\": {\n \"id\": \"q3bmGKIvx5S-\",\n \"pycharm\": {\n \"name\": \"#%% md\\n\"\n }\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"import clip_server\\n\",\n \"cas_path = clip_server.__path__[0]\"\n ],\n \"metadata\": {\n \"id\": \"nypR4g9EmgOj\",\n \"pycharm\": {\n \"name\": \"#%%\\n\"\n }\n },\n \"execution_count\": 1,\n \"outputs\": []\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"This YAML is directly [taken from this file](https://github.com/jina-ai/clip-as-service/blob/main/server/clip_server/torch-flow.yml). You can also customize it as you wish, [please check CLIP-as-service docs](https://clip-as-service.jina.ai/user-guides/server/#yaml-config).\"\n ],\n \"metadata\": {\n \"id\": \"5RVA1OD8ywOo\",\n \"pycharm\": {\n \"name\": \"#%% md\\n\"\n }\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"flow_yaml = f'''\\n\",\n \"jtype: Flow\\n\",\n \"with:\\n\",\n \" port: 51000\\n\",\n \"executors:\\n\",\n \" - name: clip_t\\n\",\n \" uses:\\n\",\n \" jtype: CLIPEncoder\\n\",\n \" metas:\\n\",\n \" py_modules:\\n\",\n \" - {cas_path}/executors/clip_torch.py\\n\",\n \"'''\"\n ],\n \"metadata\": {\n \"id\": \"q1BXWnXVkIZ8\",\n \"pycharm\": {\n \"name\": \"#%%\\n\"\n }\n },\n \"execution_count\": 2,\n \"outputs\": []\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"flow_yaml\"\n ],\n \"metadata\": {\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\",\n \"height\": 53\n },\n \"id\": \"Fb1PKf992rLj\",\n \"outputId\": \"a06b634a-5021-4b24-f3dc-a2c6b1d87524\",\n \"pycharm\": {\n \"name\": \"#%%\\n\"\n }\n },\n \"execution_count\": 3,\n \"outputs\": [\n {\n \"output_type\": \"execute_result\",\n \"data\": {\n \"text/plain\": [\n \"'\\\\njtype: Flow\\\\nwith:\\\\n port: 51000\\\\nexecutors:\\\\n - name: clip_t\\\\n uses:\\\\n jtype: CLIPEncoder\\\\n metas:\\\\n py_modules:\\\\n - /usr/local/lib/python3.7/dist-packages/clip_server/executors/clip_torch.py\\\\n'\"\n ],\n \"application/vnd.google.colaboratory.intrinsic+json\": {\n \"type\": \"string\"\n }\n },\n \"metadata\": {},\n \"execution_count\": 3\n }\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"## 4. Start the Flow\\n\",\n \"\\n\",\n \"It may take a minute or so on the first start, as it will download the pretrained models. To select different pretrained models, [please check CLIP-as-service docs](https://clip-as-service.jina.ai/user-guides/server/#yaml-config).\"\n ],\n \"metadata\": {\n \"id\": \"GvAeaUf4y88e\",\n \"pycharm\": {\n \"name\": \"#%% md\\n\"\n }\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"from jina import Flow\\n\",\n \"\\n\",\n \"f = Flow.load_config(flow_yaml)\\n\",\n \"f.start()\"\n ],\n \"metadata\": {\n \"id\": \"4UubypFpl8-K\",\n \"pycharm\": {\n \"name\": \"#%%\\n\"\n }\n },\n \"execution_count\": null,\n \"outputs\": []\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"Remember to close it via `f.close()` when you don't use it. But let's keep it open for now.\"\n ],\n \"metadata\": {\n \"id\": \"2BOYxmpd8YSE\",\n \"pycharm\": {\n \"name\": \"#%% md\\n\"\n }\n }\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"## 5. Set up forwarding\\n\",\n \"\\n\",\n \"By default Flow uses gRPC protocol, it is highly-efficient and feature-rich. So in this tutorial, we will use gRPC protocol and use `ngrok` for forwarding. It is possible and in fact slighly easier to set up when using `Flow(protocol='http')`, [please read the turorial here](https://colab.research.google.com/github/jina-ai/jina/blob/master/docs/Using_Jina_on_Colab.ipynb#scrollTo=0ASjGLBhXono) here I won't repeat again.\\n\",\n \"\\n\",\n \"\\n\",\n \"You will need to first sign up at https://dashboard.ngrok.com/signup (http do not need register, that's why I said it is easier)\\n\",\n \"\\n\",\n \"After signing up, you can get a token. Then simply add your token via (replacing `YOUR_TOKEN_HERE`)\"\n ],\n \"metadata\": {\n \"id\": \"1lTqYEwezDTP\",\n \"pycharm\": {\n \"name\": \"#%% md\\n\"\n }\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"!pip install pyngrok\\n\",\n \"\\n\",\n \"# remember to replace to your token! otherwise i can see your service, i mean i dont really have time to see it but nonetheless\\n\",\n \"!ngrok authtoken 2ARsKtGKj47h7y4uXMQPrIeOinS_47Mkh6jkzNjFEJWuZYNEX\"\n ],\n \"metadata\": {\n \"id\": \"PYQPKek-oG1a\",\n \"pycharm\": {\n \"name\": \"#%%\\n\"\n }\n },\n \"execution_count\": null,\n \"outputs\": []\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"!ngrok tcp 51000 --log \\\"stdout\\\"\"\n ],\n \"metadata\": {\n \"id\": \"2Hacpj4qn9nx\",\n \"pycharm\": {\n \"name\": \"#%%\\n\"\n }\n },\n \"execution_count\": null,\n \"outputs\": []\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"At the last line, you should see something like: \\n\",\n \"\\n\",\n \"```\\n\",\n \"t=2022-06-11T20:29:11+0000 lvl=info msg=\\\"started tunnel\\\" obj=tunnels name=command_line addr=//localhost:54321 url=tcp://6.tcp.ngrok.io:18096\\n\",\n \"```\\n\",\n \"\\n\",\n \"Grab the text after `url=tcp://` in my case it is `6.tcp.ngrok.io:18096`.\\n\",\n \"\\n\",\n \"Now build a client using this address from your local laptop/Python environment.\\n\",\n \"\\n\",\n \"Copy paste the code below to your local Python, remmeber to change your address.\\n\",\n \"\\n\",\n \"**Remember, if your last line is `url=tcp://6.tcp.ngrok.io:18096` then you should set `Client('grpc://6.tcp.ngrok.io:18096')`**\\n\",\n \"\\n\",\n \"### Try Embedding Task from Local\\n\",\n \"\\n\",\n \"```python\\n\",\n \"# pip install clip-client\\n\",\n \"from clip_client import Client\\n\",\n \"\\n\",\n \"c = Client('grpc://6.tcp.ngrok.io:18096')\\n\",\n \"\\n\",\n \"r = c.encode(\\n\",\n \" [\\n\",\n \" 'First do it',\\n\",\n \" 'then do it right',\\n\",\n \" 'then do it better',\\n\",\n \" 'https://picsum.photos/200',\\n\",\n \" ]\\n\",\n \")\\n\",\n \"print(r)\\n\",\n \"```\\n\",\n \"\\n\",\n \"And you will get \\n\",\n \"\\n\",\n \"```text\\n\",\n \"[[ 0.03494263 -0.23510742 0.0104599 ... -0.5229492 -0.10021973\\n\",\n \" -0.08685303]\\n\",\n \" [-0.06793213 -0.0032444 0.01506805 ... -0.50341797 -0.06143188\\n\",\n \" -0.08520508]\\n\",\n \" [ 0.15063477 -0.07922363 -0.06530762 ... -0.46484375 -0.08526611\\n\",\n \" 0.04324341]\\n\",\n \" [-0.16088867 0.10552979 -0.20581055 ... -0.41381836 0.19543457\\n\",\n \" 0.05718994]]\\n\",\n \"```\\n\",\n \"\\n\",\n \"Showing the connection is success!\\n\",\n \"\\n\",\n \"\\n\",\n \"### Try Ranking Task from Local\\n\",\n \"\\n\",\n \"```python\\n\",\n \"from docarray import Document\\n\",\n \"\\n\",\n \"from clip_client import Client\\n\",\n \"\\n\",\n \"c = Client(server='grpc://6.tcp.ngrok.io:18096/rank')\\n\",\n \"\\n\",\n \"r = c.rank(\\n\",\n \" [\\n\",\n \" Document(\\n\",\n \" uri='https://picsum.photos/id/1/300/300',\\n\",\n \" matches=[\\n\",\n \" Document(text=f'a photo of a {p}')\\n\",\n \" for p in (\\n\",\n \" 'man',\\n\",\n \" 'woman',\\n\",\n \" )\\n\",\n \" ],\\n\",\n \" )\\n\",\n \" ]\\n\",\n \")\\n\",\n \"\\n\",\n \"print(r['@m', ['text', 'scores']])\\n\",\n \"```\\n\",\n \"\\n\",\n \"```\\n\",\n \"[['a photo of a man', 'a photo of a woman'], [defaultdict(, {'clip_score': {'value': 0.5806832313537598, 'op_name': 'softmax'}, 'clip_score_cosine': {'value': 0.2178003191947937, 'op_name': 'cosine'}}), defaultdict(, {'clip_score': {'value': 0.41931676864624023, 'op_name': 'softmax'}, 'clip_score_cosine': {'value': 0.21454453468322754, 'op_name': 'cosine'}})]]\\n\",\n \"```\\n\",\n \"\\n\",\n \"\\n\",\n \"Now enjoy the free GPU/TPU to build your awesome CAS applications!\"\n ],\n \"metadata\": {\n \"id\": \"Fzxt8j3Bz9Nu\",\n \"pycharm\": {\n \"name\": \"#%% md\\n\"\n }\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"f.close()\"\n ],\n \"metadata\": {\n \"id\": \"wzj0pb7qo56c\",\n \"pycharm\": {\n \"name\": \"#%%\\n\"\n }\n },\n \"execution_count\": 11,\n \"outputs\": []\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"# Push to the Limit\\n\",\n \"\\n\",\n \"Now let's use the biggest `ViT-L/14-336px` and fully leverage all VRAM with 4 replicas, lets see if it works.\\t\"\n ],\n \"metadata\": {\n \"id\": \"c6yNVg69-vaw\",\n \"pycharm\": {\n \"name\": \"#%% md\\n\"\n }\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"flow_yaml = f'''\\n\",\n \"jtype: Flow\\n\",\n \"with:\\n\",\n \" port: 51000\\n\",\n \"executors:\\n\",\n \" - name: clip_t\\n\",\n \" uses:\\n\",\n \" jtype: CLIPEncoder\\n\",\n \" metas:\\n\",\n \" py_modules:\\n\",\n \" - {cas_path}/executors/clip_torch.py\\n\",\n \" replicas: 4\\n\",\n \"'''\"\n ],\n \"metadata\": {\n \"id\": \"uHHWk3WF_DaO\",\n \"pycharm\": {\n \"name\": \"#%%\\n\"\n }\n },\n \"execution_count\": 12,\n \"outputs\": []\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"from jina import Flow\\n\",\n \"\\n\",\n \"f = Flow.load_config(flow_yaml)\\n\",\n \"f.start()\"\n ],\n \"metadata\": {\n \"id\": \"0AGcGasu_JIv\",\n \"pycharm\": {\n \"name\": \"#%%\\n\"\n }\n },\n \"execution_count\": null,\n \"outputs\": []\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"!ngrok tcp 51000 --log \\\"stdout\\\"\"\n ],\n \"metadata\": {\n \"id\": \"DQzvwOF3_K6U\",\n \"pycharm\": {\n \"name\": \"#%%\\n\"\n }\n },\n \"execution_count\": null,\n \"outputs\": []\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"Yay it works!\"\n ],\n \"metadata\": {\n \"id\": \"8T2z6HXd_hKB\",\n \"pycharm\": {\n \"name\": \"#%% md\\n\"\n }\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [],\n \"metadata\": {\n \"id\": \"4-y_vbHW_acV\",\n \"pycharm\": {\n \"name\": \"#%%\\n\"\n }\n },\n \"execution_count\": null,\n \"outputs\": []\n }\n ]\n}" }, { "path": "docs/hosting/colab.md", "content": "# Host on Google Colab\n\n```{figure} https://clip-as-service.jina.ai/_images/colab-banner.png\n:width: 0 %\n:scale: 0 %\n```\n\n```{figure} colab-banner.png\n:scale: 0 %\n:width: 0 %\n```\n\n\nAs [Jina is fully compatible to Google Colab](https://docs.jina.ai/how-to/google-colab/), CLIP-as-service can be run smoothly on Colab as well. One can host `clip_server` on Google Colab by leveraging its free GPU/TPU resources and open up to 4 replicas of `ViT-L/14-336px`. Then you can send request from local to the server for embedding, ranking and reasoning tasks. \n\nSpecifically, the architecture is illustrated below:\n\n```{figure} cas-on-colab.svg\n:width: 70%\n```\n\n```{button-link} https://colab.research.google.com/github/jina-ai/clip-as-service/blob/main/docs/hosting/cas-on-colab.ipynb\n:color: primary\n:align: center\n\n{octicon}`link-external` Open the notebook on Google Colab \n```\n\nPlease follow the walk-through there. Enjoy the free GPU/TPU to build your awesome Jina applications!\n\n\n```{tip}\nHosing service on Google Colab is not recommended if you server aims to be long-live or permanent. It is often used for quick experiment, demonstration or leveraging its free GPU/TPU. For stable, please deploy the CLIP model on your own server.\n```\n\n\n\n" }, { "path": "docs/hosting/on-jcloud.md", "content": "# Host on JCloud\n\nEssentially `clip_server` is a Jina [Flow](https://docs.jina.ai/fundamentals/flow/). Any Jina Flow can be hosted on [JCloud](https://docs.jina.ai/fundamentals/jcloud/), hence `clip_server` can be hosted on JCloud as well. Learn more about [JCloud here](https://docs.jina.ai/fundamentals/jcloud/).\n\n\nFirst, you need a Flow YAML file for deploy. A minimum YAML file is as follows:\n\n````{tab} torch-flow.yml\n\n```yaml\njtype: Flow\nexecutors:\n - uses: jinahub+docker://CLIPTorchEncoder\n```\n\n````\n````{tab} onnx-flow.yml\n\n```yaml\njtype: Flow\nexecutors:\n - uses: jinahub+docker://CLIPOnnxEncoder\n```\n\n````\n\n```{tip}\n`port` is unnecessary here as JCloud will assign a new hostname and port for any deployed service. \n```\n\nExecutors must start with `jinahub+docker://` as it is required by JCloud. We currently provide containerized executors [`jinahub+docker://CLIPTorchEncoder`](https://cloud.jina.ai/executor/gzpbl8jh) and [`jinahub+docker://CLIPOnnxEncoder`](https://cloud.jina.ai/executor/2a7auwg2) on Jina Hub. They are automatically synced on the new release of `clip_server` module. \n\nTo enable GPU on JCloud, you need to configure it in the YAML file and use prebuilt docker GPU images. For example,\n\n```yaml\njtype: Flow\nexecutors:\n - uses: jinahub+docker://CLIPTorchEncoder/latest-gpu\n jcloud:\n resources:\n gpu: shared\n```\n\nPlease refer [here](https://docs.jina.ai/fundamentals/jcloud/yaml-spec/#gpu) for more details on using GPU in JCloud.\nNotice that you must specify a docker image GPU tag for your executor to utilize the GPU. For example `latest-gpu`. \nSee the 'Tag' section in [CLIPTorchEncoder](https://cloud.jina.ai/executor/gzpbl8jh) and [CLIPOnnxEncoder](https://cloud.jina.ai/executor/2a7auwg2) for docker image GPU tags.\n\nTo deploy,\n\n````{tab} PyTorch-backed\n```bash\njc deploy torch-flow.yml\n```\n````\n\n````{tab} ONNX-backed\n```bash\njc deploy onnx-flow.yml\n```\n````\n\n\nIf Flow is successfully deployed you will see:\n\n```{figure} jc-deploy.png\n:width: 60%\n```\n\nYou can now connect to it via client by setting `server` as the URL given by JCloud:\n\n```python\nfrom clip_client import Client\n\nc = Client(\n 'grpcs://174eb69ba3.wolf.jina.ai'\n) # This is the URL you get from previous step\nc.profile()\n```\n" }, { "path": "docs/html_extra/robots.txt", "content": "User-agent: *\nsitemap: https://clip-as-service.jina.ai/sitemap.xml" }, { "path": "docs/index.md", "content": "# Welcome to CLIP-as-service!\n\n\n```{include} ../README.md\n:start-after: \n:end-before: \n```\n\n## Try it!\n\n## Install\n\n![PyPI](https://img.shields.io/pypi/v/clip_client?color=%23ffffff&label=%20) is the latest version.\n\nMake sure you are using Python 3.7+. You can install the client and server independently. It is **not required** to install both: e.g. you can install `clip_server` on a GPU machine and `clip_client` on a local laptop.\n\n````{tab} Client\n\n```bash\npip install clip-client\n```\n\n````\n\n````{tab} Server (PyTorch)\n\n```bash\npip install clip-server\n```\n````\n\n````{tab} Server (ONNX)\n\n```bash\npip install \"clip_server[onnx]\"\n```\n\n````\n\n\n````{tab} Server (TensorRT)\n\n```bash\npip install nvidia-pyindex \npip install \"clip_server[tensorrt]\"\n```\n````\n\n````{tab} Server on Google Colab\n\n```{button-link} https://colab.research.google.com/github/jina-ai/clip-as-service/blob/main/docs/hosting/cas-on-colab.ipynb\n:color: primary\n:align: center\n\n{octicon}`link-external` Open the notebook on Google Colab \n```\n\n````\n\n\n## Quick check\n\nAfter installing, you can run the following commands for a quick connectivity check.\n\n### Start the server\n\n````{tab} Start PyTorch Server \n```bash\npython -m clip_server\n```\n````\n\n````{tab} Start ONNX Server \n```bash\npython -m clip_server onnx-flow.yml\n```\n````\n\n````{tab} Start TensorRT Server \n```bash\npython -m clip_server tensorrt-flow.yml\n```\n````\n\nAt the first time starting the server, it will download the default pretrained model, which may take a while depending on your network speed. Then you will get the address information similar to the following: \n\n```text\nโ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ ๐Ÿ”— Endpoint โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ\nโ”‚ ๐Ÿ”— Protocol GRPC โ”‚\nโ”‚ ๐Ÿ  Local 0.0.0.0:51000 โ”‚\nโ”‚ ๐Ÿ”’ Private 192.168.31.62:51000 โ”‚\n| ๐ŸŒ Public 87.105.159.191:51000 |\nโ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ \n```\n\nThis means the server is ready to serve. Note down the three addresses shown above, you will need them later.\n\n### Connect from client\n\n```{tip}\nDepending on the location of the client and server. You may use different IP addresses:\n- Client and server are on the same machine: use local address, e.g. `0.0.0.0`\n- Client and server are connected to the same router: use private network address, e.g. `192.168.3.62`\n- Server is in public network: use public network address, e.g. `87.105.159.191`\n```\n\nRun the following Python script:\n\n```python\nfrom clip_client import Client\n\nc = Client('grpc://0.0.0.0:51000')\nc.profile()\n```\n\nwill give you:\n\n```text\n Roundtrip 16ms 100%\nโ”œโ”€โ”€ Client-server network 8ms 49%\nโ””โ”€โ”€ Server 8ms 51%\n โ”œโ”€โ”€ Gateway-CLIP network 2ms 25%\n โ””โ”€โ”€ CLIP model 6ms 75%\n{'Roundtrip': 15.684750003856607, 'Client-server network': 7.684750003856607, 'Server': 8, 'Gateway-CLIP network': 2, 'CLIP model': 6}\n```\n\nIt means the client and the server are now connected. Well done!\n\n\n```{include} ../README.md\n:start-after: \n:end-before: \n```\n\n\n```{toctree}\n:caption: User Guides\n:hidden:\n\nuser-guides/client\nuser-guides/server\nuser-guides/benchmark\nuser-guides/retriever\nuser-guides/faq\n```\n\n```{toctree}\n:caption: Hosting\n:hidden:\n\n\nhosting/colab\n```\n\n```{toctree}\n:caption: Playground\n:hidden:\n\nplayground/embedding\nplayground/reasoning\nplayground/searching\n```\n\n\n```{toctree}\n:caption: Developer References\n:hidden:\n:maxdepth: 1\n\napi/clip_client\n```\n\n\n---\n{ref}`genindex` | {ref}`modindex`\n\n" }, { "path": "docs/makedoc.sh", "content": "#!/usr/bin/env bash\n\nset -ex\n\nrm -rf api && make clean\n\nmake dirhtml\n" }, { "path": "docs/playground/embedding.md", "content": "# Text & Image Embedding\n\nEmbedding is a basic task in CLIP-as-service. It means converting your input sentence or image into a fixed-length vector. In this demo, you can choose a picture, input a sentence in the textbox, or copy-paste your image URL into the text box to get a rough feeling how CLIP-as-service works.\n\nThis is *not* a search task. The images are random stock images and are related to any search results, they are mainly for saving your time on finding some random internet cat pictures. \n\nThe model is `ViT-L/14-336px` on one GPU.\n\n\n\n```{button-link} ../../_static/demo-embed.html\n:color: primary\n:align: center\n\n{octicon}`link-external` Open this playground in a new window\n```" }, { "path": "docs/playground/reasoning.md", "content": "# Visual Reasoning\n\nVisual reasoning is another basic task in CLIP-as-service. There are four basic visual reasoning skills: object recognition, object counting, color recognition, and spatial relation understanding. Despite how magic it sounds and looks, the idea is fairly simple: just input the reasoning texts as prompts, then {ref}`calling rank interface` of `clip_server`. The server will rank the prompts and return sorted prompts with scores.\n\nIn this demo, you can choose a picture, or copy-paste your image URL into the text box to get a rough feeling how visual reasoning works. Feel free to add or remove prompts and observe how it affects the ranking results.\n\nThe model is `ViT-L/14-336px` on one GPU.\n\n\n\n\n```{button-link} ../../_static/demo-text-rank.html\n:color: primary\n:align: center\n\n{octicon}`link-external` Open this playground in a new window\n```" }, { "path": "docs/playground/searching.md", "content": "# Text & Image Searching\n\nCLIP-as-service enables us to encode text and images into a common space. This is a powerful tool for many applications, such as cross-modality search.\n\n[CLIP search](../user-guides/retriever.md) is a new feature provided by CLIP-as-service. It enables us to search for images based on text/image. It calculates the similarity score based on the embeddings of the text and image. The higher the score, the more similar they are.\n\nThis demo demonstrates the text-to-image and image-to-image searching in CLIP search. You can type text query or upload the local image as a query, and it will return the top 10 similar images for you.\n\nIn this demo, we use [``Open-Image-Dataset``](https://storage.googleapis.com/openimages/web/download.html) dataset (consist of 125,346 images) to demonstrate Text & Image retrieval.\n\n\n\n```{button-link} https://jemmyshin-laion5b-streamlit-streamlit-demo-rddbqz.streamlitapp.com/\n:color: primary\n:align: center\n\n{octicon}`link-external` Open this playground in a new window\n```" }, { "path": "docs/requirements.txt", "content": "# cf. https://github.com/ryanfox/sphinx-markdown-tables/issues/36\nmarkdown<3.4.0\nsphinx\nsphinx-argparse==0.3.1\nsphinxcontrib-apidoc==0.3.0\nsphinx-autodoc-typehints==1.12.0\nsphinx_markdown_tables==0.0.15\nsphinx_copybutton==0.4.0\nsphinx-notfound-page==0.7.1\ngitpython==3.1.13\nsphinx-sitemap==2.2.0\nsphinxext-opengraph\nfuro\nmyst-parser==0.15.1\nsphinx-design\nsphinx-inline-tabs\n# sphinx-multiversion\ngit+https://github.com/Holzhaus/sphinx-multiversion.git" }, { "path": "docs/user-guides/benchmark.rst", "content": "Benchmark\n=========\n\nIn order to understand the zero-shot performance of CLIP and its limitations, we conducted a benchmark\nacross a variety of computer vision datasets (the dataset details are in the appendix). Here, thanks for the\nopen-source `CLIP Benchmark toolkit `_, we can easily reproduce the results.\n\nWe hope that this benchmark can help you to better understand the performance of CLIP models and choose the best model for your application.\n\n\nSelect the right model\n-----------------------\n\nIn general, you can select the best model for your application from different perspectives: disk usage, peak RAM and VRAM usages, QPS, and most importantly, the performance.\n\nBased on our experiments, we recommend the ViT models over the RN models for most general applications.\nMore specifically, the ``ViT-H-14::laion2b_s32b_b79k`` model and ``ViT-g-14::laion2b_s12b_b42k`` model should be first considered since they have the best or close to the best performance in most cases.\nHowever, if you are concerned about the encoding speed, you can consider other ViT models because they have higher QPS with decent performance.\nAnyway, you should choose the model that best fits your requirements.\nFor example, if you are labeling images for diabetic retinopathy, you should probably select the ``ViT-B-32::laion2b_s34b_b79k`` model since it has the best top-1 accuracy of 0.734 on zero-shot classification of the Retinopathy dataset.\nOr if you are dealing with histopathologic images, you should probably select the RN50::openai model since it has the best top-1 accuracy of 0.636 on zero-shot classification of the Patch Camelyon dataset.\n\n\nThe following sections show the performance of different models in details on different datasets and tasks.\n\n\nSize and efficiency\n-------------------------\n\nWe first present the model's size and efficiency in terms of query time and memory usage (including the peak RAM and VRAM usage).\nAll of the results are obtained on a single Nvidia TITAN RTX GPU (24GB VRAM) with default server settings.\n\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| Model | Disk Usage (MB) | Peak RAM Usage (GB) | Peak VRAM Usage (GB) | Text QPS | Image QPS |\n+========================================+==================+======================+=======================+===========+============+\n| RN50::openai | 244 | 2.99 | 1.36 | 1019 | 269 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| RN50::yfcc15m | 389 | 2.86 | 1.36 | 1083 | 262 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| RN50::cc12m | 389 | 2.84 | 1.36 | 1064 | 264 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| RN101::openai | 278 | 3.05 | 1.40 | 1047 | 222 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| RN101::yfcc15m | 457 | 2.88 | 1.40 | 1107 | 223 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| RN50x4::openai | 402 | 3.23 | 1.63 | 1047 | 218 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| RN50x16::openai | 631 | 3.63 | 2.02 | 1038 | 121 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| RN50x64::openai | 1291 | 4.08 | 2.98 | 985 | 59 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-B-32::openai | 338 | 3.20 | 1.40 | 1064 | 286 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-B-32::laion2b_e16 | 577 | 2.93 | 1.40 | 1120 | 292 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-B-32::laion400m_e31 | 577 | 2.93 | 1.40 | 1080 | 287 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-B-32::laion400m_e32 | 577 | 2.94 | 1.40 | 1092 | 289 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-B-32::laion2b-s34b-b79k | 577 | 2.94 | 1.40 | 1102 | 285 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-B-16::openai | 335 | 3.20 | 1.44 | 1064 | 260 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-B-16::laion400m_e31 | 571 | 2.93 | 1.44 | 1099 | 262 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-B-16::laion400m_e32 | 571 | 2.94 | 1.44 | 1082 | 268 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-B-16-plus-240::laion400m_e31 | 795 | 3.03 | 1.59 | 1059 | 235 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-B-16-plus-240::laion400m_e32 | 795 | 3.03 | 1.59 | 1043 | 239 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-L-14::openai | 890 | 3.66 | 2.04 | 1040 | 140 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-L-14::laion400m_e31 | 1631 | 3.43 | 2.03 | 1058 | 147 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-L-14::laion400m_e32 | 1631 | 3.42 | 2.03 | 1061 | 146 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-L-14::laion2b-s32b-b82k | 1631 | 3.43 | 2.03 | 1069 | 147 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-L-14-336::openai | 891 | 3.74 | 2.23 | 1070 | 76 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-H-14::laion2b-s32b-b79k | 3762 | 4.45 | 3.26 | 642 | 91 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| ViT-g-14::laion2b-s12b-b42k | 5214 | 5.16 | 4.00 | 639 | 69 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| M-CLIP/LABSE-Vit-L-14 | 3609 | 4.30 | 4.70 | 646 | 284 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| M-CLIP/XLM-Roberta-Large-Vit-B-32 | 4284 | 5.37 | 1.68 | 656 | 139 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| M-CLIP/XLM-Roberta-Large-Vit-B-16Plus | 4293 | 4.30 | 4.13 | 662 | 236 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n| M-CLIP/XLM-Roberta-Large-Vit-L-14 | 4293 | 4.30 | 4.97 | 1027 | 139 |\n+----------------------------------------+------------------+----------------------+-----------------------+-----------+------------+\n\n\n\nZero-shot performance\n----------------------------\n\nIn this section, we will report the zero-shot performance of the models on classification and retrieval tasks across different datasets.\nIn the following tables, we will highlight the best results in bold for each dataset (higher is better).\n\nZero-shot retrieval\n+++++++++++++++++++\n\nIn zero-shot retrieval benchmark, each model is evaluated on the following datasets: `COCO Caption `_, `Flickr8k `_ and `Flickr30k `_.\nFor the above datasets, there are five corresponding description sentences for each image written by humans.\nThe results are reported in terms of top-5 text-to-image retrieval recall, top-5 image-to-text retrieval recall and their averages.\nMore specifically, the top-5 text-to-image retrieval recall for each retrieved image is either 1 or 0.\nIt is 1 if the input text matches one of the image descriptions among the top-5.\nThe top-5 image-to-text retrieval recall for each image is the number of top-5 retrieved texts matching that image descriptions.\n\n+----------------------------------+-------------------------------------------+-------------------------------------------+-------------------------------------------+\n| Model | COCO Caption | Flickr 8k | Flickr 30k |\n| +---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| | Text to image | Image to text | Average | Text to image | Image to text | Average | Text to image | Image to text | Average |\n+==================================+===============+===============+===========+===============+===============+===========+===============+===============+===========+\n| RN50::openai | 0.529 | 0.728 | 0.629 | 0.504 | 0.690 | 0.597 | 0.392 | 0.621 | 0.506 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| RN50::yfcc15m | 0.361 | 0.534 | 0.447 | 0.238 | 0.394 | 0.316 | 0.146 | 0.278 | 0.212 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| RN50::cc12m | 0.446 | 0.607 | 0.527 | 0.302 | 0.435 | 0.369 | 0.204 | 0.316 | 0.260 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| RN101::openai | 0.555 | 0.745 | 0.650 | 0.523 | 0.694 | 0.608 | 0.415 | 0.629 | 0.522 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| RN101::yfcc15m | 0.376 | 0.549 | 0.463 | 0.251 | 0.417 | 0.334 | 0.156 | 0.296 | 0.226 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| RN50x4::openai | 0.581 | 0.767 | 0.674 | 0.558 | 0.729 | 0.643 | 0.451 | 0.671 | 0.561 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| RN50x16::openai | 0.600 | 0.787 | 0.693 | 0.597 | 0.768 | 0.682 | 0.496 | 0.713 | 0.604 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| RN50x64::openai | 0.599 | 0.803 | 0.701 | 0.629 | 0.790 | 0.709 | 0.534 | 0.756 | 0.645 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-B-32::openai | 0.560 | 0.749 | 0.654 | 0.532 | 0.699 | 0.616 | 0.413 | 0.629 | 0.521 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-B-32::laion2b_e16 | 0.647 | 0.795 | 0.721 | 0.622 | 0.760 | 0.691 | 0.507 | 0.687 | 0.597 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-B-32::laion400m_e31 | 0.600 | 0.763 | 0.682 | 0.562 | 0.736 | 0.649 | 0.438 | 0.633 | 0.536 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-B-32::laion400m_e32 | 0.600 | 0.765 | 0.682 | 0.562 | 0.736 | 0.649 | 0.437 | 0.634 | 0.536 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-B-32::laion2b_s34b_b79k | 0.654 | 0.798 | 0.726 | 0.629 | 0.778 | 0.703 | 0.513 | 0.694 | 0.603 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-B-16::openai | 0.584 | 0.767 | 0.676 | 0.564 | 0.727 | 0.646 | 0.452 | 0.671 | 0.561 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-B-16::laion400m_e31 | 0.637 | 0.796 | 0.717 | 0.620 | 0.765 | 0.692 | 0.506 | 0.697 | 0.602 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-B-16::laion400m_e32 | 0.636 | 0.796 | 0.716 | 0.620 | 0.767 | 0.694 | 0.508 | 0.697 | 0.603 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-B-16-plus-240::laion400m_e31 | 0.660 | 0.809 | 0.735 | 0.642 | 0.788 | 0.715 | 0.533 | 0.725 | 0.629 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-B-16-plus-240::laion400m_e32 | 0.662 | 0.811 | 0.736 | 0.644 | 0.791 | 0.718 | 0.535 | 0.727 | 0.631 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-L-14::openai | 0.610 | 0.793 | 0.702 | 0.599 | 0.767 | 0.683 | 0.494 | 0.717 | 0.605 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-L-14::laion400m_e31 | 0.680 | 0.821 | 0.750 | 0.675 | 0.806 | 0.741 | 0.570 | 0.751 | 0.661 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-L-14::laion400m_e32 | 0.680 | 0.821 | 0.751 | 0.675 | 0.806 | 0.740 | 0.570 | 0.751 | 0.661 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-L-14::laion2b_s32b_b82k | 0.711 | 0.840 | 0.775 | 0.712 | 0.824 | 0.768 | 0.620 | 0.789 | 0.704 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-L-14-336::openai | 0.616 | 0.812 | 0.714 | 0.629 | 0.779 | 0.704 | 0.533 | 0.741 | 0.637 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-H-14::laion2b_s32b_b79k | **0.734** | **0.861** | **0.797** | **0.746** | **0.856** | **0.801** | **0.657** | **0.823** | **0.740** |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n| ViT-g-14::laion2b_s12b_b42k | 0.724 | 0.853 | 0.788 | 0.730 | 0.846 | 0.788 | 0.639 | 0.806 | 0.722 |\n+----------------------------------+---------------+---------------+-----------+---------------+---------------+-----------+---------------+---------------+-----------+\n\nFrom the table, we observe that the ViT models outperform the RN models in general.\nMore specifically, the ``ViT-H-14::laion2b_s32b_b79k`` model and ``ViT-g-14::laion2b_s12b_b42k`` model achieve the best and second-best results on all zero-shot retrieval tasks.\nFor ViT models, the results of the same base model are better on those pre-trained with larger datasets (e.g., ``ViT-B-32::openai`` vs ``ViT-B-32::laion400m_e31`` vs ``ViT-B-32::laion2b-s34b-b79k``).\n\nZero-shot classification\n++++++++++++++++++++++++\n\nIn zero-shot classification benchmark, each model is evaluated on the following datasets: `ImageNetV2 `_, `VOC2007 `_ and 19 `VTAB datasets `_.\nThe results are shown in the following table. \nFor each dataset, we report the top-1 accuracy, which is whether the top-1 retrieved class of a image matches its true class.\n\n+----------------------------------+------------+-----------+-------------------------------------------------------------------------------------+------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+\n| Model | ImageNetV2 | VOC2007 | VTAB natural | VTAB specialized | VTAB structured |\n| | | +------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| | | | Caltech101 | CIFAR-100 | DTD | Flowers102 | Pets | Sun397 | SVHN | EuroSAT | Resisc45 | Patch Camelyon | Retinopathy | Clevr/count | Clevr/distance | dSprites/location | dSprites/orientation | SmallNORB/azimuth | SmallNORB/elevation | DMLab | KITTI/distance |\n+==================================+============+===========+============+===========+===========+============+===========+===========+===========+===========+===========+================+=============+=============+================+===================+======================+===================+=====================+===========+================+\n| RN50::openai | 0.529 | 0.650 | 0.772 | 0.403 | 0.415 | 0.660 | 0.857 | 0.894 | 0.303 | 0.408 | 0.453 | **0.636** | 0.171 | 0.217 | 0.148 | 0.034 | 0.014 | 0.056 | 0.110 | 0.145 | 0.170 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| RN50::yfcc15m | 0.214 | 0.215 | 0.402 | 0.116 | 0.122 | 0.167 | 0.174 | 0.127 | 0.157 | 0.172 | 0.123 | 0.533 | 0.358 | 0.151 | 0.158 | 0.032 | 0.024 | 0.053 | 0.120 | 0.160 | **0.336** |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| RN50::cc12m | 0.224 | 0.438 | 0.582 | 0.178 | 0.135 | 0.095 | 0.331 | 0.123 | 0.102 | 0.148 | 0.117 | 0.535 | 0.293 | 0.184 | 0.222 | 0.031 | 0.025 | 0.047 | 0.096 | 0.161 | 0.155 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| RN101::openai | 0.561 | 0.651 | 0.780 | 0.476 | 0.432 | 0.652 | 0.869 | 0.887 | 0.226 | 0.314 | 0.547 | 0.583 | 0.280 | 0.242 | 0.130 | 0.031 | 0.021 | 0.054 | 0.111 | 0.139 | 0.263 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| RN101::yfcc15m | 0.221 | 0.243 | 0.469 | 0.125 | 0.117 | 0.210 | 0.177 | 0.128 | 0.137 | 0.151 | 0.099 | 0.479 | 0.584 | 0.109 | 0.159 | 0.031 | 0.019 | 0.055 | 0.097 | 0.153 | 0.252 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| RN50x4::openai | 0.594 | 0.682 | 0.781 | 0.451 | 0.486 | 0.698 | 0.887 | 0.908 | 0.367 | 0.335 | 0.532 | 0.569 | 0.318 | 0.205 | 0.082 | 0.031 | 0.026 | 0.056 | 0.108 | 0.162 | 0.233 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| RN50x16::openai | 0.643 | 0.680 | 0.810 | 0.522 | 0.524 | 0.724 | 0.898 | 0.917 | 0.409 | 0.433 | 0.589 | 0.625 | 0.715 | 0.195 | 0.213 | 0.030 | 0.026 | 0.050 | 0.116 | 0.146 | 0.229 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| RN50x64::openai | 0.670 | 0.740 | 0.834 | 0.598 | 0.531 | 0.788 | 0.936 | 0.931 | 0.481 | 0.577 | 0.628 | 0.539 | 0.073 | 0.227 | 0.200 | 0.034 | 0.025 | 0.056 | 0.125 | 0.158 | 0.311 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-B-32::openai | 0.559 | 0.764 | 0.815 | 0.643 | 0.443 | 0.664 | 0.873 | 0.913 | 0.135 | 0.504 | 0.537 | 0.623 | 0.447 | 0.232 | 0.164 | 0.037 | 0.024 | 0.061 | **0.127** | 0.193 | 0.274 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-B-32::laion2b_e16 | 0.573 | 0.788 | 0.831 | 0.754 | 0.539 | 0.691 | 0.893 | 0.933 | 0.388 | 0.503 | 0.619 | 0.506 | 0.195 | 0.192 | 0.167 | 0.031 | 0.024 | 0.052 | 0.110 | 0.189 | 0.176 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-B-32::laion400m_e31 | 0.523 | 0.731 | 0.818 | 0.678 | 0.521 | 0.659 | 0.856 | 0.918 | 0.220 | 0.470 | 0.510 | 0.549 | 0.259 | 0.155 | 0.161 | 0.033 | 0.021 | 0.053 | 0.117 | 0.173 | 0.122 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-B-32::laion400m_e32 | 0.523 | 0.733 | 0.817 | 0.677 | 0.523 | 0.658 | 0.854 | 0.917 | 0.223 | 0.476 | 0.510 | 0.548 | 0.240 | 0.153 | 0.161 | 0.033 | 0.021 | 0.054 | 0.117 | 0.173 | 0.118 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-B-32::laion2b_s34b_b79k | 0.581 | 0.791 | 0.839 | 0.755 | 0.557 | 0.716 | 0.909 | 0.937 | 0.410 | 0.482 | 0.610 | 0.598 | **0.734** | 0.153 | 0.189 | 0.029 | **0.034** | **0.062** | 0.113 | 0.159 | 0.262 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-B-16::openai | 0.619 | 0.783 | 0.819 | 0.669 | 0.449 | 0.712 | 0.890 | 0.924 | 0.313 | 0.559 | 0.582 | 0.507 | 0.036 | 0.209 | 0.158 | 0.030 | 0.023 | 0.053 | 0.122 | 0.155 | 0.263 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-B-16::laion400m_e31 | 0.594 | 0.767 | 0.838 | 0.712 | 0.513 | 0.694 | 0.892 | 0.939 | 0.380 | 0.503 | 0.585 | 0.593 | 0.062 | 0.289 | **0.245** | 0.031 | 0.030 | 0.059 | 0.100 | 0.152 | 0.200 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-B-16::laion400m_e32 | 0.597 | 0.768 | 0.837 | 0.712 | 0.513 | 0.692 | 0.892 | 0.939 | 0.385 | 0.501 | 0.585 | 0.598 | 0.077 | 0.287 | **0.245** | 0.032 | 0.029 | 0.060 | 0.099 | 0.151 | 0.183 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-B-16-plus-240::laion400m_e31 | 0.614 | 0.764 | 0.832 | 0.733 | 0.555 | 0.706 | 0.904 | 0.940 | 0.355 | 0.569 | 0.615 | 0.551 | 0.093 | 0.240 | 0.159 | 0.041 | 0.026 | 0.056 | 0.111 | 0.149 | 0.280 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-B-16-plus-240::laion400m_e32 | 0.615 | 0.764 | 0.833 | 0.738 | 0.555 | 0.711 | 0.902 | 0.940 | 0.362 | 0.581 | 0.613 | 0.551 | 0.095 | 0.238 | 0.160 | **0.043** | 0.027 | 0.054 | 0.110 | 0.148 | 0.281 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-L-14::openai | 0.698 | 0.783 | 0.835 | 0.758 | 0.554 | 0.792 | 0.932 | 0.937 | 0.571 | 0.626 | 0.633 | 0.520 | 0.733 | 0.194 | 0.161 | 0.032 | 0.023 | 0.045 | 0.115 | 0.163 | 0.218 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-L-14::laion400m_e31 | 0.654 | 0.758 | 0.839 | 0.774 | 0.598 | 0.757 | 0.917 | 0.950 | 0.378 | 0.632 | 0.671 | 0.487 | 0.058 | 0.242 | 0.149 | 0.030 | 0.026 | 0.053 | 0.109 | 0.186 | 0.200 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-L-14::laion400m_e32 | 0.654 | 0.756 | 0.839 | 0.774 | 0.605 | 0.756 | 0.919 | 0.950 | 0.380 | 0.622 | 0.675 | 0.493 | 0.061 | 0.243 | 0.149 | 0.030 | 0.026 | 0.053 | 0.110 | 0.186 | 0.203 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-L-14::laion2b_s32b_b82k | 0.677 | 0.805 | **0.851** | 0.833 | 0.629 | 0.758 | 0.932 | 0.958 | 0.459 | 0.646 | 0.668 | 0.563 | 0.116 | 0.312 | 0.161 | 0.032 | 0.020 | 0.056 | 0.108 | **0.224** | 0.229 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-L-14-336::openai | **0.709** | 0.781 | 0.837 | 0.744 | 0.556 | 0.783 | 0.937 | 0.940 | 0.560 | 0.615 | 0.638 | 0.608 | 0.733 | 0.200 | 0.158 | 0.032 | 0.024 | 0.046 | 0.113 | 0.158 | 0.262 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-H-14::laion2b_s32b_b79k | **0.709** | 0.777 | 0.850 | **0.847** | 0.678 | **0.801** | **0.945** | 0.961 | 0.563 | **0.726** | 0.699 | 0.542 | 0.297 | 0.268 | 0.169 | 0.032 | 0.027 | 0.054 | 0.111 | 0.140 | 0.110 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n| ViT-g-14::laion2b_s12b_b42k | 0.696 | **0.811** | **0.851** | 0.839 | **0.682** | 0.776 | 0.943 | **0.962** | **0.603** | 0.648 | 0.718 | 0.560 | 0.580 | **0.332** | 0.175 | 0.036 | 0.031 | 0.060 | 0.115 | 0.190 | 0.138 |\n+----------------------------------+------------+-----------+------------+-----------+-----------+------------+-----------+-----------+-----------+-----------+-----------+----------------+-------------+-------------+----------------+-------------------+----------------------+-------------------+---------------------+-----------+----------------+\n\nFrom the table, we observe that the ViT models still outperform the RN models in most tasks, except for the Patch Camelyon dataset where ``RN50::openai`` has the best top-1 accuracy of 0.636, and the KITTI/distance dataset where ``RN50::yfcc15m`` has the best result of 0.336.\nSimilar to retrieval results, the ``ViT-H-14::laion2b_s32b_b79k`` model and ``ViT-g-14::laion2b_s12b_b42k`` model still have the best or close to the best results on 12/21 zero-shot classification tasks.\nAll models tend to perform well on ImageNetV2, VOC2007, VTAB natural and VTAB specialized (except for Retinopathy) datasets, whereas they perform poorly on VTAB structured datasets.\nWe do not observe any significant difference between the ViT models of the same base model. \n\nAppendix: Datasets description\n------------------------------\n\n* **COCO Caption** [1]_: The dataset contains over one and a half million captions describing over 330,000 images. For the training and validation images, five independent human generated captions are provided.\n\n* **Flickr 8k** [2]_: The dataset consists of 8,000 images that are each paired with five different captions which provide clear descriptions of the salient entities and events. The images were chosen from six different Flickr groups, and tend not to contain any well-known people or locations, but were manually selected to depict a variety of scenes and situations.\n\n* **Flickr 30k** [3]_: The dataset is an extension of the Flickr 8k Dataset. It consists of 158,915 crowd-sourced captions describing 31,783 images.\n\n* **ImageNetV2** [4]_: ImageNetV2 contains three test sets with 10,000 new images each. Importantly, these test sets were sampled after a decade of progress on the original ImageNet dataset. This makes the new test data independent of existing models and guarantees that the accuracy scores are not affected by adaptive overfitting.\n\n* **VOC2007** [5]_: The training data provided consists of a set of images; each image has an annotation file giving a bounding box and object class label for each object in one of the twenty classes present in the image. Note that multiple objects from multiple classes may be present in the same image.\n\n* **VTAB natural group** [6]_: The natural group represents classical vision problems. These tasks contain natural images captured using standard cameras. The classes may represent generic, fine-grained, or abstract objects.\n\n * **Caltech101**: The task consists in classifying pictures of objects (101 classes plus a background clutter class), including animals, airplanes, chairs, or scissors. The image size varies, but it typically ranges from 200-300 pixels per edge.\n\n * **CIFAR-100**: The task consists in classifying natural images (100 classes, with 500 training images each). Some examples include apples, bottles, dinosaurs, and bicycles. The image size is 32x32.\n\n * **DTD**: The task consists in classifying images of textural patterns (47 classes, with 120 training images each). Some of the textures are banded, bubbly, meshed, lined, or porous. The image size ranges between 300x300 and 640x640 pixels.\n\n * **Flowers102**: The task consists in classifying images of flowers present in the UK (102 classes, with between 40 and 248 training images per class). Azalea, Californian Poppy, Sunflower, or Petunia are some examples. Each image dimension has at least 500 pixels.\n\n * **Pets**: The task consists in classifying pictures of cat and dog breeds (37 classes with around 200 images each), including Persian cat, Chihuahua dog, English Setter dog, or Bengal cat. Images dimensions are typically 200 pixels or larger.\n\n * **Sun397**: The Sun397 task is a scenery benchmark with 397 classes and, at least, 100 images per class. Classes have a hierarchy structure, and include cathedral, staircase, shelter, river, or archipelago. The images are (colour) 200x200 pixels or larger.\n\n * **SVHN**: This task consists in classifying images of Google's street-view house numbers (10 classes, with more than 1000 training images each). The image size is 32x32 pixels.\n\n* **VTAB specialized group**: The specialized group also contains images of the world, but captured through specialist equipment. These images have different invariances to those in the specialized tasks. Nonetheless, humans recognize the structures therein, thus generic visual representations should also capture the visual concepts. It two sub-groups: remote sensing, and medical.\n\n * **EuroSAT**: The task consists in classifying Sentinel-2 satellite images into 10 different types of land use (Residential, Industrial, River, Highway, etc). The spatial resolution corresponds to 10 meters per pixel, and the image size is 64x64 pixels.\n\n * **Resisc45**: The Remote Sensing Image Scene Classification (RESISC) dataset is a scene classification task from remote sensing images. There are 45 classes, containing 700 images each, including tennis court, ship, island, lake, parking lot, sparse residential, or stadium. The image size is RGB 256x256 pixels.\n\n * **Patch Camelyon**: The Patch Camelyon dataset contains 327,680 images of histopathologic scans of lymph node sections. The classification task consists in predicting the presence of metastatic tissue in given image (i.e., two classes). All images are 96x96 pixels.\n\n * **Retinopathy**: The Diabetic Retinopathy dataset consists of image-label pairs with high-resolution retina images, and labels that indicate the presence of Diabetic Retinopahy (DR) in a 0-4 scale (No DR, Mild, Moderate, Severe, or Proliferative DR).\n\n* **VTAB structured group**: The structured group assesses comprehension of the structure of a scene, for example, object counting, or 3D depth prediction. Most of these tasks are generated from simulated environments, whose structure is easy for a human to determine, but whose domain differs greatly to datasets like ImageNet. These tasks are intended as a step towards useful representations for perceptual control.\n\n * **Clevr/count**: CLEVR is a visual question and answer dataset designed to evaluate algorithmic visual reasoning. We use just the images from this dataset, and create a synthetic task by setting the label equal to the number of objects in the images.\n\n * **Clevr/distance**: Another synthetic task we create from CLEVR consists of predicting the depth of the closest object in the image from the camera. The depths are bucketed into size bins.\n\n * **dSprites/location**: The dSprites dataset was originally designed to asses disentanglement properties of unsupervised learning algorithms. In particular, each image is a 2D shape where six factors are controlled: color, shape, scale, rotation, and (x,y) center coordinates. Images have 64x64 black-and-white pixels. This task consists in predicting the x (horizontal) coordinate of the object. The locations are bucketed into 16 bins.\n\n * **dSprites/orientation**: We create another task from dSprites consists in predicting the orientation of each object, bucketed into 16 bins.\n\n * **SmallNORB/azimuth**: The Small NORB dataset contains images of 3D-toys from 50 classes, including animals, human figures, airplanes, trucks, and cars. The image size is 640x480 pixels. In this case, we define labels depending on the azimuth (angle of horizontal deviation), in intervals of 20 degrees (18 classes).\n\n * **SmallNORB/elevation**: Another synthetic task we create from Small NORB consists in predicting the elevation in the image. There are 9 classes, corresponding to 9 different elevations ranging from 30 to 70 degrees, in intervals of 5 degrees.\n\n * **DMLab**: The DMLab (DeepMind Lab) is a set of control environments focused on 3D navigation and puzzle-solving tasks. The Dmlab dataset contains frames observed by the agent acting in the DeepMind Lab environment, which are annotated by the distance between the agent and various objects present in the environment. The goal is to evaluate the ability of a visual model to reason about distances from the visual input in 3D environments. The Dmlab dataset consists of 360x480 color images in 6 classes. The classes are {close, far, very far} x {positive reward, negative reward} respectively.\n\n * **KITTI-Dist**: The KITTI task consists in predicting the (binned) depth to the vehicle (car, van, or truck) in the image. There are 4 bins / classes.\n\n.. [1] https://arxiv.org/pdf/1504.00325.pdf\n.. [2] https://www.kaggle.com/datasets/adityajn105/flickr8k\n.. [3] https://shannon.cs.illinois.edu/DenotationGraph/\n.. [4] https://github.com/modestyachts/ImageNetV2\n.. [5] http://host.robots.ox.ac.uk/pascal/VOC/voc2007/\n.. [6] https://arxiv.org/pdf/1910.04867.pdf\n" }, { "path": "docs/user-guides/client.md", "content": "# Client API\n\nCLIP-as-service is designed in a client-server architecture. You can use `clip_client` to send images and texts to the server and receive the responses from the server. Right now, `clip_client` provides encoding, ranking, indexing, and searching functionalities. Additionally, it has many nice designs for speeding up the processing of a large amount of data:\n\n- Streaming: request sending is *not* blocked by the response receiving. Sending and receiving are two separate streams that run in parallel. Both are independent and each have separate internal buffer.\n- Batching: large requests are segmented into small batches and send in a stream. \n- Low memory footprint: only load data when needed.\n- Sync/async interface: provide `async` interface that can be easily integrated into other asynchronous system.\n- Auto-detect images and text input.\n- Support gRPC, HTTP, Websocket protocols with their TLS counterparts. \n\n\n```{tip}\nYou will need to install `clip_client` first in Python 3.7+: `pip install clip-client`.\n```\n\n(construct-client)=\n## Construct client\n\nTo use `clip_client`, you need to first construct a Client object, e.g.:\n\n```python\nfrom clip_client import Client\n\nc = Client('grpc://0.0.0.0:23456')\n```\n\nThe URL-like scheme `grpc://0.0.0.0:23456` is what you get after {ref}`running the server`. The scheme follows the format `scheme://netloc:port`:\n\n\n| Field | Description | Example |\n| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |\n| `scheme` | The protocol of the server, must be one of `grpc`, `websocket`, `http`, `grpcs`, `websockets`, `https`. Protocols end with `s` are TLS encrypted. This must match with the server protocol. | `grpc` |\n| `netloc` | The server's IP address or hostname | `192.168.0.3` |\n| `port` | The public port of the server | `51234` |\n \n\n\n## Encoding\n\n`clip_client` provides {func}`~clip_client.client.Client.encode` function that allows you to send sentences, images to the server in a streaming and sync/async manner. Encoding here means getting the fixed-length vector representation of a text or image.\n\n{func}`~clip_client.client.Client.encode` supports two basic input types:\n\n- **An iterable of `str`**, e.g. `List[str]`, `Tuple[str]`, `Generator[str]` are all acceptable.\n- **An iterable of {class}`~docarray.document.Document`**, e.g. `List[Document]`, {class}`~docarray.array.document.DocumentArray`, `Generator[Document]` are all acceptable.\n\nDepending on the input, the output of {func}`~clip_client.client.Client.encode` is different:\n\n- If the input is an iterable of `str`, then the output will be a `numpy.ndarray`.\n- If the input is an iterable of `Document`, then the output will be a `DocumentArray`.\n\nNow let's look at these two cases in details.\n\n### Input as iterable of strings\n\n- Input: each string element is auto-detected as a sentence or an image.\n- Output: a `[N, D]` shape `numpy.ndarray`, where `N` is the length of the input and `D` is the CLIP embedding size. Each row corresponds to the embedding of the input object.\n\nAny URI-like string, including relative, absolute file path, http/https path, data URI string will be considered as an image. Otherwise, it will be considered as a sentence. \n\nFor example,\n\n```python\nfrom clip_client import Client\n\nc = Client('grpc://0.0.0.0:23456')\n\nr = c.encode(\n [\n 'she smiled, with pain',\n 'apple.png',\n 'https://clip-as-service.jina.ai/_static/favicon.png',\n 'data:image/gif;base64,R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o/XBs/fNwfjZ0frl3/zy7////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7',\n ]\n)\nprint(r)\n```\n\ngives you\n\n```text\n[[-0.09136295 0.42720157 -0.05784469 ... -0.42873043 0.04472527\n 0.4437953 ]\n [ 0.43152636 0.1563695 -0.09363698 ... -0.11514216 0.1865044\n 0.15025651]\n [ 0.42862126 0.17757078 0.08584607 ... 0.23284511 -0.00929402\n 0.10993651]\n [ 0.4706376 -0.01384148 0.3877237 ... 0.1995864 -0.22621225\n -0.4837676 ]]\n```\n\n### Input as iterable of Documents\n\n```{tip}\nThis feature uses [DocArray](https://docarray.jina.ai), which is installed together with `clip_client` as an upstream dependency. You do not need to install DocArray separately.\n```\n\nIf auto-detection on a list of raw string is too \"sci-fi\" to you, then you may use `docarray.Document` to make the input more explicit and organized. `Document` can be used as a container to easily represent a sentence or an image.\n\n- Input: each `Document` must be filled with `.text` or `.uri` or `.blob` or `.tensor` attribute. \n - `Document` filled with `.text` is considered as sentence;\n - `Document` filled with `.uri` or `.blob` or `.tensor` is considered as image. If `.tensor` is filled, then its shape must be in `[H, W, C]` format.\n- Output: a `DocumentArray` of the same input length. Each `Document` object in it is the same one from the input and is now filled with `.embedding` attribute. The order of the output is the same as the input.\n\n```{note}\nIf the input `Document` is filled with both `.text` and `.uri`, then `.text` will be used.\n```\n\n```{caution}\nThe correctness of result and the order of output rely on the uniqueness of id of the input `Document`. The id will be implicitly generated if not provided. If you set the id manually, then you must make sure the id is unique, otherwise the results will not be complete.\n```\n\nThe explicitness comes from now you have to put the content into the `Document` attributes. For example, we can rewrite the above example as below:\n\n```python\nfrom clip_client import Client\nfrom docarray import Document\n\nc = Client('grpc://0.0.0.0:23456')\n\nda = [\n Document(text='she smiled, with pain'),\n Document(uri='apple.png'),\n Document(uri='apple.png').load_uri_to_image_tensor(),\n Document(blob=open('apple.png', 'rb').read()),\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n Document(\n uri='data:image/gif;base64,R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o/XBs/fNwfjZ0frl3/zy7////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7'\n ),\n]\n\nr = c.encode(da)\n```\n\nInstead of sending a list of `Document`, you can also wrap it with a `DocumentArray` and then send it:\n\n```python\nr = c.encode(DocumentArray(da))\n```\n\nNow that the return result is a `DocumentArray`, we can get a summary of it using `r.summary()`.\n\n```text\nโ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Documents Summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ\nโ”‚ โ”‚\nโ”‚ Length 6 โ”‚\nโ”‚ Homogenous Documents False โ”‚\nโ”‚ 4 Documents have attributes ('id', 'mime_type', 'uri', 'embedding') โ”‚\nโ”‚ 1 Document has attributes ('id', 'mime_type', 'text', 'embedding') โ”‚\nโ”‚ 1 Document has attributes ('id', 'embedding') โ”‚\nโ”‚ โ”‚\nโ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ\nโ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Attributes Summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ\nโ”‚ โ”‚\nโ”‚ Attribute Data type #Unique values Has empty value โ”‚\nโ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚\nโ”‚ embedding ('ndarray',) 6 False โ”‚\nโ”‚ id ('str',) 6 False โ”‚\nโ”‚ mime_type ('str',) 5 False โ”‚\nโ”‚ text ('str',) 2 False โ”‚\nโ”‚ uri ('str',) 4 False โ”‚\nโ”‚ โ”‚\nโ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ\n```\n\nTo get the embedding of all Documents, simply call `r.embeddings`:\n\n```text\n[[-0.09136295 0.42720157 -0.05784469 ... -0.42873043 0.04472527\n 0.4437953 ]\n [ 0.43152636 0.1563695 -0.09363698 ... -0.11514216 0.1865044\n 0.15025651]\n [ 0.43152636 0.1563695 -0.09363698 ... -0.11514216 0.1865044\n 0.15025651]\n [ 0.42862126 0.17757078 0.08584607 ... 0.23284511 -0.00929402\n 0.10993651]\n [ 0.4706376 -0.01384148 0.3877237 ... 0.1995864 -0.22621225\n -0.4837676 ]]\n```\n\n```{tip}\nReading an image file into bytes and put into `.blob` is possible as shown above. However, it is often unnecessary. Especially if you have a lot of images, loading all of them into memory is not a good idea. Rule of thumb, always use `.uri` and trust `clip_client` to handle it well. \n```\n\n### Async encoding\n\nTo encode `Document` in an asynchronous manner, one can use {func}`~clip_client.client.Client.aencode`.\n\n```{tip}\nDespite the sexy word \"async\", many data scientists have misconceptions about asynchronous behavior. And their motivation of using async function is often wrong. _Async is not a silver bullet._ In a simple language, you will only need `.aencode()` when there is another concurrent task that is also async. Then you want to \"overlap\" the time spending of these two tasks.\n\nIf your system is sync by design, there is nothing wrong about it. Go with `encode()` until you see a clear advantage of using `aencode()`, or until your boss tell you to do so. \n```\n\nIn the following example, there is another job `another_heavylifting_job` to represent a job like writing to database, downloading large file.\n\n```python\nimport asyncio\nfrom clip_client import Client\n\nc = Client('grpc://0.0.0.0:23456')\n\n\nasync def another_heavylifting_job():\n # can be writing to database, downloading large file\n # big IO ops\n await asyncio.sleep(3)\n\n\nasync def main():\n t1 = asyncio.create_task(another_heavylifting_job())\n t2 = asyncio.create_task(c.aencode(['hello world'] * 100))\n await asyncio.gather(t1, t2)\n\n\nasyncio.run(main())\n```\n\nThe final time cost will be less than `3s + time(t2)`.\n\n(rank-api)=\n## Ranking\n\n```{tip}\nThis feature is only available with `clip_server>=0.3.0`.\n```\n\nOne can also rank cross-modal matches via {meth}`~clip_client.client.Client.rank` or {meth}`~clip_client.client.Client.arank`. First construct a cross-modal `Document` where the root contains an image and `.matches` contain sentences to rerank. One can also construct text-to-image rerank as below:\n\n````{tab} Given image, rank sentences\n\n```python\nfrom docarray import Document\n\nd = Document(\n uri='.github/README-img/rerank.png',\n matches=[\n Document(text=f'a photo of a {p}')\n for p in (\n 'control room',\n 'lecture room',\n 'conference room',\n 'podium indoor',\n 'television studio',\n )\n ],\n)\n```\n\n````\n\n````{tab} Given sentence, rank images\n\n```python\nfrom docarray import Document\n\nd = Document(\n text='a photo of conference room',\n matches=[\n Document(uri='.github/README-img/4.png'),\n Document(uri='.github/README-img/9.png'),\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n ],\n)\n```\n\n````\n\n\n\nThen call `rank`, you can feed it with multiple Documents as a list:\n\n```python\nfrom clip_client import Client\n\nc = Client(server='grpc://0.0.0.0:23456')\nr = c.rank([d])\n\nprint(r['@m', ['text', 'scores__clip_score__value']])\n```\n\nFinally, in the return you can observe the matches are re-ranked according to `.scores['clip_score']`:\n\n```text\n[['a photo of a television studio', 'a photo of a conference room', 'a photo of a lecture room', 'a photo of a control room', 'a photo of a podium indoor'], \n[0.9920725226402283, 0.006038925610482693, 0.0009973491542041302, 0.00078492151806131, 0.00010626466246321797]]\n```\n\n(indexing)=\n## Indexing\n\n```{tip}\nThis feature is only available with `clip_client>=0.7.0`, and the server is running with \na FLOW consisting of encoder and indexer.\n``` \n\nYou can index Documents via {func}`~clip_client.client.Client.index` or {func}`~clip_client.client.Client.aindex`. \n\n```python\nfrom clip_client import Client\nfrom docarray import Document\n\nc = Client('grpc://0.0.0.0:23456')\n\nda = [\n Document(text='she smiled, with pain'),\n Document(uri='apple.png'),\n Document(uri='apple.png').load_uri_to_image_tensor(),\n Document(blob=open('apple.png', 'rb').read()),\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n Document(\n uri='data:image/gif;base64,R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o/XBs/fNwfjZ0frl3/zy7////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7'\n ),\n]\n\nr = c.index(da)\n```\nNow that the return result is a DocumentArray, we can get a summary of it.\n\n```text\nโ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Documents Summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ\nโ”‚ โ”‚\nโ”‚ Length 6 โ”‚\nโ”‚ Homogenous Documents False โ”‚\nโ”‚ 4 Documents have attributes ('id', 'mime_type', 'uri', 'embedding') โ”‚\nโ”‚ 1 Document has attributes ('id', 'mime_type', 'text', 'embedding') โ”‚\nโ”‚ 1 Document has attributes ('id', 'embedding') โ”‚\nโ”‚ โ”‚\nโ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ\nโ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Attributes Summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ\nโ”‚ โ”‚\nโ”‚ Attribute Data type #Unique values Has empty value โ”‚\nโ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚\nโ”‚ embedding ('ndarray',) 6 False โ”‚\nโ”‚ id ('str',) 6 False โ”‚\nโ”‚ mime_type ('str',) 5 False โ”‚\nโ”‚ text ('str',) 2 False โ”‚\nโ”‚ uri ('str',) 4 False โ”‚\nโ”‚ โ”‚\nโ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ\n```\n\nThe `embedding` is the output of the encoder, which is a 512-dim vector. \nNow we can use the indexer to search for the indexed Documents.\n\n\n(searching)=\n## Searching\n\n```{tip}\nThis feature is only available with `clip_client>=0.7.0`, and the server is running with \na FLOW consisting of encoder and indexer.\n``` \n\nYou can use {func}`~clip_client.client.Client.search` or {func}`~clip_client.client.Client.asearch`\nto search for relevant Documents in the index for a given query.\n\n```python\nfrom clip_client import Client\n\nc = Client('grpc://0.0.0.0:23456')\n\nresult = c.search(['smile'], limit=2)\n\nprint(result['@m', ['text', 'scores__cosine']])\n```\n\nThe results will look like this, the most relevant doc is \"she smiled, with pain\" with the cosine distance of 0.096. And the apple image has the cosine distance of 0.799.\n```text\n[['she smiled, with pain', ''], [{'value': 0.09604918956756592}, {'value': 0.7994111776351929}]]\n```\nYou can set the `limit` parameter (default is `10`) to control the number of the most similar documents to be retrieved.\n\n\n\n(profiling)=\n## Profiling\n\nYou can use {func}`~clip_client.client.Client.profile` to give a quick test on the server to make sure everything is good. \n\n```python\nfrom clip_client import Client\n\nc = Client('grpc://0.0.0.0:23456')\n\nc.profile()\n```\n\nThis give you a tree-like table showing the latency and percentage.\n\n```text\n Roundtrip 16ms 100% \nโ”œโ”€โ”€ Client-server network 12ms 75% \nโ””โ”€โ”€ Server 4ms 25% \n โ”œโ”€โ”€ Gateway-CLIP network 0ms 0% \n โ””โ”€โ”€ CLIP model 4ms 100% \n```\n\nUnder the hood, `.profile()` sends a single empty Document to the CLIP-server for encoding and calculates a summary of latency. The above tree can be read as follows: \n\n- From calling `client.encode()` to returning the results, everything counted, takes 16ms to finish.\n- Among them the time spent on the server is 4ms, the remaining 12ms is spent on the client-server communication, request packing, response unpacking.\n- During the 4ms server processing time, CLIP model takes 4ms, whereas the [Gateway](https://docs.jina.ai/fundamentals/architecture-overview/#architecture-overview) to CLIP communication takes no time.\n\n`.profile()` can also take a string argument and asks CLIP-server to encode it. This string can be a sentence, local/remote image file URI. For example:\n\n```python\nc.profile('hello, world')\nc.profile('apple.png')\nc.profile('https://docarray.jina.ai/_static/favicon.png')\n```\n\n\nSingle query latency is often very fluctuated. Running `.profile()` multiple times may give you different results. Nonetheless, it helps you understand who to blame if CLIP-as-service is running slow for you: the network? the computation? But certainly not this software itself.\n\n\n## Best practices\n\nIn this section, we will show you some best practices for using this client. We will use encoding as an example. The same applies to all other methods.\n\n### Control batch size\n\nYou can specify `.encode(..., batch_size=8)` to control how many `Document`s are sent in each request. You can play this number and find the perfect balance between network transmission and GPU utilization. \n\nIntuitively, setting `batch_size=1024` should result in very high GPU utilization on each request. However, a large batch size like this also means sending each request would take longer. Given that `clip-client` is designed with request and response streaming, large batch size would not benefit from the time overlapping between request streaming and response streaming.\n\n### Control prefetch size\n\nTo control the number of in-flight batches, you can use the `.encode(..., prefetch=100)` option. \nThe way this works is that when you send a large request, the outgoing request stream will usually finish before the incoming response stream due to the asynchronous design. \nThis is because the request handling is typically time-consuming, which can prevent the server from sending back the response and may cause it to close the connection as it thinks the incoming channel is idle. \nBy default, the client is set to a prefetch value of 100. However, it is recommended to use a lower value for expensive operations and a higher value for faster response times.\n\nFor more information about client prefetching, please refer to [Rate Limit](https://docs.jina.ai/concepts/client/rate-limit/) in Jina documentation.\n\n### Show progressbar\n\nYou can use `.encode(..., show_progress=True)` to turn on the progress bar.\n\n```{figure} images/client-pgbar.gif\n:width: 80%\n```\n\n```{hint}\nProgress bar may not show up in the PyCharm debug terminal. This is an upstream issue of `rich` package.\n```\n\n### Processing large number of Documents\n\nHere are some suggestions when encoding a large number of `Document`s:\n\n1. Use `Generator` as input to load data on-demand. You can put your data into a Generator and feed to `.encode`:\n ```python\n def data_gen():\n for _ in range(100_000):\n yield Document(uri=...)\n\n\n c = Client(...)\n c.encode(data_gen())\n ```\n Yield raw strings is also acceptable, e.g. to encode all images under a directory, you can simply do:\n ```python\n from glob import iglob\n\n c.encode(iglob('**/*.png'))\n ```\n2. Adjust the `batch_size` parameters.\n3. Adjust the `prefetch` parameters.\n4. Turn on the progressbar.\n\n````{danger}\nIn any case, avoiding the following coding:\n\n```python\nfor d in big_list:\n c.encode([d])\n```\n\nThis is extremely slow as only one document is encoded at a time, it is a bad utilization of the network and not leveraging any duplex streaming.\n````\n\n### Custom callback\n\n`clip_client` by default collects all the results and returns them to users. However, if you want to process the results on-the-fly, you can also pass a callback function when sending the request. For example, you can use the callback to save the results to a database, or render the results to a webpage. Specifically, you can specify any of the three callback functions: `on_done`, `on_error`, and `on_always`.\n\n- `on_done` is executed while streaming, after successful completion of each request\n- `on_error` is executed while streaming, whenever an error occurs in each request\n- `on_always` is always performed while streaming, no matter the success or failure of each request\n\nNote that these callbacks only work for requests (and failures) inside the stream. For `on_error`, if the failure is due to an error happening outside of streaming, then it will not be triggered. For example, a `SIGKILL` from the client OS during the handling of the request, or a networking issue, will not trigger the callback. Learn more about [handling exceptions in `on_error`](https://docs.jina.ai/concepts/client/callbacks/#handle-exceptions-in-callbacks).\n\nCallback functions take a `Response` of the type DataRequest, which contains resulting Documents, parameters, and other information. Learn more about [handling `DataRequest` in callbacks](https://docs.jina.ai/concepts/client/callbacks/#handle-datarequest-in-callbacks).\n\nIn the following example, we will use `on_done` to save the results to a database. We use a simple `dict` to simulate the database. The error is saved to log file using `on_error`. `on_always` will print the number of documents processed in each request.\n\n```python\nfrom clip_client import Client\n\ndb = {}\n\n\ndef my_on_done(resp):\n for doc in resp.docs:\n db[doc.id] = doc\n\n\ndef my_on_error(resp):\n with open('error.log', 'a') as f:\n f.write(resp)\n\n\ndef my_on_always(resp):\n print(f'{len(resp.docs)} docs processed')\n\n\nc = Client('grpc://0.0.0.0:12345')\nc.encode(\n ['hello', 'world'], on_done=my_on_done, on_error=my_on_error, on_always=my_on_always\n)\n```\n\n```{note}\nIf either `on_done` or `on_always` is specified, the default behavior of returning the results is disabled. You need to handle the results yourself.\n```\n\n### Client parallelism\n\nIn case you instanciate a `clip_client` object using the `grpc` protocol, keep in mind that `grpc` clients cannot be used in a multi-threaded environment (check [this gRPC issue](https://github.com/grpc/grpc/issues/25364) for reference).\nWhat you should do, is to rely on asynchronous programming or multi-processing rather than multi-threading.\n\nTo use `clip_client` in a Flask application, you can introduce multi-processing based parallelism to your app using `gunicorn`: \n\n```bash\ngunicorn -w 4 -b 127.0.0.1:4000 myproject:app\n```\n\nTo use `clip_client` in a FastAPI application, you have to manually restrict the thread number to 1 at the starting state of the app:\n\n```python \nimport uvicorn\nfrom fastapi import FastAPI\nfrom clip_client import Client\nfrom anyio.lowlevel import RunVar\nfrom anyio import CapacityLimiter\n\nc = Client('grpc://0.0.0.0:51001')\napp = FastAPI()\n\n@app.on_event(\"startup\")\ndef startup():\n print(\"start\")\n RunVar(\"_default_thread_limiter\").set(CapacityLimiter(1))\n\n@app.post(\"/\")\ndef encode():\n r = c.encode(['Hello world', 'Hello Jina'])\n print(r)\n```\n\nThen it can run with multiprocessing using\n\n```bash\ngunicorn myproject:app --workers 4 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:4000\n```\n\n## Appendix: Plain HTTP request via `curl`\n\n```{tip}\nSending large embeddings over plain HTTP is often not the best idea. Websocket is often a better choice, allows one to call clip-server from Javascript with much better performance. \n```\n\nIf your {ref}`server is spawned` with `protocol: http` and `cors: True`, then you do not need to call the server via Python client. You can simply do it via `curl` or Javascript by sending a JSON to `http://address:port/post`. Notice, the `/post` endpoint at the end. For example,\n\nTo encode sentences:\n\n```{code-block} bash\n---\nemphasize-lines: 3\n---\ncurl -X POST http://0.0.0.0:51000/post \\ \n -H 'Content-Type: application/json' \\\n -d '{\"data\":[{\"text\": \"First do it\"}, {\"text\": \"then do it right\"}, {\"text\": \"then do it better\"}], \"execEndpoint\":\"/\"}'\n```\n\nTo encode a local image, you need to load it as base64 string and put into the `blob` field, and be careful with the quotes there:\n\n```{code-block} bash\n---\nemphasize-lines: 3\n---\ncurl -X POST http://0.0.0.0:51000/post \\ \n -H 'Content-Type: application/json' \\\n -d '{\"data\":[{\"text\": \"First do it\"}, {\"blob\":\"'\"$( base64 test-1.jpeg)\"'\" }], \"execEndpoint\":\"/\"}'\n```\n\nTo encode a remote image, you can simply put its address into `uri` field:\n\n```{code-block} bash\n---\nemphasize-lines: 3\n---\ncurl -X POST http://0.0.0.0:51000/post \\ \n -H 'Content-Type: application/json' \\\n -d '{\"data\":[{\"text\": \"First do it\"}, {\"uri\": \"https://clip-as-service.jina.ai/_static/favicon.png\"}], \"execEndpoint\":\"/\"}'\n```\n\nRun it, you will get:\n\n```json\n{\"header\":{\"requestId\":\"8b1f4b419bc54e95ab4b63cc086233c9\",\"status\":null,\"execEndpoint\":\"/\",\"targetExecutor\":\"\"},\"parameters\":null,\"routes\":[{\"executor\":\"gateway\",\"startTime\":\"2022-04-01T15:24:28.267003+00:00\",\"endTime\":\"2022-04-01T15:24:28.328868+00:00\",\"status\":null},{\"executor\":\"clip_t\",\"startTime\":\"2022-04-01T15:24:28.267189+00:00\",\"endTime\":\"2022-04-01T15:24:28.328748+00:00\",\"status\":null}],\"data\":[{\"id\":\"b15331b8281ffde1e9fb64005af28ffd\",\"parent_id\":null,\"granularity\":null,\"adjacency\":null,\"blob\":null,\"tensor\":null,\"mime_type\":\"text/plain\",\"text\":\"hello, world!\",\"weight\":null,\"uri\":null,\"tags\":null,\"offset\":null,\"location\":null,\"embedding\":[-0.022064208984375,0.1044921875, ..., -0.1363525390625,-0.447509765625],\"modality\":null,\"evaluations\":null,\"scores\":null,\"chunks\":null,\"matches\":null}]}\n```\n\nThe embedding is inside `.data[].embedding`. If you have [jq](https://stedolan.github.io/jq/) installed, you can easily filter the embeddings out via:\n\n```{code-block} bash\n---\nemphasize-lines: 4\n---\ncurl -X POST http://0.0.0.0:51000/post \\\n -H 'Content-Type: application/json' \\\n -d '{\"data\":[{\"text\": \"hello, world!\"}, {\"blob\":\"'\"$( base64 test-1.jpeg)\"'\" }], \"execEndpoint\":\"/\"}' | \\\n jq -c '.data[] | .embedding'\n```\n\n```text\n[-0.022064208984375,0.1044921875,...]\n[-0.0750732421875,-0.166015625,...]\n```\n" }, { "path": "docs/user-guides/faq.md", "content": "# FAQ\n\nThis is a list of Frequently Asked Questions about CLIP-as-service. Feel free to suggest new entries!\n\n\nWhat is CLIP model?\n: Developed by OpenAI, CLIP (Contrastive Language-Image Pre-Training) is a neural network trained on a variety of (image, text) pairs. The original CLIP Github repository [is here](https://github.com/openai/CLIP). The introduction of the CLIP model can [be found here](https://openai.com/blog/clip/).\n\nDo I need to install `clip-server` and `clip-client` together?\n: No. You can install them separately on different machines. For example, on a GPU server, you just need `clip-server`; on your laptop, you just need `clip-client`.\n\nWhat is CLIP-as-service based on? The codebase seems quite small\n: CLIP-as-service leverages features from [Jina](https://github.com/jina-ai/jina), which itself utilizes [DocArray](https://github.com/jina-ai/docarray). Thanks to them CLIP-as-service can be quickly built with solid infrastructure and rich features.\n\nI had this AioRpcError, what should I do?\n: If you encounter the following errors, it means you client can not connect to the server.\n\n ```text\n GRPCClient@28632[E]:gRPC error: StatusCode.UNAVAILABLE failed to connect to all addresses\n the ongoing request is terminated as the server is not available or closed already\n ```\n\n ```text\n AioRpcError: `. If it still throws the same error, then your connection is broken.\n\n While it is hard to pinpoint a network problem, also out of the scope of CLIP-as-service, we here provide you a checklist that may help you to diagnose the problem: \n - Are the IP address, port, and protocol all correct?\n - Is client and server under the same network, or a different network?\n - Is your server down?\n - Is server's port open to public?\n - Is there a firewall on the server side that restricts the port?\n - Is there a firewall on the client side that restricts the port?\n - Is the security group (on Cloud providers) correctly configured?\n\nWhy \"CLIP-as-service\" why not \"CLIP-as-a-service\"\n: Kind of pay homage to BERT-as-service. It is not about grammatically correct anyhow.\n\nWhat happened to the BERT-as-service.\n: There has been no maintenance of BERT-as-service since Feb. 2019.\n\n CLIP-as-service is a huge upgrade of BERT-as-service, with more powerful universal embedding models that can handle both images and texts; and more solid and efficient microservice infrastructure developed in the last 2 years by Jina AI. The high-level API, especially the client side, is a drop-in replacement of the old BERT-as-service.\n\nWhere can I find the old codebase of BERT-as-service.\n: In the [`bert-as-service` branch](https://github.com/jina-ai/clip-as-service/tree/bert-as-service) of the repository." }, { "path": "docs/user-guides/finetuner.md", "content": "(Finetuner)=\n# Fine-tune Models\n\nAlthough CLIP-as-service has provided you a list of pre-trained models, you can also fine-tune your models. \nThis guide will show you how to use [Finetuner](https://finetuner.jina.ai) to fine-tune models and use them in CLIP-as-service.\n\nFor installation and basic usage of Finetuner, please refer to [Finetuner documentation](https://finetuner.jina.ai).\nYou can also [learn more details about fine-tuning CLIP](https://finetuner.jina.ai/tasks/text-to-image/).\n\nThis tutorial requires `finetuner >=v0.6.4`, `clip_server >=v0.6.0`.\n\n## Prepare Training Data\n\nFinetuner accepts training data and evaluation data in the form of {class}`~docarray.array.document.DocumentArray`.\nThe training data for CLIP is a list of (text, image) pairs.\nEach pair is stored in a {class}`~docarray.document.Document` which wraps two [`chunks`](https://docarray.jina.ai/fundamentals/document/nested/) with `image` and `text` modality respectively.\nYou can push the resulting {class}`~docarray.array.document.DocumentArray` to the cloud using the {meth}`~docarray.array.document.DocumentArray.push` method.\n\nWe use [fashion captioning dataset](https://github.com/xuewyang/Fashion_Captioning) as a sample dataset in this tutorial.\nThe following are examples of descriptions and image urls from the dataset.\nWe also include a preview of each image.\n\n| Description | Image URL | Preview |\n|---------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------|\n| subtly futuristic and edgy this liquid metal cuff bracelet is shaped from sculptural rectangular link | [https://n.nordstrommedia.com/id/sr3/
    58d1a13f-b6b6-4e68-b2ff-3a3af47c422e.jpeg](https://n.nordstrommedia.com/id/sr3/58d1a13f-b6b6-4e68-b2ff-3a3af47c422e.jpeg) | |\n| high quality leather construction defines a hearty boot one-piece on a tough lug sole | [https://n.nordstrommedia.com/id/sr3/
    21e7a67c-0a54-4d09-a4a4-6a0e0840540b.jpeg](https://n.nordstrommedia.com/id/sr3/21e7a67c-0a54-4d09-a4a4-6a0e0840540b.jpeg) | |\n| this shimmering tricot knit tote is traced with decorative whipstitching and diamond cut chain the two hallmark of the falabella line | [https://n.nordstrommedia.com/id/sr3/
    1d8dd635-6342-444d-a1d3-4f91a9cf222b.jpeg](https://n.nordstrommedia.com/id/sr3/1d8dd635-6342-444d-a1d3-4f91a9cf222b.jpeg) | |\n| ... | ... | ... |\n\nYou can use the following script to transform the first three entries of the dataset to a {class}`~docarray.array.document.DocumentArray` and push it to the cloud using the name `fashion-sample`.\n\n```python\nfrom docarray import Document, DocumentArray\n\ntrain_da = DocumentArray(\n [\n Document(\n chunks=[\n Document(\n content='subtly futuristic and edgy this liquid metal cuff bracelet is shaped from sculptural rectangular link',\n modality='text',\n ),\n Document(\n uri='https://n.nordstrommedia.com/id/sr3/58d1a13f-b6b6-4e68-b2ff-3a3af47c422e.jpeg',\n modality='image',\n ),\n ],\n ),\n Document(\n chunks=[\n Document(\n content='high quality leather construction defines a hearty boot one-piece on a tough lug sole',\n modality='text',\n ),\n Document(\n uri='https://n.nordstrommedia.com/id/sr3/21e7a67c-0a54-4d09-a4a4-6a0e0840540b.jpeg',\n modality='image',\n ),\n ],\n ),\n Document(\n chunks=[\n Document(\n content='this shimmering tricot knit tote is traced with decorative whipstitching and diamond cut chain the two hallmark of the falabella line',\n modality='text',\n ),\n Document(\n uri='https://n.nordstrommedia.com/id/sr3/1d8dd635-6342-444d-a1d3-4f91a9cf222b.jpeg',\n modality='image',\n ),\n ],\n ),\n ]\n)\ntrain_da.push('fashion-sample')\n```\n\nThe full dataset has been converted to `clip-fashion-train-data` and `clip-fashion-eval-data` and pushed to the cloud which can be directly used in Finetuner.\n\n## Start Finetuner\n\nYou may now create and run a fine-tuning job after login to Jina ecosystem.\n\n```python\nimport finetuner\n\nfinetuner.login()\nrun = finetuner.fit(\n model='ViT-B-32::openai',\n run_name='clip-fashion',\n train_data='clip-fashion-train-data',\n eval_data='clip-fashion-eval-data', # optional\n epochs=5,\n learning_rate=1e-5,\n loss='CLIPLoss',\n to_onnx=True,\n)\n```\n\nAfter the job started, you may use {meth}`~finetuner.run.Run.status` to check the status of the job.\n\n```python\nimport finetuner\n\nfinetuner.login()\nrun = finetuner.get_run('clip-fashion')\nprint(run.status())\n```\n\nWhen the status is `FINISHED`, you can download the tuned model to your local machine.\n\n```python\nimport finetuner\n\nfinetuner.login()\nrun = finetuner.get_run('clip-fashion')\nrun.save_artifact('clip-model')\n```\n\nYou should now get a zip file containing the tuned model named `clip-fashion.zip` under the folder `clip-model`.\n\n## Use the Model\n\nAfter unzipping the model you get from the previous step, a folder with the following structure will be generated:\n\n```text\n.\nโ””โ”€โ”€ clip-fashion/\n โ”œโ”€โ”€ config.yml\n โ”œโ”€โ”€ metadata.yml\n โ”œโ”€โ”€ metrics.yml\n โ””โ”€โ”€ models/\n โ”œโ”€โ”€ clip-text/\n โ”‚ โ”œโ”€โ”€ metadata.yml\n โ”‚ โ””โ”€โ”€ model.onnx\n โ”œโ”€โ”€ clip-vision/\n โ”‚ โ”œโ”€โ”€ metadata.yml\n โ”‚ โ””โ”€โ”€ model.onnx\n โ””โ”€โ”€ input-map.yml\n```\n\nSince the tuned model generated from Finetuner contains richer information such as metadata and config, we now transform it to simpler structure used by CLIP-as-service.\n\n* Firstly, create a new folder named `clip-fashion-cas` or name of your choice. This will be the storage of the models to use in CLIP-as-service.\n\n* Secondly, copy the textual model `clip-fashion/models/clip-text/model.onnx` into the folder `clip-fashion-cas` and rename the model to `textual.onnx`.\n\n* Similarly, copy the visual model `clip-fashion/models/clip-vision/model.onnx` into the folder `clip-fashion-cas` and rename the model to `visual.onnx`.\n\nThis is the expected structure of `clip-fashion-cas`:\n\n```text\n.\nโ””โ”€โ”€ clip-fashion-cas/\n โ”œโ”€โ”€ textual.onnx\n โ””โ”€โ”€ visual.onnx\n```\n\nIn order to use the fine-tuned model, create a custom YAML file `finetuned_clip.yml` like below. Learn more about [Flow YAML configuration](https://docs.jina.ai/fundamentals/flow/yaml-spec/) and [`clip_server` YAML configuration](https://clip-as-service.jina.ai/user-guides/server/#yaml-config).\n\n```yaml\njtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_o\n uses:\n jtype: CLIPEncoder\n metas:\n py_modules:\n - clip_server.executors.clip_onnx\n with:\n name: ViT-B-32::openai\n model_path: 'clip-fashion-cas' # path to clip-fashion-cas\n replicas: 1\n```\n\nYou can use `finetuner.describe_models()` to check the supported models in `finetuner`, you should see:\n```bash\n Finetuner backbones \nโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“\nโ”ƒ name โ”ƒ task โ”ƒ output_dim โ”ƒ architecture โ”ƒ description โ”ƒ\nโ”กโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ\nโ”‚ bert-base-cased โ”‚ text-to-text โ”‚ 768 โ”‚ transformer โ”‚ BERT model pre-trained on BookCorpus and English Wikipedia โ”‚\nโ”‚ openai/clip-vit-base-patch16 โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ CLIP base model with patch size 16 โ”‚\nโ”‚ openai/clip-vit-base-patch32 โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ CLIP base model โ”‚\nโ”‚ openai/clip-vit-large-patch14-336 โ”‚ text-to-image โ”‚ 768 โ”‚ transformer โ”‚ CLIP large model for 336x336 images โ”‚\nโ”‚ openai/clip-vit-large-patch14 โ”‚ text-to-image โ”‚ 1024 โ”‚ transformer โ”‚ CLIP large model with patch size 14 โ”‚\nโ”‚ efficientnet_b0 โ”‚ image-to-image โ”‚ 1280 โ”‚ cnn โ”‚ EfficientNet B0 pre-trained on ImageNet โ”‚\nโ”‚ efficientnet_b4 โ”‚ image-to-image โ”‚ 1792 โ”‚ cnn โ”‚ EfficientNet B4 pre-trained on ImageNet โ”‚\nโ”‚ RN101::openai โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"RN101::openai\" model โ”‚\nโ”‚ RN101-quickgelu::openai โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"RN101-quickgelu::openai\" model โ”‚\nโ”‚ RN101-quickgelu::yfcc15m โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"RN101-quickgelu::yfcc15m\" model โ”‚\nโ”‚ RN101::yfcc15m โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"RN101::yfcc15m\" model โ”‚\nโ”‚ RN50::cc12m โ”‚ text-to-image โ”‚ 1024 โ”‚ transformer โ”‚ Open CLIP \"RN50::cc12m\" model โ”‚\nโ”‚ RN50::openai โ”‚ text-to-image โ”‚ 1024 โ”‚ transformer โ”‚ Open CLIP \"RN50::openai\" model โ”‚\nโ”‚ RN50-quickgelu::cc12m โ”‚ text-to-image โ”‚ 1024 โ”‚ transformer โ”‚ Open CLIP \"RN50-quickgelu::cc12m\" model โ”‚\nโ”‚ RN50-quickgelu::openai โ”‚ text-to-image โ”‚ 1024 โ”‚ transformer โ”‚ Open CLIP \"RN50-quickgelu::openai\" model โ”‚\nโ”‚ RN50-quickgelu::yfcc15m โ”‚ text-to-image โ”‚ 1024 โ”‚ transformer โ”‚ Open CLIP \"RN50-quickgelu::yfcc15m\" model โ”‚\nโ”‚ RN50x16::openai โ”‚ text-to-image โ”‚ 768 โ”‚ transformer โ”‚ Open CLIP \"RN50x16::openai\" model โ”‚\nโ”‚ RN50x4::openai โ”‚ text-to-image โ”‚ 640 โ”‚ transformer โ”‚ Open CLIP \"RN50x4::openai\" model โ”‚\nโ”‚ RN50x64::openai โ”‚ text-to-image โ”‚ 1024 โ”‚ transformer โ”‚ Open CLIP \"RN50x64::openai\" model โ”‚\nโ”‚ RN50::yfcc15m โ”‚ text-to-image โ”‚ 1024 โ”‚ transformer โ”‚ Open CLIP \"RN50::yfcc15m\" model โ”‚\nโ”‚ ViT-B-16::laion400m_e31 โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"ViT-B-16::laion400m_e31\" model โ”‚\nโ”‚ ViT-B-16::laion400m_e32 โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"ViT-B-16::laion400m_e32\" model โ”‚\nโ”‚ ViT-B-16::openai โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"ViT-B-16::openai\" model โ”‚\nโ”‚ ViT-B-16-plus-240::laion400m_e31 โ”‚ text-to-image โ”‚ 640 โ”‚ transformer โ”‚ Open CLIP \"ViT-B-16-plus-240::laion400m_e31\" model โ”‚\nโ”‚ ViT-B-16-plus-240::laion400m_e32 โ”‚ text-to-image โ”‚ 640 โ”‚ transformer โ”‚ Open CLIP \"ViT-B-16-plus-240::laion400m_e32\" model โ”‚\nโ”‚ ViT-B-32::laion2b_e16 โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"ViT-B-32::laion2b_e16\" model โ”‚\nโ”‚ ViT-B-32::laion400m_e31 โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"ViT-B-32::laion400m_e31\" model โ”‚\nโ”‚ ViT-B-32::laion400m_e32 โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"ViT-B-32::laion400m_e32\" model โ”‚\nโ”‚ ViT-B-32::openai โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"ViT-B-32::openai\" model โ”‚\nโ”‚ ViT-B-32-quickgelu::laion400m_e31 โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"ViT-B-32-quickgelu::laion400m_e31\" model โ”‚\nโ”‚ ViT-B-32-quickgelu::laion400m_e32 โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"ViT-B-32-quickgelu::laion400m_e32\" model โ”‚\nโ”‚ ViT-B-32-quickgelu::openai โ”‚ text-to-image โ”‚ 512 โ”‚ transformer โ”‚ Open CLIP \"ViT-B-32-quickgelu::openai\" model โ”‚\nโ”‚ ViT-L-14-336::openai โ”‚ text-to-image โ”‚ 768 โ”‚ transformer โ”‚ Open CLIP \"ViT-L-14-336::openai\" model โ”‚\nโ”‚ ViT-L-14::openai โ”‚ text-to-image โ”‚ 768 โ”‚ transformer โ”‚ Open CLIP \"ViT-L-14::openai\" model โ”‚\nโ”‚ resnet152 โ”‚ image-to-image โ”‚ 2048 โ”‚ cnn โ”‚ ResNet152 pre-trained on ImageNet โ”‚\nโ”‚ resnet50 โ”‚ image-to-image โ”‚ 2048 โ”‚ cnn โ”‚ ResNet50 pre-trained on ImageNet โ”‚\nโ”‚ sentence-transformers/msmarco-distilbert-base-v3 โ”‚ text-to-text โ”‚ 768 โ”‚ transformer โ”‚ Pretrained BERT, fine-tuned on MS Marco โ”‚\nโ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€\n```\n\n\nYou can now start the `clip_server` using fine-tuned model to get a performance boost:\n\n```bash\npython -m clip_server finetuned_clip.yml\n```\n\nThat's it, enjoy ๐Ÿš€\n" }, { "path": "docs/user-guides/retriever.md", "content": "# CLIP Search\n\n\nCLIP Search is a search paradigm that uses the CLIP model to encode the text and image documents into a common vector space. \nThe search results are then retrieved by computing the cosine similarity between the query and the indexed documents.\nTechnically, CLIP search can be designed as a two-stage process: *encoding* and *indexing*.\n\n```{figure} images/retreival.png\n:width: 80%\n```\n\nAt the encoding stage, the text and image documents can be encoded into a common vector space by the CLIP model. \nIt enables us to achieve cross-modal search, i.e., we can search for images given a text query, or search for text given an image query. \nAt the indexing stage, we use the encoded vectors to build an index, which is a data structure that can be used to efficiently retrieve the most relevant documents.\nSpecifically, we use the [Annlite](https://github.com/jina-ai/annlite) indexer executor to build the index.\n\nThis chapter will walk you through the process of building a CLIP search system.\n\n\n```{tip}\nYou will need to install server first in Python 3.7+: `pip install clip-server[search]>=0.7.0`.\n```\n\n## Start the server\n\nTo start the server, you can use the following command:\n\n```bash\npython -m clip_server search_flow.yml\n```\n\nThe `search_flow.yml` is the yaml configuration file for the search flow. It defines a [Jina Flow](https://docs.jina.ai/fundamentals/flow/) to implement the CLIP search system.\nBelow is an example of the Flow YAML file, we can put it into two subsections as below:\n\n````{tab} CLIP model config\n\n```{code-block} yaml\n---\nemphasize-lines: 9\n---\n\njtype: Flow\nversion: '1'\nwith:\n port: 61000\nexecutors:\n - name: encoder\n uses:\n jtype: CLIPEncoder\n metas:\n py_modules:\n - clip_server.executors.clip_torch\n \n - name: indexer\n uses:\n jtype: AnnLiteIndexer\n with:\n n_dim: 512\n metas:\n py_modules:\n - annlite.executor\n workspace: './workspace'\n```\n\n````\n\n````{tab} Annlite indexer config\n\n```{code-block} yaml\n---\nemphasize-lines: 17,18,19\n---\n\njtype: Flow\nversion: '1'\nwith:\n port: 61000\nexecutors:\n - name: encoder\n uses:\n jtype: CLIPEncoder\n with:\n metas:\n py_modules:\n - clip_server.executors.clip_torch\n \n - name: indexer\n uses:\n jtype: AnnLiteIndexer\n with:\n n_dim: 512\n limit: 10\n metas:\n py_modules:\n - annlite.executor\n workspace: './workspace'\n```\n\n````\n\nThe first part defines the CLIP model config, which is explained [here](https://clip-as-service.jina.ai/user-guides/server/#clip-model-config).\nAnd the second part defines the Annlite indexer config, you can set the following parameters:\n\n| Parameter | Description |\n|-----------|----------------------------------------------------------------------------------------------|\n| `n_dim` | The dimension of the vector space. It should be the same as the dimension of the CLIP model. |\n| `limit` | The number of the most relevant documents to be retrieved. The default value is 10. |\n\nAnd the `workspace` parameter is the path to the workspace directory, which is used to store the index files.\n\n## Index and search documents\n\n```{tip}\nYou will need to install client first in Python 3.7+: `pip install clip-client>=0.7.0`.\n```\n\n### Index Documents\n\nTo index image or text documents in the CLIP search server, you can use the client function {func}`~clip_client.Client.index`:\n\n```python\nfrom clip_client import Client\nfrom docarray import Document\n\nclient = Client('grpc://0.0.0.0:61000')\n\nclient.index(\n [\n Document(text='she smiled, with pain'),\n Document(uri='apple.png'),\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n ]\n)\n```\n\nYou don't need to call `client.encode()` explicitly since `client.index()` will handle this for you.\n\n\n### Search Documents\n\nThen, you can use the client function {func}`~clip_client.Client.search` to search for similar documents:\n\n```python\nresult = client.search(['smile'], limit=2)\n\nprint(result['@m', ['text', 'scores__cosine']])\n```\n\nThe results will look like this, the most relevant doc is \"she smiled, with pain\" with the cosine distance of 0.096. And the apple image has the cosine distance of 0.799.\n```text\n[['she smiled, with pain', ''], [{'value': 0.09604918956756592}, {'value': 0.7994111776351929}]]\n```\nYou can set the `limit` parameter (default is `10`) to control the number of the most similar documents to be retrieved.\n\n\n### Memory Estimation\n\nHere, we will show how to estimate the memory usage of `AnnLite` indexer.\nThis is useful for determining the amount of memory required for indexing and querying.\n\nIn `AnnLite`, the memory usage is determined by the following two components:\n\n- `HNSW` indexer: N * 1.1 * (4 bytes * `dimension` + 8 bytes * `max_connection`), where N is the number of embedding vectors, `dimension` is the dimension of the embedding vectors, and `max_connection` is the maximum number of connections in the graph. \n- `cell_table`: it's almost linear to the number of columns and number of data. If the default setting is used (no columns used for filtering), the memory usage of `cell_table` is 0.12GB per million data.\nColumns used for filtering are stored in string type so the memory usage is depended on the length of the string.\n\n```{Notice}\nIf you use `AnnLiteIndexer` in your Jina Flow, the memory usage will be slightly higher since we keep a `SQLite` table in memory in order to indexing in `DocumentArray`.\n```\n\n\n## Support large-scale dataset\n\nWhen we want to index a large number of documents, for example, 100 million data or even 1 billion data, \nit's not possible to implement index operations on a single machine. **Sharding**, \na type of partitioning that separates a large dataset into smaller, faster, more easily managed parts, is needed in this case.\n\nYou need to specify the `shards` and `polling` in the YAML config:\n\n```yaml\njtype: Flow\nversion: '1'\nwith:\n port: 61000\nexecutors:\n - name: encoder\n uses:\n jtype: CLIPEncoder\n metas:\n py_modules:\n - clip_server.executors.clip_torch\n \n - name: indexer\n uses:\n jtype: AnnLiteIndexer\n with:\n n_dim: 512\n metas:\n py_modules:\n - annlite.executor\n workspace: './workspace'\n shards: 5\n polling: {'/index': 'ANY', '/search': 'ALL', '/update': 'ALL',\n '/delete': 'ALL', '/status': 'ALL'}\n```\n\n| Parameter | Description |\n|-------------|---------------------------------------------|\n| `shards` | Number of shardings. |\n| `polling` | Polling strategies for different endpoints. |\n\nThen you can perform exactly the same operations as we do on a single machine.(`/encode`, `/index` and `/search`)\n\n**Why different [polling strategies](https://docs.jina.ai/how-to/scale-out/?highlight=polling#different-polling-strategies) are needed for different endpoints?**\n\nDifferences between `ANY` and `ALL`:\n- `ANY`: requests are sent to one of the executors.\n- `ALL`: requests are sent to all executors.\n\n```{figure} images/polling_stratey.png\n:width: 80%\n\n```\n\nSince one data point only needs to be indexed once, there will only be one indexer executor that will handle this data point. Thus, `ANY` is used for `/index`. On the contrary, we use `ALL` in for `/search` since we don't know which executor stores the perfectly matched result, so the search request should be handled by all indexer executors. (The same reason for using `ALL` in `/update`, `/delete`, `/status`)\n\n```{Warning}\nIncreasing the number of shardings will definitely alleviate the memory issue, but it will increase the latency since there will be more network connections between different shards.\n```\n" }, { "path": "docs/user-guides/server.md", "content": "# Server API\n\nCLIP-as-service is designed in a client-server architecture. A server is a long-running program that receives raw sentences and images from clients, and returns CLIP embeddings to the client. Additionally, `clip_server` is optimized for speed, low memory footprint and scalability.\n- Horizontal scaling: adding more replicas easily with one argument. \n- Vertical scaling: using PyTorch JIT, ONNX or TensorRT runtime to speedup single GPU inference.\n- Supporting gRPC, HTTP, Websocket protocols with their TLS counterparts, w/o compressions.\n\nThis chapter introduces the API of the server. \n\n```{tip}\nYou will need to install server first in Python 3.7+: `pip install clip-server`.\n```\n\n(server-address)=\n## Start server\n\n\n### Start a PyTorch-backed server\n\nUnlike the client, server only has a CLI entrypoint. To start a server, run the following in the terminal:\n\n```bash\npython -m clip_server\n```\n\nNote that it is underscore `_` not the dash `-`.\n\nFirst time running will download the pretrained model (Pytorch `ViT-B/32` by default), load the model, and finally you will get the address information of the server. This information will {ref}`then be used in clients`.\n\n```{figure} images/server-start.gif\n:width: 70%\n\n```\n\n### Start a ONNX-backed server\n\nTo use ONNX runtime for CLIP, you can run:\n\n```bash\npip install \"clip_server[onnx]\"\n\npython -m clip_server onnx-flow.yml\n```\n\n\n### Start a TensorRT-backed server\n\n`nvidia-pyindex` package needs to be installed first. It allows your `pip` to fetch additional Python modules from the NVIDIA NGCโ„ข PyPI repo:\n\n```bash\npip install nvidia-pyindex\npip install \"clip_server[tensorrt]\"\n\npython -m clip_server tensorrt-flow.yml\n```\n\nOne may wonder where is this `onnx-flow.yml` or `tensorrt-flow.yml` come from. Must be a typo? Believe me, just run it. It should just work. I will explain this YAML file in the next section. \n\nThe procedure and UI of ONNX and TensorRT runtime would look the same as Pytorch runtime.\n\n## Model support\n\nThe various `CLIP` models implemented in the [OpenAI](https://github.com/openai/CLIP), [OpenCLIP](https://github.com/mlfoundations/open_clip), and [MultilingualCLIP](https://github.com/FreddeFrallan/Multilingual-CLIP) are supported. \n`ViT-B-32::openai` is used as the default model in all runtimes. \nDue to the limitation of some runtimes, not every runtime supports all models. \nPlease also note that **different models give different sizes of output dimensions**. This will affect your downstream applications. For example, switching the model from one to another make your embedding incomparable, which breaks the downstream applications. Below is a list of supported models of each runtime and its corresponding size.\n\nFor more details about the models and how to select the best model for your application, please refer to the [CLIP benchmark page](benchmark.rst).\n\n| Model | PyTorch | ONNX | TensorRT | Output Dimension |\n| ------------------------------------- | ------- | ---- | -------- | ---------------- |\n| RN50::openai | โœ… | โœ… | โœ… | 1024 |\n| RN50::yfcc15m | โœ… | โœ… | โœ… | 1024 |\n| RN50::cc12m | โœ… | โœ… | โœ… | 1024 |\n| RN101::openai | โœ… | โœ… | โœ… | 512 |\n| RN101::yfcc15m | โœ… | โœ… | โœ… | 512 |\n| RN50x4::openai | โœ… | โœ… | โœ… | 640 |\n| RN50x16::openai | โœ… | โœ… | โŒ | 768 |\n| RN50x64::openai | โœ… | โœ… | โŒ | 1024 |\n| ViT-B-32::openai | โœ… | โœ… | โœ… | 512 |\n| ViT-B-32::laion2b_e16 | โœ… | โœ… | โœ… | 512 |\n| ViT-B-32::laion400m_e31 | โœ… | โœ… | โœ… | 512 |\n| ViT-B-32::laion400m_e32 | โœ… | โœ… | โœ… | 512 |\n| ViT-B-32::laion2b-s34b-b79k | โœ… | โœ… | โŒ | 512 |\n| ViT-B-16::openai | โœ… | โœ… | โœ… | 512 |\n| ViT-B-16::laion400m_e31 | โœ… | โœ… | โœ… | 512 |\n| ViT-B-16::laion400m_e32 | โœ… | โœ… | โœ… | 512 |\n| ViT-B-16-plus-240::laion400m_e31 | โœ… | โœ… | ๐Ÿšง | 640 |\n| ViT-B-16-plus-240::laion400m_e32 | โœ… | โœ… | ๐Ÿšง | 640 |\n| ViT-L-14::openai | โœ… | โœ… | โŒ | 768 |\n| ViT-L-14::laion400m_e31 | โœ… | โœ… | โŒ | 768 |\n| ViT-L-14::laion400m_e32 | โœ… | โœ… | โŒ | 768 |\n| ViT-L-14::laion2b-s32b-b82k | โœ… | โœ… | โŒ | 768 |\n| ViT-L-14-336::openai | โœ… | โœ… | โŒ | 768 |\n| ViT-H-14::laion2b-s32b-b79k | โœ… | โœ… | โŒ | 1024 |\n| ViT-g-14::laion2b-s12b-b42k | โœ… | โœ… | โŒ | 1024 |\n| M-CLIP/LABSE-Vit-L-14 | โœ… | โœ… | โŒ | 768 |\n| M-CLIP/XLM-Roberta-Large-Vit-B-32 | โœ… | โœ… | ๐Ÿšง | 512 |\n| M-CLIP/XLM-Roberta-Large-Vit-B-16Plus | โœ… | โœ… | ๐Ÿšง | 640 |\n| M-CLIP/XLM-Roberta-Large-Vit-L-14 | โœ… | โœ… | โŒ | 768 |\n\nโœ… = Supported โ€” ๐Ÿšง = Working in progress โ€” โŒ = Not supported\n\n### Use custom model for onnx\nYou can also use your own model in ONNX runtime by specifying the model name and the path to ONNX model directory in YAML file.\nThe model directory should have the same structure as below:\n\n```text\n.\nโ””โ”€โ”€ custom-model/\n โ”œโ”€โ”€ textual.onnx\n โ””โ”€โ”€ visual.onnx\n```\n\nOne may wonder how to produce the model as described above. \nFortunately, you can simply use the [Finetuner](https://finetuner.jina.ai) to fine-tune your model based on custom dataset.\n[Finetuner](https://finetuner.jina.ai) is a cloud service that makes fine-tuning simple and fast. \nMoving the process into the cloud, [Finetuner](https://finetuner.jina.ai) handles all related complexity and infrastructure, making models performant and production ready.\n{ref}`Click here for detail instructions`.\n\n## YAML config\n\nYou may notice that there is a YAML file in our last ONNX example. All configurations are stored in this file. In fact, `python -m clip_server` does **not support** any other argument besides a YAML file. So it is the only source of the truth of your configs. \n\nTo load a YAML config from `my.yml`, simply do\n\n```bash\npython -m clip_server my.yml\n```\n\nOr one can also pipe the config via stdin:\n\n```bash\ncat my.yml | python -m clip_server -i\n```\n\nThis can be very useful when using `clip_server` in a Docker container.\n\nAnd to answer your doubt, `clip_server` has three built-in YAML configs as a part of the package resources. When you do `python -m clip_server` it loads the Pytorch config, and when you do `python -m clip_server onnx-flow.yml` it loads the ONNX config.\nIn the same way, when you do `python -m clip_server tensorrt-flow.yml` it loads the TensorRT config.\n\nLet's look at these three built-in YAML configs:\n\n````{tab} torch-flow.yml\n\n```yaml\njtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_t\n uses:\n jtype: CLIPEncoder\n metas:\n py_modules:\n - clip_server.executors.clip_torch\n```\n````\n\n````{tab} onnx-flow.yml\n\n```yaml\njtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_o\n uses:\n jtype: CLIPEncoder\n metas:\n py_modules:\n - clip_server.executors.clip_onnx\n```\n````\n\n\n````{tab} tensorrt-flow.yml\n\n```yaml\njtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_r\n uses:\n jtype: CLIPEncoder\n metas:\n py_modules:\n - clip_server.executors.clip_tensorrt\n```\n````\n\nBasically, each YAML file defines a [Jina Flow](https://docs.jina.ai/fundamentals/flow/). The complete Jina Flow YAML syntax [can be found here](https://docs.jina.ai/fundamentals/flow/yaml-spec/). General parameters of the Flow and Executor can be used here as well. But now we only highlight the most important parameters.\n\nLooking at the YAML file again, we can put it into three subsections as below:\n\n\n````{tab} CLIP model config\n\n```{code-block} yaml\n---\nemphasize-lines: 9\n---\n\njtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_t\n uses:\n jtype: CLIPEncoder\n with:\n metas:\n py_modules:\n - clip_server.executors.clip_torch\n```\n\n````\n\n````{tab} Executor config\n\n```{code-block} yaml\n---\nemphasize-lines: 6\n---\n\njtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_t\n uses:\n jtype: CLIPEncoder\n with: \n metas:\n py_modules:\n - clip_server.executors.clip_torch\n```\n\n````\n\n````{tab} Flow config\n\n```{code-block} yaml\n---\nemphasize-lines: 3,4\n---\n\njtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_t\n uses:\n jtype: CLIPEncoder\n with: \n metas:\n py_modules:\n - clip_server.executors.clip_torch\n```\n\n````\n\n### CLIP model config\n\nFor all backends, you can set the following parameters via `with`:\n\n| Parameter | Description |\n| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------- |\n| `name` | The name of the model to be used. Default 'ViT-B-32::openai'. A list of available models can be found [here](#model-support) |\n| `num_worker_preprocess` | The number of CPU workers to preprocess images and texts. Default is 4. |\n| `minibatch_size` | The size of the minibatch for preprocessing and encoding. Default is 32. Reduce this number if you encounter OOM errors. |\n\nThere are also runtime-specific parameters listed below:\n\n````{tab} PyTorch\n\n| Parameter | Description |\n| --------- | ---------------------------------------------------------------- |\n| `device` | 'cpu' or 'cuda'. Default is None, which auto-detects the device. |\n| `jit` | Whether to use JIT compilation. Default is False. |\n\n````\n\n````{tab} ONNX\n\n| Parameter | Description |\n| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `device` | 'cpu' or 'cuda'. Default is None, which auto-detects the device. |\n| `model_path` | The path to the model to be used. If not specified, the model will be downloaded or loaded from the local cache. See [here](#use-custom-model-for-onnx) to learn how to finetune custom models. |\n\n````\n\nFor example, to turn on JIT and force PyTorch running on CPU, one can do:\n\n```{code-block} yaml\n---\nemphasize-lines: 9-11\n---\n\njtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_t\n uses:\n jtype: CLIPEncoder\n with: \n jit: True\n device: cpu\n metas:\n py_modules:\n - clip_server.executors.clip_torch\n```\n\nTo use custom model in ONNX runtime, one can do:\n\n```{code-block} yaml\n---\nemphasize-lines: 9-11\n---\n\njtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_o\n uses:\n jtype: CLIPEncoder\n with:\n name: ViT-B/32\n model_path: 'custom-model'\n metas:\n py_modules:\n - clip_server.executors.clip_onnx\n```\n\n```{warning}\nThe model name should match the fine-tuned model, or you will get incorrect output.\n```\n\n### Executor config\n\nThe full list of configs for Executor can be found via `jina executor --help`. The most important one is probably `replicas`, which **allows you to run multiple CLIP models in parallel** to achieve horizontal scaling.\n\nTo scale to 4 CLIP replicas, simply adding `replicas: 4` under `uses:`:\n\n```{code-block} yaml\n---\nemphasize-lines: 7\n---\njtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_t\n replicas: 4\n uses:\n jtype: CLIPEncoder\n metas:\n py_modules:\n - clip_server.executors.clip_torch\n```\n\n(flow-config)=\n### Flow config\n\nFlow configs are the ones under top-level `with:`. We can see the `port: 51000` is configured there. Besides `port`, there are some common parameters you might need.\n\n| Parameter | Description |\n| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `protocol` | Communication protocol between server and client. Can be `grpc`, `http`, `websocket`. |\n| `cors` | Only effective when `protocol=http`. If set, a CORS middleware is added to FastAPI frontend to allow cross-origin access. |\n| `prefetch` | Control the maximum streamed request inside the Flow at any given time, default is `None`, means no limit. Setting `prefetch` to a small number helps solving the OOM problem, but may slow down the streaming a bit. |\n\n\nAs an example, to set `protocol` and `prefetch`, one can modify the YAML as follows:\n\n```{code-block} yaml\n---\nemphasize-lines: 5,6\n---\n\njtype: Flow\nversion: '1'\nwith:\n port: 51000\n protocol: websocket\n prefetch: 10\nexecutors:\n - name: clip_t\n replicas: 4\n uses:\n jtype: CLIPEncoder\n metas:\n py_modules:\n - clip_server.executors.clip_torch\n```\n\n## Environment variables\n\n\nTo start a server with more verbose logging,\n\n```bash\nJINA_LOG_LEVEL=DEBUG python -m clip_server\n```\n\n```{figure} images/server-log.gif\n:width: 70%\n\n```\n\nTo run CLIP-server on 3rd GPU,\n\n```bash\nCUDA_VISIBLE_DEVICES=2 python -m clip_server\n```\n\n### Serve on Multiple GPUs\n\nIf you have multiple GPU devices, you can leverage them via `CUDA_VISIBLE_DEVICES=RR`. For example, if you have 3 GPUs and your Flow YAML says `replicas: 5`, then \n\n```bash\nCUDA_VISIBLE_DEVICES=RR python -m clip_server\n```\n\nWill assign GPU devices to the following round-robin fashion:\n\n| GPU device | Replica ID |\n| ---------- | ---------- |\n| 0 | 0 |\n| 1 | 1 |\n| 2 | 2 |\n| 0 | 3 |\n| 1 | 4 |\n\n\nYou can also restrict the visible devices in round-robin assigment by `CUDA_VISIBLE_DEVICES=RR0:2`, where `0:2` has the same meaning as Python slice. This will create the following assigment:\n\n| GPU device | Replica ID |\n| ---------- | ---------- |\n| 0 | 0 |\n| 1 | 1 |\n| 0 | 2 |\n| 1 | 3 |\n| 0 | 4 |\n\n\n```{tip}\nIn pratice, we found it is unnecessary to run `clip_server` on multiple GPUs for two reasons:\n- A single replica even with largest `ViT-L/14-336px` takes only 3.5GB VRAM.\n- Real network traffic never utilizes GPU in 100%.\n\nBased on these two points, it makes more sense to have multiple replicas on a single GPU comparing to have multiple replicas on different GPU, which is kind of waste of resources. `clip_server` scales pretty well by interleaving the GPU time with mulitple replicas.\n```\n\n## Monitor with Prometheus and Grafana\n\nTo monitor the performance of the service, you can enable the Prometheus metrics in the Flow YAML:\n\n```{code-block} yaml\n---\nemphasize-lines: 5,6,14,15\n---\n\njtype: Flow\nversion: '1'\nwith:\n port: 51000\n monitoring: True\n port_monitoring: 9090\nexecutors:\n - name: clip_t\n uses:\n jtype: CLIPEncoder\n metas:\n py_modules:\n - clip_server.executors.clip_torch\n monitoring: true\n port_monitoring: 9091\n```\n\nThis enables Prometheus metrics on both Gateway and the CLIP Executor.\n\nRunning it gives you:\n\n```{figure} images/server-start-monitoring.gif\n:width: 80%\n\n```\n\nwhich exposes two additional endpoints:\n- `http://localhost:9090` for the Gateway\n- `http://localhost:9091` for the CLIP Executor\n\n\nTo visualize the metrics in Grafana, you can import this [JSON file of an example dashboard](https://clip-as-service.jina.ai/_static/cas-grafana.json). You will get something as follows:\n\n```{figure} images/grafana-dashboard.png\n:width: 80%\n\n```\n\n\nFor more information on monitoring a Flow, [please read here](https://docs.jina.ai/fundamentals/flow/monitoring-flow/). \n\n## Serve with TLS\n\nYou can turn on TLS for HTTP and gRPC protocols. Your Flow YAML should be changed to the following:\n\n```{code-block} yaml\n---\nemphasize-lines: 4,5,7-10\n---\njtype: Flow\nversion: '1'\nwith:\n port: 8443\n protocol: http\n cors: true\n uvicorn_kwargs:\n ssl_keyfile_password: blahblah\n ssl_certfile: cert.pem\n ssl_keyfile: key.pem\n```\n\nHere, `protocol` can be either `http` or `grpc`; `cert.pem` or `key.pem` represent both parts of a certificate, key being the private key to the certificate and crt being the signed certificate. You can run the following command in terminal:\n\n```bash\nopenssl req -newkey rsa:4096 -nodes -sha512 -x509 -days 3650 -nodes -out cert.pem -keyout key.pem -subj \"/CN=\"\n```\n\nNote that if you are using `protocol: grpc` then `/CN=` must strictly follow the IP address or the domain name of your server. Mismatch IP or domain name would throw an exception.\n\nCertificate and keys can be also generated via [letsencrypt.org](https://letsencrypt.org/), which is a free SSL provider.\n\n```{warning}\nNote that note every port support HTTPS. Commonly support ports are: `443`, `2053`, `2083`, `2087`, `2096`, `8443`.\n```\n\n```{warning}\nIf you are using Cloudflare proxied DNS, please be aware:\n- you need to turn on gRPC support manually, [please follow the guide here](https://support.cloudflare.com/hc/en-us/articles/360050483011-Understanding-Cloudflare-gRPC-support);\n- the free tier of Cloudflare has 100s hard limit on the timeout, meaning sending big batch to a CPU server may throw 524 to the client-side.\n```\n\nWhen the server is successfully running, you can connect to it via client by setting `server` to `https://` or `grpcs://` as follows:\n\n```python\nfrom clip_client import Client\n\nc = Client('grpcs://:2096')\n\nr = c.encode(\n [\n 'First do it',\n 'then do it right',\n 'then do it better',\n 'https://picsum.photos/200',\n ]\n)\n```\n\n## Serve in Docker Container\n\nYou can run the server inside a Docker container. We provide a Dockerfile in the repository, which is CUDA-enabled with optimized package installation. \n\n### Build\n\nWe have a list of {ref}`pre-built images available on Docker Hub`. If they are too big for you to download, you may consider built it yourself as follows:\n\n```bash\ngit clone https://github.com/jina-ai/clip-as-service.git\ndocker build . -f Dockerfiles/server.Dockerfile --build-arg GROUP_ID=$(id -g ${USER}) --build-arg USER_ID=$(id -u ${USER}) -t jinaai/clip-server\n```\n\n```{tip}\nThe build argument `--build-arg GROUP_ID=$(id -g ${USER}) --build-arg USER_ID=$(id -u ${USER})` is optional, but having them is highly recommended as it allows you to reuse host's cache with the correct access.\n```\n\n\n### Run\n\n````{tab} PyTorch\n```bash\ndocker run -p 51009:51000 -v $HOME/.cache:/home/cas/.cache --gpus all jinaai/clip-server\n```\n````\n````{tab} ONNX\n```bash\ndocker run -p 51009:51000 -v $HOME/.cache:/home/cas/.cache --gpus all jinaai/clip-server:master-onnx onnx-flow.yml\n```\n````\n````{tab} TensorRT\n```bash\ndocker run -p 51009:51000 -v $HOME/.cache:/home/cas/.cache --gpus all jinaai/clip-server:master-tensorrt tensorrt-flow.yml\n```\n````\n\nHere, `51009` is the public port on the host and `51000` is the {ref}`in-container port defined inside YAML`. The argument `-v $HOME/.cache:/home/cas/.cache` leverages host's cache and prevents you to download the same model next time on start. \n\nDue to the limitation of the terminal inside Docker container, you will **not** see the classic Jina progress bar on start. Instead, you will face a few minutes awkward silent while model downloading and then see \"Flow is ready to serve\" dialog.\n\nTo pass a YAML config from the host, one can do:\n\n````{tab} PyTorch\n```bash\ncat my.yml | docker run -i -p 51009:51000 -v $HOME/.cache:/home/cas/.cache --gpus all jinaai/clip-server -i\n```\n````\n````{tab} ONNX\n```bash\ncat my.yml | docker run -i -p 51009:51000 -v $HOME/.cache:/home/cas/.cache --gpus all jinaai/clip-server:master-onnx -i\n```\n````\n````{tab} TensorRT\n```bash\ncat my.yml | docker run -i -p 51009:51000 -v $HOME/.cache:/home/cas/.cache --gpus all jinaai/clip-server:master-tensorrt -i\n```\n````\n\nThe CLI usage is the same {ref}`as described here `.\n\n```{tip}\nYou can enable debug logging via: `docker run --env JINA_LOG_LEVEL=debug ...`\n```\n\n(prebuild-images)=\n### Pre-built images\n\nWe have prebuilt images with CUDA support.\n\nThe Docker image name always starts with `jinaai/clip-server` followed by a tag composed of three parts:\n\n```text\njinaai/clip-server:{version}{extra}\n```\n\n- `{version}`: The version of Jina. Possible values:\n - `latest`: the last release;\n - `master`: the master branch of `jina-ai/jina` repository;\n - `x.y.z`: the release of a particular version;\n - `x.y` and `x`: the alias to the last `x.y.z` patch release, i.e. `x.y` = `x.y.max(z)`;\n- `{extra}`: the extra dependency installed along with `clip_server`. Possible values:\n - ` `: Pytorch backend;\n - `-onnx`: ONNX backend; \n - `-tensorrt`: TensorRT backend;\n\n#### Image alias and updates\n\n| Event | Updated images | Aliases |\n| -------------------- | ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |\n| On merge into `main` | `jinaai/clip-server:master{extra}` | |\n| On `x.y.z` release | `jinaai/clip-server:x.y.z{extra}` | `jinaai/clip-server:latest{python_version}{extra}`, `jinaai/clip-server:x.y{python_version}{extra}`, `jinaai/clip-server:x{python_version}{extra}` |\n\n3 images are built on the event listed above, i.e. taking the combination of:\n - `{extra} = [\"\", \"-onnx\", \"-tensorrt\"]`\n\n#### Image size on different tags\n\n```{warning}\n[Due to a known bug in shields.io/Docker Hub API](https://github.com/badges/shields/issues/7583), the following badge may show \"invalid\" status randomly.\n```\n\n| Image Size |\n| ----------------------------------------------------------------------------------------------------------------------------------------- |\n| ![](https://img.shields.io/docker/image-size/jinaai/clip-server/latest?label=jinaai%2Fclip-server%3Alatest&logo=docker) |\n| ![](https://img.shields.io/docker/image-size/jinaai/clip-server/latest-onnx?label=jinaai%2Fclip-server%3Alatest-onnx&logo=docker) |\n| ![](https://img.shields.io/docker/image-size/jinaai/clip-server/latest-tensorrt?label=jinaai%2Fclip-server%3Alatest-tensorrt&logo=docker) |\n| ![](https://img.shields.io/docker/image-size/jinaai/clip-server/master?label=jinaai%2Fclip-server%3Amaster&logo=docker) |\n| ![](https://img.shields.io/docker/image-size/jinaai/clip-server/master-onnx?label=jinaai%2Fclip-server%3Amaster-onnx&logo=docker) |\n| ![](https://img.shields.io/docker/image-size/jinaai/clip-server/master-tensorrt?label=jinaai%2Fclip-server%3Amaster-tensorrt&logo=docker) |\n\n" }, { "path": "scripts/MANIFEST.in", "content": "include LICENSE\nprune tests/\nprune **/tests/\n" }, { "path": "scripts/benchmark.py", "content": "import random\nimport time\nfrom typing import Optional\nimport threading\nimport click\nimport numpy as np\nfrom docarray import Document, DocumentArray\n\n\ndef warn(*args, **kwargs):\n pass\n\n\nimport warnings\n\nwarnings.warn = warn\n\n\nnp.random.seed(123)\n\n\nclass BenchmarkClient(threading.Thread):\n def __init__(\n self,\n server: str,\n batch_size: int = 1,\n modality: str = 'text',\n num_iter: Optional[int] = 100,\n image_sample: str = None,\n **kwargs,\n ):\n \"\"\"\n @param server: the CLIP-as-service server URI\n @param batch_size: number of batch sample\n @param num_iter: number of repeat run per experiment\n @param image_sample: uri of the test image\n \"\"\"\n assert num_iter > 2, 'num_iter must be greater than 2'\n super().__init__()\n self.server = server\n self.batch_size = batch_size\n self.modality = modality\n self.image_sample = image_sample\n self.num_iter = num_iter\n self.avg_time = 0\n\n def run(self):\n try:\n from clip_client import Client\n except ImportError:\n raise ImportError(\n 'clip_client module is not available. it is required for benchmarking.'\n 'Please use \"\"pip install clip-client\" to install it.'\n )\n\n if self.modality == 'text':\n from clip_server.model.simple_tokenizer import SimpleTokenizer\n\n tokenizer = SimpleTokenizer()\n vocab = list(tokenizer.encoder.keys())\n batch = DocumentArray(\n [\n Document(text=' '.join(random.choices(vocab, k=78)))\n for _ in range(self.batch_size)\n ]\n )\n elif self.modality == 'image':\n batch = DocumentArray(\n [\n Document(blob=open(self.image_sample, 'rb').read())\n for _ in range(self.batch_size)\n ]\n )\n else:\n raise ValueError(f'The modality \"{self.modality}\" is unsupported')\n\n client = Client(self.server)\n\n time_costs = []\n for _ in range(self.num_iter):\n start = time.perf_counter()\n r = client.encode(batch, batch_size=self.batch_size)\n time_costs.append(time.perf_counter() - start)\n self.avg_time = np.mean(time_costs[2:])\n\n\n@click.command(name='clip-as-service benchmark')\n@click.argument('server')\n@click.option(\n '--batch_sizes',\n multiple=True,\n type=int,\n default=[1, 8, 16, 32, 64],\n help='number of batch',\n)\n@click.option(\n '--num_iter', default=10, help='number of repeat run per experiment (must > 2)'\n)\n@click.option(\n \"--concurrent_clients\",\n multiple=True,\n type=int,\n default=[1, 4, 16, 32, 64],\n help='number of concurrent clients per experiment',\n)\n@click.option(\"--image_sample\", help='path to the image sample file')\ndef main(server, batch_sizes, num_iter, concurrent_clients, image_sample):\n # wait until the server is ready\n for batch_size in batch_sizes:\n for num_client in concurrent_clients:\n all_clients = [\n BenchmarkClient(\n server,\n batch_size=batch_size,\n num_iter=num_iter,\n modality='image' if (image_sample is not None) else 'text',\n image_sample=image_sample,\n )\n for _ in range(num_client)\n ]\n\n for bc in all_clients:\n bc.start()\n\n clients_speed = []\n for bc in all_clients:\n bc.join()\n clients_speed.append(batch_size / bc.avg_time)\n\n max_speed, min_speed, avg_speed = (\n max(clients_speed),\n min(clients_speed),\n np.mean(clients_speed),\n )\n\n print(\n '(concurrent client=%d, batch_size=%d) avg speed: %.3f\\tmax speed: %.3f\\tmin speed: %.3f'\n % (num_client, batch_size, avg_speed, max_speed, min_speed),\n flush=True,\n )\n\n\nif __name__ == '__main__':\n main()\n" }, { "path": "scripts/black.sh", "content": "#!/bin/bash\npip install black==22.3.0\narrVar=()\necho we ignore non-*.py files and files generated from protobuf\nexcluded_files=(\n docarray/proto/docarray_pb2.py\n docs/conf.py\n)\nfor changed_file in $CHANGED_FILES; do\n if [[ ${changed_file} == *.py ]] && ! [[ \" ${excluded_files[@]} \" =~ \" ${changed_file} \" ]]; then\n echo checking ${changed_file}\n arrVar+=(${changed_file})\n fi\ndone\nif [ ${#arrVar[@]} -ne 0 ]; then\n black -S --check \"${arrVar[@]}\"\nfi\n" }, { "path": "scripts/docstrings_lint.sh", "content": "#!/bin/bash\n# required in order to get the status of all the files at once\npip install darglint==1.6.0\npip install pydocstyle==5.1.1\necho ====================================================================================\necho DOCSTRINGS LINT: checking $CHANGED_FILES\necho ------------------------------------------------------------------------------------\necho 'removing files under /tests...'\narrVar=()\n# we ignore tests files\nfor changed_file in $CHANGED_FILES; do\n case ${changed_file} in\n tests/* | \\\n .github/* | \\\n scripts/* | \\\n docarray/resources/* | \\\n docs/* | \\\n setup.py | \\\n fastentrypoints.py)\n ;;*)\n echo keeping ${changed_file}\n arrVar+=(${changed_file})\n ;;\n esac\ndone\n\n# if array is empty\nif [ ${#arrVar[@]} -eq 0 ]; then\n echo 'nothing to check'\n exit 0\nfi\n\nDARGLINT_OUTPUT=$(darglint -v 2 -s sphinx \"${arrVar[@]}\"); PYDOCSTYLE_OUTPUT=$(pydocstyle --select=D101,D102,D103 \"${arrVar[@]}\")\n# status captured here\nif [[ -z \"$PYDOCSTYLE_OUTPUT\" ]] && [[ -z \"$DARGLINT_OUTPUT\" ]]; then\n echo 'OK'\n exit 0\nelse\n echo 'failure. make sure to check the guide for docstrings: https://docarray.jina.ai/chapters/docstring.html'\n echo $DARGLINT_OUTPUT\n echo $PYDOCSTYLE_OUTPUT\n exit 1\nfi\necho ====================================================================================\n" }, { "path": "scripts/get-all-test-paths.sh", "content": "#!/usr/bin/env bash\n\nset -ex\n\nBATCH_SIZE=3\n#declare -a array1=( \"tests/unit/test_*.py\" )\n#declare -a array2=( $(ls -d tests/unit/*/ | grep -v '__pycache__' | grep -v 'array') )\n#declare -a array3=( \"tests/unit/array/*.py\" )\ndeclare -a mixins=( $(find tests -name \"test_*.py\" | grep -v 'test_tensorrt.py') )\ndeclare -a array4=( \"$(echo \"${mixins[@]}\" | xargs -n$BATCH_SIZE)\" )\n# array5 is currently empty because in the array/ directory, mixins is the only directory\n# but add the following in case new directories are created in array/\ndeclare -a array5=( $(ls -d tests/unit/array/*/ | grep -v '__pycache__' | grep -v 'mixins') )\ndest=( \"${array1[@]}\" \"${array2[@]}\" \"${array3[@]}\" \"${array4[@]}\" \"${array5[@]}\" )\n\nprintf '%s\\n' \"${dest[@]}\" | jq -R . | jq -cs .\n" }, { "path": "scripts/get-last-release-note.py", "content": "## under jina root dir\n# python scripts/get-last-release-note.py\n## result in root/tmp.md\n\nwith open('CHANGELOG.md') as fp:\n n = []\n for v in fp:\n if v.startswith('## Release Note'):\n n.clear()\n n.append(v)\n\nwith open('tmp.md', 'w') as fp:\n fp.writelines(n)\n" }, { "path": "scripts/get-requirements.py", "content": "## under clip-as-service root dir\n# python scripts/get-requirments.py $PIP_TAG /path/to/requirements.txt\n\nimport sys\nfrom distutils.core import run_setup\n\nresult = run_setup(\"./server/setup.py\", stop_after=\"init\")\n\nwith open(sys.argv[2], 'w') as fp:\n fp.write('\\n'.join(result.install_requires) + '\\n')\n if sys.argv[1]:\n fp.write('\\n'.join(result.extras_require[sys.argv[1]]) + '\\n')\n" }, { "path": "scripts/onnx_helper.py", "content": "def convert_float_to_float16(model_path: str, output_model_path: str):\n import onnx\n from onnxmltools.utils.float16_converter import (\n convert_float_to_float16_model_path,\n )\n\n new_onnx_model = convert_float_to_float16_model_path(model_path)\n\n onnx.save(new_onnx_model, output_model_path)\n\n # Alternate approach\n # from onnx import load_model\n # from onnxruntime.transformers import optimizer, onnx_model\n #\n # # optimized_model = optimizer.optimize_model(model_path, model_type='bert')\n #\n # model = load_model(model_path)\n # optimized_model = onnx_model.OnnxModel(model)\n #\n # if hasattr(optimized_model, 'convert_float32_to_float16'):\n # optimized_model.convert_float_to_float16()\n # else:\n # optimized_model.convert_model_float32_to_float16()\n #\n # self._textual_path = f'{self._textual_path[:-5]}_optimized.onnx'\n # optimized_model.save_model_to_file(output_model_path)\n\n\ndef quantize(model_path: str, output_model_path: str):\n \"\"\"\n Quantize the weights of the model from float32 to in8 to allow very efficient inference on modern CPU\n Uses unsigned ints for activation values, signed ints for weights, per\n https://onnxruntime.ai/docs/performance/quantization.html#data-type-selection\n it is faster on most CPU architectures\n Args:\n onnx_model_path: Path to location the exported ONNX model is stored\n Returns: The Path generated for the quantized\n \"\"\"\n from onnxruntime.quantization import quantize_dynamic, QuantType\n\n quantize_dynamic(\n model_input=model_path,\n model_output=output_model_path,\n per_channel=True,\n reduce_range=True, # should be the same as per_channel\n activation_type=QuantType.QUInt8,\n weight_type=QuantType.QInt8, # per docs, signed is faster on most CPUs\n optimize_model=True,\n op_types_to_quantize=[\"MatMul\", \"Attention\", \"Mul\", \"Add\"],\n extra_options={\"WeightSymmetric\": False, \"MatMulConstBOnly\": True},\n ) # op_types_to_quantize=['MatMul', 'Relu', 'Add', 'Mul' ],\n" }, { "path": "scripts/release.sh", "content": "#!/usr/bin/env bash\n\n# Requirements\n# brew install hub\n# npm install -g git-release-notes\n# pip install twine wheel\n\nset -ex\n\nINIT_FILE='client/clip_client/__init__.py'\nVER_TAG='__version__ = '\nRELEASENOTE='./node_modules/.bin/git-release-notes'\n\nfunction escape_slashes {\n sed 's/\\//\\\\\\//g'\n}\n\nfunction update_ver_line {\n local OLD_LINE_PATTERN=$1\n local NEW_LINE=$2\n local FILE=$3\n\n local NEW=$(echo \"${NEW_LINE}\" | escape_slashes)\n sed -i '/'\"${OLD_LINE_PATTERN}\"'/s/.*/'\"${NEW}\"'/' \"${FILE}\"\n head -n10 ${FILE}\n}\n\n\nfunction clean_build {\n rm -rf dist\n rm -rf *.egg-info\n rm -rf build\n}\n\nfunction pub_pypi {\n # publish to pypi\n cd $1\n clean_build\n python setup.py sdist\n twine upload dist/*\n clean_build\n cd -\n}\n\nfunction git_commit {\n git config --local user.email \"dev-bot@jina.ai\"\n git config --local user.name \"Jina Dev Bot\"\n git tag \"v$RELEASE_VER\" -m \"$(cat ./CHANGELOG.tmp)\"\n git add client/clip_client/__init__.py server/clip_server/__init__.py ./CHANGELOG.md\n git commit -m \"chore(version): the next version will be $NEXT_VER\" -m \"build($RELEASE_ACTOR): $RELEASE_REASON\"\n}\n\n\n\nfunction make_release_note {\n ${RELEASENOTE} ${LAST_VER}..HEAD .github/release-template.ejs > ./CHANGELOG.tmp\n head -n10 ./CHANGELOG.tmp\n printf '\\n%s\\n\\n%s\\n%s\\n\\n%s\\n\\n%s\\n\\n' \"$(cat ./CHANGELOG.md)\" \"\" \"## Release Note (\\`${RELEASE_VER}\\`)\" \"> Release time: $(date +'%Y-%m-%d %H:%M:%S')\" \"$(cat ./CHANGELOG.tmp)\" > ./CHANGELOG.md\n}\n\nBRANCH=$(git rev-parse --abbrev-ref HEAD)\n\nif [[ \"$BRANCH\" != \"main\" ]]; then\n printf \"You are not at main branch, exit\\n\";\n exit 1;\nfi\n\nLAST_UPDATE=`git show --no-notes --format=format:\"%H\" $BRANCH | head -n 1`\nLAST_COMMIT=`git show --no-notes --format=format:\"%H\" origin/$BRANCH | head -n 1`\n\nif [ $LAST_COMMIT != $LAST_UPDATE ]; then\n printf \"Your local $BRANCH is behind the remote master, exit\\n\"\n exit 1;\nfi\n\n# release the current version\nexport RELEASE_VER=$(sed -n '/^__version__/p' $INIT_FILE | cut -d \\' -f2)\nLAST_VER=$(git tag -l | sort -V | tail -n1)\nprintf \"last version: \\e[1;32m$LAST_VER\\e[0m\\n\"\n\nif [[ $1 == \"final\" ]]; then\n printf \"this will be a final release: \\e[1;33m$RELEASE_VER\\e[0m\\n\"\n\n NEXT_VER=$(echo $RELEASE_VER | awk -F. -v OFS=. 'NF==1{print ++$NF}; NF>1{$NF=sprintf(\"%0*d\", length($NF), ($NF+1)); print}')\n printf \"bump master version to: \\e[1;32m$NEXT_VER\\e[0m\\n\"\n\n make_release_note\n\n pub_pypi client\n pub_pypi server\n cp scripts/MANIFEST.in ./\n cp scripts/setup.py ./\n pub_pypi \".\"\n\n VER_TAG_NEXT=$VER_TAG\\'${NEXT_VER}\\'\n update_ver_line \"$VER_TAG\" \"$VER_TAG_NEXT\" 'client/clip_client/__init__.py'\n update_ver_line \"$VER_TAG\" \"$VER_TAG_NEXT\" 'server/clip_server/__init__.py'\n RELEASE_REASON=\"$2\"\n RELEASE_ACTOR=\"$3\"\n git_commit\nelif [[ $1 == 'rc' ]]; then\n printf \"this will be a release candidate: \\e[1;33m$RELEASE_VER\\e[0m\\n\"\n DOT_RELEASE_VER=$(echo $RELEASE_VER | sed \"s/rc/\\./\")\n NEXT_VER=$(echo $DOT_RELEASE_VER | awk -F. -v OFS=. 'NF==1{print ++$NF}; NF>1{$NF=sprintf(\"%0*d\", length($NF), ($NF+1)); print}')\n NEXT_VER=$(echo $NEXT_VER | sed \"s/\\.\\([^.]*\\)$/rc\\1/\")\n printf \"bump master version to: \\e[1;32m$NEXT_VER\\e[0m, this will be the next version\\n\"\n\n make_release_note\n\n pub_pypi client\n pub_pypi server\n cp scripts/MANIFEST.in ./\n cp scripts/setup.py ./\n pub_pypi \".\"\n\n VER_TAG_NEXT=$VER_TAG\\'${NEXT_VER}\\'\n update_ver_line \"$VER_TAG\" \"$VER_TAG_NEXT\" 'client/clip_client/__init__.py'\n update_ver_line \"$VER_TAG\" \"$VER_TAG_NEXT\" 'server/clip_server/__init__.py'\n RELEASE_REASON=\"$2\"\n RELEASE_ACTOR=\"$3\"\n git_commit\nelse\n # as a prerelease, pypi update only, no back commit etc.\n COMMITS_SINCE_LAST_VER=$(git rev-list $LAST_VER..HEAD --count)\n NEXT_VER=$RELEASE_VER\".dev\"$COMMITS_SINCE_LAST_VER\n printf \"this will be a developmental release: \\e[1;33m$NEXT_VER\\e[0m\\n\"\n\n VER_TAG_NEXT=$VER_TAG\\'${NEXT_VER}\\'\n update_ver_line \"$VER_TAG\" \"$VER_TAG_NEXT\" 'client/clip_client/__init__.py'\n update_ver_line \"$VER_TAG\" \"$VER_TAG_NEXT\" 'server/clip_server/__init__.py'\n\n pub_pypi client\n pub_pypi server\n cp scripts/MANIFEST.in ./\n cp scripts/setup.py ./\n pub_pypi \".\"\nfi\n" }, { "path": "scripts/setup.py", "content": "import sys\nfrom os import path\n\nfrom setuptools import find_packages\nfrom setuptools import setup\n\nif sys.version_info < (3, 7, 0):\n raise OSError(f'Clip-as-service requires Python >=3.7, but yours is {sys.version}')\n\ntry:\n pkg_name = 'clip-as-service'\n libinfo_py = path.join('server/clip_server/__init__.py')\n libinfo_content = open(libinfo_py, 'r', encoding='utf8').readlines()\n version_line = [l.strip() for l in libinfo_content if l.startswith('__version__')][\n 0\n ]\n exec(version_line) # gives __version__\nexcept FileNotFoundError:\n __version__ = '0.0.0'\n\ntry:\n with open('README.md', encoding='utf8') as fp:\n _long_description = fp.read()\nexcept FileNotFoundError:\n _long_description = ''\n\nsetup(\n name=pkg_name,\n packages=find_packages(),\n version=__version__,\n include_package_data=True,\n description='Embed images and sentences into fixed-length vectors via CLIP',\n author='Jina AI',\n author_email='hello@jina.ai',\n license='Apache 2.0',\n url='https://github.com/jina-ai/clip-as-service',\n download_url='https://github.com/jina-ai/clip-as-service/tags',\n long_description=_long_description,\n long_description_content_type='text/markdown',\n zip_safe=False,\n setup_requires=['setuptools>=18.0', 'wheel'],\n install_requires=['clip-server', 'clip-client'],\n classifiers=[\n 'Development Status :: 5 - Production/Stable',\n 'Intended Audience :: Developers',\n 'Intended Audience :: Education',\n 'Intended Audience :: Science/Research',\n 'Programming Language :: Python :: 3.7',\n 'Programming Language :: Python :: 3.8',\n 'Programming Language :: Python :: 3.9',\n 'Programming Language :: Unix Shell',\n 'Environment :: Console',\n 'License :: OSI Approved :: Apache Software License',\n 'Operating System :: OS Independent',\n 'Topic :: Database :: Database Engines/Servers',\n 'Topic :: Scientific/Engineering :: Artificial Intelligence',\n 'Topic :: Internet :: WWW/HTTP :: Indexing/Search',\n 'Topic :: Scientific/Engineering :: Image Recognition',\n 'Topic :: Multimedia :: Video',\n 'Topic :: Scientific/Engineering',\n 'Topic :: Scientific/Engineering :: Mathematics',\n 'Topic :: Software Development',\n 'Topic :: Software Development :: Libraries',\n 'Topic :: Software Development :: Libraries :: Python Modules',\n ],\n project_urls={\n 'Documentation': 'https://clip-as-service.jina.ai/',\n 'Source': 'https://github.com/jina-ai/clip-as-service',\n 'Tracker': 'https://github.com/jina-ai/clip-as-service/issues',\n },\n keywords='jina openai clip deep-learning cross-modal multi-modal neural-search',\n)\n" }, { "path": "server/MANIFEST.in", "content": "recursive-include clip_server/resources *\ninclude clip_server/*.yml\n" }, { "path": "server/clip_server/__init__.py", "content": "__version__ = '0.8.4'\n" }, { "path": "server/clip_server/__main__.py", "content": "import inspect\nimport os\nimport sys\n\nif __name__ == '__main__':\n if 'NO_VERSION_CHECK' not in os.environ:\n from clip_server.helper import is_latest_version\n\n is_latest_version(github_repo='clip-as-service')\n\n from jina import Flow\n\n if len(sys.argv) > 1:\n if sys.argv[1] == '-i':\n _input = sys.stdin.read()\n else:\n _input = sys.argv[1]\n else:\n _input = 'torch-flow.yml'\n\n f = Flow.load_config(\n _input,\n extra_search_paths=[os.path.dirname(inspect.getfile(inspect.currentframe()))],\n )\n with f:\n f.block()\n" }, { "path": "server/clip_server/executors/__init__.py", "content": "" }, { "path": "server/clip_server/executors/clip_onnx.py", "content": "import os\nimport warnings\nfrom functools import partial\nfrom multiprocessing.pool import ThreadPool\nfrom typing import Dict, Optional\n\nimport onnxruntime as ort\nfrom clip_server.executors.helper import (\n preproc_image,\n preproc_text,\n set_rank,\n split_img_txt_da,\n)\nfrom clip_server.model import clip\nfrom clip_server.model.clip_onnx import CLIPOnnxModel\nfrom clip_server.model.tokenization import Tokenizer\nfrom jina import DocumentArray, Executor, requests\nfrom opentelemetry.trace import NoOpTracer, Span\n\n\nclass CLIPEncoder(Executor):\n def __init__(\n self,\n name: str = 'ViT-B-32::openai',\n device: Optional[str] = None,\n num_worker_preprocess: int = 4,\n minibatch_size: int = 32,\n access_paths: str = '@r',\n model_path: Optional[str] = None,\n dtype: Optional[str] = None,\n **kwargs,\n ):\n \"\"\"\n :param name: The name of the model to be used. Default 'ViT-B-32::openai'. A list of available models can be\n found at https://clip-as-service.jina.ai/user-guides/server/#model-support\n :param device: 'cpu' or 'cuda'. Default is None, which auto-detects the device.\n :param num_worker_preprocess: The number of CPU workers to preprocess images and texts. Default is 4.\n :param minibatch_size: The size of the minibatch for preprocessing and encoding. Default is 32. Reduce this\n number if you encounter OOM errors.\n :param access_paths: The access paths to traverse on the input documents to get the images and texts to be\n processed. Visit https://docarray.jina.ai/fundamentals/documentarray/access-elements for more details.\n :param model_path: The path to the model to be used. If not specified, the model will be downloaded or loaded\n from the local cache. Visit https://clip-as-service.jina.ai/user-guides/server/#use-custom-model-for-onnx\n to learn how to finetune custom models.\n :param dtype: inference data type, if None defaults to 'fp32' if device == 'cpu' else 'fp16'.\n \"\"\"\n super().__init__(**kwargs)\n import torch\n\n if not device:\n device = 'cuda' if torch.cuda.is_available() else 'cpu'\n self._device = device\n if not dtype:\n dtype = 'fp32' if self._device in ('cpu', torch.device('cpu')) else 'fp16'\n self._dtype = dtype\n\n self._minibatch_size = minibatch_size\n self._access_paths = access_paths\n if 'traversal_paths' in kwargs:\n warnings.warn(\n f'`traversal_paths` is deprecated. Use `access_paths` instead.'\n )\n self._access_paths = kwargs['traversal_paths']\n\n self._num_worker_preprocess = num_worker_preprocess\n self._pool = ThreadPool(processes=num_worker_preprocess)\n\n self._model = CLIPOnnxModel(name, model_path, dtype)\n self._tokenizer = Tokenizer(name)\n\n self._image_transform = clip._transform_blob(self._model.image_size)\n\n # define the priority order for the execution providers\n providers = ['CPUExecutionProvider']\n\n # prefer CUDA Execution Provider over CPU Execution Provider\n if self._device.startswith('cuda'):\n providers.insert(0, 'CUDAExecutionProvider')\n\n sess_options = ort.SessionOptions()\n\n # Enables all available optimizations including layout optimizations\n sess_options.graph_optimization_level = (\n ort.GraphOptimizationLevel.ORT_ENABLE_ALL\n )\n\n if not self._device.startswith('cuda') and (\n 'OMP_NUM_THREADS' not in os.environ\n and hasattr(self.runtime_args, 'replicas')\n ):\n replicas = getattr(self.runtime_args, 'replicas', 1)\n num_threads = max(1, torch.get_num_threads() * 2 // replicas)\n if num_threads < 2:\n warnings.warn(\n f'Too many replicas ({replicas}) vs too few threads {num_threads} may result in '\n f'sub-optimal performance.'\n )\n\n # Run the operators in the graph in parallel (not support the CUDA Execution Provider)\n sess_options.execution_mode = ort.ExecutionMode.ORT_PARALLEL\n\n # The number of threads used to parallelize the execution of the graph (across nodes)\n sess_options.inter_op_num_threads = 1\n sess_options.intra_op_num_threads = max(num_threads, 1)\n\n self._model.start_sessions(\n sess_options=sess_options, providers=providers, dtype=dtype\n )\n\n if not self.tracer:\n self.tracer = NoOpTracer()\n\n def _preproc_images(self, docs: 'DocumentArray', drop_image_content: bool):\n with self.monitor(\n name='preprocess_images_seconds',\n documentation='images preprocess time in seconds',\n ):\n with self.tracer.start_as_current_span('preprocess_images'):\n return preproc_image(\n docs,\n preprocess_fn=self._image_transform,\n return_np=True,\n drop_image_content=drop_image_content,\n dtype=self._dtype,\n )\n\n def _preproc_texts(self, docs: 'DocumentArray'):\n with self.monitor(\n name='preprocess_texts_seconds',\n documentation='texts preprocess time in seconds',\n ):\n with self.tracer.start_as_current_span('preprocess_images'):\n return preproc_text(docs, tokenizer=self._tokenizer, return_np=True)\n\n @requests(on='/rank')\n async def rank(self, docs: 'DocumentArray', parameters: Dict, **kwargs):\n _drop_image_content = parameters.get('drop_image_content', False)\n await self.encode(docs['@r,m'], drop_image_content=_drop_image_content)\n\n set_rank(docs)\n\n @requests\n async def encode(\n self,\n docs: 'DocumentArray',\n tracing_context=None,\n parameters: Dict = {},\n **kwargs,\n ):\n with self.tracer.start_as_current_span(\n 'encode', context=tracing_context\n ) as span:\n span.set_attribute('device', self._device)\n span.set_attribute('runtime', 'onnx')\n access_paths = parameters.get('access_paths', self._access_paths)\n if 'traversal_paths' in parameters:\n warnings.warn(\n f'`traversal_paths` is deprecated. Use `access_paths` instead.'\n )\n access_paths = parameters['traversal_paths']\n _drop_image_content = parameters.get('drop_image_content', False)\n\n _img_da = DocumentArray()\n _txt_da = DocumentArray()\n for d in docs[access_paths]:\n split_img_txt_da(d, _img_da, _txt_da)\n\n with self.tracer.start_as_current_span('inference') as inference_span:\n inference_span.set_attribute('drop_image_content', _drop_image_content)\n inference_span.set_attribute('minibatch_size', self._minibatch_size)\n inference_span.set_attribute('has_img_da', True if _img_da else False)\n inference_span.set_attribute('has_txt_da', True if _txt_da else False)\n # for image\n if _img_da:\n with self.tracer.start_as_current_span(\n 'img_minibatch_encoding'\n ) as img_encode_span:\n for minibatch, batch_data in _img_da.map_batch(\n partial(\n self._preproc_images,\n drop_image_content=_drop_image_content,\n ),\n batch_size=self._minibatch_size,\n pool=self._pool,\n ):\n with self.monitor(\n name='encode_images_seconds',\n documentation='images encode time in seconds',\n ):\n minibatch.embeddings = self._model.encode_image(\n batch_data\n )\n\n # for text\n if _txt_da:\n with self.tracer.start_as_current_span(\n 'txt_minibatch_encoding'\n ) as txt_encode_span:\n for minibatch, batch_data in _txt_da.map_batch(\n self._preproc_texts,\n batch_size=self._minibatch_size,\n pool=self._pool,\n ):\n with self.monitor(\n name='encode_texts_seconds',\n documentation='texts encode time in seconds',\n ):\n minibatch.embeddings = self._model.encode_text(\n batch_data\n )\n\n return docs\n" }, { "path": "server/clip_server/executors/clip_tensorrt.py", "content": "import warnings\nfrom functools import partial\nfrom multiprocessing.pool import ThreadPool\nfrom typing import Dict, Optional\n\nimport numpy as np\nfrom clip_server.executors.helper import (\n preproc_image,\n preproc_text,\n set_rank,\n split_img_txt_da,\n)\nfrom clip_server.model import clip\nfrom clip_server.model.clip_trt import CLIPTensorRTModel\nfrom clip_server.model.tokenization import Tokenizer\nfrom jina import DocumentArray, Executor, requests\nfrom opentelemetry.trace import NoOpTracer, Span\n\n\nclass CLIPEncoder(Executor):\n def __init__(\n self,\n name: str = 'ViT-B-32::openai',\n device: str = 'cuda',\n num_worker_preprocess: int = 4,\n minibatch_size: int = 32,\n access_paths: str = '@r',\n **kwargs,\n ):\n \"\"\"\n :param name: The name of the model to be used. Default 'ViT-B-32::openai'. A list of available models can be\n found at https://clip-as-service.jina.ai/user-guides/server/#model-support\n :param device: 'cpu' or 'cuda'. Default is 'cuda' since TensorRT is only supported on CUDA.\n :param num_worker_preprocess: The number of CPU workers to preprocess images and texts. Default is 4.\n :param minibatch_size: The size of the minibatch for preprocessing and encoding. Default is 32. Reduce this\n number if you encounter OOM errors.\n :param access_paths: The access paths to traverse on the input documents to get the images and texts to be\n processed. Visit https://docarray.jina.ai/fundamentals/documentarray/access-elements for more details.\n \"\"\"\n super().__init__(**kwargs)\n\n self._num_worker_preprocess = num_worker_preprocess\n self._pool = ThreadPool(processes=num_worker_preprocess)\n\n self._minibatch_size = minibatch_size\n self._access_paths = access_paths\n if 'traversal_paths' in kwargs:\n warnings.warn(\n f'`traversal_paths` is deprecated. Use `access_paths` instead.'\n )\n self._access_paths = kwargs['traversal_paths']\n\n self._device = device\n\n import torch\n\n assert self._device.startswith('cuda'), (\n f'can not perform inference on {self._device}'\n f' with Nvidia TensorRT as backend'\n )\n\n assert (\n torch.cuda.is_available()\n ), \"CUDA/GPU is not available on Pytorch. Please check your CUDA installation\"\n\n self._model = CLIPTensorRTModel(name)\n\n self._model.start_engines()\n\n self._tokenizer = Tokenizer(name)\n self._image_transform = clip._transform_blob(self._model.image_size)\n\n if not self.tracer:\n self.tracer = NoOpTracer()\n\n def _preproc_images(self, docs: 'DocumentArray', drop_image_content: bool):\n with self.monitor(\n name='preprocess_images_seconds',\n documentation='images preprocess time in seconds',\n ):\n with self.tracer.start_as_current_span('preprocess_images'):\n return preproc_image(\n docs,\n preprocess_fn=self._image_transform,\n device=self._device,\n return_np=False,\n drop_image_content=drop_image_content,\n )\n\n def _preproc_texts(self, docs: 'DocumentArray'):\n with self.monitor(\n name='preprocess_texts_seconds',\n documentation='texts preprocess time in seconds',\n ):\n with self.tracer.start_as_current_span('preprocess_images'):\n return preproc_text(\n docs,\n tokenizer=self._tokenizer,\n device=self._device,\n return_np=False,\n )\n\n @requests(on='/rank')\n async def rank(self, docs: 'DocumentArray', parameters: Dict, **kwargs):\n _drop_image_content = parameters.get('drop_image_content', False)\n await self.encode(docs['@r,m'], drop_image_content=_drop_image_content)\n\n set_rank(docs)\n\n @requests\n async def encode(\n self,\n docs: 'DocumentArray',\n tracing_context=None,\n parameters: Dict = {},\n **kwargs,\n ):\n with self.tracer.start_as_current_span(\n 'encode', context=tracing_context\n ) as span:\n span.set_attribute('device', self._device)\n span.set_attribute('runtime', 'tensorrt')\n access_paths = parameters.get('access_paths', self._access_paths)\n if 'traversal_paths' in parameters:\n warnings.warn(\n f'`traversal_paths` is deprecated. Use `access_paths` instead.'\n )\n access_paths = parameters['traversal_paths']\n _drop_image_content = parameters.get('drop_image_content', False)\n\n _img_da = DocumentArray()\n _txt_da = DocumentArray()\n for d in docs[access_paths]:\n split_img_txt_da(d, _img_da, _txt_da)\n\n with self.tracer.start_as_current_span('inference') as inference_span:\n inference_span.set_attribute('drop_image_content', _drop_image_content)\n inference_span.set_attribute('minibatch_size', self._minibatch_size)\n inference_span.set_attribute('has_img_da', True if _img_da else False)\n inference_span.set_attribute('has_txt_da', True if _txt_da else False)\n # for image\n if _img_da:\n with self.tracer.start_as_current_span(\n 'img_minibatch_encoding'\n ) as img_encode_span:\n for minibatch, batch_data in _img_da.map_batch(\n partial(\n self._preproc_images,\n drop_image_content=_drop_image_content,\n ),\n batch_size=self._minibatch_size,\n pool=self._pool,\n ):\n with self.monitor(\n name='encode_images_seconds',\n documentation='images encode time in seconds',\n ):\n minibatch.embeddings = (\n self._model.encode_image(batch_data)\n .detach()\n .cpu()\n .numpy()\n .astype(np.float32)\n )\n\n # for text\n if _txt_da:\n with self.tracer.start_as_current_span(\n 'txt_minibatch_encoding'\n ) as txt_encode_span:\n for minibatch, batch_data in _txt_da.map_batch(\n self._preproc_texts,\n batch_size=self._minibatch_size,\n pool=self._pool,\n ):\n with self.monitor(\n name='encode_texts_seconds',\n documentation='texts encode time in seconds',\n ):\n minibatch.embeddings = (\n self._model.encode_text(batch_data)\n .detach()\n .cpu()\n .numpy()\n .astype(np.float32)\n )\n\n return docs\n" }, { "path": "server/clip_server/executors/clip_torch.py", "content": "import os\nimport warnings\nfrom functools import partial\nfrom multiprocessing.pool import ThreadPool\nfrom typing import Dict, Union, Optional\n\nimport numpy as np\nimport torch\nfrom clip_server.executors.helper import (\n preproc_image,\n preproc_text,\n set_rank,\n split_img_txt_da,\n)\nfrom clip_server.helper import __cast_dtype__\nfrom clip_server.model import clip\nfrom clip_server.model.clip_model import CLIPModel\nfrom clip_server.model.tokenization import Tokenizer\nfrom jina import DocumentArray, Executor, requests\nfrom opentelemetry.trace import NoOpTracer, Span\n\n\nclass CLIPEncoder(Executor):\n def __init__(\n self,\n name: str = 'ViT-B-32::openai',\n device: Optional[str] = None,\n jit: bool = False,\n num_worker_preprocess: int = 4,\n minibatch_size: int = 32,\n access_paths: str = '@r',\n dtype: Optional[Union[str, torch.dtype]] = None,\n **kwargs,\n ):\n \"\"\"\n :param name: The name of the model to be used. Default 'ViT-B-32::openai'. A list of available models can be\n found at https://clip-as-service.jina.ai/user-guides/server/#model-support\n :param device: 'cpu' or 'cuda'. Default is None, which auto-detects the device.\n :param jit: Whether to use JIT compilation. Default is False.\n :param num_worker_preprocess: The number of CPU workers to preprocess images and texts. Default is 4.\n :param minibatch_size: The size of the minibatch for preprocessing and encoding. Default is 32. Reduce this\n number if you encounter OOM errors.\n :param access_paths: The access paths to traverse on the input documents to get the images and texts to be\n processed. Visit https://docarray.jina.ai/fundamentals/documentarray/access-elements for more details.\n :param dtype: inference data type, if None defaults to torch.float32 if device == 'cpu' else torch.float16.\n \"\"\"\n super().__init__(**kwargs)\n\n self._minibatch_size = minibatch_size\n self._access_paths = access_paths\n if 'traversal_paths' in kwargs:\n warnings.warn(\n f'`traversal_paths` is deprecated. Use `access_paths` instead.'\n )\n self._access_paths = kwargs['traversal_paths']\n\n if not device:\n device = 'cuda' if torch.cuda.is_available() else 'cpu'\n self._device = device\n if isinstance(dtype, str):\n dtype = __cast_dtype__.get(dtype)\n elif not dtype:\n dtype = (\n torch.float32\n if self._device in ('cpu', torch.device('cpu'))\n else torch.float16\n )\n self._dtype = dtype\n\n if not self._device.startswith('cuda') and (\n 'OMP_NUM_THREADS' not in os.environ\n and hasattr(self.runtime_args, 'replicas')\n ):\n replicas = getattr(self.runtime_args, 'replicas', 1)\n num_threads = max(1, torch.get_num_threads() // replicas)\n if num_threads < 2:\n warnings.warn(\n f'Too many replicas ({replicas}) vs too few threads {num_threads} may result in '\n f'sub-optimal performance.'\n )\n\n # NOTE: make sure to set the threads right after the torch import,\n # and `torch.set_num_threads` always take precedence over environment variables `OMP_NUM_THREADS`.\n # For more details, please see https://pytorch.org/docs/stable/generated/torch.set_num_threads.html\n torch.set_num_threads(max(num_threads, 1))\n torch.set_num_interop_threads(1)\n\n self._num_worker_preprocess = num_worker_preprocess\n self._pool = ThreadPool(processes=num_worker_preprocess)\n\n self._model = CLIPModel(\n name, device=self._device, jit=jit, dtype=dtype, **kwargs\n )\n self._tokenizer = Tokenizer(name)\n self._image_transform = clip._transform_blob(self._model.image_size)\n\n if not self.tracer:\n self.tracer = NoOpTracer()\n\n def _preproc_images(self, docs: 'DocumentArray', drop_image_content: bool):\n with self.monitor(\n name='preprocess_images_seconds',\n documentation='images preprocess time in seconds',\n ):\n with self.tracer.start_as_current_span('preprocess_images'):\n return preproc_image(\n docs,\n preprocess_fn=self._image_transform,\n device=self._device,\n return_np=False,\n drop_image_content=drop_image_content,\n dtype=self._dtype,\n )\n\n def _preproc_texts(self, docs: 'DocumentArray'):\n with self.monitor(\n name='preprocess_texts_seconds',\n documentation='texts preprocess time in seconds',\n ):\n with self.tracer.start_as_current_span('preprocess_images'):\n return preproc_text(\n docs,\n tokenizer=self._tokenizer,\n device=self._device,\n return_np=False,\n )\n\n @requests(on='/rank')\n async def rank(self, docs: 'DocumentArray', parameters: Dict, **kwargs):\n _drop_image_content = parameters.get('drop_image_content', False)\n await self.encode(docs['@r,m'], drop_image_content=_drop_image_content)\n\n set_rank(docs)\n\n @requests\n async def encode(\n self,\n docs: 'DocumentArray',\n tracing_context=None,\n parameters: Dict = {},\n **kwargs,\n ):\n with self.tracer.start_as_current_span(\n 'encode', context=tracing_context\n ) as span:\n span.set_attribute('device', self._device)\n span.set_attribute('runtime', 'torch')\n access_paths = parameters.get('access_paths', self._access_paths)\n if 'traversal_paths' in parameters:\n warnings.warn(\n f'`traversal_paths` is deprecated. Use `access_paths` instead.'\n )\n access_paths = parameters['traversal_paths']\n _drop_image_content = parameters.get('drop_image_content', False)\n\n _img_da = DocumentArray()\n _txt_da = DocumentArray()\n for d in docs[access_paths]:\n split_img_txt_da(d, _img_da, _txt_da)\n\n with self.tracer.start_as_current_span('inference') as inference_span:\n with torch.inference_mode():\n inference_span.set_attribute(\n 'drop_image_content', _drop_image_content\n )\n inference_span.set_attribute('minibatch_size', self._minibatch_size)\n inference_span.set_attribute(\n 'has_img_da', True if _img_da else False\n )\n inference_span.set_attribute(\n 'has_txt_da', True if _txt_da else False\n )\n # for image\n if _img_da:\n with self.tracer.start_as_current_span(\n 'img_minibatch_encoding'\n ) as img_encode_span:\n img_encode_span.set_attribute(\n 'num_pool_workers', self._num_worker_preprocess\n )\n for minibatch, batch_data in _img_da.map_batch(\n partial(\n self._preproc_images,\n drop_image_content=_drop_image_content,\n ),\n batch_size=self._minibatch_size,\n pool=self._pool,\n ):\n with self.monitor(\n name='encode_images_seconds',\n documentation='images encode time in seconds',\n ):\n minibatch.embeddings = (\n self._model.encode_image(**batch_data)\n .cpu()\n .numpy()\n .astype(np.float32)\n )\n\n # for text\n if _txt_da:\n with self.tracer.start_as_current_span(\n 'txt_minibatch_encoding'\n ) as txt_encode_span:\n txt_encode_span.set_attribute(\n 'num_pool_workers', self._num_worker_preprocess\n )\n for minibatch, batch_data in _txt_da.map_batch(\n self._preproc_texts,\n batch_size=self._minibatch_size,\n pool=self._pool,\n ):\n with self.monitor(\n name='encode_texts_seconds',\n documentation='texts encode time in seconds',\n ):\n minibatch.embeddings = (\n self._model.encode_text(**batch_data)\n .cpu()\n .numpy()\n .astype(np.float32)\n )\n\n return docs\n" }, { "path": "server/clip_server/executors/helper.py", "content": "from typing import Tuple, List, Callable, Any, Dict, Union\nimport torch\nimport numpy as np\nfrom docarray import Document, DocumentArray\nfrom docarray.math.distance.numpy import cosine\nfrom clip_server.helper import __cast_dtype__\n\n\nfrom clip_server.model.tokenization import Tokenizer\n\n\ndef numpy_softmax(x: 'np.ndarray', axis: int = -1) -> 'np.ndarray':\n max = np.max(x, axis=axis, keepdims=True)\n e_x = np.exp(x - max)\n div = np.sum(e_x, axis=axis, keepdims=True)\n f_x = e_x / div\n return f_x\n\n\ndef preproc_image(\n da: 'DocumentArray',\n preprocess_fn: Callable,\n device: str = 'cpu',\n return_np: bool = False,\n drop_image_content: bool = False,\n dtype: Union[str, torch.dtype] = torch.float32,\n) -> Tuple['DocumentArray', Dict]:\n\n if isinstance(dtype, str):\n dtype = __cast_dtype__.get(dtype)\n\n tensors_batch = []\n\n for d in da:\n content = d.content\n if d.tensor is not None:\n d.convert_image_tensor_to_blob()\n elif d.content_type != 'blob' and d.uri:\n # in case user uses HTTP protocol and send data via curl not using .blob (base64), but in .uri\n d.load_uri_to_blob()\n\n tensors_batch.append(preprocess_fn(d.blob).detach())\n\n # recover doc content\n d.content = content\n if drop_image_content:\n d.pop('blob', 'tensor')\n\n tensors_batch = torch.stack(tensors_batch).type(dtype)\n\n if return_np:\n tensors_batch = tensors_batch.cpu().numpy()\n else:\n tensors_batch = tensors_batch.to(device)\n\n return da, {'pixel_values': tensors_batch}\n\n\ndef preproc_text(\n da: 'DocumentArray',\n tokenizer: 'Tokenizer',\n device: str = 'cpu',\n return_np: bool = False,\n) -> Tuple['DocumentArray', Dict]:\n\n inputs = tokenizer(da.texts)\n inputs['input_ids'] = inputs['input_ids'].detach()\n\n if return_np:\n inputs['input_ids'] = inputs['input_ids'].cpu().numpy().astype(np.int32)\n inputs['attention_mask'] = (\n inputs['attention_mask'].cpu().numpy().astype(np.int32)\n )\n else:\n inputs['input_ids'] = inputs['input_ids'].to(device)\n inputs['attention_mask'] = inputs['attention_mask'].to(device)\n\n da[:, 'mime_type'] = 'text'\n return da, inputs\n\n\ndef split_img_txt_da(doc: 'Document', img_da: 'DocumentArray', txt_da: 'DocumentArray'):\n if doc.text:\n txt_da.append(doc)\n elif doc.blob or (doc.tensor is not None) or doc.uri:\n img_da.append(doc)\n\n\ndef set_rank(docs, _logit_scale=np.exp(4.60517)):\n queries = docs\n candidates = docs['@m']\n\n query_embeddings = queries.embeddings # Q X D\n candidate_embeddings = candidates.embeddings # C = Sum(C_q1, C_q2, C_q3,...) x D\n cosine_scores = 1 - cosine(\n query_embeddings, candidate_embeddings\n ) # Q x C Block matix\n start_idx = 0\n for q, _cosine_scores in zip(docs, cosine_scores):\n\n _candidates = q.matches\n\n end_idx = start_idx + len(_candidates)\n\n _candidate_cosines = _cosine_scores[start_idx:end_idx]\n _candidate_softmaxs = numpy_softmax(_logit_scale * _candidate_cosines)\n for c, _c_score, _s_score in zip(\n _candidates, _candidate_cosines, _candidate_softmaxs\n ):\n c.scores['clip_score'].value = _s_score\n c.scores['clip_score'].op_name = 'softmax'\n\n c.scores['clip_score_cosine'].value = _c_score\n c.scores['clip_score_cosine'].op_name = 'cosine'\n\n start_idx = end_idx\n\n _candidates.embeddings = None # remove embedding to save bandwidth\n\n final = sorted(\n _candidates, key=lambda _m: _m.scores['clip_score'].value, reverse=True\n )\n\n q.matches = final\n\n\ndef get_image_size(name: str):\n from clip_server.model.pretrained_models import _VISUAL_MODEL_IMAGE_SIZE\n\n return _VISUAL_MODEL_IMAGE_SIZE[name]\n" }, { "path": "server/clip_server/helper.py", "content": "import json\nimport os\nimport sys\nimport threading\nimport torch\nfrom packaging.version import Version\nfrom urllib.request import Request, urlopen\n\nimport pkg_resources\nfrom rich import print\nfrom rich.panel import Panel\n\n__resources_path__ = os.path.join(\n os.path.dirname(\n sys.modules.get('clip_server').__file__\n if 'clip_server' in sys.modules\n else __file__\n ),\n 'resources',\n)\n\n\n__cast_dtype__ = {'fp16': torch.float16, 'fp32': torch.float32, 'bf16': torch.bfloat16}\n\n\ndef _version_check(package: str = None, github_repo: str = None):\n try:\n\n if not package:\n package = vars(sys.modules[__name__])['__package__']\n if not github_repo:\n github_repo = package\n\n cur_ver = Version(pkg_resources.get_distribution(package).version)\n req = Request(\n f'https://pypi.python.org/pypi/{package}/json',\n headers={'User-Agent': 'Mozilla/5.0'},\n )\n with urlopen(\n req, timeout=1\n ) as resp: # 'with' is important to close the resource after use\n j = json.load(resp)\n releases = j.get('releases', {})\n latest_release_ver = max(\n Version(v) for v in releases.keys() if '.dev' not in v\n )\n if cur_ver < latest_release_ver:\n print(\n Panel(\n f'You are using [b]{package} {cur_ver}[/b], but [bold green]{latest_release_ver}[/] is available. '\n f'You may upgrade it via [b]pip install -U {package}[/b]. [link=https://github.com/jina-ai/{github_repo}/releases]Read Changelog here[/link].',\n title=':new: New version available!',\n width=50,\n )\n )\n except:\n # no network, too slow, PyPi is down\n pass\n\n\ndef is_latest_version(package: str = None, github_repo: str = None) -> None:\n \"\"\"Check if there is a latest version from Pypi, set env `NO_VERSION_CHECK` to disable it.\n\n :param package: package name if none auto-detected\n :param github_repo: repo name that contains CHANGELOG if none then the same as package name\n \"\"\"\n\n threading.Thread(target=_version_check, args=(package, github_repo)).start()\n" }, { "path": "server/clip_server/model/__init__.py", "content": "" }, { "path": "server/clip_server/model/clip.py", "content": "# Originally from https://github.com/openai/CLIP. MIT License, Copyright (c) 2021 OpenAI\n\nimport io\n\nimport pillow_avif\nfrom PIL import Image\nfrom torchvision.transforms import Compose, Resize, CenterCrop, ToTensor, Normalize\n\ntry:\n from torchvision.transforms import InterpolationMode\n\n BICUBIC = InterpolationMode.BICUBIC\nexcept ImportError:\n BICUBIC = Image.BICUBIC\n\n\ndef _convert_image_to_rgb(image):\n return image.convert('RGB')\n\n\ndef _blob2image(blob):\n return Image.open(io.BytesIO(blob))\n\n\ndef _transform_blob(n_px):\n return Compose(\n [\n _blob2image,\n Resize(n_px, interpolation=BICUBIC),\n CenterCrop(n_px),\n _convert_image_to_rgb,\n ToTensor(),\n Normalize(\n (0.48145466, 0.4578275, 0.40821073),\n (0.26862954, 0.26130258, 0.27577711),\n ),\n ]\n )\n\n\ndef _transform_ndarray(n_px):\n return Compose(\n [\n ToTensor(),\n Resize(n_px, interpolation=BICUBIC),\n CenterCrop(n_px),\n Normalize(\n (0.48145466, 0.4578275, 0.40821073),\n (0.26862954, 0.26130258, 0.27577711),\n ),\n ]\n )\n" }, { "path": "server/clip_server/model/clip_model.py", "content": "from clip_server.model.pretrained_models import (\n _OPENCLIP_MODELS,\n _MULTILINGUALCLIP_MODELS,\n _CNCLIP_MODELS,\n _VISUAL_MODEL_IMAGE_SIZE,\n)\n\n\nclass BaseCLIPModel:\n def __init__(self, name: str, **kwargs):\n super().__init__()\n self._name = name\n\n @staticmethod\n def get_model_name(name: str):\n return name\n\n @property\n def model_name(self):\n return self.__class__.get_model_name(self._name)\n\n @property\n def image_size(self):\n return _VISUAL_MODEL_IMAGE_SIZE.get(self.model_name, None)\n\n\nclass CLIPModel(BaseCLIPModel):\n def __new__(cls, name: str, **kwargs):\n if cls is CLIPModel:\n if name in _OPENCLIP_MODELS:\n from clip_server.model.openclip_model import OpenCLIPModel\n\n instance = super().__new__(OpenCLIPModel)\n elif name in _MULTILINGUALCLIP_MODELS:\n from clip_server.model.mclip_model import MultilingualCLIPModel\n\n instance = super().__new__(MultilingualCLIPModel)\n elif name in _CNCLIP_MODELS:\n from clip_server.model.cnclip_model import CNClipModel\n\n instance = super().__new__(CNClipModel)\n else:\n raise ValueError(\n 'CLIP model {} not found; below is a list of all available models:\\n{}'.format(\n name,\n ''.join(\n [\n '\\t- {}\\n'.format(i)\n for i in list(_OPENCLIP_MODELS.keys())\n + list(_MULTILINGUALCLIP_MODELS.keys())\n + list(_CNCLIP_MODELS.keys())\n ]\n ),\n )\n )\n else:\n instance = super().__new__(cls)\n return instance\n" }, { "path": "server/clip_server/model/clip_onnx.py", "content": "import os\nfrom typing import Dict, Optional\n\nfrom clip_server.model.pretrained_models import (\n download_model,\n _OPENCLIP_MODELS,\n _MULTILINGUALCLIP_MODELS,\n)\nfrom clip_server.model.clip_model import BaseCLIPModel\n\n_S3_BUCKET = (\n 'https://clip-as-service.s3.us-east-2.amazonaws.com/models/onnx/' # Deprecated\n)\n_S3_BUCKET_V2 = 'https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d53657276696365/onnx/'\n_MODELS = {\n 'RN50::openai': (\n ('RN50/textual.onnx', '722418bfe47a1f5c79d1f44884bb3103'),\n ('RN50/visual.onnx', '5761475db01c3abb68a5a805662dcd10'),\n ),\n 'RN50::yfcc15m': (\n ('RN50-yfcc15m/textual.onnx', '4ff2ea7228b9d2337b5440d1955c2108'),\n ('RN50-yfcc15m/visual.onnx', '87daa9b4a67449b5390a9a73b8c15772'),\n ),\n 'RN50::cc12m': (\n ('RN50-cc12m/textual.onnx', '78fa0ae0ea47aca4b8864f709c48dcec'),\n ('RN50-cc12m/visual.onnx', '0e04bf92f3c181deea2944e322ebee77'),\n ),\n 'RN101::openai': (\n ('RN101/textual.onnx', '2d9efb7d184c0d68a369024cedfa97af'),\n ('RN101/visual.onnx', '0297ebc773af312faab54f8b5a622d71'),\n ),\n 'RN101::yfcc15m': (\n ('RN101-yfcc15m/textual.onnx', '7aa2a4e3d5b960998a397a6712389f08'),\n ('RN101-yfcc15m/visual.onnx', '681a72dd91c9c79464947bf29b623cb4'),\n ),\n 'RN50x4::openai': (\n ('RN50x4/textual.onnx', 'd9d63d3fe35fb14d4affaa2c4e284005'),\n ('RN50x4/visual.onnx', '16afe1e35b85ad862e8bbdb12265c9cb'),\n ),\n 'RN50x16::openai': (\n ('RN50x16/textual.onnx', '1525785494ff5307cadc6bfa56db6274'),\n ('RN50x16/visual.onnx', '2a293d9c3582f8abe29c9999e47d1091'),\n ),\n 'RN50x64::openai': (\n ('RN50x64/textual.onnx', '3ae8ade74578eb7a77506c11bfbfaf2c'),\n ('RN50x64/visual.onnx', '1341f10b50b3aca6d2d5d13982cabcfc'),\n ),\n 'ViT-B-32::openai': (\n ('ViT-B-32/textual.onnx', 'bd6d7871e8bb95f3cc83aff3398d7390'),\n ('ViT-B-32/visual.onnx', '88c6f38e522269d6c04a85df18e6370c'),\n ),\n 'ViT-B-32::laion2b_e16': (\n ('ViT-B-32-laion2b_e16/textual.onnx', 'aa6eac88fe77d21f337e806417957497'),\n ('ViT-B-32-laion2b_e16/visual.onnx', '0cdc00a9dfad560153d40aced9df0c8f'),\n ),\n 'ViT-B-32::laion400m_e31': (\n ('ViT-B-32-laion400m_e31/textual.onnx', '832f417bf1b3f1ced8f9958eda71665c'),\n ('ViT-B-32-laion400m_e31/visual.onnx', '62326b925ae342313d4cc99c2741b313'),\n ),\n 'ViT-B-32::laion400m_e32': (\n ('ViT-B-32-laion400m_e32/textual.onnx', '93284915937ba42a2b52ae8d3e5283a0'),\n ('ViT-B-32-laion400m_e32/visual.onnx', 'db220821a31fe9795fd8c2ba419078c5'),\n ),\n 'ViT-B-32::laion2b-s34b-b79k': (\n ('ViT-B-32-laion2b-s34b-b79k/textual.onnx', '84af5ae53da56464c76e67fe50fddbe9'),\n ('ViT-B-32-laion2b-s34b-b79k/visual.onnx', 'a2d4cbd1cf2632cd09ffce9b40bfd8bd'),\n ),\n 'ViT-B-16::openai': (\n ('ViT-B-16/textual.onnx', '6f0976629a446f95c0c8767658f12ebe'),\n ('ViT-B-16/visual.onnx', 'd5c03bfeef1abbd9bede54a8f6e1eaad'),\n ),\n 'ViT-B-16::laion400m_e31': (\n ('ViT-B-16-laion400m_e31/textual.onnx', '5db27763c06c06c727c90240264bf4f7'),\n ('ViT-B-16-laion400m_e31/visual.onnx', '04a6a780d855a36eee03abca64cd5361'),\n ),\n 'ViT-B-16::laion400m_e32': (\n ('ViT-B-16-laion400m_e32/textual.onnx', '9abe000a51b6f1cbaac8fde601b16725'),\n ('ViT-B-16-laion400m_e32/visual.onnx', 'd38c144ac3ad7fbc1966f88ff8fa522f'),\n ),\n 'ViT-B-16-plus-240::laion400m_e31': (\n (\n 'ViT-B-16-plus-240-laion400m_e31/textual.onnx',\n '2b524e7a530a98010cc7e57756937c5c',\n ),\n (\n 'ViT-B-16-plus-240-laion400m_e31/visual.onnx',\n 'a78989da3300fd0c398a9877dd26a9f1',\n ),\n ),\n 'ViT-B-16-plus-240::laion400m_e32': (\n (\n 'ViT-B-16-plus-240-laion400m_e32/textual.onnx',\n '53c8d26726b386ca0749207876482907',\n ),\n (\n 'ViT-B-16-plus-240-laion400m_e32/visual.onnx',\n '7a32c4272c1ee46f734486570d81584b',\n ),\n ),\n 'ViT-L-14::openai': (\n ('ViT-L-14/textual.onnx', '325380b31af4837c2e0d9aba2fad8e1b'),\n ('ViT-L-14/visual.onnx', '53f5b319d3dc5d42572adea884e31056'),\n ),\n 'ViT-L-14::laion400m_e31': (\n ('ViT-L-14-laion400m_e31/textual.onnx', '36216b85e32668ea849730a54e1e09a4'),\n ('ViT-L-14-laion400m_e31/visual.onnx', '15fa5a24916e2a58325c5cf70350c300'),\n ),\n 'ViT-L-14::laion400m_e32': (\n ('ViT-L-14-laion400m_e32/textual.onnx', '8ba5b76ba71992923470c0261b10a67c'),\n ('ViT-L-14-laion400m_e32/visual.onnx', '49db3ba92bd816001e932530ad92d76c'),\n ),\n 'ViT-L-14::laion2b-s32b-b82k': (\n ('ViT-L-14-laion2b-s32b-b82k/textual.onnx', 'da36a6cbed4f56abf576fdea8b6fe2ee'),\n ('ViT-L-14-laion2b-s32b-b82k/visual.onnx', '1e337a190abba6a8650237dfae4740b7'),\n ),\n 'ViT-L-14-336::openai': (\n ('ViT-L-14@336px/textual.onnx', '78fab479f136403eed0db46f3e9e7ed2'),\n ('ViT-L-14@336px/visual.onnx', 'f3b1f5d55ca08d43d749e11f7e4ba27e'),\n ),\n 'ViT-H-14::laion2b-s32b-b79k': (\n ('ViT-H-14-laion2b-s32b-b79k/textual.onnx', '41e73c0c871d0e8e5d5e236f917f1ec3'),\n ('ViT-H-14-laion2b-s32b-b79k/visual.zip', '38151ea5985d73de94520efef38db4e7'),\n ),\n 'ViT-g-14::laion2b-s12b-b42k': (\n ('ViT-g-14-laion2b-s12b-b42k/textual.onnx', 'e597b7ab4414ecd92f715d47e79a033f'),\n ('ViT-g-14-laion2b-s12b-b42k/visual.zip', '6d0ac4329de9b02474f4752a5d16ba82'),\n ),\n # older version name format\n 'RN50': (\n ('RN50/textual.onnx', '722418bfe47a1f5c79d1f44884bb3103'),\n ('RN50/visual.onnx', '5761475db01c3abb68a5a805662dcd10'),\n ),\n 'RN101': (\n ('RN101/textual.onnx', '2d9efb7d184c0d68a369024cedfa97af'),\n ('RN101/visual.onnx', '0297ebc773af312faab54f8b5a622d71'),\n ),\n 'RN50x4': (\n ('RN50x4/textual.onnx', 'd9d63d3fe35fb14d4affaa2c4e284005'),\n ('RN50x4/visual.onnx', '16afe1e35b85ad862e8bbdb12265c9cb'),\n ),\n 'RN50x16': (\n ('RN50x16/textual.onnx', '1525785494ff5307cadc6bfa56db6274'),\n ('RN50x16/visual.onnx', '2a293d9c3582f8abe29c9999e47d1091'),\n ),\n 'RN50x64': (\n ('RN50x64/textual.onnx', '3ae8ade74578eb7a77506c11bfbfaf2c'),\n ('RN50x64/visual.onnx', '1341f10b50b3aca6d2d5d13982cabcfc'),\n ),\n 'ViT-B/32': (\n ('ViT-B-32/textual.onnx', 'bd6d7871e8bb95f3cc83aff3398d7390'),\n ('ViT-B-32/visual.onnx', '88c6f38e522269d6c04a85df18e6370c'),\n ),\n 'ViT-B/16': (\n ('ViT-B-16/textual.onnx', '6f0976629a446f95c0c8767658f12ebe'),\n ('ViT-B-16/visual.onnx', 'd5c03bfeef1abbd9bede54a8f6e1eaad'),\n ),\n 'ViT-L/14': (\n ('ViT-L-14/textual.onnx', '325380b31af4837c2e0d9aba2fad8e1b'),\n ('ViT-L-14/visual.onnx', '53f5b319d3dc5d42572adea884e31056'),\n ),\n 'ViT-L/14@336px': (\n ('ViT-L-14@336px/textual.onnx', '78fab479f136403eed0db46f3e9e7ed2'),\n ('ViT-L-14@336px/visual.onnx', 'f3b1f5d55ca08d43d749e11f7e4ba27e'),\n ),\n # MultilingualCLIP models\n 'M-CLIP/LABSE-Vit-L-14': (\n ('M-CLIP-LABSE-Vit-L-14/textual.onnx', '03727820116e63c7d19c72bb5d839488'),\n ('M-CLIP-LABSE-Vit-L-14/visual.onnx', 'a78028eab30084c3913edfb0c8411f15'),\n ),\n 'M-CLIP/XLM-Roberta-Large-Vit-B-32': (\n (\n 'M-CLIP-XLM-Roberta-Large-Vit-B-32/textual.zip',\n '41f51ec9af4754d11c7b7929e2caf5b9',\n ),\n (\n 'M-CLIP-XLM-Roberta-Large-Vit-B-32/visual.onnx',\n '5f18f68ac94e294863bfd1f695c8c5ca',\n ),\n ),\n 'M-CLIP/XLM-Roberta-Large-Vit-B-16Plus': (\n (\n 'M-CLIP-XLM-Roberta-Large-Vit-B-16Plus/textual.zip',\n '6c3e55f7d2d6c12f2c1f1dd36fdec607',\n ),\n (\n 'M-CLIP-XLM-Roberta-Large-Vit-B-16Plus/visual.onnx',\n '467a3ef3e5f50abcf850c3db9e705f8e',\n ),\n ),\n 'M-CLIP/XLM-Roberta-Large-Vit-L-14': (\n (\n 'M-CLIP-XLM-Roberta-Large-Vit-L-14/textual.zip',\n '3dff00335dc3093acb726dab975ae57d',\n ),\n (\n 'M-CLIP-XLM-Roberta-Large-Vit-L-14/visual.onnx',\n 'a78028eab30084c3913edfb0c8411f15',\n ),\n ),\n}\n\n\nclass CLIPOnnxModel(BaseCLIPModel):\n def __init__(\n self, name: str, model_path: str = None, dtype: Optional[str] = 'fp32'\n ):\n super().__init__(name)\n self._dtype = dtype\n if name in _MODELS:\n if not model_path:\n cache_dir = os.path.expanduser(\n f'~/.cache/clip/{name.replace(\"/\", \"-\").replace(\"::\", \"-\")}'\n )\n textual_model_name, textual_model_md5 = _MODELS[name][0]\n self._textual_path = download_model(\n url=_S3_BUCKET_V2 + textual_model_name,\n target_folder=cache_dir,\n md5sum=textual_model_md5,\n with_resume=True,\n )\n visual_model_name, visual_model_md5 = _MODELS[name][1]\n self._visual_path = download_model(\n url=_S3_BUCKET_V2 + visual_model_name,\n target_folder=cache_dir,\n md5sum=visual_model_md5,\n with_resume=True,\n )\n else:\n if os.path.isdir(model_path):\n self._textual_path = os.path.join(model_path, 'textual.onnx')\n self._visual_path = os.path.join(model_path, 'visual.onnx')\n if not os.path.isfile(self._textual_path) or not os.path.isfile(\n self._visual_path\n ):\n raise RuntimeError(\n f'The given model path {model_path} does not contain `textual.onnx` and `visual.onnx`'\n )\n else:\n raise RuntimeError(\n f'The given model path {model_path} should be a folder containing both '\n f'`textual.onnx` and `visual.onnx`.'\n )\n else:\n raise RuntimeError(\n 'CLIP model {} not found or not supports ONNX backend; below is a list of all available models:\\n{}'.format(\n name,\n ''.join(['\\t- {}\\n'.format(i) for i in list(_MODELS.keys())]),\n )\n )\n\n @staticmethod\n def get_model_name(name: str):\n if name in _OPENCLIP_MODELS:\n from clip_server.model.openclip_model import OpenCLIPModel\n\n return OpenCLIPModel.get_model_name(name)\n elif name in _MULTILINGUALCLIP_MODELS:\n from clip_server.model.mclip_model import MultilingualCLIPModel\n\n return MultilingualCLIPModel.get_model_name(name)\n\n return name\n\n def start_sessions(\n self,\n dtype,\n **kwargs,\n ):\n import onnxruntime as ort\n\n def _load_session(model_path: str, model_type: str, dtype: str):\n if model_path.endswith('.zip') or dtype == 'fp16':\n import tempfile\n\n with tempfile.TemporaryDirectory() as tmp_dir:\n tmp_model_path = tmp_dir + f'/{model_type}.onnx'\n if model_path.endswith('.zip'):\n import zipfile\n\n with zipfile.ZipFile(model_path, 'r') as zip_ref:\n zip_ref.extractall(tmp_dir)\n model_path = tmp_model_path\n if dtype == 'fp16':\n import onnx\n from onnxmltools.utils import float16_converter\n\n model_fp16 = (\n float16_converter.convert_float_to_float16_model_path(\n model_path\n )\n )\n onnx.save_model(model_fp16, tmp_model_path)\n return ort.InferenceSession(tmp_model_path, **kwargs)\n return ort.InferenceSession(model_path, **kwargs)\n\n self._visual_session = _load_session(self._visual_path, 'visual', dtype)\n self._textual_session = _load_session(self._textual_path, 'textual', dtype)\n self._visual_session.disable_fallback()\n self._textual_session.disable_fallback()\n\n def encode_image(self, image_input: Dict):\n (visual_output,) = self._visual_session.run(None, image_input)\n return visual_output\n\n def encode_text(self, text_input: Dict):\n (textual_output,) = self._textual_session.run(None, text_input)\n return textual_output\n" }, { "path": "server/clip_server/model/clip_trt.py", "content": "import os\nfrom typing import Dict\n\ntry:\n import tensorrt as trt\n from tensorrt.tensorrt import Logger, Runtime\n\n from clip_server.model.trt_utils import load_engine, build_engine, save_engine\nexcept ImportError:\n raise ImportError(\n \"It seems that TensorRT is not yet installed. \"\n \"It is required when you declare TensorRT backend.\"\n \"Please find installation instruction on \"\n \"https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html\"\n )\nfrom clip_server.model.pretrained_models import (\n _OPENCLIP_MODELS,\n _MULTILINGUALCLIP_MODELS,\n)\nfrom clip_server.model.clip_model import BaseCLIPModel\nfrom clip_server.model.clip_onnx import _MODELS as ONNX_MODELS\n\n_MODELS = [\n 'RN50::openai',\n 'RN50::yfcc15m',\n 'RN50::cc12m',\n 'RN101::openai',\n 'RN101::yfcc15m',\n 'RN50x4::openai',\n 'ViT-B-32::openai',\n 'ViT-B-32::laion2b_e16',\n 'ViT-B-32::laion400m_e31',\n 'ViT-B-32::laion400m_e32',\n 'ViT-B-16::openai',\n 'ViT-B-16::laion400m_e31',\n 'ViT-B-16::laion400m_e32',\n # older version name format\n 'RN50',\n 'RN101',\n 'RN50x4',\n # 'RN50x16',\n # 'RN50x64',\n 'ViT-B/32',\n 'ViT-B/16',\n # 'ViT-L/14',\n # 'ViT-L/14@336px',\n]\n\n\nclass CLIPTensorRTModel(BaseCLIPModel):\n def __init__(\n self,\n name: str,\n ):\n super().__init__(name)\n\n if name in _MODELS:\n cache_dir = os.path.expanduser(\n f'~/.cache/clip/{name.replace(\"/\", \"-\").replace(\"::\", \"-\")}'\n )\n\n self._textual_path = os.path.join(\n cache_dir,\n f'textual.{ONNX_MODELS[name][0][1]}.trt',\n )\n self._visual_path = os.path.join(\n cache_dir,\n f'visual.{ONNX_MODELS[name][1][1]}.trt',\n )\n\n if not os.path.exists(self._textual_path) or not os.path.exists(\n self._visual_path\n ):\n from clip_server.model.clip_onnx import CLIPOnnxModel\n\n trt_logger: Logger = trt.Logger(trt.Logger.ERROR)\n runtime: Runtime = trt.Runtime(trt_logger)\n onnx_model = CLIPOnnxModel(name)\n\n visual_engine = build_engine(\n runtime=runtime,\n onnx_file_path=onnx_model._visual_path,\n logger=trt_logger,\n min_shape=(1, 3, onnx_model.image_size, onnx_model.image_size),\n optimal_shape=(\n 768,\n 3,\n onnx_model.image_size,\n onnx_model.image_size,\n ),\n max_shape=(\n 1024,\n 3,\n onnx_model.image_size,\n onnx_model.image_size,\n ),\n workspace_size=10000 * 1024 * 1024,\n fp16=False,\n int8=False,\n )\n save_engine(visual_engine, self._visual_path)\n\n text_engine = build_engine(\n runtime=runtime,\n onnx_file_path=onnx_model._textual_path,\n logger=trt_logger,\n min_shape=(1, 77),\n optimal_shape=(768, 77),\n max_shape=(1024, 77),\n workspace_size=10000 * 1024 * 1024,\n fp16=False,\n int8=False,\n )\n save_engine(text_engine, self._textual_path)\n else:\n raise RuntimeError(\n 'CLIP model {} not found or not supports Nvidia TensorRT backend; below is a list of all available models:\\n{}'.format(\n name,\n ''.join(['\\t- {}\\n'.format(i) for i in list(_MODELS.keys())]),\n )\n )\n\n @staticmethod\n def get_model_name(name: str):\n if name in _OPENCLIP_MODELS:\n from clip_server.model.openclip_model import OpenCLIPModel\n\n return OpenCLIPModel.get_model_name(name)\n elif name in _MULTILINGUALCLIP_MODELS:\n from clip_server.model.mclip_model import MultilingualCLIPModel\n\n return MultilingualCLIPModel.get_model_name(name)\n\n return name\n\n def start_engines(self):\n trt_logger: Logger = trt.Logger(trt.Logger.ERROR)\n runtime: Runtime = trt.Runtime(trt_logger)\n self._textual_engine = load_engine(runtime, self._textual_path)\n self._visual_engine = load_engine(runtime, self._visual_path)\n\n def encode_image(self, image_input: Dict):\n (visual_output,) = self._visual_engine(image_input)\n return visual_output\n\n def encode_text(self, text_input: Dict):\n (textual_output,) = self._textual_engine(text_input)\n return textual_output\n" }, { "path": "server/clip_server/model/cnclip_model.py", "content": "# Originally from https://github.com/OFA-Sys/Chinese-CLIP. MIT License.\n\nimport torch\n\nfrom clip_server.model.clip_model import CLIPModel\nfrom clip_server.model.pretrained_models import _VISUAL_MODEL_IMAGE_SIZE\nfrom cn_clip.clip import load_from_name\n\n_CNCLIP_MODEL_MAPS = {\n 'CN-CLIP/ViT-B-16': 'ViT-B-16',\n 'CN-CLIP/ViT-L-14': 'ViT-L-14',\n 'CN-CLIP/ViT-L-14-336': 'ViT-L-14-336',\n 'CN-CLIP/ViT-H-14': 'ViT-H-14',\n 'CN-CLIP/RN50': 'RN50',\n}\n\n\nclass CNClipModel(CLIPModel):\n def __init__(\n self,\n name: str,\n device: str = 'cpu',\n jit: bool = False,\n dtype: str = None,\n **kwargs\n ):\n super().__init__(name, **kwargs)\n self._name = _CNCLIP_MODEL_MAPS[name]\n\n self._model, self._preprocess = load_from_name(\n _CNCLIP_MODEL_MAPS[name], device=device\n )\n self._model.eval()\n\n @staticmethod\n def get_model_name(name: str):\n return _CNCLIP_MODEL_MAPS[name]\n\n def encode_text(self, input_ids: 'torch.Tensor', **kwargs):\n return self._model.encode_text(input_ids).detach()\n\n def encode_image(self, pixel_values: 'torch.Tensor', **kwargs):\n return self._model.encode_image(pixel_values).detach()\n\n @property\n def model_name(self):\n return self.__class__.get_model_name(self._name)\n\n @property\n def image_size(self):\n return _VISUAL_MODEL_IMAGE_SIZE.get(self._name, None)\n" }, { "path": "server/clip_server/model/flash_attention.py", "content": "import torch\nimport torch.nn as nn\nfrom torch import Tensor\nfrom typing import Optional, Tuple\n\nfrom torch.nn.functional import linear\nfrom flash_attn.flash_attn_interface import flash_attn_unpadded_func\n\n\nclass MultiheadAttention(nn.MultiheadAttention):\n def __init__(\n self,\n embed_dim,\n num_heads,\n dropout=0,\n bias=True,\n add_bias_kv=False,\n add_zero_attn=False,\n kdim=None,\n vdim=None,\n batch_first=False,\n device=None,\n dtype=None,\n ) -> None:\n super().__init__(\n embed_dim,\n num_heads,\n dropout,\n bias,\n add_bias_kv,\n add_zero_attn,\n kdim,\n vdim,\n batch_first,\n device,\n dtype,\n )\n\n def attention(\n self,\n q,\n k,\n v,\n batch_size=1,\n seqlen=77,\n softmax_scale=None,\n attention_dropout=0.0,\n causal=False,\n cu_seqlens=None,\n max_s=None,\n need_weights=False,\n ):\n \"\"\"Implements the multihead softmax attention.\n Arguments\n ---------\n q,k,v: The tensor containing the query, key, and value. each of (B*S, H, D)\n key_padding_mask: a bool tensor of shape (B, S)\n\n \"\"\"\n\n if cu_seqlens is None:\n max_s = seqlen\n cu_seqlens = torch.arange(\n 0,\n (batch_size + 1) * seqlen,\n step=seqlen,\n dtype=torch.int32,\n device=q.device,\n )\n output = flash_attn_unpadded_func(\n q,\n k,\n v,\n cu_seqlens,\n cu_seqlens,\n max_s,\n max_s,\n attention_dropout,\n softmax_scale=softmax_scale,\n causal=causal,\n )\n\n return output\n\n def forward(\n self,\n query: Tensor,\n key: Tensor,\n value: Tensor,\n key_padding_mask: Optional[Tensor] = None,\n need_weights: bool = False,\n attn_mask: Optional[Tensor] = None,\n average_attn_weights: bool = True,\n ) -> Tuple[Tensor, Optional[Tensor]]:\n # set up shape vars\n seqlen, batch_size, embed_dim = query.shape\n\n # in-projection and rearrange `b s (h d) -> (b s) h d`\n q, k, v = linear(query, self.in_proj_weight, self.in_proj_bias).chunk(3, dim=-1)\n q = (\n q.transpose(0, 1)\n .contiguous()\n .view(batch_size * seqlen, self.num_heads, self.head_dim)\n )\n k = (\n k.transpose(0, 1)\n .contiguous()\n .view(batch_size * seqlen, self.num_heads, self.head_dim)\n )\n v = (\n v.transpose(0, 1)\n .contiguous()\n .view(batch_size * seqlen, self.num_heads, self.head_dim)\n )\n\n # flash attention (use causal mask)\n causal = attn_mask is not None\n attn_output = self.attention(q, k, v, batch_size, seqlen, causal=causal)\n\n # out-projection\n # `(b s) h d -> s b (h d)`\n attn_output = attn_output.contiguous().view(\n batch_size, seqlen, self.num_heads, self.head_dim\n )\n attn_output = (\n attn_output.transpose(0, 1).contiguous().view(seqlen, batch_size, embed_dim)\n )\n attn_output = linear(attn_output, self.out_proj.weight, self.out_proj.bias)\n\n return attn_output, None\n" }, { "path": "server/clip_server/model/mclip_model.py", "content": "# Originally from https://github.com/FreddeFrallan/Multilingual-CLIP. MIT License, Copyright (c) 2022 Multilingual-CLIP\n\nimport transformers\nimport torch\n\nfrom clip_server.model.clip_model import CLIPModel\nfrom clip_server.model.openclip_model import OpenCLIPModel\n\n_CLIP_MODEL_MAPS = {\n 'M-CLIP/XLM-Roberta-Large-Vit-B-32': 'ViT-B-32::openai',\n 'M-CLIP/XLM-Roberta-Large-Vit-L-14': 'ViT-L-14::openai',\n 'M-CLIP/XLM-Roberta-Large-Vit-B-16Plus': 'ViT-B-16-plus-240::laion400m_e31',\n 'M-CLIP/LABSE-Vit-L-14': 'ViT-L-14::openai',\n}\n\n\nclass MCLIPConfig(transformers.PretrainedConfig):\n model_type = \"M-CLIP\"\n\n def __init__(\n self,\n modelBase: str = 'xlm-roberta-large',\n transformerDimSize: int = 1024,\n imageDimSize: int = 768,\n **kwargs\n ):\n self.transformerDimensions = transformerDimSize\n self.numDims = imageDimSize\n self.modelBase = modelBase\n super().__init__(**kwargs)\n\n\nclass MultilingualCLIP(transformers.PreTrainedModel):\n config_class = MCLIPConfig\n\n def __init__(self, config, *args, **kwargs):\n super().__init__(config, *args, **kwargs)\n self.transformer = transformers.AutoModel.from_pretrained(config.modelBase)\n self.LinearTransformation = torch.nn.Linear(\n in_features=config.transformerDimensions, out_features=config.numDims\n )\n\n def forward(self, input_ids: torch.Tensor, attention_mask: torch.Tensor, **kwargs):\n embs = self.transformer(\n input_ids=input_ids, attention_mask=attention_mask, **kwargs\n )[0]\n embs = (embs * attention_mask.unsqueeze(2)).sum(dim=1) / attention_mask.sum(\n dim=1\n )[:, None]\n return self.LinearTransformation(embs)\n\n\nclass MultilingualCLIPModel(CLIPModel):\n def __init__(self, name: str, device: str = 'cpu', jit: bool = False, **kwargs):\n super().__init__(name, **kwargs)\n self._mclip_model = MultilingualCLIP.from_pretrained(name)\n self._mclip_model.to(device=device)\n self._mclip_model.eval()\n self._model = OpenCLIPModel(_CLIP_MODEL_MAPS[name], device=device, jit=jit)\n\n @staticmethod\n def get_model_name(name: str):\n return _CLIP_MODEL_MAPS[name].split('::')[0]\n\n def encode_text(\n self, input_ids: 'torch.Tensor', attention_mask: 'torch.Tensor', **kwargs\n ):\n return self._mclip_model(\n input_ids=input_ids, attention_mask=attention_mask, **kwargs\n )\n\n def encode_image(self, pixel_values: torch.Tensor):\n return self._model.encode_image(pixel_values)\n" }, { "path": "server/clip_server/model/model.py", "content": "\"\"\" CLIP Model\n\nAdapted from https://github.com/mlfoundations/open_clip.\n\nOriginally MIT License, Copyright (c) 2012-2021 Gabriel Ilharco, Mitchell Wortsman,\nNicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar,\nJohn Miller, Hongseok Namkoong, Hannaneh Hajishirzi, Ali Farhadi,\nLudwig Schmidt\n\"\"\"\n\nimport warnings\nimport torch\nimport numpy as np\nfrom torch import nn\nfrom dataclasses import dataclass\nfrom typing import Tuple, Union, Optional\nfrom copy import deepcopy\nfrom clip_server.helper import __cast_dtype__\nfrom open_clip.transformer import QuickGELU, LayerNorm, LayerNormFp32, Attention\nfrom open_clip.timm_model import TimmModel\nfrom open_clip.factory import _MODEL_CONFIGS\nfrom open_clip.hf_model import HFTextEncoder\nfrom open_clip.transformer import ResidualAttentionBlock as _ResidualAttentionBlock\nfrom open_clip.transformer import Transformer as _Transformer\nfrom open_clip.transformer import VisionTransformer as _VisionTransformer\nfrom open_clip.transformer import TextTransformer as _TextTransformer\nfrom open_clip.modified_resnet import ModifiedResNet as _ModifiedResNet\nfrom open_clip.model import CustomTextCLIP as _CustomTextCLIP\nfrom open_clip.model import CLIP as _CLIP\n\n# Use flash attention\ntry:\n from clip_server.model.flash_attention import MultiheadAttention\n\n FLASH_ATTENTION_AVAILABLE = True\nexcept:\n FLASH_ATTENTION_AVAILABLE = False\n\n\nclass ModifiedResNet(_ModifiedResNet):\n def forward(self, x):\n # To handle fp16 inference\n x = x.type(self.conv1.weight.dtype)\n return super().forward(x)\n\n\nclass ResidualAttentionBlock(_ResidualAttentionBlock):\n def __init__(\n self, width: int, heads: int, dtype: torch.dtype = torch.float32, **kwargs\n ):\n super().__init__(width, heads, **kwargs)\n head_dim = width // heads\n flash_attention = head_dim % 8 == 0 and head_dim <= 128\n\n self.attn = (\n MultiheadAttention(width, heads)\n if FLASH_ATTENTION_AVAILABLE\n and torch.cuda.is_available()\n and dtype in (torch.float16, torch.bfloat16)\n and flash_attention\n else nn.MultiheadAttention(width, heads)\n )\n\n\nclass Transformer(_Transformer):\n def __init__(self, layers: int, dtype: torch.dtype = torch.float32, **kwargs):\n super().__init__(layers=layers, **kwargs)\n self.resblocks = nn.ModuleList(\n [ResidualAttentionBlock(dtype=dtype, **kwargs) for _ in range(layers)]\n )\n\n\nclass VisionTransformer(_VisionTransformer):\n def __init__(\n self,\n image_size: int,\n patch_size: int,\n global_average_pool: bool,\n output_dim: int,\n dtype: torch.dtype = torch.float32,\n **kwargs,\n ):\n super().__init__(\n image_size,\n patch_size,\n global_average_pool=global_average_pool,\n output_dim=output_dim,\n **kwargs,\n )\n self.transformer = Transformer(dtype=dtype, **kwargs)\n\n def forward(self, x: torch.Tensor):\n dtype = self.transformer.get_cast_dtype()\n x = x.to(dtype)\n return super().forward(x)\n\n\nclass TextTransformer(_TextTransformer):\n def __init__(\n self,\n context_length: int,\n vocab_size: int,\n output_dim: int,\n dtype: torch.dtype = torch.float32,\n **kwargs,\n ):\n super().__init__(context_length, vocab_size, output_dim=output_dim, **kwargs)\n self.transformer = Transformer(dtype=dtype, **kwargs)\n self.init_parameters()\n\n\n@dataclass\nclass CLIPVisionCfg:\n layers: Union[Tuple[int, int, int, int], int] = 12\n width: int = 768\n head_width: int = 64\n mlp_ratio: float = 4.0\n patch_size: int = 16\n image_size: Union[Tuple[int, int], int] = 224\n ls_init_value: Optional[float] = None # layer scale initial value\n global_average_pool: bool = False # whether to global average pool the last embedding layer, instead of using CLS token (https://arxiv.org/abs/2205.01580)\n timm_model_name: str = (\n None # a valid model name overrides layers, width, patch_size\n )\n timm_model_pretrained: bool = (\n False # use (imagenet) pretrained weights for named model\n )\n timm_pool: str = (\n 'avg' # feature pooling for timm model ('abs_attn', 'rot_attn', 'avg', '')\n )\n timm_proj: str = (\n 'linear' # linear projection for timm model output ('linear', 'mlp', '')\n )\n timm_proj_bias: bool = False # enable bias final projection\n\n\n@dataclass\nclass CLIPTextCfg:\n context_length: int = 77\n vocab_size: int = 49408\n width: int = 512\n heads: int = 8\n layers: int = 12\n ls_init_value: Optional[float] = None # layer scale initial value\n hf_model_name: str = None\n hf_tokenizer_name: str = None\n hf_model_pretrained: bool = True\n proj: str = 'mlp'\n pooler_type: str = 'mean_pooler'\n\n\ndef _build_vision_tower(\n embed_dim: int,\n vision_cfg: CLIPVisionCfg,\n quick_gelu: bool = False,\n dtype: Optional[torch.dtype] = torch.float32,\n):\n if isinstance(vision_cfg, dict):\n vision_cfg = CLIPVisionCfg(**vision_cfg)\n\n # OpenAI models are pretrained w/ QuickGELU but native nn.GELU is both faster and more\n # memory efficient in recent PyTorch releases (>= 1.10).\n # NOTE: timm models always use native GELU regardless of quick_gelu flag.\n act_layer = QuickGELU if quick_gelu else nn.GELU\n\n if vision_cfg.timm_model_name:\n visual = TimmModel(\n model_name=vision_cfg.timm_model_name,\n pretrained=vision_cfg.timm_model_pretrained,\n pool=vision_cfg.timm_pool,\n proj=vision_cfg.timm_proj,\n proj_bias=vision_cfg.timm_proj_bias,\n embed_dim=embed_dim,\n image_size=vision_cfg.image_size,\n )\n act_layer = (\n nn.GELU\n ) # so that text transformer doesn't use QuickGELU w/ timm models\n elif isinstance(vision_cfg.layers, (tuple, list)):\n vision_heads = vision_cfg.width * 32 // vision_cfg.head_width\n visual = ModifiedResNet(\n layers=vision_cfg.layers,\n output_dim=embed_dim,\n heads=vision_heads,\n image_size=vision_cfg.image_size,\n width=vision_cfg.width,\n )\n else:\n vision_heads = vision_cfg.width // vision_cfg.head_width\n norm_layer = (\n LayerNormFp32 if dtype in (torch.float16, torch.bfloat16) else LayerNorm\n )\n visual = VisionTransformer(\n image_size=vision_cfg.image_size,\n patch_size=vision_cfg.patch_size,\n width=vision_cfg.width,\n layers=vision_cfg.layers,\n heads=vision_heads,\n mlp_ratio=vision_cfg.mlp_ratio,\n ls_init_value=vision_cfg.ls_init_value,\n global_average_pool=vision_cfg.global_average_pool,\n output_dim=embed_dim,\n act_layer=act_layer,\n norm_layer=norm_layer,\n dtype=dtype,\n )\n\n return visual\n\n\ndef _build_text_tower(\n embed_dim: int,\n text_cfg: CLIPTextCfg,\n quick_gelu: bool = False,\n dtype: Optional[torch.dtype] = torch.float32,\n):\n if isinstance(text_cfg, dict):\n text_cfg = CLIPTextCfg(**text_cfg)\n\n if text_cfg.hf_model_name:\n text = HFTextEncoder(\n text_cfg.hf_model_name,\n output_dim=embed_dim,\n proj=text_cfg.proj,\n pooler_type=text_cfg.pooler_type,\n pretrained=text_cfg.hf_model_pretrained,\n )\n else:\n act_layer = QuickGELU if quick_gelu else nn.GELU\n norm_layer = (\n LayerNormFp32 if dtype in (torch.float16, torch.bfloat16) else LayerNorm\n )\n\n text = TextTransformer(\n context_length=text_cfg.context_length,\n vocab_size=text_cfg.vocab_size,\n width=text_cfg.width,\n heads=text_cfg.heads,\n layers=text_cfg.layers,\n ls_init_value=text_cfg.ls_init_value,\n output_dim=embed_dim,\n act_layer=act_layer,\n norm_layer=norm_layer,\n dtype=dtype,\n )\n return text\n\n\nclass CustomTextCLIP(_CustomTextCLIP):\n def __init__(\n self,\n embed_dim: int,\n vision_cfg: CLIPVisionCfg,\n text_cfg: CLIPTextCfg,\n quick_gelu: bool = False,\n dtype: Optional[torch.dtype] = torch.float32,\n ):\n super().__init__(embed_dim, vision_cfg, text_cfg, quick_gelu, dtype)\n self.visual = _build_vision_tower(\n embed_dim=embed_dim,\n vision_cfg=vision_cfg,\n quick_gelu=quick_gelu,\n dtype=dtype,\n )\n self.text = _build_text_tower(\n embed_dim=embed_dim, text_cfg=text_cfg, quick_gelu=quick_gelu, dtype=dtype\n )\n\n\nclass CLIP(_CLIP):\n def __init__(\n self,\n embed_dim: int,\n vision_cfg: CLIPVisionCfg,\n text_cfg: CLIPTextCfg,\n quick_gelu: bool = False,\n dtype: Optional[torch.dtype] = torch.float32,\n ):\n nn.Module.__init__(self)\n\n self.visual = _build_vision_tower(\n embed_dim=embed_dim,\n vision_cfg=vision_cfg,\n quick_gelu=quick_gelu,\n dtype=dtype,\n )\n text = _build_text_tower(\n embed_dim=embed_dim, text_cfg=text_cfg, quick_gelu=quick_gelu, dtype=dtype\n )\n self.transformer = text.transformer\n self.vocab_size = text.vocab_size\n self.token_embedding = text.token_embedding\n self.positional_embedding = text.positional_embedding\n self.ln_final = text.ln_final\n self.text_projection = text.text_projection\n self.register_buffer('attn_mask', text.attn_mask, persistent=False)\n self.logit_scale = nn.Parameter(torch.ones([]) * np.log(1 / 0.07))\n\n\ndef convert_weights_to_lp(model: nn.Module, dtype=torch.float16):\n \"\"\"Convert applicable model parameters to low-precision (bf16 or fp16)\"\"\"\n\n def _convert_weights(l):\n if isinstance(l, (nn.Conv1d, nn.Conv2d, nn.Linear)):\n l.weight.data = l.weight.data.to(dtype)\n if l.bias is not None:\n l.bias.data = l.bias.data.to(dtype)\n\n if isinstance(l, (nn.MultiheadAttention, Attention)):\n for attr in [\n *[f\"{s}_proj_weight\" for s in [\"in\", \"q\", \"k\", \"v\"]],\n \"in_proj_bias\",\n \"bias_k\",\n \"bias_v\",\n ]:\n tensor = getattr(l, attr)\n if tensor is not None:\n tensor.data = tensor.data.to(dtype)\n\n for name in [\"text_projection\", \"proj\"]:\n if hasattr(l, name):\n attr = getattr(l, name)\n if attr is not None:\n attr.data = attr.data.to(dtype)\n\n model.apply(_convert_weights)\n\n\nconvert_weights_to_fp16 = convert_weights_to_lp # backwards compat\n\n\ndef load_state_dict(checkpoint_path: str, map_location='cpu'):\n checkpoint = torch.load(checkpoint_path, map_location=map_location)\n if isinstance(checkpoint, dict) and 'state_dict' in checkpoint:\n state_dict = checkpoint['state_dict']\n else:\n state_dict = checkpoint\n if next(iter(state_dict.items()))[0].startswith('module'):\n state_dict = {k[7:]: v for k, v in state_dict.items()}\n return state_dict\n\n\ndef build_model_from_openai_state_dict(\n state_dict: dict,\n quick_gelu: bool = False,\n dtype: torch.dtype = torch.float16,\n):\n vit = \"visual.proj\" in state_dict\n\n if vit:\n vision_width = state_dict[\"visual.conv1.weight\"].shape[0]\n vision_layers = len(\n [\n k\n for k in state_dict.keys()\n if k.startswith(\"visual.\") and k.endswith(\".attn.in_proj_weight\")\n ]\n )\n vision_patch_size = state_dict[\"visual.conv1.weight\"].shape[-1]\n grid_size = round(\n (state_dict[\"visual.positional_embedding\"].shape[0] - 1) ** 0.5\n )\n image_size = vision_patch_size * grid_size\n else:\n counts: list = [\n len(\n set(\n k.split(\".\")[2]\n for k in state_dict\n if k.startswith(f\"visual.layer{b}\")\n )\n )\n for b in [1, 2, 3, 4]\n ]\n vision_layers = tuple(counts)\n vision_width = state_dict[\"visual.layer1.0.conv1.weight\"].shape[0]\n output_width = round(\n (state_dict[\"visual.attnpool.positional_embedding\"].shape[0] - 1) ** 0.5\n )\n vision_patch_size = None\n assert (\n output_width**2 + 1\n == state_dict[\"visual.attnpool.positional_embedding\"].shape[0]\n )\n image_size = output_width * 32\n\n embed_dim = state_dict[\"text_projection\"].shape[1]\n context_length = state_dict[\"positional_embedding\"].shape[0]\n vocab_size = state_dict[\"token_embedding.weight\"].shape[0]\n transformer_width = state_dict[\"ln_final.weight\"].shape[0]\n transformer_heads = transformer_width // 64\n transformer_layers = len(\n set(\n k.split(\".\")[2]\n for k in state_dict\n if k.startswith(f\"transformer.resblocks\")\n )\n )\n\n vision_cfg = CLIPVisionCfg(\n layers=vision_layers,\n width=vision_width,\n patch_size=vision_patch_size,\n image_size=image_size,\n )\n text_cfg = CLIPTextCfg(\n context_length=context_length,\n vocab_size=vocab_size,\n width=transformer_width,\n heads=transformer_heads,\n layers=transformer_layers,\n )\n model = CLIP(\n embed_dim=embed_dim,\n vision_cfg=vision_cfg,\n text_cfg=text_cfg,\n quick_gelu=quick_gelu, # OpenAI models were trained with QuickGELU\n dtype=dtype,\n )\n\n for key in [\"input_resolution\", \"context_length\", \"vocab_size\"]:\n state_dict.pop(key, None)\n\n convert_weights_to_fp16(model)\n model.load_state_dict(state_dict)\n\n return model.eval()\n\n\ndef load_openai_model(\n model_path: str,\n device: Union[str, torch.device] = 'cuda' if torch.cuda.is_available() else 'cpu',\n dtype: Optional[Union[str, torch.dtype]] = None,\n jit: bool = True,\n):\n \"\"\"Load a CLIP model\n\n Parameters\n ----------\n model_path : str\n The path to a model checkpoint containing the state_dict\n dtype: str\n Model precision, if None defaults to 'fp32' if device == 'cpu' else 'fp16'.\n device : Union[str, torch.device]\n The device to put the loaded model\n jit : bool\n Whether to load the optimized JIT model (default) or more hackable non-JIT model.\n Returns\n -------\n model : torch.nn.Module\n The CLIP model\n preprocess : Callable[[PIL.Image], torch.Tensor]\n A torchvision transform that converts a PIL image into a tensor that the returned model can take as its input\n \"\"\"\n if isinstance(dtype, str):\n dtype = __cast_dtype__.get(dtype, 'amp')\n elif dtype is None:\n dtype = (\n torch.float32 if device in ('cpu', torch.device('cpu')) else torch.float16\n )\n try:\n # loading JIT archive\n model = torch.jit.load(model_path, map_location=device if jit else \"cpu\").eval()\n state_dict = None\n except RuntimeError:\n # loading saved state dict\n if jit:\n warnings.warn(\n f\"File {model_path} is not a JIT archive. Loading as a state dict instead\"\n )\n jit = False\n state_dict = torch.load(model_path, map_location=\"cpu\")\n\n if not jit:\n # Build a non-jit model from the OpenAI jitted model state dict\n try:\n model = build_model_from_openai_state_dict(\n state_dict or model.state_dict(), dtype=dtype\n )\n except KeyError:\n sd = {k[7:]: v for k, v in state_dict[\"state_dict\"].items()}\n model = build_model_from_openai_state_dict(sd, dtype=dtype)\n\n # model from OpenAI state dict is in manually cast fp16 mode, must be converted for AMP/fp32/bf16 use\n model = model.to(device)\n if dtype == torch.float32 or (\n isinstance(dtype, str) and dtype.startswith('amp')\n ):\n model.float()\n elif dtype == torch.bfloat16:\n convert_weights_to_lp(model, dtype=torch.bfloat16)\n\n return model\n\n # patch the device names\n device_holder = torch.jit.trace(\n lambda: torch.ones([]).to(torch.device(device)), example_inputs=[]\n )\n device_node = [\n n\n for n in device_holder.graph.findAllNodes(\"prim::Constant\")\n if \"Device\" in repr(n)\n ][-1]\n\n def patch_device(module):\n try:\n graphs = [module.graph] if hasattr(module, \"graph\") else []\n except RuntimeError:\n graphs = []\n\n if hasattr(module, \"forward1\"):\n graphs.append(module.forward1.graph)\n\n for graph in graphs:\n for node in graph.findAllNodes(\"prim::Constant\"):\n if \"value\" in node.attributeNames() and str(node[\"value\"]).startswith(\n \"cuda\"\n ):\n node.copyAttributes(device_node)\n\n model.apply(patch_device)\n patch_device(model.encode_image)\n patch_device(model.encode_text)\n\n # patch dtype to float32 (typically for CPU)\n if dtype == torch.float32:\n float_holder = torch.jit.trace(\n lambda: torch.ones([]).float(), example_inputs=[]\n )\n float_input = list(float_holder.graph.findNode(\"aten::to\").inputs())[1]\n float_node = float_input.node()\n\n def patch_float(module):\n try:\n graphs = [module.graph] if hasattr(module, \"graph\") else []\n except RuntimeError:\n graphs = []\n\n if hasattr(module, \"forward1\"):\n graphs.append(module.forward1.graph)\n\n for graph in graphs:\n for node in graph.findAllNodes(\"aten::to\"):\n inputs = list(node.inputs())\n for i in [\n 1,\n 2,\n ]: # dtype can be the second or third argument to aten::to()\n if inputs[i].node()[\"value\"] == 5:\n inputs[i].node().copyAttributes(float_node)\n\n model.apply(patch_float)\n patch_float(model.encode_image)\n patch_float(model.encode_text)\n model.float()\n\n # ensure image_size attr available at consistent location for both jit and non-jit\n model.visual.image_size = model.input_resolution.item()\n return model\n\n\ndef load_openclip_model(\n model_name: str,\n model_path: str,\n device: Union[str, torch.device] = 'cpu',\n jit: bool = False,\n force_quick_gelu: bool = False,\n force_custom_text: bool = False,\n pretrained_image: bool = False,\n dtype: Optional[Union[str, torch.dtype]] = None,\n):\n if isinstance(dtype, str):\n dtype = __cast_dtype__.get(dtype)\n elif dtype is None:\n dtype = (\n torch.float32 if device in ('cpu', torch.device('cpu')) else torch.float16\n )\n\n model_name = model_name.replace(\n '/', '-'\n ) # for callers using old naming with / in ViT names\n\n if model_name in _MODEL_CONFIGS:\n model_cfg = deepcopy(_MODEL_CONFIGS[model_name])\n else:\n raise RuntimeError(f'Model config for {model_name} not found.')\n\n if force_quick_gelu:\n # override for use of QuickGELU on non-OpenAI transformer models\n model_cfg[\"quick_gelu\"] = True\n\n if pretrained_image:\n if 'timm_model_name' in model_cfg.get('vision_cfg', {}):\n # pretrained weight loading for timm models set via vision_cfg\n model_cfg['vision_cfg']['timm_model_pretrained'] = True\n else:\n assert (\n False\n ), 'pretrained image towers currently only supported for timm models'\n\n custom_text = (\n model_cfg.pop('custom_text', False)\n or force_custom_text\n or ('hf_model_name' in model_cfg['text_cfg'])\n )\n\n if custom_text:\n model = CustomTextCLIP(**model_cfg, dtype=dtype)\n else:\n model = CLIP(**model_cfg, dtype=dtype)\n\n model.eval()\n model.load_state_dict(load_state_dict(model_path))\n model.to(device=device)\n\n if dtype in (torch.float16, torch.bfloat16):\n convert_weights_to_lp(model, dtype=dtype)\n\n if jit:\n model = torch.jit.script(model)\n\n return model\n" }, { "path": "server/clip_server/model/openclip_model.py", "content": "# Originally from https://github.com/mlfoundations/open_clip.\n#\n# Copyright (c) 2012-2021 Gabriel Ilharco, Mitchell Wortsman,\n# Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar,\n# John Miller, Hongseok Namkoong, Hannaneh Hajishirzi, Ali Farhadi,\n# Ludwig Schmidt\n\nfrom clip_server.model.clip_model import CLIPModel\nfrom clip_server.model.pretrained_models import get_model_url_md5, download_model\nfrom clip_server.model.model import load_openai_model, load_openclip_model\n\nimport torch\n\n\nclass OpenCLIPModel(CLIPModel):\n def __init__(\n self,\n name: str,\n device: str = 'cpu',\n jit: bool = False,\n dtype: str = None,\n **kwargs\n ):\n super().__init__(name, **kwargs)\n\n if '::' in name:\n model_name, pretrained = name.split('::')\n else:\n model_name = name\n pretrained = 'openai'\n\n self._model_name = model_name\n\n model_url, md5sum = get_model_url_md5(name)\n model_path = download_model(model_url, md5sum=md5sum)\n\n if pretrained == 'openai':\n self._model = load_openai_model(\n model_path=model_path, device=device, jit=jit, dtype=dtype\n )\n else:\n self._model = load_openclip_model(\n model_name=self._model_name,\n model_path=model_path,\n device=device,\n jit=jit,\n dtype=dtype,\n )\n\n @staticmethod\n def get_model_name(name: str):\n if '::' in name:\n model_name, pretrained = name.split('::')\n else:\n model_name = name\n if model_name == 'ViT-L/14@336px':\n return 'ViT-L-14-336'\n return model_name.replace('/', '-')\n\n def encode_text(self, input_ids: 'torch.Tensor', **kwargs):\n return self._model.encode_text(input_ids)\n\n def encode_image(self, pixel_values: 'torch.Tensor', **kwargs):\n return self._model.encode_image(pixel_values)\n" }, { "path": "server/clip_server/model/pretrained_models.py", "content": "import os\nimport hashlib\nimport shutil\nimport urllib\n\n\n_OPENCLIP_S3_BUCKET = 'https://clip-as-service.s3.us-east-2.amazonaws.com/models/torch'\n_OPENCLIP_MODELS = {\n 'RN50::openai': ('RN50.pt', '9140964eaaf9f68c95aa8df6ca13777c'),\n 'RN50::yfcc15m': ('RN50-yfcc15m.pt', 'e9c564f91ae7dc754d9043fdcd2a9f22'),\n 'RN50::cc12m': ('RN50-cc12m.pt', '37cb01eb52bb6efe7666b1ff2d7311b5'),\n 'RN101::openai': ('RN101.pt', 'fa9d5f64ebf152bc56a18db245071014'),\n 'RN101::yfcc15m': ('RN101-yfcc15m.pt', '48f7448879ce25e355804f6bb7928cb8'),\n 'RN50x4::openai': ('RN50x4.pt', '03830990bc768e82f7fb684cde7e5654'),\n 'RN50x16::openai': ('RN50x16.pt', '83d63878a818c65d0fb417e5fab1e8fe'),\n 'RN50x64::openai': ('RN50x64.pt', 'a6631a0de003c4075d286140fc6dd637'),\n 'ViT-B-32::openai': ('ViT-B-32.pt', '3ba34e387b24dfe590eeb1ae6a8a122b'),\n 'ViT-B-32::laion2b_e16': (\n 'ViT-B-32-laion2b_e16.pt',\n 'df08de3d9f2dc53c71ea26e184633902',\n ),\n 'ViT-B-32::laion400m_e31': (\n 'ViT-B-32-laion400m_e31.pt',\n 'ca8015f98ab0f8780510710681d7b73e',\n ),\n 'ViT-B-32::laion400m_e32': (\n 'ViT-B-32-laion400m_e32.pt',\n '359e0dba4a419f175599ee0c63a110d8',\n ),\n 'ViT-B-32::laion2b-s34b-b79k': (\n 'ViT-B-32-laion2b-s34b-b79k.bin',\n '2fc036aea9cd7306f5ce7ce6abb8d0bf',\n ),\n 'ViT-B-16::openai': ('ViT-B-16.pt', '44c3d804ecac03d9545ac1a3adbca3a6'),\n 'ViT-B-16::laion400m_e31': (\n 'ViT-B-16-laion400m_e31.pt',\n '31306a44224cc46fec1bc3b82fd0c4e6',\n ),\n 'ViT-B-16::laion400m_e32': (\n 'ViT-B-16-laion400m_e32.pt',\n '07283adc5c17899f2ed22d82b563c54b',\n ),\n 'ViT-B-16-plus-240::laion400m_e31': (\n 'ViT-B-16-plus-240-laion400m_e31.pt',\n 'c88f453644a998ecb094d878a2f0738d',\n ),\n 'ViT-B-16-plus-240::laion400m_e32': (\n 'ViT-B-16-plus-240-laion400m_e32.pt',\n 'e573af3cef888441241e35022f30cc95',\n ),\n 'ViT-L-14::openai': ('ViT-L-14.pt', '096db1af569b284eb76b3881534822d9'),\n 'ViT-L-14::laion400m_e31': (\n 'ViT-L-14-laion400m_e31.pt',\n '09d223a6d41d2c5c201a9da618d833aa',\n ),\n 'ViT-L-14::laion400m_e32': (\n 'ViT-L-14-laion400m_e32.pt',\n 'a76cde1bc744ca38c6036b920c847a89',\n ),\n 'ViT-L-14::laion2b-s32b-b82k': (\n 'ViT-L-14-laion2b-s32b-b82k.bin',\n '4d2275fc7b2d7ee9db174f9b57ddecbd',\n ),\n 'ViT-L-14-336::openai': ('ViT-L-14-336px.pt', 'b311058cae50cb10fbfa2a44231c9473'),\n 'ViT-H-14::laion2b-s32b-b79k': (\n 'ViT-H-14-laion2b-s32b-b79k.bin',\n '2aa6c46521b165a0daeb8cdc6668c7d3',\n ),\n 'ViT-g-14::laion2b-s12b-b42k': (\n 'ViT-g-14-laion2b-s12b-b42k.bin',\n '3bf99353f6f1829faac0bb155be4382a',\n ),\n 'roberta-ViT-B-32::laion2b-s12b-b32k': (\n 'roberta-ViT-B-32-laion2b-s12b-b32k.bin',\n '76d4c9d13774cc15fa0e2b1b94a8402c',\n ),\n 'xlm-roberta-base-ViT-B-32::laion5b-s13b-b90k': (\n 'xlm-roberta-base-ViT-B-32-laion5b-s13b-b90k.bin',\n 'f68abc07ef349720f1f880180803142d',\n ),\n 'xlm-roberta-large-ViT-H-14::frozen_laion5b_s13b_b90k': (\n 'xlm-roberta-large-ViT-H-14-frozen_laion5b_s13b_b90k.bin',\n 'b49991239a419d704fdba59c42d5536d',\n ),\n # older version name format\n 'RN50': ('RN50.pt', '9140964eaaf9f68c95aa8df6ca13777c'),\n 'RN101': ('RN101.pt', 'fa9d5f64ebf152bc56a18db245071014'),\n 'RN50x4': ('RN50x4.pt', '03830990bc768e82f7fb684cde7e5654'),\n 'RN50x16': ('RN50x16.pt', '83d63878a818c65d0fb417e5fab1e8fe'),\n 'RN50x64': ('RN50x64.pt', 'a6631a0de003c4075d286140fc6dd637'),\n 'ViT-B/32': ('ViT-B-32.pt', '3ba34e387b24dfe590eeb1ae6a8a122b'),\n 'ViT-B/16': ('ViT-B-16.pt', '44c3d804ecac03d9545ac1a3adbca3a6'),\n 'ViT-L/14': ('ViT-L-14.pt', '096db1af569b284eb76b3881534822d9'),\n 'ViT-L/14@336px': ('ViT-L-14-336px.pt', 'b311058cae50cb10fbfa2a44231c9473'),\n}\n\n_MULTILINGUALCLIP_MODELS = {\n 'M-CLIP/XLM-Roberta-Large-Vit-B-32': (),\n 'M-CLIP/XLM-Roberta-Large-Vit-L-14': (),\n 'M-CLIP/XLM-Roberta-Large-Vit-B-16Plus': (),\n 'M-CLIP/LABSE-Vit-L-14': (),\n}\n\n_CNCLIP_MODELS = {\n 'CN-CLIP/ViT-B-16': (),\n 'CN-CLIP/ViT-L-14': (),\n 'CN-CLIP/ViT-L-14-336': (),\n 'CN-CLIP/ViT-H-14': (),\n 'CN-CLIP/RN50': (),\n}\n\n_VISUAL_MODEL_IMAGE_SIZE = {\n 'RN50': 224,\n 'RN101': 224,\n 'RN50x4': 288,\n 'RN50x16': 384,\n 'RN50x64': 448,\n 'ViT-B-32': 224,\n 'roberta-ViT-B-32': 224,\n 'xlm-roberta-base-ViT-B-32': 224,\n 'ViT-B-16': 224,\n 'Vit-B-16Plus': 240,\n 'ViT-B-16-plus-240': 240,\n 'ViT-L-14': 224,\n 'ViT-L-14-336': 336,\n 'ViT-H-14': 224,\n 'xlm-roberta-large-ViT-H-14': 224,\n 'ViT-g-14': 224,\n}\n\n\ndef md5file(filename: str):\n hash_md5 = hashlib.md5()\n with open(filename, 'rb') as f:\n for chunk in iter(lambda: f.read(4096), b\"\"):\n hash_md5.update(chunk)\n\n return hash_md5.hexdigest()\n\n\ndef get_model_url_md5(name: str):\n model_pretrained = _OPENCLIP_MODELS[name]\n if len(model_pretrained) == 0: # not on s3\n return None, None\n else:\n return (_OPENCLIP_S3_BUCKET + '/' + model_pretrained[0], model_pretrained[1])\n\n\ndef download_model(\n url: str,\n target_folder: str = os.path.expanduser(\"~/.cache/clip\"),\n md5sum: str = None,\n with_resume: bool = True,\n max_attempts: int = 3,\n) -> str:\n os.makedirs(target_folder, exist_ok=True)\n filename = os.path.basename(url)\n\n download_target = os.path.join(target_folder, filename)\n\n if os.path.exists(download_target):\n if not os.path.isfile(download_target):\n raise FileExistsError(f'{download_target} exists and is not a regular file')\n\n actual_md5sum = md5file(download_target)\n if (not md5sum) or actual_md5sum == md5sum:\n return download_target\n\n from rich.progress import (\n DownloadColumn,\n Progress,\n TextColumn,\n TimeRemainingColumn,\n TransferSpeedColumn,\n )\n\n progress = Progress(\n \" \\n\", # divide this bar from Flow's bar\n TextColumn(\"[bold blue]{task.fields[filename]}\", justify=\"right\"),\n \"[progress.percentage]{task.percentage:>3.1f}%\",\n \"โ€ข\",\n DownloadColumn(),\n \"โ€ข\",\n TransferSpeedColumn(),\n \"โ€ข\",\n TimeRemainingColumn(),\n )\n\n with progress:\n task = progress.add_task('download', filename=filename, start=False)\n\n for _ in range(max_attempts):\n tmp_file_path = download_target + '.part'\n resume_byte_pos = (\n os.path.getsize(tmp_file_path) if os.path.exists(tmp_file_path) else 0\n )\n\n try:\n # resolve the 403 error by passing a valid user-agent\n req = urllib.request.Request(url, headers={'User-Agent': 'Mozilla/5.0'})\n total_bytes = int(\n urllib.request.urlopen(req).info().get('Content-Length', -1)\n )\n mode = 'ab' if (with_resume and resume_byte_pos) else 'wb'\n\n with open(tmp_file_path, mode) as output:\n progress.update(task, total=total_bytes)\n progress.start_task(task)\n\n if resume_byte_pos and with_resume:\n progress.update(task, advance=resume_byte_pos)\n req.headers['Range'] = f'bytes={resume_byte_pos}-'\n\n with urllib.request.urlopen(req) as source:\n while True:\n buffer = source.read(8192)\n if not buffer:\n break\n\n output.write(buffer)\n progress.update(task, advance=len(buffer))\n\n actual_md5 = md5file(tmp_file_path)\n if (md5sum and actual_md5 == md5sum) or (not md5sum):\n shutil.move(tmp_file_path, download_target)\n return download_target\n else:\n os.remove(tmp_file_path)\n raise RuntimeError(\n f'MD5 mismatch: expected {md5sum}, got {actual_md5}'\n )\n\n except Exception as ex:\n progress.console.print(\n f'Failed to download {url} with {ex!r} at the {_}th attempt'\n )\n progress.reset(task)\n\n raise RuntimeError(\n f'Failed to download {url} within retry limit {max_attempts}'\n )\n" }, { "path": "server/clip_server/model/simple_tokenizer.py", "content": "# Originally from https://github.com/openai/CLIP. MIT License, Copyright (c) 2021 OpenAI\n\nimport gzip\nimport html\nimport os\nimport regex as re\nfrom functools import lru_cache\n\nimport ftfy\n\nfrom clip_server.helper import __resources_path__\n\n\n@lru_cache()\ndef default_bpe():\n return os.path.join(__resources_path__, 'bpe_simple_vocab_16e6.txt.gz')\n\n\n@lru_cache()\ndef bytes_to_unicode():\n \"\"\"\n Returns list of utf-8 byte and a corresponding list of unicode strings.\n The reversible bpe codes work on unicode strings.\n This means you need a large # of unicode characters in your vocab if you want to avoid UNKs.\n When you're at something like a 10B token dataset you end up needing around 5K for decent coverage.\n This is a signficant percentage of your normal, say, 32K bpe vocab.\n To avoid that, we want lookup tables between utf-8 bytes and unicode strings.\n And avoids mapping to whitespace/control characters the bpe code barfs on.\n \"\"\"\n bs = (\n list(range(ord(\"!\"), ord(\"~\") + 1))\n + list(range(ord(\"ยก\"), ord(\"ยฌ\") + 1))\n + list(range(ord(\"ยฎ\"), ord(\"รฟ\") + 1))\n )\n cs = bs[:]\n n = 0\n for b in range(2**8):\n if b not in bs:\n bs.append(b)\n cs.append(2**8 + n)\n n += 1\n cs = [chr(n) for n in cs]\n return dict(zip(bs, cs))\n\n\ndef get_pairs(word):\n \"\"\"Return set of symbol pairs in a word.\n Word is represented as tuple of symbols (symbols being variable-length strings).\n \"\"\"\n pairs = set()\n prev_char = word[0]\n for char in word[1:]:\n pairs.add((prev_char, char))\n prev_char = char\n return pairs\n\n\ndef basic_clean(text):\n text = ftfy.fix_text(text)\n text = html.unescape(html.unescape(text))\n return text.strip()\n\n\ndef whitespace_clean(text):\n text = re.sub(r'\\s+', ' ', text)\n text = text.strip()\n return text\n\n\nclass SimpleTokenizer(object):\n def __init__(self, bpe_path: str = default_bpe()):\n self.byte_encoder = bytes_to_unicode()\n self.byte_decoder = {v: k for k, v in self.byte_encoder.items()}\n merges = gzip.open(bpe_path).read().decode(\"utf-8\").split('\\n')\n merges = merges[1 : 49152 - 256 - 2 + 1]\n merges = [tuple(merge.split()) for merge in merges]\n vocab = list(bytes_to_unicode().values())\n vocab = vocab + [v + '' for v in vocab]\n for merge in merges:\n vocab.append(''.join(merge))\n vocab.extend(['<|startoftext|>', '<|endoftext|>'])\n self.encoder = dict(zip(vocab, range(len(vocab))))\n self.decoder = {v: k for k, v in self.encoder.items()}\n self.bpe_ranks = dict(zip(merges, range(len(merges))))\n self.cache = {\n '<|startoftext|>': '<|startoftext|>',\n '<|endoftext|>': '<|endoftext|>',\n }\n self.pat = re.compile(\n r\"\"\"<\\|startoftext\\|>|<\\|endoftext\\|>|'s|'t|'re|'ve|'m|'ll|'d|[\\p{L}]+|[\\p{N}]|[^\\s\\p{L}\\p{N}]+\"\"\",\n re.IGNORECASE,\n )\n\n def bpe(self, token):\n if token in self.cache:\n return self.cache[token]\n word = tuple(token[:-1]) + (token[-1] + '',)\n pairs = get_pairs(word)\n\n if not pairs:\n return token + ''\n\n while True:\n bigram = min(pairs, key=lambda pair: self.bpe_ranks.get(pair, float('inf')))\n if bigram not in self.bpe_ranks:\n break\n first, second = bigram\n new_word = []\n i = 0\n while i < len(word):\n try:\n j = word.index(first, i)\n new_word.extend(word[i:j])\n i = j\n except:\n new_word.extend(word[i:])\n break\n\n if word[i] == first and i < len(word) - 1 and word[i + 1] == second:\n new_word.append(first + second)\n i += 2\n else:\n new_word.append(word[i])\n i += 1\n new_word = tuple(new_word)\n word = new_word\n if len(word) == 1:\n break\n else:\n pairs = get_pairs(word)\n word = ' '.join(word)\n self.cache[token] = word\n return word\n\n def encode(self, text):\n bpe_tokens = []\n text = whitespace_clean(basic_clean(text)).lower()\n for token in re.findall(self.pat, text):\n token = ''.join(self.byte_encoder[b] for b in token.encode('utf-8'))\n bpe_tokens.extend(\n self.encoder[bpe_token] for bpe_token in self.bpe(token).split(' ')\n )\n return bpe_tokens\n\n def decode(self, tokens):\n text = ''.join([self.decoder[token] for token in tokens])\n text = (\n bytearray([self.byte_decoder[c] for c in text])\n .decode('utf-8', errors='replace')\n .replace('', ' ')\n )\n return text\n" }, { "path": "server/clip_server/model/tokenization.py", "content": "import torch\nfrom typing import List, Union\nfrom clip_server.model.pretrained_models import (\n _MULTILINGUALCLIP_MODELS,\n _CNCLIP_MODELS,\n)\n\n\nclass Tokenizer:\n def __init__(self, name: str, **kwargs):\n self._name = name\n if name in _MULTILINGUALCLIP_MODELS:\n import transformers\n\n self._tokenizer = transformers.AutoTokenizer.from_pretrained(name)\n elif name in _CNCLIP_MODELS:\n import cn_clip.clip as cnclip\n\n self._tokenizer = cnclip\n else:\n from clip_server.model.simple_tokenizer import SimpleTokenizer\n\n self._tokenizer = SimpleTokenizer()\n\n def __call__(\n self,\n texts: Union[str, List[str]],\n context_length: int = 77,\n truncate: bool = True,\n ):\n \"\"\"\n :param texts: An input string or a list of input strings to tokenize\n :param context_length: The context length to use; all English CLIP models use 77 as the context length.\n for Chinese CLIP models, context_length = 52, if the number of characters is bigger than 50, sentence will be truncate and omit the part left\n :param truncate: Whether to truncate the text in case its encoding is longer than the context length.\n\n :return: A dict of tokenized representations of the input strings and their corresponding attention masks with both\n shape = [batch size, context_length]\n \"\"\"\n if self._name in _CNCLIP_MODELS:\n return self._tokenize(texts, context_length=52)\n else:\n return self._tokenize(\n texts, context_length=context_length, truncate=truncate\n )\n\n def _tokenize(\n self,\n texts: Union[str, List[str]],\n context_length: int = 77,\n truncate: bool = True,\n ) -> dict:\n if isinstance(texts, str):\n texts = [texts]\n if self._name in _MULTILINGUALCLIP_MODELS:\n result = self._tokenizer(\n texts,\n max_length=context_length,\n return_attention_mask=True,\n return_tensors='pt',\n padding=True,\n truncation=True,\n )\n return {\n 'input_ids': result['input_ids'],\n 'attention_mask': result['attention_mask'],\n }\n elif self._name in _CNCLIP_MODELS:\n result = self._tokenizer.tokenize(\n texts=texts,\n context_length=52, # in all cnclip baseline model context length is 52\n )\n attn_mask = result.clone()\n attn_mask[result != 0] = 1\n return {\n \"input_ids\": result,\n \"attention_mask\": attn_mask,\n }\n else:\n sot_token = self._tokenizer.encoder['<|startoftext|>']\n eot_token = self._tokenizer.encoder['<|endoftext|>']\n all_tokens = [\n [sot_token] + self._tokenizer.encode(text) + [eot_token]\n for text in texts\n ]\n\n input_ids = torch.zeros(len(all_tokens), context_length, dtype=torch.long)\n attention_mask = torch.zeros(\n len(all_tokens), context_length, dtype=torch.long\n )\n\n for i, tokens in enumerate(all_tokens):\n if len(tokens) > context_length:\n if truncate:\n tokens = tokens[:context_length]\n tokens[-1] = eot_token\n else:\n raise RuntimeError(\n f'Input {texts[i]} is too long for context length {context_length}'\n )\n input_ids[i, : len(tokens)] = torch.tensor(tokens)\n attention_mask[i, : len(tokens)] = 1\n\n return {'input_ids': input_ids, 'attention_mask': attention_mask}\n" }, { "path": "server/clip_server/model/trt_utils.py", "content": "# Originally from https://github.com/ELS-RD/transformer-deploy.\n# Apache License, Version 2.0, Copyright (c) 2022 Lefebvre Dalloz Services\n\nfrom typing import Callable, Dict, List, OrderedDict, Tuple\n\nimport tensorrt as trt\nimport torch\nfrom tensorrt import ICudaEngine, IExecutionContext\nfrom tensorrt.tensorrt import (\n Builder,\n IBuilderConfig,\n IElementWiseLayer,\n ILayer,\n INetworkDefinition,\n IOptimizationProfile,\n IReduceLayer,\n Logger,\n OnnxParser,\n Runtime,\n)\n\n\n\"\"\"\nAll the tooling to ease TensorRT usage.\n\"\"\"\n\n\ndef fix_fp16_network(network_definition: INetworkDefinition) -> INetworkDefinition:\n \"\"\"\n Mixed precision on TensorRT can generate scores very far from Pytorch because of some operator being saturated.\n Indeed, FP16 can't store very large and very small numbers like FP32.\n Here, we search for some patterns of operators to keep in FP32, in most cases, it is enough to fix the inference\n and don't hurt performances.\n :param network_definition: graph generated by TensorRT after parsing ONNX file (during the model building)\n :return: patched network definition\n \"\"\"\n # search for patterns which may overflow in FP16 precision, we force FP32 precisions for those nodes\n for layer_index in range(network_definition.num_layers - 1):\n layer: ILayer = network_definition.get_layer(layer_index)\n next_layer: ILayer = network_definition.get_layer(layer_index + 1)\n # POW operation usually followed by mean reduce\n if (\n layer.type == trt.LayerType.ELEMENTWISE\n and next_layer.type == trt.LayerType.REDUCE\n ):\n # casting to get access to op attribute\n layer.__class__ = IElementWiseLayer\n next_layer.__class__ = IReduceLayer\n if layer.op == trt.ElementWiseOperation.POW:\n layer.precision = trt.DataType.FLOAT\n next_layer.precision = trt.DataType.FLOAT\n layer.set_output_type(index=0, dtype=trt.DataType.FLOAT)\n next_layer.set_output_type(index=0, dtype=trt.DataType.FLOAT)\n return network_definition\n\n\ndef build_engine(\n runtime: Runtime,\n onnx_file_path: str,\n logger: Logger,\n min_shape: Tuple[int, int],\n optimal_shape: Tuple[int, int],\n max_shape: Tuple[int, int],\n workspace_size: int,\n fp16: bool,\n int8: bool,\n) -> ICudaEngine:\n \"\"\"\n Convert ONNX file to TensorRT engine.\n It supports dynamic shape, however it's advised to keep sequence length fix as it hurts performance otherwise.\n Dynamic batch size don't hurt performance and is highly advised.\n :param runtime: global variable shared accross inference call / model building\n :param onnx_file_path: path to the ONNX file\n :param logger: specific logger to TensorRT\n :param min_shape: the minimal shape of input tensors. It's advised to set first dimension (batch size) to 1\n :param optimal_shape: input tensor shape used for optimizations\n :param max_shape: maximal input tensor shape\n :param workspace_size: GPU memory to use during the building, more is always better. If there is not enough memory,\n some optimization may fail, and the whole conversion process will crash.\n :param fp16: enable FP16 precision, it usually provide a 20-30% boost compared to ONNX Runtime.\n :param int8: enable INT-8 quantization, best performance but model should have been quantized.\n :return: TensorRT engine to use during inference\n \"\"\"\n with trt.Builder(logger) as builder: # type: Builder\n with builder.create_network(\n flags=1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)\n ) as network_definition: # type: INetworkDefinition\n with trt.OnnxParser(\n network_definition, logger\n ) as parser: # type: OnnxParser\n builder.max_batch_size = max_shape[0] # max batch size\n config: IBuilderConfig = builder.create_builder_config()\n config.max_workspace_size = workspace_size\n # to enable complete trt inspector debugging, only for TensorRT >= 8.2\n config.profiling_verbosity = trt.ProfilingVerbosity.DETAILED\n # disable CUDNN optimizations\n config.set_tactic_sources(\n tactic_sources=1 << int(trt.TacticSource.CUBLAS)\n | 1 << int(trt.TacticSource.CUBLAS_LT)\n )\n if int8:\n config.set_flag(trt.BuilderFlag.INT8)\n if fp16:\n config.set_flag(trt.BuilderFlag.FP16)\n config.set_flag(trt.BuilderFlag.DISABLE_TIMING_CACHE)\n # https://github.com/NVIDIA/TensorRT/issues/1196 (sometimes big diff in output when using FP16)\n config.set_flag(trt.BuilderFlag.OBEY_PRECISION_CONSTRAINTS)\n with open(onnx_file_path, \"rb\") as f:\n parser.parse(f.read())\n profile: IOptimizationProfile = builder.create_optimization_profile()\n for num_input in range(network_definition.num_inputs):\n profile.set_shape(\n input=network_definition.get_input(num_input).name,\n min=min_shape,\n opt=optimal_shape,\n max=max_shape,\n )\n config.add_optimization_profile(profile)\n if fp16:\n network_definition = fix_fp16_network(network_definition)\n trt_engine = builder.build_serialized_network(\n network_definition, config\n )\n engine: ICudaEngine = runtime.deserialize_cuda_engine(trt_engine)\n assert (\n engine is not None\n ), \"error during engine generation, check error messages above :-(\"\n return engine\n\n\ndef get_output_tensors(\n context: trt.IExecutionContext,\n host_inputs: List[torch.Tensor],\n input_binding_idxs: List[int],\n output_binding_idxs: List[int],\n) -> List[torch.Tensor]:\n \"\"\"\n Reserve memory in GPU for input and output tensors.\n :param context: TensorRT context shared accross inference steps\n :param host_inputs: input tensor\n :param input_binding_idxs: indexes of each input vector (should be the same than during building)\n :param output_binding_idxs: indexes of each output vector (should be the same than during building)\n :return: tensors where output will be stored\n \"\"\"\n # explicitly set dynamic input shapes, so dynamic output shapes can be computed internally\n for host_input, binding_index in zip(host_inputs, input_binding_idxs):\n context.set_binding_shape(binding_index, tuple(host_input.shape))\n assert context.all_binding_shapes_specified\n device_outputs: List[torch.Tensor] = []\n for binding_index in output_binding_idxs:\n # TensorRT computes output shape based on input shape provided above\n output_shape = context.get_binding_shape(binding_index)\n # allocate buffers to hold output results\n output = torch.empty(tuple(output_shape), device=\"cuda\")\n device_outputs.append(output)\n return device_outputs\n\n\ndef infer_tensorrt(\n context: IExecutionContext,\n host_inputs: OrderedDict[str, torch.Tensor],\n input_binding_idxs: List[int],\n output_binding_idxs: List[int],\n) -> List[torch.Tensor]:\n \"\"\"\n Perform inference with TensorRT.\n :param context: shared variable\n :param host_inputs: input tensor\n :param input_binding_idxs: input tensor indexes\n :param output_binding_idxs: output tensor indexes\n :return: output tensor\n \"\"\"\n input_tensors: List[torch.Tensor] = list()\n for tensor in host_inputs.values():\n assert isinstance(\n tensor, torch.Tensor\n ), f\"unexpected tensor type: {tensor.dtype}\"\n\n if tensor.dtype == torch.int64:\n # warning: small changes in output if int64 is used instead of int32\n tensor = tensor.type(torch.int32)\n # tensor = tensor.to(\"cuda\")\n input_tensors.append(tensor)\n # calculate input shape, bind it, allocate GPU memory for the output\n output_tensors: List[torch.Tensor] = get_output_tensors(\n context, input_tensors, input_binding_idxs, output_binding_idxs\n )\n bindings = [int(i.data_ptr()) for i in input_tensors + output_tensors]\n assert context.execute_async_v2(\n bindings, torch.cuda.current_stream().cuda_stream\n ), \"failure during execution of inference\"\n torch.cuda.current_stream().synchronize() # sync all CUDA ops\n return output_tensors\n\n\ndef load_engine(\n runtime: Runtime, engine_file_path: str, profile_index: int = 0\n) -> Callable[[Dict[str, torch.Tensor]], torch.Tensor]:\n \"\"\"\n Load serialized TensorRT engine.\n :param runtime: shared variable\n :param engine_file_path: path to the serialized engine\n :param profile_index: which profile to load, 0 if you have not used multiple profiles\n :return: A function to perform inference\n \"\"\"\n with open(file=engine_file_path, mode=\"rb\") as f:\n engine: ICudaEngine = runtime.deserialize_cuda_engine(f.read())\n stream: int = torch.cuda.current_stream().cuda_stream\n context: IExecutionContext = engine.create_execution_context()\n context.set_optimization_profile_async(\n profile_index=profile_index, stream_handle=stream\n )\n # retrieve input/output IDs\n input_binding_idxs, output_binding_idxs = get_binding_idxs(\n engine, profile_index\n ) # type: List[int], List[int]\n\n def tensorrt_model(inputs: Dict[str, torch.Tensor]) -> torch.Tensor:\n return infer_tensorrt(\n context=context,\n host_inputs=inputs,\n input_binding_idxs=input_binding_idxs,\n output_binding_idxs=output_binding_idxs,\n )\n\n return tensorrt_model\n\n\ndef save_engine(engine: ICudaEngine, engine_file_path: str) -> None:\n \"\"\"\n Serialize TensorRT engine to file.\n :param engine: TensorRT engine\n :param engine_file_path: output path\n \"\"\"\n with open(engine_file_path, \"wb\") as f:\n f.write(engine.serialize())\n\n\ndef get_binding_idxs(engine: trt.ICudaEngine, profile_index: int):\n \"\"\"\n Calculate start/end binding indices for current context's profile\n https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#opt_profiles_bindings\n :param engine: TensorRT engine generated during the model building\n :param profile_index: profile to use (several profiles can be set during building)\n :return: input and output tensor indexes\n \"\"\"\n num_bindings_per_profile = engine.num_bindings // engine.num_optimization_profiles\n start_binding = profile_index * num_bindings_per_profile\n end_binding = (\n start_binding + num_bindings_per_profile\n ) # Separate input and output binding indices for convenience\n input_binding_idxs: List[int] = []\n output_binding_idxs: List[int] = []\n for binding_index in range(start_binding, end_binding):\n if engine.binding_is_input(binding_index):\n input_binding_idxs.append(binding_index)\n else:\n output_binding_idxs.append(binding_index)\n return input_binding_idxs, output_binding_idxs\n" }, { "path": "server/clip_server/onnx-flow.yml", "content": "jtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_o\n uses:\n jtype: CLIPEncoder\n metas:\n py_modules:\n - clip_server.executors.clip_onnx\n timeout_ready: 3000000\n replicas: 1" }, { "path": "server/clip_server/tensorrt-flow.yml", "content": "jtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_r\n uses:\n jtype: CLIPEncoder\n metas:\n py_modules:\n - clip_server.executors.clip_tensorrt\n timeout_ready: 3000000\n replicas: 1" }, { "path": "server/clip_server/torch-flow.yml", "content": "jtype: Flow\nversion: '1'\nwith:\n port: 51000\nexecutors:\n - name: clip_t\n uses:\n jtype: CLIPEncoder\n metas:\n py_modules:\n - clip_server.executors.clip_torch\n timeout_ready: 3000000\n replicas: 1" }, { "path": "server/setup.py", "content": "import sys\nfrom os import path\n\nfrom setuptools import find_packages, setup\n\nif sys.version_info < (3, 7, 0):\n raise OSError(f'CLIP-as-service requires Python >=3.7, but yours is {sys.version}')\n\ntry:\n pkg_name = 'clip-server'\n libinfo_py = path.join(\n path.dirname(__file__), pkg_name.replace('-', '_'), '__init__.py'\n )\n libinfo_content = open(libinfo_py, 'r', encoding='utf8').readlines()\n version_line = [l.strip() for l in libinfo_content if l.startswith('__version__')][\n 0\n ]\n exec(version_line) # gives __version__\nexcept FileNotFoundError:\n __version__ = '0.0.0'\n\ntry:\n with open('../README.md', encoding='utf8') as fp:\n _long_description = fp.read()\nexcept FileNotFoundError:\n _long_description = ''\n\nsetup(\n name=pkg_name,\n packages=find_packages(),\n version=__version__,\n include_package_data=True,\n description='Embed images and sentences into fixed-length vectors via CLIP',\n author='Jina AI',\n author_email='hello@jina.ai',\n license='Apache 2.0',\n url='https://github.com/jina-ai/clip-as-service',\n download_url='https://github.com/jina-ai/clip-as-service/tags',\n long_description=_long_description,\n long_description_content_type='text/markdown',\n zip_safe=False,\n setup_requires=['setuptools>=18.0', 'wheel'],\n install_requires=[\n 'ftfy',\n 'torch',\n 'regex',\n 'torchvision<=0.13.0' if sys.version_info <= (3, 7, 2) else 'torchvision',\n 'jina>=3.12.0',\n 'docarray==0.21.0',\n 'prometheus-client',\n 'open_clip_torch>=2.8.0,<2.9.0',\n 'pillow-avif-plugin',\n ],\n extras_require={\n 'onnx': [\n 'onnx',\n 'onnxmltools<1.12.0',\n ]\n + (\n ['onnxruntime-gpu<=1.13.1']\n if sys.platform != 'darwin'\n else ['onnxruntime<=1.13.1']\n ),\n 'tensorrt': [\n 'nvidia-tensorrt==8.4.1.5',\n ],\n 'transformers': ['transformers>=4.16.2'],\n 'search': ['annlite>=0.3.10'],\n 'flash-attn': ['flash-attn'],\n 'cn_clip': ['cn_clip'],\n },\n classifiers=[\n 'Development Status :: 5 - Production/Stable',\n 'Intended Audience :: Developers',\n 'Intended Audience :: Education',\n 'Intended Audience :: Science/Research',\n 'Programming Language :: Python :: 3.7',\n 'Programming Language :: Python :: 3.8',\n 'Programming Language :: Python :: 3.9',\n 'Programming Language :: Python :: 3.10',\n 'Programming Language :: Unix Shell',\n 'Environment :: Console',\n 'License :: OSI Approved :: Apache Software License',\n 'Operating System :: OS Independent',\n 'Topic :: Database :: Database Engines/Servers',\n 'Topic :: Scientific/Engineering :: Artificial Intelligence',\n 'Topic :: Internet :: WWW/HTTP :: Indexing/Search',\n 'Topic :: Scientific/Engineering :: Image Recognition',\n 'Topic :: Multimedia :: Video',\n 'Topic :: Scientific/Engineering',\n 'Topic :: Scientific/Engineering :: Mathematics',\n 'Topic :: Software Development',\n 'Topic :: Software Development :: Libraries',\n 'Topic :: Software Development :: Libraries :: Python Modules',\n ],\n project_urls={\n 'Documentation': 'https://clip-as-service.jina.ai',\n 'Source': 'https://github.com/jina-ai/clip-as-service/',\n 'Tracker': 'https://github.com/jina-ai/clip-as-service/issues',\n },\n keywords='jina openai clip deep-learning cross-modal multi-modal neural-search',\n)\n" }, { "path": "tests/__init__.py", "content": "import os\n\nos.environ['OMP_NUM_THREADS'] = '1'\n" }, { "path": "tests/conftest.py", "content": "import pytest\nfrom jina import helper, Flow\n\n\n@pytest.fixture(scope='session')\ndef port_generator():\n generated_ports = set()\n\n def random_port():\n port = helper.random_port()\n while port in generated_ports:\n port = helper.random_port()\n generated_ports.add(port)\n return port\n\n return random_port\n\n\n@pytest.fixture(scope='session', params=['onnx', 'torch', 'onnx_custom'])\ndef make_flow(port_generator, request):\n if request.param != 'onnx_custom':\n if request.param == 'onnx':\n from clip_server.executors.clip_onnx import CLIPEncoder\n else:\n from clip_server.executors.clip_torch import CLIPEncoder\n\n f = Flow(port=port_generator()).add(name=request.param, uses=CLIPEncoder)\n else:\n import os\n from clip_server.executors.clip_onnx import CLIPEncoder\n\n f = Flow(port=port_generator()).add(\n name=request.param,\n uses=CLIPEncoder,\n uses_with={\n 'model_path': os.path.expanduser('~/.cache/clip/ViT-B-32-openai')\n },\n )\n with f:\n yield f\n\n\n@pytest.fixture(scope='session', params=['torch'])\ndef make_torch_flow(port_generator, request):\n from clip_server.executors.clip_torch import CLIPEncoder\n\n f = Flow(port=port_generator()).add(name=request.param, uses=CLIPEncoder)\n with f:\n yield f\n\n\n@pytest.fixture(scope='session', params=['tensorrt'])\ndef make_trt_flow(port_generator, request):\n from clip_server.executors.clip_tensorrt import CLIPEncoder\n\n f = Flow(port=port_generator()).add(name=request.param, uses=CLIPEncoder)\n with f:\n yield f\n\n\n@pytest.fixture(params=['torch'])\ndef make_search_flow(tmpdir, port_generator, request):\n from clip_server.executors.clip_torch import CLIPEncoder\n from annlite.executor import AnnLiteIndexer\n\n f = (\n Flow(port=port_generator())\n .add(name=request.param, uses=CLIPEncoder)\n .add(\n name='annlite',\n uses=AnnLiteIndexer,\n workspace=tmpdir,\n uses_with={'n_dim': 512},\n )\n )\n with f:\n yield f\n" }, { "path": "tests/test_asyncio.py", "content": "import asyncio\nimport os\nimport pytest\n\nfrom clip_client import Client\nfrom docarray import Document, DocumentArray\n\n\nasync def another_heavylifting_job():\n await asyncio.sleep(3)\n\n\n@pytest.mark.asyncio\nasync def test_async_encode(make_flow):\n c = Client(server=f'grpc://0.0.0.0:{make_flow.port}')\n t1 = asyncio.create_task(another_heavylifting_job())\n t2 = asyncio.create_task(c.aencode(['hello world'] * 10))\n await asyncio.gather(t1, t2)\n assert t2.result().shape\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n DocumentArray([Document(text='hello, world'), Document(text='goodbye, world')]),\n DocumentArray(\n [\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n text='hello, world',\n ),\n ]\n ),\n DocumentArray.from_files(\n f'{os.path.dirname(os.path.abspath(__file__))}/**/*.jpg'\n ),\n ],\n)\n@pytest.mark.asyncio\nasync def test_async_docarray_preserve_original_inputs(make_flow, inputs):\n c = Client(server=f'grpc://0.0.0.0:{make_flow.port}')\n t1 = asyncio.create_task(another_heavylifting_job())\n t2 = asyncio.create_task(c.aencode(inputs if not callable(inputs) else inputs()))\n await asyncio.gather(t1, t2)\n assert isinstance(t2.result(), DocumentArray)\n assert inputs[0] is t2.result()[0]\n assert t2.result().embeddings.shape\n assert t2.result().contents == inputs.contents\n assert not t2.result()[0].tensor\n assert inputs[0] is t2.result()[0]\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n [Document(id=str(i), text='hello, world') for i in range(20)],\n DocumentArray([Document(id=str(i), text='hello, world') for i in range(20)]),\n ],\n)\n@pytest.mark.asyncio\nasync def test_async_docarray_preserve_original_order(make_flow, inputs):\n c = Client(server=f'grpc://0.0.0.0:{make_flow.port}')\n t1 = asyncio.create_task(another_heavylifting_job())\n t2 = asyncio.create_task(\n c.aencode(inputs if not callable(inputs) else inputs(), batch_size=1)\n )\n await asyncio.gather(t1, t2)\n assert isinstance(t2.result(), DocumentArray)\n for i in range(len(inputs)):\n assert inputs[i] is t2.result()[i]\n assert inputs[i].id == str(i)\n" }, { "path": "tests/test_client.py", "content": "import os\nimport random\nimport time\nimport pytest\nimport numpy as np\nfrom docarray import Document, DocumentArray\nfrom jina import Flow, Executor, requests\n\n\nclass Exec1(Executor):\n @requests\n async def aencode(self, docs, **kwargs):\n time.sleep(random.random() * 1)\n docs.embeddings = np.random.rand(len(docs), 10)\n\n\nclass Exec2(Executor):\n def __init__(self, server_host: str = '', **kwargs):\n super().__init__(**kwargs)\n from clip_client.client import Client\n\n self._client = Client(server=server_host)\n\n @requests\n async def process(self, docs, **kwargs):\n results = await self._client.aencode(docs, batch_size=2)\n return results\n\n\nclass ErrorExec(Executor):\n @requests\n def foo(self, docs, **kwargs):\n raise NotImplementedError\n\n\ndef test_client_concurrent_requests(port_generator):\n\n f1 = Flow(port=port_generator()).add(uses=Exec1)\n\n f2 = Flow(protocol='http').add(\n uses=Exec2, uses_with={'server_host': f'grpc://0.0.0.0:{f1.port}'}\n )\n\n with f1, f2:\n import jina\n from multiprocessing.pool import ThreadPool\n\n def run_post(docs):\n c = jina.clients.Client(port=f2.port, protocol='http')\n results = c.post(on='/', inputs=docs, request_size=2)\n # assert set([d.id for d in results]) != set([d.id for d in docs])\n return results\n\n def generate_docs(tag):\n return DocumentArray(\n [Document(id=f'{tag}_{i}', text='hello') for i in range(20)]\n )\n\n with ThreadPool(5) as p:\n results = p.map(run_post, [generate_docs(f't{k}') for k in range(5)])\n\n for r in results:\n assert len(set([d.id[:2] for d in r])) == 1\n\n\ndef test_client_large_input(make_torch_flow):\n from clip_client.client import Client\n\n inputs = ['hello' for _ in range(600)]\n\n c = Client(server=f'grpc://0.0.0.0:{make_torch_flow.port}')\n with pytest.warns(UserWarning):\n c.encode(inputs if not callable(inputs) else inputs())\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n [],\n DocumentArray(),\n ],\n)\n@pytest.mark.parametrize('endpoint', ['encode', 'rank', 'index', 'search'])\n@pytest.mark.asyncio\ndef test_empty_input(make_torch_flow, inputs, endpoint):\n from clip_client.client import Client\n\n c = Client(server=f'grpc://0.0.0.0:{make_torch_flow.port}')\n\n r = getattr(c, endpoint)(inputs if not callable(inputs) else inputs())\n if endpoint == 'encode':\n if isinstance(inputs, DocumentArray):\n assert isinstance(r, DocumentArray)\n else:\n assert isinstance(r, list)\n else:\n assert isinstance(r, DocumentArray)\n assert len(r) == 0\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n [],\n DocumentArray(),\n ],\n)\n@pytest.mark.parametrize('endpoint', ['aencode', 'arank', 'aindex', 'asearch'])\n@pytest.mark.asyncio\nasync def test_async_empty_input(make_torch_flow, inputs, endpoint):\n from clip_client.client import Client\n\n c = Client(server=f'grpc://0.0.0.0:{make_torch_flow.port}')\n\n r = await getattr(c, endpoint)(inputs if not callable(inputs) else inputs())\n if endpoint == 'aencode':\n if isinstance(inputs, DocumentArray):\n assert isinstance(r, DocumentArray)\n else:\n assert isinstance(r, list)\n else:\n assert isinstance(r, DocumentArray)\n assert len(r) == 0\n\n\n@pytest.mark.parametrize('endpoint', ['encode', 'rank', 'index', 'search'])\ndef test_wrong_input_type(make_torch_flow, endpoint):\n from clip_client.client import Client\n\n c = Client(server=f'grpc://0.0.0.0:{make_torch_flow.port}')\n\n with pytest.raises(Exception):\n getattr(c, endpoint)('hello')\n\n\n@pytest.mark.parametrize('endpoint', ['aencode', 'arank', 'aindex', 'asearch'])\n@pytest.mark.asyncio\nasync def test_wrong_input_type(make_torch_flow, endpoint):\n from clip_client.client import Client\n\n c = Client(server=f'grpc://0.0.0.0:{make_torch_flow.port}')\n\n with pytest.raises(Exception):\n await getattr(c, endpoint)('hello')\n\n\n@pytest.mark.parametrize('endpoint', ['encode', 'rank', 'index', 'search'])\n@pytest.mark.slow\ndef test_custom_on_done(make_torch_flow, mocker, endpoint):\n from clip_client.client import Client\n\n c = Client(server=f'grpc://0.0.0.0:{make_torch_flow.port}')\n\n on_done_mock = mocker.Mock()\n on_error_mock = mocker.Mock()\n on_always_mock = mocker.Mock()\n\n r = getattr(c, endpoint)(\n DocumentArray(\n [Document(text='hello', matches=DocumentArray([Document(text='jina')]))]\n ),\n on_done=on_done_mock,\n on_error=on_error_mock,\n on_always=on_always_mock,\n )\n assert r is None\n on_done_mock.assert_called_once()\n on_error_mock.assert_not_called()\n on_always_mock.assert_called_once()\n\n\n@pytest.mark.parametrize('endpoint', ['aencode', 'arank', 'aindex', 'asearch'])\n@pytest.mark.slow\n@pytest.mark.asyncio\nasync def test_async_custom_on_done(make_torch_flow, mocker, endpoint):\n from clip_client.client import Client\n\n c = Client(server=f'grpc://0.0.0.0:{make_torch_flow.port}')\n\n on_done_mock = mocker.Mock()\n on_error_mock = mocker.Mock()\n on_always_mock = mocker.Mock()\n\n r = await getattr(c, endpoint)(\n DocumentArray(\n [Document(text='hello', matches=DocumentArray([Document(text='jina')]))]\n ),\n on_done=on_done_mock,\n on_error=on_error_mock,\n on_always=on_always_mock,\n )\n assert r is None\n on_done_mock.assert_called_once()\n on_error_mock.assert_not_called()\n on_always_mock.assert_called_once()\n\n\n@pytest.mark.parametrize('endpoint', ['encode', 'rank', 'index', 'search'])\n@pytest.mark.slow\ndef test_custom_on_error(port_generator, mocker, endpoint):\n from clip_client.client import Client\n\n f = Flow(port=port_generator()).add(uses=ErrorExec)\n\n with f:\n c = Client(server=f'grpc://0.0.0.0:{f.port}')\n\n on_done_mock = mocker.Mock()\n on_error_mock = mocker.Mock()\n on_always_mock = mocker.Mock()\n\n r = getattr(c, endpoint)(\n DocumentArray(\n [Document(text='hello', matches=DocumentArray([Document(text='jina')]))]\n ),\n on_done=on_done_mock,\n on_error=on_error_mock,\n on_always=on_always_mock,\n )\n assert r is None\n on_done_mock.assert_not_called()\n on_error_mock.assert_called_once()\n on_always_mock.assert_called_once()\n\n\n@pytest.mark.parametrize('endpoint', ['aencode', 'arank', 'aindex', 'asearch'])\n@pytest.mark.slow\n@pytest.mark.asyncio\nasync def test_async_custom_on_error(port_generator, mocker, endpoint):\n from clip_client.client import Client\n\n f = Flow(port=port_generator()).add(uses=ErrorExec)\n\n with f:\n c = Client(server=f'grpc://0.0.0.0:{f.port}')\n\n on_done_mock = mocker.Mock()\n on_error_mock = mocker.Mock()\n on_always_mock = mocker.Mock()\n\n r = await getattr(c, endpoint)(\n DocumentArray(\n [Document(text='hello', matches=DocumentArray([Document(text='jina')]))]\n ),\n on_done=on_done_mock,\n on_error=on_error_mock,\n on_always=on_always_mock,\n )\n assert r is None\n on_done_mock.assert_not_called()\n on_error_mock.assert_called_once()\n on_always_mock.assert_called_once()\n" }, { "path": "tests/test_helper.py", "content": "import pytest\nimport numpy as np\nfrom clip_server.executors.helper import numpy_softmax\nfrom clip_server.executors.helper import split_img_txt_da\nfrom clip_server.executors.helper import preproc_image\nfrom docarray import Document, DocumentArray\n\n\n@pytest.mark.parametrize('shape', [(5, 10), (5, 10, 10)])\n@pytest.mark.parametrize('axis', [-1, 1, 0])\ndef test_numpy_softmax(shape, axis):\n import torch\n\n logits = np.random.random(shape)\n\n np_softmax = numpy_softmax(logits, axis=axis)\n torch_softmax = torch.from_numpy(logits).softmax(dim=axis).numpy()\n np.testing.assert_array_almost_equal(np_softmax, torch_softmax)\n\n np_softmax = numpy_softmax(logits, axis=axis)\n torch_softmax = torch.from_numpy(logits).softmax(dim=axis).numpy()\n np.testing.assert_array_almost_equal(np_softmax, torch_softmax)\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n (\n DocumentArray(\n [\n Document(text='hello, world'),\n Document(text='goodbye, world'),\n Document(\n text='hello, world',\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n ),\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n ),\n ]\n ),\n (3, 1),\n ),\n (\n DocumentArray(\n [\n Document(text='hello, world'),\n Document(tensor=np.array([0, 1, 2])),\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png'\n ).load_uri_to_blob(),\n Document(\n tensor=np.array([0, 1, 2]),\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n ),\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n ),\n ]\n ),\n (1, 4),\n ),\n (\n DocumentArray(\n [\n Document(text='hello, world'),\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n ]\n ),\n (1, 1),\n ),\n ],\n)\ndef test_split_img_txt_da(inputs):\n txt_da = DocumentArray()\n img_da = DocumentArray()\n for doc in inputs[0]:\n split_img_txt_da(doc, img_da, txt_da)\n assert len(txt_da) == inputs[1][0]\n assert len(img_da) == inputs[1][1]\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n DocumentArray(\n [\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n ).load_uri_to_image_tensor(),\n ]\n )\n ],\n)\ndef test_preproc_image(inputs):\n from clip_server.model import clip\n\n preprocess_fn = clip._transform_blob(224)\n da, pixel_values = preproc_image(inputs, preprocess_fn, drop_image_content=True)\n assert len(da) == 1\n assert not da[0].blob\n assert not da[0].tensor\n assert pixel_values.get('pixel_values') is not None\n" }, { "path": "tests/test_model.py", "content": "import pytest\nfrom clip_server.model.clip_model import CLIPModel\nfrom clip_server.model.clip_onnx import CLIPOnnxModel\nfrom clip_server.model.openclip_model import OpenCLIPModel\nfrom clip_server.model.mclip_model import MultilingualCLIPModel\nfrom clip_server.model.cnclip_model import CNClipModel\n\n\n@pytest.mark.parametrize(\n 'name, model_cls',\n [\n ('ViT-L/14@336px', OpenCLIPModel),\n ('RN50::openai', OpenCLIPModel),\n ('roberta-ViT-B-32::laion2b-s12b-b32k', OpenCLIPModel),\n ('M-CLIP/LABSE-Vit-L-14', MultilingualCLIPModel),\n ('CN-CLIP/ViT-B-16', CNClipModel),\n ],\n)\ndef test_torch_model(name, model_cls):\n model = CLIPModel(name)\n assert model.__class__ == model_cls\n\n\n@pytest.mark.parametrize(\n 'name',\n [\n 'RN50::openai',\n 'ViT-H-14::laion2b-s32b-b79k',\n 'M-CLIP/LABSE-Vit-L-14',\n ],\n)\ndef test_onnx_model(name):\n CLIPOnnxModel(name)\n\n\n@pytest.mark.gpu\n@pytest.mark.parametrize(\n 'name',\n ['ViT-H-14::laion2b-s32b-b79k'],\n)\ndef test_large_onnx_model_fp16(name):\n from clip_server.executors.clip_onnx import CLIPEncoder\n\n CLIPEncoder(name, dtype='fp16')\n" }, { "path": "tests/test_ranker.py", "content": "import os\n\nimport numpy as np\nimport pytest\nfrom docarray import DocumentArray, Document\n\nfrom clip_client import Client\nfrom clip_server.executors.clip_onnx import CLIPEncoder as ONNXCLILPEncoder\nfrom clip_server.executors.clip_torch import CLIPEncoder as TorchCLIPEncoder\n\n\n@pytest.mark.asyncio\n@pytest.mark.parametrize('encoder_class', [TorchCLIPEncoder, ONNXCLILPEncoder])\nasync def test_torch_executor_rank_img2texts(encoder_class):\n ce = encoder_class()\n\n da = DocumentArray.from_files(\n f'{os.path.dirname(os.path.abspath(__file__))}/**/*.jpg'\n )\n for d in da:\n d.matches.append(Document(text='hello, world!'))\n d.matches.append(Document(text='goodbye, world!'))\n d.matches.append(Document(text='goodbye,!'))\n d.matches.append(Document(text='good world!'))\n d.matches.append(Document(text='good!'))\n d.matches.append(Document(text='world!'))\n\n await ce.rank(da, {})\n print(da['@m', 'scores__clip_score__value'])\n for d in da:\n for c in d.matches:\n assert c.scores['clip_score'].value is not None\n assert not c.tensor\n org_score = d.matches[:, 'scores__clip_score__value']\n assert org_score == list(sorted(org_score, reverse=True))\n assert not d.tensor\n\n\n@pytest.mark.asyncio\n@pytest.mark.parametrize('encoder_class', [TorchCLIPEncoder, ONNXCLILPEncoder])\nasync def test_torch_executor_rank_text2imgs(encoder_class):\n ce = encoder_class()\n db = DocumentArray(\n [Document(text='hello, world!'), Document(text='goodbye, world!')]\n )\n for d in db:\n d.matches.extend(\n DocumentArray.from_files(\n f'{os.path.dirname(os.path.abspath(__file__))}/**/*.jpg'\n )\n )\n await ce.rank(db, {})\n print(db['@m', 'scores__clip_score__value'])\n for d in db:\n for c in d.matches:\n assert c.scores['clip_score'].value is not None\n assert c.scores['clip_score_cosine'].value is not None\n assert not c.tensor\n np.testing.assert_almost_equal(\n sum(c.scores['clip_score'].value for c in d.matches), 1\n )\n assert not d.tensor\n assert not d.blob\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n [\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n matches=[\n Document(text='hello, world'),\n Document(text='goodbye, world'),\n ],\n ),\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n matches=[\n Document(text='hello, world'),\n Document(text='goodbye, world'),\n ],\n ),\n ],\n DocumentArray(\n [\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n matches=[\n Document(text='hello, world'),\n Document(text='goodbye, world'),\n ],\n ),\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n matches=[\n Document(text='hello, world'),\n Document(text='goodbye, world'),\n ],\n ),\n ]\n ),\n lambda: (\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n matches=[\n Document(text='hello, world'),\n Document(text='goodbye, world'),\n ],\n )\n for _ in range(10)\n ),\n DocumentArray(\n [\n Document(\n text='hello, world',\n matches=[\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png'\n ),\n Document(\n uri=f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg'\n ),\n ],\n )\n ]\n ),\n ],\n)\ndef test_docarray_inputs(make_flow, inputs):\n c = Client(server=f'grpc://0.0.0.0:{make_flow.port}')\n r = c.rank(inputs if not callable(inputs) else inputs())\n assert not r[0].tensor\n assert isinstance(r, DocumentArray)\n rv1 = r['@m', 'scores__clip_score__value']\n rv2 = r['@m', 'scores__clip_score_cosine__value']\n for v1, v2 in zip(rv1, rv2):\n assert v1 is not None\n assert v1 > 0\n assert v2 is not None\n assert v2 > 0\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n [\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n matches=[\n Document(text='hello, world'),\n Document(text='goodbye, world'),\n ],\n ),\n ],\n DocumentArray(\n [\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n matches=[\n Document(text='hello, world'),\n Document(text='goodbye, world'),\n ],\n ),\n ]\n ),\n lambda: (\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n matches=[\n Document(text='hello, world'),\n Document(text='goodbye, world'),\n ],\n )\n for _ in range(1)\n ),\n DocumentArray(\n [\n Document(\n text='hello, world',\n matches=[\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png'\n ),\n Document(\n uri=f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg'\n ),\n ],\n )\n ]\n ),\n ],\n)\n@pytest.mark.asyncio\nasync def test_async_arank(make_flow, inputs):\n c = Client(server=f'grpc://0.0.0.0:{make_flow.port}')\n r = await c.arank(inputs if not callable(inputs) else inputs())\n assert not r[0].tensor\n assert isinstance(r, DocumentArray)\n rv = r['@m', 'scores__clip_score__value']\n for v in rv:\n assert v is not None\n assert v > 0\n np.testing.assert_almost_equal(sum(rv), 1.0)\n\n rv = r['@m', 'scores__clip_score_cosine__value']\n for v in rv:\n assert v is not None\n assert -1.0 <= v <= 1.0\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n [\n Document(\n id=str(i), text='A', matches=[Document(text='B'), Document(text='C')]\n )\n for i in range(20)\n ],\n DocumentArray(\n [\n Document(\n id=str(i),\n text='A',\n matches=[Document(text='B'), Document(text='C')],\n )\n for i in range(20)\n ]\n ),\n ],\n)\ndef test_docarray_preserve_original_order(make_flow, inputs):\n c = Client(server=f'grpc://0.0.0.0:{make_flow.port}')\n r = c.rank(inputs, batch_size=1)\n assert isinstance(r, DocumentArray)\n for i in range(len(inputs)):\n assert inputs[i] is r[i]\n assert inputs[i].id == str(i)\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n [\n Document(\n id=str(i), text='A', matches=[Document(text='B'), Document(text='C')]\n )\n for i in range(20)\n ],\n DocumentArray(\n [\n Document(\n id=str(i),\n text='A',\n matches=[Document(text='B'), Document(text='C')],\n )\n for i in range(20)\n ]\n ),\n ],\n)\n@pytest.mark.asyncio\nasync def test_async_docarray_preserve_original_order(make_flow, inputs):\n c = Client(server=f'grpc://0.0.0.0:{make_flow.port}')\n r = await c.arank(inputs, batch_size=1)\n assert isinstance(r, DocumentArray)\n for i in range(len(inputs)):\n assert inputs[i] is r[i]\n assert inputs[i].id == str(i)\n" }, { "path": "tests/test_search.py", "content": "import os\n\nimport numpy as np\nimport pytest\nfrom docarray import DocumentArray, Document\n\nfrom clip_client import Client\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n [Document(text='hello, world'), Document(text='goodbye, world')],\n DocumentArray([Document(text='hello, world'), Document(text='goodbye, world')]),\n lambda: (Document(text='hello, world') for _ in range(10)),\n DocumentArray(\n [\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n Document(\n uri=f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg'\n ),\n Document(text='hello, world'),\n Document(\n uri=f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg'\n ).load_uri_to_image_tensor(),\n ]\n ),\n DocumentArray.from_files(\n f'{os.path.dirname(os.path.abspath(__file__))}/**/*.jpg'\n ),\n ],\n)\n@pytest.mark.parametrize('limit', [1, 2])\ndef test_index_search(make_search_flow, inputs, limit):\n c = Client(server=f'grpc://0.0.0.0:{make_search_flow.port}')\n\n r = c.index(inputs if not callable(inputs) else inputs())\n assert isinstance(r, DocumentArray)\n assert r.embeddings.shape[1] == 512\n\n r = c.search(inputs if not callable(inputs) else inputs(), limit=limit)\n assert isinstance(r, DocumentArray)\n for d in r:\n assert len(d.matches) == limit\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n [Document(text='hello, world'), Document(text='goodbye, world')],\n DocumentArray([Document(text='hello, world'), Document(text='goodbye, world')]),\n lambda: (Document(text='hello, world') for _ in range(10)),\n DocumentArray(\n [\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n Document(\n uri=f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg'\n ),\n Document(text='hello, world'),\n Document(\n uri=f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg'\n ).load_uri_to_image_tensor(),\n ]\n ),\n DocumentArray.from_files(\n f'{os.path.dirname(os.path.abspath(__file__))}/**/*.jpg'\n ),\n ],\n)\n@pytest.mark.parametrize('limit', [1, 2])\n@pytest.mark.asyncio\nasync def test_async_index_search(make_search_flow, inputs, limit):\n c = Client(server=f'grpc://0.0.0.0:{make_search_flow.port}')\n r = await c.aindex(inputs if not callable(inputs) else inputs())\n assert isinstance(r, DocumentArray)\n\n assert r.embeddings.shape[1] == 512\n\n r = await c.asearch(inputs if not callable(inputs) else inputs(), limit=limit)\n assert isinstance(r, DocumentArray)\n for d in r:\n assert len(d.matches) == limit\n" }, { "path": "tests/test_server.py", "content": "import os\n\nimport pytest\nfrom clip_server.model.clip import _transform_ndarray, _transform_blob\nfrom clip_server.model.pretrained_models import download_model\nfrom docarray import Document\nfrom jina import Flow\nimport numpy as np\n\n\ndef test_server_download(tmpdir):\n download_model(\n url='https://clip-as-service.jina.ai/_static/favicon.png',\n target_folder=tmpdir,\n md5sum='43104e468ddd23c55bc662d84c87a7f8',\n with_resume=False,\n )\n target_path = os.path.join(tmpdir, 'favicon.png')\n file_size = os.path.getsize(target_path)\n assert file_size > 0\n\n part_path = target_path + '.part'\n with open(target_path, 'rb') as source, open(part_path, 'wb') as part_out:\n buf = source.read(10)\n part_out.write(buf)\n\n os.remove(target_path)\n\n download_model(\n url='https://clip-as-service.jina.ai/_static/favicon.png',\n target_folder=tmpdir,\n md5sum='43104e468ddd23c55bc662d84c87a7f8',\n with_resume=True,\n )\n assert os.path.getsize(target_path) == file_size\n assert not os.path.exists(part_path)\n\n\n@pytest.mark.parametrize('md5', ['ABC', None, '43104e468ddd23c55bc662d84c87a7f8'])\ndef test_server_download_md5(tmpdir, md5):\n if md5 != 'ABC':\n download_model(\n url='https://clip-as-service.jina.ai/_static/favicon.png',\n target_folder=tmpdir,\n md5sum=md5,\n with_resume=False,\n )\n else:\n with pytest.raises(Exception):\n download_model(\n url='https://clip-as-service.jina.ai/_static/favicon.png',\n target_folder=tmpdir,\n md5sum=md5,\n with_resume=False,\n )\n\n\ndef test_server_download_not_regular_file(tmpdir):\n with pytest.raises(Exception):\n download_model(\n url='https://clip-as-service.jina.ai/_static/favicon.png',\n target_folder=tmpdir,\n md5sum='',\n with_resume=False,\n )\n download_model(\n url='https://docarray.jina.ai/_static/',\n target_folder=tmpdir,\n md5sum='',\n with_resume=False,\n )\n\n\ndef test_make_onnx_flow_wrong_name_path():\n from clip_server.executors.clip_onnx import CLIPEncoder\n\n with pytest.raises(Exception):\n encoder = CLIPEncoder(\n 'ABC', model_path=os.path.expanduser('~/.cache/clip/ViT-B-32')\n )\n\n with pytest.raises(Exception) as info:\n encoder = CLIPEncoder('ViT-B/32', model_path='~/.cache/')\n\n\n@pytest.mark.parametrize(\n 'image_uri',\n [\n f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg',\n 'https://clip-as-service.jina.ai/_static/favicon.png',\n ],\n)\n@pytest.mark.parametrize('size', [224, 288, 384, 448])\ndef test_server_preprocess_ndarray_image(image_uri, size):\n d1 = Document(uri=image_uri)\n d1.load_uri_to_blob()\n d2 = Document(uri=image_uri)\n d2.load_uri_to_image_tensor()\n\n t1 = _transform_blob(size)(d1.blob).numpy()\n t2 = _transform_ndarray(size)(d2.tensor).numpy()\n assert t1.shape == t2.shape\n\n\n@pytest.mark.parametrize(\n 'tensor',\n [\n np.random.random([100, 100, 3]),\n np.random.random([1, 1, 3]),\n np.random.random([5, 50, 3]),\n ],\n)\ndef test_transform_arbitrary_tensor(tensor):\n d = Document(tensor=tensor)\n assert _transform_ndarray(224)(d.tensor).numpy().shape == (3, 224, 224)\n" }, { "path": "tests/test_simple.py", "content": "import os\n\nimport pytest\nfrom docarray import Document, DocumentArray\nfrom jina import Flow\n\nfrom clip_client.client import Client\n\n\n@pytest.mark.parametrize('protocol', ['grpc', 'http', 'websocket', 'other'])\n@pytest.mark.parametrize('jit', [True, False])\ndef test_protocols(port_generator, protocol, jit, pytestconfig):\n from clip_server.executors.clip_torch import CLIPEncoder\n\n if protocol == 'other':\n with pytest.raises(ValueError):\n Client(server=f'{protocol}://0.0.0.0:8000')\n return\n\n f = Flow(port=port_generator(), protocol=protocol).add(\n uses=CLIPEncoder, uses_with={'jit': jit}\n )\n with f:\n c = Client(server=f'{protocol}://0.0.0.0:{f.port}')\n c.profile()\n c.profile(content='hello world')\n c.profile(content=f'{pytestconfig.rootdir}/tests/img/00000.jpg')\n\n\n@pytest.mark.gpu\n@pytest.mark.parametrize(\n 'inputs',\n [\n ['hello, world', 'goodbye, world'],\n ('hello, world', 'goodbye, world'),\n lambda: ('hello, world' for _ in range(10)),\n [\n 'https://clip-as-service.jina.ai/_static/favicon.png',\n f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg',\n 'hello, world',\n ],\n ],\n)\ndef test_plain_inputs(make_flow, inputs):\n c = Client(server=f'grpc://0.0.0.0:{make_flow.port}')\n r = c.encode(inputs if not callable(inputs) else inputs())\n assert (\n r.shape[0] == len(list(inputs)) if not callable(inputs) else len(list(inputs()))\n )\n\n\n@pytest.mark.gpu\n@pytest.mark.parametrize(\n 'inputs',\n [\n [Document(text='hello, world'), Document(text='goodbye, world')],\n DocumentArray([Document(text='hello, world'), Document(text='goodbye, world')]),\n lambda: (Document(text='hello, world') for _ in range(10)),\n DocumentArray(\n [\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n Document(\n uri=f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg'\n ),\n Document(text='hello, world'),\n Document(\n uri=f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg'\n ).load_uri_to_image_tensor(),\n ]\n ),\n DocumentArray.from_files(\n f'{os.path.dirname(os.path.abspath(__file__))}/**/*.jpg'\n ),\n ],\n)\ndef test_docarray_inputs(make_flow, inputs):\n c = Client(server=f'grpc://0.0.0.0:{make_flow.port}')\n r = c.encode(inputs if not callable(inputs) else inputs())\n assert isinstance(r, DocumentArray)\n assert r.embeddings.shape\n assert not r[0].tensor\n if hasattr(inputs, '__len__'):\n assert inputs[0] is r[0]\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n DocumentArray([Document(text='hello, world'), Document(text='goodbye, world')]),\n DocumentArray(\n [\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n text='hello, world',\n ),\n ]\n ),\n DocumentArray.from_files(\n f'{os.path.dirname(os.path.abspath(__file__))}/**/*.jpg'\n ),\n ],\n)\ndef test_docarray_preserve_original_inputs(make_flow, inputs):\n c = Client(server=f'grpc://0.0.0.0:{make_flow.port}')\n r = c.encode(inputs if not callable(inputs) else inputs())\n assert isinstance(r, DocumentArray)\n assert r.embeddings.shape\n assert r.contents == inputs.contents\n assert not r[0].tensor\n assert inputs[0] is r[0]\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n DocumentArray([Document(text='hello, world'), Document(text='goodbye, world')]),\n DocumentArray(\n [\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n text='hello, world',\n ),\n ]\n ),\n DocumentArray.from_files(\n f'{os.path.dirname(os.path.abspath(__file__))}/**/*.jpg'\n ),\n ],\n)\ndef test_docarray_traversal(make_flow, inputs):\n from jina import Client as _Client\n\n da = DocumentArray.empty(1)\n da[0].chunks = inputs\n\n c = _Client(host=f'grpc://0.0.0.0', port=make_flow.port)\n r1 = c.post(on='/', inputs=da, parameters={'traversal_paths': '@c'})\n assert isinstance(r1, DocumentArray)\n assert r1[0].chunks.embeddings.shape[0] == len(inputs)\n assert not r1[0].tensor\n assert not r1[0].blob\n assert not r1[0].chunks[0].tensor\n assert not r1[0].chunks[0].blob\n\n r2 = c.post(on='/', inputs=da, parameters={'access_paths': '@c'})\n assert isinstance(r2, DocumentArray)\n assert r2[0].chunks.embeddings.shape[0] == len(inputs)\n assert not r2[0].tensor\n assert not r2[0].blob\n assert not r2[0].chunks[0].tensor\n assert not r2[0].chunks[0].blob\n\n\n@pytest.mark.parametrize(\n 'inputs',\n [\n [Document(id=str(i), text='hello, world') for i in range(20)],\n DocumentArray([Document(id=str(i), text='hello, world') for i in range(20)]),\n ],\n)\ndef test_docarray_preserve_original_order(make_flow, inputs):\n c = Client(server=f'grpc://0.0.0.0:{make_flow.port}')\n r = c.encode(inputs if not callable(inputs) else inputs(), batch_size=1)\n assert isinstance(r, DocumentArray)\n for i in range(len(inputs)):\n assert inputs[i] is r[i]\n assert inputs[i].id == str(i)\n" }, { "path": "tests/test_tensorrt.py", "content": "import os\n\nimport pytest\nimport numpy as np\nfrom docarray import Document, DocumentArray\nfrom jina import Flow\n\nfrom clip_client.client import Client\n\n\n@pytest.mark.gpu\n@pytest.mark.parametrize(\n 'inputs',\n [\n [Document(text='hello, world'), Document(text='goodbye, world')],\n DocumentArray([Document(text='hello, world'), Document(text='goodbye, world')]),\n lambda: (Document(text='hello, world') for _ in range(10)),\n DocumentArray(\n [\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n Document(\n uri=f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg'\n ),\n Document(text='hello, world'),\n Document(\n uri=f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg'\n ).load_uri_to_image_tensor(),\n ]\n ),\n DocumentArray.from_files(\n f'{os.path.dirname(os.path.abspath(__file__))}/**/*.jpg'\n ),\n ],\n)\ndef test_docarray_inputs(make_trt_flow, inputs):\n c = Client(server=f'grpc://0.0.0.0:{make_trt_flow.port}')\n r = c.encode(inputs if not callable(inputs) else inputs())\n assert isinstance(r, DocumentArray)\n assert r.embeddings.shape\n if hasattr(inputs, '__len__'):\n assert inputs[0] is r[0]\n\n\n@pytest.mark.gpu\n@pytest.mark.asyncio\n@pytest.mark.parametrize(\n 'd',\n [\n Document(\n uri='https://clip-as-service.jina.ai/_static/favicon.png',\n matches=[Document(text='hello, world'), Document(text='goodbye, world')],\n ),\n Document(\n text='hello, world',\n matches=[\n Document(uri='https://clip-as-service.jina.ai/_static/favicon.png'),\n Document(\n uri=f'{os.path.dirname(os.path.abspath(__file__))}/img/00000.jpg'\n ),\n ],\n ),\n ],\n)\nasync def test_async_arank(make_trt_flow, d):\n c = Client(server=f'grpc://0.0.0.0:{make_trt_flow.port}')\n r = await c.arank([d])\n assert isinstance(r, DocumentArray)\n assert d is r[0]\n rv = r['@m', 'scores__clip_score__value']\n for v in rv:\n assert v is not None\n assert v > 0\n np.testing.assert_almost_equal(sum(rv), 1.0)\n\n rv = r['@m', 'scores__clip_score_cosine__value']\n for v in rv:\n assert v is not None\n assert -1.0 <= v <= 1.0\n" }, { "path": "tests/test_tokenization.py", "content": "import pytest\nfrom clip_server.model.tokenization import Tokenizer\n\n\n@pytest.mark.parametrize(\n 'name', ['ViT-L/14@336px', 'M-CLIP/XLM-Roberta-Large-Vit-B-32']\n)\ndef test_tokenizer_name(name):\n tokenizer = Tokenizer(name)\n\n result = tokenizer('hello world')\n assert result['input_ids'].shape == result['attention_mask'].shape\n assert result['input_ids'].shape[0] == 1\n\n result = tokenizer(['hello world', 'welcome to the world'])\n assert result['input_ids'].shape == result['attention_mask'].shape\n assert result['input_ids'].shape[0] == 2\n" } ]