Repository: alievk/avatarify-python Branch: master Commit: 862182cfbca5 Files: 37 Total size: 116.0 KB Directory structure: gitextract_8_53yfwn/ ├── .github/ │ └── ISSUE_TEMPLATE/ │ ├── bug-report.md │ └── discussion.md ├── .gitignore ├── .gitmodules ├── Dockerfile ├── LICENSE.md ├── README.md ├── afy/ │ ├── arguments.py │ ├── cam_fomm.py │ ├── camera_selector.py │ ├── networking.py │ ├── predictor_local.py │ ├── predictor_remote.py │ ├── predictor_worker.py │ ├── utils.py │ └── videocaptureasync.py ├── avatarify.ipynb ├── avatars/ │ └── .geetkeep ├── config.yaml ├── docs/ │ └── README.md ├── requirements.txt ├── requirements_client.txt ├── run.sh ├── run_mac.sh ├── run_windows.bat ├── scripts/ │ ├── create_virtual_camera.sh │ ├── download_data.sh │ ├── get_ngrok.sh │ ├── install.sh │ ├── install_docker.sh │ ├── install_mac.sh │ ├── install_windows.bat │ ├── open_tunnel_ngrok.sh │ ├── open_tunnel_ssh.sh │ ├── settings.sh │ └── settings_windows.bat └── var/ └── log/ └── .geetkeep ================================================ FILE CONTENTS ================================================ ================================================ FILE: .github/ISSUE_TEMPLATE/bug-report.md ================================================ --- name: Bug report about: Create a report to help us improve title: '' labels: '' assignees: '' --- **Describe the bug** **To Reproduce** **Info (please complete the following information):** - OS (e.g., Linux): - GPU model: - Any other relevant information: **Screenshots** **Logs** ================================================ FILE: .github/ISSUE_TEMPLATE/discussion.md ================================================ --- name: Discussion about: Discuss a new feature or improvement. title: '' labels: '' assignees: '' --- ================================================ FILE: .gitignore ================================================ vox-adv-cpk.pth.tar __pycache__ *.log ================================================ FILE: .gitmodules ================================================ ================================================ FILE: Dockerfile ================================================ FROM nvcr.io/nvidia/cuda:10.0-cudnn7-runtime-ubuntu18.04 RUN DEBIAN_FRONTEND=noninteractive apt-get -qq update \ && DEBIAN_FRONTEND=noninteractive apt-get -qqy install curl python3-pip python3-tk python3.7-dev ffmpeg git less nano libsm6 libxext6 libxrender-dev \ && rm -rf /var/lib/apt/lists/* # use python3.7 by default RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.6 1 \ && update-alternatives --install /usr/bin/python python /usr/bin/python3.7 2 \ && update-alternatives --set python /usr/bin/python3.7 \ && update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 1 \ && update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.7 2 \ && update-alternatives --set python3 /usr/bin/python3.7 \ && python -m pip install --upgrade setuptools pip wheel ARG PYTORCH_WHEEL="https://download.pytorch.org/whl/cu100/torch-1.0.0-cp37-cp37m-linux_x86_64.whl" ARG FACE_ALIGNMENT_GIT="git+https://github.com/1adrianb/face-alignment" ARG AVATARIFY_COMMIT="a300fcaadb6a6964e69d4a90db9e7d72bb87e791" ARG FOMM_COMMIT="efbe0a6f17b38360ff9a446fddfbb3ce5493534c" RUN git clone https://github.com/alievk/avatarify-python.git /app/avatarify && cd /app/avatarify && git checkout ${AVATARIFY_COMMIT} \ && git clone https://github.com/alievk/first-order-model.git /app/avatarify/fomm && cd /app/avatarify/fomm && git checkout ${FOMM_COMMIT} WORKDIR /app/avatarify RUN bash scripts/download_data.sh RUN pip3 install ${PYTORCH_WHEEL} -r requirements.txt \ && pip3 install ${PYTORCH_WHEEL} -r fomm/requirements.txt \ && rm -rf /root/.cache/pip ENV PYTHONPATH="/app/avatarify:/app/avatarify/fomm" EXPOSE 5557 EXPOSE 5558 CMD ["python3", "afy/cam_fomm.py", "--config", "fomm/config/vox-adv-256.yaml", "--checkpoint", "vox-adv-cpk.pth.tar", "--virt-cam", "9", "--relative", "--adapt_scale", "--is-worker"] ================================================ FILE: LICENSE.md ================================================ # Attribution-NonCommercial 4.0 International Creative Commons Corporation (“Creative Commons”) is not a law firm and does not provide legal services or legal advice. Distribution of Creative Commons public licenses does not create a lawyer-client or other relationship. Creative Commons makes its licenses and related information available on an “as-is” basis. Creative Commons gives no warranties regarding its licenses, any material licensed under their terms and conditions, or any related information. Creative Commons disclaims all liability for damages resulting from their use to the fullest extent possible. ### Using Creative Commons Public Licenses Creative Commons public licenses provide a standard set of terms and conditions that creators and other rights holders may use to share original works of authorship and other material subject to copyright and certain other rights specified in the public license below. The following considerations are for informational purposes only, are not exhaustive, and do not form part of our licenses. * __Considerations for licensors:__ Our public licenses are intended for use by those authorized to give the public permission to use material in ways otherwise restricted by copyright and certain other rights. Our licenses are irrevocable. Licensors should read and understand the terms and conditions of the license they choose before applying it. Licensors should also secure all rights necessary before applying our licenses so that the public can reuse the material as expected. Licensors should clearly mark any material not subject to the license. This includes other CC-licensed material, or material used under an exception or limitation to copyright. [More considerations for licensors](http://wiki.creativecommons.org/Considerations_for_licensors_and_licensees#Considerations_for_licensors). * __Considerations for the public:__ By using one of our public licenses, a licensor grants the public permission to use the licensed material under specified terms and conditions. If the licensor’s permission is not necessary for any reason–for example, because of any applicable exception or limitation to copyright–then that use is not regulated by the license. Our licenses grant only permissions under copyright and certain other rights that a licensor has authority to grant. Use of the licensed material may still be restricted for other reasons, including because others have copyright or other rights in the material. A licensor may make special requests, such as asking that all changes be marked or described. Although not required by our licenses, you are encouraged to respect those requests where reasonable. [More considerations for the public](http://wiki.creativecommons.org/Considerations_for_licensors_and_licensees#Considerations_for_licensees). ## Creative Commons Attribution-NonCommercial 4.0 International Public License By exercising the Licensed Rights (defined below), You accept and agree to be bound by the terms and conditions of this Creative Commons Attribution-NonCommercial 4.0 International Public License ("Public License"). To the extent this Public License may be interpreted as a contract, You are granted the Licensed Rights in consideration of Your acceptance of these terms and conditions, and the Licensor grants You such rights in consideration of benefits the Licensor receives from making the Licensed Material available under these terms and conditions. ### Section 1 – Definitions. a. __Adapted Material__ means material subject to Copyright and Similar Rights that is derived from or based upon the Licensed Material and in which the Licensed Material is translated, altered, arranged, transformed, or otherwise modified in a manner requiring permission under the Copyright and Similar Rights held by the Licensor. For purposes of this Public License, where the Licensed Material is a musical work, performance, or sound recording, Adapted Material is always produced where the Licensed Material is synched in timed relation with a moving image. b. __Adapter's License__ means the license You apply to Your Copyright and Similar Rights in Your contributions to Adapted Material in accordance with the terms and conditions of this Public License. c. __Copyright and Similar Rights__ means copyright and/or similar rights closely related to copyright including, without limitation, performance, broadcast, sound recording, and Sui Generis Database Rights, without regard to how the rights are labeled or categorized. For purposes of this Public License, the rights specified in Section 2(b)(1)-(2) are not Copyright and Similar Rights. d. __Effective Technological Measures__ means those measures that, in the absence of proper authority, may not be circumvented under laws fulfilling obligations under Article 11 of the WIPO Copyright Treaty adopted on December 20, 1996, and/or similar international agreements. e. __Exceptions and Limitations__ means fair use, fair dealing, and/or any other exception or limitation to Copyright and Similar Rights that applies to Your use of the Licensed Material. f. __Licensed Material__ means the artistic or literary work, database, or other material to which the Licensor applied this Public License. g. __Licensed Rights__ means the rights granted to You subject to the terms and conditions of this Public License, which are limited to all Copyright and Similar Rights that apply to Your use of the Licensed Material and that the Licensor has authority to license. h. __Licensor__ means the individual(s) or entity(ies) granting rights under this Public License. i. __NonCommercial__ means not primarily intended for or directed towards commercial advantage or monetary compensation. For purposes of this Public License, the exchange of the Licensed Material for other material subject to Copyright and Similar Rights by digital file-sharing or similar means is NonCommercial provided there is no payment of monetary compensation in connection with the exchange. j. __Share__ means to provide material to the public by any means or process that requires permission under the Licensed Rights, such as reproduction, public display, public performance, distribution, dissemination, communication, or importation, and to make material available to the public including in ways that members of the public may access the material from a place and at a time individually chosen by them. k. __Sui Generis Database Rights__ means rights other than copyright resulting from Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended and/or succeeded, as well as other essentially equivalent rights anywhere in the world. l. __You__ means the individual or entity exercising the Licensed Rights under this Public License. Your has a corresponding meaning. ### Section 2 – Scope. a. ___License grant.___ 1. Subject to the terms and conditions of this Public License, the Licensor hereby grants You a worldwide, royalty-free, non-sublicensable, non-exclusive, irrevocable license to exercise the Licensed Rights in the Licensed Material to: A. reproduce and Share the Licensed Material, in whole or in part, for NonCommercial purposes only; and B. produce, reproduce, and Share Adapted Material for NonCommercial purposes only. 2. __Exceptions and Limitations.__ For the avoidance of doubt, where Exceptions and Limitations apply to Your use, this Public License does not apply, and You do not need to comply with its terms and conditions. 3. __Term.__ The term of this Public License is specified in Section 6(a). 4. __Media and formats; technical modifications allowed.__ The Licensor authorizes You to exercise the Licensed Rights in all media and formats whether now known or hereafter created, and to make technical modifications necessary to do so. The Licensor waives and/or agrees not to assert any right or authority to forbid You from making technical modifications necessary to exercise the Licensed Rights, including technical modifications necessary to circumvent Effective Technological Measures. For purposes of this Public License, simply making modifications authorized by this Section 2(a)(4) never produces Adapted Material. 5. __Downstream recipients.__ A. __Offer from the Licensor – Licensed Material.__ Every recipient of the Licensed Material automatically receives an offer from the Licensor to exercise the Licensed Rights under the terms and conditions of this Public License. B. __No downstream restrictions.__ You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, the Licensed Material if doing so restricts exercise of the Licensed Rights by any recipient of the Licensed Material. 6. __No endorsement.__ Nothing in this Public License constitutes or may be construed as permission to assert or imply that You are, or that Your use of the Licensed Material is, connected with, or sponsored, endorsed, or granted official status by, the Licensor or others designated to receive attribution as provided in Section 3(a)(1)(A)(i). b. ___Other rights.___ 1. Moral rights, such as the right of integrity, are not licensed under this Public License, nor are publicity, privacy, and/or other similar personality rights; however, to the extent possible, the Licensor waives and/or agrees not to assert any such rights held by the Licensor to the limited extent necessary to allow You to exercise the Licensed Rights, but not otherwise. 2. Patent and trademark rights are not licensed under this Public License. 3. To the extent possible, the Licensor waives any right to collect royalties from You for the exercise of the Licensed Rights, whether directly or through a collecting society under any voluntary or waivable statutory or compulsory licensing scheme. In all other cases the Licensor expressly reserves any right to collect such royalties, including when the Licensed Material is used other than for NonCommercial purposes. ### Section 3 – License Conditions. Your exercise of the Licensed Rights is expressly made subject to the following conditions. a. ___Attribution.___ 1. If You Share the Licensed Material (including in modified form), You must: A. retain the following if it is supplied by the Licensor with the Licensed Material: i. identification of the creator(s) of the Licensed Material and any others designated to receive attribution, in any reasonable manner requested by the Licensor (including by pseudonym if designated); ii. a copyright notice; iii. a notice that refers to this Public License; iv. a notice that refers to the disclaimer of warranties; v. a URI or hyperlink to the Licensed Material to the extent reasonably practicable; B. indicate if You modified the Licensed Material and retain an indication of any previous modifications; and C. indicate the Licensed Material is licensed under this Public License, and include the text of, or the URI or hyperlink to, this Public License. 2. You may satisfy the conditions in Section 3(a)(1) in any reasonable manner based on the medium, means, and context in which You Share the Licensed Material. For example, it may be reasonable to satisfy the conditions by providing a URI or hyperlink to a resource that includes the required information. 3. If requested by the Licensor, You must remove any of the information required by Section 3(a)(1)(A) to the extent reasonably practicable. 4. If You Share Adapted Material You produce, the Adapter's License You apply must not prevent recipients of the Adapted Material from complying with this Public License. ### Section 4 – Sui Generis Database Rights. Where the Licensed Rights include Sui Generis Database Rights that apply to Your use of the Licensed Material: a. for the avoidance of doubt, Section 2(a)(1) grants You the right to extract, reuse, reproduce, and Share all or a substantial portion of the contents of the database for NonCommercial purposes only; b. if You include all or a substantial portion of the database contents in a database in which You have Sui Generis Database Rights, then the database in which You have Sui Generis Database Rights (but not its individual contents) is Adapted Material; and c. You must comply with the conditions in Section 3(a) if You Share all or a substantial portion of the contents of the database. For the avoidance of doubt, this Section 4 supplements and does not replace Your obligations under this Public License where the Licensed Rights include other Copyright and Similar Rights. ### Section 5 – Disclaimer of Warranties and Limitation of Liability. a. __Unless otherwise separately undertaken by the Licensor, to the extent possible, the Licensor offers the Licensed Material as-is and as-available, and makes no representations or warranties of any kind concerning the Licensed Material, whether express, implied, statutory, or other. This includes, without limitation, warranties of title, merchantability, fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. Where disclaimers of warranties are not allowed in full or in part, this disclaimer may not apply to You.__ b. __To the extent possible, in no event will the Licensor be liable to You on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this Public License or use of the Licensed Material, even if the Licensor has been advised of the possibility of such losses, costs, expenses, or damages. Where a limitation of liability is not allowed in full or in part, this limitation may not apply to You.__ c. The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability. ### Section 6 – Term and Termination. a. This Public License applies for the term of the Copyright and Similar Rights licensed here. However, if You fail to comply with this Public License, then Your rights under this Public License terminate automatically. b. Where Your right to use the Licensed Material has terminated under Section 6(a), it reinstates: 1. automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or 2. upon express reinstatement by the Licensor. For the avoidance of doubt, this Section 6(b) does not affect any right the Licensor may have to seek remedies for Your violations of this Public License. c. For the avoidance of doubt, the Licensor may also offer the Licensed Material under separate terms or conditions or stop distributing the Licensed Material at any time; however, doing so will not terminate this Public License. d. Sections 1, 5, 6, 7, and 8 survive termination of this Public License. ### Section 7 – Other Terms and Conditions. a. The Licensor shall not be bound by any additional or different terms or conditions communicated by You unless expressly agreed. b. Any arrangements, understandings, or agreements regarding the Licensed Material not stated herein are separate from and independent of the terms and conditions of this Public License. ### Section 8 – Interpretation. a. For the avoidance of doubt, this Public License does not, and shall not be interpreted to, reduce, limit, restrict, or impose conditions on any use of the Licensed Material that could lawfully be made without permission under this Public License. b. To the extent possible, if any provision of this Public License is deemed unenforceable, it shall be automatically reformed to the minimum extent necessary to make it enforceable. If the provision cannot be reformed, it shall be severed from this Public License without affecting the enforceability of the remaining terms and conditions. c. No term or condition of this Public License will be waived and no failure to comply consented to unless expressly agreed to by the Licensor. d. Nothing in this Public License constitutes or may be interpreted as a limitation upon, or waiver of, any privileges and immunities that apply to the Licensor or You, including from the legal processes of any jurisdiction or authority. > Creative Commons is not a party to its public licenses. Notwithstanding, Creative Commons may elect to apply one of its public licenses to material it publishes and in those instances will be considered the “Licensor.” Except for the limited purpose of indicating that material is shared under a Creative Commons public license or as otherwise permitted by the Creative Commons policies published at [creativecommons.org/policies](http://creativecommons.org/policies), Creative Commons does not authorize the use of the trademark “Creative Commons” or any other trademark or logo of Creative Commons without its prior written consent including, without limitation, in connection with any unauthorized modifications to any of its public licenses or any other arrangements, understandings, or agreements concerning use of licensed material. For the avoidance of doubt, this paragraph does not form part of the public licenses. > > Creative Commons may be contacted at creativecommons.org ================================================ FILE: README.md ================================================ ![](docs/mona.gif) # Avatarify Python Photorealistic avatars for video-conferencing. Avatarify Python requires manually downloading and installing some dependencies, and is therefore best suited for users who have some experience with command line applications. [Avatarify Desktop](https://github.com/alievk/avatarify-desktop), which aims to be easier to install and use, is recommended for most users. If you still want to use Avatarify Python, proceed to the [install instructions](docs/). Based on [First Order Motion Model](https://github.com/AliaksandrSiarohin/first-order-model). ## News - **7 Mar 2021.** Renamed project to Avatarify Python to distinguish it from other versions of Avatarify - **14 December 2020.** Released Avatarify Desktop. Check it out [here](https://github.com/alievk/avatarify-desktop). - **11 July 2020.** Added Docker support. Now you can run Avatarify from Docker on [Linux](https://github.com/alievk/avatarify-python/blob/master/docs/README.md#docker). Thanks to [mikaelhg](https://github.com/mikaelhg) and [mintmaker](https://github.com/mintmaker) for contribution! - **22 May 2020.** Added [Google Colab](https://colab.research.google.com/github/alievk/avatarify/blob/master/avatarify.ipynb) mode. Now you can run Avatarify on any computer without GPU! - **7 May 2020.** Added remote GPU support for all platforms (based on [mynameisfiber's](https://github.com/mynameisfiber) solution). [Demo](https://youtu.be/3Dz_bUIPYFM). Deployment [instructions](https://github.com/alievk/avatarify-python/wiki/Remote-GPU). - **24 April 2020.** Added Windows installation [tutorial](https://www.youtube.com/watch?v=lym9ANVb120). - **17 April 2020.** Created Slack community. Please join via [invitation link](https://join.slack.com/t/avatarify/shared_invite/zt-dyoqy8tc-~4U2ObQ6WoxuwSaWKKVOgg). - **15 April 2020.** Added [StyleGAN-generated](https://www.thispersondoesnotexist.com) avatars. Just press `Q` and now you drive a person that never existed. Every time you push the button – new avatar is sampled. - **13 April 2020.** Added Windows support (kudos to [9of9](https://github.com/9of9)). ## Avatarify apps We have deployed Avatarify on iOS and Android devices using our proprietary inference engine. The iOS version features the Life mode for recording animations in real time. However, the Life mode is not available on Android devices due to the diversity of the devices we have to support. [drawing](https://apps.apple.com/app/apple-store/id1512669147?pt=121960189&ct=GitHub&mt=8) [drawing](https://play.google.com/store/apps/details?id=com.avatarify.android) ================================================ FILE: afy/arguments.py ================================================ from argparse import ArgumentParser parser = ArgumentParser() parser.add_argument("--config", help="path to config") parser.add_argument("--checkpoint", default='vox-cpk.pth.tar', help="path to checkpoint to restore") parser.add_argument("--relative", dest="relative", action="store_true", help="use relative or absolute keypoint coordinates") parser.add_argument("--adapt_scale", dest="adapt_scale", action="store_true", help="adapt movement scale based on convex hull of keypoints") parser.add_argument("--no-pad", dest="no_pad", action="store_true", help="don't pad output image") parser.add_argument("--enc_downscale", default=1, type=float, help="Downscale factor for encoder input. Improves performance with cost of quality.") parser.add_argument("--virt-cam", type=int, default=0, help="Virtualcam device ID") parser.add_argument("--no-stream", action="store_true", help="On Linux, force no streaming") parser.add_argument("--verbose", action="store_true", help="Print additional information") parser.add_argument("--hide-rect", action="store_true", default=False, help="Hide the helper rectangle in preview window") parser.add_argument("--avatars", default="./avatars", help="path to avatars directory") parser.add_argument("--is-worker", action="store_true", help="Whether to run this process as a remote GPU worker") parser.add_argument("--is-client", action="store_true", help="Whether to run this process as a client") parser.add_argument("--in-port", type=int, default=5557, help="Remote worker input port") parser.add_argument("--out-port", type=int, default=5558, help="Remote worker output port") parser.add_argument("--in-addr", type=str, default=None, help="Socket address for incoming messages, like example.com:5557") parser.add_argument("--out-addr", type=str, default=None, help="Socker address for outcoming messages, like example.com:5558") parser.add_argument("--jpg_quality", type=int, default=95, help="Jpeg copression quality for image transmission") parser.set_defaults(relative=False) parser.set_defaults(adapt_scale=False) parser.set_defaults(no_pad=False) opt = parser.parse_args() if opt.is_client and (opt.in_addr is None or opt.out_addr is None): raise ValueError("You have to set --in-addr and --out-addr") ================================================ FILE: afy/cam_fomm.py ================================================ import os, sys from sys import platform as _platform import glob import yaml import time import requests import numpy as np import cv2 from afy.videocaptureasync import VideoCaptureAsync from afy.arguments import opt from afy.utils import info, Once, Tee, crop, pad_img, resize, TicToc import afy.camera_selector as cam_selector log = Tee('./var/log/cam_fomm.log') # Where to split an array from face_alignment to separate each landmark LANDMARK_SLICE_ARRAY = np.array([17, 22, 27, 31, 36, 42, 48, 60]) if _platform == 'darwin': if not opt.is_client: info('\nOnly remote GPU mode is supported for Mac (use --is-client and --connect options to connect to the server)') info('Standalone version will be available lately!\n') exit() def is_new_frame_better(source, driving, predictor): global avatar_kp global display_string if avatar_kp is None: display_string = "No face detected in avatar." return False if predictor.get_start_frame() is None: display_string = "No frame to compare to." return True driving_smaller = resize(driving, (128, 128))[..., :3] new_kp = predictor.get_frame_kp(driving) if new_kp is not None: new_norm = (np.abs(avatar_kp - new_kp) ** 2).sum() old_norm = (np.abs(avatar_kp - predictor.get_start_frame_kp()) ** 2).sum() out_string = "{0} : {1}".format(int(new_norm * 100), int(old_norm * 100)) display_string = out_string log(out_string) return new_norm < old_norm else: display_string = "No face found!" return False def load_stylegan_avatar(): url = "https://thispersondoesnotexist.com/image" r = requests.get(url, headers={'User-Agent': "My User Agent 1.0"}).content image = np.frombuffer(r, np.uint8) image = cv2.imdecode(image, cv2.IMREAD_COLOR) image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) image = resize(image, (IMG_SIZE, IMG_SIZE)) return image def load_images(IMG_SIZE = 256): avatars = [] filenames = [] images_list = sorted(glob.glob(f'{opt.avatars}/*')) for i, f in enumerate(images_list): if f.endswith('.jpg') or f.endswith('.jpeg') or f.endswith('.png'): img = cv2.imread(f) if img is None: log("Failed to open image: {}".format(f)) continue if img.ndim == 2: img = np.tile(img[..., None], [1, 1, 3]) img = img[..., :3][..., ::-1] img = resize(img, (IMG_SIZE, IMG_SIZE)) avatars.append(img) filenames.append(f) return avatars, filenames def change_avatar(predictor, new_avatar): global avatar, avatar_kp, kp_source avatar_kp = predictor.get_frame_kp(new_avatar) kp_source = None avatar = new_avatar predictor.set_source_image(avatar) def draw_rect(img, rw=0.6, rh=0.8, color=(255, 0, 0), thickness=2): h, w = img.shape[:2] l = w * (1 - rw) // 2 r = w - l u = h * (1 - rh) // 2 d = h - u img = cv2.rectangle(img, (int(l), int(u)), (int(r), int(d)), color, thickness) def kp_to_pixels(arr): '''Convert normalized landmark locations to screen pixels''' return ((arr + 1) * 127).astype(np.int32) def draw_face_landmarks(img, face_kp, color=(20, 80, 255)): if face_kp is not None: img = cv2.polylines(img, np.split(kp_to_pixels(face_kp), LANDMARK_SLICE_ARRAY), False, color) def print_help(): info('\n\n=== Control keys ===') info('1-9: Change avatar') for i, fname in enumerate(avatar_names): key = i + 1 name = fname.split('/')[-1] info(f'{key}: {name}') info('W: Zoom camera in') info('S: Zoom camera out') info('A: Previous avatar in folder') info('D: Next avatar in folder') info('Q: Get random avatar') info('X: Calibrate face pose') info('I: Show FPS') info('ESC: Quit') info('\nFull key list: https://github.com/alievk/avatarify#controls') info('\n\n') def draw_fps(frame, fps, timing, x0=10, y0=20, ystep=30, fontsz=0.5, color=(255, 255, 255)): frame = frame.copy() cv2.putText(frame, f"FPS: {fps:.1f}", (x0, y0 + ystep * 0), 0, fontsz * IMG_SIZE / 256, color, 1) cv2.putText(frame, f"Model time (ms): {timing['predict']:.1f}", (x0, y0 + ystep * 1), 0, fontsz * IMG_SIZE / 256, color, 1) cv2.putText(frame, f"Preproc time (ms): {timing['preproc']:.1f}", (x0, y0 + ystep * 2), 0, fontsz * IMG_SIZE / 256, color, 1) cv2.putText(frame, f"Postproc time (ms): {timing['postproc']:.1f}", (x0, y0 + ystep * 3), 0, fontsz * IMG_SIZE / 256, color, 1) return frame def draw_landmark_text(frame, thk=2, fontsz=0.5, color=(0, 0, 255)): frame = frame.copy() cv2.putText(frame, "ALIGN FACES", (60, 20), 0, fontsz * IMG_SIZE / 255, color, thk) cv2.putText(frame, "THEN PRESS X", (60, 245), 0, fontsz * IMG_SIZE / 255, color, thk) return frame def draw_calib_text(frame, thk=2, fontsz=0.5, color=(0, 0, 255)): frame = frame.copy() cv2.putText(frame, "FIT FACE IN RECTANGLE", (40, 20), 0, fontsz * IMG_SIZE / 255, color, thk) cv2.putText(frame, "W - ZOOM IN", (60, 40), 0, fontsz * IMG_SIZE / 255, color, thk) cv2.putText(frame, "S - ZOOM OUT", (60, 60), 0, fontsz * IMG_SIZE / 255, color, thk) cv2.putText(frame, "THEN PRESS X", (60, 245), 0, fontsz * IMG_SIZE / 255, color, thk) return frame def select_camera(config): cam_config = config['cam_config'] cam_id = None if os.path.isfile(cam_config): with open(cam_config, 'r') as f: cam_config = yaml.load(f, Loader=yaml.FullLoader) cam_id = cam_config['cam_id'] else: cam_frames = cam_selector.query_cameras(config['query_n_cams']) if cam_frames: if len(cam_frames) == 1: cam_id = list(cam_frames)[0] else: cam_id = cam_selector.select_camera(cam_frames, window="CLICK ON YOUR CAMERA") log(f"Selected camera {cam_id}") with open(cam_config, 'w') as f: yaml.dump({'cam_id': cam_id}, f) else: log("No cameras are available") return cam_id if __name__ == "__main__": with open('config.yaml', 'r') as f: config = yaml.load(f, Loader=yaml.FullLoader) global display_string display_string = "" IMG_SIZE = 256 log('Loading Predictor') predictor_args = { 'config_path': opt.config, 'checkpoint_path': opt.checkpoint, 'relative': opt.relative, 'adapt_movement_scale': opt.adapt_scale, 'enc_downscale': opt.enc_downscale } if opt.is_worker: from afy import predictor_worker predictor_worker.run_worker(opt.in_port, opt.out_port) sys.exit(0) elif opt.is_client: from afy import predictor_remote try: predictor = predictor_remote.PredictorRemote( in_addr=opt.in_addr, out_addr=opt.out_addr, **predictor_args ) except ConnectionError as err: log(err) sys.exit(1) predictor.start() else: from afy import predictor_local predictor = predictor_local.PredictorLocal( **predictor_args ) cam_id = select_camera(config) if cam_id is None: exit(1) cap = VideoCaptureAsync(cam_id) cap.start() avatars, avatar_names = load_images() enable_vcam = not opt.no_stream ret, frame = cap.read() stream_img_size = frame.shape[1], frame.shape[0] if enable_vcam: if _platform in ['linux', 'linux2']: try: import pyfakewebcam except ImportError: log("pyfakewebcam is not installed.") exit(1) stream = pyfakewebcam.FakeWebcam(f'/dev/video{opt.virt_cam}', *stream_img_size) else: enable_vcam = False # log("Virtual camera is supported only on Linux.") # if not enable_vcam: # log("Virtual camera streaming will be disabled.") cur_ava = 0 avatar = None change_avatar(predictor, avatars[cur_ava]) passthrough = False cv2.namedWindow('cam', cv2.WINDOW_GUI_NORMAL) cv2.moveWindow('cam', 500, 250) frame_proportion = 0.9 frame_offset_x = 0 frame_offset_y = 0 overlay_alpha = 0.0 preview_flip = False output_flip = False find_keyframe = False is_calibrated = False show_landmarks = False fps_hist = [] fps = 0 show_fps = False print_help() try: while True: tt = TicToc() timing = { 'preproc': 0, 'predict': 0, 'postproc': 0 } green_overlay = False tt.tic() ret, frame = cap.read() if not ret: log("Can't receive frame (stream end?). Exiting ...") break frame = frame[..., ::-1] frame_orig = frame.copy() frame, (frame_offset_x, frame_offset_y) = crop(frame, p=frame_proportion, offset_x=frame_offset_x, offset_y=frame_offset_y) frame = resize(frame, (IMG_SIZE, IMG_SIZE))[..., :3] if find_keyframe: if is_new_frame_better(avatar, frame, predictor): log("Taking new frame!") green_overlay = True predictor.reset_frames() timing['preproc'] = tt.toc() if passthrough: out = frame elif is_calibrated: tt.tic() out = predictor.predict(frame) if out is None: log('predict returned None') timing['predict'] = tt.toc() else: out = None tt.tic() key = cv2.waitKey(1) if cv2.getWindowProperty('cam', cv2.WND_PROP_VISIBLE) < 1.0: break elif is_calibrated and cv2.getWindowProperty('avatarify', cv2.WND_PROP_VISIBLE) < 1.0: break if key == 27: # ESC break elif key == ord('d'): cur_ava += 1 if cur_ava >= len(avatars): cur_ava = 0 passthrough = False change_avatar(predictor, avatars[cur_ava]) elif key == ord('a'): cur_ava -= 1 if cur_ava < 0: cur_ava = len(avatars) - 1 passthrough = False change_avatar(predictor, avatars[cur_ava]) elif key == ord('w'): frame_proportion -= 0.05 frame_proportion = max(frame_proportion, 0.1) elif key == ord('s'): frame_proportion += 0.05 frame_proportion = min(frame_proportion, 1.0) elif key == ord('H'): frame_offset_x -= 1 elif key == ord('h'): frame_offset_x -= 5 elif key == ord('K'): frame_offset_x += 1 elif key == ord('k'): frame_offset_x += 5 elif key == ord('J'): frame_offset_y -= 1 elif key == ord('j'): frame_offset_y -= 5 elif key == ord('U'): frame_offset_y += 1 elif key == ord('u'): frame_offset_y += 5 elif key == ord('Z'): frame_offset_x = 0 frame_offset_y = 0 frame_proportion = 0.9 elif key == ord('x'): predictor.reset_frames() if not is_calibrated: cv2.namedWindow('avatarify', cv2.WINDOW_GUI_NORMAL) cv2.moveWindow('avatarify', 600, 250) is_calibrated = True show_landmarks = False elif key == ord('z'): overlay_alpha = max(overlay_alpha - 0.1, 0.0) elif key == ord('c'): overlay_alpha = min(overlay_alpha + 0.1, 1.0) elif key == ord('r'): preview_flip = not preview_flip elif key == ord('t'): output_flip = not output_flip elif key == ord('f'): find_keyframe = not find_keyframe elif key == ord('o'): show_landmarks = not show_landmarks elif key == ord('q'): try: log('Loading StyleGAN avatar...') avatar = load_stylegan_avatar() passthrough = False change_avatar(predictor, avatar) except: log('Failed to load StyleGAN avatar') elif key == ord('l'): try: log('Reloading avatars...') avatars, avatar_names = load_images() passthrough = False log("Images reloaded") except: log('Image reload failed') elif key == ord('i'): show_fps = not show_fps elif 48 < key < 58: cur_ava = min(key - 49, len(avatars) - 1) passthrough = False change_avatar(predictor, avatars[cur_ava]) elif key == 48: passthrough = not passthrough elif key != -1: log(key) if overlay_alpha > 0: preview_frame = cv2.addWeighted( avatar, overlay_alpha, frame, 1.0 - overlay_alpha, 0.0) else: preview_frame = frame.copy() if show_landmarks: # Dim the background to make it easier to see the landmarks preview_frame = cv2.convertScaleAbs(preview_frame, alpha=0.5, beta=0.0) draw_face_landmarks(preview_frame, avatar_kp, (200, 20, 10)) frame_kp = predictor.get_frame_kp(frame) draw_face_landmarks(preview_frame, frame_kp) if preview_flip: preview_frame = cv2.flip(preview_frame, 1) if green_overlay: green_alpha = 0.8 overlay = preview_frame.copy() overlay[:] = (0, 255, 0) preview_frame = cv2.addWeighted( preview_frame, green_alpha, overlay, 1.0 - green_alpha, 0.0) timing['postproc'] = tt.toc() if find_keyframe: preview_frame = cv2.putText(preview_frame, display_string, (10, 220), 0, 0.5 * IMG_SIZE / 256, (255, 255, 255), 1) if show_fps: preview_frame = draw_fps(preview_frame, fps, timing) if not is_calibrated: preview_frame = draw_calib_text(preview_frame) elif show_landmarks: preview_frame = draw_landmark_text(preview_frame) if not opt.hide_rect: draw_rect(preview_frame) cv2.imshow('cam', preview_frame[..., ::-1]) if out is not None: if not opt.no_pad: out = pad_img(out, stream_img_size) if output_flip: out = cv2.flip(out, 1) if enable_vcam: out = resize(out, stream_img_size) stream.schedule_frame(out) cv2.imshow('avatarify', out[..., ::-1]) fps_hist.append(tt.toc(total=True)) if len(fps_hist) == 10: fps = 10 / (sum(fps_hist) / 1000) fps_hist = [] except KeyboardInterrupt: log("main: user interrupt") log("stopping camera") cap.stop() cv2.destroyAllWindows() if opt.is_client: log("stopping remote predictor") predictor.stop() log("main: exit") ================================================ FILE: afy/camera_selector.py ================================================ import cv2 import numpy as np import yaml from afy.utils import log g_selected_cam = None def query_cameras(n_cams): cam_frames = {} cap = None for camid in range(n_cams): log(f"Trying camera with id {camid}") cap = cv2.VideoCapture(camid) if not cap.isOpened(): log(f"Camera with id {camid} is not available") continue ret, frame = cap.read() if not ret or frame is None: log(f"Could not read from camera with id {camid}") cap.release() continue for i in range(10): ret, frame = cap.read() cam_frames[camid] = frame.copy() cap.release() return cam_frames def make_grid(images, cell_size=(320, 240), cols=2): w0, h0 = cell_size _rows = len(images) // cols + int(len(images) % cols) _cols = min(len(images), cols) grid = np.zeros((h0 * _rows, w0 * _cols, 3), dtype=np.uint8) for i, (camid, img) in enumerate(images.items()): img = cv2.resize(img, (w0, h0)) # add rect img = cv2.rectangle(img, (1, 1), (w0 - 1, h0 - 1), (0, 0, 255), 2) # add id img = cv2.putText(img, f'Camera {camid}', (10, 30), 0, 1, (0, 255, 0), 2) c = i % cols r = i // cols grid[r * h0:(r + 1) * h0, c * w0:(c + 1) * w0] = img[..., :3] return grid def mouse_callback(event, x, y, flags, userdata): global g_selected_cam if event == 1: cell_size, grid_cols, cam_frames = userdata c = x // cell_size[0] r = y // cell_size[1] camid = r * grid_cols + c if camid < len(cam_frames): g_selected_cam = camid def select_camera(cam_frames, window="Camera selector"): cell_size = 320, 240 grid_cols = 2 grid = make_grid(cam_frames, cols=grid_cols) # to fit the text if only one cam available if grid.shape[1] == 320: cell_size = 640, 480 grid = cv2.resize(grid, cell_size) cv2.putText(grid, f'Click on the web camera to use', (10, grid.shape[0] - 30), 0, 0.7, (200, 200, 200), 2) cv2.namedWindow(window) cv2.setMouseCallback(window, mouse_callback, (cell_size, grid_cols, cam_frames)) cv2.imshow(window, grid) while True: key = cv2.waitKey(10) if g_selected_cam is not None: break if key == 27: break cv2.destroyAllWindows() if g_selected_cam is not None: return list(cam_frames)[g_selected_cam] else: return list(cam_frames)[0] if __name__ == '__main__': with open('config.yaml', 'r') as f: config = yaml.load(f, Loader=yaml.FullLoader) cam_frames = query_cameras(config['query_n_cams']) if cam_frames: selected_cam = select_camera(cam_frames) print(f"Selected camera {selected_cam}") else: log("No cameras are available") ================================================ FILE: afy/networking.py ================================================ import zmq import numpy as np import msgpack import msgpack_numpy as m m.patch() from afy.utils import log def check_connection(socket, timeout=1000): old_rcvtimeo = socket.RCVTIMEO socket.RCVTIMEO = timeout try: data = msgpack.packb(([], {})) socket.send_data('hello', data) attr_recv, data_recv = socket.recv_data() response = msgpack.unpackb(data_recv) except zmq.error.Again: return False finally: socket.RCVTIMEO = old_rcvtimeo log(f"Response to hello is {response}") return response == 'OK' class SerializingSocket(zmq.Socket): """Numpy array serialization methods. Based on https://github.com/jeffbass/imagezmq/blob/master/imagezmq/imagezmq.py#L291 Used for sending / receiving OpenCV images, which are Numpy arrays. Also used for sending / receiving jpg compressed OpenCV images. """ def send_array(self, A, msg='NoName', flags=0, copy=True, track=False): """Sends a numpy array with metadata and text message. Sends a numpy array with the metadata necessary for reconstructing the array (dtype,shape). Also sends a text msg, often the array or image name. Arguments: A: numpy array or OpenCV image. msg: (optional) array name, image name or text message. flags: (optional) zmq flags. copy: (optional) zmq copy flag. track: (optional) zmq track flag. """ md = dict( msg=msg, dtype=str(A.dtype), shape=A.shape, ) self.send_json(md, flags | zmq.SNDMORE) return self.send(A, flags, copy=copy, track=track) def send_data(self, msg='NoName', data=b'00', flags=0, copy=True, track=False): """Send a jpg buffer with a text message. Sends a jpg bytestring of an OpenCV image. Also sends text msg, often the image name. Arguments: msg: image name or text message. data: binary data to be sent. flags: (optional) zmq flags. copy: (optional) zmq copy flag. track: (optional) zmq track flag. """ md = dict(msg=msg, ) self.send_json(md, flags | zmq.SNDMORE) return self.send(data, flags, copy=copy, track=track) def recv_array(self, flags=0, copy=True, track=False): """Receives a numpy array with metadata and text message. Receives a numpy array with the metadata necessary for reconstructing the array (dtype,shape). Returns the array and a text msg, often the array or image name. Arguments: flags: (optional) zmq flags. copy: (optional) zmq copy flag. track: (optional) zmq track flag. Returns: msg: image name or text message. A: numpy array or OpenCV image reconstructed with dtype and shape. """ md = self.recv_json(flags=flags) msg = self.recv(flags=flags, copy=copy, track=track) A = np.frombuffer(msg, dtype=md['dtype']) return (md['msg'], A.reshape(md['shape'])) def recv_data(self, flags=0, copy=True, track=False): """Receives a jpg buffer and a text msg. Receives a jpg bytestring of an OpenCV image. Also receives a text msg, often the image name. Arguments: flags: (optional) zmq flags. copy: (optional) zmq copy flag. track: (optional) zmq track flag. Returns: msg: image name or text message. data: bytestring, containing data. """ md = self.recv_json(flags=flags) # metadata text data = self.recv(flags=flags, copy=copy, track=track) return (md['msg'], data) class SerializingContext(zmq.Context): _socket_class = SerializingSocket ================================================ FILE: afy/predictor_local.py ================================================ from scipy.spatial import ConvexHull import torch import yaml from modules.keypoint_detector import KPDetector from modules.generator_optim import OcclusionAwareGenerator from sync_batchnorm import DataParallelWithCallback import numpy as np import face_alignment def normalize_kp(kp_source, kp_driving, kp_driving_initial, adapt_movement_scale=False, use_relative_movement=False, use_relative_jacobian=False): if adapt_movement_scale: source_area = ConvexHull(kp_source['value'][0].data.cpu().numpy()).volume driving_area = ConvexHull(kp_driving_initial['value'][0].data.cpu().numpy()).volume adapt_movement_scale = np.sqrt(source_area) / np.sqrt(driving_area) else: adapt_movement_scale = 1 kp_new = {k: v for k, v in kp_driving.items()} if use_relative_movement: kp_value_diff = (kp_driving['value'] - kp_driving_initial['value']) kp_value_diff *= adapt_movement_scale kp_new['value'] = kp_value_diff + kp_source['value'] if use_relative_jacobian: jacobian_diff = torch.matmul(kp_driving['jacobian'], torch.inverse(kp_driving_initial['jacobian'])) kp_new['jacobian'] = torch.matmul(jacobian_diff, kp_source['jacobian']) return kp_new def to_tensor(a): return torch.tensor(a[np.newaxis].astype(np.float32)).permute(0, 3, 1, 2) / 255 class PredictorLocal: def __init__(self, config_path, checkpoint_path, relative=False, adapt_movement_scale=False, device=None, enc_downscale=1): self.device = device or ('cuda' if torch.cuda.is_available() else 'cpu') self.relative = relative self.adapt_movement_scale = adapt_movement_scale self.start_frame = None self.start_frame_kp = None self.kp_driving_initial = None self.config_path = config_path self.checkpoint_path = checkpoint_path self.generator, self.kp_detector = self.load_checkpoints() self.fa = face_alignment.FaceAlignment(face_alignment.LandmarksType._2D, flip_input=True, device=self.device) self.source = None self.kp_source = None self.enc_downscale = enc_downscale def load_checkpoints(self): with open(self.config_path) as f: config = yaml.load(f, Loader=yaml.FullLoader) generator = OcclusionAwareGenerator(**config['model_params']['generator_params'], **config['model_params']['common_params']) generator.to(self.device) kp_detector = KPDetector(**config['model_params']['kp_detector_params'], **config['model_params']['common_params']) kp_detector.to(self.device) checkpoint = torch.load(self.checkpoint_path, map_location=self.device) generator.load_state_dict(checkpoint['generator']) kp_detector.load_state_dict(checkpoint['kp_detector']) generator.eval() kp_detector.eval() return generator, kp_detector def reset_frames(self): self.kp_driving_initial = None def set_source_image(self, source_image): self.source = to_tensor(source_image).to(self.device) self.kp_source = self.kp_detector(self.source) if self.enc_downscale > 1: h, w = int(self.source.shape[2] / self.enc_downscale), int(self.source.shape[3] / self.enc_downscale) source_enc = torch.nn.functional.interpolate(self.source, size=(h, w), mode='bilinear') else: source_enc = self.source self.generator.encode_source(source_enc) def predict(self, driving_frame): assert self.kp_source is not None, "call set_source_image()" with torch.no_grad(): driving = to_tensor(driving_frame).to(self.device) if self.kp_driving_initial is None: self.kp_driving_initial = self.kp_detector(driving) self.start_frame = driving_frame.copy() self.start_frame_kp = self.get_frame_kp(driving_frame) kp_driving = self.kp_detector(driving) kp_norm = normalize_kp(kp_source=self.kp_source, kp_driving=kp_driving, kp_driving_initial=self.kp_driving_initial, use_relative_movement=self.relative, use_relative_jacobian=self.relative, adapt_movement_scale=self.adapt_movement_scale) out = self.generator(self.source, kp_source=self.kp_source, kp_driving=kp_norm) out = np.transpose(out['prediction'].data.cpu().numpy(), [0, 2, 3, 1])[0] out = (np.clip(out, 0, 1) * 255).astype(np.uint8) return out def get_frame_kp(self, image): kp_landmarks = self.fa.get_landmarks(image) if kp_landmarks: kp_image = kp_landmarks[0] kp_image = self.normalize_alignment_kp(kp_image) return kp_image else: return None @staticmethod def normalize_alignment_kp(kp): kp = kp - kp.mean(axis=0, keepdims=True) area = ConvexHull(kp[:, :2]).volume area = np.sqrt(area) kp[:, :2] = kp[:, :2] / area return kp def get_start_frame(self): return self.start_frame def get_start_frame_kp(self): return self.start_frame_kp ================================================ FILE: afy/predictor_remote.py ================================================ from arguments import opt from networking import SerializingContext, check_connection from utils import Logger, TicToc, AccumDict, Once import multiprocessing as mp import queue import cv2 import numpy as np import zmq import msgpack import msgpack_numpy as m m.patch() PUT_TIMEOUT = 0.1 # s GET_TIMEOUT = 0.1 # s RECV_TIMEOUT = 1000 # ms QUEUE_SIZE = 100 class PredictorRemote: def __init__(self, *args, in_addr=None, out_addr=None, **kwargs): self.in_addr = in_addr self.out_addr = out_addr self.predictor_args = (args, kwargs) self.timing = AccumDict() self.log = Logger('./var/log/predictor_remote.log', verbose=opt.verbose) self.send_queue = mp.Queue(QUEUE_SIZE) self.recv_queue = mp.Queue(QUEUE_SIZE) self.worker_alive = mp.Value('i', 0) self.send_process = mp.Process( target=self.send_worker, args=(self.in_addr, self.send_queue, self.worker_alive), name="send_process" ) self.recv_process = mp.Process( target=self.recv_worker, args=(self.out_addr, self.recv_queue, self.worker_alive), name="recv_process" ) self._i_msg = -1 def start(self): self.worker_alive.value = 1 self.send_process.start() self.recv_process.start() self.init_remote_worker() def stop(self): self.worker_alive.value = 0 self.log("join worker processes...") self.send_process.join(timeout=5) self.recv_process.join(timeout=5) self.send_process.terminate() self.recv_process.terminate() def init_remote_worker(self): return self._send_recv_async('__init__', self.predictor_args, critical=True) def __getattr__(self, attr): is_critical = attr != 'predict' return lambda *args, **kwargs: self._send_recv_async(attr, (args, kwargs), critical=is_critical) def _send_recv_async(self, method, args, critical): self._i_msg += 1 args, kwargs = args tt = TicToc() tt.tic() if method == 'predict': image = args[0] assert isinstance(image, np.ndarray), 'Expected image' ret_code, data = cv2.imencode(".jpg", image, [int(cv2.IMWRITE_JPEG_QUALITY), opt.jpg_quality]) else: data = msgpack.packb((args, kwargs)) self.timing.add('PACK', tt.toc()) meta = { 'name': method, 'critical': critical, 'id': self._i_msg } self.log("send", meta) if critical: self.send_queue.put((meta, data)) while True: meta_recv, data_recv = self.recv_queue.get() if meta_recv == meta: break else: try: # TODO: find good timeout self.send_queue.put((meta, data), timeout=PUT_TIMEOUT) except queue.Full: self.log('send_queue is full') try: meta_recv, data_recv = self.recv_queue.get(timeout=GET_TIMEOUT) except queue.Empty: self.log('recv_queue is empty') return None self.log("recv", meta_recv) tt.tic() if meta_recv['name'] == 'predict': result = cv2.imdecode(np.frombuffer(data_recv, dtype='uint8'), -1) else: result = msgpack.unpackb(data_recv) self.timing.add('UNPACK', tt.toc()) if opt.verbose: Once(self.timing, per=1) return result @staticmethod def send_worker(address, send_queue, worker_alive): timing = AccumDict() log = Logger('./var/log/send_worker.log', opt.verbose) ctx = SerializingContext() sender = ctx.socket(zmq.PUSH) sender.connect(address) log(f"Sending to {address}") try: while worker_alive.value: tt = TicToc() try: msg = send_queue.get(timeout=GET_TIMEOUT) except queue.Empty: continue tt.tic() sender.send_data(*msg) timing.add('SEND', tt.toc()) if opt.verbose: Once(timing, log, per=1) except KeyboardInterrupt: log("send_worker: user interrupt") finally: worker_alive.value = 0 sender.disconnect(address) sender.close() ctx.destroy() log("send_worker exit") @staticmethod def recv_worker(address, recv_queue, worker_alive): timing = AccumDict() log = Logger('./var/log/recv_worker.log') ctx = SerializingContext() receiver = ctx.socket(zmq.PULL) receiver.connect(address) receiver.RCVTIMEO = RECV_TIMEOUT log(f"Receiving from {address}") try: while worker_alive.value: tt = TicToc() try: tt.tic() msg = receiver.recv_data() timing.add('RECV', tt.toc()) except zmq.error.Again: continue try: recv_queue.put(msg, timeout=PUT_TIMEOUT) except queue.Full: log('recv_queue full') continue if opt.verbose: Once(timing, log, per=1) except KeyboardInterrupt: log("recv_worker: user interrupt") finally: worker_alive.value = 0 receiver.disconnect(address) receiver.close() ctx.destroy() log("recv_worker exit") ================================================ FILE: afy/predictor_worker.py ================================================ from predictor_local import PredictorLocal from arguments import opt from networking import SerializingContext, check_connection from utils import Logger, TicToc, AccumDict, Once import cv2 import numpy as np import zmq import msgpack import msgpack_numpy as m m.patch() import queue import multiprocessing as mp import traceback import time PUT_TIMEOUT = 1 # s GET_TIMEOUT = 1 # s RECV_TIMEOUT = 1000 # ms QUEUE_SIZE = 100 # class PredictorLocal(): # def __init__(self, *args, **kwargs): # pass # def __getattr__(self, *args, **kwargs): # return lambda *args, **kwargs: None class PredictorWorker(): def __init__(self, in_port=None, out_port=None): self.recv_queue = mp.Queue(QUEUE_SIZE) self.send_queue = mp.Queue(QUEUE_SIZE) self.worker_alive = mp.Value('i', 0) self.recv_process = mp.Process(target=self.recv_worker, args=(in_port, self.recv_queue, self.worker_alive)) self.predictor_process = mp.Process(target=self.predictor_worker, args=(self.recv_queue, self.send_queue, self.worker_alive)) self.send_process = mp.Process(target=self.send_worker, args=(out_port, self.send_queue, self.worker_alive)) def run(self): self.worker_alive.value = 1 self.recv_process.start() self.predictor_process.start() self.send_process.start() try: self.recv_process.join() self.predictor_process.join() self.send_process.join() except KeyboardInterrupt: pass @staticmethod def recv_worker(port, recv_queue, worker_alive): timing = AccumDict() log = Logger('./var/log/recv_worker.log', verbose=opt.verbose) ctx = SerializingContext() socket = ctx.socket(zmq.PULL) socket.bind(f"tcp://*:{port}") socket.RCVTIMEO = RECV_TIMEOUT log(f'Receiving on port {port}', important=True) try: while worker_alive.value: tt = TicToc() try: tt.tic() msg = socket.recv_data() timing.add('RECV', tt.toc()) except zmq.error.Again: log("recv timeout") continue #log('recv', msg[0]) method, data = msg if method['critical']: recv_queue.put(msg) else: try: recv_queue.put(msg, block=False) except queue.Full: log('recv_queue full') Once(timing, log, per=1) except KeyboardInterrupt: log("recv_worker: user interrupt", important=True) worker_alive.value = 0 log("recv_worker exit", important=True) @staticmethod def predictor_worker(recv_queue, send_queue, worker_alive): predictor = None predictor_args = () timing = AccumDict() log = Logger('./var/log/predictor_worker.log', verbose=opt.verbose) try: while worker_alive.value: tt = TicToc() try: method, data = recv_queue.get(timeout=GET_TIMEOUT) except queue.Empty: continue # get the latest non-critical request from the queue # don't skip critical request while not recv_queue.empty() and not method['critical']: log(f"skip {method}") method, data = recv_queue.get() log("working on", method) try: tt.tic() if method['name'] == 'predict': image = cv2.imdecode(np.frombuffer(data, dtype='uint8'), -1) else: args = msgpack.unpackb(data) timing.add('UNPACK', tt.toc()) except ValueError: log("Invalid Message", important=True) continue tt.tic() if method['name'] == "hello": result = "OK" elif method['name'] == "__init__": if args == predictor_args: log("Same config as before... reusing previous predictor") else: del predictor predictor_args = args predictor = PredictorLocal(*predictor_args[0], **predictor_args[1]) log("Initialized predictor with:", predictor_args, important=True) result = True tt.tic() # don't account for init elif method['name'] == 'predict': assert predictor is not None, "Predictor was not initialized" result = getattr(predictor, method['name'])(image) else: assert predictor is not None, "Predictor was not initialized" result = getattr(predictor, method['name'])(*args[0], **args[1]) timing.add('CALL', tt.toc()) tt.tic() if method['name'] == 'predict': assert isinstance(result, np.ndarray), f'Expected np.ndarray, got {result.__class__}' ret_code, data_send = cv2.imencode(".jpg", result, [int(cv2.IMWRITE_JPEG_QUALITY), opt.jpg_quality]) else: data_send = msgpack.packb(result) timing.add('PACK', tt.toc()) if method['critical']: send_queue.put((method, data_send)) else: try: send_queue.put((method, data_send), block=False) except queue.Full: log("send_queue full") pass Once(timing, log, per=1) except KeyboardInterrupt: log("predictor_worker: user interrupt", important=True) except Exception as e: log("predictor_worker error", important=True) traceback.print_exc() worker_alive.value = 0 log("predictor_worker exit", important=True) @staticmethod def send_worker(port, send_queue, worker_alive): timing = AccumDict() log = Logger('./var/log/send_worker.log', verbose=opt.verbose) ctx = SerializingContext() socket = ctx.socket(zmq.PUSH) socket.bind(f"tcp://*:{port}") log(f'Sending on port {port}', important=True) try: while worker_alive.value: tt = TicToc() try: method, data = send_queue.get(timeout=GET_TIMEOUT) except queue.Empty: log("send queue empty") continue # get the latest non-critical request from the queue # don't skip critical request while not send_queue.empty() and not method['critical']: log(f"skip {method}") method, data = send_queue.get() log("sending", method) tt.tic() socket.send_data(method, data) timing.add('SEND', tt.toc()) Once(timing, log, per=1) except KeyboardInterrupt: log("predictor_worker: user interrupt", important=True) worker_alive.value = 0 log("send_worker exit", important=True) def run_worker(in_port=None, out_port=None): worker = PredictorWorker(in_port=in_port, out_port=out_port) worker.run() ================================================ FILE: afy/utils.py ================================================ import sys import time from collections import defaultdict import numpy as np import cv2 def log(*args, file=sys.stderr, **kwargs): time_str = f'{time.time():.6f}' print(f'[{time_str}]', *args, file=file, **kwargs) def info(*args, file=sys.stdout, **kwargs): print(*args, file=file, **kwargs) class Tee(object): def __init__(self, filename, mode='w', terminal=sys.stderr): self.file = open(filename, mode, buffering=1) self.terminal = terminal def __del__(self): self.file.close() def write(self, *args, **kwargs): log(*args, file=self.file, **kwargs) log(*args, file=self.terminal, **kwargs) def __call__(self, *args, **kwargs): return self.write(*args, **kwargs) def flush(self): self.file.flush() class Logger(): def __init__(self, filename, verbose=True): self.tee = Tee(filename) self.verbose = verbose def __call__(self, *args, important=False, **kwargs): if not self.verbose and not important: return self.tee(*args, **kwargs) class Once(): _id = {} def __init__(self, what, who=log, per=1e12): """ Do who(what) once per seconds. what: args for who who: callable per: frequency in seconds. """ assert callable(who) now = time.time() if what not in Once._id or now - Once._id[what] > per: who(what) Once._id[what] = now class TicToc: def __init__(self): self.t = None self.t_init = time.time() def tic(self): self.t = time.time() def toc(self, total=False): if total: return (time.time() - self.t_init) * 1000 assert self.t, 'You forgot to call tic()' return (time.time() - self.t) * 1000 def tocp(self, str): t = self.toc() log(f"{str} took {t:.4f}ms") return t class AccumDict: def __init__(self, num_f=3): self.d = defaultdict(list) self.num_f = num_f def add(self, k, v): self.d[k] += [v] def __dict__(self): return self.d def __getitem__(self, key): return self.d[key] def __str__(self): s = '' for k in self.d: if not self.d[k]: continue cur = self.d[k][-1] avg = np.mean(self.d[k]) format_str = '{:.%df}' % self.num_f cur_str = format_str.format(cur) avg_str = format_str.format(avg) s += f'{k} {cur_str} ({avg_str})\t\t' return s def __repr__(self): return self.__str__() def clamp(value, min_value, max_value): return max(min(value, max_value), min_value) def crop(img, p=0.7, offset_x=0, offset_y=0): h, w = img.shape[:2] x = int(min(w, h) * p) l = (w - x) // 2 r = w - l u = (h - x) // 2 d = h - u offset_x = clamp(offset_x, -l, w - r) offset_y = clamp(offset_y, -u, h - d) l += offset_x r += offset_x u += offset_y d += offset_y return img[u:d, l:r], (offset_x, offset_y) def pad_img(img, target_size, default_pad=0): sh, sw = img.shape[:2] w, h = target_size pad_w, pad_h = default_pad, default_pad if w / h > 1: pad_w += int(sw * (w / h) - sw) // 2 else: pad_h += int(sh * (h / w) - sh) // 2 out = np.pad(img, [[pad_h, pad_h], [pad_w, pad_w], [0,0]], 'constant') return out def resize(img, size, version='cv'): return cv2.resize(img, size) ================================================ FILE: afy/videocaptureasync.py ================================================ # https://github.com/gilbertfrancois/video-capture-async import threading import cv2 import time WARMUP_TIMEOUT = 10.0 class VideoCaptureAsync: def __init__(self, src=0, width=640, height=480): self.src = src self.cap = cv2.VideoCapture(self.src) if not self.cap.isOpened(): raise RuntimeError("Cannot open camera") self.cap.set(cv2.CAP_PROP_FRAME_WIDTH, width) self.cap.set(cv2.CAP_PROP_FRAME_HEIGHT, height) self.grabbed, self.frame = self.cap.read() self.started = False self.read_lock = threading.Lock() def set(self, var1, var2): self.cap.set(var1, var2) def isOpened(self): return self.cap.isOpened() def start(self): if self.started: print('[!] Asynchronous video capturing has already been started.') return None self.started = True self.thread = threading.Thread(target=self.update, args=(), daemon=True) self.thread.start() # (warmup) wait for the first successfully grabbed frame warmup_start_time = time.time() while not self.grabbed: warmup_elapsed_time = (time.time() - warmup_start_time) if warmup_elapsed_time > WARMUP_TIMEOUT: raise RuntimeError(f"Failed to succesfully grab frame from the camera (timeout={WARMUP_TIMEOUT}s). Try to restart.") time.sleep(0.5) return self def update(self): while self.started: grabbed, frame = self.cap.read() if not grabbed or frame is None or frame.size == 0: continue with self.read_lock: self.grabbed = grabbed self.frame = frame def read(self): while True: with self.read_lock: frame = self.frame.copy() grabbed = self.grabbed break return grabbed, frame def stop(self): self.started = False self.thread.join() def __exit__(self, exec_type, exc_value, traceback): self.cap.release() ================================================ FILE: avatarify.ipynb ================================================ { "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "Avatarify", "provenance": [], "collapsed_sections": [], "authorship_tag": "ABX9TyMqm8Y/SbrWZMeNGn+BbnaN", "include_colab_link": true }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "accelerator": "GPU" }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "view-in-github", "colab_type": "text" }, "source": [ "\"Open" ] }, { "cell_type": "markdown", "metadata": { "id": "pEl-Q4OpsLb9", "colab_type": "text" }, "source": [ "# Avatarify Colab Server\n", "\n", "This Colab notebook is for running Avatarify rendering server. It allows you to run Avatarify on your computer **without GPU** in this way:\n", "\n", "1. When this notebook is executed, it starts listening for incoming requests from your computer;\n", "1. You start the client on your computer and it connects to the notebook and starts sending requests;\n", "1. This notebooks receives the requests from your computer, renders avatar images and sends them back;\n", "\n", "To this end, all the heavy work is offloaded from your computer to this notebook so you don't need to have a beafy hardware on your PC anymore.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "wRpMPl7VyeoD", "colab_type": "text" }, "source": [ "## Start the server\n", "Run the cells below (Shift+Enter) sequentially and pay attention to the hints and instructions included in this notebook.\n", "\n", "At the end you will get a command for running the client on your computer.\n", "\n", "## Start the client\n", "\n", "Make sure you have installed the latest version of Avatarify on your computer. Refer to the [README](https://github.com/alievk/avatarify#install) for the instructions.\n", "\n", "When it's ready execute this notebook and get the command for running the client on your computer.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "8h0f5WQEjgbH", "colab_type": "text" }, "source": [ "### Technical details\n", "\n", "The client on your computer connects to the server via `ngrok` TCP tunnel or a reverse `ssh` tunnel.\n", "\n", "`ngrok`, while easy to use, can induce a considerable network lag ranging from dozens of milliseconds to a second. This can lead to a poor experience.\n", "\n", "A more stable connection could be established using a reverse `ssh` tunnel to a host with a public IP, like an AWS `t3.micro` (free) instance. This notebook provides a script for creating a tunnel, but launching an instance in a cloud is on your own (find the manual below)." ] }, { "cell_type": "markdown", "metadata": { "id": "6ZI4EvKaNUhL", "colab_type": "text" }, "source": [ "# Install" ] }, { "cell_type": "markdown", "metadata": { "id": "9fYm9X3X125H", "colab_type": "text" }, "source": [ "### Avatarify\n", "Follow the steps below to clone Avatarify and install the dependencies." ] }, { "cell_type": "code", "metadata": { "id": "LC1q-hdat-JP", "colab_type": "code", "colab": {} }, "source": [ "!cd /content\n", "!rm -rf *" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "kE4_YSbsiX_O", "colab_type": "code", "colab": {} }, "source": [ "!git clone https://github.com/alievk/avatarify.git" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "I8fTBhOEUM_c", "colab_type": "code", "colab": {} }, "source": [ "cd avatarify" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "FONInDgZUmcZ", "colab_type": "code", "colab": {} }, "source": [ "!git clone https://github.com/alievk/first-order-model.git fomm\n", "!pip install face-alignment==1.0.0 msgpack_numpy pyyaml==5.1" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "hPgXoqE_gAyD", "colab_type": "code", "colab": {} }, "source": [ "!scripts/download_data.sh" ], "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "j1soT4zEEFzp", "colab_type": "text" }, "source": [ "### ngrok\n", "Follow the steps below to setup ngrok. You will also need to sign up on the ngrok site and get your authtoken (free).\n" ] }, { "cell_type": "code", "metadata": { "id": "bbptNwHL1s61", "colab_type": "code", "colab": {} }, "source": [ "# Download ngrok\n", "!scripts/get_ngrok.sh" ], "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "4qk1FCeeaviZ", "colab_type": "text" }, "source": [ "# Run\n", "Start here if the runtime was restarted after installation." ] }, { "cell_type": "code", "metadata": { "id": "_f2iYcQVI2ss", "colab_type": "code", "colab": {} }, "source": [ "cd /content/avatarify" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "qxK_ZZjPz_Rr", "colab_type": "code", "colab": {} }, "source": [ "#!git pull origin" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "MA-h22jF6-ks", "colab_type": "code", "colab": {} }, "source": [ "from subprocess import Popen, PIPE\n", "import shlex\n", "import json\n", "import time\n", "\n", "\n", "def run_with_pipe(command):\n", " commands = list(map(shlex.split,command.split(\"|\")))\n", " ps = Popen(commands[0], stdout=PIPE, stderr=PIPE)\n", " for command in commands[1:]:\n", " ps = Popen(command, stdin=ps.stdout, stdout=PIPE, stderr=PIPE)\n", " return ps.stdout.readlines()\n", "\n", "\n", "def get_tunnel_adresses():\n", " info = run_with_pipe(\"curl http://localhost:4040/api/tunnels\")\n", " assert info\n", "\n", " info = json.loads(info[0])\n", " for tunnel in info['tunnels']:\n", " url = tunnel['public_url']\n", " port = url.split(':')[-1]\n", " local_port = tunnel['config']['addr'].split(':')[-1]\n", " print(f'{url} -> {local_port} [{tunnel[\"name\"]}]')\n", " if tunnel['name'] == 'input':\n", " in_addr = url\n", " elif tunnel['name'] == 'output':\n", " out_addr = url\n", " else:\n", " print(f'unknown tunnel: {tunnel[\"name\"]}')\n", "\n", " return in_addr, out_addr" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "RfHa02CBWoNN", "colab_type": "code", "colab": {} }, "source": [ "# Input and output ports for communication\n", "local_in_port = 5557\n", "local_out_port = 5558" ], "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "BcULGnhGJJjC", "colab_type": "text" }, "source": [ "# Start the worker\n" ] }, { "cell_type": "code", "metadata": { "id": "8PnArK75mRqx", "colab_type": "code", "colab": {} }, "source": [ "# (Re)Start the worker\n", "with open('/tmp/run.txt', 'w') as f:\n", " ps = Popen(\n", " shlex.split(f'./run.sh --is-worker --in-port {local_in_port} --out-port {local_out_port} --no-vcam --no-conda'),\n", " stdout=f, stderr=f)\n", " time.sleep(3)" ], "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "XfUqQxMtRSvc", "colab_type": "text" }, "source": [ "This command should print lines if the worker is successfully started" ] }, { "cell_type": "code", "metadata": { "id": "W0eY8gkBqUJG", "colab_type": "code", "colab": {} }, "source": [ "!ps aux | grep 'python3 afy/cam_fomm.py' | grep -v grep | tee /tmp/ps_run\n", "!if [[ $(cat /tmp/ps_run | wc -l) == \"0\" ]]; then echo \"Worker failed to start\"; cat /tmp/run.txt; else echo \"Worker started\"; fi" ], "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "tz9gpLD0IsCL", "colab_type": "text" }, "source": [ "# Open ngrok tunnel" ] }, { "cell_type": "markdown", "metadata": { "id": "gyB7XIxL0XpD", "colab_type": "text" }, "source": [ "#### Get ngrok token\n", "Go to https://dashboard.ngrok.com/auth/your-authtoken (sign up if required), copy your authtoken and put it below." ] }, { "cell_type": "code", "metadata": { "id": "YDtPpi77AkQ1", "colab_type": "code", "colab": {} }, "source": [ "# Paste your authtoken here in quotes\n", "authtoken = \"1cBzFFwzSlaLhlRPXIHJiVLqtiQ_2cVsonJXe52B6DDyp8su7\"" ], "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "gASaDrsFXLXA", "colab_type": "text" }, "source": [ "Set your region\n", "\n", "Code | Region\n", "--- | ---\n", "us | United States\n", "eu | Europe\n", "ap | Asia/Pacific\n", "au | Australia\n", "sa | South America\n", "jp | Japan\n", "in | India" ] }, { "cell_type": "code", "metadata": { "id": "r5e9VR9NYckJ", "colab_type": "code", "colab": {} }, "source": [ "# Set your region here in quotes\n", "region = \"eu\"" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "jZ5_PE_EHpCg", "colab_type": "code", "colab": {} }, "source": [ "config =\\\n", "f\"\"\"\n", "version: 2\n", "authtoken: {authtoken}\n", "region: {region}\n", "console_ui: False\n", "tunnels:\n", " input:\n", " addr: {local_in_port}\n", " proto: tcp \n", " output:\n", " addr: {local_out_port}\n", " proto: tcp\n", "\"\"\"\n", "\n", "with open('ngrok.conf', 'w') as f:\n", " f.write(config)" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "Z49OEhAdDI7Y", "colab_type": "code", "colab": {} }, "source": [ "# (Re)Open tunnel\n", "ps = Popen('./scripts/open_tunnel_ngrok.sh', stdout=PIPE, stderr=PIPE)\n", "time.sleep(3)" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "JAyPH2t2C64H", "colab_type": "code", "colab": {} }, "source": [ "# Get tunnel addresses\n", "try:\n", " in_addr, out_addr = get_tunnel_adresses()\n", " print(\"Tunnel opened\")\n", "except Exception as e:\n", " [print(l.decode(), end='') for l in ps.stdout.readlines()]\n", " print(\"Something went wrong, reopen the tunnel\")" ], "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "dc6rg2HQAocK", "colab_type": "text" }, "source": [ "### [Optional] AWS proxy\n", "Alternatively you can create a ssh reverse tunnel to an AWS `t3.micro` instance (it's free). It has lower latency than ngrok.\n", "\n", "1. In your AWS console go to Services -> EC2 -> Instances -> Launch Instance;\n", "1. Choose `Ubuntu Server 18.04 LTS` AMI;\n", "1. Choose `t3.micro` instance type and press Review and launch;\n", "1. Confirm your key pair and press Launch instances;\n", "1. Go to the security group of this instance and edit inbound rules. Add TCP ports 5557 and 5558 and set Source to Anywhere. Press Save rules;\n", "1. ssh into the instance (you can find the command in the Instances if you click on the Connect button) and add this line in the end of `/etc/ssh/sshd_config`:\n", "```\n", "GatewayPorts yes\n", "```\n", "then restart `sshd`\n", "```\n", "sudo service sshd restart\n", "```\n", "1. Copy your `key_pair.pem` by dragging and dropping it into avatarify folder in this notebook;\n", "1. Use the command below to open the tunnel;\n", "1. Start client with a command (substitute `run_mac.sh` with `run_windows.bat` or `run.sh`)\n", "```\n", "./run_mac.sh --is-client --in-addr tcp://instace.compute.amazonaws.com:5557 --out-addr tcp://instance.compute.amazonaws.com:5558\n", "```" ] }, { "cell_type": "code", "metadata": { "id": "zdN5Qj2BCYsr", "colab_type": "code", "colab": {} }, "source": [ "# Open reverse ssh tunnel (uncomment line below)\n", "# !./scripts/open_tunnel_ssh.sh key_pair.pem ubuntu@instance.compute.amazonaws.com" ], "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "ccZ24BT4Jdis", "colab_type": "text" }, "source": [ "# Start the client\n", "When you run the cell below it will print a command. Run this command on your computer:\n", "\n", "1. Open a terminal (in Windows open `Anaconda Prompt`);\n", "2. Change working directory to the `avatarify` directory:
\n", "* Windows (change `C:\\path\\to\\avatarify` to your path)
\n", "`cd C:\\path\\to\\avatarify`

\n", "* Mac/Linux (change `/path/to/avatarify` to your path)
\n", "`cd /path/to/avatarify`\n", "3. Copy-paste to the terminal the command below and run;\n", "4. It can take some time to connect (usually up to 10 seconds). If the preview window doesn't appear in a minute or two, look for the errors above in this notebook and report in the [issues](https://github.com/alievk/avatarify/issues) or [Slack](https://join.slack.com/t/avatarify/shared_invite/zt-dyoqy8tc-~4U2ObQ6WoxuwSaWKKVOgg)." ] }, { "cell_type": "code", "metadata": { "id": "4gaqS0mZWF1V", "colab_type": "code", "colab": {} }, "source": [ "print('Copy-paste to the terminal the command below and run (press Enter)\\n')\n", "print('Mac:')\n", "print(f'./run_mac.sh --is-client --in-addr {in_addr} --out-addr {out_addr}')\n", "print('\\nWindows:')\n", "print(f'run_windows.bat --is-client --in-addr {in_addr} --out-addr {out_addr}')\n", "print('\\nLinux:')\n", "print(f'./run.sh --is-client --in-addr {in_addr} --out-addr {out_addr}')" ], "execution_count": 0, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "M3h92xQ9KA-R", "colab_type": "text" }, "source": [ "# Logs" ] }, { "cell_type": "markdown", "metadata": { "id": "SvodbjapKBQi", "colab_type": "text" }, "source": [ "If something doesn't work as expected, please run the cells below and include the logs in your report." ] }, { "cell_type": "code", "metadata": { "id": "0GeT7KxON0Ke", "colab_type": "code", "colab": {}, "cellView": "form" }, "source": [ "#@title\n", "!cat ./var/log/cam_fomm.log | head -100" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "P1FQcdzwqdce", "colab_type": "code", "colab": {}, "cellView": "form" }, "source": [ "#@title\n", "!cat ./var/log/recv_worker.log | tail -100" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "YThWBXCf_yzI", "colab_type": "code", "colab": {}, "cellView": "form" }, "source": [ "#@title\n", "!cat ./var/log/predictor_worker.log | tail -100" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "zhJzygCP_6p3", "colab_type": "code", "colab": {}, "cellView": "form" }, "source": [ "#@title\n", "!cat ./var/log/send_worker.log | tail -100" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "nrzNffhR_8HQ", "colab_type": "code", "colab": {} }, "source": [ "" ], "execution_count": 0, "outputs": [] } ] } ================================================ FILE: avatars/.geetkeep ================================================ ================================================ FILE: config.yaml ================================================ # how many cameras to query at the first start query_n_cams: 4 # camera configuration path cam_config: ./cam.yaml ================================================ FILE: docs/README.md ================================================ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/alievk/avatarify/blob/master/avatarify.ipynb) :arrow_forward: [Demo](https://youtu.be/Q7LFDT-FRzs) :arrow_forward: [AI-generated Elon Musk](https://youtu.be/lONuXGNqLO0) ## Table of Contents - [Requirements](#requirements) - [Install](#install) - [Download network weights](#download-network-weights) - [Linux](#linux) - [Mac](#mac) - [Windows](#windows) - [Remote GPU](#remote-gpu) - [Docker](#docker) - [Setup avatars](#setup-avatars) - [Run](#run) - [Linux](#linux-1) - [Mac](#mac-1) - [Windows](#windows-1) - [Controls](#controls) - [Driving your avatar](#driving-your-avatar) - [Configure video meeting app](#configure-video-meeting-app) - [Skype](#skype) - [Zoom](#zoom) - [Teams](#teams) - [Slack](#slack) - [Uninstall](#uninstall) - [Contribution](#contribution) - [FAQ](#faq) - [Troubleshooting](#troubleshooting) ## Requirements You can run Avatarify in two modes: *locally* and *remotely*. To run Avatarify *locally* you need a CUDA-enabled (NVIDIA) video card. Otherwise it will fallback to the central processor and run very slowly. These are performance metrics for some hardware: - GeForce GTX 1080 Ti: **33 frames per second** - GeForce GTX 1070: **15 frames per second** - GeForce GTX 950: **9 frames per second** You can also run Avatarify *remotely* on [Google Colab](https://colab.research.google.com/github/alievk/avatarify/blob/master/avatarify.ipynb) (easy) or on a [dedicated server](https://github.com/alievk/avatarify-python/wiki/Remote-GPU) with a GPU (harder). There are no special PC requirements for this mode, only a stable internet connection. Of course, you also need a webcam! ## Install #### Download network weights Download model's weights from [here](https://openavatarify.s3-avatarify.com/weights/vox-adv-cpk.pth.tar) or [here](https://yadi.sk/d/M0FWpz2ExBfgAA) or [here](https://drive.google.com/file/d/1coUCdyRXDbpWnEkA99NLNY60mb9dQ_n3/view?usp=sharing) [228 MB, md5sum `8a45a24037871c045fbb8a6a8aa95ebc`] #### Linux Linux uses `v4l2loopback` to create virtual camera. 1. Download [Miniconda Python 3.7](https://docs.conda.io/en/latest/miniconda.html#linux-installers) and install using command: ```bash bash Miniconda3-latest-Linux-x86_64.sh ``` 2. Clone `avatarify` and install its dependencies (sudo privelege is required): ```bash git clone https://github.com/alievk/avatarify-python.git cd avatarify-python bash scripts/install.sh ``` 3. [Download network weights](#download-network-weights) and place `vox-adv-cpk.pth.tar` file in the `avatarify-python` directory (don't unpack it). #### Mac We will use [CamTwist](http://camtwiststudio.com) to create virtual camera for Mac. 1. Install [Miniconda Python 3.7](https://docs.conda.io/en/latest/miniconda.html#macosx-installers) or use *Homebrew Cask*: `brew install --cask miniconda`. 2. [Download](https://github.com/alievk/avatarify-python/archive/master.zip) and unpack the repository or use `git`: ```bash git clone https://github.com/alievk/avatarify-python.git cd avatarify-python bash scripts/install_mac.sh ``` 3. Download and install [CamTwist](http://camtwiststudio.com) from [here](http://camtwiststudio.com/download). It's easy. #### Windows :arrow_forward: [Video tutorial](https://youtu.be/lym9ANVb120) This guide is tested for Windows 10. 1. Install [Miniconda Python 3.8](https://docs.conda.io/en/latest/miniconda.html#windows-installers). 2. Install [Git](https://git-scm.com/download/win). 3. Press Windows button and type "miniconda". Run suggested Anaconda Prompt. 4. Download and install Avatarify (please copy-paste these commands and don't change them): ```bash git clone https://github.com/alievk/avatarify-python.git cd avatarify-python scripts\install_windows.bat ``` 5. [Download network weights](#download-network-weights) and place `vox-adv-cpk.pth.tar` file in the `avatarify-python` directory (don't unpack it). 6. Run `run_windows.bat`. If installation was successful, two windows "cam" and "avatarify" will appear. Leave these windows open for the next installation steps. 7. Install [OBS Studio](https://obsproject.com/) for capturing Avatarify output. 8. Install [VirtualCam plugin](https://obsproject.com/forum/resources/obs-virtualcam.539/). Choose `Install and register only 1 virtual camera`. 9. Run OBS Studio. 10. In the Sources section, press on Add button ("+" sign), select Windows Capture and press OK. In the appeared window, choose "[python.exe]: avatarify" in Window drop-down menu and press OK. Then select Edit -> Transform -> Fit to screen. 11. In OBS Studio, go to Tools -> VirtualCam. Check AutoStart, set Buffered Frames to 0 and press Start. 12. Now `OBS-Camera` camera should be available in Zoom (or other videoconferencing software). The steps 10-11 are required only once during setup. #### Remote GPU You can offload the heavy work to [Google Colab](https://colab.research.google.com/github/alievk/avatarify/blob/master/avatarify.ipynb) or a [server with a GPU](https://github.com/alievk/avatarify-python/wiki/Remote-GPU) and use your laptop just to communicate the video stream. The server and client software are native and dockerized available. ### Docker Docker images are only availabe on Linux. 1. Install Docker following the [Documentation](https://docs.docker.com/engine/install/). Then run this [step](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user) to make docker available for your user. 2. For using the gpu (hardly recommended): Install nvidia drivers and [nvidia docker](https://github.com/NVIDIA/nvidia-docker#quickstart). 3. Clone `avatarify-python` and install its dependencies (v4l2loopback kernel module): ```bash git clone https://github.com/alievk/avatarify-python.git cd avatarify-python bash scripts/install_docker.sh ``` 4. Build the Dockerfile: ```bash cd avatarify-python docker build -t avatarify . ``` ## Setup avatars Avatarify comes with a standard set of avatars of famous people, but you can extend this set simply copying your avatars into `avatars` folder. Follow these advices for better visual quality: * Make square crop of your avatar picture. * Crop avatar's face so that it's not too close not too far. Use standard avatars as reference. * Prefer pictures with uniform background. It will diminish visual artifacts. ## Run Your web cam must be plugged-in. **Note:** run your video-conferencing app only after Avatarify is started. #### Linux The run script will create virtual camera `/dev/video9`. You can change these settings in `scripts/settings.sh`. You can use command `v4l2-ctl --list-devices` to list all devices in your system. Run: ```bash bash run.sh ``` If you haven't installed a GPU add the `--no-gpus` flag. In order to use Docker add the `--docker` flag. `cam` and `avatarify` windows will pop-up. The `cam` window is for controlling your face position and `avatarify` is for the avatar animation preview. Please follow these [recommendations](#driving-your-avatar) to drive your avatars. #### Mac *Note*: On Mac Avatarify runs only with [Google Colab](https://colab.research.google.com/github/alievk/avatarify/blob/master/avatarify.ipynb) or a [dedicated server](https://github.com/alievk/avatarify-python/wiki/Remote-GPU) with GPU. Please find where you downloaded `avatarify` and substitute path `/path/to/avatarify` below. 1. To run Avatarify please follow instructions for [Google Colab](https://colab.research.google.com/github/alievk/avatarify/blob/master/avatarify.ipynb) or a [dedicated server](https://github.com/alievk/avatarify-python/wiki/Remote-GPU). 2. Go to [CamTwist](http://camtwiststudio.com). 3. Choose `Desktop+` and press `Select`. 4. In the `Settings` section choose `Confine to Application Window` and select `python (avatarify)` from the drop-down menu. `cam` and `avatarify` windows will pop-up. The `cam` window is for controlling your face position and `avatarify` is for the avatar animation preview. Please follow these [recommendations](#driving-your-avatar) to drive your avatars. #### Windows 1. In Anaconda Prompt: ``` cd C:\path\to\avatarify run_windows.bat ``` 2. Run OBS Studio. It should automaitcally start streaming video from Avatarify to `OBS-Camera`. `cam` and `avatarify` windows will pop-up. The `cam` window is for controlling your face position and `avatarify` is for the avatar animation preview. Please follow these [recommendations](#driving-your-avatar) to drive your avatars. **Note:** To reduce video latency, in OBS Studio right click on the preview window and uncheck Enable Preview. ## Controls Keys | Controls --- | --- 1-9 | These will immediately switch between the first 9 avatars. Q | Turns on StyleGAN-generated avatar. Every time you push the button – new avatar is sampled. 0 | Toggles avatar display on and off. A/D | Previous/next avatar in folder. W/S | Zoom camera in/out. U/H/J/K | Translate camera. `H` - left, `K` - right, `U` - up, `J` - Down by 5 pixels. Add `Shift` to adjust by 1 pixel. Shift-Z | Reset camera zoom and translation Z/C | Adjust avatar target overlay opacity. X | Reset reference frame. F | Toggle reference frame search mode. R | Mirror reference window. T | Mirror output window. L | Reload avatars. I | Show FPS O | Toggle face detection overlay. ESC | Quit ## Driving your avatar These are the main principles for driving your avatar: * Align your face in the camera window as closely as possible in proportion and position to the target avatar. Use zoom in/out function (W/S keys) and camera left, right, up, down translation (U/H/J/K keys). When you have aligned, hit 'X' to use this frame as reference to drive the rest of the animation * Use the image overlay function (Z/C keys) or the face detection overlay function (O key) to match your and avatar's face expressions as close as possible Alternatively, you can hit 'F' for the software to attempt to find a better reference frame itself. This will slow down the framerate, but while this is happening, you can keep moving your head around: the preview window will flash green when it finds your facial pose is a closer match to the avatar than the one it is currently using. You will see two numbers displayed as well: the first number is how closely you are currently aligned to the avatar, and the second number is how closely the reference frame is aligned. You want to get the first number as small as possible - around 10 is usually a good alignment. When you are done, press 'F' again to exit reference frame search mode. You don't need to be exact, and some other configurations can yield better results still, but it's usually a good starting point. ## Configure video meeting app Avatarify supports any video-conferencing app where video input source can be changed (Zoom, Skype, Hangouts, Slack, ...). Here are a few examples how to configure particular app to use Avatarify. ### Skype Go to Settings -> Audio & Video, choose `avatarify` (Linux), `CamTwist` (Mac) or `OBS-Camera` (Windows) camera. ### Zoom Go to Settings -> Video and choose `avatarify` (Linux), `CamTwist` (Mac) or `OBS-Camera` (Windows) from Camera drop-down menu. ### Teams Go to your profile picture -> Settings -> Devices and choose `avatarify` (Linux), `CamTwist` (Mac) or `OBS-Camera` (Windows) from Camera drop-down menu. ### Slack Make a call, allow browser using cameras, click on Settings icon, choose `avatarify` (Linux), `CamTwist` (Mac) or `OBS-Camera` (Windows) in Video settings drop-down menu. ## Uninstall To remove Avatarify and its related programs follow the [instructions](https://github.com/alievk/avatarify-python/wiki/Removing-Avatarify) in the Wiki. ## Contribution Our goal is to democratize photorealistic avatars for video-conferencing. To make the technology even more accessible, we have to tackle the following problems: * ~~Add support for more platforms (Linux and Mac are already supported).~~ * ~~Remote GPU support. This is a work in progress.~~ * Porting to non-CUDA GPUs (Intel integrated GPUs, AMD GPUs, etc) and optimization. The goal is to run Avatarify real-time (at least 10FPS) on modern laptops. Please make pull requests if you have any improvements or bug-fixes. ## FAQ Q: **Do I need any knowledge of programming to run Avatarify?** A: Not really, but you need some beginner-level knowledge of the command line. For Windows we recorded a video [tutorial](https://www.youtube.com/watch?v=lym9ANVb120), so it’ll be easy to install. Q: **Why does it work so slow on my Macbook?** A: The model used in Avatarify requires a CUDA-enabled NVIDIA GPU to perform heavy computations. Macbooks don’t have such GPUs, and for processing use CPU, which has much less computing power to run Avatarify smoothly. Q: **I don’t have a NVIDIA GPU, can I run it?** A: You still can run it without a NVIDIA GPU, but with drastically reduced performance (<1fps). Q: **I have an ATI GPU (e.g. Radeon). Why does it work so slow?** A: To run the neural network Avatarify uses PyTorch library, which is optimized for CUDA. If PyTorch can’t find a CUDA-enabled GPU in your system it will fallback to CPU. The performance on the CPU will be much worse. Q: **How to add a new avatar?** A: It’s easy. All you need is to find a picture of your avatar and put it in the `avatars` folder. [More](https://github.com/alievk/avatarify-python#setup-avatars). Q: **My avatar looks distorted.** A: You need to calibrate your face position. Please follow the [tips](https://github.com/alievk/avatarify-python#driving-your-avatar) or watch the video [tutorial](https://youtu.be/lym9ANVb120?t=662). Q: **Can I use a cloud GPU?** A: This is work in progress. See the relevant [discussion](https://github.com/alievk/avatarify-python/issues/115). Q: **Avatarify crashed, what to do?** A: First, try to find your error in the [troubleshooting](https://github.com/alievk/avatarify-python#troubleshooting) section. If it is not there, try to find it in the [issues](https://github.com/alievk/avatarify-python/issues). If you couldn’t find your issue there, please open a new one using the issue template. Q: **Can I use Avatarify for commercial purposes?** A: No. Avatarify and First Order Motion Model are licensed under Creative Commons Non-Commercial license, which prohibits commercial use. Q: **What video conferencing apps does Avatarify support?** A: Avatarify creates a virtual camera which can be plugged into any app where video input source can be changed (Zoom, Skype, Hangouts, Slack, ...). Q: **Where can I discuss Avatarify-related topics with the community?** A: We have Slack. Please join: [](https://join.slack.com/t/avatarify/shared_invite/zt-dyoqy8tc-~4U2ObQ6WoxuwSaWKKVOgg) ## Troubleshooting Please follow the [Wiki](https://github.com/alievk/avatarify-python/wiki/Troubleshooting) page. ================================================ FILE: requirements.txt ================================================ opencv-python==4.2.0.34 face-alignment==1.3.3 pyzmq==20.0.0 msgpack-numpy==0.4.7.1 pyyaml==5.4 requests==2.32.0 pyfakewebcam==0.1.0 ================================================ FILE: requirements_client.txt ================================================ numpy==1.15.0 PyYAML==5.1 requests==2.32.0 msgpack-numpy==0.4.5 pyzmq==19.0.0 opencv-python==3.4.5.20 ================================================ FILE: run.sh ================================================ #!/usr/bin/env bash # set -x ENABLE_CONDA=1 ENABLE_VCAM=1 KILL_PS=1 USE_DOCKER=0 IS_WORKER=0 IS_CLIENT=0 DOCKER_IS_LOCAL_CLIENT=0 DOCKER_NO_GPU=0 FOMM_CONFIG=fomm/config/vox-adv-256.yaml FOMM_CKPT=vox-adv-cpk.pth.tar ARGS="" DOCKER_ARGS="" while (( "$#" )); do case "$1" in --no-conda) ENABLE_CONDA=0 shift ;; --no-vcam) ENABLE_VCAM=0 ARGS="$ARGS --no-stream" shift ;; --keep-ps) KILL_PS=0 shift ;; --docker) USE_DOCKER=1 shift ;; --no-gpus) DOCKER_NO_GPU=1 shift ;; --is-worker) IS_WORKER=1 ARGS="$ARGS $1" DOCKER_ARGS="$DOCKER_ARGS -p 5557:5557 -p 5558:5558" shift ;; --is-client) IS_CLIENT=1 ARGS="$ARGS $1" shift ;; --is-local-client) IS_CLIENT=1 DOCKER_IS_LOCAL_CLIENT=1 ARGS="$ARGS --is-client" shift ;; *|-*|--*) ARGS="$ARGS $1" shift ;; esac done eval set -- "$ARGS" if [[ $USE_DOCKER == 0 ]]; then if [[ $KILL_PS == 1 ]]; then kill -9 $(ps aux | grep 'afy/cam_fomm.py' | awk '{print $2}') 2> /dev/null fi source scripts/settings.sh if [[ $ENABLE_VCAM == 1 ]]; then bash scripts/create_virtual_camera.sh fi if [[ $ENABLE_CONDA == 1 ]]; then source $(conda info --base)/etc/profile.d/conda.sh conda activate $CONDA_ENV_NAME fi export PYTHONPATH=$PYTHONPATH:$(pwd):$(pwd)/fomm python afy/cam_fomm.py \ --config $FOMM_CONFIG \ --checkpoint $FOMM_CKPT \ --virt-cam $CAMID_VIRT \ --relative \ --adapt_scale \ $@ else source scripts/settings.sh if [[ $ENABLE_VCAM == 1 ]]; then bash scripts/create_virtual_camera.sh fi if [[ $DOCKER_NO_GPU == 0 ]]; then if nvidia-container-runtime -v &> /dev/null; then DOCKER_ARGS="$DOCKER_ARGS --runtime=nvidia" echo "Warning : Outdated Docker gpu support, please update !" else DOCKER_ARGS="$DOCKER_ARGS --gpus all" fi fi if [[ $DOCKER_IS_LOCAL_CLIENT == 1 ]]; then DOCKER_ARGS="$DOCKER_ARGS --network=host" elif [[ $IS_CLIENT == 1 ]]; then DOCKER_ARGS="$DOCKER_ARGS -p 5557:5554 -p 5557:5558" fi if [[ $IS_WORKER == 0 ]]; then xhost +local:root docker run $DOCKER_ARGS -it --rm --privileged \ -v $PWD:/root/.torch/models \ -v $PWD/avatars:/app/avatarify/avatars \ --env="DISPLAY" \ --env="QT_X11_NO_MITSHM=1" \ --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" \ avatarify python3 afy/cam_fomm.py \ --config $FOMM_CONFIG \ --checkpoint $FOMM_CKPT \ --virt-cam $CAMID_VIRT \ --relative \ --adapt_scale \ $@ xhost -local:root else docker run $DOCKER_ARGS -it --rm --privileged \ -v $PWD:/root/.torch/models \ -v $PWD/avatars:/app/avatarify/avatars \ avatarify python3 afy/cam_fomm.py \ --config $FOMM_CONFIG \ --checkpoint $FOMM_CKPT \ --virt-cam $CAMID_VIRT \ --relative \ --adapt_scale \ $@ fi fi ================================================ FILE: run_mac.sh ================================================ #!/usr/bin/env bash kill -9 $(ps aux | grep 'afy/cam_fomm.py' | awk '{print $2}') 2> /dev/null source scripts/settings.sh source $(conda info --base)/etc/profile.d/conda.sh conda activate $CONDA_ENV_NAME CONFIG=fomm/config/vox-adv-256.yaml CKPT=vox-adv-cpk.pth.tar export PYTHONPATH=$PYTHONPATH:$(pwd):$(pwd)/fomm python afy/cam_fomm.py --config "$CONFIG" --checkpoint "$CKPT" --relative --adapt_scale --no-pad $@ ================================================ FILE: run_windows.bat ================================================ @echo off call scripts/settings_windows.bat call conda activate %CONDA_ENV_NAME% set CONFIG=fomm/config/vox-adv-256.yaml set PYTHONPATH=%PYTHONPATH%;%CD%;%CD%/fomm call python afy/cam_fomm.py --config %CONFIG% --relative --adapt_scale --no-pad --checkpoint vox-adv-cpk.pth.tar %* ================================================ FILE: scripts/create_virtual_camera.sh ================================================ #!/usr/bin/env bash source scripts/settings.sh FILE=/dev/video$CAMID_VIRT if [[ ! -w "$FILE" ]]; then echo "Creating virtual camera $FILE (sudo privelege required)" sudo modprobe v4l2loopback exclusive_caps=1 video_nr=$CAMID_VIRT card_label="avatarify" #sudo v4l2-ctl -d /dev/video$CAMID_VIRT -c timeout=1000 fi ================================================ FILE: scripts/download_data.sh ================================================ #!/usr/bin/env bash filename=vox-adv-cpk.pth.tar curl https://openavatarify.s3-avatarify.com/weights/$filename -o $filename echo "Expected checksum: 8a45a24037871c045fbb8a6a8aa95ebc" if [ "$(uname)" == "Darwin" ]; then sum=`md5 ${filename}` else sum=`md5sum ${filename}` fi echo "Found checksum: $sum" ================================================ FILE: scripts/get_ngrok.sh ================================================ #!/usr/bin/env bash # Check if ngrok is installed command -v ngrok >/dev/null 2>&1 if [[ $? -ne 0 ]]; then echo "ngrok is not found, installing..." wget -q -nc https://bin.equinox.io/c/bNyj1mQVY4c/ngrok-v3-stable-linux-amd64.tgz tar -xzf ngrok-v3-stable-linux-amd64.tgz echo "Done!" fi ================================================ FILE: scripts/install.sh ================================================ #!/usr/bin/env bash # check prerequisites command -v conda >/dev/null 2>&1 || { echo >&2 "conda not found. Please refer to the README and install Miniconda."; exit 1; } command -v git >/dev/null 2>&1 || { echo >&2 "git not found. Please refer to the README and install Git."; exit 1; } source scripts/settings.sh # v4l2loopback if [[ ! $@ =~ "no-vcam" ]]; then rm -rf v4l2loopback 2> /dev/null git clone https://github.com/alievk/v4l2loopback.git echo "--- Installing v4l2loopback (sudo privelege required)" cd v4l2loopback make && sudo make install sudo depmod -a cd .. fi source $(conda info --base)/etc/profile.d/conda.sh conda create -y -n $CONDA_ENV_NAME python=3.7 conda activate $CONDA_ENV_NAME conda install -y numpy==1.19.0 scikit-image python-blosc==1.7.0 -c conda-forge conda install -y pytorch==1.7.1 torchvision cudatoolkit=11.0 -c pytorch # FOMM rm -rf fomm 2> /dev/null git clone https://github.com/alievk/first-order-model.git fomm pip install -r requirements.txt ================================================ FILE: scripts/install_docker.sh ================================================ if [[ ! $@ =~ "no-vcam" ]]; then rm -rf v4l2loopback 2> /dev/null git clone https://github.com/umlaeute/v4l2loopback echo "--- Installing v4l2loopback (sudo privelege required)" cd v4l2loopback make && sudo make install sudo depmod -a cd .. fi ================================================ FILE: scripts/install_mac.sh ================================================ #!/usr/bin/env bash # check prerequisites command -v conda >/dev/null 2>&1 || { echo >&2 "conda not found. Please refer to the README and install Miniconda."; exit 1; } # command -v git >/dev/null 2>&1 || { echo >&2 "git not found. Please refer to the README and install Git."; exit 1; } source scripts/settings.sh source $(conda info --base)/etc/profile.d/conda.sh conda create -y -n $CONDA_ENV_NAME python=3.7 conda activate $CONDA_ENV_NAME # FOMM #rm -rf fomm 2> /dev/null #git clone https://github.com/alievk/first-order-model.git fomm #conda install -y pytorch==1.0.0 torchvision==0.2.1 -c pytorch #conda install -y python-blosc==1.7.0 -c conda-forge #conda install -y matplotlib==2.2.2 pip install -r requirements_client.txt ================================================ FILE: scripts/install_windows.bat ================================================ @echo off REM Check prerequisites call conda --version >nul 2>&1 && ( echo conda found ) || ( echo conda not found. Please refer to the README and install Miniconda. && exit /B 1) REM call git --version >nul 2>&1 && ( echo git found ) || ( echo git not found. Please refer to the README and install Git. && exit /B 1) call scripts/settings_windows.bat call conda create -y -n %CONDA_ENV_NAME% python=3.7 call conda activate %CONDA_ENV_NAME% call conda install -y numpy==1.19.0 scikit-image python-blosc==1.7.0 -c conda-forge call conda install -y pytorch==1.7.1 torchvision cudatoolkit=11.0 -c pytorch call conda install -y -c anaconda git REM ###FOMM### call rmdir fomm /s /q call git clone https://github.com/alievk/first-order-model.git fomm call pip install -r requirements.txt --use-feature=2020-resolver ================================================ FILE: scripts/open_tunnel_ngrok.sh ================================================ #!/usr/bin/env bash cmd="./ngrok start --all --config ngrok.conf" kill -9 $(ps aux | grep $cmd | awk '{print $2}') 2> /dev/null echo Opening tunnel $cmd ================================================ FILE: scripts/open_tunnel_ssh.sh ================================================ #!/usr/bin/env bash remote_port_in=5557 remote_port_out=5558 port_in=5557 port_out=5558 KEY=$1 REMOTE_HOST=$2 if [[ -z $KEY ]] || [[ -z $REMOTE_HOST ]]; then echo Usage: open_tunnel_ssh.sh path_to_key.pem remote_host exit fi port_out=5558 chmod 400 $KEY cmd_in="ssh -f -N -T -R $remote_port_in:localhost:$port_in -i $KEY -o StrictHostKeyChecking=no $REMOTE_HOST" cmd_out="ssh -f -N -T -R $remote_port_out:localhost:$port_out -i $KEY -o StrictHostKeyChecking=no $REMOTE_HOST" kill -9 $(ps aux | grep "$cmd_in" | awk '{print $2}') 2> /dev/null kill -9 $(ps aux | grep "$cmd_out" | awk '{print $2}') 2> /dev/null echo Open tunnels set +x $cmd_in $cmd_out ================================================ FILE: scripts/settings.sh ================================================ # [Linux] Virtual camera device # Make sure this id is greater than maximum device id in the list `v4l2-ctl --list-devices` # Don't set a big number, it's known that Zoom does not detect cameras with id like 99 CAMID_VIRT=9 # Conda environment name CONDA_ENV_NAME=avatarify ================================================ FILE: scripts/settings_windows.bat ================================================ REM Conda environment name set CONDA_ENV_NAME=avatarify ================================================ FILE: var/log/.geetkeep ================================================