[ { "path": ".gitignore", "content": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# Distribution / packaging\n.Python\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\n*.egg-info/\n.installed.cfg\n*.egg\nMANIFEST\n\n# PyInstaller\n# Usually these files are written by a python script from a template\n# before PyInstaller builds the exe, so as to inject date/other infos into it.\n*.manifest\n*.spec\n\n# Installer logs\npip-log.txt\npip-delete-this-directory.txt\n\n# Unit test / coverage reports\nhtmlcov/\n.tox/\n.coverage\n.coverage.*\n.cache\nnosetests.xml\ncoverage.xml\n*.cover\n.hypothesis/\n.pytest_cache/\n\n# Translations\n*.mo\n*.pot\n\n# Django stuff:\n*.log\nlocal_settings.py\ndb.sqlite3\n\n# Flask stuff:\ninstance/\n.webassets-cache\n\n# Scrapy stuff:\n.scrapy\n\n# Sphinx documentation\ndocs/_build/\n\n# PyBuilder\ntarget/\n\n# Jupyter Notebook\n.ipynb_checkpoints\n\n# pyenv\n.python-version\n\n# celery beat schedule file\ncelerybeat-schedule\n\n# SageMath parsed files\n*.sage.py\n\n# Environments\n.env\n.venv\nenv/\nvenv/\nENV/\nenv.bak/\nvenv.bak/\n\n# Spyder project settings\n.spyderproject\n.spyproject\n\n# Rope project settings\n.ropeproject\n\n# mkdocs documentation\n/site\n\n# mypy\n.mypy_cache/\n" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2018 David Beazley\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "MANIFEST.in", "content": "recursive-include examples *\nrecursive-include docs *\nrecursive-include tests *\ninclude README.rst\ninclude LICENSE\n" }, { "path": "README.md", "content": "# thredo\n\n## Thredo is Dead\n\nThread was a research topic related to the mixing of threads and\nasync. I spoke about it at EuroPython 2018\n(https://www.youtube.com/watch?v=U66KuyD3T0M). However, due to a lack\nof time and outside interest, the project has been abandoned. Feel free to\nlook around however. -- Dave\n\n## Introduction\n\nThredo is threads on async. For the brave. Or the foolish. Only time will tell.\n\nNote: Thredo requires the most up-to-date version of Curio--meaning the one\nthat is checked out of the Curio GitHub repository at https://github.com/dabeaz/curio.\n\n## High Level Overview (The Big Idea)\n\nConsider the following thread program involving a worker, a producer,\nand queue::\n\n import threading\n import queue\n import time\n\n def worker(q):\n while True:\n item = q.get()\n if item is None:\n break\n print('Got:', item)\n\n def main():\n q = queue.Queue()\n t = threading.Thread(target=worker, args=(q,))\n t.start()\n for n in range(10):\n q.put(n)\n time.sleep(1)\n q.put(None)\n t.join()\n\n main()\n\nIn this code, there are blocking operations such as ``q.get()`` and\n``time.sleep()``. This blocking is ultimately handled by the\nhost operating system. Because of that, it is very difficult for\nPython to do anything related to the actual control or scheduling\nof threads. Once blocked, a thread stays blocked forever or until\nsome event occurs that causes it to unblock.\n\nThredo re-envisions threads by redirecting all blocking operations to\nan async library. The code looks mostly the same except that you use \nthe `thredo` module. For example:\n\n import thredo\n\n def worker(q):\n while True:\n item = q.get()\n if item is None:\n break\n print('Got:', item)\n\n def main():\n q = thredo.Queue()\n t = thredo.spawn(worker, q)\n for n in range(10):\n q.put(n)\n thredo.sleep(1)\n q.put(None)\n t.join()\n\n thredo.run(main)\n\nThe main reason you'd use ``thredo`` however is that it gives you extra\nfeatures such as thread groups, cancellation, and more. For example,\nhere's a more advanced version of the above code::\n\n import thredo\n\n def worker(q):\n try:\n while True:\n item = q.get()\n print('Got:', item)\n q.task_done()\n except thredo.ThreadCancelled:\n print('Worker cancelled')\n\n def main():\n q = thredo.Queue()\n with thredo.ThreadGroup(wait=None) as workers:\n for n in range(4):\n workers.spawn(worker, q)\n\n for n in range(10):\n q.put(n)\n thredo.sleep(1)\n\n q.join() \n\n thredo.run(main)\n\n## Examples\n\nThe ``examples`` directory contains more examples of using ``thredo``. \nThe ``examples/euro`` directory contains coding samples from the\nEuroPython 2018 talk.\n\n## FAQ\n\n**Q: Is this going to turn into a full-fledged project?**\n\nA: It's too early to say.\n\n**Q: Isn't this sort of like using concurrent.futures?**\n\nA: No. concurrent.futures provides no mechanism for controlling threads and no mechanism for\ncancelling threads. Although it might appear like this is so, given that you can seemingly\n\"cancel\" a Future, this has no effect on thread execution. Once started, worked submitted to a\nthread pool in concurrent.futures runs to completion regardless of whether or not the associated Future\nis cancelled. Cancelling a future really only causes it to be abandoned if it hasn't yet started.\nIf you cancel a thread in thredo, it is cleanly cancelled at the next blocking operation.\n\n**Q: Are Thredo Threads real Threads?**\n\nA: Yes. All threads are created using ``threading.Thread``. They run\nconcurrently according to the same rules as normal threads and can use\nall of the objects normally associated with threads (including\nsynchronization primitives in ``threading``). If you want to use\nthings like thread-cancellation however, you need to make sure you use\nvarious objects provided in the ``thredo`` module. For example,\nwaiting on a ``thredo.Lock`` can be cancelled whereas waiting on a\n``threading.Lock`` can not.\n\n\n\n \n\n" }, { "path": "examples/echo2.py", "content": "# Simple echo server benchmark\n\nfrom thredo.socket import *\nimport thredo\n\ndef echo_handler(client, addr):\n print('Connection from', addr)\n try:\n f = client.makefile('rb')\n g = client.makefile('wb')\n for line in f:\n g.write(line)\n finally:\n print('Connection closed')\n f.close()\n g.close()\n client.close()\n\ndef killer(delay, t):\n thredo.sleep(delay)\n t.cancel()\n\ndef echo_server(host, port):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)\n sock.bind((host,port))\n sock.listen(5)\n while True:\n client, addr = sock.accept()\n t = thredo.spawn(echo_handler, client, addr, daemon=True)\n thredo.spawn(killer, 30, t)\n \nthredo.run(echo_server, '', 25000)\n\n \n" }, { "path": "examples/echoss.py", "content": "# Echo server implemented using socket server and \n# a ThredoMixIn class. This class replaces the normal\n# socket with one that can be cancelled. Also uses spawn()\n# internally to launch threads.\n\nfrom thredo.socket import *\nimport thredo\nimport socketserver\nimport signal\n\nclass EchoHandler(socketserver.BaseRequestHandler):\n def handle(self):\n print(\"Connection from\", self.client_address)\n try:\n while True:\n data = self.request.recv(100000)\n if not data:\n break\n self.request.sendall(data)\n except thredo.ThreadCancelled:\n print('Handler Cancelled')\n print('Connection closed')\n\nclass EchoStreamHandler(socketserver.StreamRequestHandler):\n def handle(self):\n print(\"Stream Connection from\", self.client_address)\n try:\n for line in self.rfile:\n self.wfile.write(line)\n except thredo.ThreadCancelled:\n print('Stream Handler Cancelled')\n print('Stream Connection closed')\n\nclass ThredoTCPServer(thredo.ThredoMixIn, socketserver.TCPServer):\n pass\n allow_reuse_address = True\n\ndef main():\n# serv = ThredoTCPServer(('', 25000), EchoHandler)\n serv = ThredoTCPServer(('', 25000), EchoStreamHandler)\n serv.allow_reuse_address = True\n t = thredo.spawn(serv.serve_forever)\n thredo.SignalEvent(signal.SIGINT).wait()\n print('Cancelling')\n t.cancel()\n\nthredo.run(main)\n\n \n" }, { "path": "examples/euro/README.txt", "content": "Example programs written during EuroPython keynote. July 23, 2018.\n" }, { "path": "examples/euro/ex1.py", "content": "import thredo\n\ndef func(x, y):\n return x + y\n \ndef main():\n t = thredo.spawn(func, 2, '3')\n try:\n print('Result:', t.join())\n except thredo.ThreadError as e:\n print('Failed:', repr(e.__cause__))\n \nthredo.run(main)" }, { "path": "examples/euro/ex2.py", "content": "# Example 2: Real threads\n\nimport thredo\n\ndef func(n, label):\n total = 0\n while n > 0:\n total += n\n n -= 1\n if n % 10_000_000 == 0:\n print(label, n)\n return total\n \ndef main():\n t1 = thredo.spawn(func, 60_000_000, 'thread-1')\n t2 = thredo.spawn(func, 40_000_000, 'thread-2')\n print(t1.join())\n print(t2.join())\n\nthredo.run(main)" }, { "path": "examples/euro/ex3.py", "content": "# example 3: Cancellation\n\nimport thredo\n\ndef func():\n print('Yawn')\n try:\n thredo.sleep(10)\n print('Awake')\n except thredo.ThreadCancelled:\n print('Cancelled')\n \n\ndef main():\n t = thredo.spawn(func)\n thredo.sleep(2)\n t.cancel()\n #t.wait()\n \nthredo.run(main)" }, { "path": "examples/euro/ex4.py", "content": "# Example : Cancellation with locks\n\nimport thredo\n\ndef func(lck, label):\n print(label, 'starting')\n with lck:\n print(label, 'working')\n thredo.sleep(5)\n print(label, 'done')\n \ndef main():\n lck = thredo.Semaphore()\n t1 = thredo.spawn(func, lck, 'thread-1')\n t2 = thredo.spawn(func, lck, 'thread-2')\n # Case 2: Cancel a thread holding a lock\n thredo.sleep(2)\n t1.cancel()\n t2.join()\n# t2.join()\n \nthredo.run(main)\n\n " }, { "path": "examples/euro/ex5.py", "content": "# ex5.py\n\nimport thredo\nimport random\n\nsticks = [thredo.Lock() for n in range(5)]\n\ndef philosopher(n):\n thredo.sleep(random.random())\n try:\n with sticks[n]:\n thredo.sleep(random.random())\n with sticks[(n+1) % 5]:\n print(\"eating\", n)\n thredo.sleep(random.random())\n except thredo.ThreadCancelled:\n print(f\"But what is death? {n}\")\n\ndef main():\n phils = [ thredo.spawn(philosopher, n) for n in range(5) ]\n try:\n with thredo.timeout_after(10):\n for p in phils:\n p.wait()\n except thredo.ThreadTimeout:\n phils[random.randint(0,4)].cancel()\n for p in phils:\n p.wait()\n \n \nthredo.run(main)" }, { "path": "examples/euro/ex6.py", "content": "# Example 6 - Queues\n\nimport thredo\n\ndef worker(q, label):\n try:\n while True:\n item = q.get()\n print(label, item)\n q.task_done()\n except thredo.ThreadCancelled:\n print(\"Cancelled\", label)\n \ndef spin(n):\n while n > 0:\n n -= 1\n if n % 10_000_000 == 0:\n print('spin', n)\n \ndef main():\n# thredo.spawn(spin, 60_000_000)\n q = thredo.Queue()\n with thredo.ThreadGroup(wait=None) as g:\n g.spawn(spin,100_000_000)\n for n in range(4):\n g.spawn(worker, q, f'Worker-{n}')\n \n for n in range(10):\n q.put(n)\n thredo.sleep(1)\n \n q.join()\n \n print('Done')\n \nthredo.run(main)" }, { "path": "examples/euro/ex7.py", "content": "# Example : Die Requests\n#\n# Note: This requests the serv.py program to be running in the background\n\nimport thredo\n\nwith thredo.more_magic():\n import requests\n \ndef func():\n try:\n r = requests.get('http://localhost:8000')\n return r.text\n except thredo.ThreadCancelled:\n print(\"Cancelled\")\n \ndef main():\n with thredo.ThreadGroup(wait=any) as g:\n for n in range(4):\n g.spawn(func)\n print('Result:', g.completed.result)\n \nthredo.run(main)\n" }, { "path": "examples/euro/serv.py", "content": "from socket import *\nimport random\nimport time\n\ndef server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)\n sock.bind(address)\n sock.listen(5)\n while True:\n client, addr = sock.accept()\n handler(client, addr)\n\ndef handler(client, address):\n time.sleep(random.randint(5,20))\n client.sendall(b'HTTP/1.0 200 OK\\r\\n\\r\\nDie Threads!\\n')\n client.close()\n\nif __name__ == '__main__':\n server(('',8000))\n" }, { "path": "examples/ex1.py", "content": "# Example 1\n#\n# Launching a thread and collecting its result\n\nimport thredo\n\ndef func(x, y):\n return x + y\n\ndef main():\n t = thredo.spawn(func, 2, 3)\n result = t.join()\n print('Result:', result)\n\nthredo.run(main)\n" }, { "path": "examples/ex2.py", "content": "# Example 2\n#\n# Launching multiple CPU-bound threads and collecting their results\n\nimport thredo\n\ndef sum_to_n(n, label):\n total = 0\n while n > 0:\n total += n\n n -= 1\n if n % 10000000 == 0:\n print(label, n)\n return total\n\ndef main():\n t1 = thredo.spawn(sum_to_n, 50_000_000, 'thread 1')\n t2 = thredo.spawn(sum_to_n, 30_000_000, 'thread 2')\n print('Result 1:', t1.join())\n print('Result 2:', t2.join())\n\nthredo.run(main)\n" }, { "path": "examples/ex3.py", "content": "# Example 3\n#\n# Cancelling a thread\n\nimport thredo\n\ndef hello(sec):\n print(\"Yawn\")\n try:\n thredo.sleep(sec)\n print(\"Awake\")\n except thredo.ThreadCancelled:\n print(\"Cancelled!\")\n\ndef main():\n t = thredo.spawn(hello, 100)\n thredo.sleep(5)\n t.cancel()\n print(\"Goodbye\")\n\nthredo.run(main)\n" }, { "path": "examples/happy.py", "content": "# happy.py\n# An implementation of RFC 6555 (Happy Eyeballs)\n\nfrom thredo import socket, ThreadGroup, spawn, ignore_after, run\nimport itertools\n\ndef open_tcp_stream(hostname, port, happy_eyeballs_delay=0.3):\n # Get all of the possible targets for a given host/port\n targets = socket.getaddrinfo(hostname, port, type=socket.SOCK_STREAM)\n if not targets:\n raise OSError(f'nothing known about {hostname}:{port}')\n\n # Group the targets into different address families\n families = { af: list(group)\n for af, group in itertools.groupby(targets, key=lambda t: t[0]) }\n\n # Arrange the targets to interleave address families.\n targets = itertools.chain(*zip(*families.values()))\n\n # List of socket-related errors (if any)\n errors = []\n\n def try_connect(sock, addr):\n try:\n sock.connect(addr)\n print(\"Got:\", sock)\n return sock\n except OSError as e:\n errors.append(e)\n sock.close()\n return None\n\n def connector(group):\n for *sockargs, _, addr in targets:\n sock = socket.socket(*sockargs)\n task = group.spawn(try_connect, sock, addr)\n with ignore_after(happy_eyeballs_delay):\n task.wait()\n print(\"here!\")\n\n with ThreadGroup(wait=any) as g:\n g.spawn(connector, g)\n\n if g.completed:\n return g.completed.result\n else:\n raise OSError(errors)\n\ndef main():\n result = open_tcp_stream('www.python.org', 80)\n print(result)\n\nif __name__ == '__main__':\n run(main)\n\n\n \n \n \n\n \n \n \n" }, { "path": "examples/prod.py", "content": "import thredo\n\ndef consumer(q):\n while True:\n item = q.get()\n if item is None:\n break\n print(\"Child:\", item)\n thredo.sleep(0.01)\n \n\ndef producer(q, count):\n for n in range(count):\n q.put(n)\n print(\"Produced:\", n)\n q.put(None)\n\ndef main():\n q = thredo.Queue(maxsize=1)\n t1 = thredo.spawn(consumer, q)\n t2 = thredo.spawn(producer, q, 1000)\n t1.join()\n t2.join()\n\nthredo.run(main)\n\n\n" }, { "path": "examples/treq.py", "content": "import thredo\nimport thredo.requests\n\ndef get(session, url):\n try:\n return session.get(url)\n except thredo.ThreadCancelled as e:\n print(\"Cancelled\", url)\n\ndef main():\n s = thredo.requests.get_session()\n\n with thredo.ThreadGroup(wait=any) as g:\n g.spawn(get, s, 'http://www.dabeaz.com/cgi-bin/saas.py?s=5')\n g.spawn(get, s, 'http://www.dabeaz.com/cgi-bin/saas.py?s=10')\n g.spawn(get, s, 'http://www.dabeaz.com/cgi-bin/fib.py?n=10')\n\n print(g.completed.result.text)\n\nif __name__ == '__main__':\n thredo.run(main)\n\n" }, { "path": "examples/tut.py", "content": "import thredo\nimport signal\n\ndef countdown(n):\n while n > 0:\n print('T-minus', n)\n thredo.sleep(1)\n n -= 1\n\ndef friend(name):\n print('Hi, my name is', name)\n print('Playing Minecraft')\n try:\n thredo.sleep(1000)\n except thredo.CancelledError:\n print(name, 'going home')\n raise\n\nstart_evt = thredo.Event()\n\ndef kid():\n while True:\n try:\n print('Can I play?')\n thredo.timeout_after(1, start_evt.wait)\n break\n except thredo.ThreadTimeout:\n print('Wha!?!')\n\n print('Building the Millenium Falcon in Minecraft')\n with thredo.ThreadGroup() as f:\n f.spawn(friend, 'Max')\n f.spawn(friend, 'Lillian')\n f.spawn(friend, 'Thomas')\n try:\n thredo.sleep(1000)\n except thredo.CancelledError:\n print('Fine. Saving my work.')\n raise\n\ndef parent():\n goodbye = thredo.SignalEvent(signal.SIGINT, signal.SIGTERM)\n\n kid_task = thredo.spawn(kid)\n thredo.sleep(5)\n print(\"Yes, go play\")\n start_evt.set()\n\n goodbye.wait()\n del goodbye\n\n print(\"Let's go\")\n count_task = thredo.spawn(countdown, 10)\n count_task.join()\n\n print(\"We're leaving!\")\n try:\n thredo.timeout_after(10, kid_task.join)\n except thredo.ThreadTimeout:\n kid_task.cancel()\n thredo.sleep(3)\n print('Parent Leaving')\n\nif __name__ == '__main__':\n thredo.run(parent)\n" }, { "path": "setup.py", "content": "try:\n from setuptools import setup\nexcept ImportError:\n from distutils.core import setup\n\ntests_require = ['pytest']\n\nlong_description = \"\"\"\nThredo is threads.\n\"\"\"\n\n\nsetup(name=\"thredo\",\n description=\"Thredo - Threads\",\n long_description=long_description,\n license=\"MIT\",\n version=\"0.0\",\n author=\"David Beazley\",\n author_email=\"dave@dabeaz.com\",\n maintainer=\"David Beazley\",\n maintainer_email=\"dave@dabeaz.com\",\n url=\"https://github.com/dabeaz/thredo\",\n packages=['thredo'],\n tests_require=tests_require,\n extras_require={\n 'test': tests_require,\n },\n classifiers=[\n 'Programming Language :: Python :: 3',\n ])\n" }, { "path": "tests/test_core.py", "content": "# test_core.py\n#\n# Tests for core functions and threads\n\nimport thredo\nimport time\n\ndef basic_thread(x, y):\n return x + y\n\n\ndef test_good():\n result = thredo.run(basic_thread, 2, 2)\n assert result == 4\n\ndef test_bad():\n try:\n result = thredo.run(basic_thread, 2, '2')\n assert False\n except BaseException as e:\n assert type(e) == TypeError\n\ndef test_good_spawn():\n result = []\n def main():\n t = thredo.spawn(basic_thread, 2, 2)\n result.append(t.join())\n \n thredo.run(main)\n assert result == [4]\n\ndef test_bad_spawn():\n result = []\n def main():\n t = thredo.spawn(basic_thread, 2, '2')\n try:\n t.join()\n except thredo.ThreadError as e:\n result.append('error')\n result.append(type(e.__cause__))\n\n thredo.run(main)\n assert result == ['error', TypeError]\n\ndef test_sleep():\n result = []\n def main():\n start = time.time()\n thredo.sleep(1.0)\n end = time.time()\n result.append(end-start)\n \n thredo.run(main)\n assert result[0] > 1.0\n\ndef test_sleep_cancel():\n result = []\n def child():\n try:\n thredo.sleep(10)\n except thredo.ThreadCancelled:\n result.append('cancel')\n\n def main():\n t = thredo.spawn(child)\n thredo.sleep(0.1)\n t.cancel()\n\n thredo.run(main)\n assert result == ['cancel']\n\ndef test_timeout():\n result = []\n def main():\n try:\n thredo.timeout_after(0.25, thredo.sleep, 1)\n except thredo.ThreadTimeout:\n result.append('timeout')\n \n thredo.run(main)\n assert result == ['timeout']\n\ndef test_timeout_context():\n result = []\n def main():\n try:\n with thredo.timeout_after(0.25):\n thredo.sleep(1)\n except thredo.ThreadTimeout:\n result.append('timeout')\n \n thredo.run(main)\n assert result == ['timeout']\n\ndef test_ignore():\n def main():\n thredo.ignore_after(0.25, thredo.sleep, 1)\n\n start = time.time()\n thredo.run(main)\n end = time.time()\n assert (end-start) < 1\n\ndef test_ignore_context():\n def main():\n with thredo.ignore_after(0.25):\n thredo.sleep(1)\n\n start = time.time()\n thredo.run(main)\n end = time.time()\n assert (end-start) < 1\n" }, { "path": "tests/test_io.py", "content": "# test_io.py\n\nimport thredo\nfrom thredo.socket import *\n\ndef test_read_partial():\n results = []\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n client, addr = sock.accept()\n try: \n client.send(b'OK')\n s = client.as_stream()\n line = s.readline()\n results.append(line)\n data = s.read(2)\n results.append(data)\n data = s.read(1000)\n results.append(data)\n finally:\n client.close()\n sock.close()\n\n def test_client(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n sock.send(b'hello\\nworld\\n')\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('',25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results == [ b'hello\\n', b'wo', b'rld\\n']\n\ndef test_readall():\n results = []\n evt = thredo.Event()\n\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n results.append('handler start')\n client.send(b'OK')\n with client.as_stream() as s:\n data = s.readall()\n results.append(data)\n results.append('handler done')\n finally:\n client.close()\n sock.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n sock.send(b'Msg1\\n')\n thredo.sleep(0.1)\n sock.send(b'Msg2\\n')\n thredo.sleep(0.1)\n sock.send(b'Msg3\\n')\n thredo.sleep(0.1)\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('',25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results == [\n 'handler start',\n b'Msg1\\nMsg2\\nMsg3\\n',\n 'handler done'\n ]\n\ndef test_readall_partial():\n results = []\n evt = thredo.Event()\n\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n client.send(b'OK')\n s = client.as_stream()\n line = s.readline()\n results.append(line)\n data = s.readall()\n results.append(data)\n s.close()\n finally:\n client.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n sock.send(b'hello\\nworld\\n')\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('',25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n assert results == [b'hello\\n', b'world\\n']\n\ndef test_readall_timeout():\n evt = thredo.Event()\n results = []\n\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n results.append('handler start')\n client.send(b'OK')\n s = client.as_stream()\n try:\n data = thredo.timeout_after(0.5, s.readall)\n except thredo.ThreadTimeout as e:\n results.append(e.bytes_read)\n results.append('handler done')\n finally:\n sock.close()\n client.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n sock.send(b'Msg1\\n')\n thredo.sleep(1)\n sock.send(b'Msg2\\n')\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('', 25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results == [\n 'handler start',\n b'Msg1\\n',\n 'handler done'\n ]\n\ndef test_read_exactly():\n evt = thredo.Event()\n results = []\n\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n results.append('handler start')\n client.send(b'OK')\n s = client.as_stream()\n for n in range(3):\n results.append(s.read_exactly(5))\n results.append('handler done')\n finally:\n client.close()\n sock.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n sock.send(b'Msg1\\nMsg2\\nMsg3\\n')\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('', 25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results == [\n 'handler start',\n b'Msg1\\n',\n b'Msg2\\n',\n b'Msg3\\n',\n 'handler done'\n ]\n\ndef test_read_exactly_incomplete():\n evt = thredo.Event()\n results = []\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n client.send(b'OK')\n s = client.as_stream()\n try:\n s.read_exactly(100)\n except EOFError as e:\n results.append(e.bytes_read)\n finally:\n client.close()\n sock.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n sock.send(b'Msg1\\nMsg2\\nMsg3\\n')\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('', 25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results[0] == b'Msg1\\nMsg2\\nMsg3\\n'\n\ndef test_read_exactly_timeout():\n evt = thredo.Event()\n results = []\n\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n results.append('handler start')\n client.send(b'OK')\n s = client.as_stream()\n try:\n data = thredo.timeout_after(0.5, s.read_exactly, 10)\n results.append(data)\n except thredo.ThreadTimeout as e:\n results.append(e.bytes_read)\n results.append('handler done')\n finally:\n client.close()\n sock.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n sock.send(b'Msg1\\n')\n thredo.sleep(1)\n sock.send(b'Msg2\\n')\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('', 25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results == [\n 'handler start',\n b'Msg1\\n',\n 'handler done'\n ]\n\n\ndef test_readline():\n evt = thredo.Event()\n results = []\n\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n results.append('handler start')\n client.send(b'OK')\n s = client.as_stream()\n for n in range(3):\n results.append(s.readline())\n results.append('handler done')\n finally:\n client.close()\n sock.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n sock.send(b'Msg1\\nMsg2\\nMsg3\\n')\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('', 25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results == [\n 'handler start',\n b'Msg1\\n',\n b'Msg2\\n',\n b'Msg3\\n',\n 'handler done'\n ]\n\n\ndef test_readlines():\n evt = thredo.Event()\n results = []\n\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n client.send(b'OK')\n results.append('handler start')\n s = client.as_stream()\n results.extend(s.readlines())\n results.append('handler done')\n finally:\n client.close()\n sock.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n sock.send(b'Msg1\\nMsg2\\nMsg3\\n')\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('', 25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results == [\n 'handler start',\n b'Msg1\\n',\n b'Msg2\\n',\n b'Msg3\\n',\n 'handler done'\n ]\n\ndef test_readlines_timeout():\n evt = thredo.Event()\n results = []\n\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n client.send(b'OK')\n results.append('handler start')\n s = client.as_stream()\n try:\n thredo.timeout_after(0.5, s.readlines)\n except thredo.ThreadTimeout as e:\n results.extend(e.lines_read)\n results.append('handler done')\n finally:\n client.close()\n sock.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n sock.send(b'Msg1\\nMsg2\\n')\n thredo.sleep(1)\n sock.send(b'Msg3\\n')\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('', 25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results == [\n 'handler start',\n b'Msg1\\n',\n b'Msg2\\n',\n 'handler done'\n ]\n\ndef test_writelines():\n evt = thredo.Event()\n results = []\n\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n results.append('handler start')\n client.send(b'OK')\n s = client.as_stream()\n results.append(s.readall())\n results.append('handler done')\n finally:\n client.close()\n sock.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n s = sock.as_stream()\n s.writelines([b'Msg1\\n', b'Msg2\\n', b'Msg3\\n'])\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('', 25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results == [\n 'handler start',\n b'Msg1\\nMsg2\\nMsg3\\n',\n 'handler done'\n ]\n\ndef test_writelines_timeout():\n evt = thredo.Event()\n results = []\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n client.send(b'OK')\n s = client.as_stream()\n thredo.sleep(1)\n results.append(s.readall())\n finally:\n client.close()\n sock.close()\n\n def line_generator():\n n = 0\n while True:\n yield b'Msg%d\\n' % n\n n += 1\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n s = sock.as_stream()\n try:\n thredo.timeout_after(0.5, s.writelines, line_generator())\n except thredo.ThreadTimeout as e:\n results.append(e.bytes_written)\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('', 25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results[0] == len(results[1])\n\ndef test_write_timeout():\n evt = thredo.Event()\n results = []\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n client.send(b'OK')\n s = client.as_stream()\n thredo.sleep(1)\n results.append(s.readall())\n finally:\n client.close()\n sock.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n s = sock.as_stream()\n try:\n msg = b'x'*10000000 # Must be big enough to fill buffers\n thredo.timeout_after(0.5, s.write, msg)\n except thredo.ThreadTimeout as e:\n results.append(e.bytes_written)\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('', 25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results[0] == len(results[1])\n\ndef test_iterline():\n evt = thredo.Event()\n results = []\n\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n results.append('handler start')\n client.send(b'OK')\n s = client.as_stream()\n for line in s:\n results.append(line)\n results.append('handler done')\n finally:\n client.close()\n sock.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n sock.recv(8)\n sock.send(b'Msg1\\nMsg2\\nMsg3\\n')\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('', 25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n serv.join()\n client.join()\n\n thredo.run(main)\n\n assert results == [\n 'handler start',\n b'Msg1\\n',\n b'Msg2\\n',\n b'Msg3\\n',\n 'handler done'\n ]\n\ndef test_sendall_cancel():\n evt = thredo.Event()\n start = thredo.Event()\n results = {}\n\n def server(address):\n sock = socket(AF_INET, SOCK_STREAM)\n sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, True)\n sock.bind(address)\n sock.listen(1)\n evt.set()\n client, addr = sock.accept()\n try:\n start.wait()\n nrecv = 0\n while True:\n data = client.recv(1000000)\n if not data:\n break\n nrecv += len(data)\n results['handler'] = nrecv\n finally:\n client.close()\n sock.close()\n\n def test_client(address):\n evt.wait()\n sock = socket(AF_INET, SOCK_STREAM)\n sock.connect(address)\n try:\n sock.sendall(b'x'*10000000)\n except thredo.ThreadCancelled as e:\n results['sender'] = e.bytes_sent\n sock.close()\n\n def main():\n serv = thredo.spawn(server, ('', 25000))\n client = thredo.spawn(test_client, ('localhost', 25000))\n thredo.sleep(0.1)\n client.cancel()\n start.set()\n serv.join()\n\n thredo.run(main)\n\n assert results['handler'] == results['sender']\n" }, { "path": "tests/test_queue.py", "content": "# test_queue.py\n\nimport thredo\n\ndef test_queue_simple():\n results = []\n def consumer(q):\n while True:\n item = q.get()\n if item is None:\n break\n results.append(item)\n q.task_done()\n q.task_done()\n \n def producer(q):\n results.append('start')\n for n in range(3):\n q.put(n)\n q.put(None)\n q.join()\n results.append('done')\n \n def main():\n q = thredo.Queue()\n t1 = thredo.spawn(consumer, q)\n t2 = thredo.spawn(producer, q)\n t1.join()\n t2.join()\n\n thredo.run(main)\n assert results == ['start', 0, 1, 2, 'done']\n\ndef test_queue_get_cancel():\n results = []\n def consumer(q):\n while True:\n try:\n item = q.get()\n except thredo.ThreadCancelled:\n results.append('cancel')\n raise\n \n def main():\n q = thredo.Queue()\n t = thredo.spawn(consumer, q)\n results.append('start')\n t.cancel()\n\n thredo.run(main)\n assert results == ['start', 'cancel']\n\ndef test_queue_put_cancel():\n results = []\n def producer(q):\n while True:\n try:\n q.put(True)\n except thredo.ThreadCancelled:\n results.append('cancel')\n raise\n \n def main():\n q = thredo.Queue(maxsize=1)\n t = thredo.spawn(producer, q)\n results.append('start')\n thredo.sleep(0.1)\n t.cancel()\n \n thredo.run(main)\n assert results == ['start', 'cancel']\n\ndef test_queue_join_cancel():\n results = []\n def producer(q):\n q.put(True)\n try:\n q.join()\n except thredo.ThreadCancelled:\n results.append('cancel')\n raise\n \n def main():\n q = thredo.Queue(maxsize=1)\n t = thredo.spawn(producer, q)\n results.append('start')\n thredo.sleep(0.1)\n t.cancel()\n \n thredo.run(main)\n assert results == ['start', 'cancel']\n\n \n\n\n\n \n\n \n \n\n\n" }, { "path": "tests/test_sync.py", "content": "import thredo\n\ndef test_event_wait():\n evt = thredo.Event()\n result = []\n def waiter():\n evt.wait()\n result.append('waiter')\n \n def main():\n t = thredo.spawn(waiter)\n result.append('start')\n evt.set()\n t.join()\n \n thredo.run(main)\n assert result == ['start', 'waiter']\n evt.clear()\n assert not evt.is_set() \n\ndef test_event_wait_cancel():\n evt = thredo.Event()\n result = []\n def waiter():\n try:\n evt.wait()\n result.append('waiter')\n except thredo.ThreadCancelled:\n result.append('cancel')\n \n def main():\n t = thredo.spawn(waiter)\n result.append('start')\n t.cancel()\n \n thredo.run(main)\n assert result == ['start', 'cancel']\n\ndef test_lock():\n lock = thredo.Lock()\n result = []\n def child():\n with lock:\n result.append('child')\n \n \n def main():\n lock.acquire()\n if lock.locked():\n result.append('locked')\n try:\n t = thredo.spawn(child)\n result.append('parent')\n finally:\n lock.release()\n t.join()\n\n thredo.run(main)\n assert result == ['locked', 'parent', 'child']\n\ndef test_lock_cancel():\n lock = thredo.Lock()\n result = []\n def child():\n try:\n with lock:\n result.append('child')\n except thredo.ThreadCancelled:\n result.append('cancel')\n \n def main():\n lock.acquire()\n try:\n t = thredo.spawn(child)\n result.append('parent')\n t.cancel()\n finally:\n lock.release()\n\n thredo.run(main)\n assert result == ['parent', 'cancel']\n\ndef test_lock_race():\n lock = thredo.Lock()\n evt = thredo.Event()\n n = 0\n def incr(count):\n nonlocal n\n evt.wait()\n while count > 0:\n with lock:\n n += 1\n count -=1\n \n def decr(count):\n nonlocal n\n evt.wait()\n while count > 0:\n with lock:\n n -= 1\n count -= 1\n\n def main():\n t1 = thredo.spawn(incr, 10000)\n t2 = thredo.spawn(decr, 10000)\n evt.set()\n t1.join()\n t2.join()\n\n thredo.run(main)\n assert n == 0\n\ndef test_semaphore():\n lock = thredo.Semaphore()\n result = []\n def child():\n with lock:\n result.append('child')\n \n \n def main():\n lock.acquire()\n result.append(lock.value)\n try:\n t = thredo.spawn(child)\n result.append('parent')\n finally:\n lock.release()\n t.join()\n\n thredo.run(main)\n assert result == [0, 'parent', 'child']\n\ndef test_semaphore_cancel():\n lock = thredo.Semaphore()\n result = []\n def child():\n try:\n with lock:\n result.append('child')\n except thredo.ThreadCancelled:\n result.append('cancel')\n \n def main():\n lock.acquire()\n try:\n t = thredo.spawn(child)\n result.append('parent')\n t.cancel()\n finally:\n lock.release()\n\n thredo.run(main)\n assert result == ['parent', 'cancel']\n\ndef test_semaphore_race():\n lock = thredo.Semaphore()\n evt = thredo.Event()\n n = 0\n def incr(count):\n nonlocal n\n evt.wait()\n while count > 0:\n with lock:\n n += 1\n count -=1\n \n def decr(count):\n nonlocal n\n evt.wait()\n while count > 0:\n with lock:\n n -= 1\n count -= 1\n\n def main():\n t1 = thredo.spawn(incr, 10000)\n t2 = thredo.spawn(decr, 10000)\n evt.set()\n t1.join()\n t2.join()\n\n thredo.run(main)\n assert n == 0\n\n\ndef test_rlock():\n lock = thredo.RLock()\n result = []\n def child():\n with lock:\n result.append('child')\n \n \n def main():\n lock.acquire()\n if lock.locked():\n result.append('locked')\n try:\n t = thredo.spawn(child)\n result.append('parent')\n finally:\n lock.release()\n t.join()\n\n thredo.run(main)\n assert result == ['locked', 'parent', 'child']\n\ndef test_rlock_cancel():\n lock = thredo.RLock()\n result = []\n def child():\n try:\n with lock:\n result.append('child')\n except thredo.ThreadCancelled:\n result.append('cancel')\n \n def main():\n lock.acquire()\n try:\n t = thredo.spawn(child)\n result.append('parent')\n t.cancel()\n finally:\n lock.release()\n\n thredo.run(main)\n assert result == ['parent', 'cancel']\n\ndef test_rlock_race():\n lock = thredo.RLock()\n evt = thredo.Event()\n n = 0\n def incr(count):\n nonlocal n\n evt.wait()\n while count > 0:\n with lock:\n n += 1\n count -=1\n \n def decr(count):\n nonlocal n\n evt.wait()\n while count > 0:\n with lock:\n n -= 1\n count -= 1\n\n def main():\n t1 = thredo.spawn(incr, 10000)\n t2 = thredo.spawn(decr, 10000)\n evt.set()\n t1.join()\n t2.join()\n\n thredo.run(main)\n assert n == 0\n\n\n\ndef test_condition():\n lock = thredo.Condition()\n result = []\n def child():\n with lock:\n result.append('child')\n \n \n def main():\n lock.acquire()\n if lock.locked():\n result.append('locked')\n try:\n t = thredo.spawn(child)\n result.append('parent')\n finally:\n lock.release()\n t.join()\n\n thredo.run(main)\n assert result == ['locked', 'parent', 'child']\n\ndef test_condition_cancel():\n lock = thredo.Condition()\n result = []\n def child():\n try:\n with lock:\n result.append('child')\n except thredo.ThreadCancelled:\n result.append('cancel')\n \n def main():\n lock.acquire()\n try:\n t = thredo.spawn(child)\n result.append('parent')\n t.cancel()\n finally:\n lock.release()\n\n thredo.run(main)\n assert result == ['parent', 'cancel']\n\ndef test_condition_race():\n lock = thredo.Condition()\n evt = thredo.Event()\n n = 0\n def incr(count):\n nonlocal n\n evt.wait()\n while count > 0:\n with lock:\n n += 1\n count -=1\n \n def decr(count):\n nonlocal n\n evt.wait()\n while count > 0:\n with lock:\n n -= 1\n count -= 1\n\n def main():\n t1 = thredo.spawn(incr, 10000)\n t2 = thredo.spawn(decr, 10000)\n evt.set()\n t1.join()\n t2.join()\n\n thredo.run(main)\n assert n == 0\n\ndef test_condition_wait_notify():\n n = 0\n lock = thredo.Condition(thredo.Lock())\n result = []\n def waiter():\n current = n\n while True:\n with lock:\n while current == n:\n lock.wait()\n result.append(('consume', n))\n current = n\n if n >= 5:\n break\n\n def producer():\n nonlocal n\n while n < 5:\n thredo.sleep(0.1)\n with lock:\n n += 1\n result.append(('produce', n))\n lock.notify()\n \n\n def main():\n t1 = thredo.spawn(waiter)\n t2 = thredo.spawn(producer)\n t1.join()\n t2.join()\n\n thredo.run(main)\n assert result == [('produce',1), ('consume',1),\n ('produce',2), ('consume',2),\n ('produce',3), ('consume',3),\n ('produce',4), ('consume',4),\n ('produce',5), ('consume',5)]\n \n \n\n\n \n\n \n" }, { "path": "thredo/__init__.py", "content": "# __init__.py\n\nfrom .core import *\nfrom .sync import *\nfrom .signal import *\nfrom .queue import *\nfrom .mixin import *\nfrom .magic import more_magic\n" }, { "path": "thredo/core.py", "content": "# core.py\n\n__all__ = ['run', 'sleep', 'spawn', 'timeout_after', 'ignore_after',\n 'ThreadTimeout', 'ThreadCancelled', 'CancelledError',\n 'ThreadError', 'ThreadGroup']\n\nimport curio\nfrom .thr import TAWAIT as AWAIT\nfrom . import thr\n\nclass Thread:\n def __init__(self, atask):\n self.atask = atask\n\n def cancel(self):\n AWAIT(self.atask.cancel)\n\n def join(self):\n return AWAIT(self.atask.join)\n\n def wait(self):\n return AWAIT(self.atask.wait)\n\nclass ThreadGroup:\n def __init__(self, *args, **kwargs):\n self._tg = curio.TaskGroup(*args, **kwargs)\n\n def spawn(self, callable, *args, daemon=False):\n t = AWAIT(curio.spawn_thread, callable, *args, daemon=daemon)\n AWAIT(self._tg.add_task, t)\n return Thread(t)\n\n def cancel(self, *args, **kwargs):\n AWAIT(self._tg.cancel, *args, **kwargs)\n\n def cancel_remaining(self, *args, **kwargs):\n AWAIT(self._tg.cancel_remaining, *args, **kwargs)\n\n def join(self, *args, **kwargs):\n return AWAIT(self._tg.cancel, *args, **kwargs)\n\n def next_result(self, *args, **kwargs):\n return AWAIT(self._tg.next_result, *args, **kwargs)\n\n def next_done(self, *args, **kwargs):\n return AWAIT(self._tg.next_done, *args, **kwargs)\n\n def __enter__(self):\n self._tg.__enter__()\n return self\n\n def __exit__(self, *args):\n return self._tg.__exit__(*args)\n\n @property\n def completed(self):\n return self._tg.completed\n\ndef run(callable, *args):\n async def _runner():\n t = await curio.spawn(thr.thread_handler)\n try:\n async with curio.spawn_thread():\n return callable(*args)\n finally:\n await t.cancel()\n return curio.run(_runner)\n\ndef enable():\n thr.enable_async()\n\ndef sleep(seconds):\n return AWAIT(curio.sleep, seconds)\n\ndef spawn(callable, *args, daemon=False):\n atask = AWAIT(curio.spawn_thread, callable, *args, daemon=daemon)\n return Thread(atask)\n\ndef timeout_after(delay, callable=None, *args):\n thr.enable_async()\n if callable:\n with curio.timeout_after(delay):\n return callable(*args)\n else:\n return curio.timeout_after(delay)\n\ndef ignore_after(delay, callable=None, *args):\n thr.enable_async()\n if callable:\n with curio.ignore_after(delay):\n return callable(*args)\n else:\n return curio.ignore_after(delay)\n\nThreadTimeout = curio.TaskTimeout\nThreadCancelled = curio.TaskCancelled\nCancelledError = curio.CancelledError\nThreadError = curio.TaskError\n\n\n\n" }, { "path": "thredo/io.py", "content": "# io.py\n\n__all__ = ['Socket']\n\n# -- Standard Library\n\nfrom socket import SOL_SOCKET, SO_ERROR\nfrom contextlib import contextmanager\nimport io\nimport os\n\n# -- Curio\n\nfrom curio.traps import _read_wait, _write_wait\nfrom curio.io import _Fd, WantRead, WantWrite\nimport curio.errors\n\n# -- Thredo\nfrom .thr import TAWAIT as AWAIT\n\n# This socket class mirrors the functionality in the Curio socket\n# class. An important facet of the design is that it still relies\n# upon non-blocking I/O and eager evaluation. Curio only enters\n# the picture if operations actually block.\n\nclass Socket(object):\n '''\n Non-blocking wrapper around a socket object.\n '''\n\n def __init__(self, sock):\n self._socket = sock\n self._socket.setblocking(False)\n self._fileno = _Fd(sock.fileno())\n\n # Commonly used bound methods\n self._socket_send = sock.send\n self._socket_recv = sock.recv\n\n def __repr__(self):\n return '' % (self._socket)\n\n def __getattr__(self, name):\n return getattr(self._socket, name)\n\n def fileno(self):\n return int(self._fileno)\n\n def settimeout(self, seconds):\n if seconds:\n raise RuntimeError('Use timeout_after() to set a timeout')\n\n def gettimeout(self):\n return None\n\n def dup(self):\n return type(self)(self._socket.dup())\n\n def makefile(self, mode, buffering=0, *, encoding=None, errors=None, newline=None):\n if 'b' not in mode:\n raise RuntimeError('File can only be created in binary mode')\n f = self._socket.makefile(mode, buffering=buffering)\n return FileStream(f)\n\n def as_stream(self):\n return SocketStream(self._socket)\n\n def recv(self, maxsize, flags=0):\n while True:\n try:\n return self._socket_recv(maxsize, flags)\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n\n def recv_into(self, buffer, nbytes=0, flags=0):\n while True:\n try:\n return self._socket.recv_into(buffer, nbytes, flags)\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n\n def send(self, data, flags=0):\n while True:\n try:\n return self._socket_send(data, flags)\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n except WantRead: \n AWAIT(_read_wait, self._fileno)\n\n def sendall(self, data, flags=0):\n with memoryview(data).cast('B') as buffer:\n total_sent = 0\n try:\n while buffer:\n try:\n nsent = self._socket_send(buffer, flags)\n total_sent += nsent\n buffer = buffer[nsent:]\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n except WantRead: \n AWAIT(_read_wait, self._fileno)\n except curio.errors.CancelledError as e:\n e.bytes_sent = total_sent\n raise\n\n def accept(self):\n while True:\n try:\n client, addr = self._socket.accept()\n client = Socket(client)\n client.__class__ = type(self)\n return client, addr\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n\n def connect_ex(self, address):\n try:\n self.connect(address)\n return 0\n except OSError as e:\n return e.errno\n\n def connect(self, address):\n try:\n result = self._socket.connect(address)\n if getattr(self, 'do_handshake_on_connect', False):\n self.do_handshake()\n return result\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n err = self._socket.getsockopt(SOL_SOCKET, SO_ERROR)\n if err != 0:\n raise OSError(err, 'Connect call failed %s' % (address,))\n if getattr(self, 'do_handshake_on_connect', False):\n self.do_handshake()\n\n def recvfrom(self, buffersize, flags=0):\n while True:\n try:\n return self._socket.recvfrom(buffersize, flags)\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n\n def recvfrom_into(self, buffer, bytes=0, flags=0):\n while True:\n try:\n return self._socket.recvfrom_into(buffer, bytes, flags)\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n\n def sendto(self, bytes, flags_or_address, address=None):\n if address:\n flags = flags_or_address\n else:\n address = flags_or_address\n flags = 0\n while True:\n try:\n return self._socket.sendto(bytes, flags, address)\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n\n def recvmsg(self, bufsize, ancbufsize=0, flags=0):\n while True:\n try:\n return self._socket.recvmsg(bufsize, ancbufsize, flags)\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n\n def recvmsg_into(self, buffers, ancbufsize=0, flags=0):\n while True:\n try:\n return self._socket.recvmsg_into(buffers, ancbufsize, flags)\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n\n def sendmsg(self, buffers, ancdata=(), flags=0, address=None):\n while True:\n try:\n return self._socket.sendmsg(buffers, ancdata, flags, address)\n except WantRead:\n AWAIT(_write_wait, self._fileno)\n\n # Special functions for SSL\n def do_handshake(self):\n while True:\n try:\n return self._socket.do_handshake()\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n\n def close(self):\n if self._socket:\n self._socket.close()\n self._socket = None\n self._fileno = -1\n\n def shutdown(self, how):\n if self._socket:\n self._socket.shutdown(how)\n\n def __enter__(self):\n self._socket.__enter__()\n return self\n\n def __exit__(self, *args):\n self._socket.__exit__(*args)\n\n\n\nMAX_READ = 65536\n\nclass StreamBase(object):\n '''\n Base class for file-like objects.\n '''\n\n def __init__(self, fileobj):\n self._file = fileobj\n self._fileno = _Fd(fileobj.fileno())\n self._buffer = bytearray()\n\n def __repr__(self):\n return '' % (type(self).__name__, self._file)\n\n def fileno(self):\n return int(self._fileno)\n\n # ---- Methods that must be implemented in child classes\n def _read(self, maxbytes=-1): \n raise NotImplementedError()\n\n def write(self, data): \n raise NotImplementedError()\n\n # ---- General I/O methods for streams\n def read(self, maxbytes=-1):\n buf = self._buffer\n if buf:\n if maxbytes < 0 or len(buf) <= maxbytes:\n data = bytes(buf)\n buf.clear()\n else:\n data = bytes(buf[:maxbytes])\n del buf[:maxbytes]\n else:\n data = self._read(maxbytes)\n return data\n\n def readall(self):\n chunks = []\n maxread = 65536\n if self._buffer:\n chunks.append(bytes(self._buffer))\n self._buffer.clear()\n while True:\n try:\n chunk = self.read(maxread)\n except curio.errors.CancelledError as e:\n e.bytes_read = b''.join(chunks)\n raise\n if not chunk:\n return b''.join(chunks)\n chunks.append(chunk)\n if len(chunk) == maxread:\n maxread *= 2\n\n def read_exactly(self, nbytes):\n chunks = []\n while nbytes > 0:\n try:\n chunk = self.read(nbytes)\n except curio.errors.CancelledError as e:\n e.bytes_read = b''.join(chunks)\n raise\n if not chunk:\n e = EOFError('Unexpected end of data')\n e.bytes_read = b''.join(chunks)\n raise e\n chunks.append(chunk)\n nbytes -= len(chunk)\n return b''.join(chunks)\n\n def readinto(self, memory):\n with memoryview(memory).cast('B') as view:\n remaining = len(view)\n total_read = 0\n\n # It's possible that there is data already buffered on this stream. \n # If so, we have to copy into the memory buffer first.\n buffered = len(self._buffer)\n tocopy = remaining if (remaining < buffered) else buffered\n if tocopy:\n view[:tocopy] = self._buffer[:tocopy]\n del self._buffer[:tocopy]\n remaining -= tocopy\n total_read += tocopy\n\n # To emulate behavior of synchronous readinto(), we read all available\n # bytes up to the buffer size.\n while remaining > 0:\n try:\n nrecv = self._readinto_impl(view[total_read:total_read+remaining])\n\n # On proper file objects, None might be returned to indicate blocking\n if nrecv is None:\n AWAIT(_read_wait, self._fileno)\n elif nrecv == 0:\n break\n else:\n total_read += nrecv\n remaining -= nrecv\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n return total_read\n \n def readline(self, maxlen=None):\n while True:\n nl_index = self._buffer.find(b'\\n')\n if nl_index >= 0:\n resp = bytes(self._buffer[:nl_index + 1])\n del self._buffer[:nl_index + 1]\n return resp\n data = self._read(MAX_READ)\n if data == b'':\n resp = bytes(self._buffer)\n self._buffer.clear()\n return resp\n self._buffer.extend(data)\n\n def readlines(self):\n lines = []\n try:\n for line in self:\n lines.append(line)\n return lines\n except curio.errors.CancelledError as e:\n e.lines_read = lines\n raise\n\n def writelines(self, lines):\n nwritten = 0\n for line in lines:\n try:\n self.write(line)\n nwritten += len(line)\n except curio.errors.CancelledError as e:\n e.bytes_written += nwritten\n raise\n\n def flush(self):\n pass\n\n def close(self):\n self.flush()\n if self._file:\n self._file.close()\n self._file = None\n self._fileno = -1\n\n def __iter__(self):\n return self\n\n def __next__(self):\n line = self.readline()\n if line:\n return line\n else:\n raise StopIteration\n\n def __enter__(self):\n return self\n\n def __exit__(self, *args):\n return self.close()\n\n\nclass FileStream(StreamBase):\n '''\n Wrapper around a file-like object. File is put into non-blocking mode.\n The underlying file must be in binary mode.\n '''\n\n def __init__(self, fileobj):\n assert not isinstance(fileobj, io.TextIOBase), 'Only binary mode files allowed'\n super().__init__(fileobj)\n os.set_blocking(int(self._fileno), False)\n\n # Common bound methods\n self._file_read = fileobj.read\n self._readinto_impl = getattr(fileobj, 'readinto', None)\n self._file_write = fileobj.write\n\n def _read(self, maxbytes=-1):\n while True:\n # In non-blocking mode, a file-like object might return None if no data is\n # available. Alternatively, we'll catch the usual blocking exceptions just to be safe\n try:\n data = self._file_read(maxbytes)\n if data is None:\n AWAIT(_read_wait, self._fileno)\n else:\n return data\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n except WantWrite: \n AWAIT(_write_wait, self._fileno)\n\n def write(self, data):\n nwritten = 0\n view = memoryview(data).cast('B')\n try:\n while view:\n try:\n nbytes = self._file_write(view)\n if nbytes is None:\n raise BlockingIOError()\n nwritten += nbytes\n view = view[nbytes:]\n except WantWrite as e:\n if hasattr(e, 'characters_written'):\n nwritten += e.characters_written\n view = view[e.characters_written:]\n AWAIT(_write_wait, self._fileno)\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n return nwritten\n\n except curio.errors.CancelledError as e:\n e.bytes_written = nwritten\n raise\n\n def flush(self):\n if not self._file:\n return\n while True:\n try:\n return self._file.flush()\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n\nclass SocketStream(StreamBase):\n '''\n Stream wrapper for a socket.\n '''\n\n def __init__(self, sock):\n super().__init__(sock)\n sock.setblocking(False)\n\n # Common bound methods\n self._socket_recv = sock.recv\n self._readinto_impl = sock.recv_into\n self._socket_send = sock.send\n\n def _read(self, maxbytes=-1):\n while True:\n try:\n data = self._socket_recv(maxbytes if maxbytes > 0 else MAX_READ)\n return data\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n\n def write(self, data):\n nwritten = 0\n view = memoryview(data).cast('B')\n try:\n while view:\n try:\n nbytes = self._socket_send(view)\n nwritten += nbytes\n view = view[nbytes:]\n except WantWrite:\n AWAIT(_write_wait, self._fileno)\n except WantRead:\n AWAIT(_read_wait, self._fileno)\n return nwritten\n except curio.errors.CancelledError as e:\n e.bytes_written = nwritten\n raise\n" }, { "path": "thredo/magic.py", "content": "# magic.py\n\nfrom contextlib import contextmanager\nimport sys\n\nfrom . import socket\n\n__all__ = ['more_magic']\n\n@contextmanager\ndef more_magic():\n sockmod = sys.modules['socket']\n sys.modules['socket'] = socket\n try:\n yield\n finally:\n sys.modules['socket'] = sockmod\n" }, { "path": "thredo/mixin.py", "content": "# mixin.py\n\n__all__ = ['ThredoMixIn']\n\nfrom . import io\nfrom . import core\n\nclass ThredoMixIn:\n '''\n Mixin class that can be used to make standard socketserver objects to\n use thredo\n '''\n _threads = None\n\n def server_activate(self):\n super().server_activate()\n self.socket = io.Socket(self.socket)\n\n def serve_forever(self):\n while True:\n self.handle_request()\n \n def handle_request(self):\n try:\n self._handle_request_noblock()\n except core.ThreadCancelled:\n threads = self._threads\n self._threads = None\n if threads:\n for thread in threads:\n thread.cancel()\n raise\n\n def process_request_thread(self, request, client_address):\n try:\n self.finish_request(request, client_address)\n except Exception:\n self.handle_error(request, client_address)\n finally:\n self.shutdown_request(request)\n\n def process_request(self, request, client_address):\n t = core.spawn(self.process_request_thread, request, client_address)\n if self._threads is None:\n self._threads = []\n self._threads.append(t)\n\n def server_close(self):\n super().server_close()\n threads = self._threads\n self._threads = None\n if threads:\n for thread in threads:\n thread.join()\n\n\n" }, { "path": "thredo/queue.py", "content": "# queue.py\n#\n# A basic queue\n\n__all__ = [ 'Queue' ]\n\nimport curio\n\n# -- Thredo\nfrom .thr import TAWAIT as AWAIT\n\nclass Queue(object):\n def __init__(self, maxsize=0):\n self._queue = curio.Queue(maxsize)\n\n def empty(self):\n return self._queue.empty()\n\n def full(self):\n return self._queue.full()\n\n def get(self):\n return AWAIT(self._queue.get)\n\n def join(self):\n return AWAIT(self._queue.join)\n\n def put(self, item):\n return AWAIT(self._queue.put, item)\n\n def qsize(self):\n return self._queue.qsize()\n\n def task_done(self):\n return AWAIT(self._queue.task_done)\n\n def __iter__(self):\n return self\n\n def __next__(self):\n return self.get()\n" }, { "path": "thredo/requests.py", "content": "# requests.py\n#\n# Session adapter that allows requests to use thredo socket objects.\n# This is a bit of plumbing, but it's a clean interface that doesn't\n# require any monkeypatching or other low-level magic\n\n__all__ = ['get_session']\n\n# -- Thredo\nfrom . import socket\n\n# -- Requests/third party\nimport requests\nfrom requests.adapters import HTTPAdapter\nfrom requests.packages.urllib3 import PoolManager, HTTPConnectionPool\nfrom requests.packages.urllib3 import HTTPSConnectionPool\nfrom http.client import HTTPConnection, HTTPSConnection\n\nclass ThredoAdapter(HTTPAdapter):\n def init_poolmanager(self, connections, maxsize, block):\n self.poolmanager = ThredoPoolManager(num_pools=connections,\n maxsize=maxsize,\n block=block)\n\n\nclass ThredoPoolManager(PoolManager):\n def _new_pool(self, scheme, host, port):\n # Important!\n if scheme == 'http':\n return ThredoHTTPConnectionPool(host, port, **self.connection_pool_kw)\n\n if scheme == 'https':\n return ThredoHTTPSConnectionPool(host, port, **self.connection_pool_kw)\n\n return super(PoolManager, self)._new_pool(self, scheme, host, port)\n\n\nclass ThredoHTTPConnectionPool(HTTPConnectionPool):\n def _new_conn(self):\n self.num_connections += 1\n return ThredoHTTPConnection(host=self.host,\n port=self.port)\n\n\nclass ThredoHTTPSConnectionPool(HTTPSConnectionPool):\n def _new_conn(self):\n self.num_connections += 1\n return ThredoHTTPSConnection(host=self.host,\n port=self.port)\n\n\nclass ThredoHTTPConnection(HTTPConnection):\n def connect(self):\n \"\"\"Connect to the host and port specified in __init__.\"\"\"\n # Uses thredo\n self.sock = socket.create_connection((self.host, self.port),\n self.timeout, self.source_address)\n # Important!\n if self._tunnel_host:\n self._tunnel()\n\nclass ThredoHTTPSConnection(HTTPSConnection):\n def connect(self):\n \"\"\"Connect to the host and port specified in __init__.\"\"\"\n # Uses thredo\n self.sock = socket.create_connection((self.host, self.port),\n self.timeout, self.source_address)\n # Important!\n if self._tunnel_host:\n server_hostname = self._tunnel_host\n else:\n server_hostname = self.host\n \n self.sock = self._context.wrap_socket(self.sock, server_hostname=server_hostname)\n\ndef get_session():\n s = requests.Session()\n s.mount('http://', ThredoAdapter())\n s.mount('https://', ThredoAdapter())\n return s\n\n" }, { "path": "thredo/signal.py", "content": "# signal.py\n#\n# Signal related functionality\n\n__all__ = ['SignalEvent']\n\nimport curio\n\n# -- Thredo\nfrom .thr import TAWAIT as AWAIT\n\nclass SignalEvent:\n def __init__(self, *args, **kwargs):\n async def _run():\n self._sigevt = curio.SignalEvent(*args, **kwargs)\n AWAIT(_run)\n\n def __del__(self):\n async def _run():\n del self._sigevt\n AWAIT(_run)\n\n def wait(self):\n AWAIT(self._sigevt.wait)\n\n def set(self):\n AWAIT(self._sigevt.set)\n\n" }, { "path": "thredo/socket.py", "content": "# socket.py\n#\n# Standin for the standard socket library. The entire contents of stdlib socket are\n# made available here. \n\nimport socket as _socket\n\n__all__ = _socket.__all__\n\nfrom socket import *\nfrom socket import _GLOBAL_DEFAULT_TIMEOUT\n\nfrom functools import wraps\nfrom . import io\nimport sys\n \n@wraps(_socket.socket)\ndef socket(*args, **kwargs):\n return io.Socket(_socket.socket(*args, **kwargs))\n\nclass socket(io.Socket):\n def __init__(self, *args, **kwargs):\n super().__init__(_socket.socket(*args, **kwargs))\n\n# @staticmethod\n# def __new__(cls, *args, **kwargs):\n# return super().__new__(cls, _socket.socket(*args, **kwargs))\n\n@wraps(_socket.socketpair)\ndef socketpair(*args, **kwargs):\n s1, s2 = _socket.socketpair(*args, **kwargs)\n return io.Socket(s1), io.Socket(s2)\n\n@wraps(_socket.fromfd)\ndef fromfd(*args, **kwargs):\n return io.Socket(_socket.fromfd(*args, **kwargs))\n\n# Replacements for blocking functions related to domain names and DNS\n\n@wraps(_socket.create_connection)\ndef create_connection(*args, **kwargs):\n sock = _socket.create_connection(*args, **kwargs)\n return io.Socket(sock)\n\n\n" }, { "path": "thredo/sync.py", "content": "# sync.py\n#\n# The basic synchronization primitives such as locks, semaphores, and condition variables.\n\n__all__ = [ 'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Condition' ]\n\nimport curio\n\n# -- Thredo\nfrom .thr import TAWAIT as AWAIT\n\nclass Event(object):\n def __init__(self):\n self._evt = curio.Event()\n\n def is_set(self):\n return self._evt.is_set()\n\n def clear(self):\n self._evt.clear()\n\n def wait(self):\n AWAIT(self._evt.wait)\n\n def set(self):\n AWAIT(self._evt.set)\n\n# Base class for all synchronization primitives that operate as context managers.\n\nclass _LockBase(object):\n def __enter__(self):\n self._lock.__enter__()\n\n def __exit__(self, *args):\n self._lock.__exit__(*args)\n\n def acquire(self):\n AWAIT(self._lock.acquire)\n\n def release(self):\n AWAIT(self._lock.release)\n\n def locked(self):\n return self._lock.locked()\n\nclass Lock(_LockBase):\n def __init__(self):\n self._lock = curio.Lock()\n\nclass RLock(_LockBase):\n def __init__(self):\n self._lock = curio.RLock()\n\nclass Semaphore(_LockBase):\n def __init__(self):\n self._lock = curio.Semaphore()\n\n @property\n def value(self):\n return self._lock.value\n\nclass BoundedSemaphore(Semaphore):\n def __init__(self):\n self._lock = curio.BoundedSemaphore()\n\nclass Condition(_LockBase):\n def __init__(self, lock=None):\n if lock is None:\n self._lock = curio.Condition(curio.Lock())\n else:\n self._lock = curio.Condition(lock._lock)\n\n def locked(self):\n return self._lock.locked()\n\n def wait(self):\n AWAIT(self._lock.wait)\n \n def wait_for(self, predicate):\n AWAIT(self._lock.wait_for, predicate)\n\n def notify(self, n=1):\n AWAIT(self._lock.notify, n)\n\n def notify_all(self):\n AWAIT(self._lock.notify_all)\n\n" }, { "path": "thredo/thr.py", "content": "# thr.py\n#\n# Functions that allow normal threads to promote into Curio async threads.\n\nfrom concurrent.futures import Future\nfrom curio.thread import is_async_thread, _locals, AWAIT, AsyncThread\nfrom curio import spawn, UniversalQueue\n\n_request_queue = None\n\ndef TAWAIT(coro, *args, **kwargs):\n '''\n Ensure that the caller is an async thread (promoting if necessary),\n then await for a coroutine\n '''\n if not is_async_thread():\n enable_async()\n return AWAIT(coro, *args, **kwargs)\n\ndef thread_atexit(callable):\n _locals.thread_exit.atexit(callable)\n\ndef enable_async():\n '''\n Enable asynchronous operations in an existing thread. This only\n shuts down once the thread exits.\n '''\n if hasattr(_locals, 'thread'):\n return\n\n if _request_queue is None:\n raise RuntimeError('Curio: enable_threads not used')\n\n fut = Future()\n _request_queue.put(('start', threading.current_thread(), fut))\n _locals.thread = fut.result()\n _locals.thread_exit = ThreadAtExit()\n \n def shutdown(thread=_locals.thread, rq=_request_queue):\n fut = Future()\n rq.put(('stop', thread, fut))\n fut.result()\n _locals.thread_exit.atexit(shutdown)\n\nasync def thread_handler():\n '''\n Special handler function that allows Curio to respond to\n threads that want to access async functions. This handler\n must be spawned manually in code that wants to allow normal\n threads to promote to Curio async threads.\n '''\n global _request_queue\n assert _request_queue is None, \"thread_handler already running\"\n _request_queue = queue.UniversalQueue()\n\n try:\n while True:\n request, thr, fut = await _request_queue.get()\n if request == 'start':\n athread = AsyncThread(None)\n athread._task = await spawn(athread._coro_runner, daemon=True)\n athread._thread = thr\n fut.set_result(athread)\n elif request == 'stop':\n thr._request.set_result(None)\n await thr._task.join()\n fut.set_result(None)\n finally:\n _request_queue = None\n \n" } ]