Repository: sogisha/fyuneru Branch: master Commit: cff2af1e28f1 Files: 28 Total size: 84.4 KB Directory structure: gitextract_v0a9sp1m/ ├── .gitignore ├── README.md ├── config.dev.json ├── doc/ │ ├── TODO.md │ └── cryptography.md ├── fyuneru/ │ ├── __init__.py │ ├── ipc/ │ │ ├── __init__.py │ │ ├── __protocol.py │ │ ├── client.py │ │ ├── server.py │ │ ├── tools.py │ │ └── url.py │ ├── net/ │ │ ├── __init__.py │ │ ├── protocol.py │ │ └── vnet.py │ └── util/ │ ├── __init__.py │ ├── __xsalsa20.py │ ├── config.py │ ├── crypto.py │ ├── debug.py │ ├── droproot.py │ ├── pidfile.py │ └── procmgr.py ├── proxy.icmp.py ├── proxy.shadowsocks.py ├── proxy.tcp.py ├── proxy.xmpp.py └── run_as.py ================================================ FILE CONTENTS ================================================ ================================================ FILE: .gitignore ================================================ *.sw? *.pyc test.* config*.json !config.dev.json *.odg ================================================ FILE: README.md ================================================ Fyuneru v1.1 ============ **向下拉动，见简体中文版本。** **Fyuneru** lets you set up a server and a client computer within a virtual ethernet, whose data frames over the virtual network cable are proxified parallel via imagnative a lot of protocols. It also provides basical encryption, which makes analyzing the traffic more difficult. This software is licensed under GPLv2. ## Principle Fyuneru ships with 2 parts: a core, and several proxies. The **core** behaves on the server and client computer in an identical manner. It first sets up the virtual network interface using Linux's TUN device and deals with intercepting IP frames sending to and coming from it. When these frames are leaving our computer, they are encrypted to ensure the confidentiality and integrity. The core communicates with serveral proxies using UNIX Sockets in UDP Datagrams, which are also managed by Fyuneru. The proxies are choosen randomly to receive encrypted packets from the core. Then, how the proxies send these packets, varies. Currently we have officially written 2 proxies by using Shadowsocks and XMPP protocol. Shadowsocks sends UDP packets using an encrypted connection directly to a server with little protocol footprint, and XMPP sends our packets using TCP via up to 2 intermediate servers. You may configure Fyuneru to initiate several such proxies, making a lot of proxy paths. Notice that the core on both server and client behaves the same: it chooses for every IP frame a random proxy, therefore for a given connection or session, the actual IP frame of request and response travels highly likely over different routes with different protocols and even different IPs(in case of XMPP, etc). This schema should confuse an observer, e.g. a firewall. ## Installation ### System Requirements 1. Operating System: currently tested over Fedora 21 and Ubuntu 14.04; 1. Installed Python2 environment; 1. Dependencies installed, see below. ### Dependencies 1. `salsa20`, a python module providing encryption for our program. Use `sudo pip install salsa20` to get this. 2. `xmpppy`, if you want to use XMPP proxies. I have forked an library on github at , follow the instructions to get it installed. 3. `shadowsocks-libev`, you are likely to compile it on your self. I have also forked one at , follow the instructions to compile it and install. ## Usage ### Add user and group for Fyuneru You have to set up your system with an user and a group, in which Fyuneru will run. Recommended is `nobody` for both. ### Configuration Install Fyuneru on both your server and your client. Before running, create a file named `config.json` and place it in the same folder as the `run_as.py`. Both server and client needs such a file, make sure they're **identical**. An example of `config.json` is below. Read instructions below to get explanations! ``` { "version": "1.1", "core": { "server": { "ip": "10.1.0.1" }, "client": { "ip": "10.1.0.2" }, "user": { "uidname": "nobody", "gidname": "nobody" }, "key": "DEVELOPMENT ONLY" }, "proxies": { "proxy-ss-01": { "type": "shadowsocks", "server": { "bin": "/usr/local/bin/ss-server", "ip": "", "port": 31000, "forward-to": 10081 }, "client": { "bin": "/usr/local/bin/ss-tunnel", "port": 10080, "proxy": { "ip": "", "port": 31000 } } }, "proxy-xmpp-01": { "type": "xmpp", "server": { "jid": "jid1@test.com", "password": "jid1_password" }, "client": { "jid": "jid2@example.com", "password": "jid2_password" } } } } ``` #### Section `core` 1. Set `core.server.ip` and `core.client.ip` as the desired virtual IP addresses for both computers. 1. `core.key` must be random and unique for your own security. 1. `core.user.uidname` and `core.user.gidname` are UID/GID allocated to Fyuneru. After setting up necessary network devices, Fyuneru will jump to this UID/GID and give up root privileges. #### Section `proxies` To configure using a specific type of proxy, specify parameters in `proxies` section. You have to give each proxy a name. `proxy-ss-01` or `proxy-xmpp-01` are examples. They are going to appear in the log output. ##### To add a Shadowsocks proxy You have to write the section of a proxy like this: ``` "": { "type": "shadowsocks", "server": { "bin": "/usr/local/bin/ss-server", "ip": "", "port": 31000, "forward-to": 10081 }, "client": { "bin": "/usr/local/bin/ss-tunnel", "port": 10080, "proxy": { "ip": "", "port": 31000 } } } ``` 1. `type` must be `shadowsocks` to tell the program start a Shadowsocks proxy. 1. `server.bin` points to the `ss-server` binary, and `client.bin` to the `ss-tunnel` binary. 2. `server.ip` and `server.port` are IP and port, on which the server should listen in Internet. 3. `server.forward-to` is the port used by the server-side adapter of Fyuneru. Choose it as you like. Similarily is `client.port` the port used by the client-side adapter of Fyuneru. 4. `client.proxy` is optional. If you want to ask the client to connect to a port-forwarded server, not directly to the server port, use this. Otherwise we'll just connect to `server.ip`:`server.port` directly. ##### To add a XMPP proxy ``` "": { "type": "xmpp", "server": { "jid": "jid1@test.com", "password": "jid1_password" }, "client": { "jid": "jid2@example.com", "password": "jid2_password" } } ``` 1. `type` must be `xmpp` to tell the program start a XMPP proxy. 2. Both server and client requires the same format, providing a `jid` and a `password`. You have to apply for 2 XMPP accounts(they don't have to be from the same service provider, but the client account's provider should be reachable from your Internet environment). ### Run Find the `run_as.py`, run `python run_as.py s` for setting up a server, and `python run_as.py c` for setting up the client. You need root priviledge to do this. Add `--debug` after `run_as.py` will enable the debug mode. The IP frames will be dumped to the console. --- Fyuneru ======= **Scroll up for English version.** **Fyuneru**允许您将服务器和客户端计算机用虚拟的以太网连接。 在虚拟的网线上传递的数据帧实际借助可以想象的一系列协议传送。 它同时提供基本的加密，使得分析流量更加困难。 本软件在GPLv2协议下进行许可。 ## 原理 Fyuneru包含两个部分：核心，和一系列代理。 **核心**在服务器和客户端上的行为是相同的。 它首先使用Linux的TUN设备建立一个虚拟网卡，然后收发这个网卡上的IP数据帧。 这些数据帧离开我们的计算机之前会被加密，以便保证它们的机密性和完整性。 核心使用Unix Socket和各个代理之间使用UDP数据包通信。这些代理也是Fyuneru管理的。 核心随机地向代理发送数据包，之后，代理用自己的不同方式将这些数据包发到服务器。 当前我们提供2种代理：利用Shadowsocks和XMPP协议的。 Shadowsocks使用一种难以分析特征的协议向服务器直接发出UDP数据包， XMPP将数据包通过TCP连接传递，但利用了最多2个中间服务器。 您可以配置Fyuneru启动多个这样的代理进程，以便设定多个代理路径。 注意到，服务器和客户端的核心有相同的行为：对每个IP数据帧，它都随机选择一个代理。 即使是对于同样的连接会话，实际上的IP数据帧也是通过不同的路径、不同的协议甚至不同的IP传递的（例如XMPP协议）。 这种方式可以扰乱例如防火墙这样的观察者。 ## 安装 ### 系统需求 1. 操作系统：当前在Fedora 21和Ubuntu 14.04上进行了测试。 1. 需要有Python2的环境。 1. 需要安装如下所述的依赖包。 ### 依赖包 1. `salsa20`, 为我们的程序提供加密的Python模块。使用如下命令安装： `sudo pip install salsa20` 2. `xmpppy`, 如果要使用XMPP代理的话。可以从我在github上fork的地址找到： ，按照上面的指示安装。 3. `shadowsocks-libev`, 您可能需要自己编译安装。我在github上也fork了一份： ，同样按照上面的指示编译安装。 ## 用法 ### 添加Fyuneru使用的用户和组 您需要在系统中添加一个用户和一个组，Fyuneru将以它们的身份运行。 推荐用`nobody`作为用户和组。 ### 配置 您需要在服务器和客户端上都安装Fyuneru。在运行前，创建一个`config.json`文件， 将其放在和`run_as.py`同样的目录下。 服务器和客户端都需要这样一个文件，且请确认它们完全一致。 `config.json`的内容类似如下，请根据后文的指示具体配置。 ``` { "version": "1.1", "core": { "server": { "ip": "10.1.0.1" }, "client": { "ip": "10.1.0.2" }, "user": { "uidname": "nobody", "gidname": "nobody" }, "key": "DEVELOPMENT ONLY" }, "proxies": { "proxy-ss-01": { "type": "shadowsocks", "server": { "bin": "/usr/local/bin/ss-server", "ip": "", "port": 31000, "forward-to": 10081 }, "client": { "bin": "/usr/local/bin/ss-tunnel", "port": 10080, "proxy": { "ip": "", "port": 31000 } } }, "proxy-xmpp-01": { "type": "xmpp", "server": { "jid": "jid1@test.com", "password": "jid1_password" }, "client": { "jid": "jid2@example.com", "password": "jid2_password" } } } } ``` #### `core`部分 1. 将`core.server.ip`和`core.client.ip`设定为服务器和客户端的虚拟IP地址。 1. `core.key`必须是一个随机的密钥，这是为了您的安全着想。 1. `core.user.uidname`和`core.user.gidname`是分配给Fyuneru使用的UID/GID。 在设定了虚拟网卡等设备后，Fyuneru将会以这个身份运行，放弃root权限。 #### `proxies`部分 为了配置某个代理，需要在`proxies`部分中增加对应的内容。 您需要给每个代理取一个名字，例如示例中的`proxy-ss-01`或`proxy-xmpp-01`。 它们会出现在日志中。 ##### 要添加一个Shadowsocks代理 您需要将代理部分的配置写成类似如下的形式： ``` "": { "type": "shadowsocks", "server": { "bin": "/usr/local/bin/ss-server", "ip": "", "port": 31000, "forward-to": 10081 }, "client": { "bin": "/usr/local/bin/ss-tunnel", "port": 10080, "proxy": { "ip": "", "port": 31000 } } } ``` 1. `type`必须是`shadowsocks`，这样程序就会启动一个Shadowsocks代理。 1. `server.bin`指向`ss-server`这个程序的二进制文件，`client.bin`指向`ss-tunnel`的二进制文件。 2. `server.ip`和`server.port`是服务器在互联网上监听所用的IP地址和端口号。 3. `server.forward-to`是服务器端Fyuneru的代理程序所用的内部端口号。 按照您想要的数字选择。 同样地，`client.port`是客户端Fyuneru代理程序所用的端口号。 4. `client.proxy`是可选的。如果您为Shadowsocks设定了一个端口转发， 可以用这个方式让客户端连接到被转发的IP地址和端口上。 如果您不指定，将会使用由`server.ip`:`server.port`确定的目标地址。 ##### 要添加一个XMPP代理 ``` "": { "type": "xmpp", "server": { "jid": "jid1@test.com", "password": "jid1_password" }, "client": { "jid": "jid2@example.com", "password": "jid2_password" } } ``` 1. `type`必须是`xmpp`，这样程序会启动一个XMPP代理。 2. 服务器和客户端都需要同样形式的配置：`jid`和`password`。 您需要申请2个XMPP帐号（他们不一定来自同一个服务提供商， 但客户端所用的帐号所在的服务器应当能在您的互联网环境中可以直接访问）。 ### 运行 找到`run_as.py`，用命令`python run_as.py s`来启动服务器。 用`python run_as.py c`来启动客户端。您需要提供root权限。 在`run_as.py`之后添加`--debug`标志将会进入调试模式。 在此模式下，将会在控制台输出收发的IP数据帧。 ================================================ FILE: config.dev.json ================================================ { "version": "1.0", "core": { "server": { "ip": "10.1.0.1" }, "client": { "ip": "10.1.0.2" }, "user": { "uidname": "nobody", "gidname": "nobody" }, "key": "DEVELOPMENT ONLY" }, "proxies": { "proxy-ws-01": { "type": "websocket", "server": { "port": 7501 }, "client": { "url": "ws://127.0.0.1/login/" } }, "proxy-ss-01": { "type": "shadowsocks", "tunnel": { "binary":{ "server": "/usr/local/bin/ss-server", "client": "/usr/local/bin/ss-tunnel" }, "server": { "ip": "127.0.0.1", "port": 31000 }, "client": { "ip": "127.0.0.1", "port": 10080 } }, "entrance": "127.0.0.1" } } } ================================================ FILE: doc/TODO.md ================================================ 任务列表 ======== ## v1.2(未确定) [ ] 1. 在客户端上设立控制端口，以便开发web界面，查看UDP流量、延迟，管理开放情况 [ ] 2. 考虑一种控制协议，可以通过上一条所述方式控制服务器端程序 [ ] 3. 修改Readme中的语法错误 [ ] 4. 用图显示本地各端口的延时情况。 ## v1.1 [ DONE ] 1. 重构虚拟网卡为一个类，用纯Python实现，删除了对python-pytun的依赖 [ DONE ] 2. 修改代理接口的协议，也许使用Unix Socket，改进握手的协议，将接口的 性能统计（通过DataPacket的时刻）也加入。 [delete] 2.1 v1.1-using-unix-socket 分支，使用Unix Socket作为代理进程和主进程 之间的通信方式。需要一种统一的IPC模块来完成任务。 (由于UnixSocket的特性，准备放弃此特性，重写由UDP连接的IPC） [ DONE ] 2.2 使用UDP构建IPC服务器类，并将对代理程序的配置转用IPC完成 [ DONE ] 3. 为Shadowsocks代理增加在服务器重启后也能自动恢复连接的机制（Bugfix） [ DONE ] 4. 修改加密协议，将加密后的UDP数据包长度随机化 此功能目前没有在代码中启用。 [ DONE ] 5. 编写基于XMPP的代理(使用xmpppy) [ pend ] 6. 使用Python的logging模块来管理log输出。 ## 已完成的改进 1. 在IP帧封包中加入发送时刻，使得客户端和服务器都可确定某个UDP端口（或者说某个 代理机制）是否堵塞。 1. 检测是否有root权限及在启动TUN后放弃root权限 1. config.json 加入version键，判断config.json是否兼容于本版本的程序。 1. config.json 修改，直接对（服务器-客户机）端口对进行配置，并针对每个端口对传 递具体代理程序的情况。 1. 将Debug输出改成彩色。 ## 远期计划 或 对其他开发者的建议 下面列的改进是远期计划中的目标，当前并没有实际的编程实现。 1. 在服务器上使用账户管理： 1. 可以自动更换通信的对称密钥，使用类似ZRTP的机制，或者传统的公钥体系完成 认证 1. 用一个服务器为多个虚拟网卡提供服务，将服务器变成虚拟局域网的路由器。在 本条实现后，可以开发额外的项目，实现多个虚拟网卡组网，甚至基于局域网内 UDP的无服务器音视频加密聊天。 2. （放弃）使用multiprocessing模块代替Unix Socket进行IPC （当前在v1.1-multiprocessing分支中进行开发）。 ================================================ FILE: doc/cryptography.md ================================================ On the Cryptography in Fyuneru ============================== This article will discuss how cryptography is being used in our program Fyuneru. In Fyuneru, one of the most important tasks is the protection of data packets we have got and to put into the virtual network interface. Such packets are encrypted and authenticated before its been send over a proxy, and is authenticated before decrypted after received from a proxy. The encryption is independent of the proxy protocol. Encryption is based on simple symmetric keys. Therefore anyone who wants to deploy a private proxy server, should use SSH or similar relative trustful means to send its symmetric key to the server. The reason why not using an asymmetric method for negotiating a key is, that: 1) This cannot guarentee more security if the server we are using as proxy is not secure. 2) That we want to use construct such ciphertexts, that it has no more characteristics as its length. It has to appear as pseudo-random. No flag bits, no plaintexts... The encryption schema is shown as following diagram: 1. Cleartext: | BUFFER arbitary length | | DATALEN | | RANDOM PADDING | Bytes | 2 | ? | ? | 2. Encryption: | ENCRYPTED BUFFER ................................ | | IV | HMAC | Bytes | 24 | 20 | | 3. Result: | PSEUDORANDOM BUFFER OF RANDOM LENGTH ........................ | Bytes | ? | Each raw plaintext buffer being encrypted has a length of no more than 0xFFFF - 20 - 24 - 2 = 65489 bytes 0xFFFF is the maximal length of a UDP packet, which is being used in our program as intermediate packets between core and proxies. 20 for a SHA1-HMAC result and 24 for an IV, 2 for the length recording in cleartext. Random padding can be applied in the cleartext, to conceal the real length of the buffer. We use **HMAC-SHA1** to authenticate the **encrypted buffer** instead of plaintext in following reasons: 1. A good HMAC should be indistinguishable against random bytes without knowledge of key. Putting it outside the cleartext and making it to authenticate the cihpertext makes it easier(with a known HMAC key!) to find out invalid packets without decryption. 2. This is secure enough. SHA1 is attacked only in theory, and that's even not HMAC(HMAC-MD5 is still secure enough, think of that). Even an attacker has got some advances in theory, it doesn't mean he will be able to use it detecting a real-time traffic effectively(making it hard to do deep packet analysis). 3. HMAC-SHA1 is 12 bytes shorter as SHA256 or more secure hash algorithms. Unrevealing the HMAC key will not lead to the leak of encryption key, since the HMAC key is derived from encryption key. If however someone has got this key, he will be able to trick our program with seemingly legal traffic, and he cannot distinguish or get a proof, that it is us not him that has sent anything. We use **XSalsa20** to encrypt the buffer. XSalsa20 is a stream cipher, and this is good since it will produce the same bytes of output as the cleartext, block ciphers will make bytes always multiples of some 16 or 32 bytes, which makes it distinguishable against a carefully observer. ================================================ FILE: fyuneru/__init__.py ================================================ # -*- coding: utf-8 -*- __all__ = [\ "config", "crypto", "debug", "droproot", "procmgr", "protocol" ] ================================================ FILE: fyuneru/ipc/__init__.py ================================================ ================================================ FILE: fyuneru/ipc/__protocol.py ================================================ """ IPC used data packets generation and parsing ============================================ This module defines several data packet classes, and a trying function for reading a buffer. """ import pickle TYPE_DATAPACKET = 0x01 TYPE_HEARTBEAT = 0x02 TYPE_QUERY = 0x03 TYPE_INFO = 0x04 class WrongTypeOfPacketException(Exception): pass ############################################################################## # Data packet, carries a buffer. class DataPacket: buffer = '' def __init__(self, buf=None): if None == buf: return if ord(buf[0]) != TYPE_DATAPACKET: raise WrongTypeOfPacketException() self.buffer = buf[1:] def __str__(self): return chr(TYPE_DATAPACKET) + self.buffer # Heartbeat packet, carries nothing. class HeartbeatPacket: def __init__(self, buf=None): if None == buf: return if ord(buf[0]) != TYPE_HEARTBEAT: raise WrongTypeOfPacketException() def __str__(self): return \ chr(TYPE_HEARTBEAT) +\ "Across the Great Wall, we can reach every corner in the world." # Query packet, carries a question text. class QueryPacket: question = '' arguments = {} def __init__(self, buf=None): if None == buf: return if ord(buf[0]) != TYPE_QUERY: raise WrongTypeOfPacketException() obj = pickle.loads(buf[1:]) self.question = obj["question"] self.arguments = obj["arguments"] def __str__(self): obj = {"question": self.question, "arguments": self.arguments} return chr(TYPE_QUERY) + pickle.dumps(obj) # Info packet, carries an info text. class InfoPacket: def __init__(self, buf=None): if None == buf: return if ord(buf[0]) != TYPE_INFO: raise WrongTypeOfPacketException() self.__dict__ = pickle.loads(buf[1:]) def __setattr__(self, name, value): self.__dict__[name] = value def __getattr__(self, name): return getattr(self.__dict__, name) def __str__(self): return chr(TYPE_INFO) + pickle.dumps(self.__dict__) ############################################################################## def loadBufferToPacket(buf): tries = [DataPacket, HeartbeatPacket, QueryPacket, InfoPacket] success = False for packetClass in tries: try: loaded = packetClass(buf) success = True break except WrongTypeOfPacketException: continue except Exception,e: raise e if success: return loaded return None ############################################################################## __all__ = ['DataPacket', 'HeartbeatPacket', 'QueryPacket', 'InfoPacket', 'loadBufferToPacket'] if __name__ == '__main__': pin1 = InfoPacket() pin2 = InfoPacket() pin1.attr1 = '1' pin1.attr2 = '2' pin2.attr1 = '3' pin2.attr2 = '4' pin3 = InfoPacket(str(pin1)) print pin3.attr1 print pin3.attr2 pin4 = InfoPacket(str(pin2)) print pin4.attr1 print pin4.attr2 ================================================ FILE: fyuneru/ipc/client.py ================================================ # -*- coding: utf-8 -*- """ Internal Fyuneru Socket for Proxy Processes A fyuneru socket basing on UDP socket is defined. It is always a listening UDP socket on a port in local addr, which does automatic handshake with given internal magic word, and provides additionally abilities like encryption underlays, traffic statistics, etc. """ from logging import debug, info, warning, error, exception from time import time from struct import pack, unpack from socket import socket, AF_INET, SOCK_DGRAM from fyuneru.util.crypto import Authenticator from __protocol import * from .url import IPCServerURL ############################################################################## class InternalSocketClient: __sock = None __name = None __peer = (None, None) connected = False broken = False __lastbeatSent = 0 __lastbeatRecv = 0 __infoHandler = None def __init__(self, serverURL): server = IPCServerURL(serverURL) self.__sock = socket(AF_INET, SOCK_DGRAM) self.__authenticator = Authenticator(server.key) self.__peer = (server.host, server.port) self.name = server.user def __getattr__(self, name): return getattr(self.__sock, name) # ---------- heartbeat related def __registerLastBeatSent(self): self.__lastbeatSent = time() def __registerLastBeatRecv(self): self.__lastbeatRecv = time() self.connected = True self.broken = False # ---------- internal mechanism dealing with outbound/inbound data def __sendPacket(self, packet): """Send a packet class to a destination using local socket.""" s = self.__authenticator.sign(str(packet)) try: self.__sock.sendto(s, self.__peer) except Exception,e: exception(e) self.connected = False self.broken = True def __recvBuffer(self, buf, sender): """Receive a buffer, unpack into packet, and dispatch it to different handlers. Returns buffer when unpacked is a DataPacket. Otherwise None.""" # filter out traffic that's not originating from what we thought if sender != self.__peer: return None # See if is a data packet, which is special. buf = self.__authenticator.verify(buf) if not buf: return None # signature check failed packet = loadBufferToPacket(buf) if not packet: return None if isinstance(packet, DataPacket): return packet.buffer # If not, call different handlers to handle this. if isinstance(packet, HeartbeatPacket): self.__handleHeartbeatPacket(packet) return None if isinstance(packet, InfoPacket): self.__handleInfoPacket(packet) return None # ---------- inner handlers for different packets def __handleHeartbeatPacket(self, packet): # heart beat reply received, answer if self.connected == False: debug("IPC client connected.") self.__registerLastBeatRecv() def __handleInfoPacket(self, packet): if self.__infoHandler: self.__infoHandler(packet) # ---------- public functions def doQuery(self, fillerFunc): packet = QueryPacket() s = fillerFunc(packet) if s: debug("Sent a query packet.") self.__sendPacket(packet) def onInfo(self, handler): self.__infoHandler = handler def close(self): debug("IPC socket shutting down...") try: self.__sock.close() except Exception,e: error("Error closing socket: %s" % e) def heartbeat(self): tdiffSent = time() - self.__lastbeatSent tdiffRecv = time() - self.__lastbeatRecv if not self.connected or tdiffSent > 2: try: self.__registerLastBeatSent() self.__sendPacket(HeartbeatPacket()) if not self.connected: debug("IPC heartbeat sent to server.") except Exception,e: exception(e) error("Heartbeat of IPC connection at client failed.") self.connected = False self.broken = True if self.connected and tdiffRecv > 5: warning("Stale IPC connection at client detected.") self.connected = False self.broken = True def receive(self): buf, sender = self.__sock.recvfrom(65536) buf = self.__recvBuffer(buf, sender) # pre handling this buffer if not buf: return None # digested within other mechanism. exit. return buf def send(self, buf): if not self.connected: return try: packet = DataPacket() packet.buffer = buf self.__sendPacket(packet) except Exception,e: exception(e) error("Failed sending buffer to IPC server.") self.connected = False # this peer may not work self.broken = True ================================================ FILE: fyuneru/ipc/server.py ================================================ # -*- coding: utf-8 -*- """ Internal Fyuneru Socket for Proxy Processes A fyuneru socket basing on UDP socket is defined. It is always a listening UDP socket on a port in local addr, which does automatic handshake with given internal magic word, and provides additionally abilities like encryption underlays, traffic statistics, etc. """ import os from logging import debug, info, warning, error, exception from time import time from struct import pack, unpack from socket import socket, AF_INET, SOCK_DGRAM import random from ..util.crypto import Crypto, Authenticator from fyuneru.util.crypto import randrange from __protocol import * IPCPort = 64089 ############################################################################## class InternalSocketServer: __sockpath = None __sock = None local = None IPCKey = None peers = {} __answerFunctions = {} # for IPC query/info service def __init__(self, key): self.IPCKey = os.urandom(32) self.__crypto = Crypto(key) self.__authenticator = Authenticator(self.IPCKey) self.local = ("127.0.0.1", IPCPort) self.__sock = socket(AF_INET, SOCK_DGRAM) self.__sock.bind(self.local) def __getattr__(self, name): return getattr(self.__sock, name) def __registerPeer(self, addrTuple, timestamp=None): """Register a peer's activity. This implies we have heard from this peer. If timestamp is given, it will be used to update the last network reception time.""" now = time() if not self.peers.has_key(addrTuple): self.peers[addrTuple] = {\ "recv": False, "send": False, "heartbeat": now, } return if timestamp: self.peers[addrTuple]["recv"] = timestamp self.peers[addrTuple]["heartbeat"] = now def __choosePeer(self): """Choose a peer randomly. This implies we are going to send a packet to this peer, and thus the sending timing will be updated.""" possiblePeers = [i for i in self.peers if self.peers[i] != False] if len(possiblePeers) < 1: return None peer = possiblePeers[randrange(0, len(possiblePeers))] self.peers[peer]["send"] = time() return peer def __sendPacket(self, packet, to): """Send a packet class to a destination using local socket.""" s = self.__authenticator.sign(str(packet)) self.__sock.sendto(s, to) def __recvBuffer(self, buf, sender): """Receive a buffer, unpack into packet, and dispatch it to different handlers. Returns buffer when unpacked is a DataPacket. Otherwise None.""" # See if is a data packet, which is special. buf = self.__authenticator.verify(buf) if not buf: return None # signature check failed packet = loadBufferToPacket(buf) if not packet: return None if isinstance(packet, DataPacket): return packet.buffer # If not, call different handlers to handle this. if isinstance(packet, HeartbeatPacket): self.__handleHeartbeatPacket(packet, sender) return None if isinstance(packet, QueryPacket): self.__handleQueryPacket(packet, sender) return None # ---------- inner handlers for different packets def __handleHeartbeatPacket(self, packet, sender): # If this is a greeting word, register this as a new connected peer # and answer. self.__registerPeer(sender) self.__sendPacket(packet, sender) def __handleQueryPacket(self, packet, sender): question = packet.question debug("Got a query packet.") if self.__answerFunctions.has_key(question): # a new answer formular answer = InfoPacket() # call handler func to fill in the answer formular s = self.__answerFunctions[question](packet.arguments, answer) # send the answer back if s: self.__sendPacket(answer, sender) # ---------- public functions def onQuery(self, question, answerfunc): self.__answerFunctions[question] = answerfunc def close(self): # close socket debug("Internal socket shutting down...") try: self.__sock.close() except Exception,e: error("Error closing socket.") exception(e) def clean(self): # reserved for doing clean up jobs relating to the peer delays removeList = [] now = time() for each in self.peers: if not self.peers[each]: # if peer has been marked as False, because of errors, etc removeList.append(each) continue # if we have not heard from peer for some while, take it as stale if now - self.peers[each]["heartbeat"] > 5: removeList.append(each) if len(removeList) > 0: for each in removeList: # delete peers: forget them(no more tasks will be assigned) self.peers[each] = False del self.peers[each] warning(\ "Following proxies are removed due to irresponsibility: \n" + " \n".join(["%s:%d" % i for i in removeList]) ) def receive(self): buf, sender = self.__sock.recvfrom(65536) buf = self.__recvBuffer(buf, sender) # pre handling this buffer if not buf: return None # digested within other mechanism. exit. # Otherwise, this is data buffer and has to be decrypted correctly. decryption = self.__crypto.decrypt(buf) if not decryption: return None # Decrypted data must be also good formated. if len(decryption) < 8: return None header = decryption[:8] timestamp = unpack('> (32 - (b)))) & 0xFFFFFFFF for i in xrange(0, 8): # half of r(=16), r/2 rounds x[ 4] ^= R(x[ 0]+x[12], 7) x[ 8] ^= R(x[ 4]+x[ 0], 9) x[12] ^= R(x[ 8]+x[ 4],13) x[ 0] ^= R(x[12]+x[ 8],18) x[ 9] ^= R(x[ 5]+x[ 1], 7) x[13] ^= R(x[ 9]+x[ 5], 9) x[ 1] ^= R(x[13]+x[ 9],13) x[ 5] ^= R(x[ 1]+x[13],18) x[14] ^= R(x[10]+x[ 6], 7) x[ 2] ^= R(x[14]+x[10], 9) x[ 6] ^= R(x[ 2]+x[14],13) x[10] ^= R(x[ 6]+x[ 2],18) x[ 3] ^= R(x[15]+x[11], 7) x[ 7] ^= R(x[ 3]+x[15], 9) x[11] ^= R(x[ 7]+x[ 3],13) x[15] ^= R(x[11]+x[ 7],18) x[ 1] ^= R(x[ 0]+x[ 3], 7) x[ 2] ^= R(x[ 1]+x[ 0], 9) x[ 3] ^= R(x[ 2]+x[ 1],13) x[ 0] ^= R(x[ 3]+x[ 2],18) x[ 6] ^= R(x[ 5]+x[ 4], 7) x[ 7] ^= R(x[ 6]+x[ 5], 9) x[ 4] ^= R(x[ 7]+x[ 6],13) x[ 5] ^= R(x[ 4]+x[ 7],18) x[11] ^= R(x[10]+x[ 9], 7) x[ 8] ^= R(x[11]+x[10], 9) x[ 9] ^= R(x[ 8]+x[11],13) x[10] ^= R(x[ 9]+x[ 8],18) x[12] ^= R(x[15]+x[14], 7) x[13] ^= R(x[12]+x[15], 9) x[14] ^= R(x[13]+x[12],13) x[15] ^= R(x[14]+x[13],18) for i in xrange(0, 16): oubuf[i] = (inbuf[i] + x[i]) & 0xFFFFFFFF def __streamXor(self, iv, key, buf): c2u = lambda src, f:\ src[f] + (src[f+1] << 4) + (src[f+2] << 8) + (src[f+3] << 12) key = bytearray(key) iv = bytearray(iv) len0 = len(buf) buf = bytearray(buf) + bytearray('0' * 64) if len(key) != 32 or len(iv) != 28: raise Exception("Key must be 32 bytes, IV must be 28 bytes") temp = uintArray(16) b1i, b1o = uintArray(16), uintArray(16) b2i, b2o = uintArray(16), uintArray(16) salsa20Constants = (0x61707865, 0x3320646e, 0x79622d32, 0x6b206574) b1i[0], b1i[5], b1i[10], b1i[15] = salsa20Constants b1i[1], b1i[2], b1i[3], b1i[4] = \ c2u(key, 0), c2u(key, 4), c2u(key, 8), c2u(key, 12) b1i[11], b1i[12], b1i[13], b1i[14] = \ c2u(key, 16), c2u(key, 20), c2u(key, 24), c2u(key, 28) b1i[6], b1i[7], b1i[8], b1i[9] = \ c2u(iv, 0), c2u(iv, 4), c2u(iv, 8), c2u(iv, 12) self.__salsa20Core(temp, b1i, b1o) b2i[0], b2i[5], b2i[10], b2i[15] = salsa20Constants b2i[1], b2i[2], b2i[3], b2i[4] = b1o[0], b1o[5], b1o[10], b1o[15] b2i[11], b2i[12], b2i[13], b2i[14] = b1o[6], b1o[7], b1o[8], b1o[9] b2i[6], b2i[7] = c2u(iv, 16), c2u(iv, 20) b2i[8] = c2u(iv, 24) i = 0 output = bytearray('0' * (len0 + 64)) b2i[9] = 0 while i < len0: self.__salsa20Core(temp, b2i, b2o) for j in xrange(0, 16): output[i+0] = buf[i+0] ^ ((b2o[j] & 0x000000FF)) output[i+1] = buf[i+1] ^ ((b2o[j] & 0x0000FF00) >> 8) output[i+2] = buf[i+2] ^ ((b2o[j] & 0x00FF0000) >> 16) output[i+3] = buf[i+3] ^ ((b2o[j] & 0xFF000000) >> 24) i += 4 b2i[9] += 1 return str(output[:len0]) def encrypt(self, iv, key, buf): return self.__streamXor(iv, key, buf) def decrypt(self, iv, key, buf): return self.__streamXor(iv, key, buf) if __name__ == "__main__": from salsa20 import XSalsa20_xor key = bytearray([0] * 32) key[0] = 0x80 iv = bytearray([0] * 28) buf = bytearray([0] * 50) c = XSalsa20() print c.encrypt(iv, key, buf).encode('hex') print XSalsa20_xor(str(buf), str(iv)[:24], str(key)).encode('hex') exit() key = '0' * 32 iv = '1' * 28 buf = 't' * 1684 c = XSalsa20() repeat = 1000 import time a = time.time() for i in xrange(0, repeat): ourres = c.encrypt(iv, key, buf) b = time.time() print len(buf) * repeat * 1.0 / (b - a) ================================================ FILE: fyuneru/util/config.py ================================================ # -*- coding: utf-8 -*- """ Config reader and manager for the core program Class Configuration provides the functionalities for loading the `config.json` and getting them parsed, and for generating the commands necessary for initializing the proxy subprocesses(more details defined in module `proxyconf`). """ import os from distutils.version import StrictVersion from json import loads from ..ipc.url import IPCServerURL VERSION_REQUIREMENT = "1.1" # required version of `config.json` proxies = { "shadowsocks": "proxy.shadowsocks.py", "xmpp": "proxy.xmpp.py", } ############################################################################## class CoreConfiguration: pass class ProxyConfiguration: pass class ConfigFileException(Exception): pass ############################################################################## class Configuration: def __checkKeyExistence(self, parentDict, *keys): for each in keys: if not parentDict.has_key(each): return False return True def __loadCore(self, core): # parse json.core and save to class attributes # get key for cryptography self.key = str(core["key"]) if type(self.key) != str: raise ConfigFileException("core.key is invalid.") # get IP for server and client if not core["server"].has_key("ip"): raise ConfigFileException("core.server.ip must be specified.") if not core["client"].has_key("ip"): raise ConfigFileException("core.client.ip must be specified.") self.serverIP = core["server"]["ip"] self.clientIP = core["client"]["ip"] # get UID/GID names if not core["user"].has_key('uidname'): raise ConfigFileException("core.user.uidname must be specified.") if not core["user"].has_key('gidname'): raise ConfigFileException("core.user.gidname must be specified.") self.user = (core["user"]["uidname"], core["user"]["gidname"]) def __loadProxyAllocations(self, proxies): # get and validate proxy and ports allocations self.__proxies = {} for proxyName in proxies: proxyConfig = proxies[proxyName] self.__proxies[proxyName] = proxyConfig def listProxies(self): return self.__proxies.keys() def getProxyConfig(self, name): if not self.__proxies.has_key(name): raise ConfigFileException("No such proxy defined.") proxyConfig = self.__proxies[name] return proxyConfig def getProxyInitParameters(self, proxyName, IPCServer, debug=False): proxyConfig = self.__proxies[proxyName] proxyType = proxyConfig["type"] cmd = ["python", proxies[proxyType]] if debug: cmd += ["--debug"] url = IPCServerURL() url.host = IPCServer.local[0] url.port = IPCServer.local[1] url.user = proxyName url.key = IPCServer.IPCKey cmd += [str(url)] return cmd def getCoreInitParameters(self, mode): ret = CoreConfiguration() ret.uid = self.user[0] ret.gid = self.user[1] ret.key = self.key if mode == 's': ret.localIP = self.serverIP ret.remoteIP = self.clientIP elif mode == 'c': ret.localIP = self.clientIP ret.remoteIP = self.serverIP else: raise Exception("Invalid run mode, either `c` or `s`.") return ret def __init__(self, config): # try load the configuration file string, and parse into JSON. try: json = loads(config) except Exception,e: raise ConfigFileException("config.json parse error.") # read config file version declaration jsonVersion = "0.0.0" if json.has_key("version"): jsonVersion = json["version"] if StrictVersion(jsonVersion) < StrictVersion(VERSION_REQUIREMENT): raise ConfigFileException("config.json version too old.") # check existence of 'core' and 'proxies' entry if not self.__checkKeyExistence(json, 'core', 'proxies'): raise ConfigFileException("config.json incomplete.") # check entries of core jsonCore = json["core"] if not self.__checkKeyExistence(\ jsonCore, 'server', 'client', 'user', 'key', ): raise ConfigFileException("config.json incomplete.") self.__loadCore(jsonCore) # check for proxy allocations jsonProxies = json["proxies"] self.__loadProxyAllocations(jsonProxies) ############################################################################## if __name__ == "__main__": j = open('../config.json', 'r').read() #print j config = Configuration(j) lst = config.listProxies() for n in lst: proxyConfig = config.getProxyConfig(n) print proxyConfig.getInitCommand('s') print proxyConfig.getInitCommand('c') ================================================ FILE: fyuneru/util/crypto.py ================================================ """ Cryptographical Services ======================== ## Authenticator in IPC communication In IPC communications, data are not encrypted, but it has to be authenticated since any process can practically send data to a UDP port our process listening on. We currently use HMAC-MD5, for local usage this should be enough and performance comes first. ## Encryption and Decryption for Internet Traffic ### Schema 1. Cleartext: | BUFFER arbitary length | | DATALEN | | RANDOM PADDING | Bytes | 2 | ? | ? | 2. Encryption: | ENCRYPTED BUFFER ................................ | | IV | HMAC | Bytes | 24 | 20 | | 3. Result: | PSEUDORANDOM BUFFER OF RANDOM LENGTH ........................ | Bytes | ? | ### Remarks 1. We use HMAC-SHA1 to authenticate the encrypted buffer in following reasons: 1.1 A good HMAC should be indistinguishable against random bytes without knowledge of key. Putting it outside the cleartext and making it to authenticate the cihpertext makes it easier(with a known HMAC key!) to find out invalid packets without decryption. 1.2 This is secure enough. SHA1 is attacked only in theory, and that's even not HMAC(HMAC-MD5 is still secure enough, think of that). Even an attacker has got some advances in theory, it doesn't mean he will be able to use it detecting a real-time traffic effectively(making it hard to do deep packet analysis). 1.3 Unrevealing the HMAC key will not lead to the leak of encryption key, since the HMAC key is derived from encryption key. If however someone has got this key, he will be able to trick our program with seemingly legal traffic, and he cannot distinguish or get a proof, that it is us not him that has sent anything. 1.4 HMAC-SHA1 is 12 bytes shorter as SHA256. """ import hashlib import hmac import math from os import urandom from struct import pack, unpack from salsa20 import XSalsa20_xor _HASHALGO = hashlib.sha1 _HASHLEN = len(_HASHALGO('').digest()) _RESULT_SIZE = 0xFFFF - 24 - 2 - _HASHLEN class CryptoException(Exception): pass def randint(a, b): i = unpack(' bufferLength: return randSize - bufferLength return 0 else: maxSize = int(min(_RESULT_SIZE - bufferLength, bufferLength * 2.0)) return randint(0, maxSize) def KDF1(secret, length): """ISO-18033-2 defined simple KDF function. Notice we assume secret is already long and random enough.""" global _HASHALGO, _HASHLEN d = int(math.ceil(length * 1.0 / _HASHLEN)) T = "" for i in xrange(0, d): C = pack(" _RESULT_SIZE: raise CryptoException('Buffer too large to be encrypted.') buflen = len(buf) lenstr = pack('> 4) & 0xF ihl = version_ihl & 0xF iph_length = ihl * 4 ttl = iph[5] protocol = iph[6] s_addr = socket.inet_ntoa(iph[8]) d_addr = socket.inet_ntoa(iph[9]) payload = buf[iph_length:] ip_meta = { "version": version, "length": str(ihl), "TTL": ttl, "protocol": protocol, "src": str(s_addr), "dst": str(d_addr), } return ip_meta, payload def showPacket(buf): width = 16 lines = [] origbuf = buf while buf != '': lines.append(buf[:width]) buf = buf[width:] ret = '' start = 0 for line in lines: hexstr = '' asciistr = '' for c in line: ordc = ord(c) hexstr += '%02x ' % ordc if ordc <= 0x7E and ordc >= 0x20: asciistr += c else: asciistr += '.' start += len(line) hexstr = hexstr.ljust(3 * width) ret += "%08x: %s: %s\n" % (start, hexstr, asciistr) ipReport = '' try: ipMeta, ipPayload = _decodeIPFrame(origbuf) ipReport = "Ver %s :: TTL %d :: Protocol %s :: Src %s :: Dst %s" % (\ ipMeta["version"], ipMeta["TTL"], ipMeta["protocol"], ipMeta["src"], ipMeta["dst"] ) ret = ipReport + "\n" + ret except Exception,e: print e pass return ret.strip() ############################################################################## if __name__ == '__main__': import os print showPacket(os.urandom(200)) ================================================ FILE: fyuneru/util/droproot.py ================================================ """ Function to drop root privileges This has to be used in 2 places: * `run_as.py`, whose root privilege will be passed to `tunnel.py`. Root is no more needed after `tunnel.py` is started, and will be dropped. * `tunnel.py`, which will need root to setup TUN device. After that, root will be dropped. """ import grp import os import pwd def dropRoot(uid_name='nobody', gid_name='nobody'): if os.getuid() != 0: # We're not root so, like, whatever dude return # Get the uid/gid from the name running_uid = pwd.getpwnam(uid_name).pw_uid running_gid = grp.getgrnam(gid_name).gr_gid # Remove group privileges os.setgroups([]) # Try setting the new uid/gid os.setgid(running_gid) os.setuid(running_uid) # Ensure a very conservative umask old_umask = os.umask(077) ================================================ FILE: fyuneru/util/pidfile.py ================================================ # -*- coding: utf-8 -*- import os import time class ProcessRunningException(Exception): pass class PidfileNonExistentException(Exception): pass class PidfileWatcher: """Watches if a pidfile exists. If exist at time of initialization of this instance, may also be used to check if the pidfile is changed or deleted. This is useful for subprocesses who may want to exit after main process is terminated. """ __lastcheck = 0 pid = None pidfile = None def __init__(self, path): self.pidfile = path self.__lastcheck = time.time() try: pidfile = open(self.pidfile, 'r') self.pid = pidfile.read() pidfile.close() except: raise PidfileNonExistentException() def check(self): try: pidfile = open(self.pidfile, 'r') pid = pidfile.read() pidfile.close() except: return False return pid == self.pid class PidfileCreator: def __init__(self, path, log=sys.stdout.write, warn=sys.stderr.write): self.pidfile = path self.log = log self.warn = warn def __enter__(self): try: self.pidfd = os.open(self.pidfile, os.O_CREAT|os.O_WRONLY|os.O_EXCL) self.log('locked pidfile %s' % self.pidfile) except OSError as e: if e.errno == errno.EEXIST: pid = self._check() if pid: self.pidfd = None raise ProcessRunningException('process already running in %s as pid %s' % (self.pidfile, pid)) else: os.remove(self.pidfile) self.warn('removed staled lockfile %s' % (self.pidfile)) self.pidfd = os.open(self.pidfile, os.O_CREAT|os.O_WRONLY|os.O_EXCL) else: raise os.write(self.pidfd, str(os.getpid())) os.close(self.pidfd) return self def __exit__(self, t, e, tb): # return false to raise, true to pass if t is None: # normal condition, no exception self._remove() return True elif t is PidfileProcessRunningException: # do not remove the other process lockfile return False else: # other exception if self.pidfd: # this was our lockfile, removing self._remove() return False def _remove(self): self.log('removed pidfile %s' % self.pidfile) os.remove(self.pidfile) def _check(self): """check if a process is still running the process id is expected to be in pidfile, which should exist. if it is still running, returns the pid, if not, return False.""" with open(self.pidfile, 'r') as f: try: pidstr = f.read() pid = int(pidstr) except ValueError: # not an integer self.log("not an integer: %s" % pidstr) return False try: os.kill(pid, 0) except OSError: self.log("can't deliver signal to %s" % pid) return False else: return pid ================================================ FILE: fyuneru/util/procmgr.py ================================================ # -*- coding: utf-8 -*- import signal import subprocess import time from logging import debug, info, warning, error class ProcessManagerException(Exception): pass class ProcessManager: __processes = {} __commands = {} def __init__(self): signal.signal(signal.SIGTERM, self.killall) def __pollAll(self): for each in self.__processes: try: self.processes[each].poll() except: pass def __startProcess(self, name, command): info("ProcessManager starting [%s]: %s" % (name, " ".join(command))) proc = subprocess.Popen(command) self.__processes[name] = proc self.__commands[name] = command def new(self, name, command): if self.__processes.has_key(name): raise ProcessManagerException(\ "Child process already registered with name [%s]" % name) self.__startProcess(name, command) def killall(self, tolerance=1.0): for each in self.__processes: self.kill(each, tolerance) def kill(self, name, tolerance=1.0): if not self.__processes.has_key(name): return True sigtermSuccess = False if tolerance: try: info("ProcessManager stopping [%s] gracefully." % name) self.__processes[name].terminate() time.sleep(1.0) self.__processes[name].poll() sigtermSuccess = (self.__processes[name].returncode != None) except: pass if sigtermSuccess: info("ProcessManager confirmed stop of [%s]." % name) return True warning("ProcessManager couldn't stop [%s] with tolerance." % name) try: self.__processes[name].kill() info("ProcessManager killed [%s]." % name) return True except Exception,e: error(("ProcessManager cannot stop [%s]. " % name) + \ "It may has been already killed. Giving up.") return False def restart(self, name, tolerance=1.0): if not self.__commands.has_key(name): return False if not self.kill(name, tolerance): return False command = self.__commands[name] self.__startProcess(name, command) def wait(self, name): self.__processes[name].wait() class ParentProcessWatcher: __last = 0 def __init__(self, pid, terminator): self.__pid = pid self.__func = terminator self.__last = time.time() def watch(self): now = time.time() if now - self.__last < 10: return proclist = subprocess.check_output(['ps', '-e']).split('\n') strpid = str(self.__pid) found = False for each in proclist: if each.strip().startswith(strpid): found = True return info("Watching: %d" % self.__last) if found: self.__last = now else: warning("Parent pid [%d] not present anymore." % self.__pid) self.__func(None, None) ================================================ FILE: proxy.icmp.py ================================================ """ ICMP Proxy Process ================== This is a very simple proxy that puts our encrypted payloads into ICMP packets. Since encrypted payloads will be recognized using cryptographical means, we won't do anything on the payload. ICMP Proxy is currently only one-way: client->server is okay, reverse is not supported. You have to use other means of proxies to get replied. """ # TODO: modify fyuneru.intsck to make server not try to use this tunnel as # answer. import argparse import signal from select import select from fyuneru.ipc.client import InternalSocketClient from fyuneru.droproot import dropRoot def log(x): print "proxy-icmp-client: %s" % x # ----------- parse arguments parser = argparse.ArgumentParser() # drop privilege to ... parser.add_argument("--uidname", metavar="UID_NAME", type=str, required=True) parser.add_argument("--gidname", metavar="GID_NAME", type=str, required=True) parser.add_argument("--client-send-to", type=str, required=False) args = parser.parse_args() ############################################################################## if None == args.client_send_to: # run as server mode sck = socket.socket(socket.AF_INET,socket.SOCK_RAW,socket.IPPROTO_ICMP) sck.setsockopt(socket.SOL_IP, socket.IP_HDRINCL, 1) dropRoot(args.uidname, args.gidname) else: # run as client mode pass local = InternalSocketClient(args.socket) ############################################################################## def doExit(signum, frame): global local, proxy try: local.close() except: pass try: proxy.xmpp.disconnect() except: pass print "exit now." exit() signal.signal(signal.SIGTERM, doExit) ############################################################################## sockets = { proxy.xmpp.Connection._sock: 'proxy', local: 'local', } while True: try: local.heartbeat() r, w, _ = select(sockets.keys(), [], [], 1) for each in r: if sockets[each] == 'proxy': proxy.xmpp.Process(1) for b in proxy.recvQueue: log("Received %d bytes, sending to core." % len(b)) local.send(b) proxy.recvQueue = [] if sockets[each] == 'local': recv = local.receive() if not recv: continue log("Received %d bytes, sending to tunnel." % len(recv)) proxy.send(recv) except KeyboardInterrupt: doExit(None,None) ================================================ FILE: proxy.shadowsocks.py ================================================ # -*- coding: utf-8 -*- import argparse import os import sys from select import select import socket import signal import time import logging from logging import info, debug, warning, error import hashlib import hmac from fyuneru.ipc.client import InternalSocketClient from fyuneru.util.droproot import dropRoot from fyuneru.util.procmgr import ProcessManager from fyuneru.util.debug import configLoggingModule from fyuneru.ipc.url import IPCServerURL from fyuneru.ipc.tools import InitConfigWaiter ENCRYPTION_METHOD = 'aes-256-cfb' ############################################################################## # ----------- parse arguments parser = argparse.ArgumentParser() parser.add_argument("--debug", action="store_true", default=False) parser.add_argument("IPC_SERVER_URL", type=str) args = parser.parse_args() ############################################################################## configLoggingModule(args.debug) ############################################################################## # use command line to initialize IPC client ipc = InternalSocketClient(args.IPC_SERVER_URL) queried = InitConfigWaiter(ipc).wait() if not queried: error("Configuration timed out. Exit.") ipc.close() sys.exit(1) ############################################################################## debug("Drop privilege to %s:%s" % queried["user"]) dropRoot(*queried["user"]) ############################################################################## # start shadowsocks process procmgr = ProcessManager() sharedsecret= hmac.HMAC( str(ipc.name + '-shadowsocks'), queried["key"], hashlib.sha256 ).digest().encode('base64').strip() proxyConfig = queried["config"] forwardToPort = proxyConfig["server"]["forward-to"] # exit port at server if 'c' == queried["mode"]: # CLIENT mode if proxyConfig["client"].has_key("proxy"): connectIP = proxyConfig["client"]["proxy"]["ip"] connectPort = proxyConfig["client"]["proxy"]["port"] else: connectIP = proxyConfig["server"]["ip"] connectPort = proxyConfig["server"]["port"] sscmd = [ proxyConfig["client"]["bin"], # shadowsocks-libev '-U', # UDP relay only '-L', "127.0.0.1:%d" % forwardToPort, # destinating UDP addr '-k', sharedsecret, # key '-s', connectIP, # server host '-p', str(connectPort), # server port '-b', "127.0.0.1", # local addr '-l', str(proxyConfig["client"]["port"]), # local port(entrance) '-m', ENCRYPTION_METHOD, # encryption method ] elif 's' == queried['mode']: # SERVER mode sscmd = [ proxyConfig["server"]["bin"], # shadowsocks-libev '-U', # UDP relay only '-k', sharedsecret, # key '-s', proxyConfig["server"]["ip"], # server host '-p', str(proxyConfig["server"]["port"]), # server port '-m', ENCRYPTION_METHOD, # encryption method ] else: sys.exit(127) procmgr.new('shadowsocks', sscmd) ############################################################################## proxySocket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) if 's' == queried["mode"]: proxySocket.bind(('127.0.0.1', forwardToPort)) proxyPeer = None # not knowing where to send data back else: # send to local tunnel entrance proxyPeer = ('127.0.0.1', proxyConfig["client"]["port"]) ############################################################################## def doExit(signum, frame): global ipc, proxySocket, procmgr try: ipc.close() except: pass try: proxySocket.close() except: pass try: procmgr.killall() except: pass info("Exit now.") exit() signal.signal(signal.SIGTERM, doExit) ############################################################################## while True: try: ipc.heartbeat() selected = select([ipc, proxySocket], [], [], 1.0) if len(selected) < 1: continue readables = selected[0] for each in readables: if each == ipc: buf = ipc.receive() if None == buf: continue if None == proxyPeer: continue debug("Received %d bytes, sending to tunnel." % len(buf)) proxySocket.sendto(buf, proxyPeer) if each == proxySocket: buf, sender = each.recvfrom(65536) proxyPeer = sender debug("Received %d bytes, sending back to core." % len(buf)) ipc.send(buf) if ipc.broken: doExit(None, None) except KeyboardInterrupt: doExit(None, None) ================================================ FILE: proxy.tcp.py ================================================ # -*- coding: utf-8 -*- """ UDP over TCP proxy for Fyuneru. """ import argparse import os from select import select import socket import signal import time import logging from logging import info, debug, warning, error from fyuneru.ipc.client import InternalSocketClient from fyuneru.util.droproot import dropRoot from fyuneru.util.debug import configLoggingModule ############################################################################## # ----------- parse arguments parser = argparse.ArgumentParser() # if enable debug mode parser.add_argument("--debug", action="store_true", default=False) # drop privilege to ... parser.add_argument("--uidname", metavar="UID_NAME", type=str, required=True) parser.add_argument("--gidname", metavar="GID_NAME", type=str, required=True) # mode for this script to run parser.add_argument(\ "--mode", type=str, choices=["server", "client"], required=True ) args = parser.parse_args() ############################################################################## configLoggingModule(args.debug) ############################################################################## dropRoot(args.uidname, args.gidname) ############################################################################## class Datagram2Stream: __buffer = "" def put(self, datagram): self.__buffer += datagram.encode('base64').strip() + '\n' def get(self, size=32768): ret = self.__buffer[:size] self.__buffer = self.__buffer[size:] return ret class Stream2Datagram: __buffer = "" def put(self, buf): self.__buffer += buf def get(self): n = self.__buffer.find('\n') if n < 0: return None self.__buffer = self.__buffer[n:].strip() try: ret = self.__buffer[:n].strip().decode('base64') return ret except: return None ############################################################################## localSocket = InternalSocketClient() proxySocket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) if 'server' == args.mode: proxySocket.bind(('127.0.0.1', args.FORWARD_TO)) proxyPeer = None # not knowing where to send data back else: proxyPeer = ('127.0.0.1', args.l) # send to local tunnel entrance ############################################################################## def doExit(signum, frame): global localSocket, proxySocket, procmgr try: localSocket.close() except: pass try: proxySocket.close() except: pass try: procmgr.killall() except: pass info("Exit now.") exit() signal.signal(signal.SIGTERM, doExit) ############################################################################## while True: try: localSocket.heartbeat() selected = select([localSocket, proxySocket], [], [], 1.0) if len(selected) < 1: continue readables = selected[0] for each in readables: if each == localSocket: buf = localSocket.receive() if None == buf: continue if None == proxyPeer: continue debug("Received %d bytes, sending to tunnel." % len(buf)) proxySocket.sendto(buf, proxyPeer) if each == proxySocket: buf, sender = each.recvfrom(65536) proxyPeer = sender debug("Received %d bytes, sending back to core." % len(buf)) localSocket.send(buf) if localSocket.broken: doExit(None, None) except KeyboardInterrupt: doExit(None, None) ================================================ FILE: proxy.xmpp.py ================================================ """ XMPP Proxy Process ================== This proxy utilizes the `xmpppy` library. You have to first install it before using this. If sudo pip install xmpppy doesn't work, you may have to download it from and install manually. """ import argparse import sys import signal from select import select import logging from logging import info, warning, debug, exception, error import xmpp from fyuneru.ipc.client import InternalSocketClient from fyuneru.util.droproot import dropRoot from fyuneru.util.debug import configLoggingModule from fyuneru.ipc.url import IPCServerURL from fyuneru.ipc.tools import InitConfigWaiter ############################################################################## # ----------- parse arguments parser = argparse.ArgumentParser() parser.add_argument("--debug", action="store_true", default=False) parser.add_argument("IPC_SERVER_URL", type=str) args = parser.parse_args() ############################################################################## configLoggingModule(args.debug) # use command line to initialize IPC client ipc = InternalSocketClient(args.IPC_SERVER_URL) queried = InitConfigWaiter(ipc).wait() if not queried: error("Configuration timed out. Exit.") ipc.close() sys.exit(1) ############################################################################## debug("Drop privilege to %s:%s" % queried["user"]) dropRoot(*queried["user"]) ############################################################################## class SocketXMPPProxyException(Exception): pass class SocketXMPPProxy: def __init__(self, jid, password, peer): self.__jid = xmpp.protocol.JID(jid) self.__peer = xmpp.protocol.JID(peer) self.__peerJIDStripped = self.__peer.getStripped() self.xmpp = xmpp.Client(\ self.__jid.getDomain(), debug=["always", "socket", "nodebuilder", "dispatcher"] ) connection = self.xmpp.connect() if not connection: raise SocketXMPPProxyException("Unable to connect.") authentication = self.xmpp.auth(\ self.__jid.getNode(), password ) if not authentication: raise SocketXMPPProxyException("Authentication error.") self.__registerHandlers() self.__sendPresence() def __registerHandlers(self): self.xmpp.RegisterHandler('message', self.message) def __sendPresence(self): presence = xmpp.protocol.Presence(priority=999) self.xmpp.send(presence) recvQueue = [] def message(self, con, event): msgtype = event.getType() fromjid = event.getFrom().getStripped() if fromjid != self.__peerJIDStripped: return if msgtype in ('chat', 'normal', None): body = event.getBody() try: self.recvQueue.append(body.decode('base64')) except: pass def send(self, buf): buf = buf.encode('base64') message = xmpp.protocol.Message(to=self.__peer, body=buf, typ='chat') self.xmpp.send(message) ############################################################################## proxyConfig = queried["config"] if 's' == queried["mode"]: localJID = proxyConfig["server"]["jid"] localPassword = proxyConfig["server"]["password"] remoteJID = proxyConfig["client"]["jid"] elif 'c' == queried["mode"]: localJID = proxyConfig["client"]["jid"] localPassword = proxyConfig["client"]["password"] remoteJID = proxyConfig["server"]["jid"] else: sys.exit(127) proxy = SocketXMPPProxy(localJID, localPassword, remoteJID) info("XMPP proxy from [%s](local) to [%s](remote)." % (localJID, remoteJID)) ############################################################################## def doExit(signum, frame): global ipc, proxy try: ipc.close() except: pass try: proxy.xmpp.disconnect() except: pass print "exit now." exit() signal.signal(signal.SIGTERM, doExit) ############################################################################## sockets = { proxy.xmpp.Connection._sock: 'proxy', ipc: 'ipc', } while True: try: if ipc.broken: doExit(None, None) ipc.heartbeat() r, w, _ = select(sockets.keys(), [], [], 1) for each in r: if sockets[each] == 'proxy': proxy.xmpp.Process(1) for b in proxy.recvQueue: debug("Received %d bytes, sending to core." % len(b)) ipc.send(b) proxy.recvQueue = [] if sockets[each] == 'ipc': recv = ipc.receive() if not recv: continue debug("Received %d bytes, sending to tunnel." % len(recv)) proxy.send(recv) except KeyboardInterrupt: doExit(None,None) except Exception,e: exception(e) ================================================ FILE: run_as.py ================================================ #!/usr/bin/env python # -*- coding: utf-8 -*- import argparse import os from select import select import signal import sys import logging from logging import info, debug, warning, error, exception, critical from fyuneru.net.vnet import VirtualNetworkInterface from fyuneru.util.config import Configuration from fyuneru.util.debug import showPacket, configLoggingModule from fyuneru.util.droproot import dropRoot from fyuneru.net.protocol import DataPacket, DataPacketException from fyuneru.ipc.server import InternalSocketServer from fyuneru.util.procmgr import ProcessManager ############################################################################## parser = argparse.ArgumentParser() parser = argparse.ArgumentParser(description=""" This is the initator of Fyeneru proxy. Before running, put a `config.json` in the same path as this file. Requires root priviledge for running this script. """) parser.add_argument(\ "--debug", action="store_true", default=False, help = "Print debug info, e.g. packet data." ) parser.add_argument(\ "mode", metavar="MODE", type=str, choices=['s', 'c'], help=""" Either 'c' or 's', respectively for client mode and server mode. """ ) args = parser.parse_args() PATH = os.path.realpath(os.path.dirname(sys.argv[0])) MODE = args.mode MTU = 1400 ############################################################################## # ---------- config log/debug functions configLoggingModule(args.debug) # ---------- load and parse configuration file config = Configuration(open(os.path.join(PATH, 'config.json'), 'r').read()) coreConfig = config.getCoreInitParameters(MODE) # ---------- initialize IPC and ProcessManager ipc = InternalSocketServer(coreConfig.key) processes = ProcessManager() # ---------- config TUN device and start up if "client" == args.mode: info("Running as client.") else: info("Running as server.") tun = VirtualNetworkInterface(coreConfig.localIP, coreConfig.remoteIP) tun.netmask = "255.255.255.0" tun.mtu = MTU tun.up() info("%s: up now." % tun.name) # ----------- prepare info for IPC clients(part of each proxy process) # register answer functions with ipc.onQuery(question, func), providing # services for IPC client to get its necessary information def ipcOnQueryInit(argv, answer): global config, MODE try: proxyName = argv["name"] proxyConfig = config.getProxyConfig(proxyName) answer.title = 'init' answer.uid = config.user[0] answer.gid = config.user[1] answer.config = proxyConfig answer.key = config.key answer.mode = MODE except Exception,e: exception(e) error("We cannot answer an init query.") return False return True ipc.onQuery('init', ipcOnQueryInit) # ---------- initialize proxy processes for proxyName in config.listProxies(): proxyCommand = config.getProxyInitParameters(proxyName, ipc, args.debug) processes.new(proxyName, proxyCommand) # ---------- drop root privileges dropRoot(coreConfig.uid, coreConfig.gid) ############################################################################## # register exit function, and start IO loop reads = [ipc, tun] # for `select` function def doExit(signum, frame): global reads, processes info("Exit now.") # first close TUN devices for each in reads: each.close() # kill processes t = 1.0 # second(s) waiting for exit try: processes.killall(t) info("Exiting. Wait %f seconds for child processes to exit." % t) except Exception,e: error("Exiting, error: %s" % e) info("Good bye.") sys.exit() signal.signal(signal.SIGTERM, doExit) signal.signal(signal.SIGINT, doExit) while True: try: # ---------- deal with I/O things readables = select(reads, [], [], 5.0)[0] for each in readables: if each == tun: # ---------- forward packets came from tun0 buf = each.read(65536) # each.read(each.mtu) # pack buf with timestamp packet = DataPacket() packet.data = buf # encrypt and sign buf ipc.send(str(packet)) debug(showPacket(buf)) if each == ipc: # ---------- receive packets from internet buf = each.receive() if buf == None: # Received buffer being digested by InternalSocket itself, # either some internal mechanism packet, or packet with # wrong destination, or packet decryption failed... continue try: packet = DataPacket(buf) except DataPacketException, e: # if failed reading the packet debug("[%d] --> %s: Bad packet - %s" % (i, tun.name, e)) continue # send buf to network interface tun.write(packet.data) debug(showPacket(packet.data)) # ---------- deal with tunnel delay timings ipc.clean() except KeyboardInterrupt: doExit(None, None)