Repository: sogisha/fyuneru
Branch: master
Commit: cff2af1e28f1
Files: 28
Total size: 84.4 KB
Directory structure:
gitextract_v0a9sp1m/
├── .gitignore
├── README.md
├── config.dev.json
├── doc/
│ ├── TODO.md
│ └── cryptography.md
├── fyuneru/
│ ├── __init__.py
│ ├── ipc/
│ │ ├── __init__.py
│ │ ├── __protocol.py
│ │ ├── client.py
│ │ ├── server.py
│ │ ├── tools.py
│ │ └── url.py
│ ├── net/
│ │ ├── __init__.py
│ │ ├── protocol.py
│ │ └── vnet.py
│ └── util/
│ ├── __init__.py
│ ├── __xsalsa20.py
│ ├── config.py
│ ├── crypto.py
│ ├── debug.py
│ ├── droproot.py
│ ├── pidfile.py
│ └── procmgr.py
├── proxy.icmp.py
├── proxy.shadowsocks.py
├── proxy.tcp.py
├── proxy.xmpp.py
└── run_as.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
*.sw?
*.pyc
test.*
config*.json
!config.dev.json
*.odg
================================================
FILE: README.md
================================================
Fyuneru v1.1
============
**向下拉动，见简体中文版本。**
**Fyuneru** lets you set up a server and a client computer within a virtual
ethernet, whose data frames over the virtual network cable are proxified
parallel via imagnative a lot of protocols. It also provides basical
encryption, which makes analyzing the traffic more difficult.
This software is licensed under GPLv2.
## Principle
Fyuneru ships with 2 parts: a core, and several proxies.
The **core** behaves on the server and client computer in an identical manner.
It first sets up the virtual network interface using Linux's TUN device and
deals with intercepting IP frames sending to and coming from it. When these
frames are leaving our computer, they are encrypted to ensure the
confidentiality and integrity.
The core communicates with serveral proxies using UNIX Sockets in UDP
Datagrams, which are also managed by Fyuneru. The proxies are choosen randomly
to receive encrypted packets from the core. Then, how the proxies send these
packets, varies.
Currently we have officially written 2 proxies by using Shadowsocks and XMPP
protocol. Shadowsocks sends UDP packets using an encrypted connection directly
to a server with little protocol footprint, and XMPP sends our packets using
TCP via up to 2 intermediate servers. You may configure Fyuneru to initiate
several such proxies, making a lot of proxy paths.
Notice that the core on both server and client behaves the same: it chooses for
every IP frame a random proxy, therefore for a given connection or session, the
actual IP frame of request and response travels highly likely over different
routes with different protocols and even different IPs(in case of XMPP, etc).
This schema should confuse an observer, e.g. a firewall.
## Installation
### System Requirements
1. Operating System: currently tested over Fedora 21 and Ubuntu 14.04;
1. Installed Python2 environment;
1. Dependencies installed, see below.
### Dependencies
1. `salsa20`, a python module providing encryption for our program. Use
`sudo pip install salsa20` to get this.
2. `xmpppy`, if you want to use XMPP proxies. I have forked an library on
github at <https://github.com/sogisha/xmpppy>, follow the instructions
to get it installed.
3. `shadowsocks-libev`, you are likely to compile it on your self. I have also
forked one at <https://github.com/sogisha/shadowsocks-libev>, follow the
instructions to compile it and install.
## Usage
### Add user and group for Fyuneru
You have to set up your system with an user and a group, in which Fyuneru
will run. Recommended is `nobody` for both.
### Configuration
Install Fyuneru on both your server and your client. Before running, create a
file named `config.json` and place it in the same folder as the `run_as.py`.
Both server and client needs such a file, make sure they're **identical**.
An example of `config.json` is below. Read instructions below to get
explanations!
```
{
"version": "1.1",
"core": {
"server": {
"ip": "10.1.0.1"
},
"client": {
"ip": "10.1.0.2"
},
"user": {
"uidname": "nobody",
"gidname": "nobody"
},
"key": "DEVELOPMENT ONLY"
},
"proxies": {
"proxy-ss-01": {
"type": "shadowsocks",
"server": {
"bin": "/usr/local/bin/ss-server",
"ip": "<SERVER-IP>",
"port": 31000,
"forward-to": 10081
},
"client": {
"bin": "/usr/local/bin/ss-tunnel",
"port": 10080,
"proxy": {
"ip": "<PROXY-IP>",
"port": 31000
}
}
},
"proxy-xmpp-01": {
"type": "xmpp",
"server": {
"jid": "jid1@test.com",
"password": "jid1_password"
},
"client": {
"jid": "jid2@example.com",
"password": "jid2_password"
}
}
}
}
```
#### Section `core`
1. Set `core.server.ip` and `core.client.ip` as the desired virtual IP
addresses for both computers.
1. `core.key` must be random and unique for your own security.
1. `core.user.uidname` and `core.user.gidname` are UID/GID allocated to
Fyuneru. After setting up necessary network devices, Fyuneru will jump
to this UID/GID and give up root privileges.
#### Section `proxies`
To configure using a specific type of proxy, specify parameters in `proxies`
section.
You have to give each proxy a name. `proxy-ss-01` or `proxy-xmpp-01` are
examples. They are going to appear in the log output.
##### To add a Shadowsocks proxy
You have to write the section of a proxy like this:
```
"<PROXY-NAME>": {
"type": "shadowsocks",
"server": {
"bin": "/usr/local/bin/ss-server",
"ip": "<SERVER-IP>",
"port": 31000,
"forward-to": 10081
},
"client": {
"bin": "/usr/local/bin/ss-tunnel",
"port": 10080,
"proxy": {
"ip": "<PROXY-IP>",
"port": 31000
}
}
}
```
1. `type` must be `shadowsocks` to tell the program start a Shadowsocks proxy.
1. `server.bin` points to the `ss-server` binary, and `client.bin` to the
`ss-tunnel` binary.
2. `server.ip` and `server.port` are IP and port, on which the server should
listen in Internet.
3. `server.forward-to` is the port used by the server-side adapter of Fyuneru.
Choose it as you like. Similarily is `client.port` the port used by the
client-side adapter of Fyuneru.
4. `client.proxy` is optional. If you want to ask the client to connect to a
port-forwarded server, not directly to the server port, use this. Otherwise
we'll just connect to `server.ip`:`server.port` directly.
##### To add a XMPP proxy
```
"<PROXY-NAME>": {
"type": "xmpp",
"server": {
"jid": "jid1@test.com",
"password": "jid1_password"
},
"client": {
"jid": "jid2@example.com",
"password": "jid2_password"
}
}
```
1. `type` must be `xmpp` to tell the program start a XMPP proxy.
2. Both server and client requires the same format, providing a `jid` and
a `password`. You have to apply for 2 XMPP accounts(they don't have to be
from the same service provider, but the client account's provider should be
reachable from your Internet environment).
### Run
Find the `run_as.py`, run `python run_as.py s` for setting up a server, and
`python run_as.py c` for setting up the client. You need root priviledge to
do this.
Add `--debug` after `run_as.py` will enable the debug mode. The IP frames will
be dumped to the console.
---
Fyuneru
=======
**Scroll up for English version.**
**Fyuneru**允许您将服务器和客户端计算机用虚拟的以太网连接。
在虚拟的网线上传递的数据帧实际借助可以想象的一系列协议传送。
它同时提供基本的加密，使得分析流量更加困难。
本软件在GPLv2协议下进行许可。
## 原理
Fyuneru包含两个部分：核心，和一系列代理。
**核心**在服务器和客户端上的行为是相同的。
它首先使用Linux的TUN设备建立一个虚拟网卡，然后收发这个网卡上的IP数据帧。
这些数据帧离开我们的计算机之前会被加密，以便保证它们的机密性和完整性。
核心使用Unix Socket和各个代理之间使用UDP数据包通信。这些代理也是Fyuneru管理的。
核心随机地向代理发送数据包，之后，代理用自己的不同方式将这些数据包发到服务器。
当前我们提供2种代理：利用Shadowsocks和XMPP协议的。
Shadowsocks使用一种难以分析特征的协议向服务器直接发出UDP数据包，
XMPP将数据包通过TCP连接传递，但利用了最多2个中间服务器。
您可以配置Fyuneru启动多个这样的代理进程，以便设定多个代理路径。
注意到，服务器和客户端的核心有相同的行为：对每个IP数据帧，它都随机选择一个代理。
即使是对于同样的连接会话，实际上的IP数据帧也是通过不同的路径、不同的协议甚至不同的IP传递的（例如XMPP协议）。
这种方式可以扰乱例如防火墙这样的观察者。
## 安装
### 系统需求
1. 操作系统：当前在Fedora 21和Ubuntu 14.04上进行了测试。
1. 需要有Python2的环境。
1. 需要安装如下所述的依赖包。
### 依赖包
1. `salsa20`, 为我们的程序提供加密的Python模块。使用如下命令安装：
`sudo pip install salsa20`
2. `xmpppy`, 如果要使用XMPP代理的话。可以从我在github上fork的地址找到：
<https://github.com/sogisha/xmpppy>，按照上面的指示安装。
3. `shadowsocks-libev`, 您可能需要自己编译安装。我在github上也fork了一份：
<https://github.com/sogisha/shadowsocks-libev>，同样按照上面的指示编译安装。
## 用法
### 添加Fyuneru使用的用户和组
您需要在系统中添加一个用户和一个组，Fyuneru将以它们的身份运行。
推荐用`nobody`作为用户和组。
### 配置
您需要在服务器和客户端上都安装Fyuneru。在运行前，创建一个`config.json`文件，
将其放在和`run_as.py`同样的目录下。
服务器和客户端都需要这样一个文件，且请确认它们完全一致。
`config.json`的内容类似如下，请根据后文的指示具体配置。
```
{
"version": "1.1",
"core": {
"server": {
"ip": "10.1.0.1"
},
"client": {
"ip": "10.1.0.2"
},
"user": {
"uidname": "nobody",
"gidname": "nobody"
},
"key": "DEVELOPMENT ONLY"
},
"proxies": {
"proxy-ss-01": {
"type": "shadowsocks",
"server": {
"bin": "/usr/local/bin/ss-server",
"ip": "<SERVER-IP>",
"port": 31000,
"forward-to": 10081
},
"client": {
"bin": "/usr/local/bin/ss-tunnel",
"port": 10080,
"proxy": {
"ip": "<PROXY-IP>",
"port": 31000
}
}
},
"proxy-xmpp-01": {
"type": "xmpp",
"server": {
"jid": "jid1@test.com",
"password": "jid1_password"
},
"client": {
"jid": "jid2@example.com",
"password": "jid2_password"
}
}
}
}
```
#### `core`部分
1. 将`core.server.ip`和`core.client.ip`设定为服务器和客户端的虚拟IP地址。
1. `core.key`必须是一个随机的密钥，这是为了您的安全着想。
1. `core.user.uidname`和`core.user.gidname`是分配给Fyuneru使用的UID/GID。
在设定了虚拟网卡等设备后，Fyuneru将会以这个身份运行，放弃root权限。
#### `proxies`部分
为了配置某个代理，需要在`proxies`部分中增加对应的内容。
您需要给每个代理取一个名字，例如示例中的`proxy-ss-01`或`proxy-xmpp-01`。
它们会出现在日志中。
##### 要添加一个Shadowsocks代理
您需要将代理部分的配置写成类似如下的形式：
```
"<PROXY-NAME>": {
"type": "shadowsocks",
"server": {
"bin": "/usr/local/bin/ss-server",
"ip": "<SERVER-IP>",
"port": 31000,
"forward-to": 10081
},
"client": {
"bin": "/usr/local/bin/ss-tunnel",
"port": 10080,
"proxy": {
"ip": "<PROXY-IP>",
"port": 31000
}
}
}
```
1. `type`必须是`shadowsocks`，这样程序就会启动一个Shadowsocks代理。
1. `server.bin`指向`ss-server`这个程序的二进制文件，`client.bin`指向`ss-tunnel`的二进制文件。
2. `server.ip`和`server.port`是服务器在互联网上监听所用的IP地址和端口号。
3. `server.forward-to`是服务器端Fyuneru的代理程序所用的内部端口号。
按照您想要的数字选择。
同样地，`client.port`是客户端Fyuneru代理程序所用的端口号。
4. `client.proxy`是可选的。如果您为Shadowsocks设定了一个端口转发，
可以用这个方式让客户端连接到被转发的IP地址和端口上。
如果您不指定，将会使用由`server.ip`:`server.port`确定的目标地址。
##### 要添加一个XMPP代理
```
"<PROXY-NAME>": {
"type": "xmpp",
"server": {
"jid": "jid1@test.com",
"password": "jid1_password"
},
"client": {
"jid": "jid2@example.com",
"password": "jid2_password"
}
}
```
1. `type`必须是`xmpp`，这样程序会启动一个XMPP代理。
2. 服务器和客户端都需要同样形式的配置：`jid`和`password`。
您需要申请2个XMPP帐号（他们不一定来自同一个服务提供商，
但客户端所用的帐号所在的服务器应当能在您的互联网环境中可以直接访问）。
### 运行
找到`run_as.py`，用命令`python run_as.py s`来启动服务器。
用`python run_as.py c`来启动客户端。您需要提供root权限。
在`run_as.py`之后添加`--debug`标志将会进入调试模式。
在此模式下，将会在控制台输出收发的IP数据帧。
================================================
FILE: config.dev.json
================================================
{
"version": "1.0",
"core": {
"server": {
"ip": "10.1.0.1"
},
"client": {
"ip": "10.1.0.2"
},
"user": {
"uidname": "nobody",
"gidname": "nobody"
},
"key": "DEVELOPMENT ONLY"
},
"proxies": {
"proxy-ws-01": {
"type": "websocket",
"server": {
"port": 7501
},
"client": {
"url": "ws://127.0.0.1/login/"
}
},
"proxy-ss-01": {
"type": "shadowsocks",
"tunnel": {
"binary":{
"server": "/usr/local/bin/ss-server",
"client": "/usr/local/bin/ss-tunnel"
},
"server": {
"ip": "127.0.0.1",
"port": 31000
},
"client": {
"ip": "127.0.0.1",
"port": 10080
}
},
"entrance": "127.0.0.1"
}
}
}
================================================
FILE: doc/TODO.md
================================================
任务列表
========
## v1.2(未确定)
[ ] 1. 在客户端上设立控制端口，以便开发web界面，查看UDP流量、延迟，管理开放情况
[ ] 2. 考虑一种控制协议，可以通过上一条所述方式控制服务器端程序
[ ] 3. 修改Readme中的语法错误
[ ] 4. 用图显示本地各端口的延时情况。
## v1.1
[ DONE ] 1. 重构虚拟网卡为一个类，用纯Python实现，删除了对python-pytun的依赖
[ DONE ] 2. 修改代理接口的协议，也许使用Unix Socket，改进握手的协议，将接口的
性能统计（通过DataPacket的时刻）也加入。
[delete] 2.1 v1.1-using-unix-socket 分支，使用Unix Socket作为代理进程和主进程
之间的通信方式。需要一种统一的IPC模块来完成任务。
(由于UnixSocket的特性，准备放弃此特性，重写由UDP连接的IPC）
[ DONE ] 2.2 使用UDP构建IPC服务器类，并将对代理程序的配置转用IPC完成
[ DONE ] 3. 为Shadowsocks代理增加在服务器重启后也能自动恢复连接的机制（Bugfix）
[ DONE ] 4. 修改加密协议，将加密后的UDP数据包长度随机化
此功能目前没有在代码中启用。
[ DONE ] 5. 编写基于XMPP的代理(使用xmpppy)
[ pend ] 6. 使用Python的logging模块来管理log输出。
## 已完成的改进
1. 在IP帧封包中加入发送时刻，使得客户端和服务器都可确定某个UDP端口（或者说某个
代理机制）是否堵塞。
1. 检测是否有root权限及在启动TUN后放弃root权限
1. config.json 加入version键，判断config.json是否兼容于本版本的程序。
1. config.json 修改，直接对（服务器-客户机）端口对进行配置，并针对每个端口对传
递具体代理程序的情况。
1. 将Debug输出改成彩色。
## 远期计划 或 对其他开发者的建议
下面列的改进是远期计划中的目标，当前并没有实际的编程实现。
1. 在服务器上使用账户管理：
1. 可以自动更换通信的对称密钥，使用类似ZRTP的机制，或者传统的公钥体系完成
认证
1. 用一个服务器为多个虚拟网卡提供服务，将服务器变成虚拟局域网的路由器。在
本条实现后，可以开发额外的项目，实现多个虚拟网卡组网，甚至基于局域网内
UDP的无服务器音视频加密聊天。
2. （放弃）使用multiprocessing模块代替Unix Socket进行IPC
（当前在v1.1-multiprocessing分支中进行开发）。
================================================
FILE: doc/cryptography.md
================================================
On the Cryptography in Fyuneru
==============================
This article will discuss how cryptography is being used in our program
Fyuneru.
In Fyuneru, one of the most important tasks is the protection of data packets
we have got and to put into the virtual network interface. Such packets are
encrypted and authenticated before its been send over a proxy, and is
authenticated before decrypted after received from a proxy. The encryption
is independent of the proxy protocol.
Encryption is based on simple symmetric keys. Therefore anyone who wants to
deploy a private proxy server, should use SSH or similar relative trustful
means to send its symmetric key to the server. The reason why not using an
asymmetric method for negotiating a key is, that:
1) This cannot guarentee more security if the server we are using as proxy
is not secure.
2) That we want to use construct such ciphertexts, that it has no more
characteristics as its length. It has to appear as pseudo-random. No
flag bits, no plaintexts...
The encryption schema is shown as following diagram:
1. Cleartext:
| BUFFER arbitary length |
| DATALEN | | RANDOM PADDING |
Bytes | 2 | ? | ? |
2. Encryption:
| ENCRYPTED BUFFER ................................ |
| IV | HMAC |
Bytes | 24 | 20 | |
3. Result:
| PSEUDORANDOM BUFFER OF RANDOM LENGTH ........................ |
Bytes | ? |
Each raw plaintext buffer being encrypted has a length of no more than
0xFFFF - 20 - 24 - 2 = 65489 bytes
0xFFFF is the maximal length of a UDP packet, which is being used in our
program as intermediate packets between core and proxies. 20 for a SHA1-HMAC
result and 24 for an IV, 2 for the length recording in cleartext.
Random padding can be applied in the cleartext, to conceal the real length
of the buffer.
We use **HMAC-SHA1** to authenticate the **encrypted buffer** instead of
plaintext in following reasons:
1. A good HMAC should be indistinguishable against random bytes without
knowledge of key. Putting it outside the cleartext and making it to
authenticate the cihpertext makes it easier(with a known HMAC key!) to
find out invalid packets without decryption.
2. This is secure enough. SHA1 is attacked only in theory, and that's even
not HMAC(HMAC-MD5 is still secure enough, think of that). Even an
attacker has got some advances in theory, it doesn't mean he will be
able to use it detecting a real-time traffic effectively(making it hard
to do deep packet analysis).
3. HMAC-SHA1 is 12 bytes shorter as SHA256 or more secure hash algorithms.
Unrevealing the HMAC key will not lead to the leak of encryption key, since the
HMAC key is derived from encryption key. If however someone has got this key,
he will be able to trick our program with seemingly legal traffic, and he
cannot distinguish or get a proof, that it is us not him that has sent
anything.
We use **XSalsa20** to encrypt the buffer. XSalsa20 is a stream cipher, and
this is good since it will produce the same bytes of output as the cleartext,
block ciphers will make bytes always multiples of some 16 or 32 bytes, which
makes it distinguishable against a carefully observer.
================================================
FILE: fyuneru/__init__.py
================================================
# -*- coding: utf-8 -*-
__all__ = [\
"config",
"crypto",
"debug",
"droproot",
"procmgr",
"protocol"
]
================================================
FILE: fyuneru/ipc/__init__.py
================================================
================================================
FILE: fyuneru/ipc/__protocol.py
================================================
"""
IPC used data packets generation and parsing
============================================
This module defines several data packet classes, and a trying function for
reading a buffer.
"""
import pickle
TYPE_DATAPACKET = 0x01
TYPE_HEARTBEAT = 0x02
TYPE_QUERY = 0x03
TYPE_INFO = 0x04
class WrongTypeOfPacketException(Exception): pass
##############################################################################
# Data packet, carries a buffer.
class DataPacket:
buffer = ''
def __init__(self, buf=None):
if None == buf: return
if ord(buf[0]) != TYPE_DATAPACKET: raise WrongTypeOfPacketException()
self.buffer = buf[1:]
def __str__(self):
return chr(TYPE_DATAPACKET) + self.buffer
# Heartbeat packet, carries nothing.
class HeartbeatPacket:
def __init__(self, buf=None):
if None == buf: return
if ord(buf[0]) != TYPE_HEARTBEAT: raise WrongTypeOfPacketException()
def __str__(self):
return \
chr(TYPE_HEARTBEAT) +\
"Across the Great Wall, we can reach every corner in the world."
# Query packet, carries a question text.
class QueryPacket:
question = ''
arguments = {}
def __init__(self, buf=None):
if None == buf: return
if ord(buf[0]) != TYPE_QUERY: raise WrongTypeOfPacketException()
obj = pickle.loads(buf[1:])
self.question = obj["question"]
self.arguments = obj["arguments"]
def __str__(self):
obj = {"question": self.question, "arguments": self.arguments}
return chr(TYPE_QUERY) + pickle.dumps(obj)
# Info packet, carries an info text.
class InfoPacket:
def __init__(self, buf=None):
if None == buf: return
if ord(buf[0]) != TYPE_INFO: raise WrongTypeOfPacketException()
self.__dict__ = pickle.loads(buf[1:])
def __setattr__(self, name, value):
self.__dict__[name] = value
def __getattr__(self, name):
return getattr(self.__dict__, name)
def __str__(self):
return chr(TYPE_INFO) + pickle.dumps(self.__dict__)
##############################################################################
def loadBufferToPacket(buf):
tries = [DataPacket, HeartbeatPacket, QueryPacket, InfoPacket]
success = False
for packetClass in tries:
try:
loaded = packetClass(buf)
success = True
break
except WrongTypeOfPacketException:
continue
except Exception,e:
raise e
if success: return loaded
return None
##############################################################################
__all__ = ['DataPacket', 'HeartbeatPacket', 'QueryPacket', 'InfoPacket', 'loadBufferToPacket']
if __name__ == '__main__':
pin1 = InfoPacket()
pin2 = InfoPacket()
pin1.attr1 = '1'
pin1.attr2 = '2'
pin2.attr1 = '3'
pin2.attr2 = '4'
pin3 = InfoPacket(str(pin1))
print pin3.attr1
print pin3.attr2
pin4 = InfoPacket(str(pin2))
print pin4.attr1
print pin4.attr2
================================================
FILE: fyuneru/ipc/client.py
================================================
# -*- coding: utf-8 -*-
"""
Internal Fyuneru Socket for Proxy Processes
A fyuneru socket basing on UDP socket is defined. It is always a listening UDP
socket on a port in local addr, which does automatic handshake with given
internal magic word, and provides additionally abilities like encryption
underlays, traffic statistics, etc.
"""
from logging import debug, info, warning, error, exception
from time import time
from struct import pack, unpack
from socket import socket, AF_INET, SOCK_DGRAM
from fyuneru.util.crypto import Authenticator
from __protocol import *
from .url import IPCServerURL
##############################################################################
class InternalSocketClient:
__sock = None
__name = None
__peer = (None, None)
connected = False
broken = False
__lastbeatSent = 0
__lastbeatRecv = 0
__infoHandler = None
def __init__(self, serverURL):
server = IPCServerURL(serverURL)
self.__sock = socket(AF_INET, SOCK_DGRAM)
self.__authenticator = Authenticator(server.key)
self.__peer = (server.host, server.port)
self.name = server.user
def __getattr__(self, name):
return getattr(self.__sock, name)
# ---------- heartbeat related
def __registerLastBeatSent(self):
self.__lastbeatSent = time()
def __registerLastBeatRecv(self):
self.__lastbeatRecv = time()
self.connected = True
self.broken = False
# ---------- internal mechanism dealing with outbound/inbound data
def __sendPacket(self, packet):
"""Send a packet class to a destination using local socket."""
s = self.__authenticator.sign(str(packet))
try:
self.__sock.sendto(s, self.__peer)
except Exception,e:
exception(e)
self.connected = False
self.broken = True
def __recvBuffer(self, buf, sender):
"""Receive a buffer, unpack into packet, and dispatch it to different
handlers. Returns buffer when unpacked is a DataPacket. Otherwise
None."""
# filter out traffic that's not originating from what we thought
if sender != self.__peer: return None
# See if is a data packet, which is special.
buf = self.__authenticator.verify(buf)
if not buf: return None # signature check failed
packet = loadBufferToPacket(buf)
if not packet: return None
if isinstance(packet, DataPacket): return packet.buffer
# If not, call different handlers to handle this.
if isinstance(packet, HeartbeatPacket):
self.__handleHeartbeatPacket(packet)
return None
if isinstance(packet, InfoPacket):
self.__handleInfoPacket(packet)
return None
# ---------- inner handlers for different packets
def __handleHeartbeatPacket(self, packet):
# heart beat reply received, answer
if self.connected == False: debug("IPC client connected.")
self.__registerLastBeatRecv()
def __handleInfoPacket(self, packet):
if self.__infoHandler: self.__infoHandler(packet)
# ---------- public functions
def doQuery(self, fillerFunc):
packet = QueryPacket()
s = fillerFunc(packet)
if s:
debug("Sent a query packet.")
self.__sendPacket(packet)
def onInfo(self, handler):
self.__infoHandler = handler
def close(self):
debug("IPC socket shutting down...")
try:
self.__sock.close()
except Exception,e:
error("Error closing socket: %s" % e)
def heartbeat(self):
tdiffSent = time() - self.__lastbeatSent
tdiffRecv = time() - self.__lastbeatRecv
if not self.connected or tdiffSent > 2:
try:
self.__registerLastBeatSent()
self.__sendPacket(HeartbeatPacket())
if not self.connected: debug("IPC heartbeat sent to server.")
except Exception,e:
exception(e)
error("Heartbeat of IPC connection at client failed.")
self.connected = False
self.broken = True
if self.connected and tdiffRecv > 5:
warning("Stale IPC connection at client detected.")
self.connected = False
self.broken = True
def receive(self):
buf, sender = self.__sock.recvfrom(65536)
buf = self.__recvBuffer(buf, sender) # pre handling this buffer
if not buf: return None # digested within other mechanism. exit.
return buf
def send(self, buf):
if not self.connected: return
try:
packet = DataPacket()
packet.buffer = buf
self.__sendPacket(packet)
except Exception,e:
exception(e)
error("Failed sending buffer to IPC server.")
self.connected = False # this peer may not work
self.broken = True
================================================
FILE: fyuneru/ipc/server.py
================================================
# -*- coding: utf-8 -*-
"""
Internal Fyuneru Socket for Proxy Processes
A fyuneru socket basing on UDP socket is defined. It is always a listening UDP
socket on a port in local addr, which does automatic handshake with given
internal magic word, and provides additionally abilities like encryption
underlays, traffic statistics, etc.
"""
import os
from logging import debug, info, warning, error, exception
from time import time
from struct import pack, unpack
from socket import socket, AF_INET, SOCK_DGRAM
import random
from ..util.crypto import Crypto, Authenticator
from fyuneru.util.crypto import randrange
from __protocol import *
IPCPort = 64089
##############################################################################
class InternalSocketServer:
__sockpath = None
__sock = None
local = None
IPCKey = None
peers = {}
__answerFunctions = {} # for IPC query/info service
def __init__(self, key):
self.IPCKey = os.urandom(32)
self.__crypto = Crypto(key)
self.__authenticator = Authenticator(self.IPCKey)
self.local = ("127.0.0.1", IPCPort)
self.__sock = socket(AF_INET, SOCK_DGRAM)
self.__sock.bind(self.local)
def __getattr__(self, name):
return getattr(self.__sock, name)
def __registerPeer(self, addrTuple, timestamp=None):
"""Register a peer's activity. This implies we have heard from this
peer. If timestamp is given, it will be used to update the last network
reception time."""
now = time()
if not self.peers.has_key(addrTuple):
self.peers[addrTuple] = {\
"recv": False,
"send": False,
"heartbeat": now,
}
return
if timestamp: self.peers[addrTuple]["recv"] = timestamp
self.peers[addrTuple]["heartbeat"] = now
def __choosePeer(self):
"""Choose a peer randomly. This implies we are going to send a packet
to this peer, and thus the sending timing will be updated."""
possiblePeers = [i for i in self.peers if self.peers[i] != False]
if len(possiblePeers) < 1: return None
peer = possiblePeers[randrange(0, len(possiblePeers))]
self.peers[peer]["send"] = time()
return peer
def __sendPacket(self, packet, to):
"""Send a packet class to a destination using local socket."""
s = self.__authenticator.sign(str(packet))
self.__sock.sendto(s, to)
def __recvBuffer(self, buf, sender):
"""Receive a buffer, unpack into packet, and dispatch it to different
handlers. Returns buffer when unpacked is a DataPacket. Otherwise
None."""
# See if is a data packet, which is special.
buf = self.__authenticator.verify(buf)
if not buf: return None # signature check failed
packet = loadBufferToPacket(buf)
if not packet: return None
if isinstance(packet, DataPacket): return packet.buffer
# If not, call different handlers to handle this.
if isinstance(packet, HeartbeatPacket):
self.__handleHeartbeatPacket(packet, sender)
return None
if isinstance(packet, QueryPacket):
self.__handleQueryPacket(packet, sender)
return None
# ---------- inner handlers for different packets
def __handleHeartbeatPacket(self, packet, sender):
# If this is a greeting word, register this as a new connected peer
# and answer.
self.__registerPeer(sender)
self.__sendPacket(packet, sender)
def __handleQueryPacket(self, packet, sender):
question = packet.question
debug("Got a query packet.")
if self.__answerFunctions.has_key(question):
# a new answer formular
answer = InfoPacket()
# call handler func to fill in the answer formular
s = self.__answerFunctions[question](packet.arguments, answer)
# send the answer back
if s: self.__sendPacket(answer, sender)
# ---------- public functions
def onQuery(self, question, answerfunc):
self.__answerFunctions[question] = answerfunc
def close(self):
# close socket
debug("Internal socket shutting down...")
try:
self.__sock.close()
except Exception,e:
error("Error closing socket.")
exception(e)
def clean(self):
# reserved for doing clean up jobs relating to the peer delays
removeList = []
now = time()
for each in self.peers:
if not self.peers[each]:
# if peer has been marked as False, because of errors, etc
removeList.append(each)
continue
# if we have not heard from peer for some while, take it as stale
if now - self.peers[each]["heartbeat"] > 5:
removeList.append(each)
if len(removeList) > 0:
for each in removeList:
# delete peers: forget them(no more tasks will be assigned)
self.peers[each] = False
del self.peers[each]
warning(\
"Following proxies are removed due to irresponsibility: \n" +
" \n".join(["%s:%d" % i for i in removeList])
)
def receive(self):
buf, sender = self.__sock.recvfrom(65536)
buf = self.__recvBuffer(buf, sender) # pre handling this buffer
if not buf: return None # digested within other mechanism. exit.
# Otherwise, this is data buffer and has to be decrypted correctly.
decryption = self.__crypto.decrypt(buf)
if not decryption: return None
# Decrypted data must be also good formated.
if len(decryption) < 8: return None
header = decryption[:8]
timestamp = unpack('<d', header)[0]
buf = decryption[8:]
# Only then we will recognize this as a legal status update from this
# peer. Refresh the peer record with updated receiving timings.
self.__registerPeer(sender, min(timestamp, time()))
return buf
def send(self, buf):
# Choose a peer randomly
peer = self.__choosePeer()
if not peer:
error("Not even one proxy found. Dropping a packet.")
return
# Prepare for the data that's going to be sent to this peer
header = pack('<d', time())
encryption = self.__crypto.encrypt(header + buf)
# Send to this peer. If anything goes wrong, mark this peer as False
try:
packet = DataPacket()
packet.buffer = encryption
self.__sendPacket(packet, peer)
except Exception,e:
exception(e) # for debug
warning(\
("Failed sending to proxy listening at %s:%d." % peer) +
"This proxy will be removed."
)
self.peers[peer] = False # this peer may not work
================================================
FILE: fyuneru/ipc/tools.py
================================================
"""
Tool functions for IPC
"""
from select import select
from logging import info, debug, exception, error
class InitConfigWaiter:
"""For clients. Send an IPC query with `init` command and block execution,
until a packet with proper answer is returned."""
__queried = False
def __queryFiller(self, packet):
packet.question = 'init'
packet.arguments = {"name": self.__ipc.name}
return True
def __infoReader(self, packet):
try:
if packet.title != 'init': return
self.__queried = {
"user": (packet.uid, packet.gid),
"config": packet.config,
"key": packet.key,
"mode": packet.mode,
}
except Exception,e:
exception(e)
def __init__(self, ipc):
self.__ipc = ipc
ipc.onInfo(lambda p: self.__infoReader(p))
def wait(self):
info("Waiting for configuration.")
i = 0
while i < 5:
self.__ipc.doQuery(lambda p: self.__queryFiller(p))
r = select([self.__ipc], [], [], 1.0)[0]
i += 1
if len(r) < 1: continue
self.__ipc.receive()
if self.__queried: break
if not self.__queried: return None
return self.__queried
================================================
FILE: fyuneru/ipc/url.py
================================================
"""
URL for starting IPC client
===========================
This tells the IPC client how to find the IPC server, do authentication
and identify itself to server.
"""
_URLPREFIX = "fyuneru-ipc://"
class InvalidIPCServerURLException(Exception): pass
class IPCServerURL:
host = "127.0.0.1"
port = 64089
key = ""
user = ""
def __init__(self, url=None):
if None == url: return
try:
if not url.startswith(_URLPREFIX): raise
url = url[len(_URLPREFIX):]
split = url.replace('@', ':').split(':')
user, key, host, port = split
user = user.decode('hex')
key = key.decode('hex')
port = int(port)
if not port in xrange(0, 65536): raise
self.host, self.port, self.key, self.user = host, port, key, user
except:
raise InvalidIPCServerURLException()
def __str__(self):
user = self.user.encode('hex')
key = self.key.encode('hex')
return _URLPREFIX + ("%s:%s@%s:%d" % (user, key, self.host, self.port))
if __name__ == "__main__":
url = IPCServerURL()
url.key = 'kadsjfakl'
url.user = 'proxy1'
print str(url)
url2 = IPCServerURL(str(url))
print url2.host, url2.port, url2.key, url2.user
================================================
FILE: fyuneru/net/__init__.py
================================================
================================================
FILE: fyuneru/net/protocol.py
================================================
# -*- coding: utf-8 -*-
"""
Packets of Fyuneru
"""
from struct import pack, unpack
SIGN_DATAPACKET = 0x01
##############################################################################
# DataPacket
class DataPacketException(Exception):
pass
class DataPacket:
data = ''
def __unpack(self, buf):
if(len(buf) < 1):
raise DataPacketException("Not a valid data packet.")
sign = unpack('<B', buf[:1])[0]
if sign != SIGN_DATAPACKET:
raise DataPacketException("Not a valid data packet.")
self.data = buf[1:]
def __str__(self):
"""Pack this packet into a string."""
buf = pack('<B', SIGN_DATAPACKET)
buf += self.data
return buf
def __init__(self, buf=None):
"""Read a packet with given `buf`, or construct an empty packet."""
if None != buf:
if type(buf) != str:
raise DataPacketException("Not a valid data packet.")
self.__unpack(buf)
return
##############################################################################
#
if __name__ == '__main__':
packet = DataPacket()
packet.data = 'abcdefg'
print DataPacket(str(packet))
================================================
FILE: fyuneru/net/vnet.py
================================================
# -*- coding: utf-8 -*-
import os
import fcntl
import struct
from logging import info, debug, critical, exception
from select import select
TUNSETIFF = 0x400454ca
IFF_TUN = 0x0001 # Set up TUN device
IFF_TAP = 0x0002 # Set up TAP device
IFF_NO_PI = 0x1000 # Without this flag, received frame will have 4 bytes
# for flags and protocol(each 2 bytes)
class VirtualNetworkInterfaceException(Exception): pass
class VirtualNetworkInterface:
mtu = 1200
netmask = "255.255.255.0"
def __getTUNDeviceLocation(self):
if os.path.exists("/dev/net/tun"): return "/dev/net/tun"
if os.path.exists("/dev/tun"): return "/dev/tun"
critical("TUN/TAP device not found on this OS!")
raise VirtualNetworkInterfaceException("No TUN/TAP device.")
def __init__(self, ip, dstip, netmask="255.255.255.0"):
self.addr = ip
self.dstaddr = dstip
self.netmask = netmask
try:
self.__tun = os.open(self.__getTUNDeviceLocation(), os.O_RDWR)
tun = fcntl.ioctl(\
self.__tun,
TUNSETIFF,
struct.pack("16sH", "fyuneru-%d", IFF_TUN)
)
except Exception,e:
exception(e)
raise VirtualNetworkInterfaceException(\
"Cannot set TUN/TAP device."
)
self.name = tun[:16].strip("\x00")
def up(self):
os.system("ifconfig %s inet %s netmask %s pointopoint %s" %\
(self.name, self.addr, self.netmask, self.dstaddr)
)
os.system("ifconfig %s mtu %d up" % (self.name, self.mtu))
info(\
"""%s: mtu %d addr %s netmask %s dstaddr %s""" % \
(\
self.name,
self.mtu,
self.addr,
self.netmask,
self.dstaddr
)
)
def fileno(self):
return self.__tun
def write(self, buf):
os.write(self.__tun, buf)
def read(self, size=65536):
return os.read(self.__tun, size)
def close(self):
try:
os.close(self.__tun)
except:
pass
if __name__ == "__main__":
vn = VirtualNetworkInterface({\
"ip": "10.1.0.2",
"dstip": "10.1.0.1",
"netmask": "255.255.255.0",
})
vn.up()
select([vn], [], [])
================================================
FILE: fyuneru/util/__init__.py
================================================
================================================
FILE: fyuneru/util/__xsalsa20.py
================================================
"""
Modified XSalsa20/16 Cipher Implementation in Pure Python
======================================================
This library provides a pure python implementation of XSalsa20/16 cipher.
This library is a modification of XSalsa20 cipher and adapted for fyuneru.
Changed is merely the nonce length from 24 bytes to 28 bytes. The extra 4 bytes
are given as higher 32 bits of the counter, making the counter only able
to count with 32 bits. Since our usage of this cipher uses a different nonce
for each plaintext being encrypted, but each plaintext will not be more than
65536 bytes(describable with just 2 bytes), this is better suited.
This implementation is not audited and may have errors that lead to serious
problems. And it's slow.
"""
import array
import math
uintArray = lambda l: array.array('I', [0] * l)
uintArray = lambda l: [0] * l
class XSalsa20:
def __salsa20Core(self, x, inbuf, oubuf):
for i in xrange(0, 16): x[i] = inbuf[i]
R = lambda a,b: (((a) << (b)) | ((a) >> (32 - (b)))) & 0xFFFFFFFF
for i in xrange(0, 8): # half of r(=16), r/2 rounds
x[ 4] ^= R(x[ 0]+x[12], 7)
x[ 8] ^= R(x[ 4]+x[ 0], 9)
x[12] ^= R(x[ 8]+x[ 4],13)
x[ 0] ^= R(x[12]+x[ 8],18)
x[ 9] ^= R(x[ 5]+x[ 1], 7)
x[13] ^= R(x[ 9]+x[ 5], 9)
x[ 1] ^= R(x[13]+x[ 9],13)
x[ 5] ^= R(x[ 1]+x[13],18)
x[14] ^= R(x[10]+x[ 6], 7)
x[ 2] ^= R(x[14]+x[10], 9)
x[ 6] ^= R(x[ 2]+x[14],13)
x[10] ^= R(x[ 6]+x[ 2],18)
x[ 3] ^= R(x[15]+x[11], 7)
x[ 7] ^= R(x[ 3]+x[15], 9)
x[11] ^= R(x[ 7]+x[ 3],13)
x[15] ^= R(x[11]+x[ 7],18)
x[ 1] ^= R(x[ 0]+x[ 3], 7)
x[ 2] ^= R(x[ 1]+x[ 0], 9)
x[ 3] ^= R(x[ 2]+x[ 1],13)
x[ 0] ^= R(x[ 3]+x[ 2],18)
x[ 6] ^= R(x[ 5]+x[ 4], 7)
x[ 7] ^= R(x[ 6]+x[ 5], 9)
x[ 4] ^= R(x[ 7]+x[ 6],13)
x[ 5] ^= R(x[ 4]+x[ 7],18)
x[11] ^= R(x[10]+x[ 9], 7)
x[ 8] ^= R(x[11]+x[10], 9)
x[ 9] ^= R(x[ 8]+x[11],13)
x[10] ^= R(x[ 9]+x[ 8],18)
x[12] ^= R(x[15]+x[14], 7)
x[13] ^= R(x[12]+x[15], 9)
x[14] ^= R(x[13]+x[12],13)
x[15] ^= R(x[14]+x[13],18)
for i in xrange(0, 16): oubuf[i] = (inbuf[i] + x[i]) & 0xFFFFFFFF
def __streamXor(self, iv, key, buf):
c2u = lambda src, f:\
src[f] + (src[f+1] << 4) + (src[f+2] << 8) + (src[f+3] << 12)
key = bytearray(key)
iv = bytearray(iv)
len0 = len(buf)
buf = bytearray(buf) + bytearray('0' * 64)
if len(key) != 32 or len(iv) != 28:
raise Exception("Key must be 32 bytes, IV must be 28 bytes")
temp = uintArray(16)
b1i, b1o = uintArray(16), uintArray(16)
b2i, b2o = uintArray(16), uintArray(16)
salsa20Constants = (0x61707865, 0x3320646e, 0x79622d32, 0x6b206574)
b1i[0], b1i[5], b1i[10], b1i[15] = salsa20Constants
b1i[1], b1i[2], b1i[3], b1i[4] = \
c2u(key, 0), c2u(key, 4), c2u(key, 8), c2u(key, 12)
b1i[11], b1i[12], b1i[13], b1i[14] = \
c2u(key, 16), c2u(key, 20), c2u(key, 24), c2u(key, 28)
b1i[6], b1i[7], b1i[8], b1i[9] = \
c2u(iv, 0), c2u(iv, 4), c2u(iv, 8), c2u(iv, 12)
self.__salsa20Core(temp, b1i, b1o)
b2i[0], b2i[5], b2i[10], b2i[15] = salsa20Constants
b2i[1], b2i[2], b2i[3], b2i[4] = b1o[0], b1o[5], b1o[10], b1o[15]
b2i[11], b2i[12], b2i[13], b2i[14] = b1o[6], b1o[7], b1o[8], b1o[9]
b2i[6], b2i[7] = c2u(iv, 16), c2u(iv, 20)
b2i[8] = c2u(iv, 24)
i = 0
output = bytearray('0' * (len0 + 64))
b2i[9] = 0
while i < len0:
self.__salsa20Core(temp, b2i, b2o)
for j in xrange(0, 16):
output[i+0] = buf[i+0] ^ ((b2o[j] & 0x000000FF))
output[i+1] = buf[i+1] ^ ((b2o[j] & 0x0000FF00) >> 8)
output[i+2] = buf[i+2] ^ ((b2o[j] & 0x00FF0000) >> 16)
output[i+3] = buf[i+3] ^ ((b2o[j] & 0xFF000000) >> 24)
i += 4
b2i[9] += 1
return str(output[:len0])
def encrypt(self, iv, key, buf):
return self.__streamXor(iv, key, buf)
def decrypt(self, iv, key, buf):
return self.__streamXor(iv, key, buf)
if __name__ == "__main__":
from salsa20 import XSalsa20_xor
key = bytearray([0] * 32)
key[0] = 0x80
iv = bytearray([0] * 28)
buf = bytearray([0] * 50)
c = XSalsa20()
print c.encrypt(iv, key, buf).encode('hex')
print XSalsa20_xor(str(buf), str(iv)[:24], str(key)).encode('hex')
exit()
key = '0' * 32
iv = '1' * 28
buf = 't' * 1684
c = XSalsa20()
repeat = 1000
import time
a = time.time()
for i in xrange(0, repeat):
ourres = c.encrypt(iv, key, buf)
b = time.time()
print len(buf) * repeat * 1.0 / (b - a)
================================================
FILE: fyuneru/util/config.py
================================================
# -*- coding: utf-8 -*-
"""
Config reader and manager for the core program
Class Configuration provides the functionalities for loading the `config.json`
and getting them parsed, and for generating the commands necessary for
initializing the proxy subprocesses(more details defined in module
`proxyconf`).
"""
import os
from distutils.version import StrictVersion
from json import loads
from ..ipc.url import IPCServerURL
VERSION_REQUIREMENT = "1.1" # required version of `config.json`
proxies = {
"shadowsocks": "proxy.shadowsocks.py",
"xmpp": "proxy.xmpp.py",
}
##############################################################################
class CoreConfiguration: pass
class ProxyConfiguration: pass
class ConfigFileException(Exception): pass
##############################################################################
class Configuration:
def __checkKeyExistence(self, parentDict, *keys):
for each in keys:
if not parentDict.has_key(each):
return False
return True
def __loadCore(self, core):
# parse json.core and save to class attributes
# get key for cryptography
self.key = str(core["key"])
if type(self.key) != str:
raise ConfigFileException("core.key is invalid.")
# get IP for server and client
if not core["server"].has_key("ip"):
raise ConfigFileException("core.server.ip must be specified.")
if not core["client"].has_key("ip"):
raise ConfigFileException("core.client.ip must be specified.")
self.serverIP = core["server"]["ip"]
self.clientIP = core["client"]["ip"]
# get UID/GID names
if not core["user"].has_key('uidname'):
raise ConfigFileException("core.user.uidname must be specified.")
if not core["user"].has_key('gidname'):
raise ConfigFileException("core.user.gidname must be specified.")
self.user = (core["user"]["uidname"], core["user"]["gidname"])
def __loadProxyAllocations(self, proxies):
# get and validate proxy and ports allocations
self.__proxies = {}
for proxyName in proxies:
proxyConfig = proxies[proxyName]
self.__proxies[proxyName] = proxyConfig
def listProxies(self):
return self.__proxies.keys()
def getProxyConfig(self, name):
if not self.__proxies.has_key(name):
raise ConfigFileException("No such proxy defined.")
proxyConfig = self.__proxies[name]
return proxyConfig
def getProxyInitParameters(self, proxyName, IPCServer, debug=False):
proxyConfig = self.__proxies[proxyName]
proxyType = proxyConfig["type"]
cmd = ["python", proxies[proxyType]]
if debug: cmd += ["--debug"]
url = IPCServerURL()
url.host = IPCServer.local[0]
url.port = IPCServer.local[1]
url.user = proxyName
url.key = IPCServer.IPCKey
cmd += [str(url)]
return cmd
def getCoreInitParameters(self, mode):
ret = CoreConfiguration()
ret.uid = self.user[0]
ret.gid = self.user[1]
ret.key = self.key
if mode == 's':
ret.localIP = self.serverIP
ret.remoteIP = self.clientIP
elif mode == 'c':
ret.localIP = self.clientIP
ret.remoteIP = self.serverIP
else:
raise Exception("Invalid run mode, either `c` or `s`.")
return ret
def __init__(self, config):
# try load the configuration file string, and parse into JSON.
try:
json = loads(config)
except Exception,e:
raise ConfigFileException("config.json parse error.")
# read config file version declaration
jsonVersion = "0.0.0"
if json.has_key("version"):
jsonVersion = json["version"]
if StrictVersion(jsonVersion) < StrictVersion(VERSION_REQUIREMENT):
raise ConfigFileException("config.json version too old.")
# check existence of 'core' and 'proxies' entry
if not self.__checkKeyExistence(json, 'core', 'proxies'):
raise ConfigFileException("config.json incomplete.")
# check entries of core
jsonCore = json["core"]
if not self.__checkKeyExistence(\
jsonCore,
'server',
'client',
'user',
'key',
):
raise ConfigFileException("config.json incomplete.")
self.__loadCore(jsonCore)
# check for proxy allocations
jsonProxies = json["proxies"]
self.__loadProxyAllocations(jsonProxies)
##############################################################################
if __name__ == "__main__":
j = open('../config.json', 'r').read()
#print j
config = Configuration(j)
lst = config.listProxies()
for n in lst:
proxyConfig = config.getProxyConfig(n)
print proxyConfig.getInitCommand('s')
print proxyConfig.getInitCommand('c')
================================================
FILE: fyuneru/util/crypto.py
================================================
"""
Cryptographical Services
========================
## Authenticator in IPC communication
In IPC communications, data are not encrypted, but it has to be authenticated
since any process can practically send data to a UDP port our process listening
on. We currently use HMAC-MD5, for local usage this should be enough and
performance comes first.
## Encryption and Decryption for Internet Traffic
### Schema
1. Cleartext:
| BUFFER arbitary length |
| DATALEN | | RANDOM PADDING |
Bytes | 2 | ? | ? |
2. Encryption:
| ENCRYPTED BUFFER ................................ |
| IV | HMAC |
Bytes | 24 | 20 | |
3. Result:
| PSEUDORANDOM BUFFER OF RANDOM LENGTH ........................ |
Bytes | ? |
### Remarks
1. We use HMAC-SHA1 to authenticate the encrypted buffer in following reasons:
1.1 A good HMAC should be indistinguishable against random bytes without
knowledge of key. Putting it outside the cleartext and making it to
authenticate the cihpertext makes it easier(with a known HMAC key!) to find
out invalid packets without decryption.
1.2 This is secure enough. SHA1 is attacked only in theory, and that's even not
HMAC(HMAC-MD5 is still secure enough, think of that). Even an attacker has
got some advances in theory, it doesn't mean he will be able to use it
detecting a real-time traffic effectively(making it hard to do deep packet
analysis).
1.3 Unrevealing the HMAC key will not lead to the leak of encryption key, since
the HMAC key is derived from encryption key. If however someone has got
this key, he will be able to trick our program with seemingly legal
traffic, and he cannot distinguish or get a proof, that it is us not him
that has sent anything.
1.4 HMAC-SHA1 is 12 bytes shorter as SHA256.
"""
import hashlib
import hmac
import math
from os import urandom
from struct import pack, unpack
from salsa20 import XSalsa20_xor
_HASHALGO = hashlib.sha1
_HASHLEN = len(_HASHALGO('').digest())
_RESULT_SIZE = 0xFFFF - 24 - 2 - _HASHLEN
class CryptoException(Exception): pass
def randint(a, b):
i = unpack('<L', urandom(4))[0]
return a + i % (b-a+1)
def randrange(a, b):
i = unpack('<L', urandom(4))[0]
return a + i % (b-a)
def decidePaddingLength(bufferLength):
global _RESULT_SIZE
return 0
if bufferLength < 1500:
randSize = randint(0, 1500)
if randSize > bufferLength:
return randSize - bufferLength
return 0
else:
maxSize = int(min(_RESULT_SIZE - bufferLength, bufferLength * 2.0))
return randint(0, maxSize)
def KDF1(secret, length):
"""ISO-18033-2 defined simple KDF function. Notice we assume secret is
already long and random enough."""
global _HASHALGO, _HASHLEN
d = int(math.ceil(length * 1.0 / _HASHLEN))
T = ""
for i in xrange(0, d):
C = pack("<H", i)
T = T + _HASHALGO(secret + C).digest()
return T[:length]
KDF = KDF1
##############################################################################
class Authenticator:
"""Authentication service for IPC mechanism. Provides signing and
verifying. For signing a buffer, the HMAC result will be returned together
with original buffer. This result can then be passed to the verify
function on another process. If verification succeeded, return buffer. If
not, return None."""
__ALGORITHM_CONFIG = (hashlib.md5, 16) # 16 bytes output for MD5.
def __init__(self, key):
self.__origHMAC = hmac.new(key, '', self.__ALGORITHM_CONFIG[0])
def __HMAC(self, buf):
worker = self.__origHMAC.copy()
worker.update(buf)
return worker.digest()
def sign(self, buf):
signature = self.__HMAC(buf)
return signature + buf
def verify(self, buf):
hlen = self.__ALGORITHM_CONFIG[1]
if len(buf) < hlen: return None
signature = buf[:hlen]
buf = buf[hlen:]
if self.__HMAC(buf) != signature: return None
return buf
class Crypto:
"""Crypto service for traffic over Internet."""
def __init__(self, passphrase):
self.__KEY = KDF(passphrase, 32)
HMACKEY = KDF(self.__KEY, _HASHLEN)
self.__origHMAC = hmac.new(HMACKEY, '', _HASHALGO)
def __HMAC(self, buf):
worker = self.__origHMAC.copy()
worker.update(buf)
return worker.digest()
def encrypt(self, buf):
if len(buf) > _RESULT_SIZE:
raise CryptoException('Buffer too large to be encrypted.')
buflen = len(buf)
lenstr = pack('<H', buflen)
padding = '0' * decidePaddingLength(buflen) # TODO discuss security here!
cleartext = lenstr + buf + padding
iv = urandom(24) # since IV will be exposed, has to be crypto. random.
ciphertext = XSalsa20_xor(cleartext, iv, self.__KEY)
hmac = self.__HMAC(ciphertext)
return iv + hmac + ciphertext
def decrypt(self, buf):
if len(buf) < 24 + _HASHLEN:
return False
iv = buf[:24]
hmac = buf[24:24+_HASHLEN]
ciphertext = buf[24+_HASHLEN:]
hmac2 = self.__HMAC(ciphertext)
if hmac2 != hmac: return False
decrypted = XSalsa20_xor(ciphertext, iv, self.__KEY)
lenstr = decrypted[:2]
buflen = unpack('<H', lenstr)[0]
buf = decrypted[2:][:buflen]
if len(buf) != buflen: return False
return buf
if __name__ == '__main__':
encryptor = Crypto('test')
decryptor = Crypto('test')
import time
a = time.time()
for x in xrange(0, 1024):
encrypted = encryptor.encrypt('a' * 400)
decrypted = encryptor.decrypt(encrypted)
if not decrypted:
print "*"
b = time.time()
print "done in %f seconds" % (b-a)
# print len(encrypted), decryptor.decrypt(encrypted)
================================================
FILE: fyuneru/util/debug.py
================================================
import socket
from struct import *
import logging
import sys
import os
##############################################################################
def configLoggingModule(debug):
logLevel = logging.INFO
if debug: logLevel = logging.DEBUG
procname = os.path.basename(sys.argv[0])
logFormat = \
"\n=== [%%(asctime)-15s] [%s|%d]: %%(levelname)s\n %%(message)s"\
% (procname, os.getpid())
logging.basicConfig(\
level=logLevel,
format=logFormat
)
##############################################################################
def colorify(text, color):
colors = {\
"blue": '\033[94m',
"green": '\033[92m',
"red": '\033[91m',
}
ENDC = '\033[0m'
BOLD = '\033[1m'
UNDERLINE = '\033[4m'
return colors[color] + text + ENDC
##############################################################################
def _decodeIPFrame(buf):
# meaning this is not raw packet, but with 4 bytes prefixed as in TUN
# device defined
if buf[:2] == '\x00\x00': buf = buf[4:]
ip_header = buf[0:20]
iph = unpack('!BBHHHBBH4s4s' , ip_header)
version_ihl = iph[0]
version = (version_ihl >> 4) & 0xF
ihl = version_ihl & 0xF
iph_length = ihl * 4
ttl = iph[5]
protocol = iph[6]
s_addr = socket.inet_ntoa(iph[8])
d_addr = socket.inet_ntoa(iph[9])
payload = buf[iph_length:]
ip_meta = {
"version": version,
"length": str(ihl),
"TTL": ttl,
"protocol": protocol,
"src": str(s_addr),
"dst": str(d_addr),
}
return ip_meta, payload
def showPacket(buf):
width = 16
lines = []
origbuf = buf
while buf != '':
lines.append(buf[:width])
buf = buf[width:]
ret = ''
start = 0
for line in lines:
hexstr = ''
asciistr = ''
for c in line:
ordc = ord(c)
hexstr += '%02x ' % ordc
if ordc <= 0x7E and ordc >= 0x20:
asciistr += c
else:
asciistr += '.'
start += len(line)
hexstr = hexstr.ljust(3 * width)
ret += "%08x: %s: %s\n" % (start, hexstr, asciistr)
ipReport = ''
try:
ipMeta, ipPayload = _decodeIPFrame(origbuf)
ipReport = "Ver %s :: TTL %d :: Protocol %s :: Src %s :: Dst %s" % (\
ipMeta["version"],
ipMeta["TTL"],
ipMeta["protocol"],
ipMeta["src"],
ipMeta["dst"]
)
ret = ipReport + "\n" + ret
except Exception,e:
print e
pass
return ret.strip()
##############################################################################
if __name__ == '__main__':
import os
print showPacket(os.urandom(200))
================================================
FILE: fyuneru/util/droproot.py
================================================
"""
Function to drop root privileges
<http://stackoverflow.com/questions/2699907/dropping-root-permissions-in-python>
This has to be used in 2 places:
* `run_as.py`, whose root privilege will be passed to `tunnel.py`. Root is
no more needed after `tunnel.py` is started, and will be dropped.
* `tunnel.py`, which will need root to setup TUN device. After that, root will
be dropped.
"""
import grp
import os
import pwd
def dropRoot(uid_name='nobody', gid_name='nobody'):
if os.getuid() != 0:
# We're not root so, like, whatever dude
return
# Get the uid/gid from the name
running_uid = pwd.getpwnam(uid_name).pw_uid
running_gid = grp.getgrnam(gid_name).gr_gid
# Remove group privileges
os.setgroups([])
# Try setting the new uid/gid
os.setgid(running_gid)
os.setuid(running_uid)
# Ensure a very conservative umask
old_umask = os.umask(077)
================================================
FILE: fyuneru/util/pidfile.py
================================================
# -*- coding: utf-8 -*-
import os
import time
class ProcessRunningException(Exception): pass
class PidfileNonExistentException(Exception): pass
class PidfileWatcher:
"""Watches if a pidfile exists. If exist at time of initialization of this
instance, may also be used to check if the pidfile is changed or deleted.
This is useful for subprocesses who may want to exit after main process
is terminated.
"""
__lastcheck = 0
pid = None
pidfile = None
def __init__(self, path):
self.pidfile = path
self.__lastcheck = time.time()
try:
pidfile = open(self.pidfile, 'r')
self.pid = pidfile.read()
pidfile.close()
except:
raise PidfileNonExistentException()
def check(self):
try:
pidfile = open(self.pidfile, 'r')
pid = pidfile.read()
pidfile.close()
except:
return False
return pid == self.pid
class PidfileCreator:
def __init__(self, path, log=sys.stdout.write, warn=sys.stderr.write):
self.pidfile = path
self.log = log
self.warn = warn
def __enter__(self):
try:
self.pidfd = os.open(self.pidfile, os.O_CREAT|os.O_WRONLY|os.O_EXCL)
self.log('locked pidfile %s' % self.pidfile)
except OSError as e:
if e.errno == errno.EEXIST:
pid = self._check()
if pid:
self.pidfd = None
raise ProcessRunningException('process already running in %s as pid %s' % (self.pidfile, pid))
else:
os.remove(self.pidfile)
self.warn('removed staled lockfile %s' % (self.pidfile))
self.pidfd = os.open(self.pidfile, os.O_CREAT|os.O_WRONLY|os.O_EXCL)
else:
raise
os.write(self.pidfd, str(os.getpid()))
os.close(self.pidfd)
return self
def __exit__(self, t, e, tb):
# return false to raise, true to pass
if t is None:
# normal condition, no exception
self._remove()
return True
elif t is PidfileProcessRunningException:
# do not remove the other process lockfile
return False
else:
# other exception
if self.pidfd:
# this was our lockfile, removing
self._remove()
return False
def _remove(self):
self.log('removed pidfile %s' % self.pidfile)
os.remove(self.pidfile)
def _check(self):
"""check if a process is still running
the process id is expected to be in pidfile, which should exist.
if it is still running, returns the pid, if not, return False."""
with open(self.pidfile, 'r') as f:
try:
pidstr = f.read()
pid = int(pidstr)
except ValueError:
# not an integer
self.log("not an integer: %s" % pidstr)
return False
try:
os.kill(pid, 0)
except OSError:
self.log("can't deliver signal to %s" % pid)
return False
else:
return pid
================================================
FILE: fyuneru/util/procmgr.py
================================================
# -*- coding: utf-8 -*-
import signal
import subprocess
import time
from logging import debug, info, warning, error
class ProcessManagerException(Exception): pass
class ProcessManager:
__processes = {}
__commands = {}
def __init__(self):
signal.signal(signal.SIGTERM, self.killall)
def __pollAll(self):
for each in self.__processes:
try:
self.processes[each].poll()
except:
pass
def __startProcess(self, name, command):
info("ProcessManager starting [%s]: %s" % (name, " ".join(command)))
proc = subprocess.Popen(command)
self.__processes[name] = proc
self.__commands[name] = command
def new(self, name, command):
if self.__processes.has_key(name):
raise ProcessManagerException(\
"Child process already registered with name [%s]" % name)
self.__startProcess(name, command)
def killall(self, tolerance=1.0):
for each in self.__processes:
self.kill(each, tolerance)
def kill(self, name, tolerance=1.0):
if not self.__processes.has_key(name): return True
sigtermSuccess = False
if tolerance:
try:
info("ProcessManager stopping [%s] gracefully." % name)
self.__processes[name].terminate()
time.sleep(1.0)
self.__processes[name].poll()
sigtermSuccess = (self.__processes[name].returncode != None)
except:
pass
if sigtermSuccess:
info("ProcessManager confirmed stop of [%s]." % name)
return True
warning("ProcessManager couldn't stop [%s] with tolerance." % name)
try:
self.__processes[name].kill()
info("ProcessManager killed [%s]." % name)
return True
except Exception,e:
error(("ProcessManager cannot stop [%s]. " % name) + \
"It may has been already killed. Giving up.")
return False
def restart(self, name, tolerance=1.0):
if not self.__commands.has_key(name): return False
if not self.kill(name, tolerance):
return False
command = self.__commands[name]
self.__startProcess(name, command)
def wait(self, name):
self.__processes[name].wait()
class ParentProcessWatcher:
__last = 0
def __init__(self, pid, terminator):
self.__pid = pid
self.__func = terminator
self.__last = time.time()
def watch(self):
now = time.time()
if now - self.__last < 10: return
proclist = subprocess.check_output(['ps', '-e']).split('\n')
strpid = str(self.__pid)
found = False
for each in proclist:
if each.strip().startswith(strpid):
found = True
return
info("Watching: %d" % self.__last)
if found:
self.__last = now
else:
warning("Parent pid [%d] not present anymore." % self.__pid)
self.__func(None, None)
================================================
FILE: proxy.icmp.py
================================================
"""
ICMP Proxy Process
==================
This is a very simple proxy that puts our encrypted payloads into ICMP packets.
Since encrypted payloads will be recognized using cryptographical means, we
won't do anything on the payload.
ICMP Proxy is currently only one-way: client->server is okay, reverse is not
supported. You have to use other means of proxies to get replied.
"""
# TODO: modify fyuneru.intsck to make server not try to use this tunnel as
# answer.
import argparse
import signal
from select import select
from fyuneru.ipc.client import InternalSocketClient
from fyuneru.droproot import dropRoot
def log(x):
print "proxy-icmp-client: %s" % x
# ----------- parse arguments
parser = argparse.ArgumentParser()
# drop privilege to ...
parser.add_argument("--uidname", metavar="UID_NAME", type=str, required=True)
parser.add_argument("--gidname", metavar="GID_NAME", type=str, required=True)
parser.add_argument("--client-send-to", type=str, required=False)
args = parser.parse_args()
##############################################################################
if None == args.client_send_to:
# run as server mode
sck = socket.socket(socket.AF_INET,socket.SOCK_RAW,socket.IPPROTO_ICMP)
sck.setsockopt(socket.SOL_IP, socket.IP_HDRINCL, 1)
dropRoot(args.uidname, args.gidname)
else:
# run as client mode
pass
local = InternalSocketClient(args.socket)
##############################################################################
def doExit(signum, frame):
global local, proxy
try:
local.close()
except:
pass
try:
proxy.xmpp.disconnect()
except:
pass
print "exit now."
exit()
signal.signal(signal.SIGTERM, doExit)
##############################################################################
sockets = {
proxy.xmpp.Connection._sock: 'proxy',
local: 'local',
}
while True:
try:
local.heartbeat()
r, w, _ = select(sockets.keys(), [], [], 1)
for each in r:
if sockets[each] == 'proxy':
proxy.xmpp.Process(1)
for b in proxy.recvQueue:
log("Received %d bytes, sending to core." % len(b))
local.send(b)
proxy.recvQueue = []
if sockets[each] == 'local':
recv = local.receive()
if not recv: continue
log("Received %d bytes, sending to tunnel." % len(recv))
proxy.send(recv)
except KeyboardInterrupt:
doExit(None,None)
================================================
FILE: proxy.shadowsocks.py
================================================
# -*- coding: utf-8 -*-
import argparse
import os
import sys
from select import select
import socket
import signal
import time
import logging
from logging import info, debug, warning, error
import hashlib
import hmac
from fyuneru.ipc.client import InternalSocketClient
from fyuneru.util.droproot import dropRoot
from fyuneru.util.procmgr import ProcessManager
from fyuneru.util.debug import configLoggingModule
from fyuneru.ipc.url import IPCServerURL
from fyuneru.ipc.tools import InitConfigWaiter
ENCRYPTION_METHOD = 'aes-256-cfb'
##############################################################################
# ----------- parse arguments
parser = argparse.ArgumentParser()
parser.add_argument("--debug", action="store_true", default=False)
parser.add_argument("IPC_SERVER_URL", type=str)
args = parser.parse_args()
##############################################################################
configLoggingModule(args.debug)
##############################################################################
# use command line to initialize IPC client
ipc = InternalSocketClient(args.IPC_SERVER_URL)
queried = InitConfigWaiter(ipc).wait()
if not queried:
error("Configuration timed out. Exit.")
ipc.close()
sys.exit(1)
##############################################################################
debug("Drop privilege to %s:%s" % queried["user"])
dropRoot(*queried["user"])
##############################################################################
# start shadowsocks process
procmgr = ProcessManager()
sharedsecret= hmac.HMAC(
str(ipc.name + '-shadowsocks'),
queried["key"],
hashlib.sha256
).digest().encode('base64').strip()
proxyConfig = queried["config"]
forwardToPort = proxyConfig["server"]["forward-to"] # exit port at server
if 'c' == queried["mode"]: # CLIENT mode
if proxyConfig["client"].has_key("proxy"):
connectIP = proxyConfig["client"]["proxy"]["ip"]
connectPort = proxyConfig["client"]["proxy"]["port"]
else:
connectIP = proxyConfig["server"]["ip"]
connectPort = proxyConfig["server"]["port"]
sscmd = [
proxyConfig["client"]["bin"], # shadowsocks-libev
'-U', # UDP relay only
'-L', "127.0.0.1:%d" % forwardToPort, # destinating UDP addr
'-k', sharedsecret, # key
'-s', connectIP, # server host
'-p', str(connectPort), # server port
'-b', "127.0.0.1", # local addr
'-l', str(proxyConfig["client"]["port"]), # local port(entrance)
'-m', ENCRYPTION_METHOD, # encryption method
]
elif 's' == queried['mode']: # SERVER mode
sscmd = [
proxyConfig["server"]["bin"], # shadowsocks-libev
'-U', # UDP relay only
'-k', sharedsecret, # key
'-s', proxyConfig["server"]["ip"], # server host
'-p', str(proxyConfig["server"]["port"]), # server port
'-m', ENCRYPTION_METHOD, # encryption method
]
else:
sys.exit(127)
procmgr.new('shadowsocks', sscmd)
##############################################################################
proxySocket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
if 's' == queried["mode"]:
proxySocket.bind(('127.0.0.1', forwardToPort))
proxyPeer = None # not knowing where to send data back
else:
# send to local tunnel entrance
proxyPeer = ('127.0.0.1', proxyConfig["client"]["port"])
##############################################################################
def doExit(signum, frame):
global ipc, proxySocket, procmgr
try:
ipc.close()
except:
pass
try:
proxySocket.close()
except:
pass
try:
procmgr.killall()
except:
pass
info("Exit now.")
exit()
signal.signal(signal.SIGTERM, doExit)
##############################################################################
while True:
try:
ipc.heartbeat()
selected = select([ipc, proxySocket], [], [], 1.0)
if len(selected) < 1:
continue
readables = selected[0]
for each in readables:
if each == ipc:
buf = ipc.receive()
if None == buf: continue
if None == proxyPeer: continue
debug("Received %d bytes, sending to tunnel." % len(buf))
proxySocket.sendto(buf, proxyPeer)
if each == proxySocket:
buf, sender = each.recvfrom(65536)
proxyPeer = sender
debug("Received %d bytes, sending back to core." % len(buf))
ipc.send(buf)
if ipc.broken: doExit(None, None)
except KeyboardInterrupt:
doExit(None, None)
================================================
FILE: proxy.tcp.py
================================================
# -*- coding: utf-8 -*-
"""
UDP over TCP proxy for Fyuneru.
"""
import argparse
import os
from select import select
import socket
import signal
import time
import logging
from logging import info, debug, warning, error
from fyuneru.ipc.client import InternalSocketClient
from fyuneru.util.droproot import dropRoot
from fyuneru.util.debug import configLoggingModule
##############################################################################
# ----------- parse arguments
parser = argparse.ArgumentParser()
# if enable debug mode
parser.add_argument("--debug", action="store_true", default=False)
# drop privilege to ...
parser.add_argument("--uidname", metavar="UID_NAME", type=str, required=True)
parser.add_argument("--gidname", metavar="GID_NAME", type=str, required=True)
# mode for this script to run
parser.add_argument(\
"--mode",
type=str,
choices=["server", "client"],
required=True
)
args = parser.parse_args()
##############################################################################
configLoggingModule(args.debug)
##############################################################################
dropRoot(args.uidname, args.gidname)
##############################################################################
class Datagram2Stream:
__buffer = ""
def put(self, datagram):
self.__buffer += datagram.encode('base64').strip() + '\n'
def get(self, size=32768):
ret = self.__buffer[:size]
self.__buffer = self.__buffer[size:]
return ret
class Stream2Datagram:
__buffer = ""
def put(self, buf):
self.__buffer += buf
def get(self):
n = self.__buffer.find('\n')
if n < 0: return None
self.__buffer = self.__buffer[n:].strip()
try:
ret = self.__buffer[:n].strip().decode('base64')
return ret
except:
return None
##############################################################################
localSocket = InternalSocketClient()
proxySocket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
if 'server' == args.mode:
proxySocket.bind(('127.0.0.1', args.FORWARD_TO))
proxyPeer = None # not knowing where to send data back
else:
proxyPeer = ('127.0.0.1', args.l) # send to local tunnel entrance
##############################################################################
def doExit(signum, frame):
global localSocket, proxySocket, procmgr
try:
localSocket.close()
except:
pass
try:
proxySocket.close()
except:
pass
try:
procmgr.killall()
except:
pass
info("Exit now.")
exit()
signal.signal(signal.SIGTERM, doExit)
##############################################################################
while True:
try:
localSocket.heartbeat()
selected = select([localSocket, proxySocket], [], [], 1.0)
if len(selected) < 1:
continue
readables = selected[0]
for each in readables:
if each == localSocket:
buf = localSocket.receive()
if None == buf: continue
if None == proxyPeer: continue
debug("Received %d bytes, sending to tunnel." % len(buf))
proxySocket.sendto(buf, proxyPeer)
if each == proxySocket:
buf, sender = each.recvfrom(65536)
proxyPeer = sender
debug("Received %d bytes, sending back to core." % len(buf))
localSocket.send(buf)
if localSocket.broken: doExit(None, None)
except KeyboardInterrupt:
doExit(None, None)
================================================
FILE: proxy.xmpp.py
================================================
"""
XMPP Proxy Process
==================
This proxy utilizes the `xmpppy` library. You have to first install it before
using this. If
sudo pip install xmpppy
doesn't work, you may have to download it from <http://xmpppy.sourceforge.net>
and install manually.
"""
import argparse
import sys
import signal
from select import select
import logging
from logging import info, warning, debug, exception, error
import xmpp
from fyuneru.ipc.client import InternalSocketClient
from fyuneru.util.droproot import dropRoot
from fyuneru.util.debug import configLoggingModule
from fyuneru.ipc.url import IPCServerURL
from fyuneru.ipc.tools import InitConfigWaiter
##############################################################################
# ----------- parse arguments
parser = argparse.ArgumentParser()
parser.add_argument("--debug", action="store_true", default=False)
parser.add_argument("IPC_SERVER_URL", type=str)
args = parser.parse_args()
##############################################################################
configLoggingModule(args.debug)
# use command line to initialize IPC client
ipc = InternalSocketClient(args.IPC_SERVER_URL)
queried = InitConfigWaiter(ipc).wait()
if not queried:
error("Configuration timed out. Exit.")
ipc.close()
sys.exit(1)
##############################################################################
debug("Drop privilege to %s:%s" % queried["user"])
dropRoot(*queried["user"])
##############################################################################
class SocketXMPPProxyException(Exception): pass
class SocketXMPPProxy:
def __init__(self, jid, password, peer):
self.__jid = xmpp.protocol.JID(jid)
self.__peer = xmpp.protocol.JID(peer)
self.__peerJIDStripped = self.__peer.getStripped()
self.xmpp = xmpp.Client(\
self.__jid.getDomain(),
debug=["always", "socket", "nodebuilder", "dispatcher"]
)
connection = self.xmpp.connect()
if not connection:
raise SocketXMPPProxyException("Unable to connect.")
authentication = self.xmpp.auth(\
self.__jid.getNode(),
password
)
if not authentication:
raise SocketXMPPProxyException("Authentication error.")
self.__registerHandlers()
self.__sendPresence()
def __registerHandlers(self):
self.xmpp.RegisterHandler('message', self.message)
def __sendPresence(self):
presence = xmpp.protocol.Presence(priority=999)
self.xmpp.send(presence)
recvQueue = []
def message(self, con, event):
msgtype = event.getType()
fromjid = event.getFrom().getStripped()
if fromjid != self.__peerJIDStripped: return
if msgtype in ('chat', 'normal', None):
body = event.getBody()
try:
self.recvQueue.append(body.decode('base64'))
except:
pass
def send(self, buf):
buf = buf.encode('base64')
message = xmpp.protocol.Message(to=self.__peer, body=buf, typ='chat')
self.xmpp.send(message)
##############################################################################
proxyConfig = queried["config"]
if 's' == queried["mode"]:
localJID = proxyConfig["server"]["jid"]
localPassword = proxyConfig["server"]["password"]
remoteJID = proxyConfig["client"]["jid"]
elif 'c' == queried["mode"]:
localJID = proxyConfig["client"]["jid"]
localPassword = proxyConfig["client"]["password"]
remoteJID = proxyConfig["server"]["jid"]
else:
sys.exit(127)
proxy = SocketXMPPProxy(localJID, localPassword, remoteJID)
info("XMPP proxy from [%s](local) to [%s](remote)." % (localJID, remoteJID))
##############################################################################
def doExit(signum, frame):
global ipc, proxy
try:
ipc.close()
except:
pass
try:
proxy.xmpp.disconnect()
except:
pass
print "exit now."
exit()
signal.signal(signal.SIGTERM, doExit)
##############################################################################
sockets = {
proxy.xmpp.Connection._sock: 'proxy',
ipc: 'ipc',
}
while True:
try:
if ipc.broken: doExit(None, None)
ipc.heartbeat()
r, w, _ = select(sockets.keys(), [], [], 1)
for each in r:
if sockets[each] == 'proxy':
proxy.xmpp.Process(1)
for b in proxy.recvQueue:
debug("Received %d bytes, sending to core." % len(b))
ipc.send(b)
proxy.recvQueue = []
if sockets[each] == 'ipc':
recv = ipc.receive()
if not recv: continue
debug("Received %d bytes, sending to tunnel." % len(recv))
proxy.send(recv)
except KeyboardInterrupt:
doExit(None,None)
except Exception,e:
exception(e)
================================================
FILE: run_as.py
================================================
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import argparse
import os
from select import select
import signal
import sys
import logging
from logging import info, debug, warning, error, exception, critical
from fyuneru.net.vnet import VirtualNetworkInterface
from fyuneru.util.config import Configuration
from fyuneru.util.debug import showPacket, configLoggingModule
from fyuneru.util.droproot import dropRoot
from fyuneru.net.protocol import DataPacket, DataPacketException
from fyuneru.ipc.server import InternalSocketServer
from fyuneru.util.procmgr import ProcessManager
##############################################################################
parser = argparse.ArgumentParser()
parser = argparse.ArgumentParser(description="""
This is the initator of Fyeneru proxy. Before running, put a `config.json`
in the same path as this file.
Requires root priviledge for running this script.
""")
parser.add_argument(\
"--debug",
action="store_true",
default=False,
help = "Print debug info, e.g. packet data."
)
parser.add_argument(\
"mode",
metavar="MODE",
type=str,
choices=['s', 'c'],
help="""
Either 'c' or 's', respectively for client mode and server mode.
"""
)
args = parser.parse_args()
PATH = os.path.realpath(os.path.dirname(sys.argv[0]))
MODE = args.mode
MTU = 1400
##############################################################################
# ---------- config log/debug functions
configLoggingModule(args.debug)
# ---------- load and parse configuration file
config = Configuration(open(os.path.join(PATH, 'config.json'), 'r').read())
coreConfig = config.getCoreInitParameters(MODE)
# ---------- initialize IPC and ProcessManager
ipc = InternalSocketServer(coreConfig.key)
processes = ProcessManager()
# ---------- config TUN device and start up
if "client" == args.mode:
info("Running as client.")
else:
info("Running as server.")
tun = VirtualNetworkInterface(coreConfig.localIP, coreConfig.remoteIP)
tun.netmask = "255.255.255.0"
tun.mtu = MTU
tun.up()
info("%s: up now." % tun.name)
# ----------- prepare info for IPC clients(part of each proxy process)
# register answer functions with ipc.onQuery(question, func), providing
# services for IPC client to get its necessary information
def ipcOnQueryInit(argv, answer):
global config, MODE
try:
proxyName = argv["name"]
proxyConfig = config.getProxyConfig(proxyName)
answer.title = 'init'
answer.uid = config.user[0]
answer.gid = config.user[1]
answer.config = proxyConfig
answer.key = config.key
answer.mode = MODE
except Exception,e:
exception(e)
error("We cannot answer an init query.")
return False
return True
ipc.onQuery('init', ipcOnQueryInit)
# ---------- initialize proxy processes
for proxyName in config.listProxies():
proxyCommand = config.getProxyInitParameters(proxyName, ipc, args.debug)
processes.new(proxyName, proxyCommand)
# ---------- drop root privileges
dropRoot(coreConfig.uid, coreConfig.gid)
##############################################################################
# register exit function, and start IO loop
reads = [ipc, tun] # for `select` function
def doExit(signum, frame):
global reads, processes
info("Exit now.")
# first close TUN devices
for each in reads: each.close()
# kill processes
t = 1.0 # second(s) waiting for exit
try:
processes.killall(t)
info("Exiting. Wait %f seconds for child processes to exit." % t)
except Exception,e:
error("Exiting, error: %s" % e)
info("Good bye.")
sys.exit()
signal.signal(signal.SIGTERM, doExit)
signal.signal(signal.SIGINT, doExit)
while True:
try:
# ---------- deal with I/O things
readables = select(reads, [], [], 5.0)[0]
for each in readables:
if each == tun:
# ---------- forward packets came from tun0
buf = each.read(65536) # each.read(each.mtu)
# pack buf with timestamp
packet = DataPacket()
packet.data = buf
# encrypt and sign buf
ipc.send(str(packet))
debug(showPacket(buf))
if each == ipc:
# ---------- receive packets from internet
buf = each.receive()
if buf == None:
# Received buffer being digested by InternalSocket itself,
# either some internal mechanism packet, or packet with
# wrong destination, or packet decryption failed...
continue
try:
packet = DataPacket(buf)
except DataPacketException, e:
# if failed reading the packet
debug("[%d] --> %s: Bad packet - %s" % (i, tun.name, e))
continue
# send buf to network interface
tun.write(packet.data)
debug(showPacket(packet.data))
# ---------- deal with tunnel delay timings
ipc.clean()
except KeyboardInterrupt:
doExit(None, None)
gitextract_v0a9sp1m/ ├── .gitignore ├── README.md ├── config.dev.json ├── doc/ │ ├── TODO.md │ └── cryptography.md ├── fyuneru/ │ ├── __init__.py │ ├── ipc/ │ │ ├── __init__.py │ │ ├── __protocol.py │ │ ├── client.py │ │ ├── server.py │ │ ├── tools.py │ │ └── url.py │ ├── net/ │ │ ├── __init__.py │ │ ├── protocol.py │ │ └── vnet.py │ └── util/ │ ├── __init__.py │ ├── __xsalsa20.py │ ├── config.py │ ├── crypto.py │ ├── debug.py │ ├── droproot.py │ ├── pidfile.py │ └── procmgr.py ├── proxy.icmp.py ├── proxy.shadowsocks.py ├── proxy.tcp.py ├── proxy.xmpp.py └── run_as.py
SYMBOL INDEX (149 symbols across 19 files)
FILE: fyuneru/ipc/__protocol.py
class WrongTypeOfPacketException (line 16) | class WrongTypeOfPacketException(Exception): pass
class DataPacket (line 22) | class DataPacket:
method __init__ (line 26) | def __init__(self, buf=None):
method __str__ (line 31) | def __str__(self):
class HeartbeatPacket (line 36) | class HeartbeatPacket:
method __init__ (line 38) | def __init__(self, buf=None):
method __str__ (line 42) | def __str__(self):
class QueryPacket (line 49) | class QueryPacket:
method __init__ (line 54) | def __init__(self, buf=None):
method __str__ (line 61) | def __str__(self):
class InfoPacket (line 67) | class InfoPacket:
method __init__ (line 69) | def __init__(self, buf=None):
method __setattr__ (line 74) | def __setattr__(self, name, value):
method __getattr__ (line 77) | def __getattr__(self, name):
method __str__ (line 80) | def __str__(self):
function loadBufferToPacket (line 85) | def loadBufferToPacket(buf):
FILE: fyuneru/ipc/client.py
class InternalSocketClient (line 24) | class InternalSocketClient:
method __init__ (line 38) | def __init__(self, serverURL):
method __getattr__ (line 46) | def __getattr__(self, name):
method __registerLastBeatSent (line 51) | def __registerLastBeatSent(self):
method __registerLastBeatRecv (line 54) | def __registerLastBeatRecv(self):
method __sendPacket (line 61) | def __sendPacket(self, packet):
method __recvBuffer (line 71) | def __recvBuffer(self, buf, sender):
method __handleHeartbeatPacket (line 94) | def __handleHeartbeatPacket(self, packet):
method __handleInfoPacket (line 99) | def __handleInfoPacket(self, packet):
method doQuery (line 104) | def doQuery(self, fillerFunc):
method onInfo (line 111) | def onInfo(self, handler):
method close (line 114) | def close(self):
method heartbeat (line 121) | def heartbeat(self):
method receive (line 139) | def receive(self):
method send (line 146) | def send(self, buf):
FILE: fyuneru/ipc/server.py
class InternalSocketServer (line 27) | class InternalSocketServer:
method __init__ (line 38) | def __init__(self, key):
method __getattr__ (line 48) | def __getattr__(self, name):
method __registerPeer (line 51) | def __registerPeer(self, addrTuple, timestamp=None):
method __choosePeer (line 68) | def __choosePeer(self):
method __sendPacket (line 77) | def __sendPacket(self, packet, to):
method __recvBuffer (line 82) | def __recvBuffer(self, buf, sender):
method __handleHeartbeatPacket (line 103) | def __handleHeartbeatPacket(self, packet, sender):
method __handleQueryPacket (line 109) | def __handleQueryPacket(self, packet, sender):
method onQuery (line 122) | def onQuery(self, question, answerfunc):
method close (line 125) | def close(self):
method clean (line 134) | def clean(self):
method receive (line 156) | def receive(self):
method send (line 178) | def send(self, buf):
FILE: fyuneru/ipc/tools.py
class InitConfigWaiter (line 7) | class InitConfigWaiter:
method __queryFiller (line 13) | def __queryFiller(self, packet):
method __infoReader (line 18) | def __infoReader(self, packet):
method __init__ (line 30) | def __init__(self, ipc):
method wait (line 34) | def wait(self):
FILE: fyuneru/ipc/url.py
class InvalidIPCServerURLException (line 11) | class InvalidIPCServerURLException(Exception): pass
class IPCServerURL (line 13) | class IPCServerURL:
method __init__ (line 20) | def __init__(self, url=None):
method __str__ (line 35) | def __str__(self):
FILE: fyuneru/net/protocol.py
class DataPacketException (line 16) | class DataPacketException(Exception):
class DataPacket (line 19) | class DataPacket:
method __unpack (line 23) | def __unpack(self, buf):
method __str__ (line 31) | def __str__(self):
method __init__ (line 37) | def __init__(self, buf=None):
FILE: fyuneru/net/vnet.py
class VirtualNetworkInterfaceException (line 15) | class VirtualNetworkInterfaceException(Exception): pass
class VirtualNetworkInterface (line 17) | class VirtualNetworkInterface:
method __getTUNDeviceLocation (line 22) | def __getTUNDeviceLocation(self):
method __init__ (line 28) | def __init__(self, ip, dstip, netmask="255.255.255.0"):
method up (line 47) | def up(self):
method fileno (line 64) | def fileno(self):
method write (line 67) | def write(self, buf):
method read (line 70) | def read(self, size=65536):
method close (line 73) | def close(self):
FILE: fyuneru/util/__xsalsa20.py
class XSalsa20 (line 25) | class XSalsa20:
method __salsa20Core (line 27) | def __salsa20Core(self, x, inbuf, oubuf):
method __streamXor (line 65) | def __streamXor(self, iv, key, buf):
method encrypt (line 111) | def encrypt(self, iv, key, buf):
method decrypt (line 114) | def decrypt(self, iv, key, buf):
FILE: fyuneru/util/config.py
class CoreConfiguration (line 27) | class CoreConfiguration: pass
class ProxyConfiguration (line 28) | class ProxyConfiguration: pass
class ConfigFileException (line 30) | class ConfigFileException(Exception): pass
class Configuration (line 34) | class Configuration:
method __checkKeyExistence (line 36) | def __checkKeyExistence(self, parentDict, *keys):
method __loadCore (line 42) | def __loadCore(self, core):
method __loadProxyAllocations (line 65) | def __loadProxyAllocations(self, proxies):
method listProxies (line 73) | def listProxies(self):
method getProxyConfig (line 76) | def getProxyConfig(self, name):
method getProxyInitParameters (line 82) | def getProxyInitParameters(self, proxyName, IPCServer, debug=False):
method getCoreInitParameters (line 97) | def getCoreInitParameters(self, mode):
method __init__ (line 115) | def __init__(self, config):
FILE: fyuneru/util/crypto.py
class CryptoException (line 66) | class CryptoException(Exception): pass
function randint (line 68) | def randint(a, b):
function randrange (line 72) | def randrange(a, b):
function decidePaddingLength (line 76) | def decidePaddingLength(bufferLength):
function KDF1 (line 88) | def KDF1(secret, length):
class Authenticator (line 102) | class Authenticator:
method __init__ (line 111) | def __init__(self, key):
method __HMAC (line 114) | def __HMAC(self, buf):
method sign (line 119) | def sign(self, buf):
method verify (line 123) | def verify(self, buf):
class Crypto (line 133) | class Crypto:
method __init__ (line 136) | def __init__(self, passphrase):
method __HMAC (line 141) | def __HMAC(self, buf):
method encrypt (line 146) | def encrypt(self, buf):
method decrypt (line 161) | def decrypt(self, buf):
FILE: fyuneru/util/debug.py
function configLoggingModule (line 9) | def configLoggingModule(debug):
function colorify (line 26) | def colorify(text, color):
function _decodeIPFrame (line 39) | def _decodeIPFrame(buf):
function showPacket (line 70) | def showPacket(buf):
FILE: fyuneru/util/droproot.py
function dropRoot (line 16) | def dropRoot(uid_name='nobody', gid_name='nobody'):
FILE: fyuneru/util/pidfile.py
class ProcessRunningException (line 8) | class ProcessRunningException(Exception): pass
class PidfileNonExistentException (line 9) | class PidfileNonExistentException(Exception): pass
class PidfileWatcher (line 11) | class PidfileWatcher:
method __init__ (line 21) | def __init__(self, path):
method check (line 31) | def check(self):
class PidfileCreator (line 41) | class PidfileCreator:
method __init__ (line 42) | def __init__(self, path, log=sys.stdout.write, warn=sys.stderr.write):
method __enter__ (line 47) | def __enter__(self):
method __exit__ (line 68) | def __exit__(self, t, e, tb):
method _remove (line 84) | def _remove(self):
method _check (line 88) | def _check(self):
FILE: fyuneru/util/procmgr.py
class ProcessManagerException (line 9) | class ProcessManagerException(Exception): pass
class ProcessManager (line 11) | class ProcessManager:
method __init__ (line 16) | def __init__(self):
method __pollAll (line 19) | def __pollAll(self):
method __startProcess (line 26) | def __startProcess(self, name, command):
method new (line 32) | def new(self, name, command):
method killall (line 38) | def killall(self, tolerance=1.0):
method kill (line 42) | def kill(self, name, tolerance=1.0):
method restart (line 67) | def restart(self, name, tolerance=1.0):
method wait (line 74) | def wait(self, name):
class ParentProcessWatcher (line 78) | class ParentProcessWatcher:
method __init__ (line 80) | def __init__(self, pid, terminator):
method watch (line 85) | def watch(self):
FILE: proxy.icmp.py
function log (line 24) | def log(x):
function doExit (line 55) | def doExit(signum, frame):
FILE: proxy.shadowsocks.py
function doExit (line 118) | def doExit(signum, frame):
FILE: proxy.tcp.py
class Datagram2Stream (line 53) | class Datagram2Stream:
method put (line 57) | def put(self, datagram):
method get (line 60) | def get(self, size=32768):
class Stream2Datagram (line 65) | class Stream2Datagram:
method put (line 69) | def put(self, buf):
method get (line 72) | def get(self):
function doExit (line 97) | def doExit(signum, frame):
FILE: proxy.xmpp.py
class SocketXMPPProxyException (line 62) | class SocketXMPPProxyException(Exception): pass
class SocketXMPPProxy (line 64) | class SocketXMPPProxy:
method __init__ (line 66) | def __init__(self, jid, password, peer):
method __registerHandlers (line 89) | def __registerHandlers(self):
method __sendPresence (line 92) | def __sendPresence(self):
method message (line 98) | def message(self, con, event):
method send (line 109) | def send(self, buf):
function doExit (line 133) | def doExit(signum, frame):
FILE: run_as.py
function ipcOnQueryInit (line 87) | def ipcOnQueryInit(argv, answer):
function doExit (line 122) | def doExit(signum, frame):
Condensed preview — 28 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (96K chars).
[
{
"path": ".gitignore",
"chars": 55,
"preview": "*.sw?\n*.pyc\ntest.*\nconfig*.json\n!config.dev.json\n*.odg\n"
},
{
"path": "README.md",
"chars": 10904,
"preview": "Fyuneru v1.1\n============\n\n\n**向下拉动，见简体中文版本。**\n\n**Fyuneru** lets you set up a server and a client computer within a virtu"
},
{
"path": "config.dev.json",
"chars": 1086,
"preview": "{\n \"version\": \"1.0\",\n \"core\": {\n \"server\": {\n \"ip\": \"10.1.0.1\"\n },\n \"client\": {\n "
},
{
"path": "doc/TODO.md",
"chars": 1284,
"preview": "任务列表\n========\n\n## v1.2(未确定)\n\n[ ] 1. 在客户端上设立控制端口，以便开发web界面，查看UDP流量、延迟，管理开放情况\n\n[ ] 2. 考虑一种控制协议，可以通过上一条所述方式控制服务器端"
},
{
"path": "doc/cryptography.md",
"chars": 3518,
"preview": "On the Cryptography in Fyuneru\n==============================\n\nThis article will discuss how cryptography is being used "
},
{
"path": "fyuneru/__init__.py",
"chars": 127,
"preview": "# -*- coding: utf-8 -*-\n\n__all__ = [\\\n \"config\",\n \"crypto\",\n \"debug\",\n \"droproot\",\n \"procmgr\",\n \"proto"
},
{
"path": "fyuneru/ipc/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "fyuneru/ipc/__protocol.py",
"chars": 3069,
"preview": "\"\"\"\nIPC used data packets generation and parsing\n============================================\n\nThis module defines sever"
},
{
"path": "fyuneru/ipc/client.py",
"chars": 5021,
"preview": "\n# -*- coding: utf-8 -*-\n\n\"\"\"\nInternal Fyuneru Socket for Proxy Processes\n\nA fyuneru socket basing on UDP socket is defi"
},
{
"path": "fyuneru/ipc/server.py",
"chars": 7048,
"preview": "# -*- coding: utf-8 -*-\n\n\"\"\"\nInternal Fyuneru Socket for Proxy Processes\n\nA fyuneru socket basing on UDP socket is defin"
},
{
"path": "fyuneru/ipc/tools.py",
"chars": 1309,
"preview": "\"\"\"\nTool functions for IPC\n\"\"\"\nfrom select import select\nfrom logging import info, debug, exception, error\n\nclass InitCo"
},
{
"path": "fyuneru/ipc/url.py",
"chars": 1299,
"preview": "\"\"\"\nURL for starting IPC client\n===========================\n\nThis tells the IPC client how to find the IPC server, do au"
},
{
"path": "fyuneru/net/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "fyuneru/net/protocol.py",
"chars": 1227,
"preview": "# -*- coding: utf-8 -*-\n\n\"\"\"\nPackets of Fyuneru\n\n\"\"\"\n\nfrom struct import pack, unpack\n\nSIGN_DATAPACKET = 0x01\n\n#########"
},
{
"path": "fyuneru/net/vnet.py",
"chars": 2406,
"preview": "# -*- coding: utf-8 -*-\n\nimport os\nimport fcntl\nimport struct\nfrom logging import info, debug, critical, exception\nfrom "
},
{
"path": "fyuneru/util/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "fyuneru/util/__xsalsa20.py",
"chars": 5057,
"preview": "\"\"\"\nModified XSalsa20/16 Cipher Implementation in Pure Python\n======================================================\n\nTh"
},
{
"path": "fyuneru/util/config.py",
"chars": 5089,
"preview": "# -*- coding: utf-8 -*-\n\n\"\"\"\nConfig reader and manager for the core program\n\nClass Configuration provides the functional"
},
{
"path": "fyuneru/util/crypto.py",
"chars": 6145,
"preview": "\"\"\"\nCryptographical Services\n========================\n\n## Authenticator in IPC communication\n\nIn IPC communications, dat"
},
{
"path": "fyuneru/util/debug.py",
"chars": 2814,
"preview": "import socket\nfrom struct import * \nimport logging\nimport sys\nimport os\n\n###############################################"
},
{
"path": "fyuneru/util/droproot.py",
"chars": 911,
"preview": "\"\"\"\nFunction to drop root privileges\n<http://stackoverflow.com/questions/2699907/dropping-root-permissions-in-python>\n\nT"
},
{
"path": "fyuneru/util/pidfile.py",
"chars": 3319,
"preview": "# -*- coding: utf-8 -*-\n\nimport os\nimport time\n\n\n\nclass ProcessRunningException(Exception): pass\nclass PidfileNonExisten"
},
{
"path": "fyuneru/util/procmgr.py",
"chars": 3126,
"preview": "# -*- coding: utf-8 -*-\n\nimport signal\nimport subprocess\nimport time\nfrom logging import debug, info, warning, error\n\n\nc"
},
{
"path": "proxy.icmp.py",
"chars": 2569,
"preview": "\n\"\"\"\nICMP Proxy Process\n==================\n\nThis is a very simple proxy that puts our encrypted payloads into ICMP packe"
},
{
"path": "proxy.shadowsocks.py",
"chars": 5047,
"preview": "# -*- coding: utf-8 -*-\n\nimport argparse\nimport os\nimport sys\nfrom select import select\nimport socket\nimport signal\nimpo"
},
{
"path": "proxy.tcp.py",
"chars": 3702,
"preview": "# -*- coding: utf-8 -*-\n\n\"\"\"\nUDP over TCP proxy for Fyuneru.\n\"\"\"\n\nimport argparse\nimport os\nfrom select import select\nim"
},
{
"path": "proxy.xmpp.py",
"chars": 4978,
"preview": "\"\"\"\nXMPP Proxy Process\n==================\n\nThis proxy utilizes the `xmpppy` library. You have to first install it before"
},
{
"path": "run_as.py",
"chars": 5275,
"preview": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\nimport argparse\nimport os\nfrom select import select\nimport signal\nimport "
}
]
About this extraction
This page contains the full source code of the sogisha/fyuneru GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 28 files (84.4 KB), approximately 22.5k tokens, and a symbol index with 149 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.