Repository: robsco-git/spreadsheet_server
Branch: master
Commit: 8f37b92fe277
Files: 27
Total size: 87.0 KB
Directory structure:
gitextract_sjf5n4b7/
├── .coveragerc
├── .dockerignore
├── .editorconfig
├── .flake8
├── .gitignore
├── COPYING
├── Dockerfile
├── README.md
├── client.py
├── connection.py
├── coverage.sh
├── docker_run.sh
├── example_client.py
├── log/
│ └── .gitignore
├── monitor.py
├── pyproject.toml
├── request_handler.py
├── requirements.txt
├── saved_spreadsheets/
│ └── .gitignore
├── server.py
├── spreadsheets/
│ └── .gitignore
└── tests/
├── __init__.py
├── context.py
├── example.ods
├── test_client.py
├── test_connection.py
└── test_monitor.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .coveragerc
================================================
[report]
omit =
*/venv/*
/usr/lib/libreoffice/program/uno.py
================================================
FILE: .dockerignore
================================================
.*
venv
log
saved_spreadsheets
spreadsheets
================================================
FILE: .editorconfig
================================================
# EditorConfig is awesome: http://EditorConfig.org
# top-most EditorConfig file
root = true
# Unix-style newlines with a newline ending every file
[*]
end_of_line = lf
insert_final_newline = true
# Matches multiple files with brace expansion notation
# Set default charset
[*.{js,py}]
charset = utf-8
# 4 space indentation
[*.py]
indent_style = space
indent_size = 4
# Tab indentation (no size specified)
[Makefile]
indent_style = tab
# Indentation override for all JS under lib directory
[*.{js,ts,tsx,scss}]
indent_style = space
indent_size = 2
# Matches the exact files either package.json or .travis.yml
[{package.json,.travis.yml}]
indent_style = space
indent_size = 2
================================================
FILE: .flake8
================================================
# This is an example .flake8 config, used when developing *Black* itself.
# Keep in sync with setup.cfg which is used for source packages.
[flake8]
ignore = E203, E266, E501, W503, F403, F401, C901, E711, E712, E402
max-line-length = 79
max-complexity = 18
select = B,C,E,F,W,T4,B9
================================================
FILE: .gitignore
================================================
venv
.#*
\#*
*~
__pycache__
.~lock*
spreadsheets/*
log/*.log
*.pyc
saved_spreadsheets/*
.coverage
htmlcov
.vscode
/todo.org
================================================
FILE: COPYING
================================================
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Lesser General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License.
================================================
FILE: Dockerfile
================================================
FROM debian:sid-slim
RUN apt-get update && \
apt-get upgrade -y && \
apt-get install -y --no-install-recommends wget python3 python3-dev python3-pip python3-virtualenv python3-uno libreoffice-calc && \
rm -rf /var/cache/apt/archives /var/lib/apt/lists/*
COPY * /spreadsheet_server/
WORKDIR /spreadsheet_server
RUN mkdir -p /spreadsheet_server/log
RUN virtualenv --system-site-packages -p python3 /opt/virtualenv
ENV PATH="/opt/virtualenv/bin:$PATH"
RUN pip3 install --no-cache-dir -r requirements.txt
CMD ["python3", "-c", "from server import SpreadsheetServer; spreadsheet_server = SpreadsheetServer(host='0.0.0.0'); spreadsheet_server.run()"]
================================================
FILE: README.md
================================================
# spreadsheet_server
## Introduction
spreadsheet_server was built to aid rapid web tool development where the logic
was already implemented in Microsoft Excel/LibreOffice Calc. Instead of
rewriting the logic from scratch, this tool was born.
The tool has been developed to work on a headless GNU/Linux where the server
and client are on the same machine.
## Features
- 'Instant' access to cells in the spreadsheets as they open in LibreOffice Calc.
- All the function calculation support and power of LibreOffice Calc.
- A given spreadsheet is locked (within python, not on disk) when it is accessed to prevent state irregularities across multiple concurrent connections to the same spreadsheet.
- Monitoring of a directory with automatic loading and unloading of spreadsheets.
- By default, when a spreadsheet file changes on disk, it will be closed and
opened in LibreOffice.
- Spreadsheets can be saved - useful for debugging purposes.
## Installation
### Docker
```
docker build . --tag="spreadsheet_server"
docker run -v ./spreadsheets:/spreadsheet_server/spreadsheets -p 127.0.0.1:5555:5555 --name spreadsheet_server spreadsheet_server
```
### Ubuntu Server 20.04
```
sudo apt-get update && sudo apt-get upgrade
sudo apt-get install git gcc python3 python3-dev python3-virtualenv libreoffice-calc python3-uno
git clone https://github.com/robsco-git/spreadsheet_server.git
cd spreadsheet_server
mkdir -p ~/.virtualenvs
virtualenv --system-site-packages -p python3 ~/.virtualenvs/spreadsheet_server
source ~/.virtualenvs/spreadsheet_server/bin/activate
pip install -r requirements.txt
python server.py
# Copy spreadsheets (.xlsx, .ods etc) into ./spreadsheets
```
## Usage
Place your spreadsheets in './spreadsheets'.
Make sure you have a correctly set up virtualenv the required packages are installed
(see below), then run the server:
```
python server.py
```
You can also create a SpreadsheetServer object (from server.py) and then call
the method 'run' on the object to start the server. This allows the for
customisation of the default settings. Have a look in server.py to see what can
be set.
example_client.py is provided for an overview of how to use the functions
exposed by the client. This is a good place to start.
## How it works
- A LibreOffice instance is launched by 'server.py' in a headless state.
- By default, the './spreadsheets' directory is polled every 5 seconds for file
changes.
- New spreadsheets in the directory are opened with LibreOffice and removed
spreadsheets are closed in LibreOffice.
- The 'client.py' connects to the server and can update cells and retrieve
their calculated content.
## Questions
What is UNO? What is Python-UNO? What is PyOO?
The first few paragraphs of the PyOO README should answer most of your questions:
https://github.com/seznam/pyoo/blob/master/README.rst
## Notes
### Symbolic links and lock files
If you symbolically link a spreadsheet itself, the lock files that LibreOffice
creates and uses are stored in the directory where the file is located, not in the
directory where the symbolic link is located. It is recommended that you place your
spreadsheet(s) into a directory and symbolically link that directory into the
'./spreadsheets' directory. This way, LibreOffice will always be able to locate the
lock files it needs. You can use a directory per project if you like.
## Tests
You can run the all the current tests with 'python -m unittest discover'.
Use './coverage.sh' to run the coverage analysis of the current tests and have a look
in the generated htmlcov directory. You will need the 'coverage' installed to the
virtualenv:
```
pip install coverage
```
================================================
FILE: client.py
================================================
# Copyright (C) 2016 Robert Scott
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
import socket
import json
import traceback
import struct
import select
IP, PORT = "localhost", 5555
TIMEOUT = 10
class SpreadsheetClient:
def __init__(self, spreadsheet, ip=IP, port=PORT):
try:
self.sock = self.__connect(ip, port)
except socket.error:
raise RuntimeError("Could not connect to the server.")
else:
self.__set_spreadsheet(spreadsheet)
def __connect(self, ip, port):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip, port))
return sock
def __set_spreadsheet(self, spreadsheet):
self.__send(["SPREADSHEET", spreadsheet])
received = self.__receive()
if received == "NOT FOUND" or received != "OK":
self.disconnect()
raise RuntimeError("The requested spreadsheet was not found.")
def set_cells(self, sheet, cell_ref, data):
"""Set the value(s) for a single cell or a cell range.
'sheet' is either a 0-based index or the string name of the sheet.
'cell_ref' is a LibreOffice style cell reference. eg. "A1" or "D7:G42".
For a single cell, 'data' is a single string, int or float value.
For a one dimensional (only horizontal or only vertical) range of
cells, 'data' is a list. For a two dimensional range of cells, 'data'
is a list of lists. For example setting the 'cell_ref' "A1:C3"
requires 'data' of the format:
[[A1, B1, C1], [A2, B2, C2], [A3, B3, C3]].
"""
self.__send(["SET", sheet, cell_ref, data])
received = self.__receive()
if type(received) == dict:
# The server is retuning an error
raise RuntimeError(received["ERROR"])
def get_sheet_names(self):
"""Returns a list of all sheet names in the workbook."""
self.__send(["GET_SHEETS"])
sheet_names = self.__receive()
if sheet_names == "ERROR":
raise Exception("Could not retrieve sheet names.")
return sheet_names
def get_cells(self, sheet, cell_ref):
"""Get the value of a single cell or a cell range from the server
and return it or them.
'sheet' is either a 0-based index or the string name of the sheet.
'cell_ref' is what one would use in LibreOffice Calc. Eg. "ABC945" or
"A3:F75".
A single cell is returned for a single value.
A list of cell values is returned for a one dimensional range of cells.
A list of lists is returned for a two dimensional range of cells.
"""
self.__send(["GET", sheet, cell_ref])
cells = self.__receive()
if type(cells) == dict:
# The server is retuning an error
raise RuntimeError(cells["ERROR"])
return cells
def save_spreadsheet(self, filename):
"""Save the spreadsheet in its current state on the server. The
server determines where it is saved."""
self.__send(["SAVE", filename])
return self.__receive()
def __send(self, msg):
"""Encode msg into json and then send it over the socket."""
json_msg = json.dumps(msg)
json_msg = bytes(json_msg, "utf-8")
# Prepend the length of the string to the meg
json_msg = struct.pack(">I", len(json_msg)) + json_msg
try:
self.sock.sendall(json_msg)
except: # noqa
traceback.print_exc()
raise Exception("Could not send message to server")
def __receive(self):
"""Receive a message from the client, convert the received utf-8
bytes into a string then decode if from json."""
raw_msg_length = self.__receive_length(4)
if not raw_msg_length:
return False
msg_length = struct.unpack(">I", raw_msg_length)[0]
recv = self.__receive_length(msg_length)
if recv == b"":
# The connection has been closed.
raise Exception("Connection to server closed!")
received = str(recv, encoding="utf-8")
received = json.loads(received)
return received
def __receive_length(self, length):
"""Receive length number of bytes from the client."""
data = b""
while len(data) < length:
ready = select.select([self.sock], [], [], TIMEOUT)
if ready[0]:
packet = self.sock.recv(length - len(data))
if not packet:
return b""
data += packet
else:
# Did not recieve on the socket within the timeout
return b""
return data
def disconnect(self):
"""Disconnect from the server."""
try:
self.sock.shutdown(socket.SHUT_RDWR)
except socket.error:
# The client has already disconnected
pass
self.sock.close()
================================================
FILE: connection.py
================================================
# Copyright (C) 2016 Robert Scott
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
# USA.
import logging
from math import pow
from werkzeug.utils import secure_filename
from threading import ThreadError
import os
CELL_REF_ERROR_STR = "Cell range is invalid."
class SpreadsheetConnection:
"""Handles connections to the spreadsheets opened by soffice (LibreOffice).
"""
def __init__(self, spreadsheet, lock, save_path):
self.spreadsheet = spreadsheet
self.lock = lock
self.save_path = save_path
def lock_spreadsheet(self):
"""Lock the spreadsheet.
The getting and setting cell functions rely on a given spreadsheet
being locked. This insures simulations requests to the same spreadsheet
do not interfere with one another.
"""
self.lock.acquire()
def unlock_spreadsheet(self):
""" Unlock the spreadsheet and return a 'success' boolean."""
try:
self.lock.release()
return True
except (RuntimeError, ThreadError):
return False
def __get_xy_index(self, cell_ref):
chars = [c for c in cell_ref if c.isalpha()]
nums = [n for n in cell_ref if n.isdigit()]
alpha_index = 0
for i, c in enumerate(chars):
# The base index for this character is in the range of 0 to 25
c_index = ord(c.upper()) - 65
if i == len(chars) - 1:
# The simple case of the least significant character.
# Eg. The 'J' in 'AMJ'.
alpha_index += c_index
else:
# The index for additional characters to the left are
# calculated c_index * 26^(n) where n is the characters
# position to the left.
# Need to increment c_index for correct multiplication
c_index += 1
alpha_index += c_index * pow(26, len(chars) - i - 1)
num_index = int("".join(nums)) - 1 # zero-based
alpha_index = int(alpha_index)
# Check max values
# Column can not be > AMJ == 1023
if alpha_index >= 1024:
raise ValueError(CELL_REF_ERROR_STR)
# Row can not be > 1048576
if num_index >= 1048576:
raise ValueError(CELL_REF_ERROR_STR)
return alpha_index, num_index
def __is_single_cell(self, cell_ref):
if len(cell_ref.split(":")) == 1:
return True
return False
def __check_single_cell(self, cell_ref):
if not self.__is_single_cell(cell_ref):
raise ValueError(
"Expected a single cell reference. A cell range was given."
)
def __cell_to_index(self, cell_ref):
"""Convert a spreadsheet style single cell or cell reference, to a zero
based numerical index.
'cell_ref' is what one would use in LibreOffice Calc. Eg. "ABC945".
Returned is: {"row_index": int, "column_index": int}.
"""
alpha_index, num_index = self.__get_xy_index(cell_ref)
return {"row_index": num_index, "column_index": alpha_index}
def __cell_range_to_index(self, cell_ref):
"""Convert a spreadsheet style range reference to zero-based numerical
indecies that describe the start and end points of the cell range.
'cell_ref' is what one would use in LibreOffice Calc. Eg. "A1" or
"A1:D6".
Returned is: {"row_start": int, "row_end": int,
"column_start": int, "column_end": int}
"""
left_ref, right_ref = cell_ref.split(":")
left_alpha_index, left_num_index = self.__get_xy_index(left_ref)
right_alpha_index, right_num_index = self.__get_xy_index(right_ref)
return {
"row_start": left_num_index,
"row_end": right_num_index,
"column_start": left_alpha_index,
"column_end": right_alpha_index,
}
def __check_for_lock(self):
if not self.lock.locked():
raise RuntimeError(
"Lock for this spreadsheet has not been aquired."
)
def __convert_to_float_if_numeric(self, value):
"""If value is a string representation of a number, convert it to a
float. Otherwise, simply return the string.
"""
try:
return float(value)
except (ValueError, TypeError):
return value
def __check_list(self, data):
if not isinstance(data, list):
raise ValueError("Expecting list type.")
def __check_1D_list(self, data):
self.__check_list(data)
if isinstance(data[0], list):
raise ValueError("Got 2D list when expecting 1D list.")
for x, cell in enumerate(data):
data[x] = self.__convert_to_float_if_numeric(cell)
return data
def set_cells(self, sheet, cell_ref, value):
"""Set the value(s) for a single cell or a cell range. This can be used
when it is not known if 'cell_ref' refers to a single cell or a range
See 'set_cell' and 'set_cell_range' for more information.
"""
self.__validate_sheet_name(sheet)
self.__validate_cell_ref(cell_ref)
if self.__is_single_cell(cell_ref):
self.set_cell(sheet, cell_ref, value)
else:
self.set_cell_range(sheet, cell_ref, value)
def set_cell(self, sheet, cell_ref, value):
"""Set the value of a single cell.
'sheet' is either a 0-based index or the string name of the sheet.
'cell_ref' is a LibreOffice style cell reference. eg. "A1".
'value' is a single string, int or float value.
"""
self.__check_single_cell(cell_ref)
self.__check_for_lock()
r = self.__cell_to_index(cell_ref)
sheet = self.spreadsheet.sheets[sheet]
if isinstance(value, list):
raise ValueError(
"Expectin a single cell. \
A list of cells was given."
)
value = self.__convert_to_float_if_numeric(value)
sheet[r["row_index"], r["column_index"]].value = value
def set_cell_range(self, sheet, cell_ref, data):
"""Set the values for a cell range.
'sheet' is either a 0-based index or the string name of the sheet.
'cell_ref' is a LibreOffice style cell reference. eg. "D7:G42".
For a one dimensional (only horizontal or only vertical) range of
cells, 'data' is a list. For a two dimensional range of cells, 'data'
is a list of lists. For example setting the 'cell_ref' "A1:C3"
requires 'data' of the format:
[[A1, B1, C1], [A2, B2, C2], [A3, B3, C3]].
"""
self.__check_for_lock()
r = self.__cell_range_to_index(cell_ref)
sheet = self.spreadsheet.sheets[sheet]
if r["row_start"] == r["row_end"]: # A row of cells
data = self.__check_1D_list(data)
sheet[
r["row_start"], r["column_start"] : r["column_end"] + 1
].values = data
elif r["column_start"] == r["column_end"]: # A column of cells
data = self.__check_1D_list(data)
sheet[
r["row_start"] : r["row_end"] + 1, r["column_start"]
].values = data
else: # A grid of cells
self.__check_list(data)
for x, row in enumerate(data):
if not isinstance(row, list):
raise ValueError("Expected a list of cells.")
for y, cell in enumerate(row):
data[x][y] = self.__convert_to_float_if_numeric(cell)
sheet[
r["row_start"] : r["row_end"] + 1,
r["column_start"] : r["column_end"] + 1,
].values = data
def get_sheet_names(self):
"""Returns a list of all sheet names in the workbook."""
return [s.name for s in self.spreadsheet.sheets]
def __validate_cell_ref(self, cell_ref):
""" A cell ref must be of the LibreOffice format
e.g. A1 or A1:ABC123."""
if type(cell_ref) is not str:
raise ValueError(CELL_REF_ERROR_STR)
if not cell_ref[0].isalpha():
raise ValueError(CELL_REF_ERROR_STR)
if not cell_ref[-1].isdigit():
raise ValueError(CELL_REF_ERROR_STR)
if ":" in cell_ref:
# Check the second alpha if it exists
if not cell_ref[cell_ref.index(":") + 1].isalpha():
raise ValueError(CELL_REF_ERROR_STR)
# Check the start of the range has a numeric component
if not cell_ref[cell_ref.index(":") - 1].isdigit():
raise ValueError(CELL_REF_ERROR_STR)
# Check for any unallowed characters
for ref in cell_ref:
if not ref.isdigit() and not ref.isalpha() and ref != ":":
raise ValueError(CELL_REF_ERROR_STR)
# TODO - Check range for sanity
# Reversed ranges should be allowed, they just need to be flipped
# e.g. "A5:A1" must become "A1:A5"
# Also need to convert "A1:A1" to "A1"
def __validate_sheet_name(self, sheet):
"""Don't want to send an invalid sheet to pyoo."""
ERROR_STR = "Sheet name is invalid."
sheet_names = self.get_sheet_names()
if type(sheet) is int:
if sheet < 0 or sheet > len(sheet_names) - 1:
raise ValueError(ERROR_STR)
elif type(sheet) is str:
if sheet not in sheet_names:
raise ValueError(ERROR_STR)
else:
raise ValueError(ERROR_STR)
def get_cells(self, sheet, cell_ref):
"""Gets the value(s) of a single cell or a cell range. This can be used
when it is not known if 'cell_ref' refers to a single cell or a range.
See 'get_cell' and 'get_cell_range' for more information.
"""
self.__validate_sheet_name(sheet)
self.__validate_cell_ref(cell_ref)
if self.__is_single_cell(cell_ref):
return self.get_cell(sheet, cell_ref)
else:
return self.get_cell_range(sheet, cell_ref)
def get_cell(self, sheet, cell_ref):
"""Returns the value of a single cell.
'sheet' is either a 0-based index or the string name of the sheet.
'cell_ref' is what one would use in LibreOffice Calc. Eg. "A3".
A single cell value is returned.
"""
self.__check_single_cell(cell_ref)
r = self.__cell_to_index(cell_ref)
sheet = self.spreadsheet.sheets[sheet]
return sheet[r["row_index"], r["column_index"]].value
def get_cell_range(self, sheet, cell_ref):
"""Returns the values of a range of cells.
'sheet' is either a 0-based index or the string name of the sheet.
'cell_ref' is what one would use in LibreOffice Calc. Eg. "A3:F75".
A list of cell values is returned for a one dimensional range of cells.
A list of lists is returned for a two dimensional range of cells.
"""
r = self.__cell_range_to_index(cell_ref)
sheet = self.spreadsheet.sheets[sheet]
logging.debug("Requested cell area: " + str(r))
# Cell ranges are requested as: [vertical area, horizontal area]
if r["row_start"] == r["row_end"]: # A row of cells was requested
return sheet[
r["row_start"], r["column_start"] : r["column_end"] + 1
].values
elif r["column_start"] == r["column_end"]: # A column of cells
return sheet[
r["row_start"] : r["row_end"] + 1, r["column_start"]
].values
else: # A grid of cells
return sheet[
r["row_start"] : r["row_end"] + 1,
r["column_start"] : r["column_end"] + 1,
].values
def save_spreadsheet(self, filename):
"""Save the spreadsheet in it's current state.
'filename' is the name of the file.
"""
if self.lock.locked():
filename = secure_filename(filename)
self.spreadsheet.save(os.path.join(self.save_path, filename))
return True
else:
return False
================================================
FILE: coverage.sh
================================================
#!/bin/bash
# Run the coverage tests and genereate the html reports
#coverage run -m tests.test_connection && coverage html
coverage run -m unittest discover && coverage html
================================================
FILE: docker_run.sh
================================================
#!/bin/bash
sudo docker build . -t spreadsheet_server
sudo docker stop spreadsheet_server
sudo docker rm spreadsheet_server
sudo docker run -d \
-v /home/robsco/code/spreadsheet_server/spreadsheets:/spreadsheet_server/spreadsheets \
-v /home/robsco/code/spreadsheet_server/saved_spreadsheets:/spreadsheet_server/saved_spreadsheets \
--restart always \
-v /home/robsco/code/spreadsheet_server/log:/spreadsheet_server/log \
-p 5555:5555 \
--name spreadsheet_server \
spreadsheet_server
================================================
FILE: example_client.py
================================================
# Copyright (C) 2016 Robert Scott
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
import shutil
import os
from client import SpreadsheetClient
if __name__ == "__main__":
"""This script shows how the differnet functions exposed by client.py can be
used."""
EXAMPLE_SPREADSHEET = "example.ods"
# Copy the example spreadsheet from the tests directory into the spreadsheets
# directory
shutil.copyfile(
os.path.join("tests", EXAMPLE_SPREADSHEET),
os.path.join("spreadsheets", EXAMPLE_SPREADSHEET),
)
SHEET_NAME = "Sheet1"
print(
"Waiting for the example spreadsheet to be scanned and loaded into LibreOffice."
)
sc = SpreadsheetClient(EXAMPLE_SPREADSHEET)
# Get sheet names
sheet_names = sc.get_sheet_names()
print(sheet_names)
# Set a cell value
sc.set_cells(SHEET_NAME, "A1", 5)
# Retrieve a cell value.
cell_value = sc.get_cells(SHEET_NAME, "C3")
print(cell_value)
# Set a one dimensional cell range.
# Cells are set using the format: [A1, A2, A3]
cell_values = [1, 2, 3]
sc.set_cells(SHEET_NAME, "A1:A3", cell_values)
# Retrieve one dimensional cell range.
cell_values = sc.get_cells(SHEET_NAME, "C1:C3")
print(cell_values)
# Set a two dimensional cell range.
# Cells are set using the format: [[A1, B1, C1], [A2, B2, C2], [A3, B3, C3]]
cell_values = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
sc.set_cells(SHEET_NAME, "A1:C3", cell_values)
# Retrieve a two dimensional cell range.
cell_values = sc.get_cells(SHEET_NAME, "A1:C3")
print(cell_values)
# Save a spreadsheet - it will save into ./saved_spreadsheets
sc.save_spreadsheet(EXAMPLE_SPREADSHEET)
sc.disconnect()
os.remove(os.path.join("spreadsheets", EXAMPLE_SPREADSHEET))
================================================
FILE: log/.gitignore
================================================
# Ignore everything in this directory
*
# Except this file
!.gitignore
================================================
FILE: monitor.py
================================================
# Copyright (C) 2016 Robert Scott
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
import threading
from os import listdir, remove
from os.path import isfile, isdir, join, exists
import logging
from time import sleep
import hashlib
from glob import glob
class MonitorThread(threading.Thread):
"""Monitors the spreadsheet directory for changes."""
def __init__(
self,
spreadsheets,
locks,
hashes,
soffice,
spreadsheets_path,
monitor_frequency,
reload_on_disk_change,
):
self._stop_thread = threading.Event()
self.spreadsheets = spreadsheets
self.locks = locks
self.hashes = hashes
self.soffice = soffice
self.spreadsheets_path = spreadsheets_path
self.monitor_frequency = monitor_frequency
self.reload_on_disk_change = reload_on_disk_change
self.__delete_lock_files()
self.done_scan = False # Done an initial scan or not
super(MonitorThread, self).__init__()
def stop_thread(self):
self._stop_thread.set()
def stopped(self):
return self._stop_thread.isSet()
def initial_scan(self):
return self.done_scan
def __get_full_path(self, doc):
return join(self.spreadsheets_path, doc)
def __delete_lock_files(self):
"""Lock files can cause issues opening documents."""
lock_files = glob(
join(self.spreadsheets_path, ".~lock.*#"), recursive=True
)
for lock_file in lock_files:
remove(lock_file)
def __load_spreadsheet(self, doc):
logging.info("Loading " + doc["path"])
self.spreadsheets[doc["path"]] = self.soffice.open_spreadsheet(
self.__get_full_path(doc["path"])
)
self.locks[doc["path"]] = threading.Lock()
self.hashes[doc["path"]] = doc["hash"]
def __unload_spreadsheet(self, doc_path):
logging.info("Removing " + doc_path)
self.locks[doc_path].acquire()
self.spreadsheets[doc_path].close()
self.spreadsheets.pop(doc_path, None)
self.locks.pop(doc_path, None)
self.hashes.pop(doc_path, None)
def __check_added(self):
"""Check for new spreadsheets and loads them into LibreOffice."""
for doc in self.docs:
if doc["path"][0] != ".": # Ignore hidden files
load = True # Default to loading the spreadsheet
for key, value in self.spreadsheets.items():
if doc["path"] == key:
# Check if the file has been modified
# Does the file now have a differnet hash?
if (
self.reload_on_disk_change
and doc["hash"] != self.hashes[doc["path"]]
):
self.__unload_spreadsheet(doc["path"])
else:
load = False
break
if load:
self.__load_spreadsheet(doc)
def __check_removed(self):
"""Check for any deleted or removed spreadsheets and remove them from
LibreOffice.
"""
removed_spreadsheets = []
for key, value in self.spreadsheets.items():
removed = True
for doc in self.docs:
if key == doc["path"]:
removed = False
break
if removed:
removed_spreadsheets.append(key)
for doc_path in removed_spreadsheets:
self.__unload_spreadsheet(doc_path)
def __scan_directory(self, d):
"""Recursively scan a directory for spreadsheets."""
dir_contents = listdir(d)
for f in dir_contents:
# Ignore particular files
if f[:7] == ".~lock." or f == ".gitignore":
continue
full_path = join(d, f)
if isfile(full_path):
# Remove self.spreadsheets_path from the path
relative_path = full_path.split(self.spreadsheets_path)[1][1:]
# Calculate the MD5 hash for the file
hasher = hashlib.md5()
with open(self.__get_full_path(relative_path), "rb") as afile:
buf = afile.read()
hasher.update(buf)
h = hasher.hexdigest()
self.docs.append({"path": relative_path, "hash": h})
elif isdir(full_path):
self.__scan_directory(full_path)
def run(self):
while not self.stopped():
self.docs = []
self.__scan_directory(self.spreadsheets_path)
self.__check_removed()
self.__check_added()
self.done_scan = True
sleep(self.monitor_frequency)
================================================
FILE: pyproject.toml
================================================
[tool.black]
line-length = 79
include = '\.pyi?$'
exclude = '''
/(
\.git
| \.hg
| \.mypy_cache
| \.tox
| \.venv
| _build
| buck-out
| build
| dist
)/
'''
================================================
FILE: request_handler.py
================================================
# Copyright (C) 2016 Robert Scott
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
import json
import logging
import select
import socketserver
import struct
from socket import SHUT_RDWR
from time import sleep
from com.sun.star.uno import RuntimeException
from com.sun.star.io import IOException
from connection import SpreadsheetConnection
TIMEOUT = 10
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
def __init__(self, save_path, *args, **kwargs):
self.save_path = save_path
socketserver.TCPServer.__init__(self, *args, **kwargs)
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
def __send(self, msg):
"""Convert a message to JSON and send it to the client.
The messages are sent as utf-8 encoded bytes
"""
# What type is msg coming in as?
json_msg = json.dumps(msg)
json_msg = bytes(json_msg, "utf-8")
# Prepend the length of the string to the meg
json_msg = struct.pack(">I", len(json_msg)) + json_msg
self.request.send(json_msg)
logging.info("Sent: " + json.dumps(msg))
def __receive(self):
"""Receive a message from the client, decode it from JSON and return.
The received messages are utf-8 encoded bytes. False is returned on
failure to connect to the client, otherwise a string of the message is
returned.
"""
raw_msg_length = self.__receive_length(4)
if not raw_msg_length:
return False
msg_length = struct.unpack(">I", raw_msg_length)[0]
recv = self.__receive_length(msg_length)
if recv == b"":
# The connection is closed.
return False
recv_json = str(recv, encoding="utf-8")
recv_string = json.loads(recv_json)
logging.info("Received: " + str(recv_string))
return recv_string
def __receive_length(self, length):
"""Receive length number of bytes from the client."""
data = b""
while len(data) < length:
ready = select.select([self.request], [], [], TIMEOUT)
if ready[0]:
packet = self.request.recv(length - len(data))
if not packet:
return b""
data += packet
else:
logging.warning("Waited too long to recieve from the client.")
return b""
return data
def __make_connection(self):
"""Handle first request to server and check that it adheres to the
protocol.
"""
def protocol_error():
# Invalid connection protocol
logging.error(
"Client attempted to connect using and invalid protocol."
)
self.__send("PROTOCOL ERROR")
self.__close_connection()
return False
data = self.__receive()
if type(data) != list:
return protocol_error()
if len(data) != 2:
return protocol_error()
if data[0] != "SPREADSHEET":
return protocol_error()
# If there is a KeyError when looking up the spreadsheets name, wait
# a bit and try again
max_attempts = self.server.monitor_frequency + 1
attempt = 0
while 1:
if attempt >= max_attempts: # soffice process isin't coming up
logging.debug("Waited too long for spreadsheet")
# We can assume the spreadsheet does not exist
self.__send("NOT FOUND")
self.__close_connection()
return False
try:
self.con = SpreadsheetConnection(
self.server.spreadsheets[data[1]],
self.server.locks[data[1]],
self.server.save_path,
)
break
except KeyError:
pass
attempt += 1
logging.debug("Waiting for spreadsheet")
sleep(1)
# If the spreadsheet was sucessfully connected to
if attempt != max_attempts:
self.__send("OK")
self.con.lock_spreadsheet()
return True
def __close_connection(self):
"""Unlock the spreadsheet and close the connection to the client."""
try:
if self.con.lock.locked:
self.con.unlock_spreadsheet()
except (UnboundLocalError, AttributeError):
# con was never created.
pass
try:
self.request.shutdown(SHUT_RDWR)
except OSError:
# The client has already disconnected.
pass
logging.debug("Closing socket for ThreadedTCPRequestHandler")
self.request.close()
def __main_loop(self):
while True:
data = self.__receive()
if data == False:
# The connection has been lost.
break
elif data[0] == "SET":
try:
self.con.set_cells(data[1], data[2], data[3])
except (ValueError, RuntimeException) as e:
self.__send({"ERROR": str(e)})
else:
self.__send("OK")
elif data[0] == "GET":
try:
cells = self.con.get_cells(data[1], data[2])
except (ValueError, RuntimeException) as e:
self.__send({"ERROR": str(e)})
else:
self.__send(cells)
elif data[0] == "GET_SHEETS":
sheet_names = self.con.get_sheet_names()
self.__send(sheet_names)
elif data[0] == "SAVE":
try:
self.con.save_spreadsheet(data[1])
except (IOException, OSError) as e:
self.__send({"ERROR": str(e)})
else:
self.__send("OK")
def handle(self):
"""Make a connection to the client, run the main protocol loop and
close the connection.
"""
if self.__make_connection():
self.__main_loop()
self.__close_connection()
================================================
FILE: requirements.txt
================================================
pyoo==1.4
Werkzeug==3.1.3
================================================
FILE: saved_spreadsheets/.gitignore
================================================
# Ignore everything in this directory
*
# Except this file
!.gitignore
================================================
FILE: server.py
================================================
# Copyright (C) 2016 Robert Scott
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
import logging
import os
import socket
import subprocess
import sys
import tempfile
import threading
from time import sleep
import pyoo
from monitor import MonitorThread
from request_handler import ThreadedTCPRequestHandler, ThreadedTCPServer
this_dir = os.path.dirname(os.path.realpath(__file__))
SAVE_PATH = os.path.join(this_dir, "saved_spreadsheets")
SPREADSHEETS_PATH = os.path.join(this_dir, "spreadsheets")
SOFFICE_LOG = os.path.join(this_dir, "log", "soffice.log")
LOG_FILE = os.path.join(this_dir, "log", "server.log")
SOFFICE_PROCNAME = "soffice.bin"
HOST, PORT = "localhost", 5555
SOFFICE_PIPE = "soffice_headless"
MONITOR_FREQ = 5 # In seconds
LOG_LEVEL = logging.DEBUG
class SpreadsheetServer:
def __init__(
self,
soffice_log=SOFFICE_LOG,
host=HOST,
port=PORT,
soffice_pipe=SOFFICE_PIPE,
spreadsheets_path=SPREADSHEETS_PATH,
monitor_frequency=MONITOR_FREQ,
reload_on_disk_change=True,
ask_kill=False,
save_path=SAVE_PATH,
log_level=LOG_LEVEL,
log_file=LOG_FILE,
):
# Where the output from LibreOffice is logged to
self.soffice_log = soffice_log
self.host = host # The address on which the server is listening
self.port = port # The port on which the server is listening
# The name of the pipe set up by LibreOffice that pyoo will connect to.
self.soffice_pipe = soffice_pipe
# The frequency, in seconds, at which the directory containing the
# spreadsheets is polled.
self.monitor_frequency = monitor_frequency
# Whether or not to close and open a spreadsheet with the file changes on
# disk
self.reload_on_disk_change = reload_on_disk_change
# Whether or not to interactively ask the user if they want to kill an
# existing LibreOffice process.
self.ask_kill = ask_kill
# Where to save the spreadsheets when requested by the client.
self.save_path = save_path
# Where to look for spreadsheets to load.
self.spreadsheets_path = spreadsheets_path
self.spreadsheets = {} # Each pyoo spreadsheet object.
self.locks = {} # A lock for each spreadsheet.
self.hashes = {} # A hash of the file contents for each spreadsheet.
self.log_level = log_level
self.log_file = log_file # Where 'logging' logs to
self.libreoffice_temp_dir = tempfile.TemporaryDirectory()
def __logging(self):
"""Set up logging."""
logging.basicConfig(
format="%(asctime)s:%(levelname)s:%(message)s",
datefmt="%Y%m%d %H:%M:%S",
# filename=self.log_file,
level=self.log_level,
)
def __start_soffice(self):
def get_soffice_binay_path():
try:
return str(
subprocess.check_output(["which", "soffice"])[:-1],
encoding="utf-8",
)
except subprocess.CalledProcessError:
raise RuntimeError(
"The soffice binary was not found. Is LibreOffice installed?"
)
soffice_path = get_soffice_binay_path()
command = (
soffice_path
+ " -env:UserInstallation=file://"
+ self.libreoffice_temp_dir.name
+ ' --accept="pipe,name='
+ self.soffice_pipe
+ ';urp;" --norestore --nologo --nodefault --headless --invisible --nocrashreport --nofirststartwizard'
)
self.logfile = open(self.soffice_log, "w")
self.soffice_process = subprocess.Popen(
command, shell=True, stdout=self.logfile, stderr=self.logfile
)
def __connect_to_soffice(self):
"""Make a connection to soffice and fail if it can not connect."""
MAX_ATTEMPTS = 10
attempt = 0
while 1:
if attempt > MAX_ATTEMPTS: # soffice process isin't coming up
raise RuntimeError(
"Could not connect to the soffice process. Did LibreOffice start?"
)
try:
self.soffice = pyoo.Desktop(pipe=SOFFICE_PIPE)
logging.info("Connected to soffice.")
break
except (OSError, IOError):
attempt += 1
sleep(1) # Wait for the soffice process to start
except Exception:
exc_type, exc_obj, exc_tb = sys.exc_info()
fname = os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
print(exc_type, fname, exc_tb.tb_lineno)
def __start_threaded_tcp_server(self):
"""Set up and start the TCP threaded server to handle incomming
requests.
"""
logging.info("Starting spreadsheet_server.")
MAX_ATTEMPTS = 10
attempt = 0
def start_threaded_tcp_server(attempt):
try:
self.server = ThreadedTCPServer(
self.save_path,
(self.host, self.port),
ThreadedTCPRequestHandler,
)
except (OSError, socket.error):
attempt += 1
if attempt > MAX_ATTEMPTS:
import traceback
traceback.print_exc()
print(
"Error: The port is in use. Maybe the server is already"
"running?"
)
exit()
sleep(1)
start_threaded_tcp_server(attempt) # Try again
start_threaded_tcp_server(attempt)
self.server.spreadsheets = self.spreadsheets
self.server.locks = self.locks
self.server.hashes = self.hashes
self.server.monitor_frequency = self.monitor_frequency
# Start the main server thread. This server thread will start a
# new thread to handle each client connection.
self.server_thread = threading.Thread(target=self.server.serve_forever)
self.server_thread.daemon = False # Gracefully stop child threads
self.server_thread.start()
logging.info("Server thread running. Waiting on connections...")
def __start_monitor_thread(self):
"""This thread monitors the SPREADSHEETS directory to add or remove.
"""
self.monitor_thread = MonitorThread(
self.spreadsheets,
self.locks,
self.hashes,
self.soffice,
self.spreadsheets_path,
self.monitor_frequency,
self.reload_on_disk_change,
)
self.monitor_thread.daemon = True
self.monitor_thread.start()
def __stop_monitor_thread(self):
"""Stop the monitor thread."""
self.monitor_thread.stop_thread()
self.monitor_thread.join()
def __stop_threaded_tcp_server(self):
"""Stop the ThreadedTCPServer."""
try:
self.server.shutdown()
self.server.server_close()
except AttributeError:
# The server was never set up
pass
def __kill_libreoffice(self):
"""Terminate the soffice.bin process."""
self.soffice_process.terminate()
self.soffice_process.wait()
self.libreoffice_temp_dir.cleanup()
def __close_logfile(self):
"""Close the logfile."""
self.logfile.close()
def stop(self):
"""Stop all the threads and shutdown LibreOffice."""
self.__stop_monitor_thread()
self.__stop_threaded_tcp_server()
self.__kill_libreoffice()
self.__close_logfile()
def run(self):
self.__logging()
self.__start_soffice()
self.__connect_to_soffice()
self.__start_monitor_thread()
self.__start_threaded_tcp_server()
if __name__ == "__main__":
print("Starting spreadsheet_server...")
spreadsheet_server = SpreadsheetServer(ask_kill=True)
try:
print("Logging to: " + spreadsheet_server.log_file)
print("Connecting to LibreOffice...")
spreadsheet_server.run()
print("Up and listening for connections!")
while True:
sleep(100)
except KeyboardInterrupt:
print("Shutting down server. Please wait...")
spreadsheet_server.stop()
================================================
FILE: spreadsheets/.gitignore
================================================
# Ignore everything in this directory
*
# Except this file
!.gitignore
================================================
FILE: tests/__init__.py
================================================
================================================
FILE: tests/context.py
================================================
import os
import sys
sys.path.insert(0, os.path.abspath(".."))
from connection import SpreadsheetConnection
from server import SpreadsheetServer
from monitor import MonitorThread
from request_handler import ThreadedTCPServer, ThreadedTCPRequestHandler
from client import SpreadsheetClient
================================================
FILE: tests/test_client.py
================================================
import unittest
from .context import SpreadsheetServer, SpreadsheetClient
from time import sleep
import os
import shutil
import sys
import logging
EXAMPLE_SPREADSHEET = "example.ods"
SOFFICE_PIPE = "soffice_headless"
SPREADSHEETS_PATH = "./spreadsheets"
TESTS_PATH = "./tests"
SHEET_NAME = "Sheet1"
class TestClient(unittest.TestCase):
@classmethod
def setUpClass(cls):
# Copy the example spreadsheet from the tests directory into the spreadsheets
# directory
shutil.copyfile(
TESTS_PATH + "/" + EXAMPLE_SPREADSHEET,
SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET,
)
cls.server = SpreadsheetServer(log_level=logging.CRITICAL)
cls.server.run()
@classmethod
def tearDownClass(cls):
cls.server.stop()
os.remove(SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET)
def setUp(self):
self.sc = SpreadsheetClient(EXAMPLE_SPREADSHEET)
def tearDown(self):
self.sc.disconnect()
def test_connect_invalid_spreadsheet(self):
try:
SpreadsheetClient(EXAMPLE_SPREADSHEET + "z")
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(
str(e), "The requested spreadsheet was not found."
)
# Give the ThreadedTCPServer some time to shut down correctly before
# the next test
sleep(1)
def test_get_sheet_names(self):
sheet_names = self.sc.get_sheet_names()
self.assertEqual(sheet_names, ["Sheet1"])
def test_set_cell(self):
self.sc.set_cells(SHEET_NAME, "A1", 5)
a1 = self.sc.get_cells(SHEET_NAME, "A1")
self.assertEqual(a1, 5)
def test_set_cell_invalid_sheet(self):
try:
self.sc.set_cells(SHEET_NAME + "z", "A1", 5)
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(str(e), "Sheet name is invalid.")
def test_get_cell(self):
cell_value = self.sc.get_cells(SHEET_NAME, "C3")
self.assertEqual(cell_value, 6)
def test_get_cell_invalid_sheet(self):
try:
self.sc.get_cells(SHEET_NAME + "z", "C3")
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(str(e), "Sheet name is invalid.")
def test_get_invalid_cell_numeric(self):
try:
self.sc.get_cells(SHEET_NAME, 1)
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(str(e), "Cell range is invalid.")
def test_get_invalid_cell_missing_alpha(self):
try:
self.sc.get_cells(SHEET_NAME, "1")
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(str(e), "Cell range is invalid.")
def test_get_invalid_cell_missing_numeric(self):
try:
self.sc.get_cells(SHEET_NAME, "A")
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(str(e), "Cell range is invalid.")
def test_get_invalid_cell_missing_start_numeric(self):
try:
self.sc.get_cells(SHEET_NAME, "A:B2")
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(str(e), "Cell range is invalid.")
def test_get_invalid_cell_missing_end_numeric(self):
try:
self.sc.get_cells(SHEET_NAME, "A1:B")
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(str(e), "Cell range is invalid.")
def test_get_invalid_cell_missing_start_alpha(self):
try:
self.sc.get_cells(SHEET_NAME, "1:B2")
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(str(e), "Cell range is invalid.")
def test_get_invalid_cell_missing_end_alpha(self):
try:
self.sc.get_cells(SHEET_NAME, "A1:2")
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(str(e), "Cell range is invalid.")
def test_get_invalid_cell_negative(self):
try:
self.sc.get_cells(SHEET_NAME, "A-1:B2")
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(str(e), "Cell range is invalid.")
def test_get_invalid_cell_numeric_too_large(self):
try:
self.sc.get_cells(SHEET_NAME, "A1048577")
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(str(e), "Cell range is invalid.")
def test_get_invalid_cell_alpha_too_large(self):
try:
self.sc.get_cells(SHEET_NAME, "AMK1")
self.assertTrue(False)
except RuntimeError as e:
self.assertEqual(str(e), "Cell range is invalid.")
def test_set_cell_row(self):
cell_values = [4, 5, 6]
self.sc.set_cells(SHEET_NAME, "A1:A3", cell_values)
saved_values = self.sc.get_cells(SHEET_NAME, "A1:A3")
self.assertEqual(cell_values, saved_values)
def test_get_cell_column(self):
cell_values = self.sc.get_cells(SHEET_NAME, "C1:C3")
self.assertEqual(cell_values, [3, 3.5, 6])
def test_get_cell_column_large_alpha(self):
cell_values = self.sc.get_cells(SHEET_NAME, "AF5:AF186")
self.assertEqual(cell_values, ["" for x in range(5, 186 + 1)])
def test_set_cell_range(self):
cell_values = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
self.sc.set_cells(SHEET_NAME, "A1:C3", cell_values)
saved_values = self.sc.get_cells(SHEET_NAME, "A1:C3")
self.assertEqual(cell_values, saved_values)
def test_save_spreadsheet(self):
filename = "test.ods"
self.sc.save_spreadsheet(filename)
dir_path = os.path.dirname(os.path.realpath(__file__))
saved_path = dir_path + "/../saved_spreadsheets/" + filename
self.assertTrue(os.path.exists(saved_path))
os.remove(saved_path)
def test_unicode(self):
for i in range(1000):
self.sc.set_cells(SHEET_NAME, "A1", chr(i))
cell = self.sc.get_cells(SHEET_NAME, "A1")
try:
int(chr(i))
continue
except ValueError:
pass # Not a number
self.assertEqual(chr(i), cell)
if __name__ == "__main__":
unittest.main()
================================================
FILE: tests/test_connection.py
================================================
import os
import shutil
import threading
import unittest
from signal import SIGTERM
from .context import SpreadsheetConnection, SpreadsheetServer
EXAMPLE_SPREADSHEET = "example.ods"
SOFFICE_PIPE = "soffice_headless"
SPREADSHEETS_PATH = "./spreadsheets"
TESTS_PATH = "./tests"
class TestConnection(unittest.TestCase):
@classmethod
def setUpClass(cls):
# Copy the example spreadsheet from the tests directory into the spreadsheets
# directory
shutil.copyfile(
TESTS_PATH + "/" + EXAMPLE_SPREADSHEET,
SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET,
)
cls.spreadsheet_server = SpreadsheetServer()
cls.spreadsheet_server._SpreadsheetServer__start_soffice()
cls.spreadsheet_server._SpreadsheetServer__connect_to_soffice()
@classmethod
def tearDownClass(cls):
cls.spreadsheet_server._SpreadsheetServer__kill_libreoffice()
cls.spreadsheet_server._SpreadsheetServer__close_logfile()
os.remove(SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET)
def setUp(self):
soffice = self.spreadsheet_server.soffice
self.spreadsheet = soffice.open_spreadsheet(
SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET
)
lock = threading.Lock()
self.ss_con = SpreadsheetConnection(
self.spreadsheet, lock, self.spreadsheet_server.save_path
)
def tearDown(self):
self.spreadsheet.close()
def test_lock_spreadsheet(self):
self.ss_con.lock_spreadsheet()
self.assertTrue(self.ss_con.lock.locked())
self.ss_con.unlock_spreadsheet()
def test_unlock_spreadsheet(self):
self.ss_con.lock_spreadsheet()
status = self.ss_con.unlock_spreadsheet()
self.assertTrue(status)
self.assertFalse(self.ss_con.lock.locked())
def test_unlock_spreadsheet_runtime_error(self):
status = self.ss_con.unlock_spreadsheet()
self.assertFalse(status)
self.assertFalse(self.ss_con.lock.locked())
def test_get_xy_index_first_cell(self):
(
alpha_index,
num_index,
) = self.ss_con._SpreadsheetConnection__get_xy_index(u"A1")
self.assertEqual(alpha_index, 0)
self.assertEqual(num_index, 0)
def test_get_xy_index_Z26(self):
(
alpha_index,
num_index,
) = self.ss_con._SpreadsheetConnection__get_xy_index(u"Z26")
self.assertEqual(alpha_index, 25)
self.assertEqual(num_index, 25)
def test_get_xy_index_aa3492(self):
(
alpha_index,
num_index,
) = self.ss_con._SpreadsheetConnection__get_xy_index(u"AA3492")
self.assertEqual(alpha_index, 26)
self.assertEqual(num_index, 3491)
def test_get_xy_index_aaa1024(self):
(
alpha_index,
num_index,
) = self.ss_con._SpreadsheetConnection__get_xy_index(u"AAA1024")
self.assertEqual(alpha_index, 702)
self.assertEqual(num_index, 1023)
def test_get_xy_index_aab739(self):
(
alpha_index,
num_index,
) = self.ss_con._SpreadsheetConnection__get_xy_index(u"AAB739")
self.assertEqual(alpha_index, 703)
self.assertEqual(num_index, 738)
def test_get_xy_index_aba1(self):
(
alpha_index,
num_index,
) = self.ss_con._SpreadsheetConnection__get_xy_index(u"ABA1")
self.assertEqual(alpha_index, 728)
self.assertEqual(num_index, 0)
def test_get_xy_index_abc123(self):
(
alpha_index,
num_index,
) = self.ss_con._SpreadsheetConnection__get_xy_index(u"ABC123")
self.assertEqual(alpha_index, 730)
self.assertEqual(num_index, 122)
def test_get_xy_index_last_cell(self):
(
alpha_index,
num_index,
) = self.ss_con._SpreadsheetConnection__get_xy_index(u"AMJ1048576")
self.assertEqual(alpha_index, 1023)
self.assertEqual(num_index, 1048575)
def test_is_single_cell(self):
status = self.ss_con._SpreadsheetConnection__is_single_cell(u"AMJ1")
self.assertTrue(status)
def test_is_not_single_cell(self):
status = self.ss_con._SpreadsheetConnection__is_single_cell(u"A1:Z26")
self.assertFalse(status)
def test_check_not_single_cell(self):
status = True
try:
self.ss_con._SpreadsheetConnection__check_single_cell(u"DD56:Z98")
except ValueError:
status = False
self.assertFalse(status)
def test_cell_to_index(self):
d = self.ss_con._SpreadsheetConnection__cell_to_index(u"ABC945")
self.assertTrue(d["row_index"] == 944)
self.assertTrue(d["column_index"] == 730)
def test_cell_range_to_index(self):
d = self.ss_con._SpreadsheetConnection__cell_range_to_index(u"C9:Z26")
self.assertTrue(d["row_start"] == 8)
self.assertTrue(d["row_end"] == 25)
self.assertTrue(d["column_start"] == 2)
self.assertTrue(d["column_end"] == 25)
def test_check_for_lock(self):
status = False
try:
self.ss_con._SpreadsheetConnection__check_for_lock()
except RuntimeError:
status = True
self.assertTrue(status)
def test_check_numeric(self):
value = self.ss_con._SpreadsheetConnection__convert_to_float_if_numeric(
"1"
)
self.assertTrue(type(value) is float)
def test_check_not_numeric(self):
value = self.ss_con._SpreadsheetConnection__convert_to_float_if_numeric(
u"123A"
)
self.assertTrue(type(value) is str)
def test_check_not_list(self):
status = False
try:
self.ss_con._SpreadsheetConnection__check_list(1)
except ValueError:
status = True
self.assertTrue(status)
def test_check_1D_list(self):
data = ["1", "2", "3"]
data = self.ss_con._SpreadsheetConnection__check_1D_list(data)
self.assertEqual(data, [1.0, 2.0, 3.0])
def test_check_1D_list_when_2D(self):
status = False
data = [["1", "2", "3"], ["1", "2", "3"]]
try:
data = self.ss_con._SpreadsheetConnection__check_1D_list(data)
except ValueError:
status = True
self.assertTrue(status)
def test_invalid_sheet_name(self):
self.ss_con.lock_spreadsheet()
status = False
try:
self.ss_con.set_cells(u"1Sheet1", u"A1", 1)
except ValueError:
status = True
self.ss_con.unlock_spreadsheet()
self.assertTrue(status)
def test_set_single_cell(self):
self.ss_con.lock_spreadsheet()
self.ss_con.set_cells(u"Sheet1", u"A1", 1)
self.assertEqual(self.ss_con.get_cells(u"Sheet1", u"A1"), 1)
self.ss_con.unlock_spreadsheet()
def test_set_single_cell_list_of_data(self):
self.ss_con.lock_spreadsheet()
status = False
try:
self.ss_con.set_cells(u"Sheet1", u"A1", [9, 1])
except ValueError:
status = True
self.assertTrue(status)
self.assertNotEqual(self.ss_con.get_cells(u"Sheet1", u"A1"), 9)
self.ss_con.unlock_spreadsheet()
def test_set_cell_range_columnn(self):
self.ss_con.lock_spreadsheet()
self.ss_con.set_cells(u"Sheet1", u"A1:A5", [1, 2, 3, 4, 5])
self.assertEqual(
self.ss_con.get_cells(u"Sheet1", u"A1:A5"),
(1.0, 2.0, 3.0, 4.0, 5.0),
)
self.ss_con.unlock_spreadsheet()
def test_set_cell_range_row(self):
self.ss_con.lock_spreadsheet()
self.ss_con.set_cells(u"Sheet1", u"A1:E1", [1, 2, 3, 4, 5])
self.assertEqual(
self.ss_con.get_cells(u"Sheet1", u"A1:E1"),
(1.0, 2.0, 3.0, 4.0, 5.0),
)
self.ss_con.unlock_spreadsheet()
def test_set_cell_range_2D(self):
self.ss_con.lock_spreadsheet()
self.ss_con.set_cells(u"Sheet1", u"A1:B2", [[1, 2], [3, 4]])
self.assertEqual(
self.ss_con.get_cells(u"Sheet1", u"A1:B2"),
((1.0, 2.0), (3.0, 4.0)),
)
self.ss_con.unlock_spreadsheet()
def test_set_cell_range_2D_incorrect_data(self):
self.ss_con.lock_spreadsheet()
status = False
try:
self.ss_con.set_cells(u"Sheet1", u"A1:B2", [9, 9, 9, 9])
except ValueError:
status = True
self.assertTrue(status)
self.assertNotEqual(
self.ss_con.get_cells(u"Sheet1", u"A1:B2"),
((9.0, 9.0), (9.0, 9.0)),
)
self.ss_con.unlock_spreadsheet()
def test_get_sheet_names(self):
sheet_names = self.ss_con.get_sheet_names()
self.assertEqual(sheet_names, [u"Sheet1"])
def test_save_spreadsheet(self):
path = "./saved_spreadsheets/" + EXAMPLE_SPREADSHEET + ".new"
if os.path.exists(path):
os.remove(path)
self.ss_con.lock_spreadsheet()
status = self.ss_con.save_spreadsheet(EXAMPLE_SPREADSHEET + ".new")
self.assertTrue(status)
self.assertTrue(os.path.exists(path))
self.ss_con.unlock_spreadsheet()
def test_save_spreadsheet_no_lock(self):
path = "./saved_spreadsheets/" + EXAMPLE_SPREADSHEET + ".new"
if os.path.exists(path):
os.remove(path)
status = self.ss_con.save_spreadsheet(EXAMPLE_SPREADSHEET + ".new")
self.assertFalse(status)
self.assertFalse(
os.path.exists(
"./saved_spreadsheets/" + EXAMPLE_SPREADSHEET + ".new"
)
)
if __name__ == "__main__":
unittest.main()
================================================
FILE: tests/test_monitor.py
================================================
import os
import shutil
import unittest
from time import sleep
from .context import MonitorThread, SpreadsheetClient, SpreadsheetServer
this_dir = os.path.dirname(os.path.realpath(__file__))
parent_dir = os.path.dirname(this_dir)
SPREADSHEETS_PATH = os.path.join(parent_dir, "spreadsheets")
TESTS_PATH = this_dir
SAVED_SPREADSHEETS_PATH = os.path.join(parent_dir, "saved_spreadsheets")
EXAMPLE_SPREADSHEET = "example.ods"
EXAMPLE_SPREADSHEET_MOVED = "example_moved.ods"
SOFFICE_PIPE = "soffice_headless"
SHEET_NAME = "Sheet1"
class TestMonitor(unittest.TestCase):
@classmethod
def setUpClass(cls):
# Copy the example spreadsheet from the tests directory into the spreadsheets
# directory
shutil.copyfile(
TESTS_PATH + "/" + EXAMPLE_SPREADSHEET,
SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET,
)
cls.spreadsheet_server = SpreadsheetServer()
cls.spreadsheet_server._SpreadsheetServer__start_soffice()
cls.spreadsheet_server._SpreadsheetServer__connect_to_soffice()
@classmethod
def tearDownClass(cls):
cls.spreadsheet_server._SpreadsheetServer__kill_libreoffice()
cls.spreadsheet_server._SpreadsheetServer__close_logfile()
os.remove(SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET)
def setUp(self):
self.spreadsheet_server._SpreadsheetServer__start_monitor_thread()
self.monitor_thread = self.spreadsheet_server.monitor_thread
while not self.monitor_thread.initial_scan():
sleep(0.5)
def tearDown(self):
self.spreadsheet_server._SpreadsheetServer__stop_monitor_thread()
def test_unload_spreadsheet(self):
self.monitor_thread._MonitorThread__unload_spreadsheet(
EXAMPLE_SPREADSHEET
)
spreadsheets = [
key for key, value in self.monitor_thread.spreadsheets.items()
]
locks = [key for key, value in self.monitor_thread.locks.items()]
self.assertTrue(EXAMPLE_SPREADSHEET not in spreadsheets)
self.assertTrue(EXAMPLE_SPREADSHEET not in locks)
def test_check_added_already_exists(self):
self.monitor_thread._MonitorThread__check_added()
spreadsheets = [
key for key, value in self.monitor_thread.spreadsheets.items()
]
locks = [key for key, value in self.monitor_thread.locks.items()]
self.assertTrue(EXAMPLE_SPREADSHEET in spreadsheets)
self.assertTrue(EXAMPLE_SPREADSHEET in locks)
def test_check_removed_when_renamed(self):
# Rename example.ods to example_moved.ods
current_loc = SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET
moved_loc = SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET_MOVED
os.rename(current_loc, moved_loc)
self.monitor_thread.docs = []
self.monitor_thread._MonitorThread__scan_directory(SPREADSHEETS_PATH)
self.monitor_thread._MonitorThread__check_removed()
self.monitor_thread._MonitorThread__check_added()
spreadsheets = [
key for key, value in self.monitor_thread.spreadsheets.items()
]
locks = [key for key, value in self.monitor_thread.locks.items()]
self.assertTrue(EXAMPLE_SPREADSHEET not in spreadsheets)
self.assertTrue(EXAMPLE_SPREADSHEET not in locks)
self.assertTrue(EXAMPLE_SPREADSHEET_MOVED in spreadsheets)
self.assertTrue(EXAMPLE_SPREADSHEET_MOVED in locks)
# Move it back to where it was
os.rename(moved_loc, current_loc)
def test_change_file_hash(self):
# Save the example file with a modification
current_loc = SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET
moved_loc = SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET_MOVED
shutil.copyfile(current_loc, moved_loc)
self.spreadsheet_server._SpreadsheetServer__start_threaded_tcp_server()
self.sc = SpreadsheetClient(EXAMPLE_SPREADSHEET_MOVED)
self.sc.set_cells(SHEET_NAME, "A1", 5)
self.sc.save_spreadsheet(EXAMPLE_SPREADSHEET_MOVED)
self.sc.disconnect()
current_loc = SAVED_SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET_MOVED
moved_loc = SPREADSHEETS_PATH + "/" + EXAMPLE_SPREADSHEET_MOVED
hash_before = self.monitor_thread.hashes[EXAMPLE_SPREADSHEET_MOVED]
os.rename(current_loc, moved_loc)
# Run a scan manually
self.monitor_thread.docs = []
self.monitor_thread._MonitorThread__scan_directory(SPREADSHEETS_PATH)
self.monitor_thread._MonitorThread__check_removed()
self.monitor_thread._MonitorThread__check_added()
hash_after = self.monitor_thread.hashes[EXAMPLE_SPREADSHEET_MOVED]
self.assertNotEqual(hash_before, hash_after)
self.sc = SpreadsheetClient(EXAMPLE_SPREADSHEET_MOVED)
cell = self.sc.get_cells(SHEET_NAME, "A1")
self.sc.save_spreadsheet(EXAMPLE_SPREADSHEET_MOVED)
self.sc.disconnect()
self.assertEqual(cell, 5)
self.spreadsheet_server._SpreadsheetServer__stop_threaded_tcp_server()
os.remove(moved_loc)
if __name__ == "__main__":
unittest.main()
gitextract_sjf5n4b7/
├── .coveragerc
├── .dockerignore
├── .editorconfig
├── .flake8
├── .gitignore
├── COPYING
├── Dockerfile
├── README.md
├── client.py
├── connection.py
├── coverage.sh
├── docker_run.sh
├── example_client.py
├── log/
│ └── .gitignore
├── monitor.py
├── pyproject.toml
├── request_handler.py
├── requirements.txt
├── saved_spreadsheets/
│ └── .gitignore
├── server.py
├── spreadsheets/
│ └── .gitignore
└── tests/
├── __init__.py
├── context.py
├── example.ods
├── test_client.py
├── test_connection.py
└── test_monitor.py
SYMBOL INDEX (144 symbols across 8 files)
FILE: client.py
class SpreadsheetClient (line 28) | class SpreadsheetClient:
method __init__ (line 29) | def __init__(self, spreadsheet, ip=IP, port=PORT):
method __connect (line 37) | def __connect(self, ip, port):
method __set_spreadsheet (line 42) | def __set_spreadsheet(self, spreadsheet):
method set_cells (line 50) | def set_cells(self, sheet, cell_ref, data):
method get_sheet_names (line 72) | def get_sheet_names(self):
method get_cells (line 83) | def get_cells(self, sheet, cell_ref):
method save_spreadsheet (line 105) | def save_spreadsheet(self, filename):
method __send (line 112) | def __send(self, msg):
method __receive (line 127) | def __receive(self):
method __receive_length (line 147) | def __receive_length(self, length):
method disconnect (line 168) | def disconnect(self):
FILE: connection.py
class SpreadsheetConnection (line 28) | class SpreadsheetConnection:
method __init__ (line 32) | def __init__(self, spreadsheet, lock, save_path):
method lock_spreadsheet (line 37) | def lock_spreadsheet(self):
method unlock_spreadsheet (line 47) | def unlock_spreadsheet(self):
method __get_xy_index (line 56) | def __get_xy_index(self, cell_ref):
method __is_single_cell (line 93) | def __is_single_cell(self, cell_ref):
method __check_single_cell (line 98) | def __check_single_cell(self, cell_ref):
method __cell_to_index (line 104) | def __cell_to_index(self, cell_ref):
method __cell_range_to_index (line 116) | def __cell_range_to_index(self, cell_ref):
method __check_for_lock (line 139) | def __check_for_lock(self):
method __convert_to_float_if_numeric (line 145) | def __convert_to_float_if_numeric(self, value):
method __check_list (line 155) | def __check_list(self, data):
method __check_1D_list (line 159) | def __check_1D_list(self, data):
method set_cells (line 169) | def set_cells(self, sheet, cell_ref, value):
method set_cell (line 184) | def set_cell(self, sheet, cell_ref, value):
method set_cell_range (line 208) | def set_cell_range(self, sheet, cell_ref, data):
method get_sheet_names (line 252) | def get_sheet_names(self):
method __validate_cell_ref (line 257) | def __validate_cell_ref(self, cell_ref):
method __validate_sheet_name (line 289) | def __validate_sheet_name(self, sheet):
method get_cells (line 307) | def get_cells(self, sheet, cell_ref):
method get_cell (line 322) | def get_cell(self, sheet, cell_ref):
method get_cell_range (line 338) | def get_cell_range(self, sheet, cell_ref):
method save_spreadsheet (line 371) | def save_spreadsheet(self, filename):
FILE: monitor.py
class MonitorThread (line 26) | class MonitorThread(threading.Thread):
method __init__ (line 29) | def __init__(
method stop_thread (line 56) | def stop_thread(self):
method stopped (line 59) | def stopped(self):
method initial_scan (line 62) | def initial_scan(self):
method __get_full_path (line 65) | def __get_full_path(self, doc):
method __delete_lock_files (line 68) | def __delete_lock_files(self):
method __load_spreadsheet (line 77) | def __load_spreadsheet(self, doc):
method __unload_spreadsheet (line 86) | def __unload_spreadsheet(self, doc_path):
method __check_added (line 94) | def __check_added(self):
method __check_removed (line 121) | def __check_removed(self):
method __scan_directory (line 139) | def __scan_directory(self, d):
method run (line 166) | def run(self):
FILE: request_handler.py
class ThreadedTCPServer (line 32) | class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPSer...
method __init__ (line 33) | def __init__(self, save_path, *args, **kwargs):
class ThreadedTCPRequestHandler (line 38) | class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
method __send (line 39) | def __send(self, msg):
method __receive (line 56) | def __receive(self):
method __receive_length (line 81) | def __receive_length(self, length):
method __make_connection (line 102) | def __make_connection(self):
method __close_connection (line 162) | def __close_connection(self):
method __main_loop (line 181) | def __main_loop(self):
method handle (line 217) | def handle(self):
FILE: server.py
class SpreadsheetServer (line 44) | class SpreadsheetServer:
method __init__ (line 45) | def __init__(
method __logging (line 95) | def __logging(self):
method __start_soffice (line 105) | def __start_soffice(self):
method __connect_to_soffice (line 133) | def __connect_to_soffice(self):
method __start_threaded_tcp_server (line 159) | def __start_threaded_tcp_server(self):
method __start_monitor_thread (line 210) | def __start_monitor_thread(self):
method __stop_monitor_thread (line 227) | def __stop_monitor_thread(self):
method __stop_threaded_tcp_server (line 233) | def __stop_threaded_tcp_server(self):
method __kill_libreoffice (line 244) | def __kill_libreoffice(self):
method __close_logfile (line 251) | def __close_logfile(self):
method stop (line 256) | def stop(self):
method run (line 264) | def run(self):
FILE: tests/test_client.py
class TestClient (line 16) | class TestClient(unittest.TestCase):
method setUpClass (line 18) | def setUpClass(cls):
method tearDownClass (line 31) | def tearDownClass(cls):
method setUp (line 35) | def setUp(self):
method tearDown (line 38) | def tearDown(self):
method test_connect_invalid_spreadsheet (line 41) | def test_connect_invalid_spreadsheet(self):
method test_get_sheet_names (line 54) | def test_get_sheet_names(self):
method test_set_cell (line 58) | def test_set_cell(self):
method test_set_cell_invalid_sheet (line 63) | def test_set_cell_invalid_sheet(self):
method test_get_cell (line 70) | def test_get_cell(self):
method test_get_cell_invalid_sheet (line 74) | def test_get_cell_invalid_sheet(self):
method test_get_invalid_cell_numeric (line 81) | def test_get_invalid_cell_numeric(self):
method test_get_invalid_cell_missing_alpha (line 88) | def test_get_invalid_cell_missing_alpha(self):
method test_get_invalid_cell_missing_numeric (line 95) | def test_get_invalid_cell_missing_numeric(self):
method test_get_invalid_cell_missing_start_numeric (line 102) | def test_get_invalid_cell_missing_start_numeric(self):
method test_get_invalid_cell_missing_end_numeric (line 109) | def test_get_invalid_cell_missing_end_numeric(self):
method test_get_invalid_cell_missing_start_alpha (line 116) | def test_get_invalid_cell_missing_start_alpha(self):
method test_get_invalid_cell_missing_end_alpha (line 123) | def test_get_invalid_cell_missing_end_alpha(self):
method test_get_invalid_cell_negative (line 130) | def test_get_invalid_cell_negative(self):
method test_get_invalid_cell_numeric_too_large (line 137) | def test_get_invalid_cell_numeric_too_large(self):
method test_get_invalid_cell_alpha_too_large (line 144) | def test_get_invalid_cell_alpha_too_large(self):
method test_set_cell_row (line 151) | def test_set_cell_row(self):
method test_get_cell_column (line 158) | def test_get_cell_column(self):
method test_get_cell_column_large_alpha (line 162) | def test_get_cell_column_large_alpha(self):
method test_set_cell_range (line 166) | def test_set_cell_range(self):
method test_save_spreadsheet (line 173) | def test_save_spreadsheet(self):
method test_unicode (line 184) | def test_unicode(self):
FILE: tests/test_connection.py
class TestConnection (line 15) | class TestConnection(unittest.TestCase):
method setUpClass (line 17) | def setUpClass(cls):
method tearDownClass (line 31) | def tearDownClass(cls):
method setUp (line 36) | def setUp(self):
method tearDown (line 47) | def tearDown(self):
method test_lock_spreadsheet (line 50) | def test_lock_spreadsheet(self):
method test_unlock_spreadsheet (line 55) | def test_unlock_spreadsheet(self):
method test_unlock_spreadsheet_runtime_error (line 61) | def test_unlock_spreadsheet_runtime_error(self):
method test_get_xy_index_first_cell (line 66) | def test_get_xy_index_first_cell(self):
method test_get_xy_index_Z26 (line 74) | def test_get_xy_index_Z26(self):
method test_get_xy_index_aa3492 (line 82) | def test_get_xy_index_aa3492(self):
method test_get_xy_index_aaa1024 (line 90) | def test_get_xy_index_aaa1024(self):
method test_get_xy_index_aab739 (line 98) | def test_get_xy_index_aab739(self):
method test_get_xy_index_aba1 (line 106) | def test_get_xy_index_aba1(self):
method test_get_xy_index_abc123 (line 114) | def test_get_xy_index_abc123(self):
method test_get_xy_index_last_cell (line 122) | def test_get_xy_index_last_cell(self):
method test_is_single_cell (line 130) | def test_is_single_cell(self):
method test_is_not_single_cell (line 134) | def test_is_not_single_cell(self):
method test_check_not_single_cell (line 138) | def test_check_not_single_cell(self):
method test_cell_to_index (line 147) | def test_cell_to_index(self):
method test_cell_range_to_index (line 153) | def test_cell_range_to_index(self):
method test_check_for_lock (line 162) | def test_check_for_lock(self):
method test_check_numeric (line 171) | def test_check_numeric(self):
method test_check_not_numeric (line 177) | def test_check_not_numeric(self):
method test_check_not_list (line 184) | def test_check_not_list(self):
method test_check_1D_list (line 193) | def test_check_1D_list(self):
method test_check_1D_list_when_2D (line 199) | def test_check_1D_list_when_2D(self):
method test_invalid_sheet_name (line 209) | def test_invalid_sheet_name(self):
method test_set_single_cell (line 220) | def test_set_single_cell(self):
method test_set_single_cell_list_of_data (line 226) | def test_set_single_cell_list_of_data(self):
method test_set_cell_range_columnn (line 238) | def test_set_cell_range_columnn(self):
method test_set_cell_range_row (line 247) | def test_set_cell_range_row(self):
method test_set_cell_range_2D (line 256) | def test_set_cell_range_2D(self):
method test_set_cell_range_2D_incorrect_data (line 265) | def test_set_cell_range_2D_incorrect_data(self):
method test_get_sheet_names (line 281) | def test_get_sheet_names(self):
method test_save_spreadsheet (line 285) | def test_save_spreadsheet(self):
method test_save_spreadsheet_no_lock (line 297) | def test_save_spreadsheet_no_lock(self):
FILE: tests/test_monitor.py
class TestMonitor (line 21) | class TestMonitor(unittest.TestCase):
method setUpClass (line 23) | def setUpClass(cls):
method tearDownClass (line 37) | def tearDownClass(cls):
method setUp (line 43) | def setUp(self):
method tearDown (line 49) | def tearDown(self):
method test_unload_spreadsheet (line 52) | def test_unload_spreadsheet(self):
method test_check_added_already_exists (line 66) | def test_check_added_already_exists(self):
method test_check_removed_when_renamed (line 78) | def test_check_removed_when_renamed(self):
method test_change_file_hash (line 105) | def test_change_file_hash(self):
Condensed preview — 27 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (94K chars).
[
{
"path": ".coveragerc",
"chars": 69,
"preview": "[report]\nomit =\n */venv/*\n /usr/lib/libreoffice/program/uno.py\n"
},
{
"path": ".dockerignore",
"chars": 43,
"preview": ".*\nvenv\nlog\nsaved_spreadsheets\nspreadsheets"
},
{
"path": ".editorconfig",
"chars": 681,
"preview": "# EditorConfig is awesome: http://EditorConfig.org\n\n# top-most EditorConfig file\nroot = true\n\n# Unix-style newlines with"
},
{
"path": ".flake8",
"chars": 281,
"preview": "# This is an example .flake8 config, used when developing *Black* itself.\n# Keep in sync with setup.cfg which is used fo"
},
{
"path": ".gitignore",
"chars": 124,
"preview": "venv\n.#*\n\\#*\n*~\n__pycache__\n.~lock*\nspreadsheets/*\nlog/*.log\n*.pyc\nsaved_spreadsheets/*\n.coverage\nhtmlcov\n.vscode\n/todo."
},
{
"path": "COPYING",
"chars": 18092,
"preview": " GNU GENERAL PUBLIC LICENSE\n Version 2, June 1991\n\n Copyright (C) 1989, 1991 Fr"
},
{
"path": "Dockerfile",
"chars": 661,
"preview": "FROM debian:sid-slim\n\nRUN apt-get update && \\\n apt-get upgrade -y && \\\n apt-get install -y --no-install-recommends"
},
{
"path": "README.md",
"chars": 3670,
"preview": "# spreadsheet_server\n\n## Introduction\n\nspreadsheet_server was built to aid rapid web tool development where the logic\nwa"
},
{
"path": "client.py",
"chars": 5681,
"preview": "# Copyright (C) 2016 Robert Scott\n\n# This program is free software; you can redistribute it and/or\n# modify it under the"
},
{
"path": "connection.py",
"chars": 12952,
"preview": "# Copyright (C) 2016 Robert Scott\n\n# This program is free software; you can redistribute it and/or\n# modify it under the"
},
{
"path": "coverage.sh",
"chars": 177,
"preview": "#!/bin/bash\n\n# Run the coverage tests and genereate the html reports\n\n#coverage run -m tests.test_connection && coverage"
},
{
"path": "docker_run.sh",
"chars": 512,
"preview": "#!/bin/bash\nsudo docker build . -t spreadsheet_server\nsudo docker stop spreadsheet_server\nsudo docker rm spreadsheet_ser"
},
{
"path": "example_client.py",
"chars": 2467,
"preview": "# Copyright (C) 2016 Robert Scott\n\n# This program is free software; you can redistribute it and/or\n# modify it under the"
},
{
"path": "log/.gitignore",
"chars": 72,
"preview": "# Ignore everything in this directory\n*\n# Except this file\n!.gitignore\n\n"
},
{
"path": "monitor.py",
"chars": 5561,
"preview": "# Copyright (C) 2016 Robert Scott\n\n# This program is free software; you can redistribute it and/or\n# modify it under the"
},
{
"path": "pyproject.toml",
"chars": 173,
"preview": "[tool.black]\nline-length = 79\ninclude = '\\.pyi?$'\nexclude = '''\n/(\n \\.git\n | \\.hg\n | \\.mypy_cache\n | \\.tox\n | \\.v"
},
{
"path": "request_handler.py",
"chars": 6908,
"preview": "# Copyright (C) 2016 Robert Scott\n\n# This program is free software; you can redistribute it and/or\n# modify it under the"
},
{
"path": "requirements.txt",
"chars": 25,
"preview": "pyoo==1.4\nWerkzeug==3.1.3"
},
{
"path": "saved_spreadsheets/.gitignore",
"chars": 72,
"preview": "# Ignore everything in this directory\n*\n# Except this file\n!.gitignore\n\n"
},
{
"path": "server.py",
"chars": 9142,
"preview": "# Copyright (C) 2016 Robert Scott\n\n# This program is free software; you can redistribute it and/or\n# modify it under the"
},
{
"path": "spreadsheets/.gitignore",
"chars": 72,
"preview": "# Ignore everything in this directory\n*\n# Except this file\n!.gitignore\n\n"
},
{
"path": "tests/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/context.py",
"chars": 291,
"preview": "import os\nimport sys\n\nsys.path.insert(0, os.path.abspath(\"..\"))\n\nfrom connection import SpreadsheetConnection\nfrom serve"
},
{
"path": "tests/test_client.py",
"chars": 6393,
"preview": "import unittest\nfrom .context import SpreadsheetServer, SpreadsheetClient\nfrom time import sleep\nimport os\nimport shutil"
},
{
"path": "tests/test_connection.py",
"chars": 9801,
"preview": "import os\nimport shutil\nimport threading\nimport unittest\nfrom signal import SIGTERM\n\nfrom .context import SpreadsheetCon"
},
{
"path": "tests/test_monitor.py",
"chars": 5173,
"preview": "import os\nimport shutil\nimport unittest\nfrom time import sleep\n\nfrom .context import MonitorThread, SpreadsheetClient, S"
}
]
// ... and 1 more files (download for full content)
About this extraction
This page contains the full source code of the robsco-git/spreadsheet_server GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 27 files (87.0 KB), approximately 20.7k tokens, and a symbol index with 144 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.