Showing preview only (627K chars total). Download the full file or copy to clipboard to get everything.
Repository: gobicycle/bicycle
Branch: master
Commit: 2af50e7a7295
Files: 62
Total size: 600.6 KB
Directory structure:
gitextract_x1h1gsmv/
├── .github/
│ └── workflows/
│ └── go.yml
├── Dockerfile
├── LICENSE
├── Makefile
├── README.md
├── api/
│ ├── example.http
│ ├── handlers.go
│ ├── http-client.env.json
│ └── middleware.go
├── audit/
│ └── log.go
├── blockchain/
│ ├── blockchain.go
│ ├── blockchain_test.go
│ ├── limited_client.go
│ └── shard_tracker.go
├── cmd/
│ ├── processor/
│ │ └── main.go
│ ├── testutil/
│ │ ├── http.go
│ │ ├── main.go
│ │ ├── metrics.go
│ │ └── utils.go
│ └── testwebhook/
│ └── main.go
├── config/
│ └── config.go
├── core/
│ ├── block_scanner.go
│ ├── models.go
│ ├── proxy.go
│ ├── wallets.go
│ └── withdrawal_processor.go
├── db/
│ ├── db.go
│ ├── db_test.go
│ └── tests/
│ ├── get-jetton-internal-withdrawal-tasks/
│ │ └── 01_data.up.sql
│ ├── get-ton-internal-withdrawal-tasks/
│ │ └── 01_data.up.sql
│ └── set-expired/
│ └── 01_data.up.sql
├── deploy/
│ ├── db/
│ │ ├── 01_init.down.sql
│ │ ├── 01_init.up.sql
│ │ └── 02_create_readonly_user.sh
│ ├── grafana/
│ │ ├── main/
│ │ │ ├── dashboards/
│ │ │ │ └── Payments.json
│ │ │ └── provisioning/
│ │ │ ├── dashboards/
│ │ │ │ └── payments.yml
│ │ │ └── datasources/
│ │ │ └── data_sources.yml
│ │ └── test/
│ │ ├── dashboards/
│ │ │ ├── Processor A.json
│ │ │ ├── Processor B.json
│ │ │ └── Test util.json
│ │ └── provisioning/
│ │ ├── dashboards/
│ │ │ └── payments.yml
│ │ └── datasources/
│ │ └── data_sources.yml
│ ├── manual_migrations/
│ │ ├── 0.1.x-0.2.0.sql
│ │ └── 0.4.x-0.5.0.sql
│ └── prometheus/
│ ├── main/
│ │ └── prometheus.yml
│ └── test/
│ └── prometheus.yml
├── docker-compose.yml
├── docs/
│ ├── api.apib
│ └── index.html
├── go.mod
├── go.sum
├── jettons.md
├── manual_migrations.md
├── manual_testing_plan.md
├── metrics/
│ └── metrics.go
├── queue/
│ └── queue.go
├── release_notes.md
├── technical_notes.md
├── tests/
│ └── docker-compose-tests.yml
├── threat_model.md
├── todo_list.md
└── webhook/
└── webhook.go
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/workflows/go.yml
================================================
name: Go
on:
push:
branches: [ "master" ]
workflow_dispatch: {}
jobs:
build:
runs-on: ubuntu-24.04
environment: TESTS
services:
postgres:
image: postgres
env:
POSTGRES_DB: payment_processor
POSTGRES_USER: pp_user
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- uses: actions/checkout@v3
with:
persist-credentials: false # otherwise, the token used is the GITHUB_TOKEN, instead of your personal access token.
fetch-depth: 0 # otherwise, there would be errors pushing refs to the destination repository.
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: 1.24.0
- name: Build
run: go build -v ./...
- uses: actions/cache@v4
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-2go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-2go-
- name: Install dependencies
run: |
sudo apt-get update
sudo apt-get -y install openssl ca-certificates libsodium23
- name: Install libs
run: |
wget -O libemulator.so https://github.com/ton-blockchain/ton/releases/download/v2025.06/libemulator-linux-x86_64.so
sudo cp libemulator.so /lib
- name: Update Library Cache
run: sudo ldconfig
- name: Verify Library Presence
run: ls /lib | grep libemulator.so
- name: Run Test
env:
SEED: ${{ secrets.SEED }}
SERVER: ${{ secrets.SERVER }}
KEY: ${{ secrets.KEY }}
DB_URI: postgresql://pp_user:postgres@localhost:5432/payment_processor?sslmode=disable
run: |
go test -v $(go list ./...)
================================================
FILE: Dockerfile
================================================
FROM docker.io/library/golang:1.24-bookworm AS builder
WORKDIR /build-dir
COPY go.mod .
COPY go.sum .
RUN go mod download all
COPY api api
COPY blockchain blockchain
COPY cmd cmd
COPY config config
COPY core core
COPY db db
COPY audit audit
COPY queue queue
COPY webhook webhook
COPY metrics metrics
RUN apt-get update && apt-get install -y libsodium23
ARG GIT_TAG
RUN go build -ldflags "-X main.Version=$GIT_TAG" -o /tmp/processor github.com/gobicycle/bicycle/cmd/processor
RUN go build -ldflags "-X main.Version=$GIT_TAG" -o /tmp/testutil github.com/gobicycle/bicycle/cmd/testutil
FROM docker.io/library/ubuntu:24.04 AS payment-processor
RUN apt-get update && apt-get install -y openssl ca-certificates libsodium23 wget && rm -rf /var/lib/apt/lists/*
RUN mkdir -p /app/lib
COPY --from=builder /go/pkg/mod/github.com/tonkeeper/tongo*/lib/linux /app/lib/
ENV LD_LIBRARY_PATH=/app/lib
COPY --from=builder /tmp/processor /app/processor
CMD ["/app/processor", "-v"]
FROM docker.io/library/ubuntu:24.04 AS payment-test
RUN apt-get update && apt-get install -y openssl ca-certificates libsodium23 wget && rm -rf /var/lib/apt/lists/*
RUN mkdir -p /app/lib
COPY --from=builder /go/pkg/mod/github.com/tonkeeper/tongo*/lib/linux /app/lib/
ENV LD_LIBRARY_PATH=/app/lib
COPY --from=builder /tmp/testutil /app/testutil
CMD ["/app/testutil", "-v"]
================================================
FILE: LICENSE
================================================
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
<program> Copyright (C) <year> <name of author>
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<https://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<https://www.gnu.org/licenses/why-not-lgpl.html>.
================================================
FILE: Makefile
================================================
VERSION := latest
GIT_TAG := $(shell git describe --tags --always)
build:
@echo "Building tag: $(GIT_TAG)"
docker build --build-arg GIT_TAG=$(GIT_TAG) -t payment-processor:$(VERSION) --target payment-processor .
docker build --build-arg GIT_TAG=$(GIT_TAG) -t payment-test:$(VERSION) --target payment-test .
================================================
FILE: README.md
================================================
# TON payment processor
[![Based on TON][ton-svg]][ton]
[](https://github.com/gobicycle/bicycle/actions/workflows/go.yml)
[![Telegram][telegram-svg]][telegram-url]
Microservice for accepting payments and making withdrawals to wallets in TON blockchain.
Supports TON coins and Jettons (conforming certain criteria)
Provides REST API for integration.
Service is ADNL based and interacts directly with node and do not use any third party API.
**Warning** Manual withdrawals from hot wallet is prohibited **do not withdraw funds from a hot wallet bypassing the service, this will lead to a service error**
- [How it works](#How-it-works)
- [Features](#Features)
- [Glossary](#Glossary)
- [Prerequisites](#Prerequisites)
- [Criteria for valid Jettons](#Criteria-for-valid-Jettons)
- [Deployment](#Deployment)
- [Configurable parameters](#Configurable-parameters)
- [Service deploy](#Service-deploy)
- [Payment notifications](#Payment-notifications)
- [Binary comment support](#Binary-comment-support)
- [REST API](https://gobicycle.github.io/bicycle/)
- [Technical notes](/technical_notes.md)
- [Threat model](/threat_model.md)
- [Manual migrations](/manual_migrations.md)
- [TODO list](/todo_list.md)
## How it works
The service provides the following functionality:
- `generate new deposit address` of wallet (TON or Jetton) for TON blockchain. This address you provide to customer for payment. Payments are accumulated in the hot-wallet.
- `make withdrawal` of TONs or Jettons from hot-wallet to customer wallet at TON blockchain.
### Features
* Deposit addresses can be reused for multiple payments
* Sends withdrawals with comment
* Sends withdrawals in batches using highload wallet
* Aggregates part of TONs or Jettons at cold-wallet
* Supports authorization by Bearer token
* Service withdrawals (cancellation of incorrect payments)
## Glossary
- `deposit-address` - address generated by the service to which users send payments.
- `deposit` - blockchain account with `deposit-address`
- `hot-wallet` - wallet for aggregation all incoming TONs and Jettons from deposit-addresses.
- `cold-wallet` - wallet to which part of the funds from the hot wallet is sent for security. Cold-wallet seed phrase is not used by the service.
- `user_id` - unique text value to identify deposit-addresses or withdrawal request for a specific user.
- `query_id` - unique text value to identify withdrawal request for a specific user to prevent double spending.
- `basic unit` - minimum indivisible unit for TON (e.g. for TON `basic unit` = nanoTONs) or Jetton.
- `hot_wallet_minimum_balance` - minimum TON balance in nanoTONs at hot wallet to start service.
- `hot_wallet_maximum_balance` - maximum balance (of TONs or Jettons) in basic units at hot wallet. Anything more than this amount will be withdrawn to a cold wallet.
- `minimum_withdrawal_amount` - minimum balance (of TONs or Jettons) in basic units at deposit account to make withdrawal to hot wallet. It is necessary to prevent the case when the withdrawal fee will be close to the balance on the deposit.
## Prerequisites
- Need minimum (configured) amount of TONs at HighloadV2 wallet address correlated with seed phrase or already deployed HighloadV2 wallet.
- To ensure the reliability and security of the service, you need to provide your own TON node (with lite server) on the same machine as the service. If you want to use an untrusted node (such as a rented node), you need to set `PROOF_CHECK_ENABLED=true` and specify `NETWORK_CONFIG_URL`.
- Jettons used must meet certain criteria
### Criteria for valid Jettons
- conforming to the standard [TEP-74](https://github.com/ton-blockchain/TEPs/blob/master/text/0074-jettons-standard.md)
- the Jetton wallet should not spontaneously change its balance, only with transfer.
- fee for the withdrawal of Jettons from the wallet should not be too high and meet the internal setting of the service
For more information on Jettons compatibility, see [Jettons compatibility](/jettons.md)
## Deployment
### Configurable parameters
| ENV variable | Description |
|------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `LITESERVER` | IP and port of lite server, example: `185.86.76.183:5815` |
| `LITESERVER_KEY` | public key of lite server `5v2dHtSclsGsZVbNVwTj4hQDso5xvQjzL/yPEHJevHk=`. <br/>Be careful with base64 encoding and ENV var. Use '' |
| `LITESERVER_RATE_LIMIT` | If you have a rented node with an RPS limit, set the RPS value here equal to (or preferably slightly less than) the limit. Default: 100. |
| `SEED` | seed phrase for main hot wallet. 24 words compatible with standard TON wallets |
| `DB_URI` | URI for DB connection, example: <br/>`postgresql://db_user:db_password@localhost:5432/payment_processor` |
| `POSTGRES_DB` | name of database for storing payments data |
| `POSTGRES_READONLY_PASSWORD` | password for grafana readonly db user |
| `API_PORT` | port for REST API, example `8081` |
| `API_TOKEN` | Bearer token for REST API, example `123` |
| `IS_TESTNET` | `true` if service works in TESTNET, `false` - for MAINNET. Default: `true`. |
| `JETTONS` | list of Jettons, processed by service in format: <br/>`JETTON_SYMBOL_1:MASTER_CONTRACT_ADDR_1:hot_wallet_max_balance:min_withdrawal_amount:hot_wallet_residual_balance, JETTON_SYMBOL_2:MASTER_CONTRACT_ADDR_2:hot_wallet_max_balance:min_withdrawal_amount:hot_wallet_residual_balance`, <br/>example: `TGR:kQCKt2WPGX-fh0cIAz38Ljd_OKQjoZE_cqk7QrYGsNP6wfP0:1000000:100000` |
| `TON_CUTOFFS` | cutoffs in nanoTONs in format: <br/>`hot_wallet_min_balance:hot_wallet_max_balance:min_withdrawal_amount:hot_wallet_residual_balance`, <br/> example `1000000000:100000000000:1000000000:95000000000` |
| `COLD_WALLET` | cold-wallet address, example `kQCdyiS-fIV9UVfI9Phswo4l2MA-hm8YseH3XZ_YiH9Y1ufw`. If cold wallet is not active - use non-bounceable address (use https://ton.org/address for convert) |
| `DEPOSIT_SIDE_BALANCE` | `true` - service calculates total income for user by deposit incoming, `false` - by hot wallet incoming. Default: `true`. |
| `QUEUE_ENABLED` | `true` - service sends incoming notifications to queue, `false` - sending disabled. Default: `false`. |
| `QUEUE_URI` | URI for queue client connection, example `amqp://guest:guest@payment_rabbitmq:5672/` |
| `QUEUE_NAME` | name of exchange |
| `WEBHOOK_ENDPOINT` | endpoint to send webhooks, example: `http://hostname:3333/webhook`. If the value is not set, then webhooks are not sent. |
| `WEBHOOK_TOKEN` | Bearer token for webhook request. If not set then not used. |
| `ALLOWABLE_LAG` | allowable time lag between service time and last block time in seconds, default: 15 |
| `FORWARD_TON_AMOUNT` | forward ton amount for jetton withdrawals, default: 1 nanoton |
| `PROOF_CHECK_ENABLED` | enable verification of all proofs to securely connect to an untrusted node, default: `false`. Also you need to define `NETWORK_CONFIG_URL`. |
| `NETWORK_CONFIG_URL` | the path to load the network configuration to get the trusted key block from it. This is necessary for proof verification, example: `https://ton.org/global.config.json` |
**! Be careful with `IS_TESTNET` variable.** This does not guarantee that a testnet node is being used. It is only for address checking purposes.
There are also internal service settings (fees and timeouts) that are specified in the source code in the [Config](/config/config.go) package.
Calibration parameters recommendations in [Technical notes](/technical_notes.md).
#### `hot_wallet_residual_balance` and `hot_wallet_max_balance`
In order to avoid triggering a withdrawal to a cold wallet with each receipt of funds, a hysteresis is introduced.
`hot_wallet_max_balance` - this is the amount at which the withdrawal from the hot wallet to the cold one will be triggered
`hot_wallet_residual_balance` is the amount that will remain on the hot wallet after the withdrawal
`hot_wallet_max_balance` must be greater than `hot_wallet_residual_balance`
If the `hot_wallet_residual_balance` is not set, then it is calculated using the formula:
`hot_wallet_residual_balance` = `hot_wallet_max_balance` * `hysteresis`, where hysteresis is a hardcoded value
(at the time of writing this is 0.95)
### Service deploy
**Do not use same `.env` file for `payment-processor` and other services!**
1. Build docker images from makefile
```console
make -f Makefile
```
2. Prepare `.env` file for `payment-postgres` service or fill environment variables in `docker-compose.yml` file.
Database scheme automatically init.
```console
docker-compose -f docker-compose.yml up -d payment-postgres
```
3. Prepare `.env` file for `payment-processor` service or fill environment variables in `docker-compose.yml` file.
```console
docker-compose -f docker-compose.yml up -d payment-processor
```
4. Optionally you can start Grafana for service monitoring. Prepare `.env` file for `payment-grafana` service or
fill environment variables in `docker-compose.yml` file.
```console
docker-compose -f docker-compose.yml up -d payment-grafana
```
5. Optionally you can start RabbitMQ to collect payment notifications (if `QUEUE_ENABLED` env var is `true` for payment-processor).
Prepare `.env` file for `payment-rabbitmq` service or fill environment variables in `docker-compose.yml` file.
```console
docker-compose -f docker-compose.yml up -d payment-rabbitmq
```
## Payment notifications
ATTENTION! Sending notifications does not guarantee that all notifications will be sent.
If the service is restarted after the data is saved to the database and before the notification data is sent,
these notifications will not be sent after restart.
The service has several mechanisms for notification of payments. These are webhooks and a AMQP (to RabbitMQ).
Depending on the `DEPOSIT_SIDE_BALANCE` setting, a notification is received either about the payment to the
deposit address, or about the withdrawal from the deposit to the hot wallet. Source address and comment returned if known.
Message format when `DEPOSIT_SIDE_BALANCE` == true:
```json
{
"deposit_address":"0QCdsj-u39qVlfYdpPKuAY0hTe5VIsiJcpB5Rx4tOUOyBFhL",
"time": 12345678,
"amount":"100",
"source_address":"0QAOp2OZwWdkF5HhJ0WVDspgh6HhpmHyQ3cBuBmfJ4q_AIVe",
"comment":"hello",
"tx_hash": "f9b9e7efd3a38da318a894576499f0b6af5ca2da97ccd15c5f1d291a808a0ebf",
"user_id": "123"
}
```
Message format when `DEPOSIT_SIDE_BALANCE` == false (there is fewer data, because the accumulated amount is withdrawn
from the deposit):
```json
{
"deposit_address":"0QCdsj-u39qVlfYdpPKuAY0hTe5VIsiJcpB5Rx4tOUOyBFhL",
"time": 12345678,
"amount":"200",
"tx_hash": "f9b9e7efd3a38da318a894576499f0b6af5ca2da97ccd15c5f1d291a808a0ebf",
"user_id": "123"
}
```
### Using RabbitMQ
1. Set `QUEUE_ENABLED = true` env variable
2. Set `QUEUE_URI` as described at [Configurable parameters](#Configurable-parameters)
3. Set `QUEUE_NAME` env variable. Be careful, this is not the name of a specific queue in the rabbit, but the name of
the exchange.
4. Start RabbitMQ as described at [Service deploy](#Service-deploy)
### Using webhooks
1. Set `WEBHOOK_ENDPOINT` env variable
2. Optionally set `WEBHOOK_TOKEN` env variable
When the `payment-processor` is running, it will send a `POST` request to the webhook endpoint with each payment and
wait for a response with a `200` code and an empty body. If a successful delivery response is not received after
several attempts, the service will stop with an error. If the variable `WEBHOOK_TOKEN` is set, it will also
add header `Authorization: Bearer {token}`.
## Binary comment support
Method `/v1/withdrawal/send` also supports `binary_comment`. The comment is written in a hex form. If the bits qty is not
a multiple of a byte, then the record form with a flip bit is supported, for example `9fe7_`.
A `binary_comment` is writing directly to the body of the message (for the TON transfer) and to the `forward_payload`
(for the Jetton transfer) with its opcode according to the following TLB scheme:
`binary_comment#b3ddcf7d {n:#} data:(SnakeData ~n) = InternalMsgBody;`
`crc32('binary_comment n:# data:SnakeData ~n = InternalMsgBody') = 0xb3ddcf7d`
This comment will not be displayed by the explorer as text and can be useful for transmitting metadata that
will be read by indexers.
The documentation [contains](https://docs.ton.org/develop/smart-contracts/guidelines/internal-messages#simple-message-with-comment) a standard way of writing a binary comment, but due to the fact that it is not supported
by services, an alternative recording method was chosen.
<!-- Badges -->
[ton-svg]: https://img.shields.io/badge/Based%20on-TON-blue
[ton]: https://ton.org
[telegram-url]: https://t.me/tonbicycle
[telegram-svg]: https://img.shields.io/badge/telegram-chat-blue?color=blue&logo=telegram&logoColor=white
================================================
FILE: api/example.http
================================================
###
POST {{url}}/v1/address/new
Authorization: Bearer {{token}}
Content-Type: application/json
{"user_id": "TestUser", "currency": "TON"}
###
POST {{url}}/v1/address/new
Authorization: Bearer {{token}}
Content-Type: application/json
{"user_id": "TestUser", "currency": "TGR"}
###
GET {{url}}/v1/address/all?user_id=TestUser
Authorization: Bearer {{token}}
###
GET {{url}}/v1/income?user_id=TestUser
Authorization: Bearer {{token}}
###
GET {{url}}/v1/deposit/history?user_id=TestUser¤cy=TON&limit=3&offset=0&sort_order=asc
Authorization: Bearer {{token}}
###
POST {{url}}/v1/withdrawal/send
Authorization: Bearer {{token}}
Content-Type: application/json
{"user_id": "TestUser", "query_id": "1", "currency": "TON", "amount": 200000000, "destination": "kQBFETbGASx3-6QYpPuQAKQM1s32AfSkWzbsADqt3bKDlN1A", "comment": "test_ton_withdrawal"}
###
POST {{url}}/v1/withdrawal/send
Authorization: Bearer {{token}}
Content-Type: application/json
{"user_id": "TestUser", "query_id": "2", "currency": "TGR", "amount": 1000, "destination": "kQBFETbGASx3-6QYpPuQAKQM1s32AfSkWzbsADqt3bKDlN1A", "comment": "test_jetton_withdrawal"}
###
POST {{url}}/v1/withdrawal/send
Authorization: Bearer {{token}}
Content-Type: application/json
{"user_id": "TestUser", "query_id": "3", "currency": "TON", "amount": 1000, "destination": "kQBFETbGASx3-6QYpPuQAKQM1s32AfSkWzbsADqt3bKDlN1A", "binary_comment": "9fe7_"}
###
POST {{url}}/v1/withdrawal/service/jetton
Authorization: Bearer {{token}}
Content-Type: application/json
{"owner": "0QCdsj-u39qVlfYdpPKuAY0hTe5VIsiJcpB5Rx4tOUOyBFhL", "jetton_master": "kQAbMQzuuGiCne0R7QEj9nrXsjM7gNjeVmrlBZouyC-SCALE"}
###
POST {{url}}/v1/withdrawal/service/ton
Authorization: Bearer {{token}}
Content-Type: application/json
{"from": "0QAOp2OZwWdkF5HhJ0WVDspgh6HhpmHyQ3cBuBmfJ4q_AIVe"}
###
GET {{url}}/v1/withdrawal/status?id=2
Authorization: Bearer {{token}}
###
GET {{url}}/v1/system/sync
###
GET {{url}}/metrics
###
GET {{url}}//v1/deposit/income?tx_hash=54e61136c33b94372030de8c7d02bc23a60e3de7cfad46f26258e8e722dc66b1
Authorization: Bearer {{token}}
###
GET {{url}}//v1/balance?currency=TON&address=kQAbMQzuuGiCne0R7QEj9nrXsjM7gNjeVmrlBZouyC-SCALE
Authorization: Bearer {{token}}
###
GET {{url}}//v1/balance?currency=TON
Authorization: Bearer {{token}}
###
GET {{url}}//v1/resolve?domain=wallet.ton
Authorization: Bearer {{token}}
================================================
FILE: api/handlers.go
================================================
package api
import (
"bytes"
"context"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"github.com/gobicycle/bicycle/config"
"github.com/gobicycle/bicycle/core"
"github.com/gobicycle/bicycle/metrics"
"github.com/gofrs/uuid"
"github.com/shopspring/decimal"
log "github.com/sirupsen/logrus"
"github.com/tonkeeper/tongo/boc"
"github.com/xssnick/tonutils-go/address"
"github.com/xssnick/tonutils-go/tlb"
"github.com/xssnick/tonutils-go/ton/wallet"
"math/big"
"net/http"
"strconv"
"strings"
"sync"
)
type Handler struct {
storage storage
blockchain blockchain
token string
shard byte
mutex sync.Mutex
hotWalletAddress address.Address
}
type WithdrawalRequest struct {
UserID string `json:"user_id"`
QueryID string `json:"query_id"`
Currency string `json:"currency"`
Amount core.Coins `json:"amount"`
Destination string `json:"destination"`
Comment string `json:"comment"`
BinaryComment string `json:"binary_comment"`
}
type ServiceTonWithdrawalRequest struct {
From string `json:"from"`
}
type ServiceJettonWithdrawalRequest struct {
Owner string `json:"owner"`
JettonMaster string `json:"jetton_master"`
}
type WalletAddress struct {
Address string `json:"address"`
Currency string `json:"currency"`
}
type GetAddressesResponse struct {
Addresses []WalletAddress `json:"addresses"`
}
type WithdrawalResponse struct {
ID int64 `json:"ID"`
}
type GetBalanceResponse struct {
Balance string `json:"balance"`
Status string `json:"status,omitempty"`
ProcessingAmount string `json:"total_processing_amount,omitempty"`
PendingAmount string `json:"total_pending_amount,omitempty"`
}
type ResolveResponse struct {
Address string `json:"address"`
}
type WithdrawalStatusResponse struct {
UserID string `json:"user_id"`
QueryID string `json:"query_id"`
Status core.WithdrawalStatus `json:"status"`
TxHash string `json:"tx_hash,omitempty"`
}
type GetIncomeResponse struct {
Side string `json:"counting_side"`
TotalIncomes []totalIncome `json:"total_income"`
}
type GetHistoryResponse struct {
Incomes []income `json:"incomes"`
}
type GetIncomeByTxResponse struct {
Currency string `json:"currency"`
Income income `json:"income"`
}
type totalIncome struct {
Address string `json:"deposit_address"`
Amount string `json:"amount"`
Currency string `json:"currency"`
}
type income struct {
DepositAddress string `json:"deposit_address"`
Time int64 `json:"time"`
SourceAddress string `json:"source_address,omitempty"`
Amount string `json:"amount"`
Comment string `json:"comment,omitempty"`
TxHash string `json:"tx_hash,omitempty"`
}
func NewHandler(s storage, b blockchain, token string, shard byte, hotWalletAddress address.Address) *Handler {
return &Handler{storage: s, blockchain: b, token: token, shard: shard, hotWalletAddress: hotWalletAddress}
}
func (h *Handler) getNewAddress(resp http.ResponseWriter, req *http.Request) {
var data struct {
UserID string `json:"user_id"`
Currency string `json:"currency"`
}
err := json.NewDecoder(req.Body).Decode(&data)
if err != nil {
writeHttpError(resp, http.StatusBadRequest, fmt.Sprintf("decode payload data err: %v", err))
return
}
if !isValidCurrency(data.Currency) {
writeHttpError(resp, http.StatusBadRequest, "invalid currency type")
return
}
h.mutex.Lock()
defer h.mutex.Unlock() // To prevent data race
addr, err := generateAddress(req.Context(), data.UserID, data.Currency, h.shard, h.storage, h.blockchain, h.hotWalletAddress)
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("generate address err: %v", err))
return
}
res := struct {
Address string `json:"address"`
}{Address: addr}
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(http.StatusOK)
err = json.NewEncoder(resp).Encode(res)
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
func (h *Handler) getAddresses(resp http.ResponseWriter, req *http.Request) {
userID := req.URL.Query().Get("user_id")
if userID == "" {
writeHttpError(resp, http.StatusBadRequest, "need to provide user ID")
return
}
addresses, err := getAddresses(req.Context(), userID, h.storage)
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("get addresses err: %v", err))
return
}
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(http.StatusOK)
err = json.NewEncoder(resp).Encode(addresses)
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
func (h *Handler) sendWithdrawal(resp http.ResponseWriter, req *http.Request) {
var body WithdrawalRequest
err := json.NewDecoder(req.Body).Decode(&body)
if err != nil {
writeHttpError(resp, http.StatusBadRequest, fmt.Sprintf("decode payload err: %v", err))
return
}
w, err := convertWithdrawal(body)
if err != nil {
writeHttpError(resp, http.StatusBadRequest, fmt.Sprintf("convert withdrawal err: %v", err))
return
}
unique, err := h.storage.IsWithdrawalRequestUnique(req.Context(), w)
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("check withdrawal uniquess err: %v", err))
return
} else if !unique {
writeHttpError(resp, http.StatusBadRequest, "(user_id,query_id) not unique")
return
}
_, ok := h.storage.GetWalletType(w.Destination)
if ok {
writeHttpError(resp, http.StatusBadRequest, "withdrawal to service internal addresses not supported")
return
}
id, err := h.storage.SaveWithdrawalRequest(req.Context(), w)
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("save withdrawal request err: %v", err))
return
}
r := WithdrawalResponse{ID: id}
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(http.StatusOK)
err = json.NewEncoder(resp).Encode(r)
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
func (h *Handler) getSync(resp http.ResponseWriter, req *http.Request) {
isSynced, utime, err := h.storage.IsActualBlockData(req.Context())
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("get sync from db err: %v", err))
return
}
getSyncResponse := struct {
IsSynced bool `json:"is_synced"`
BlockTime int64 `json:"last_block_gen_utime"`
}{
IsSynced: isSynced,
BlockTime: utime,
}
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(http.StatusOK)
err = json.NewEncoder(resp).Encode(getSyncResponse)
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
func (h *Handler) getWithdrawalStatus(resp http.ResponseWriter, req *http.Request) {
ids := req.URL.Query().Get("id")
if ids == "" {
writeHttpError(resp, http.StatusBadRequest, "need to provide request ID")
return
}
id, err := strconv.ParseInt(ids, 10, 64)
if err != nil {
writeHttpError(resp, http.StatusBadRequest, fmt.Sprintf("convert request ID err: %v", err))
return
}
status, err := h.storage.GetExternalWithdrawalStatus(req.Context(), id)
if errors.Is(err, core.ErrNotFound) {
writeHttpError(resp, http.StatusBadRequest, "request ID not found")
return
}
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("get external withdrawal status err: %v", err))
return
}
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(http.StatusOK)
res := WithdrawalStatusResponse{
UserID: status.UserID,
QueryID: status.QueryID,
Status: status.Status,
}
if status.TxHash != nil {
res.TxHash = fmt.Sprintf("%x", status.TxHash)
}
err = json.NewEncoder(resp).Encode(res)
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
func (h *Handler) getIncome(resp http.ResponseWriter, req *http.Request) {
id := req.URL.Query().Get("user_id")
if id == "" {
writeHttpError(resp, http.StatusBadRequest, "need to provide user ID")
return
}
totalIncomes, err := h.storage.GetIncome(req.Context(), id, config.Config.IsDepositSideCalculation)
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("get income err: %v", err))
return
}
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(http.StatusOK)
err = json.NewEncoder(resp).Encode(convertIncome(h.storage, totalIncomes))
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
func (h *Handler) getIncomeHistory(resp http.ResponseWriter, req *http.Request) {
id := req.URL.Query().Get("user_id")
if id == "" {
writeHttpError(resp, http.StatusBadRequest, "need to provide user ID")
return
}
currency := req.URL.Query().Get("currency")
if currency == "" {
writeHttpError(resp, http.StatusBadRequest, "need to provide currency")
return
}
if !isValidCurrency(currency) {
writeHttpError(resp, http.StatusBadRequest, "invalid currency type")
return
}
limit, err := strconv.Atoi(req.URL.Query().Get("limit"))
if err != nil {
writeHttpError(resp, http.StatusBadRequest, "invalid limit parameter")
return
}
offset, err := strconv.Atoi(req.URL.Query().Get("offset"))
if err != nil {
writeHttpError(resp, http.StatusBadRequest, "invalid offset parameter")
return
}
ascOrder := false
sort := strings.ToLower(req.URL.Query().Get("sort_order"))
if sort == "asc" {
ascOrder = true
} else if sort != "" && sort != "desc" {
writeHttpError(resp, http.StatusBadRequest, "invalid sort order")
return
}
history, err := h.storage.GetIncomeHistory(req.Context(), id, currency, limit, offset, ascOrder)
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("get history err: %v", err))
return
}
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(http.StatusOK)
err = json.NewEncoder(resp).Encode(convertHistory(h.storage, currency, history))
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
func (h *Handler) serviceTonWithdrawal(resp http.ResponseWriter, req *http.Request) {
var body ServiceTonWithdrawalRequest
err := json.NewDecoder(req.Body).Decode(&body)
if err != nil {
writeHttpError(resp, http.StatusBadRequest, fmt.Sprintf("decode payload err: %v", err))
return
}
w, err := convertTonServiceWithdrawal(h.storage, body)
if err != nil {
writeHttpError(resp, http.StatusBadRequest, fmt.Sprintf("convert service withdrawal err: %v", err))
return
}
memo, err := h.storage.SaveServiceWithdrawalRequest(req.Context(), w)
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("save service withdrawal request err: %v", err))
return
}
var response = struct {
Memo uuid.UUID `json:"memo"`
}{
Memo: memo,
}
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(http.StatusOK)
err = json.NewEncoder(resp).Encode(response)
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
func (h *Handler) serviceJettonWithdrawal(resp http.ResponseWriter, req *http.Request) {
var body ServiceJettonWithdrawalRequest
err := json.NewDecoder(req.Body).Decode(&body)
if err != nil {
writeHttpError(resp, http.StatusBadRequest, fmt.Sprintf("decode payload err: %v", err))
return
}
w, err := convertJettonServiceWithdrawal(h.storage, body)
if err != nil {
writeHttpError(resp, http.StatusBadRequest, fmt.Sprintf("convert service withdrawal err: %v", err))
return
}
memo, err := h.storage.SaveServiceWithdrawalRequest(req.Context(), w)
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("save service withdrawal request err: %v", err))
return
}
var response = struct {
Memo uuid.UUID `json:"memo"`
}{
Memo: memo,
}
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(http.StatusOK)
err = json.NewEncoder(resp).Encode(response)
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
func (h *Handler) getMetrics(resp http.ResponseWriter, req *http.Request) {
buf := new(bytes.Buffer)
for _, m := range metrics.AllMetrics {
err := m.Print(buf)
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("get metrics err: %v", err.Error()))
return
}
}
resp.Header().Add("Content-Type", "application/text")
resp.WriteHeader(http.StatusOK)
_, err := resp.Write(buf.Bytes())
if err != nil {
log.Errorf("buffer writing error: %v", err)
}
}
func (h *Handler) getIncomeByTx(resp http.ResponseWriter, req *http.Request) {
txHash := strings.ToLower(req.URL.Query().Get("tx_hash"))
hash, err := hex.DecodeString(txHash)
if err != nil {
writeHttpError(resp, http.StatusBadRequest, fmt.Sprintf("get tx hash err: %v", err.Error()))
return
}
if len(hash) != 32 {
writeHttpError(resp, http.StatusBadRequest, "invalid hash len")
return
}
oneIncome, currency, err := h.storage.GetIncomeByTx(req.Context(), hash)
if errors.Is(err, core.ErrNotFound) {
writeHttpError(resp, http.StatusNotFound, "transaction not found")
return
}
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("get income by tx err: %v", err))
return
}
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(http.StatusOK)
res := GetIncomeByTxResponse{Currency: currency, Income: convertOneIncome(h.storage, currency, *oneIncome)}
err = json.NewEncoder(resp).Encode(res)
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
func (h *Handler) getBalance(resp http.ResponseWriter, req *http.Request) {
currency := req.URL.Query().Get("currency")
if currency == "" {
writeHttpError(resp, http.StatusBadRequest, "need to provide currency")
return
}
if !isValidCurrency(currency) {
writeHttpError(resp, http.StatusBadRequest, "invalid currency type")
return
}
var (
tonWalletAddress core.Address
err error
balance *big.Int
status tlb.AccountStatus
res GetBalanceResponse
)
addr := req.URL.Query().Get("address")
if addr != "" {
tonWalletAddress, _, err = validateAddress(addr)
if err != nil {
writeHttpError(resp, http.StatusBadRequest, fmt.Sprintf("invalid address: %s", err.Error()))
return
}
} else {
tonWalletAddress = core.AddressMustFromTonutilsAddress(&h.hotWalletAddress)
amounts, err := h.storage.GetTotalWithdrawalAmounts(req.Context(), currency)
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("get total withdrawal amounts err: %v", err))
return
}
res.PendingAmount = amounts.Pending.String()
res.ProcessingAmount = amounts.Processing.String()
}
if currency == core.TonSymbol {
balance, status, err = h.blockchain.GetAccountCurrentState(req.Context(), tonWalletAddress.ToTonutilsAddressStd(0))
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("get TON balance err: %v", err))
return
}
res.Status = strings.ToLower(string(status))
} else {
jetton, _ := config.Config.Jettons[currency] // currency validate earlier
balance, err = h.blockchain.GetJettonBalanceByOwner(req.Context(), tonWalletAddress.ToTonutilsAddressStd(0), jetton.Master)
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("get jetton balance err: %v", err))
return
}
}
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(http.StatusOK)
res.Balance = balance.String()
err = json.NewEncoder(resp).Encode(res)
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
func (h *Handler) getResolve(resp http.ResponseWriter, req *http.Request) {
domain := req.URL.Query().Get("domain")
if domain == "" {
writeHttpError(resp, http.StatusBadRequest, "invalid domain")
return
}
addr, err := h.blockchain.DnsResolveSmc(req.Context(), domain)
if errors.Is(err, core.ErrNotFound) {
writeHttpError(resp, http.StatusNotFound, "smart contract DNS record not found")
return
}
if err != nil {
writeHttpError(resp, http.StatusInternalServerError, fmt.Sprintf("get DNS record err: %v", err))
return
}
addr.SetTestnetOnly(config.Config.Testnet)
addr.SetBounce(true)
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(http.StatusOK)
res := ResolveResponse{Address: addr.String()}
err = json.NewEncoder(resp).Encode(res)
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
func RegisterHandlers(mux *http.ServeMux, h *Handler) {
mux.HandleFunc("/v1/address/new", recoverMiddleware(authMiddleware(post(h.getNewAddress))))
mux.HandleFunc("/v1/address/all", recoverMiddleware(authMiddleware(get(h.getAddresses))))
mux.HandleFunc("/v1/withdrawal/send", recoverMiddleware(authMiddleware(post(h.sendWithdrawal))))
mux.HandleFunc("/v1/withdrawal/service/ton", recoverMiddleware(authMiddleware(post(h.serviceTonWithdrawal))))
mux.HandleFunc("/v1/withdrawal/service/jetton", recoverMiddleware(authMiddleware(post(h.serviceJettonWithdrawal))))
mux.HandleFunc("/v1/withdrawal/status", recoverMiddleware(authMiddleware(get(h.getWithdrawalStatus))))
mux.HandleFunc("/v1/system/sync", recoverMiddleware(get(h.getSync)))
mux.HandleFunc("/v1/income", recoverMiddleware(authMiddleware(get(h.getIncome))))
mux.HandleFunc("/v1/deposit/history", recoverMiddleware(authMiddleware(get(h.getIncomeHistory))))
mux.HandleFunc("/v1/deposit/income", recoverMiddleware(authMiddleware(get(h.getIncomeByTx))))
mux.HandleFunc("/v1/balance", recoverMiddleware(authMiddleware(get(h.getBalance))))
mux.HandleFunc("/v1/resolve", recoverMiddleware(authMiddleware(get(h.getResolve))))
mux.HandleFunc("/metrics", recoverMiddleware(get(h.getMetrics)))
}
func generateAddress(
ctx context.Context,
userID string,
currency string,
shard byte,
dbConn storage,
bc blockchain,
hotWalletAddress address.Address,
) (
string,
error,
) {
subwalletID, err := dbConn.GetLastSubwalletID(ctx)
if err != nil {
return "", err
}
var res string
if currency == core.TonSymbol {
w, id, err := bc.GenerateSubWallet(config.Config.Seed, shard, subwalletID+1)
if err != nil {
return "", err
}
a, err := core.AddressFromTonutilsAddress(w.Address())
if err != nil {
return "", err
}
err = dbConn.SaveTonWallet(ctx,
core.WalletData{
SubwalletID: id,
UserID: userID,
Currency: core.TonSymbol,
Type: core.TonDepositWallet,
Address: a,
},
)
if err != nil {
return "", err
}
res = a.ToUserFormat()
} else {
jetton, ok := config.Config.Jettons[currency]
if !ok {
return "", fmt.Errorf("jetton address not found")
}
proxy, addr, err := bc.GenerateDepositJettonWalletForProxy(ctx, shard, &hotWalletAddress, jetton.Master, subwalletID+1)
if err != nil {
return "", err
}
jettonWalletAddr, err := core.AddressFromTonutilsAddress(addr)
if err != nil {
return "", err
}
proxyAddr, err := core.AddressFromTonutilsAddress(proxy.Address())
if err != nil {
return "", err
}
err = dbConn.SaveJettonWallet(
ctx,
proxyAddr,
core.WalletData{
UserID: userID,
SubwalletID: proxy.SubwalletID,
Currency: currency,
Type: core.JettonDepositWallet,
Address: jettonWalletAddr,
},
false,
)
if err != nil {
return "", err
}
res = proxyAddr.ToUserFormat()
}
return res, nil
}
func getAddresses(ctx context.Context, userID string, dbConn storage) (GetAddressesResponse, error) {
var res = GetAddressesResponse{
Addresses: []WalletAddress{},
}
tonAddr, err := dbConn.GetTonWalletsAddresses(ctx, userID, []core.WalletType{core.TonDepositWallet})
if err != nil {
return GetAddressesResponse{}, err
}
jettonAddr, err := dbConn.GetJettonOwnersAddresses(ctx, userID, []core.WalletType{core.JettonDepositWallet})
if err != nil {
return GetAddressesResponse{}, err
}
for _, a := range tonAddr {
res.Addresses = append(res.Addresses, WalletAddress{Address: a.ToUserFormat(), Currency: core.TonSymbol})
}
for _, a := range jettonAddr {
res.Addresses = append(res.Addresses, WalletAddress{Address: a.Address.ToUserFormat(), Currency: a.Currency})
}
return res, nil
}
func isValidCommentLen(comment string) bool {
return len(comment) < config.MaxCommentLength
}
func isValidCurrency(cur string) bool {
if _, ok := config.Config.Jettons[cur]; ok || cur == core.TonSymbol {
return true
}
return false
}
func convertWithdrawal(w WithdrawalRequest) (core.WithdrawalRequest, error) {
if !isValidCurrency(w.Currency) {
return core.WithdrawalRequest{}, fmt.Errorf("invalid currency")
}
addr, bounceable, err := validateAddress(w.Destination)
if err != nil {
return core.WithdrawalRequest{}, fmt.Errorf("invalid destination address: %v", err)
}
if !(w.Amount.Cmp(decimal.New(0, 0)) == 1) {
return core.WithdrawalRequest{}, fmt.Errorf("amount must be > 0")
}
if w.Comment != "" && w.BinaryComment != "" {
return core.WithdrawalRequest{}, fmt.Errorf("only one type of comment can be specified (comment OR binary comment)")
}
if !isValidCommentLen(w.Comment) || !isValidCommentLen(w.BinaryComment) {
return core.WithdrawalRequest{}, fmt.Errorf("too long comment, max length allowed: %d", config.MaxCommentLength)
}
res := core.WithdrawalRequest{
UserID: w.UserID,
QueryID: w.QueryID,
Currency: w.Currency,
Amount: w.Amount,
Destination: addr,
Bounceable: bounceable,
Comment: w.Comment,
IsInternal: false,
}
if w.BinaryComment != "" {
_, err = boc.BitStringFromFiftHex(w.BinaryComment)
if err != nil {
return core.WithdrawalRequest{}, fmt.Errorf("decode binary comment error: %v", err)
}
res.BinaryComment = w.BinaryComment
}
return res, nil
}
func convertTonServiceWithdrawal(s storage, w ServiceTonWithdrawalRequest) (core.ServiceWithdrawalRequest, error) {
from, _, err := validateAddress(w.From)
if err != nil {
return core.ServiceWithdrawalRequest{}, fmt.Errorf("invalid from address: %v", err)
}
t, ok := s.GetWalletType(from)
if !ok {
return core.ServiceWithdrawalRequest{}, fmt.Errorf("unknown deposit address")
}
if t != core.JettonOwner {
return core.ServiceWithdrawalRequest{},
fmt.Errorf("service withdrawal allowed only for Jetton deposit owner")
}
return core.ServiceWithdrawalRequest{
From: from,
}, nil
}
func convertJettonServiceWithdrawal(s storage, w ServiceJettonWithdrawalRequest) (core.ServiceWithdrawalRequest, error) {
from, _, err := validateAddress(w.Owner)
if err != nil {
return core.ServiceWithdrawalRequest{}, fmt.Errorf("invalid from address: %v", err)
}
t, ok := s.GetWalletType(from)
if !ok {
return core.ServiceWithdrawalRequest{}, fmt.Errorf("unknown deposit address")
}
if t != core.JettonOwner && t != core.TonDepositWallet {
return core.ServiceWithdrawalRequest{},
fmt.Errorf("service withdrawal allowed only for Jetton deposit owner or TON deposit")
}
jetton, _, err := validateAddress(w.JettonMaster)
if err != nil {
return core.ServiceWithdrawalRequest{}, fmt.Errorf("invalid jetton master address: %v", err)
}
// currency type checks by withdrawal processor
return core.ServiceWithdrawalRequest{
From: from,
JettonMaster: &jetton,
}, nil
}
func convertIncome(dbConn storage, totalIncomes []core.TotalIncome) GetIncomeResponse {
var res = GetIncomeResponse{
TotalIncomes: []totalIncome{},
}
if config.Config.IsDepositSideCalculation {
res.Side = core.SideDeposit
} else {
res.Side = core.SideHotWallet
}
for _, b := range totalIncomes {
totIncome := totalIncome{
Amount: b.Amount.String(),
Currency: b.Currency,
}
if b.Currency == core.TonSymbol {
totIncome.Address = b.Deposit.ToUserFormat()
} else {
owner := dbConn.GetOwner(b.Deposit)
if owner == nil {
// TODO: remove fatal
log.Fatalf("can not find owner for deposit: %s", b.Deposit.ToUserFormat())
}
totIncome.Address = owner.ToUserFormat()
}
res.TotalIncomes = append(res.TotalIncomes, totIncome)
}
return res
}
func convertOneIncome(dbConn storage, currency string, oneIncome core.ExternalIncome) income {
inc := income{
Time: int64(oneIncome.Utime),
Amount: oneIncome.Amount.String(),
Comment: oneIncome.Comment,
TxHash: fmt.Sprintf("%x", oneIncome.TxHash),
}
if currency == core.TonSymbol {
inc.DepositAddress = oneIncome.To.ToUserFormat()
} else {
owner := dbConn.GetOwner(oneIncome.To)
if owner == nil {
// TODO: remove fatal
log.Fatalf("can not find owner for deposit: %s", oneIncome.To.ToUserFormat())
}
inc.DepositAddress = owner.ToUserFormat()
}
// show only std address
if len(oneIncome.From) == 32 && oneIncome.FromWorkchain != nil {
addr := address.NewAddress(0, byte(*oneIncome.FromWorkchain), oneIncome.From)
addr.SetTestnetOnly(config.Config.Testnet)
inc.SourceAddress = addr.String()
}
return inc
}
func convertHistory(dbConn storage, currency string, incomes []core.ExternalIncome) GetHistoryResponse {
var res = GetHistoryResponse{
Incomes: []income{},
}
for _, i := range incomes {
inc := convertOneIncome(dbConn, currency, i)
res.Incomes = append(res.Incomes, inc)
}
return res
}
func validateAddress(addr string) (core.Address, bool, error) {
if addr == "" {
return core.Address{}, false, fmt.Errorf("empty address")
}
a, err := address.ParseAddr(addr)
if err != nil {
return core.Address{}, false, fmt.Errorf("invalid address: %v", err)
}
if a.IsTestnetOnly() && !config.Config.Testnet {
return core.Address{}, false, fmt.Errorf("address for testnet only")
}
if a.Workchain() != core.DefaultWorkchain {
return core.Address{}, false, fmt.Errorf("address must be in %d workchain",
core.DefaultWorkchain)
}
res, err := core.AddressFromTonutilsAddress(a)
return res, a.IsBounceable(), err
}
type storage interface {
GetLastSubwalletID(ctx context.Context) (uint32, error)
SaveTonWallet(ctx context.Context, walletData core.WalletData) error
SaveJettonWallet(ctx context.Context, ownerAddress core.Address, walletData core.WalletData, notSaveOwner bool) error
GetTonWalletsAddresses(ctx context.Context, userID string, types []core.WalletType) ([]core.Address, error)
GetJettonOwnersAddresses(ctx context.Context, userID string, types []core.WalletType) ([]core.OwnerWallet, error)
SaveWithdrawalRequest(ctx context.Context, w core.WithdrawalRequest) (int64, error)
IsWithdrawalRequestUnique(ctx context.Context, w core.WithdrawalRequest) (bool, error)
IsActualBlockData(ctx context.Context) (bool, int64, error)
GetExternalWithdrawalStatus(ctx context.Context, id int64) (core.WithdrawalData, error)
GetWalletType(address core.Address) (core.WalletType, bool)
GetIncome(ctx context.Context, userID string, isDepositSide bool) ([]core.TotalIncome, error)
SaveServiceWithdrawalRequest(ctx context.Context, w core.ServiceWithdrawalRequest) (uuid.UUID, error)
GetIncomeHistory(ctx context.Context, userID string, currency string, limit int, offset int, ascOrder bool) ([]core.ExternalIncome, error)
GetOwner(address core.Address) *core.Address
GetIncomeByTx(ctx context.Context, txHash []byte) (*core.ExternalIncome, string, error)
GetTotalWithdrawalAmounts(ctx context.Context, currency string) (*core.TotalWithdrawalsAmount, error)
}
type blockchain interface {
GenerateSubWallet(seed string, shard byte, startSubWalletID uint32) (*wallet.Wallet, uint32, error)
GenerateDepositJettonWalletForProxy(
ctx context.Context,
shard byte,
proxyOwner, jettonMaster *address.Address,
startSubWalletID uint32,
) (
proxy *core.JettonProxy,
addr *address.Address,
err error,
)
GenerateDefaultWallet(seed string, isHighload bool) (*wallet.Wallet, byte, uint32, error)
GetAccountCurrentState(ctx context.Context, address *address.Address) (*big.Int, tlb.AccountStatus, error)
GetJettonBalanceByOwner(ctx context.Context, owner *address.Address, jettonMaster *address.Address) (*big.Int, error)
DnsResolveSmc(ctx context.Context, domainName string) (*address.Address, error)
}
================================================
FILE: api/http-client.env.json
================================================
{
"local": {
"url": "http://localhost:8081"
}
}
================================================
FILE: api/middleware.go
================================================
package api
import (
"crypto/subtle"
"encoding/json"
"github.com/gobicycle/bicycle/config"
log "github.com/sirupsen/logrus"
"net/http"
"runtime/debug"
"strings"
)
func recoverMiddleware(next func(http.ResponseWriter, *http.Request)) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
defer func() {
if err := recover(); err != nil {
w.WriteHeader(http.StatusInternalServerError)
log.Errorf(
"err: %v trace %v", err, debug.Stack(),
)
}
}()
next(w, r)
}
}
func authMiddleware(next func(http.ResponseWriter, *http.Request)) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
if !checkToken(r, config.Config.APIToken) {
w.WriteHeader(http.StatusUnauthorized)
return
}
next(w, r)
}
}
func get(next func(http.ResponseWriter, *http.Request)) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodGet {
writeHttpError(w, http.StatusBadRequest, "only GET method is supported")
return
}
next(w, r)
}
}
func post(next func(http.ResponseWriter, *http.Request)) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
writeHttpError(w, http.StatusBadRequest, "only POST method is supported")
return
}
next(w, r)
}
}
func checkToken(req *http.Request, token string) bool {
auth := strings.Split(req.Header.Get("authorization"), " ")
if len(auth) != 2 {
return false
}
if auth[0] != "Bearer" {
return false
}
if x := subtle.ConstantTimeCompare([]byte(auth[1]), []byte(token)); x == 1 {
return true
} // constant time comparison to prevent time attack
return false
}
func writeHttpError(resp http.ResponseWriter, status int, comment string) {
body := struct {
Error string `json:"error"`
}{
Error: comment,
}
resp.Header().Add("Content-Type", "application/json")
resp.WriteHeader(status)
err := json.NewEncoder(resp).Encode(body)
if err != nil {
log.Errorf("json encode error: %v", err)
}
}
================================================
FILE: audit/log.go
================================================
package audit
import (
"fmt"
"github.com/gobicycle/bicycle/metrics"
log "github.com/sirupsen/logrus"
"time"
)
type Severity string
const (
Error Severity = "ERROR"
Warning Severity = "WARNING"
Info Severity = "INFO"
)
type message struct {
Severity Severity
Text string
}
func pushLog(m message) {
switch m.Severity {
case Error:
log.Printf("AUDIT|%v|%v|%s", m.Severity, time.Now().Format(time.RFC1123), m.Text)
metrics.Errors.Inc()
case Warning:
log.Printf("AUDIT|%v|%v|%s", m.Severity, time.Now().Format(time.RFC1123), m.Text)
metrics.Warnings.Inc()
case Info:
log.Printf("AUDIT|%v|%v|%s", m.Severity, time.Now().Format(time.RFC1123), m.Text)
metrics.Info.Inc()
}
}
func LogTX(severity Severity, location string, hash []byte, text string) {
pushLog(message{
Severity: severity,
Text: fmt.Sprintf("%s|TX:%x|%s", location, hash, text),
})
}
func Log(severity Severity, location, event, text string) {
pushLog(message{
Severity: severity,
Text: fmt.Sprintf("%s|%s|%s", location, event, text),
})
}
================================================
FILE: blockchain/blockchain.go
================================================
package blockchain
import (
"context"
"errors"
"fmt"
"github.com/gobicycle/bicycle/config"
"github.com/gobicycle/bicycle/core"
log "github.com/sirupsen/logrus"
"github.com/tonkeeper/tongo"
"github.com/tonkeeper/tongo/boc"
tongoTlb "github.com/tonkeeper/tongo/tlb"
"github.com/tonkeeper/tongo/tvm"
"github.com/xssnick/tonutils-go/address"
"github.com/xssnick/tonutils-go/liteclient"
"github.com/xssnick/tonutils-go/tl"
"github.com/xssnick/tonutils-go/tlb"
"github.com/xssnick/tonutils-go/ton"
"github.com/xssnick/tonutils-go/ton/dns"
"github.com/xssnick/tonutils-go/ton/jetton"
"github.com/xssnick/tonutils-go/ton/wallet"
"github.com/xssnick/tonutils-go/tvm/cell"
"math"
"math/big"
"sort"
"strings"
"time"
)
type Connection struct {
client ton.APIClientWrapped
resolver *dns.Client
}
func (c *Connection) WaitForBlock(seqno uint32) ton.APIClientWrapped {
return c.client.WaitForBlock(seqno)
}
func (c *Connection) FindLastTransactionByInMsgHash(ctx context.Context, addr *address.Address, msgHash []byte, maxTxNumToScan ...int) (*tlb.Transaction, error) {
//TODO implement me
panic("implement me")
}
func (c *Connection) FindLastTransactionByOutMsgHash(ctx context.Context, addr *address.Address, msgHash []byte, maxTxNumToScan ...int) (*tlb.Transaction, error) {
//TODO implement me
panic("implement me")
}
type contract struct {
Address tongo.AccountID
Code *boc.Cell
Data *boc.Cell
}
// NewConnection creates new Blockchain connection
func NewConnection(addr, key string, rateLimit int) (*Connection, error) {
client := liteclient.NewConnectionPool()
ctx, cancel := context.WithTimeout(context.Background(), time.Second*120)
defer cancel()
err := client.AddConnection(ctx, addr, key)
if err != nil {
return nil, fmt.Errorf("connection err: %v", err.Error())
}
limitedClient := newLimitedClient(client, rateLimit)
var wrappedClient ton.APIClientWrapped
if config.Config.ProofCheckEnabled {
if config.Config.NetworkConfigUrl == "" {
return nil, fmt.Errorf("empty network config URL")
}
cfg, err := liteclient.GetConfigFromUrl(ctx, config.Config.NetworkConfigUrl)
if err != nil {
return nil, fmt.Errorf("get network config from url err: %s", err.Error())
}
wrappedClient = ton.NewAPIClient(limitedClient, ton.ProofCheckPolicySecure).WithRetry()
wrappedClient.SetTrustedBlockFromConfig(cfg)
log.Infof("Fetching and checking proofs since config init block ...")
_, err = wrappedClient.CurrentMasterchainInfo(ctx) // we fetch block just to trigger chain proof check
if err != nil {
return nil, fmt.Errorf("get masterchain info err: %s", err.Error())
}
log.Infof("Proof checks are completed")
} else {
wrappedClient = ton.NewAPIClient(limitedClient, ton.ProofCheckPolicyUnsafe).WithRetry()
}
// TODO: replace after tonutils fix
rootDNS, bcConfig, err := getConfigData(ctx, wrappedClient)
if err != nil {
return nil, fmt.Errorf("get config data err: %s", err.Error())
}
config.Config.BlockchainConfig = bcConfig
resolver := dns.NewDNSClient(wrappedClient, rootDNS)
return &Connection{
client: wrappedClient,
resolver: resolver,
}, nil
}
func getConfigData(ctx context.Context, api ton.APIClientWrapped) (*address.Address, *boc.Cell, error) {
b, err := api.CurrentMasterchainInfo(ctx)
if err != nil {
return nil, nil, fmt.Errorf("failed to get masterchain info: %w", err)
}
cfg, err := api.GetBlockchainConfig(ctx, b, 4)
if err != nil {
return nil, nil, fmt.Errorf("failed to get blockchain config: %w", err)
}
cfgDict, err := getBlockchainConfig(ctx, api.Client(), b)
if err != nil {
return nil, nil, fmt.Errorf("failed to get config dict: %w", err)
}
cfgCell, err := tlb.ToCell(cfgDict)
if err != nil {
return nil, nil, fmt.Errorf("failed to serialize blockchain config: %w", err)
}
configCell, err := boc.DeserializeBoc(cfgCell.ToBOC())
if err != nil {
return nil, nil, fmt.Errorf("failed to deserialize blockchain config: %w", err)
}
if len(configCell) != 1 {
return nil, nil, fmt.Errorf("blockchain config must conatins only one cell")
}
data := cfg.Get(4)
if data == nil {
return nil, nil, fmt.Errorf("failed to get root address from blockchain config")
}
hash, err := data.BeginParse().LoadSlice(256)
if err != nil {
return nil, nil, fmt.Errorf("failed to get root address from blockchain config 4, failed to load hash: %w", err)
}
return address.NewAddress(0, 255, hash), configCell[0], nil
}
// GenerateDefaultWallet generates HighloadV2R2 or V3R2 TON wallet with
// default subwallet_id and returns wallet, shard and subwalletID
func (c *Connection) GenerateDefaultWallet(seed string, isHighload bool) (
w *wallet.Wallet,
shard byte,
subwalletID uint32, err error,
) {
words := strings.Split(seed, " ")
if isHighload {
w, err = wallet.FromSeed(c, words, wallet.HighloadV2R2)
} else {
w, err = wallet.FromSeed(c, words, wallet.V3)
}
if err != nil {
return nil, 0, 0, err
}
return w, w.Address().Data()[0], uint32(wallet.DefaultSubwallet), nil
}
// GenerateSubWallet generates subwallet for custom shard and
// subwallet_id >= startSubWalletId and returns wallet and new subwallet_id
func (c *Connection) GenerateSubWallet(seed string, shard byte, startSubWalletID uint32) (*wallet.Wallet, uint32, error) {
words := strings.Split(seed, " ")
basic, err := wallet.FromSeed(c, words, wallet.V3)
if err != nil {
return nil, 0, err
}
for id := startSubWalletID; id < math.MaxUint32; id++ {
subWallet, err := basic.GetSubwallet(id)
if err != nil {
return nil, 0, err
}
addr, err := core.AddressFromTonutilsAddress(subWallet.Address())
if err != nil {
return nil, 0, err
}
if inShard(addr, shard) {
return subWallet, id, nil
}
}
return nil, 0, fmt.Errorf("subwallet not found")
}
// GetJettonWalletAddress generates jetton wallet address from owner and jetton master addresses
func (c *Connection) GetJettonWalletAddress(
ctx context.Context,
owner *address.Address,
jettonMaster *address.Address,
) (*address.Address, error) {
contr, err := c.getContract(ctx, jettonMaster)
if err != nil {
return nil, err
}
emulator, err := newEmulator(contr.Code, contr.Data)
if err != nil {
return nil, err
}
addr, err := getJettonWalletAddressByTVM(owner, contr.Address, emulator)
if err != nil {
return nil, err
}
res := addr.ToTonutilsAddressStd(0)
res.SetTestnetOnly(config.Config.Testnet)
return res, nil
}
func (c *Connection) GetJettonBalanceByOwner(
ctx context.Context,
owner *address.Address,
jettonMaster *address.Address,
) (*big.Int, error) {
jettonMasterClient := jetton.NewJettonMasterClient(c.client, jettonMaster)
jettonWalletClient, err := jettonMasterClient.GetJettonWallet(ctx, owner)
if err != nil {
return nil, err
}
return jettonWalletClient.GetBalance(ctx)
}
func (c *Connection) DnsResolveSmc(
ctx context.Context,
domainName string,
) (*address.Address, error) {
// TODO: it is necessary to distinguish network errors from the impossibility of resolving
domain, err := c.resolver.Resolve(ctx, domainName)
if errors.Is(err, dns.ErrNoSuchRecord) {
return nil, core.ErrNotFound
} else if err != nil {
return nil, err
}
// TODO: replace after tonutils fix
smcAddr := getWalletRecord(domain)
if smcAddr == nil {
// not wallet
return nil, core.ErrNotFound
}
return smcAddr, nil
}
func getWalletRecord(d *dns.Domain) *address.Address {
// TODO: remove after tonutils fix
rec := d.GetRecord("wallet")
if rec == nil {
return nil
}
p := rec.BeginParse()
p, err := p.LoadRef()
if err != nil {
return nil
}
category, err := p.LoadUInt(16)
if err != nil {
return nil
}
if category != 0x9fd3 { // const _CategoryContractAddr = 0x9fd3
return nil
}
addr, err := p.LoadAddr()
if err != nil {
return nil
}
return addr
}
// GenerateDepositJettonWalletForProxy
// Generates jetton wallet address for custom shard and proxy contract as owner with subwallet_id >= startSubWalletId
func (c *Connection) GenerateDepositJettonWalletForProxy(
ctx context.Context,
shard byte,
proxyOwner, jettonMaster *address.Address,
startSubWalletID uint32,
) (
proxy *core.JettonProxy,
addr *address.Address,
err error,
) {
contr, err := c.getContract(ctx, jettonMaster)
if err != nil {
return nil, nil, err
}
emulator, err := newEmulator(contr.Code, contr.Data)
if err != nil {
return nil, nil, err
}
for id := startSubWalletID; id < math.MaxUint32; id++ {
proxy, err = core.NewJettonProxy(id, proxyOwner)
if err != nil {
return nil, nil, err
}
jettonWalletAddress, err := getJettonWalletAddressByTVM(proxy.Address(), contr.Address, emulator)
if err != nil {
return nil, nil, err
}
if inShard(jettonWalletAddress, shard) {
addr = jettonWalletAddress.ToTonutilsAddressStd(0)
addr.SetTestnetOnly(config.Config.Testnet)
return proxy, addr, nil
}
}
return nil, nil, fmt.Errorf("jetton wallet address not found")
}
func (c *Connection) getContract(ctx context.Context, addr *address.Address) (contract, error) {
block, err := c.client.CurrentMasterchainInfo(ctx)
if err != nil {
return contract{}, err
}
account, err := c.WaitForBlock(block.SeqNo).GetAccount(ctx, block, addr)
if err != nil {
return contract{}, err
}
if account == nil || account.Code == nil || account.Data == nil {
return contract{}, fmt.Errorf("empty account code or data")
}
accountID, err := tongo.ParseAccountID(addr.String())
if err != nil {
return contract{}, err
}
codeCell, err := boc.DeserializeBoc(account.Code.ToBOC())
if err != nil {
return contract{}, err
}
if len(codeCell) != 1 {
return contract{}, fmt.Errorf("BOC must have only one root")
}
dataCell, err := boc.DeserializeBoc(account.Data.ToBOC())
if err != nil {
return contract{}, err
}
if len(dataCell) != 1 {
return contract{}, fmt.Errorf("BOC must have only one root")
}
return contract{
Address: accountID,
Code: codeCell[0],
Data: dataCell[0],
}, nil
}
func getJettonWalletAddressByTVM(
owner *address.Address,
jettonMaster tongo.AccountID,
emulator *tvm.Emulator,
) (core.Address, error) {
ownerAccountID, err := tongo.ParseAccountID(owner.String())
if err != nil {
return core.Address{}, err
}
slice, err := tongoTlb.TlbStructToVmCellSlice(ownerAccountID.ToMsgAddress())
if err != nil {
return core.Address{}, err
}
code, result, err := emulator.RunSmcMethod(context.Background(), jettonMaster, "get_wallet_address",
tongoTlb.VmStack{slice})
if err != nil {
return core.Address{}, err
}
if code != 0 || len(result) != 1 || result[0].SumType != "VmStkSlice" {
return core.Address{}, fmt.Errorf("tvm execution failed")
}
var msgAddress tongoTlb.MsgAddress
err = result[0].VmStkSlice.UnmarshalToTlbStruct(&msgAddress)
if err != nil {
return core.Address{}, err
}
if msgAddress.SumType != "AddrStd" {
return core.Address{}, fmt.Errorf("not std jetton wallet address")
}
if msgAddress.AddrStd.WorkchainId != core.DefaultWorkchain {
return core.Address{}, fmt.Errorf("not default workchain for jetton wallet address")
}
return core.Address(msgAddress.AddrStd.Address), nil
}
func newEmulator(code, data *boc.Cell) (*tvm.Emulator, error) {
emulator, err := tvm.NewEmulator(code, data, config.Config.BlockchainConfig, tvm.WithBalance(1_000_000_000))
if err != nil {
return nil, err
}
// TODO: try tvm.WithLazyC7Optimization()
return emulator, nil
}
// GetJettonBalance
// Get method get_wallet_data() returns (int balance, slice owner, slice jetton, cell jetton_wallet_code)
// Returns jetton balance for custom block in basic units
func (c *Connection) GetJettonBalance(ctx context.Context, address core.Address, blockID *ton.BlockIDExt) (*big.Int, error) {
jettonWallet := address.ToTonutilsAddressStd(0)
stack, err := c.RunGetMethod(ctx, blockID, jettonWallet, "get_wallet_data")
if err != nil {
if strings.Contains(err.Error(), "contract is not initialized") {
return big.NewInt(0), nil
}
return nil, fmt.Errorf("get wallet data error: %v", err)
}
res := stack.AsTuple()
if len(res) != 4 {
return nil, fmt.Errorf("invalid stack size")
}
switch res[0].(type) {
case *big.Int:
return res[0].(*big.Int), nil
default:
return nil, fmt.Errorf("invalid balance type")
}
}
// GetLastJettonBalance
// Returns jetton balance for last block in basic units
func (c *Connection) GetLastJettonBalance(ctx context.Context, address *address.Address) (*big.Int, error) {
masterID, err := c.client.CurrentMasterchainInfo(ctx)
if err != nil {
return nil, err
}
addr, err := core.AddressFromTonutilsAddress(address)
if err != nil {
return nil, err
}
return c.GetJettonBalance(ctx, addr, masterID)
}
// GetAccountCurrentState
// Returns TON balance in nanoTONs and account status
func (c *Connection) GetAccountCurrentState(ctx context.Context, address *address.Address) (*big.Int, tlb.AccountStatus, error) {
masterID, err := c.client.CurrentMasterchainInfo(ctx)
if err != nil {
return nil, "", err
}
// TODO: fix waitForBlock and 651 error
for {
select {
case <-ctx.Done():
return nil, "", core.ErrTimeoutExceeded
default:
account, err := c.client.GetAccount(ctx, masterID, address)
if err != nil && isNotReadyError(err) {
time.Sleep(time.Millisecond * 200)
continue
} else if err != nil {
return nil, "", err
}
if !account.IsActive {
return big.NewInt(0), tlb.AccountStatusNonExist, nil
}
return account.State.Balance.Nano(), account.State.Status, nil
}
}
}
// DeployTonWallet
// Deploys wallet contract and wait its activation
func (c *Connection) DeployTonWallet(ctx context.Context, wallet *wallet.Wallet) error {
balance, status, err := c.GetAccountCurrentState(ctx, wallet.Address())
if err != nil {
return err
}
if balance.Cmp(big.NewInt(0)) == 0 {
return fmt.Errorf("empty balance")
}
if status != tlb.AccountStatusActive {
err = wallet.TransferNoBounce(ctx, wallet.Address(), tlb.FromNanoTONU(0), "")
if err != nil {
return err
}
} else {
return nil
}
return c.WaitStatus(ctx, wallet.Address(), tlb.AccountStatusActive)
}
// GetTransactionIDsFromBlock
// Gets all transactions IDs from custom block
func (c *Connection) GetTransactionIDsFromBlock(ctx context.Context, blockID *ton.BlockIDExt) ([]ton.TransactionShortInfo, error) {
var (
txIDList []ton.TransactionShortInfo
after *ton.TransactionID3
next = true
)
for next {
fetchedIDs, more, err := c.client.GetBlockTransactionsV2(ctx, blockID, 256, after)
if err != nil {
return nil, err
}
txIDList = append(txIDList, fetchedIDs...)
next = more
if more {
// set load offset for next query (pagination)
after = fetchedIDs[len(fetchedIDs)-1].ID3()
}
}
// sort by LT
sort.Slice(txIDList, func(i, j int) bool {
return txIDList[i].LT < txIDList[j].LT
})
return txIDList, nil
}
// GetTransactionFromBlock
// Gets transaction from block
func (c *Connection) GetTransactionFromBlock(ctx context.Context, blockID *ton.BlockIDExt, txID ton.TransactionShortInfo) (*tlb.Transaction, error) {
tx, err := c.client.GetTransaction(ctx, blockID, address.NewAddress(0, byte(blockID.Workchain), txID.Account), txID.LT)
if err != nil {
return nil, err
}
return tx, nil
}
func inShard(addr core.Address, shard byte) bool {
return addr[0] == shard
}
func (c *Connection) getCurrentNodeTime(ctx context.Context) (time.Time, error) {
t, err := c.client.GetTime(ctx)
if err != nil {
return time.Time{}, err
}
res := time.Unix(int64(t), 0)
return res, nil
}
// CheckTime
// Checks time diff between node and local time. Due to the fact that the request to the node takes time,
// the local time is defined as the average between the beginning and end of the request.
// Returns true if time diff < cutoff.
func (c *Connection) CheckTime(ctx context.Context, cutoff time.Duration) (bool, error) {
prevTime := time.Now()
nodeTime, err := c.getCurrentNodeTime(ctx)
if err != nil {
return false, err
}
nextTime := time.Now()
midTime := prevTime.Add(nextTime.Sub(prevTime) / 2)
nodeTimeShift := midTime.Sub(nodeTime)
log.Infof("Service-Node time diff: %v", nodeTimeShift)
if nodeTimeShift > cutoff || nodeTimeShift < -cutoff {
return false, nil
}
return true, nil
}
// WaitStatus
// Waits custom status for account. Returns error if context timeout is exceeded.
// Context must be with timeout to avoid blocking!
func (c *Connection) WaitStatus(ctx context.Context, addr *address.Address, status tlb.AccountStatus) error {
for {
select {
case <-ctx.Done():
return core.ErrTimeoutExceeded
default:
_, st, err := c.GetAccountCurrentState(ctx, addr)
if err != nil {
return err
}
if st == status {
return nil
}
time.Sleep(time.Millisecond * 200)
}
}
}
// tonutils TonAPI interface methods
// GetAccount
// The method is being redefined for more stable operation.
// Gets account from prev block if impossible to get it from current block. Be careful with diff calculation between blocks.
func (c *Connection) GetAccount(ctx context.Context, block *ton.BlockIDExt, addr *address.Address) (*tlb.Account, error) {
res, err := c.client.GetAccount(ctx, block, addr)
if err != nil && isNotReadyError(err) {
prevBlock, err := c.client.LookupBlock(ctx, block.Workchain, block.Shard, block.SeqNo-1)
if err != nil {
return nil, err
}
return c.client.GetAccount(ctx, prevBlock, addr)
}
return res, err
}
func (c *Connection) SendExternalMessage(ctx context.Context, msg *tlb.ExternalMessage) error {
return c.client.SendExternalMessage(ctx, msg)
}
// RunGetMethod
// The method is being redefined for more stable operation
// Wait until BlockIsApplied. Use context with timeout.
func (c *Connection) RunGetMethod(ctx context.Context, block *ton.BlockIDExt, addr *address.Address, method string, params ...any) (*ton.ExecutionResult, error) {
for {
select {
case <-ctx.Done():
return nil, core.ErrTimeoutExceeded
default:
res, err := c.client.RunGetMethod(ctx, block, addr, method, params...)
if err != nil && isNotReadyError(err) {
time.Sleep(time.Millisecond * 200)
continue
}
return res, err
}
}
}
func (c *Connection) ListTransactions(ctx context.Context, addr *address.Address, num uint32, lt uint64, txHash []byte) ([]*tlb.Transaction, error) {
return c.client.ListTransactions(ctx, addr, num, lt, txHash)
}
func (c *Connection) Client() ton.LiteClient {
return c.client.Client()
}
func (c *Connection) CurrentMasterchainInfo(ctx context.Context) (*ton.BlockIDExt, error) {
return c.client.CurrentMasterchainInfo(ctx)
}
func (c *Connection) GetMasterchainInfo(ctx context.Context) (*ton.BlockIDExt, error) {
return c.client.GetMasterchainInfo(ctx)
}
func (c *Connection) SendExternalMessageWaitTransaction(ctx context.Context, ext *tlb.ExternalMessage) (*tlb.Transaction, *ton.BlockIDExt, []byte, error) {
return c.client.SendExternalMessageWaitTransaction(ctx, ext)
}
func getBlockchainConfig(ctx context.Context, client ton.LiteClient, block *ton.BlockIDExt) (*cell.Dictionary, error) {
var resp tl.Serializable
var err error
err = client.QueryLiteserver(ctx, ton.GetConfigAll{
Mode: 0b1111111111,
BlockID: block,
}, &resp)
if err != nil {
return nil, err
}
switch t := resp.(type) {
case ton.ConfigAll:
stateExtra, err := ton.CheckShardMcStateExtraProof(block, []*cell.Cell{t.StateProof, t.ConfigProof})
if err != nil {
return nil, fmt.Errorf("incorrect proof: %w", err)
}
return stateExtra.ConfigParams.Config.Params, nil
case ton.LSError:
return nil, t
}
return nil, fmt.Errorf("unexpected response from node")
}
================================================
FILE: blockchain/blockchain_test.go
================================================
package blockchain
import (
"bytes"
"context"
"github.com/gobicycle/bicycle/core"
"github.com/xssnick/tonutils-go/address"
"github.com/xssnick/tonutils-go/tlb"
"github.com/xssnick/tonutils-go/ton/jetton"
"github.com/xssnick/tonutils-go/ton/wallet"
"math/big"
"math/rand"
"os"
"testing"
"time"
)
var (
jettonMasterAddress, _ = address.ParseAddr("kQCKt2WPGX-fh0cIAz38Ljd_OKQjoZE_cqk7QrYGsNP6wfP0") // TGR in Testnet
activeAccount, _ = address.ParseAddr("kQCOSEttz9aEGXkjd1h_NJsQqOca3T-Pld5zSIPHcYZIxsyf")
notActiveAccount, _ = address.ParseAddr("kQAkRRJ1RiViVHY2UmUhWCFjdiZBeEYnhkhxI1JTJFNUNG9v")
)
func connect(t *testing.T) *Connection {
server := os.Getenv("SERVER")
if server == "" {
t.Fatal("empty server var")
}
key := os.Getenv("KEY")
if key == "" {
t.Fatal("empty key var")
}
c, err := NewConnection(server, key, 100)
if err != nil {
t.Fatal("connections err: ", err)
}
return c
}
func getSeed() string {
seed := os.Getenv("SEED")
if seed == "" {
panic("empty seed")
}
return seed
}
func Test_NewConnection(t *testing.T) {
connect(t)
}
func Test_GenerateDefaultWallet(t *testing.T) {
c := connect(t)
seed := getSeed()
hlWallet, shard, id, err := c.GenerateDefaultWallet(seed, false)
if err != nil {
t.Fatal("gen default wallet err: ", err)
}
if hlWallet.Address().Data()[0] != shard {
t.Fatal("invalid shard")
}
if id != wallet.DefaultSubwallet {
t.Fatal("invalid subwallet ID")
}
w, shard, id, err := c.GenerateDefaultWallet(seed, true)
if err != nil {
t.Fatal("gen default wallet err: ", err)
}
if w.Address().Data()[0] != shard {
t.Fatal("invalid shard")
}
if id != wallet.DefaultSubwallet {
t.Fatal("invalid subwallet ID")
}
}
func Test_GenerateSubWallet(t *testing.T) {
c := connect(t)
seed := getSeed()
for i := 0; i < 10; i++ {
shard := byte(rand.Intn(255))
startSubWalletID := rand.Uint32()
subWallet, subWalletID, err := c.GenerateSubWallet(seed, shard, startSubWalletID)
if err != nil {
t.Fatal("gen sub wallet err: ", err)
}
if subWalletID <= startSubWalletID {
t.Fatal("invalid subwallet ID")
}
if subWallet.Address().Data()[0] != shard {
t.Fatal("invalid shard")
}
}
}
func Test_GetJettonWalletAddress(t *testing.T) {
c := connect(t)
seed := getSeed()
ctx, cancel := context.WithTimeout(context.Background(), time.Second*30)
defer cancel()
owner, _, _, err := c.GenerateDefaultWallet(seed, true)
if err != nil {
t.Fatal("gen owner wallet err: ", err)
}
jettonWalletAddr, err := c.GetJettonWalletAddress(ctx, owner.Address(), jettonMasterAddress)
if err != nil {
t.Fatal("get jetton wallet address err: ", err)
}
master := jetton.NewJettonMasterClient(c.client, jettonMasterAddress)
jettonWallet, err := master.GetJettonWallet(ctx, owner.Address())
if err != nil {
t.Fatal("get jetton wallet address by tonutils method err: ", err)
}
if !bytes.Equal(jettonWallet.Address().Data(), jettonWalletAddr.Data()) ||
jettonWallet.Address().Workchain() != jettonWalletAddr.Workchain() {
t.Fatal("invalid jetton wallet address")
}
}
func Test_GenerateJettonWalletAddressForProxy(t *testing.T) {
c := connect(t)
seed := getSeed()
ctx, cancel := context.WithTimeout(context.Background(), time.Second*30)
defer cancel()
owner, _, _, err := c.GenerateDefaultWallet(seed, true)
if err != nil {
t.Fatal("gen owner wallet err: ", err)
}
master := jetton.NewJettonMasterClient(c.client, jettonMasterAddress)
for i := 0; i < 10; i++ {
shard := byte(rand.Intn(255))
startSubWalletID := rand.Uint32()
proxy, jettonWalletAddr, err := c.GenerateDepositJettonWalletForProxy(ctx, shard, owner.Address(), jettonMasterAddress, startSubWalletID)
if err != nil {
t.Fatal("gen sub wallet err: ", err)
}
if proxy == nil {
t.Fatal("nil owner wallet")
}
if jettonWalletAddr == nil {
t.Fatal("nil jetton wallet address")
}
if proxy.SubwalletID <= startSubWalletID {
t.Fatal("invalid subwallet ID")
}
if jettonWalletAddr.Data()[0] != shard {
t.Fatal("invalid shard")
}
jettonWallet, err := master.GetJettonWallet(ctx, proxy.Address())
if err != nil {
t.Fatal("get jetton wallet address by tonutils method err: ", err)
}
if jettonWallet.Address().String() != jettonWalletAddr.String() {
t.Fatal("invalid jetton wallet address")
}
}
}
func Test_GetJettonBalance(t *testing.T) {
c := connect(t)
ctx, cancel := context.WithTimeout(context.Background(), time.Second*30)
defer cancel()
block, err := c.client.CurrentMasterchainInfo(ctx)
if err != nil {
t.Fatal("get current masterchain err: ", err)
}
coreAddr1 := core.AddressMustFromTonutilsAddress(activeAccount)
coreAddr2 := core.AddressMustFromTonutilsAddress(notActiveAccount)
b1, err := c.GetJettonBalance(ctx, coreAddr1, block)
if err != nil {
t.Fatal("get balance: ", err)
}
if b1.Cmp(big.NewInt(0)) != 1 {
t.Fatal("empty balance: ", err)
}
b2, err := c.GetJettonBalance(ctx, coreAddr2, block)
if err != nil {
t.Fatal("get balance: ", err)
}
if b2.Cmp(big.NewInt(0)) != 0 {
t.Fatal("not empty balance: ", err)
}
}
func Test_GetAccountCurrentState(t *testing.T) {
c := connect(t)
ctx, cancel := context.WithTimeout(context.Background(), time.Second*30)
defer cancel()
b1, st1, err := c.GetAccountCurrentState(ctx, activeAccount)
if err != nil {
t.Fatal("get acc current state err: ", err)
}
if b1.Cmp(big.NewInt(0)) != 1 || st1 != tlb.AccountStatusActive {
t.Fatal("acc not active")
}
b2, st2, err := c.GetAccountCurrentState(ctx, notActiveAccount)
if err != nil {
t.Fatal("get acc current state err: ", err)
}
if b2.Cmp(big.NewInt(0)) != 0 || st2 != tlb.AccountStatusNonExist {
t.Fatal("acc active")
}
}
func Test_DeployTonWallet(t *testing.T) {
c := connect(t)
seed := getSeed()
ctx, cancel := context.WithTimeout(context.Background(), time.Second*200)
defer cancel()
amount := tlb.FromNanoTONU(100_000_000)
mainWallet, _, _, err := c.GenerateDefaultWallet(seed, true)
if err != nil {
t.Fatal("gen main wallet err: ", err)
}
b, st, err := c.GetAccountCurrentState(ctx, mainWallet.Address())
if err != nil {
t.Fatal("get acc current state err: ", err)
}
if b.Cmp(amount.Nano()) != 1 || st != tlb.AccountStatusActive {
t.Fatal("wallet not active")
}
newWallet, err := mainWallet.GetSubwallet(rand.Uint32())
if err != nil {
t.Fatal("gen new wallet err: ", err)
}
//fmt.Printf("Main wallet: %v\n", mainWallet.Address().String())
//fmt.Printf("New wallet: %v\n", newWallet.Address().String())
_, st, err = c.GetAccountCurrentState(ctx, newWallet.Address())
if err != nil {
t.Fatal("get acc current state err: ", err)
}
if st != tlb.AccountStatusNonExist {
t.Log("wallet not empty")
t.Skip()
}
err = mainWallet.TransferNoBounce(
ctx,
newWallet.Address(),
amount,
"",
true,
)
if err != nil {
t.Fatal("transfer err: ", err)
}
err = c.WaitStatus(ctx, newWallet.Address(), tlb.AccountStatusUninit)
if err != nil {
t.Fatal("wait uninit err: ", err)
}
err = c.DeployTonWallet(ctx, newWallet)
if err != nil {
t.Fatal("deploy new wallet err: ", err)
}
err = newWallet.Send(ctx, &wallet.Message{
Mode: 128 + 32, // 128 + 32 send all and destroy
InternalMessage: &tlb.InternalMessage{
IHRDisabled: true,
Bounce: false,
DstAddr: mainWallet.Address(),
Amount: tlb.FromNanoTONU(0),
Body: nil,
},
}, false)
if err != nil {
t.Fatal("send withdrawal err: ", err)
}
err = c.WaitStatus(ctx, newWallet.Address(), tlb.AccountStatusNonExist)
if err != nil {
t.Fatal("wait empty err: ", err)
}
}
func Test_GetTransactionIDsFromBlock(t *testing.T) {
c := connect(t)
ctx, cancel := context.WithTimeout(context.Background(), time.Second*30)
defer cancel()
masterID, err := c.client.CurrentMasterchainInfo(ctx)
if err != nil {
t.Fatal("get last block err: ", err)
}
_, err = c.GetTransactionIDsFromBlock(ctx, masterID)
if err != nil {
t.Fatal("get tx ids err: ", err)
}
}
func Test_GetTransactionFromBlock(t *testing.T) {
c := connect(t)
ctx, cancel := context.WithTimeout(context.Background(), time.Second*120)
defer cancel()
for {
masterID, err := c.client.CurrentMasterchainInfo(ctx)
if err != nil {
t.Fatal("get last block err: ", err)
}
txIDs, err := c.GetTransactionIDsFromBlock(ctx, masterID)
if err != nil {
t.Fatal("get tx ids err: ", err)
}
if len(txIDs) > 0 {
tx, err := c.GetTransactionFromBlock(ctx, masterID, txIDs[0])
if err != nil {
t.Fatal("get tx err: ", err)
}
if tx == nil {
t.Fatal("nil tx")
}
break
}
}
}
func Test_CheckTime(t *testing.T) {
c := connect(t)
ctx, cancel := context.WithTimeout(context.Background(), time.Second*20)
defer cancel()
res, err := c.CheckTime(ctx, time.Second*0)
if err != nil {
t.Fatal("check time err: ", err)
}
if res == true {
t.Fatal("time diff can not be 0")
}
res, err = c.CheckTime(ctx, time.Hour*1000)
if err != nil {
t.Fatal("check time err: ", err)
}
if res == false {
t.Fatal("failed for extra large cutoff")
}
}
func Test_WaitStatus(t *testing.T) {
c := connect(t)
ctx, cancel := context.WithTimeout(context.Background(), time.Second*7)
defer cancel()
err := c.WaitStatus(ctx, activeAccount, tlb.AccountStatusActive)
if err != nil {
t.Fatal("wait status err: ", err)
}
err = c.WaitStatus(ctx, activeAccount, tlb.AccountStatusNonExist)
if err == nil {
t.Fatal("must be timeout error")
}
}
func Test_GetAccount(t *testing.T) {
c := connect(t)
ctx, cancel := context.WithTimeout(context.Background(), time.Second*120)
defer cancel()
for i := 0; i < 20; i++ {
b, err := c.client.GetMasterchainInfo(ctx)
if err != nil {
t.Fatal("get masterchain info err: ", err)
}
_, err = c.GetAccount(ctx, b, activeAccount)
if err != nil {
t.Fatal("get account err: ", err)
}
}
}
func Test_RunGetMethod(t *testing.T) {
c := connect(t)
ctx, cancel := context.WithTimeout(context.Background(), time.Second*120)
defer cancel()
for i := 0; i < 20; i++ {
b, err := c.client.GetMasterchainInfo(ctx)
if err != nil {
t.Fatal("get masterchain info err: ", err)
}
_, err = c.RunGetMethod(ctx, b, jettonMasterAddress, "get_jetton_data")
if err != nil {
t.Fatal("run get method err: ", err)
}
}
}
func Test_NextBlock(t *testing.T) {
c := connect(t)
var shard byte = 123
st := NewShardTracker(shard, nil, c)
for i := 0; i < 5; i++ {
h, _, err := st.NextBlock()
if err != nil {
t.Fatal("get next block err: ", err)
}
if !isInShard(uint64(h.Shard), shard) {
t.Fatal("next block not in shard")
}
}
}
func Test_Stop(t *testing.T) {
c := connect(t)
st := NewShardTracker(123, nil, c)
for i := 0; i < 2; i++ {
_, _, err := st.NextBlock()
if err != nil {
t.Fatal("get next block err: ", err)
}
}
st.Stop()
_, flag, err := st.NextBlock()
if err != nil {
t.Fatal("get next block err: ", err)
}
if !flag {
t.Fatal("no shutdown flag")
}
}
================================================
FILE: blockchain/limited_client.go
================================================
package blockchain
import (
"context"
"fmt"
"github.com/xssnick/tonutils-go/tl"
"github.com/xssnick/tonutils-go/ton"
"golang.org/x/time/rate"
)
type limitedLiteClient struct {
limiter *rate.Limiter
original ton.LiteClient
}
func newLimitedClient(lc ton.LiteClient, rateLimit int) *limitedLiteClient {
return &limitedLiteClient{
original: lc,
limiter: rate.NewLimiter(rate.Limit(rateLimit), 1),
}
}
func (w *limitedLiteClient) QueryLiteserver(ctx context.Context, payload tl.Serializable, result tl.Serializable) error {
err := w.limiter.Wait(ctx)
if err != nil {
return fmt.Errorf("limiter err: %w", err)
}
return w.original.QueryLiteserver(ctx, payload, result)
}
func (w *limitedLiteClient) StickyContext(ctx context.Context) context.Context {
return w.original.StickyContext(ctx)
}
func (w *limitedLiteClient) StickyNodeID(ctx context.Context) uint32 {
return w.original.StickyNodeID(ctx)
}
func (w *limitedLiteClient) StickyContextNextNode(ctx context.Context) (context.Context, error) {
return w.original.StickyContextNextNode(ctx)
}
func (w *limitedLiteClient) StickyContextNextNodeBalanced(ctx context.Context) (context.Context, error) {
return w.original.StickyContextNextNodeBalanced(ctx)
}
================================================
FILE: blockchain/shard_tracker.go
================================================
package blockchain
import (
"context"
"github.com/gobicycle/bicycle/core"
log "github.com/sirupsen/logrus"
"github.com/xssnick/tonutils-go/tlb"
"github.com/xssnick/tonutils-go/ton"
"math/bits"
"strings"
"time"
)
const ErrBlockNotApplied = "block is not applied"
const ErrBlockNotInDB = "code 651"
type ShardTracker struct {
connection *Connection
shard byte
lastKnownShardBlock *ton.BlockIDExt
lastMasterBlock *ton.BlockIDExt
buffer []core.ShardBlockHeader
gracefulShutdown bool
infoCounter, infoStep int
infoLastTime time.Time
}
// NewShardTracker creates new tracker to get blocks with specific shard attribute
func NewShardTracker(shard byte, startBlock *ton.BlockIDExt, connection *Connection) *ShardTracker {
t := &ShardTracker{
connection: connection,
shard: shard,
lastKnownShardBlock: startBlock,
buffer: make([]core.ShardBlockHeader, 0),
infoCounter: 0,
infoStep: 1000,
infoLastTime: time.Now(),
}
return t
}
// NextBlock returns next block header and graceful shutdown flag.
// (ShardBlockHeader, false) for normal operation and (empty block header, true) for graceful shutdown.
func (s *ShardTracker) NextBlock() (core.ShardBlockHeader, bool, error) {
if s.gracefulShutdown {
return core.ShardBlockHeader{}, true, nil
}
h := s.getNext()
if h != nil {
return *h, false, nil
}
// the interval between blocks can be up to 40 seconds
ctx, cancel := context.WithTimeout(context.Background(), time.Second*60)
defer cancel()
masterBlockID, err := s.getNextMasterBlockID(ctx)
if err != nil {
return core.ShardBlockHeader{}, false, err
}
exit, err := s.loadShardBlocksBatch(masterBlockID)
if err != nil {
return core.ShardBlockHeader{}, false, err
}
if exit {
log.Printf("Shard tracker sync stopped")
return core.ShardBlockHeader{}, true, nil
}
return s.NextBlock()
}
// Stop initiates graceful shutdown
func (s *ShardTracker) Stop() {
s.gracefulShutdown = true
}
func (s *ShardTracker) getNext() *core.ShardBlockHeader {
if len(s.buffer) != 0 {
h := s.buffer[0]
s.buffer = s.buffer[1:]
return &h
}
return nil
}
func (s *ShardTracker) getNextMasterBlockID(ctx context.Context) (*ton.BlockIDExt, error) {
for {
masterBlockID, err := s.connection.client.GetMasterchainInfo(ctx)
if err != nil {
// exit by context timeout
return nil, err
}
if s.lastMasterBlock == nil {
s.lastMasterBlock = masterBlockID
return masterBlockID, nil
}
if masterBlockID.SeqNo == s.lastMasterBlock.SeqNo {
time.Sleep(time.Second)
continue
}
s.lastMasterBlock = masterBlockID
return masterBlockID, nil
}
}
func (s *ShardTracker) loadShardBlocksBatch(masterBlockID *ton.BlockIDExt) (bool, error) {
var (
shards []*ton.BlockIDExt
err error
)
ctx, cancel := context.WithTimeout(context.Background(), time.Second*20)
defer cancel()
for {
shards, err = s.connection.client.GetBlockShardsInfo(ctx, masterBlockID)
if err != nil && isNotReadyError(err) { // TODO: clarify error type
time.Sleep(time.Second)
continue
} else if err != nil {
return false, err
// exit by context timeout
}
break
}
s.infoCounter = 0
batch, exit, err := s.getShardBlocksRecursively(filterByShard(shards, s.shard), nil)
if err != nil {
return false, err
}
if exit {
return true, nil
}
if len(batch) != 0 {
s.lastKnownShardBlock = batch[0].BlockIDExt
for i := len(batch) - 1; i >= 0; i-- {
s.buffer = append(s.buffer, batch[i])
}
}
return false, nil
}
func (s *ShardTracker) getShardBlocksRecursively(i *ton.BlockIDExt, batch []core.ShardBlockHeader) ([]core.ShardBlockHeader, bool, error) {
if s.gracefulShutdown {
return nil, true, nil
}
if s.lastKnownShardBlock == nil {
s.lastKnownShardBlock = i
}
isKnown := (s.lastKnownShardBlock.Shard == i.Shard) && (s.lastKnownShardBlock.SeqNo == i.SeqNo)
if isKnown {
return batch, false, nil
}
// compare seqno with filtered shard block
// handle the case when a node may reference an old block
if s.lastKnownShardBlock.SeqNo > i.SeqNo {
return []core.ShardBlockHeader{}, false, nil
}
seqnoDiff := int(i.SeqNo - s.lastKnownShardBlock.SeqNo)
if seqnoDiff > s.infoStep {
if s.infoCounter%s.infoStep == 0 {
estimatedTime := time.Duration(seqnoDiff/s.infoStep) * time.Since(s.infoLastTime)
s.infoLastTime = time.Now()
if s.infoCounter == 0 {
log.Printf("Shard tracker syncing... Seqno diff: %v Estimated time: unknown\n", seqnoDiff)
} else {
log.Printf("Shard tracker syncing... Seqno diff: %v Estimated time: %v\n", seqnoDiff, estimatedTime)
}
}
s.infoCounter++
}
ctx, cancel := context.WithTimeout(context.Background(), time.Second*20)
defer cancel()
h, err := s.connection.getShardBlocksHeader(ctx, i, s.shard)
if err != nil {
return nil, false, err
}
batch = append(batch, h)
return s.getShardBlocksRecursively(h.Parent, batch)
}
func isInShard(blockShardPrefix uint64, shard byte) bool {
if blockShardPrefix == 0 {
log.Fatalf("invalid shard_prefix")
}
prefixLen := 64 - 1 - bits.TrailingZeros64(blockShardPrefix) // without one insignificant bit
if prefixLen > 8 {
log.Fatalf("more than 256 shards is not supported")
}
res := (uint64(shard) << (64 - 8)) ^ blockShardPrefix
return bits.LeadingZeros64(res) >= prefixLen
}
func filterByShard(headers []*ton.BlockIDExt, shard byte) *ton.BlockIDExt {
for _, h := range headers {
if isInShard(uint64(h.Shard), shard) {
return h
}
}
log.Fatalf("must be at least one suitable shard block")
return nil
}
func convertBlockToShardHeader(block *tlb.Block, info *ton.BlockIDExt, shard byte) (core.ShardBlockHeader, error) {
parents, err := block.BlockInfo.GetParentBlocks()
if err != nil {
return core.ShardBlockHeader{}, err
}
parent := filterByShard(parents, shard)
return core.ShardBlockHeader{
NotMaster: block.BlockInfo.NotMaster,
GenUtime: block.BlockInfo.GenUtime,
StartLt: block.BlockInfo.StartLt,
EndLt: block.BlockInfo.EndLt,
Parent: parent,
BlockIDExt: info,
}, nil
}
// get shard block header for specific shard attribute with one parent
func (c *Connection) getShardBlocksHeader(ctx context.Context, shardBlockID *ton.BlockIDExt, shard byte) (core.ShardBlockHeader, error) {
var (
err error
block *tlb.Block
)
for {
block, err = c.client.GetBlockData(ctx, shardBlockID)
if err != nil && isNotReadyError(err) {
time.Sleep(time.Millisecond * 500)
continue
} else if err != nil {
return core.ShardBlockHeader{}, err
// exit by context timeout
}
break
}
return convertBlockToShardHeader(block, shardBlockID, shard)
}
func isNotReadyError(err error) bool {
return strings.Contains(err.Error(), ErrBlockNotApplied) || strings.Contains(err.Error(), ErrBlockNotInDB)
}
================================================
FILE: cmd/processor/main.go
================================================
package main
import (
"context"
"errors"
"fmt"
"github.com/gobicycle/bicycle/api"
"github.com/gobicycle/bicycle/blockchain"
"github.com/gobicycle/bicycle/config"
"github.com/gobicycle/bicycle/core"
"github.com/gobicycle/bicycle/db"
"github.com/gobicycle/bicycle/queue"
"github.com/gobicycle/bicycle/webhook"
log "github.com/sirupsen/logrus"
"net/http"
"os"
"os/signal"
"sync"
"syscall"
"time"
)
var Version = "dev"
func main() {
log.Infof("App version: %s", Version)
config.GetConfig()
sigChannel := make(chan os.Signal, 1)
signal.Notify(sigChannel, os.Interrupt, syscall.SIGTERM)
wg := new(sync.WaitGroup)
bcClient, err := blockchain.NewConnection(config.Config.LiteServer, config.Config.LiteServerKey, config.Config.LiteServerRateLimit)
if err != nil {
log.Fatalf("blockchain connection error: %v", err)
}
dbClient, err := db.NewConnection(config.Config.DatabaseURI)
if err != nil {
log.Fatalf("DB connection error: %v", err)
}
ctx, cancel := context.WithTimeout(context.Background(), time.Second*120)
defer cancel()
err = dbClient.LoadAddressBook(ctx)
if err != nil {
log.Fatalf("address book loading error: %v", err)
}
isTimeSynced, err := bcClient.CheckTime(ctx, config.AllowableServiceToNodeTimeDiff)
if err != nil {
log.Fatalf("get node time err: %v", err)
}
if !isTimeSynced {
log.Fatalf("Service and Node time not synced")
}
wallets, err := core.InitWallets(ctx, dbClient, bcClient, config.Config.Seed, config.Config.Jettons)
if err != nil {
log.Fatalf("Wallets initialization error: %v", err)
}
var notificators []core.Notificator
if config.Config.QueueEnabled {
queueClient, err := queue.NewAmqpClient(config.Config.QueueURI, config.Config.QueueEnabled, config.Config.QueueName)
if err != nil {
log.Fatalf("new queue client creating error: %v", err)
}
notificators = append(notificators, queueClient)
}
if config.Config.WebhookEndpoint != "" {
webhookClient, err := webhook.NewWebhookClient(config.Config.WebhookEndpoint, config.Config.WebhookToken)
if err != nil {
log.Fatalf("new webhook client creating error: %v", err)
}
notificators = append(notificators, webhookClient)
}
var tracker *blockchain.ShardTracker
block, err := dbClient.GetLastSavedBlockID(ctx)
if !errors.Is(err, core.ErrNotFound) && err != nil {
log.Fatalf("Get last saved block error: %v", err)
} else if errors.Is(err, core.ErrNotFound) {
tracker = blockchain.NewShardTracker(wallets.Shard, nil, bcClient)
} else {
tracker = blockchain.NewShardTracker(wallets.Shard, block, bcClient)
}
blockScanner := core.NewBlockScanner(wg, dbClient, bcClient, wallets.Shard, tracker, notificators)
withdrawalsProcessor := core.NewWithdrawalsProcessor(
wg, dbClient, bcClient, wallets, config.Config.ColdWallet)
withdrawalsProcessor.Start()
apiMux := http.NewServeMux()
h := api.NewHandler(dbClient, bcClient, config.Config.APIToken, wallets.Shard, *wallets.TonHotWallet.Address())
api.RegisterHandlers(apiMux, h)
go func() {
err := http.ListenAndServe(fmt.Sprintf(":%d", config.Config.APIPort), apiMux)
if err != nil {
log.Fatalf("api error: %v", err)
}
}()
go func() {
<-sigChannel
log.Printf("SIGTERM received")
blockScanner.Stop()
withdrawalsProcessor.Stop()
}()
wg.Wait()
}
================================================
FILE: cmd/testutil/http.go
================================================
package main
import (
"bytes"
"encoding/json"
"fmt"
"github.com/gobicycle/bicycle/api"
"github.com/gobicycle/bicycle/config"
"github.com/gobicycle/bicycle/core"
"github.com/gofrs/uuid"
"github.com/shopspring/decimal"
"io"
"log"
"net/http"
"time"
)
type Client struct {
client *http.Client
urlA string
urlB string
token string
userID string
}
func NewClient(urlA, urlB, token, userID string) *Client {
c := &Client{
client: &http.Client{Timeout: 10 * time.Second},
urlA: urlA,
urlB: urlB,
token: token,
userID: userID,
}
return c
}
func (s *Client) InitDeposits(host string) (map[string][]string, error) {
deposits := make(map[string][]string)
addr, err := s.GetAllAddresses(host)
if err != nil {
return nil, err
}
if len(addr.Addresses) != 0 {
for _, wa := range addr.Addresses {
deposits[wa.Currency] = append(deposits[wa.Currency], wa.Address)
}
return deposits, nil
}
for i := 0; i < depositsQty; i++ {
addr, err := s.GetNewAddress(host, core.TonSymbol)
if err != nil {
return nil, err
}
deposits[core.TonSymbol] = append(deposits[core.TonSymbol], addr)
}
for i := 0; i < depositsQty; i++ {
for cur := range config.Config.Jettons {
addr, err := s.GetNewAddress(host, cur)
if err != nil {
return nil, err
}
deposits[cur] = append(deposits[cur], addr)
}
}
log.Printf("Deposits initialized for %s", host)
return deposits, nil
}
func (s *Client) GetAllAddresses(host string) (api.GetAddressesResponse, error) {
url := fmt.Sprintf("http://%s/v1/address/all?user_id=%s", host, s.userID)
request, err := http.NewRequest("GET", url, nil)
if err != nil {
return api.GetAddressesResponse{}, err
}
request.Header.Add("Authorization", "Bearer "+s.token)
response, err := s.client.Do(request)
if err != nil {
return api.GetAddressesResponse{}, err
}
defer func() {
err := response.Body.Close()
if err != nil {
log.Fatalf("response body close error: %v", err)
}
}()
if response.StatusCode >= 300 {
return api.GetAddressesResponse{}, fmt.Errorf("response status: %v", response.Status)
}
content, err := io.ReadAll(response.Body)
if err != nil {
return api.GetAddressesResponse{}, err
}
var res api.GetAddressesResponse
err = json.Unmarshal(content, &res)
if err != nil {
return api.GetAddressesResponse{}, err
}
return res, nil
}
func (s *Client) SendWithdrawal(host, currency, destination string, amount int64) (api.WithdrawalResponse, uuid.UUID, error) {
url := fmt.Sprintf("http://%s/v1/withdrawal/send", host)
u, err := uuid.NewV4()
if err != nil {
return api.WithdrawalResponse{}, uuid.UUID{}, err
}
reqData := api.WithdrawalRequest{
UserID: s.userID,
QueryID: u.String(),
Currency: currency,
Amount: decimal.New(amount, 0),
Destination: destination,
Comment: u.String(),
}
jsonData, err := json.Marshal(reqData)
if err != nil {
return api.WithdrawalResponse{}, uuid.UUID{}, err
}
request, err := http.NewRequest("POST", url, bytes.NewBuffer(jsonData))
if err != nil {
return api.WithdrawalResponse{}, uuid.UUID{}, err
}
request.Header.Set("Content-Type", "application/json; charset=UTF-8")
request.Header.Add("Authorization", "Bearer "+s.token)
response, err := s.client.Do(request)
if err != nil {
return api.WithdrawalResponse{}, uuid.UUID{}, err
}
defer func() {
err := response.Body.Close()
if err != nil {
log.Fatalf("response body close error: %v", err)
}
}()
if response.StatusCode >= 300 {
return api.WithdrawalResponse{}, uuid.UUID{}, fmt.Errorf("response status: %v", response.Status)
}
content, err := io.ReadAll(response.Body)
if err != nil {
return api.WithdrawalResponse{}, uuid.UUID{}, err
}
var res api.WithdrawalResponse
err = json.Unmarshal(content, &res)
if err != nil {
return api.WithdrawalResponse{}, uuid.UUID{}, err
}
return res, u, nil
}
func (s *Client) GetNewAddress(host, currency string) (string, error) {
url := fmt.Sprintf("http://%s/v1/address/new", host)
reqData := struct {
UserID string `json:"user_id"`
Currency string `json:"currency"`
}{
UserID: s.userID,
Currency: currency,
}
jsonData, err := json.Marshal(reqData)
if err != nil {
return "", err
}
request, err := http.NewRequest("POST", url, bytes.NewBuffer(jsonData))
if err != nil {
return "", err
}
request.Header.Set("Content-Type", "application/json; charset=UTF-8")
request.Header.Add("Authorization", "Bearer "+s.token)
response, err := s.client.Do(request)
if err != nil {
return "", err
}
defer func() {
err := response.Body.Close()
if err != nil {
log.Fatalf("response body close error: %v", err)
}
}()
if response.StatusCode >= 300 {
return "", fmt.Errorf("response status: %v", response.Status)
}
content, err := io.ReadAll(response.Body)
if err != nil {
return "", err
}
var res struct {
Address string `json:"address"`
}
err = json.Unmarshal(content, &res)
if err != nil {
return "", err
}
return res.Address, nil
}
func (s *Client) GetWithdrawalStatus(host string, id int64) (api.WithdrawalStatusResponse, error) {
url := fmt.Sprintf("http://%s/v1/withdrawal/status?id=%v", host, id)
request, err := http.NewRequest("GET", url, nil)
if err != nil {
return api.WithdrawalStatusResponse{}, err
}
request.Header.Add("Authorization", "Bearer "+s.token)
response, err := s.client.Do(request)
if err != nil {
return api.WithdrawalStatusResponse{}, err
}
defer func() {
err := response.Body.Close()
if err != nil {
log.Fatalf("response body close error: %v", err)
}
}()
if response.StatusCode >= 300 {
return api.WithdrawalStatusResponse{}, fmt.Errorf("response status: %v", response.Status)
}
content, err := io.ReadAll(response.Body)
if err != nil {
return api.WithdrawalStatusResponse{}, err
}
var res api.WithdrawalStatusResponse
err = json.Unmarshal(content, &res)
if err != nil {
return api.WithdrawalStatusResponse{}, err
}
return res, nil
}
================================================
FILE: cmd/testutil/main.go
================================================
package main
import (
"context"
"github.com/gobicycle/bicycle/blockchain"
"github.com/gobicycle/bicycle/config"
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/xssnick/tonutils-go/address"
"log"
"net/http"
"os"
"time"
)
var (
onlyMonitoring = true
Version = "dev"
)
const (
depositsQty = 10
tonWithdrawAmount = 550_000_000
jettonWithdrawAmount = 550_000_000
tonMinCutoff = 10_000_000_000
)
func main() {
log.Printf("App version: %s", Version)
config.GetConfig()
if circulation := os.Getenv("CIRCULATION"); circulation == "true" {
onlyMonitoring = false
}
urlA := os.Getenv("HOST_A")
if urlA == "" {
log.Fatalf("empty HOST_A env var")
}
urlB := os.Getenv("HOST_B")
if urlB == "" {
log.Fatalf("empty HOST_B env var")
}
hotWalletA, err := address.ParseAddr(os.Getenv("HOT_WALLET_A"))
if err != nil {
log.Fatalf("invalid HOT_WALLET_A env var")
}
hotWalletB, err := address.ParseAddr(os.Getenv("HOT_WALLET_B"))
if err != nil {
log.Fatalf("invalid HOT_WALLET_B env var")
}
bcClient, err := blockchain.NewConnection(config.Config.LiteServer, config.Config.LiteServerKey, config.Config.LiteServerRateLimit)
if err != nil {
log.Fatalf("blockchain connection error: %v", err)
}
httpClient := NewClient(urlA, urlB, config.Config.APIToken, "TestClient")
http.Handle("/metrics", promhttp.Handler())
go func() {
log.Fatal(http.ListenAndServe(":9101", nil))
}()
depositsA, err := httpClient.InitDeposits(urlA)
if err != nil {
log.Fatalf("can not init deposits: %v", err)
}
depositsB, err := httpClient.InitDeposits(urlB)
if err != nil {
log.Fatalf("can not init deposits: %v", err)
}
payerProc := NewPayerProcessor(context.TODO(), httpClient, bcClient, depositsA, depositsB, hotWalletA, hotWalletB)
payerProc.Start()
for {
time.Sleep(time.Hour)
}
}
================================================
FILE: cmd/testutil/metrics.go
================================================
package main
import (
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
)
var (
hotWalletABalance = promauto.NewGaugeVec(
prometheus.GaugeOpts{
Name: "hot_wallet_a_balance",
Help: "Hot wallet A balance",
},
[]string{"currency"},
)
hotWalletBBalance = promauto.NewGaugeVec(
prometheus.GaugeOpts{
Name: "hot_wallet_b_balance",
Help: "Hot wallet B balance",
},
[]string{"currency"},
)
depositWalletABalance = promauto.NewGaugeVec(
prometheus.GaugeOpts{
Name: "deposit_wallet_a_balance",
Help: "Deposit wallet A balance",
},
[]string{"currency", "address"},
)
depositWalletBBalance = promauto.NewGaugeVec(
prometheus.GaugeOpts{
Name: "deposit_wallet_b_balance",
Help: "Deposit wallet B balance",
},
[]string{"currency", "address"},
)
totalBalance = promauto.NewGaugeVec(
prometheus.GaugeOpts{
Name: "total_balance",
Help: "Total balance",
},
[]string{"currency"},
)
totalLosses = promauto.NewGaugeVec(
prometheus.GaugeOpts{
Name: "total_losses",
Help: "Total losses",
},
[]string{"currency"},
)
predictedTonLoss = promauto.NewGauge(
prometheus.GaugeOpts{
Name: "predicted_ton_loss",
Help: "Predicted TON loss",
},
)
totalProcessedAmount = promauto.NewGaugeVec(
prometheus.GaugeOpts{
Name: "total_processed_amount",
Help: "Total processed amount",
},
[]string{"currency"},
)
)
================================================
FILE: cmd/testutil/utils.go
================================================
package main
import (
"bytes"
"context"
"fmt"
"github.com/gobicycle/bicycle/blockchain"
"github.com/gobicycle/bicycle/config"
"github.com/gobicycle/bicycle/core"
"github.com/gofrs/uuid"
log "github.com/sirupsen/logrus"
"github.com/xssnick/tonutils-go/address"
"github.com/xssnick/tonutils-go/tlb"
"math"
"sync"
"time"
)
type PayerProcessor struct {
client *Client
bcClient *blockchain.Connection
depositsA []Deposit
depositsB []Deposit
hotWalletsAddrA map[string]*address.Address
hotWalletsAddrB map[string]*address.Address
lastTxIDA, lastTxIDB TxID
balances *hotBalances
knownUUIDsA, knownUUIDsB map[uuid.UUID]struct{}
}
const (
TonLossForJettonExternalWithdrawal int64 = 58_376_225 // without Jetton wallet deploy except of excess // SCALE:69_579_403 TGR:47_173_046
TonLossForJettonInternalWithdrawal int64 = 55_306_226 // proxy deploy without Jetton wallet deploy except of JettonForwardAmount // SCALE:57_873_578 TGR:52_738_873
TonLossForTonExternalWithdrawal int64 = 10_232_080 // only fees
TonLossForTonInternalWithdrawal int64 = 8_034_998 // deposit wallet deploy
)
type PaymentSide string
const (
SideA PaymentSide = "A"
SideB PaymentSide = "B"
)
type TxID struct {
Lt uint64
Hash []byte
}
type Deposit struct {
Address *address.Address
JettonWallet *address.Address
Currency string
}
type hotBalances struct {
mutex sync.Mutex
hotWalletsBalancesA map[string]int64
hotWalletsBalancesB map[string]int64
}
func newHotBalances() *hotBalances {
return &hotBalances{
hotWalletsBalancesA: make(map[string]int64),
hotWalletsBalancesB: make(map[string]int64),
}
}
func (h *hotBalances) ReadBalance(walletSide PaymentSide, currency string) int64 {
h.mutex.Lock()
defer h.mutex.Unlock()
switch walletSide {
case SideA:
b, ok := h.hotWalletsBalancesA[currency]
if ok {
return b
}
return 0
case SideB:
b, ok := h.hotWalletsBalancesB[currency]
if ok {
return b
}
return 0
}
log.Fatalf("invalid payment side")
return 0
}
func (h *hotBalances) WriteBalance(walletSide PaymentSide, currency string, balance int64) {
h.mutex.Lock()
defer h.mutex.Unlock()
switch walletSide {
case SideA:
h.hotWalletsBalancesA[currency] = balance
return
case SideB:
h.hotWalletsBalancesB[currency] = balance
return
}
log.Fatalf("invalid payment side")
}
func NewPayerProcessor(
ctx context.Context,
client *Client,
bcClient *blockchain.Connection,
depositsA map[string][]string,
depositsB map[string][]string,
hotA, hotB *address.Address,
) *PayerProcessor {
// hot jetton wallets
addrA := make(map[string]*address.Address)
addrB := make(map[string]*address.Address)
addrA[core.TonSymbol] = hotA
addrB[core.TonSymbol] = hotB
lastTxIDA, err := getLastTxID(ctx, bcClient, hotA)
if err != nil {
log.Fatalf("get last TX ID A error: %v", err)
}
lastTxIDB, err := getLastTxID(ctx, bcClient, hotB)
if err != nil {
log.Fatalf("get last TX ID B error: %v", err)
}
for cur, jetton := range config.Config.Jettons {
jwA, err := bcClient.GetJettonWalletAddress(ctx, hotA, jetton.Master)
if err != nil {
log.Fatalf("get hot jetton wallet A error: %v", err)
}
jwB, err := bcClient.GetJettonWalletAddress(ctx, hotB, jetton.Master)
if err != nil {
log.Fatalf("get hot jetton wallet B error: %v", err)
}
addrA[cur] = jwA
addrB[cur] = jwB
totalProcessedAmount.WithLabelValues(cur).Set(0)
}
totalProcessedAmount.WithLabelValues(core.TonSymbol).Set(0)
p := &PayerProcessor{
client: client,
bcClient: bcClient,
depositsA: convertDeposits(bcClient, depositsA),
depositsB: convertDeposits(bcClient, depositsB),
hotWalletsAddrA: addrA,
hotWalletsAddrB: addrB,
balances: newHotBalances(),
lastTxIDA: lastTxIDA,
lastTxIDB: lastTxIDB,
knownUUIDsA: make(map[uuid.UUID]struct{}),
knownUUIDsB: make(map[uuid.UUID]struct{}),
}
return p
}
func convertDeposits(bcClient *blockchain.Connection, deposits map[string][]string) []Deposit {
var dep []Deposit
for cur, addresses := range deposits {
if cur != core.TonSymbol {
jetton := config.Config.Jettons[cur]
for _, a := range addresses {
addr, _ := address.ParseAddr(a)
jw, err := bcClient.GetJettonWalletAddress(context.Background(), addr, jetton.Master)
if err != nil {
log.Fatalf("get jetton wallet error: %v", err)
}
dep = append(dep, Deposit{Address: addr, Currency: cur, JettonWallet: jw})
}
} else {
for _, a := range addresses {
addr, err := address.ParseAddr(a)
if err != nil {
log.Fatalf("parse deposit address error: %v", err)
}
dep = append(dep, Deposit{Address: addr, Currency: cur})
}
}
}
return dep
}
func (p *PayerProcessor) Start() {
go p.balanceMonitor()
if !onlyMonitoring {
go p.startPayments(SideA)
go p.startPayments(SideB)
}
}
func (p *PayerProcessor) balanceMonitor() {
log.Infof("Balance monitor started")
startTotal := make(map[string]int64)
for {
ctx, cancel := context.WithTimeout(context.Background(), time.Second*60)
tot, err := p.updateHotWalletBalances(ctx)
if err != nil {
log.Errorf("can not update hot wallet balances: %v\n", err)
cancel()
continue
}
for _, d := range p.depositsA {
b, err := p.getDepositBalance(ctx, d)
if err != nil {
log.Errorf("can not update deposit wallet A balance: %v\n", err)
cancel()
continue
}
tot[d.Currency] = tot[d.Currency] + b
depositWalletABalance.WithLabelValues(d.Currency, d.Address.String()).Set(float64(b))
}
for _, d := range p.depositsB {
b, err := p.getDepositBalance(ctx, d)
if err != nil {
log.Errorf("can not update deposit wallet B balance: %v\n", err)
cancel()
continue
}
tot[d.Currency] = tot[d.Currency] + b
depositWalletBBalance.WithLabelValues(d.Currency, d.Address.String()).Set(float64(b))
}
for c, t := range tot {
totalBalance.WithLabelValues(c).Set(float64(t))
if _, ok := startTotal[c]; !ok {
startTotal[c] = t
}
totalLosses.WithLabelValues(c).Set(float64(t - startTotal[c]))
}
cancel()
time.Sleep(time.Millisecond * 500)
}
}
func (p *PayerProcessor) updateHotWalletBalances(ctx context.Context) (map[string]int64, error) {
totBalance := make(map[string]int64)
bA, _, err := p.bcClient.GetAccountCurrentState(ctx, p.hotWalletsAddrA[core.TonSymbol])
if err != nil {
return nil, err
}
bB, _, err := p.bcClient.GetAccountCurrentState(ctx, p.hotWalletsAddrB[core.TonSymbol])
if err != nil {
return nil, err
}
hotWalletABalance.WithLabelValues(core.TonSymbol).Set(float64(bA.Int64()))
hotWalletBBalance.WithLabelValues(core.TonSymbol).Set(float64(bB.Int64()))
totBalance[core.TonSymbol] = bA.Int64() + bB.Int64()
p.balances.WriteBalance(SideA, core.TonSymbol, bA.Int64())
p.balances.WriteBalance(SideB, core.TonSymbol, bB.Int64())
for cur := range config.Config.Jettons {
bA, err = p.bcClient.GetLastJettonBalance(ctx, p.hotWalletsAddrA[cur])
if err != nil {
return nil, err
}
bB, err = p.bcClient.GetLastJettonBalance(ctx, p.hotWalletsAddrB[cur])
if err != nil {
return nil, err
}
hotWalletABalance.WithLabelValues(cur).Set(float64(bA.Int64()))
hotWalletBBalance.WithLabelValues(cur).Set(float64(bB.Int64()))
totBalance[cur] = bA.Int64() + bB.Int64()
p.balances.WriteBalance(SideA, cur, bA.Int64())
p.balances.WriteBalance(SideB, cur, bB.Int64())
}
return totBalance, nil
}
func (p *PayerProcessor) getDepositBalance(ctx context.Context, d Deposit) (int64, error) {
if d.Currency == core.TonSymbol {
b, _, err := p.bcClient.GetAccountCurrentState(ctx, d.Address)
if err != nil {
return 0, err
}
return b.Int64(), nil
}
b, err := p.bcClient.GetLastJettonBalance(ctx, d.JettonWallet)
if err != nil {
return 0, err
}
return b.Int64(), nil
}
func (p *PayerProcessor) startPayments(side PaymentSide) {
for {
time.Sleep(time.Second * 60)
if !p.checkBalances(side) {
continue
}
ids, ww, err := p.withdrawToDeposits(side)
if err != nil {
log.Fatalf("withdraw to deposit error: %s", err)
}
log.Infof("Withdrawals sended for side %s", side)
err = p.waitWithdrawals(side, ids)
if err != nil {
log.Fatalf("wait withdrawals error: %s", err)
}
err = p.validateWithdrawals(context.TODO(), side, ww)
if err != nil {
log.Fatalf("validate withdrawals error: %s", err)
}
log.Infof("Withdrawals validated for side %s", side)
}
}
func (p *PayerProcessor) checkBalances(side PaymentSide) bool {
tonRemained := p.balances.ReadBalance(side, core.TonSymbol) - tonWithdrawAmount*depositsQty
if tonRemained < tonMinCutoff {
return false
}
for cur := range config.Config.Jettons {
jettonRemained := p.balances.ReadBalance(side, cur) - jettonWithdrawAmount*depositsQty
tonRemained = tonRemained - depositsQty*config.JettonTransferTonAmount.Nano().Int64()
if jettonRemained < 0 || tonRemained < tonMinCutoff {
return false
}
}
return true
}
type withdrawal struct {
To *address.Address
Amount int64
UUID uuid.UUID
}
func (p *PayerProcessor) withdrawToDeposits(fromSide PaymentSide) ([]int64, []withdrawal, error) {
var (
url string
deposits []Deposit
)
switch fromSide {
case SideA:
deposits = p.depositsB
url = p.client.urlA
case SideB:
deposits = p.depositsA
url = p.client.urlB
default:
return nil, nil, fmt.Errorf("invalid side")
}
var (
ww []withdrawal
ids []int64
)
for _, d := range deposits {
if d.Currency == core.TonSymbol {
r, u, err := p.client.SendWithdrawal(url, d.Currency, d.Address.String(), tonWithdrawAmount)
if err != nil {
return nil, nil, err
}
ww = append(ww, withdrawal{
To: d.Address,
Amount: tonWithdrawAmount,
UUID: u,
})
ids = append(ids, r.ID)
predictedTonLoss.Add(predictLoss(core.TonSymbol))
totalProcessedAmount.WithLabelValues(core.TonSymbol).Add(float64(tonWithdrawAmount))
} else {
r, u, err := p.client.SendWithdrawal(url, d.Currency, d.Address.String(), jettonWithdrawAmount)
if err != nil {
return nil, nil, err
}
ww = append(ww, withdrawal{
To: d.Address,
Amount: jettonWithdrawAmount,
UUID: u,
})
ids = append(ids, r.ID)
predictedTonLoss.Add(predictLoss(d.Currency))
totalProcessedAmount.WithLabelValues(d.Currency).Add(float64(jettonWithdrawAmount))
}
}
return ids, ww, nil
}
func (p *PayerProcessor) waitWithdrawals(fromSide PaymentSide, ids []int64) error {
var url string
switch fromSide {
case SideA:
url = p.client.urlA
case SideB:
url = p.client.urlB
default:
return fmt.Errorf("invalid side")
}
completed := make(map[int64]struct{})
for {
for _, id := range ids {
_, ok := completed[id]
if ok {
continue
}
r, err := p.client.GetWithdrawalStatus(url, id)
if err != nil {
return err
}
if r.Status == core.ProcessedStatus {
completed[id] = struct{}{}
} else {
time.Sleep(time.Millisecond * 200)
break
}
}
if len(completed) == len(ids) {
break
}
}
return nil
}
func (p *PayerProcessor) validateWithdrawals(ctx context.Context, fromSide PaymentSide, withdrawals []withdrawal) error {
var (
addr *address.Address
lastTxID TxID
newUUIDs []uuid.UUID
)
switch fromSide {
case SideA:
addr = p.hotWalletsAddrA[core.TonSymbol]
lastTxID = p.lastTxIDA
case SideB:
addr = p.hotWalletsAddrB[core.TonSymbol]
lastTxID = p.lastTxIDB
default:
return fmt.Errorf("invalid side")
}
txs, err := p.loadTXs(ctx, lastTxID, addr)
if err != nil {
return err
}
if len(txs) > 0 {
lastTxID = TxID{
Lt: txs[0].LT,
Hash: txs[0].Hash,
}
}
remainingTXs := withdrawals
for _, tx := range txs {
ww, uuids, err := parseTX(tx)
if err != nil {
return err
}
newUUIDs = append(newUUIDs, uuids...)
remainingTXs = compareWithdrawals(ww, remainingTXs)
}
if len(remainingTXs) > 0 {
var s string
for _, r := range remainingTXs {
s = s + r.UUID.String() + ", "
}
return fmt.Errorf("can not find withdrawals: %s", s)
}
switch fromSide {
case SideA:
p.lastTxIDA = lastTxID
err := checkDoubleSpending(newUUIDs, p.knownUUIDsA)
if err != nil {
return err
}
case SideB:
p.lastTxIDB = lastTxID
err := checkDoubleSpending(newUUIDs, p.knownUUIDsB)
if err != nil {
return err
}
}
return nil
}
func checkDoubleSpending(newUUIDs []uuid.UUID, knownUUIDs map[uuid.UUID]struct{}) error {
for _, u := range newUUIDs {
_, ok := knownUUIDs[u]
if ok {
return fmt.Errorf("double spending: %s", u.String())
}
knownUUIDs[u] = struct{}{}
}
return nil
}
func (p *PayerProcessor) loadTXs(ctx context.Context, lastTxID TxID, addr *address.Address) ([]*tlb.Transaction, error) {
newTxID, err := getLastTxID(ctx, p.bcClient, addr)
if err != nil {
return nil, err
}
currentTxID := newTxID
txs := make([]*tlb.Transaction, 0)
for {
// last transaction has 0 prev lt
if currentTxID.Lt == 0 {
break
}
list, err := p.bcClient.ListTransactions(ctx, addr, 3, currentTxID.Lt, currentTxID.Hash)
if err != nil {
return nil, err
}
// oldest = first in list
for i := len(list) - 1; i >= 0; i-- {
if bytes.Equal(list[i].Hash, lastTxID.Hash) {
return txs, nil
}
txs = append(txs, list[i])
}
// set previous info from the oldest transaction in list
currentTxID.Hash = list[0].PrevTxHash
currentTxID.Lt = list[0].PrevTxLT
}
return nil, fmt.Errorf("can not get txs")
}
func parseTX(tx *tlb.Transaction) ([]withdrawal, []uuid.UUID, error) {
var (
ww []withdrawal
uuids []uuid.UUID
msgList []tlb.Message
err error
)
if tx.OutMsgCount > 0 {
msgList, err = tx.IO.Out.ToSlice()
if err != nil {
return nil, nil, err
}
}
for _, m := range msgList {
if m.MsgType != tlb.MsgTypeInternal {
continue
}
msg := m.AsInternal()
jt, err := core.DecodeJettonTransfer(msg)
if err == nil {
u, err := uuid.FromString(jt.Comment)
if err != nil {
return nil, nil, err
}
ww = append(ww, withdrawal{
To: jt.Destination,
Amount: jt.Amount.BigInt().Int64(),
UUID: u,
})
uuids = append(uuids, u)
} else if msg.Body.BitsSize() > 32 {
u, err := uuid.FromString(core.LoadComment(msg.Body))
if err != nil {
return nil, nil, err
}
ww = append(ww, withdrawal{
To: msg.DstAddr,
Amount: msg.Amount.Nano().Int64(),
UUID: u,
})
uuids = append(uuids, u)
}
}
return ww, uuids, nil
}
func compareWithdrawals(all, target []withdrawal) []withdrawal {
var res []withdrawal
for _, t := range target {
found := false
for _, a := range all {
if t.UUID == a.UUID {
found = true
break
}
}
if !found {
res = append(res, t)
}
}
return res
}
func getLastTxID(ctx context.Context, bcClient *blockchain.Connection, address *address.Address) (TxID, error) {
b, err := bcClient.GetMasterchainInfo(ctx)
if err != nil {
return TxID{}, err
}
res, err := bcClient.GetAccount(ctx, b, address)
if err != nil {
return TxID{}, err
}
return TxID{
Lt: res.LastTxLT,
Hash: res.LastTxHash,
}, nil
}
func predictLoss(currency string) float64 {
if currency == core.TonSymbol {
cutoff := config.Config.Ton.Withdrawal.Int64()
n := math.Ceil(float64(cutoff) / float64(tonWithdrawAmount)) // number of replenishments of the deposit before withdrawal
return float64(TonLossForTonExternalWithdrawal) + float64(TonLossForTonInternalWithdrawal)/n
} else {
cutoff := config.Config.Jettons[currency].WithdrawalCutoff.Int64()
n := math.Ceil(float64(cutoff) / float64(jettonWithdrawAmount)) // number of replenishments of the deposit before withdrawal
return float64(TonLossForJettonExternalWithdrawal) + float64(TonLossForJettonInternalWithdrawal)/n
}
}
================================================
FILE: cmd/testwebhook/main.go
================================================
package main
import (
"crypto/subtle"
"fmt"
"io"
"net/http"
"strings"
)
func main() {
http.HandleFunc("/webhook", getNotification)
fmt.Printf("webhook listener started\n")
err := http.ListenAndServe(":3333", nil)
if err != nil {
panic(err)
}
}
func getNotification(resp http.ResponseWriter, req *http.Request) {
if req.Method != http.MethodPost {
fmt.Printf("Not a post request!\n")
}
checkToken(req, "123")
res, err := io.ReadAll(req.Body)
if err != nil {
fmt.Printf("notification read error: %v", err)
return
}
_ = req.Body.Close()
fmt.Printf("Notification: %s\n", res)
resp.WriteHeader(http.StatusOK)
}
func checkToken(req *http.Request, token string) {
header := req.Header.Get("authorization")
if header == "" {
fmt.Printf("no authorization header\n")
return
}
auth := strings.Split(header, " ")
if len(auth) != 2 || auth[0] != "Bearer" {
fmt.Printf("not Bearer token\n")
return
}
if x := subtle.ConstantTimeCompare([]byte(auth[1]), []byte(token)); x == 1 {
return
} // constant time comparison to prevent time attack
fmt.Printf("invalid token\n")
return
}
================================================
FILE: config/config.go
================================================
package config
import (
"log"
"math/big"
"strings"
"time"
"github.com/caarlos0/env/v6"
"github.com/shopspring/decimal"
"github.com/tonkeeper/tongo/boc"
"github.com/xssnick/tonutils-go/address"
"github.com/xssnick/tonutils-go/tlb"
)
const MaxJettonForwardTonAmount = 20_000_000
var (
JettonTransferTonAmount = tlb.FromNanoTONU(100_000_000)
JettonForwardAmount = tlb.FromNanoTONU(MaxJettonForwardTonAmount) // must be < JettonTransferTonAmount
JettonInternalForwardAmount = tlb.FromNanoTONU(1)
DefaultHotWalletHysteresis = decimal.NewFromFloat(0.95) // `hot_wallet_residual_balance` = `hot_wallet_max_balance` * `hysteresis`
ExternalMessageLifetime = 50 * time.Second
ExternalWithdrawalPeriod = 80 * time.Second // must be ExternalWithdrawalPeriod > ExternalMessageLifetime and some time for balance update
InternalWithdrawalPeriod = 80 * time.Second
ExpirationProcessorPeriod = 5 * time.Second
AllowableBlockchainLagging = 40 * time.Second // TODO: use env var
AllowableServiceToNodeTimeDiff = 2 * time.Second
)
// JettonProxyContractCode source code at https://github.com/gobicycle/ton-proxy-contract
const JettonProxyContractCode = "B5EE9C72410102010037000114FF00F4A413F4BCF2C80B010050D33331D0D3030171B0915BE0FA4030ED44D0FA4030C705F2E1939320D74A97D4018100A0FB00E8301E8A9040"
const MaxCommentLength = 1000 // qty in chars
var Config = struct {
LiteServer string `env:"LITESERVER,required"`
LiteServerKey string `env:"LITESERVER_KEY,required"`
LiteServerRateLimit int `env:"LITESERVER_RATE_LIMIT" envDefault:"100"`
Seed string `env:"SEED,required"`
DatabaseURI string `env:"DB_URI,required"`
APIPort int `env:"API_PORT,required"`
APIToken string `env:"API_TOKEN,required"`
Testnet bool `env:"IS_TESTNET" envDefault:"true"`
ColdWalletString string `env:"COLD_WALLET"`
JettonString string `env:"JETTONS"`
TonString string `env:"TON_CUTOFFS,required"`
IsDepositSideCalculation bool `env:"DEPOSIT_SIDE_BALANCE" envDefault:"true"` // TODO: rename to DEPOSIT_SIDE_CALCULATION
QueueURI string `env:"QUEUE_URI"`
QueueName string `env:"QUEUE_NAME"`
QueueEnabled bool `env:"QUEUE_ENABLED" envDefault:"false"`
ProofCheckEnabled bool `env:"PROOF_CHECK_ENABLED" envDefault:"false"`
NetworkConfigUrl string `env:"NETWORK_CONFIG_URL"`
WebhookEndpoint string `env:"WEBHOOK_ENDPOINT"`
WebhookToken string `env:"WEBHOOK_TOKEN"`
AllowableLaggingSec int `env:"ALLOWABLE_LAG"`
ForwardTonAmount int `env:"FORWARD_TON_AMOUNT" envDefault:"1"`
Jettons map[string]Jetton
Ton Cutoffs
ColdWallet *address.Address
BlockchainConfig *boc.Cell
}{}
type Jetton struct {
Master *address.Address
WithdrawalCutoff *big.Int
HotWalletMaxCutoff *big.Int
HotWalletResidual *big.Int
}
type Cutoffs struct {
HotWalletMin *big.Int
HotWalletMax *big.Int
Withdrawal *big.Int
HotWalletResidual *big.Int
}
func GetConfig() {
err := env.Parse(&Config)
if err != nil {
log.Fatalf("Can not load config: %v", err)
}
Config.Jettons = parseJettonString(Config.JettonString)
Config.Ton = parseTonString(Config.TonString)
if Config.ForwardTonAmount < 0 || Config.ForwardTonAmount > MaxJettonForwardTonAmount {
log.Fatalf("Forward TON amount for jetton transfer must be positive and less than %d", MaxJettonForwardTonAmount)
} else {
JettonForwardAmount = tlb.FromNanoTONU(uint64(Config.ForwardTonAmount))
}
if Config.ColdWalletString != "" {
coldAddr, err := address.ParseAddr(Config.ColdWalletString)
if err != nil {
log.Fatalf("Can not parse cold wallet address: %v", err)
}
if coldAddr.Type() != address.StdAddress {
log.Fatalf("Only std cold wallet address supported")
}
if coldAddr.IsTestnetOnly() && !Config.Testnet {
log.Fatalf("Can not use testnet cold wallet address for mainnet")
}
Config.ColdWallet = coldAddr
}
if Config.AllowableLaggingSec != 0 {
AllowableBlockchainLagging = time.Second * time.Duration(Config.AllowableLaggingSec)
}
}
func parseJettonString(s string) map[string]Jetton {
res := make(map[string]Jetton)
if s == "" {
return res
}
jettons := strings.Split(s, ",")
for _, j := range jettons {
data := strings.Split(j, ":")
if len(data) != 4 && len(data) != 5 {
log.Fatalf("invalid jetton data")
}
cur := data[0]
addr, err := address.ParseAddr(data[1])
if err != nil {
log.Fatalf("invalid jetton address: %v", err)
}
maxCutoff, err := decimal.NewFromString(data[2])
if err != nil {
log.Fatalf("invalid %v jetton max cutoff: %v", data[0], err)
}
withdrawalCutoff, err := decimal.NewFromString(data[3])
if err != nil {
log.Fatalf("invalid %v jetton withdrawal cutoff: %v", data[0], err)
}
residual := maxCutoff.Mul(DefaultHotWalletHysteresis)
if len(data) == 5 {
residual, err = decimal.NewFromString(data[4])
if err != nil {
log.Fatalf("invalid hot_wallet_residual_balance parameter: %v", err)
}
}
res[cur] = Jetton{
Master: addr,
WithdrawalCutoff: withdrawalCutoff.BigInt(),
HotWalletMaxCutoff: maxCutoff.BigInt(),
HotWalletResidual: residual.BigInt(),
}
}
return res
}
func parseTonString(s string) Cutoffs {
data := strings.Split(s, ":")
if len(data) != 3 && len(data) != 4 {
log.Fatalf("invalid TON cuttofs")
}
hotWalletMin, err := decimal.NewFromString(data[0])
if err != nil {
log.Fatalf("invalid TON hot wallet min cutoff: %v", err)
}
hotWalletMax, err := decimal.NewFromString(data[1])
if err != nil {
log.Fatalf("invalid TON hot wallet max cutoff: %v", err)
}
withdrawal, err := decimal.NewFromString(data[2])
if err != nil {
log.Fatalf("invalid TON withdrawal cutoff: %v", err)
}
if hotWalletMin.Cmp(hotWalletMax) == 1 {
log.Fatalf("TON hot wallet max cutoff must be greater than TON hot wallet min cutoff")
}
residual := hotWalletMax.Mul(DefaultHotWalletHysteresis)
if len(data) == 4 {
residual, err = decimal.NewFromString(data[3])
if err != nil {
log.Fatalf("invalid hot_wallet_residual_balance parameter: %v", err)
}
}
return Cutoffs{
HotWalletMin: hotWalletMin.BigInt(),
HotWalletMax: hotWalletMax.BigInt(),
Withdrawal: withdrawal.BigInt(),
HotWalletResidual: residual.BigInt(),
}
}
================================================
FILE: core/block_scanner.go
================================================
package core
import (
"context"
"fmt"
"math/big"
"sync"
"time"
"github.com/gobicycle/bicycle/audit"
"github.com/gobicycle/bicycle/config"
"github.com/gofrs/uuid"
log "github.com/sirupsen/logrus"
"github.com/tonkeeper/tongo"
"github.com/tonkeeper/tongo/boc"
tongoTlb "github.com/tonkeeper/tongo/tlb"
"github.com/xssnick/tonutils-go/address"
"github.com/xssnick/tonutils-go/tlb"
"github.com/xssnick/tonutils-go/ton"
"github.com/xssnick/tonutils-go/tvm/cell"
)
type BlockScanner struct {
db storage
blockchain blockchain
shard byte
tracker blocksTracker
wg *sync.WaitGroup
notificators []Notificator
}
type transactions struct {
Address Address
WalletType WalletType
Transactions []*tlb.Transaction
}
type jettonTransferNotificationMsg struct {
Amount Coins
Sender *address.Address
Comment string
}
type JettonTransferMsg struct {
Amount Coins
Destination *address.Address
Comment string
}
type HighLoadWalletExtMsgInfo struct {
UUID uuid.UUID
TTL time.Time
Messages *cell.Dictionary
}
type incomeNotification struct {
Deposit string `json:"deposit_address"`
Timestamp int64 `json:"time"`
Amount string `json:"amount"`
Source string `json:"source_address,omitempty"`
Comment string `json:"comment,omitempty"`
UserID string `json:"user_id"`
TxHash string `json:"tx_hash"`
}
func NewBlockScanner(
wg *sync.WaitGroup,
db storage,
blockchain blockchain,
shard byte,
tracker blocksTracker,
notificators []Notificator,
) *BlockScanner {
t := &BlockScanner{
db: db,
blockchain: blockchain,
shard: shard,
tracker: tracker,
wg: wg,
notificators: notificators,
}
t.wg.Add(1)
go t.Start()
return t
}
func (s *BlockScanner) Start() {
defer s.wg.Done()
log.Printf("Block scanner started")
for {
block, exit, err := s.tracker.NextBlock()
if err != nil {
log.Fatalf("get block error: %v", err)
}
if exit {
log.Printf("Block scanner stopped")
break
}
ctx, cancel := context.WithTimeout(context.Background(), time.Second*15)
err = s.processBlock(ctx, block)
if err != nil {
log.Fatalf("block processing error: %v", err)
}
cancel()
}
}
func (s *BlockScanner) Stop() {
s.tracker.Stop()
}
func (s *BlockScanner) processBlock(ctx context.Context, block ShardBlockHeader) error {
txIDs, err := s.blockchain.GetTransactionIDsFromBlock(ctx, block.BlockIDExt)
if err != nil {
return err
}
filteredTXs, err := s.filterTXs(ctx, block.BlockIDExt, txIDs)
if err != nil {
return err
}
e, err := s.processTXs(ctx, filteredTXs, block)
if err != nil {
return err
}
err = s.db.SaveParsedBlockData(ctx, e)
if err != nil {
return err
}
// Push notifications after saving to the database.
// Prevents duplicate sending on restart, but may result in lost notifications.
return s.pushNotifications(e)
}
func (s *BlockScanner) pushNotifications(e BlockEvents) error {
if len(s.notificators) == 0 {
return nil
}
if config.Config.IsDepositSideCalculation {
for _, ei := range e.ExternalIncomes {
err := s.pushNotification(ei.To, ei.Amount, ei.Utime, ei.From, ei.FromWorkchain, ei.Comment, ei.TxHash)
if err != nil {
return err
}
}
} else {
for _, ii := range e.InternalIncomes {
err := s.pushNotification(ii.From, ii.Amount, ii.Utime, nil, nil, "", ii.TxHash)
if err != nil {
return err
}
}
}
return nil
}
func (s *BlockScanner) pushNotification(
addr Address,
amount Coins,
timestamp uint32,
from []byte,
fromWorkchain *int32,
comment string,
txHash []byte,
) error {
owner := s.db.GetOwner(addr)
if owner != nil {
addr = *owner
}
userID, ok := s.db.GetUserID(addr)
if !ok {
return fmt.Errorf("not found UserID for deposit %s", addr.ToUserFormat())
}
notification := incomeNotification{
Deposit: addr.ToUserFormat(),
Amount: amount.String(),
Timestamp: int64(timestamp),
Comment: comment,
UserID: userID,
TxHash: fmt.Sprintf("%x", txHash),
}
if len(from) == 32 && fromWorkchain != nil {
// supports only std address
src := address.NewAddress(0, byte(*fromWorkchain), from)
src.SetTestnetOnly(config.Config.Testnet)
notification.Source = src.String()
}
for _, n := range s.notificators {
err := n.Publish(notification)
if err != nil {
return err
}
}
return nil
}
func (s *BlockScanner) filterTXs(
ctx context.Context,
blockID *ton.BlockIDExt,
ids []ton.TransactionShortInfo,
) (
[]transactions, error,
) {
txMap := make(map[Address][]*tlb.Transaction)
for _, id := range ids {
a, err := AddressFromBytes(id.Account) // must be int256 for lite api
if err != nil {
return nil, err
}
_, ok := s.db.GetWalletType(a)
if ok {
tx, err := s.blockchain.GetTransactionFromBlock(ctx, blockID, id)
if err != nil {
return nil, err
}
txMap[a] = append(txMap[a], tx)
}
}
var res []transactions
for a, txs := range txMap {
wType, _ := s.db.GetWalletType(a)
res = append(res, transactions{a, wType, txs})
}
return res, nil
}
func checkTxForSuccess(tx *tlb.Transaction) (bool, error) {
cell1, err := tlb.ToCell(tx.Description)
if err != nil {
return false, err
}
c, err := boc.DeserializeBoc(cell1.ToBOC())
if err != nil {
return false, err
}
var desc tongoTlb.TransactionDescr
err = tongoTlb.Unmarshal(c[0], &desc)
if err != nil {
return false, err
}
var fakeTx tongo.Transaction // need for check tx success via tongo
fakeTx.Description = desc
return fakeTx.IsSuccess(), nil
}
func (s *BlockScanner) processTXs(
ctx context.Context,
txs []transactions,
block ShardBlockHeader,
) (
BlockEvents, error,
) {
blockEvents := BlockEvents{Block: block}
for _, t := range txs {
switch t.WalletType {
// TODO: check order of Lt for different accounts (it is important for intermediate tx Lt)
case TonHotWallet:
hotWalletEvents, err := s.processTonHotWalletTXs(t)
if err != nil {
return BlockEvents{}, err
}
blockEvents.Append(hotWalletEvents)
case TonDepositWallet:
tonDepositEvents, err := s.processTonDepositWalletTXs(t)
if err != nil {
return BlockEvents{}, err
}
blockEvents.Append(tonDepositEvents)
case JettonDepositWallet:
jettonDepositEvents, err := s.processJettonDepositWalletTXs(ctx, t, block.BlockIDExt, block.Parent)
if err != nil {
return BlockEvents{}, err
}
blockEvents.Append(jettonDepositEvents)
}
}
return blockEvents, nil
}
func (s *BlockScanner) processTonHotWalletTXs(txs transactions) (Events, error) {
var events Events
for _, tx := range txs.Transactions {
if tx.IO.In == nil { // impossible for standard highload TON wallet
audit.LogTX(audit.Error, string(TonHotWallet), tx.Hash, "transaction without in message")
return Events{}, fmt.Errorf("anomalous behavior of the TON hot wallet")
}
switch tx.IO.In.MsgType {
case tlb.MsgTypeExternalIn:
e, err := s.processTonHotWalletExternalInMsg(tx)
if err != nil {
return Events{}, err
}
events.Append(e)
case tlb.MsgTypeInternal:
e, err := s.processTonHotWalletInternalInMsg(tx)
if err != nil {
return Events{}, err
}
events.Append(e)
default:
audit.LogTX(audit.Error, string(TonHotWallet), tx.Hash,
"transaction in message must be internal or external in")
return Events{}, fmt.Errorf("anomalous behavior of the TON hot wallet")
}
}
return events, nil
}
func (s *BlockScanner) processTonDepositWalletTXs(txs transactions) (Events, error) {
var events Events
for _, tx := range txs.Transactions {
if tx.IO.In == nil { // impossible for standard TON V3 wallet
audit.LogTX(audit.Error, string(TonDepositWallet), tx.Hash, "transaction without in message")
return Events{}, fmt.Errorf("anomalous behavior of the deposit TON wallet")
}
switch tx.IO.In.MsgType {
case tlb.MsgTypeExternalIn:
// internal withdrawal. spam or invalid external cannot invoke tx
// theoretically will be up to 4 out messages for TON V3 wallet
// external_in msg without out_msg very rare or impossible
// it is not critical for internal transfers (double spending not dangerous).
success, err := checkTxForSuccess(tx)
if err != nil {
return Events{}, err
}
if !success {
audit.LogTX(audit.Info, string(TonDepositWallet), tx.Hash, "failed transaction")
continue
}
e, err := s.processTonDepositWalletExternalInMsg(tx)
if err != nil {
return Events{}, err
}
events.Append(e)
case tlb.MsgTypeInternal:
// external payment income
// internal message can not invoke out message for TON wallet V3 except of bounce
// bounced filtered by len(tx.IO.Out) != 0
if tx.OutMsgCount != 0 {
audit.LogTX(audit.Info, string(TonDepositWallet), tx.Hash, "ton deposit filling is bounced")
continue
}
e, err := s.processTonDepositWalletInternalInMsg(tx)
if err != nil {
return Events{}, err
}
events.Append(e)
default:
audit.LogTX(audit.Error, string(TonDepositWallet), tx.Hash,
"transaction in message must be internal or external in")
return Events{}, fmt.Errorf("anomalous behavior of the deposit TON wallet")
}
}
return events, nil
}
func (s *BlockScanner) processJettonDepositWalletTXs(
ctx context.Context,
txs transactions,
blockID, prevBlockID *ton.BlockIDExt,
) (Events, error) {
var (
unknownTransactions []*tlb.Transaction
events Events
)
knownIncomeAmount := big.NewInt(0)
totalWithdrawalsAmount := big.NewInt(0)
for _, tx := range txs.Transactions {
e, knownAmount, outUnknownFound, err := s.processJettonDepositOutMsgs(tx)
if err != nil {
return Events{}, err
}
knownIncomeAmount.Add(knownIncomeAmount, knownAmount)
events.Append(e)
e, totalAmount, inUnknownFound, err := s.processJettonDepositInMsg(tx)
if err != nil {
return Events{}, err
}
totalWithdrawalsAmount.Add(totalWithdrawalsAmount, totalAmount)
events.Append(e)
if outUnknownFound || inUnknownFound { // if found some unknown messages that potentially can change Jetton balance
unknownTransactions = append(unknownTransactions, tx)
}
}
unknownIncomeAmount, err := s.calculateJettonAmounts(ctx, txs.Address, prevBlockID, blockID, knownIncomeAmount, totalWithdrawalsAmount)
if err != nil {
return Events{}, err
}
if unknownIncomeAmount.Cmp(big.NewInt(0)) == 1 { // unknownIncomeAmount > 0
unknownIncomes, err := convertUnknownJettonTxs(unknownTransactions, txs.Address, unknownIncomeAmount)
if err != nil {
return Events{}, err
}
events.ExternalIncomes = append(events.ExternalIncomes, unknownIncomes...)
}
return events, nil
}
func (s *BlockScanner) calculateJettonAmounts(
ctx context.Context,
address Address,
prevBlockID, blockID *ton.BlockIDExt,
knownIncomeAmount, totalWithdrawalsAmount *big.Int,
) (
unknownIncomeAmount *big.Int,
err error,
) {
prevBalance, err := s.blockchain.GetJettonBalance(ctx, address, prevBlockID)
if err != nil {
return nil, err
}
currentBalance, err := s.blockchain.GetJettonBalance(ctx, address, blockID)
if err != nil {
return nil, err
}
diff := big.NewInt(0)
diff.Sub(currentBalance, prevBalance) // diff = currentBalance - prevBalance
totalIncomeAmount := big.NewInt(0)
totalIncomeAmount.Add(diff, totalWithdrawalsAmount) // totalIncomeAmount = diff + totalWithdrawalsAmount
unknownIncomeAmount = big.NewInt(0)
unknownIncomeAmount.Sub(totalIncomeAmount, knownIncomeAmount) // unknownIncomeAmount = totalIncomeAmount - knownIncomeAmount
return unknownIncomeAmount, nil
}
func convertUnknownJettonTxs(txs []*tlb.Transaction, addr Address, amount *big.Int) ([]ExternalIncome, error) {
var incomes []ExternalIncome
for _, tx := range txs { // unknown sender (jetton wallet owner). do not save message sender as from.
incomes = append(incomes, ExternalIncome{
Utime: tx.Now,
Lt: tx.LT,
To: addr,
Amount: ZeroCoins(),
TxHash: tx.Hash,
})
}
if len(txs) > 0 {
incomes = append(incomes, ExternalIncome{
Utime: txs[0].Now, // mark unknown tx with first tx time
Lt: txs[0].LT,
To: addr,
Amount: NewCoins(amount),
TxHash: txs[0].Hash,
})
}
return incomes, nil
}
func decodeJettonTransferNotification(msg *tlb.InternalMessage) (jettonTransferNotificationMsg, error) {
if msg == nil {
return jettonTransferNotificationMsg{}, fmt.Errorf("nil msg")
}
payload := msg.Payload()
if payload == nil {
return jettonTransferNotificationMsg{}, fmt.Errorf("empty payload")
}
var notification struct {
_ tlb.Magic `tlb:"#7362d09c"`
QueryID uint64 `tlb:"## 64"`
Amount tlb.Coins `tlb:"."`
Sender *address.Address `tlb:"addr"`
ForwardPayload *cell.Cell `tlb:"either . ^"`
}
err := tlb.LoadFromCell(¬ification, payload.BeginParse())
if err != nil {
return jettonTransferNotificationMsg{}, err
}
return jettonTransferNotificationMsg{
Sender: notification.Sender,
Amount: NewCoins(notification.Amount.Nano()),
Comment: LoadComment(notification.ForwardPayload),
}, nil
}
func DecodeJettonTransfer(msg *tlb.InternalMessage) (JettonTransferMsg, error) {
if msg == nil {
return JettonTransferMsg{}, fmt.Errorf("nil msg")
}
payload := msg.Payload()
if payload == nil {
return JettonTransferMsg{}, fmt.Errorf("empty payload")
}
var transfer struct {
_ tlb.Magic `tlb:"#0f8a7ea5"`
QueryID uint64 `tlb:"## 64"`
Amount tlb.Coins `tlb:"."`
Destination *address.Address `tlb:"addr"`
ResponseDestination *address.Address `tlb:"addr"`
CustomPayload *cell.Cell `tlb:"maybe ^"`
ForwardTonAmount tlb.Coins `tlb:"."`
ForwardPayload *cell.Cell `tlb:"either . ^"`
}
err := tlb.LoadFromCell(&transfer, payload.BeginParse())
if err != nil {
return JettonTransferMsg{}, err
}
return JettonTransferMsg{
NewCoins(transfer.Amount.Nano()),
transfer.Destination,
LoadComment(transfer.ForwardPayload),
}, nil
}
func decodeJettonExcesses(msg *tlb.InternalMessage) (int64, error) {
if msg == nil {
return 0, fmt.Errorf("nil msg")
}
payload := msg.Payload()
if payload == nil {
return 0, fmt.Errorf("empty payload")
}
var excesses struct {
_ tlb.Magic `tlb:"#d53276db"`
QueryID uint64 `tlb:"## 64"`
}
err := tlb.LoadFromCell(&excesses, payload.BeginParse())
if err != nil {
return 0, err
}
return int64(excesses.QueryID), nil
}
func parseExternalMessage(msg *tlb.ExternalMessage) (
u uuid.UUID,
addrMap map[Address]struct{},
isValidWithdrawal bool,
err error,
) {
if msg == nil {
return uuid.UUID{}, nil, false, fmt.Errorf("nil msg")
}
addrMap = make(map[Address]struct{})
info, err := getHighLoadWalletExtMsgInfo(msg)
if err != nil {
return uuid.UUID{}, nil, false, err
}
for _, m := range info.Messages.All() {
var (
intMsg tlb.InternalMessage
addr Address
)
msgCell, err := m.Value.BeginParse().LoadRef()
if err != nil {
return uuid.UUID{}, nil, false, err
}
err = tlb.LoadFromCell(&intMsg, msgCell)
if err != nil {
return uuid.UUID{}, nil, false, err
}
jettonTransfer, err := DecodeJettonTransfer(&intMsg)
if err == nil {
addr, err = AddressFromTonutilsAddress(jettonTransfer.Destination)
if err != nil {
return uuid.UUID{}, nil, false, nil
}
} else {
addr, err = AddressFromTonutilsAddress(intMsg.DstAddr)
if err != nil {
return uuid.UUID{}, nil, false, nil
}
}
_, ok := addrMap[addr]
if ok { // not unique addresses
return uuid.UUID{}, nil, false, nil
}
addrMap[addr] = struct{}{}
}
return info.UUID, addrMap, true, nil
}
func (s *BlockScanner) failedWithdrawals(inMap map[Address]struct{}, outMap map[Address]struct{}, u uuid.UUID, txHash []byte) []ExternalWithdrawal {
var w []ExternalWithdrawal
for i := range inMap {
_, dstOk := s.db.GetWalletType(i)
if _, ok := outMap[i]; !ok && !dstOk { // !dstOk - not failed internal fee payments
w = append(w, ExternalWithdrawal{ExtMsgUuid: u, To: i, IsFailed: true, TxHash: txHash})
audit.LogTX(audit.Error, string(TonHotWallet), txHash, fmt.Sprintf("Failed external withdrawal to %v", i.ToUserFormat()))
} else if !ok && dstOk { // failed internal fee payments
// TODO: cause a fatal error or increment error counter
audit.LogTX(audit.Error, string(TonHotWallet), txHash, fmt.Sprintf("Failed internal withdrawal to %v", i.ToUserFormat()))
}
}
return w
}
func getHighLoadWalletExtMsgInfo(extMsg *tlb.ExternalMessage) (HighLoadWalletExtMsgInfo, error) {
body := extMsg.Payload()
if body == nil {
return HighLoadWalletExtMsgInfo{}, fmt.Errorf("nil body for external message")
}
hash := body.Hash() // must be 32 bytes
u, err := uuid.FromBytes(hash[:16])
if err != nil {
return HighLoadWalletExtMsgInfo{}, err
}
var data struct {
Sign []byte `tlb:"bits 512"`
SubwalletID uint32 `tlb:"## 32"`
BoundedID uint64 `tlb:"## 64"`
Messages *cell.Dictionary `tlb:"dict 16"`
}
err = tlb.LoadFromCell(&data, body.BeginParse())
if err != nil {
return HighLoadWalletExtMsgInfo{}, err
}
ttl := time.Unix(int64((data.BoundedID>>32)&0x00_00_00_00_FF_FF_FF_FF), 0)
return HighLoadWalletExtMsgInfo{UUID: u, TTL: ttl, Messages: data.Messages}, nil
}
func (s *BlockScanner) processTonHotWalletExternalInMsg(tx *tlb.Transaction) (Events, error) {
var events Events
inMsg := tx.IO.In.AsExternalIn()
// withdrawal messages must be only with different recipients for identification
u, addrMapIn, isValid, err := parseExternalMessage(inMsg)
if err != nil {
return Events{}, err
}
if !isValid {
audit.LogTX(audit.Error, string(TonHotWallet), tx.Hash, "not valid external message")
return Events{}, fmt.Errorf("not valid message")
}
addrMapOut := make(map[Address]struct{})
var outList []tlb.Message
if tx.OutMsgCount > 0 {
outList, err = tx.IO.Out.ToSlice()
if err != nil {
return Events{}, err
}
}
for _, m := range outList {
if m.MsgType != tlb.MsgTypeInternal {
audit.LogTX(audit.Error, string(TonHotWallet), tx.Hash, "not internal out message for transaction")
return Events{}, fmt.Errorf("anomalous behavior of the TON hot wallet")
}
msg := m.AsInternal()
addr, err := AddressFromTonutilsAddress(msg.DstAddr)
if err != nil {
return Events{}, fmt.Errorf("invalid address in withdrawal message")
}
dstType, dstOk := s.db.GetWalletTypeByTonutilsAddress(msg.DstAddr)
if dstOk && dstType == JettonHotWallet { // Jetton external withdrawal
jettonTransfer, err := DecodeJettonTransfer(msg)
if err != nil {
audit.LogTX(audit.Error, string(TonHotWallet), tx.Hash, "invalid jetton transfer message to hot jetton wallet")
return Events{}, fmt.Errorf("invalid jetton transfer message to hot jetton wallet")
}
a, err := AddressFromTonutilsAddress(jettonTransfer.Destination)
if err != nil {
return Events{}, fmt.Errorf("invalid address in withdrawal message")
}
events.ExternalWithdrawals = append(events.ExternalWithdrawals, ExternalWithdrawal{
ExtMsgUuid: u,
Utime: msg.CreatedAt,
Lt: msg.CreatedLT,
To: a,
Amount: jettonTransfer.Amount,
Comment: jettonTransfer.Comment,
IsFailed: false,
TxHash: tx.Hash,
})
addrMapOut[a] = struct{}{}
continue
}
if dstOk && dstType == JettonOwner { // Jetton internal withdrawal or service withdrawal
e, err := s.processTonHotWalletProxyMsg(msg)
if err != nil {
return Events{}, fmt.Errorf("jetton withdrawal error: %v", err)
}
events.Append(e)
addrMapOut[addr] = struct{}{}
continue
}
if !dstOk { // hot_wallet -> unknown_address. to filter internal fee payments
events.ExternalWithdrawals = append(events.ExternalWithdrawals, ExternalWithdrawal{
ExtMsgUuid: u,
Utime: msg.CreatedAt,
Lt: msg.CreatedLT,
To: addr,
Amount: NewCoins(msg.Amount.Nano()),
Comment: msg.Comment(),
IsFailed: false,
TxHash: tx.Hash,
})
}
addrMapOut[addr] = struct{}{}
}
events.ExternalWithdrawals = append(events.ExternalWithdrawals, s.failedWithdrawals(addrMapIn, addrMapOut, u, tx.Hash)...)
return events, nil
}
func (s *BlockScanner) processTonHotWalletProxyMsg(msg *tlb.InternalMessage) (Events, error) {
var events Events
body := msg.Payload()
internalPayload, err := body.BeginParse().LoadRef()
if err != nil {
return Events{}, fmt.Errorf("no internal payload to proxy contract: %v", err)
}
var intMsg tlb.InternalMessage
err = tlb.LoadFromCell(&intMsg, internalPayload)
if err != nil {
return Events{}, fmt.Errorf("can not decode payload message for proxy contract: %v", err)
}
destType, ok := s.db.GetWalletTypeByTonutilsAddress(intMsg.DstAddr)
// ok && destType == TonHotWallet - service TON withdrawal
// !ok - service Jetton withdrawal
if ok && destType == JettonDepositWallet { // Jetton internal withdrawal
jettonTransfer, err := DecodeJettonTransfer(&intMsg)
if err != nil {
return Events{}, fmt.Errorf("invalid jetton transfer message to deposit jetton wallet: %v", err)
}
a, err := AddressFromTonutilsAddress(jettonTransfer.Destination)
if err != nil {
return Events{}, fmt.Errorf("invalid address in withdrawal message")
}
events.SendingConfirmations = append(events.SendingConfirmations, SendingConfirmation{
Lt: msg.CreatedLT,
From: a,
Memo: jettonTransfer.Comment,
})
}
return events, nil
}
func (s *BlockScanner) processTonHotWalletInternalInMsg(tx *tlb.Transaction) (Events, error) {
var events Events
inMsg := tx.IO.In.AsInternal()
srcAddr, err := AddressFromTonutilsAddress(inMsg.SrcAddr)
if err != nil {
return Events{}, err
}
dstAddr, err := AddressFromTonutilsAddress(inMsg.DstAddr)
if err != nil {
return Events{}, err
}
srcType, srcOk := s.db.GetWalletType(srcAddr)
if !srcOk { // unknown_address -> hot_wallet. to check for external jetton transfer confirmation via excess message
queryID, err := decodeJettonExcesses(inMsg)
if err == nil {
events.WithdrawalConfirmations = append(events.WithdrawalConfirmations,
JettonWithdrawalConfirmation{queryID})
}
} else if srcOk && srcType == TonDepositWallet { // income TONs from deposit
income := InternalIncome{
Lt: inMsg.CreatedLT,
Utime: inMsg.CreatedAt,
From: srcAddr,
To: dstAddr,
Amount: NewCoins(inMsg.Amount.Nano()),
Memo: inMsg.Comment(),
IsFailed: false,
TxHash: tx.Hash,
}
success, err := checkTxForSuccess(tx)
if err != nil {
return Events{}, err
}
// TODO: check for partially failed message
if success {
events.InternalIncomes = append(events.InternalIncomes, income)
} else {
income.IsFailed = true
events.InternalIncomes = append(events.InternalIncomes, income)
}
} else if srcOk && srcType == JettonHotWallet { // income Jettons notification from Jetton hot wallet
income, err := decodeJettonTransferNotification(inMsg)
if err == nil {
sender, err := AddressFromTonutilsAddress(income.Sender)
if err != nil {
return Events{}, err
}
fromType, fromOk := s.db.GetWalletType(sender)
if !fromOk || fromType != JettonOwner { // skip transfers not from deposit wallets
return events, nil
}
events.InternalIncomes = append(events.InternalIncomes, InternalIncome{
Lt: inMsg.CreatedLT,
Utime: inMsg.CreatedAt,
From: sender, // sender == owner of jetton deposit wallet
To: srcAddr,
Amount: income.Amount,
Memo: income.Comment,
IsFailed: false,
TxHash: tx.Hash,
})
}
}
return events, nil
}
func (s *BlockScanner) processTonDepositWalletExternalInMsg(tx *tlb.Transaction) (Events, error) {
var events Events
dstAddr, err := AddressFromTonutilsAddress(tx.IO.In.AsExternalIn().DstAddr)
if err != nil {
return Events{}, err
}
var outList []tlb.Message
if tx.OutMsgCount > 0 {
outList, err = tx.IO.Out.ToSlice()
if err != nil {
return Events{}, err
}
}
for _, o := range outList {
if o.MsgType != tlb.MsgTypeInternal {
audit.LogTX(audit.Error, string(TonDepositWallet), tx.Hash, "not internal out message for transaction")
return Events{}, fmt.Errorf("anomalous behavior of the deposit TON wallet")
}
msg := o.AsInternal()
t, srcOk := s.db.GetWalletTypeByTonutilsAddress(msg.DstAddr)
if !srcOk || t != TonHotWallet {
audit.LogTX(audit.Warning, string(TonDepositWallet), tx.Hash, fmt.Sprintf("TONs withdrawal from %v to %v (not to hot wallet)",
msg.SrcAddr.String(), msg.DstAddr.String()))
continue
}
events.SendingConfirmations = append(events.SendingConfirmations, SendingConfirmation{
Lt: msg.CreatedLT,
From: dstAddr,
Memo: msg.Comment(),
})
events.InternalWithdrawals = append(events.InternalWithdrawals, InternalWithdrawal{
Utime: msg.CreatedAt,
Lt: msg.CreatedLT,
From: dstAddr,
Amount: NewCoins(msg.Amount.Nano()),
Memo: msg.Comment(),
IsFailed: false,
})
}
return events, nil
}
func (s *BlockScanner) processTonDepositWalletInternalInMsg(tx *tlb.Transaction) (Events, error) {
var (
events Events
from Address
err error
fromWorkchain *int32
)
inMsg := tx.IO.In.AsInternal()
dstAddr, err := AddressFromTonutilsAddress(inMsg.DstAddr)
if err != nil {
return Events{}, err
}
isKnownSender := false
// support only std address
if inMsg.SrcAddr.Type() == address.StdAddress {
from, err = AddressFromTonutilsAddress(inMsg.SrcAddr)
if err != nil {
return Events{}, err
}
_, isKnownSender = s.db.GetWalletType(from)
wc := inMsg.SrcAddr.Workchain()
fromWorkchain = &wc
}
if !isKnownSender { // income TONs from payer. exclude internal (hot->deposit, deposit->deposit) transfers.
events.ExternalIncomes = append(events.ExternalIncomes, ExternalIncome{
Lt: tx.LT,
Utime: tx.Now,
From: from.ToBytes(),
FromWorkchain: fromWorkchain,
To: dstAddr,
Amount: NewCoins(inMsg.Amount.Nano()),
Comment: inMsg.Comment(),
TxHash: tx.Hash,
})
}
return events, nil
}
func (s *BlockScanner) processJettonDepositOutMsgs(tx *tlb.Transaction) (Events, *big.Int, bool, error) {
var events Events
knownIncomeAmount := big.NewInt(0)
unknownMsgFound := false
var (
outList []tlb.Message
err error
)
if tx.OutMsgCount > 0 {
outList, err = tx.IO.Out.ToSlice()
if err != nil {
return Events{}, nil, false, err
}
}
for _, m := range outList { // checks for JettonTransferNotification
if m.MsgType != tlb.MsgTypeInternal {
audit.LogTX(audit.Info, string(JettonDepositWallet), tx.Hash, "sends external out message")
unknownMsgFound = true
continue
} // skip external_out msg
outMsg := m.AsInternal()
srcAddr, err := AddressFromTonutilsAddress(outMsg.SrcAddr)
if err != nil {
return Events{}, nil, false, err
}
notify, err := decodeJettonTransferNotification(outMsg)
if err != nil {
unknownMsgFound = true
continue
}
// need not check success. impossible for failed txs.
_, senderOk := s.db.GetWalletTypeByTonutilsAddress(notify.Sender)
if senderOk {
// TODO: check balance calculation for unknown transactions for service transfers
audit.LogTX(audit.Info, string(JettonDepositWallet), tx.Hash, "service Jetton transfer")
// not set unknownMsgFound = true to prevent service transfers interpretation as unknown
continue
} // some kind of internal transfer
dstAddr, err := AddressFromTonutilsAddress(outMsg.DstAddr)
if err != nil {
return Events{}, nil, false, err
}
owner := s.db.GetOwner(srcAddr)
if owner == nil {
return Events{}, nil, false, fmt.Errorf("no owner for Jetton deposit in addressbook")
}
if dstAddr != *owner {
audit.LogTX(audit.Info, string(JettonDepositWallet), tx.Hash,
"sends transfer notification message not to owner")
// interpret it as an unknown message
unknownMsgFound = true
continue
}
var (
from []byte
fromWorkchain *int32
)
if notify.Sender != nil &&
(notify.Sender.Type() == address.StdAddress || notify.Sender.Type() == address.VarAddress) {
from = notify.Sender.Data()
wc := notify.Sender.Workchain()
fromWorkchain = &wc
}
events.ExternalIncomes = append(events.ExternalIncomes, ExternalIncome{
Utime: outMsg.CreatedAt,
Lt: outMsg.CreatedLT,
From: from,
FromWorkchain: fromWorkchain,
To: srcAddr,
Amount: notify.Amount,
Comment: notify.Comment,
TxHash: tx.Hash,
})
knownIncomeAmount.Add(knownIncomeAmount, notify.Amount.BigInt())
}
return events, knownIncomeAmount, unknownMsgFound, nil
}
func (s *BlockScanner) processJettonDepositInMsg(tx *tlb.Transaction) (Events, *big.Int, bool, error) {
var events Events
unknownMsgFound := false
totalWithdrawalsAmount := big.NewInt(0)
if tx.IO.In == nil { // skip not decodable in_msg
audit.LogTX(audit.Info, string(JettonDepositWallet), tx.Hash, "transaction without in message")
// interpret it as an unknown message
return events, totalWithdrawalsAmount, true, nil
}
if tx.IO.In.MsgType != tlb.MsgTypeInternal { // skip not decodable in_msg
audit.LogTX(audit.Info, string(JettonDepositWallet), tx.Hash, "not internal in message")
// interpret it as an unknown message
return events, totalWithdrawalsAmount, true, nil
}
success, err := checkTxForSuccess(tx)
if err != nil {
return Events{}, nil, false, err
}
inMsg := tx.IO.In.AsInternal()
dstAddr, err := AddressFromTonutilsAddress(inMsg.DstAddr)
if err != nil {
return Events{}, nil, false, err
}
transfer, err := DecodeJettonTransfer(inMsg)
if err != nil {
unknownMsgFound = true
return events, totalWithdrawalsAmount, unknownMsgFound, nil
}
if !success { // failed withdrawal from deposit jetton wallet
events.InternalWithdrawals = append(events.InternalWithdrawals, InternalWithdrawal{
Utime: in
gitextract_x1h1gsmv/
├── .github/
│ └── workflows/
│ └── go.yml
├── Dockerfile
├── LICENSE
├── Makefile
├── README.md
├── api/
│ ├── example.http
│ ├── handlers.go
│ ├── http-client.env.json
│ └── middleware.go
├── audit/
│ └── log.go
├── blockchain/
│ ├── blockchain.go
│ ├── blockchain_test.go
│ ├── limited_client.go
│ └── shard_tracker.go
├── cmd/
│ ├── processor/
│ │ └── main.go
│ ├── testutil/
│ │ ├── http.go
│ │ ├── main.go
│ │ ├── metrics.go
│ │ └── utils.go
│ └── testwebhook/
│ └── main.go
├── config/
│ └── config.go
├── core/
│ ├── block_scanner.go
│ ├── models.go
│ ├── proxy.go
│ ├── wallets.go
│ └── withdrawal_processor.go
├── db/
│ ├── db.go
│ ├── db_test.go
│ └── tests/
│ ├── get-jetton-internal-withdrawal-tasks/
│ │ └── 01_data.up.sql
│ ├── get-ton-internal-withdrawal-tasks/
│ │ └── 01_data.up.sql
│ └── set-expired/
│ └── 01_data.up.sql
├── deploy/
│ ├── db/
│ │ ├── 01_init.down.sql
│ │ ├── 01_init.up.sql
│ │ └── 02_create_readonly_user.sh
│ ├── grafana/
│ │ ├── main/
│ │ │ ├── dashboards/
│ │ │ │ └── Payments.json
│ │ │ └── provisioning/
│ │ │ ├── dashboards/
│ │ │ │ └── payments.yml
│ │ │ └── datasources/
│ │ │ └── data_sources.yml
│ │ └── test/
│ │ ├── dashboards/
│ │ │ ├── Processor A.json
│ │ │ ├── Processor B.json
│ │ │ └── Test util.json
│ │ └── provisioning/
│ │ ├── dashboards/
│ │ │ └── payments.yml
│ │ └── datasources/
│ │ └── data_sources.yml
│ ├── manual_migrations/
│ │ ├── 0.1.x-0.2.0.sql
│ │ └── 0.4.x-0.5.0.sql
│ └── prometheus/
│ ├── main/
│ │ └── prometheus.yml
│ └── test/
│ └── prometheus.yml
├── docker-compose.yml
├── docs/
│ ├── api.apib
│ └── index.html
├── go.mod
├── go.sum
├── jettons.md
├── manual_migrations.md
├── manual_testing_plan.md
├── metrics/
│ └── metrics.go
├── queue/
│ └── queue.go
├── release_notes.md
├── technical_notes.md
├── tests/
│ └── docker-compose-tests.yml
├── threat_model.md
├── todo_list.md
└── webhook/
└── webhook.go
SYMBOL INDEX (419 symbols across 24 files)
FILE: api/handlers.go
type Handler (line 27) | type Handler struct
method getNewAddress (line 119) | func (h *Handler) getNewAddress(resp http.ResponseWriter, req *http.Re...
method getAddresses (line 151) | func (h *Handler) getAddresses(resp http.ResponseWriter, req *http.Req...
method sendWithdrawal (line 170) | func (h *Handler) sendWithdrawal(resp http.ResponseWriter, req *http.R...
method getSync (line 209) | func (h *Handler) getSync(resp http.ResponseWriter, req *http.Request) {
method getWithdrawalStatus (line 232) | func (h *Handler) getWithdrawalStatus(resp http.ResponseWriter, req *h...
method getIncome (line 271) | func (h *Handler) getIncome(resp http.ResponseWriter, req *http.Reques...
method getIncomeHistory (line 290) | func (h *Handler) getIncomeHistory(resp http.ResponseWriter, req *http...
method serviceTonWithdrawal (line 338) | func (h *Handler) serviceTonWithdrawal(resp http.ResponseWriter, req *...
method serviceJettonWithdrawal (line 368) | func (h *Handler) serviceJettonWithdrawal(resp http.ResponseWriter, re...
method getMetrics (line 398) | func (h *Handler) getMetrics(resp http.ResponseWriter, req *http.Reque...
method getIncomeByTx (line 415) | func (h *Handler) getIncomeByTx(resp http.ResponseWriter, req *http.Re...
method getBalance (line 444) | func (h *Handler) getBalance(resp http.ResponseWriter, req *http.Reque...
method getResolve (line 512) | func (h *Handler) getResolve(resp http.ResponseWriter, req *http.Reque...
type WithdrawalRequest (line 36) | type WithdrawalRequest struct
type ServiceTonWithdrawalRequest (line 46) | type ServiceTonWithdrawalRequest struct
type ServiceJettonWithdrawalRequest (line 50) | type ServiceJettonWithdrawalRequest struct
type WalletAddress (line 55) | type WalletAddress struct
type GetAddressesResponse (line 60) | type GetAddressesResponse struct
type WithdrawalResponse (line 64) | type WithdrawalResponse struct
type GetBalanceResponse (line 68) | type GetBalanceResponse struct
type ResolveResponse (line 75) | type ResolveResponse struct
type WithdrawalStatusResponse (line 79) | type WithdrawalStatusResponse struct
type GetIncomeResponse (line 86) | type GetIncomeResponse struct
type GetHistoryResponse (line 91) | type GetHistoryResponse struct
type GetIncomeByTxResponse (line 95) | type GetIncomeByTxResponse struct
type totalIncome (line 100) | type totalIncome struct
type income (line 106) | type income struct
function NewHandler (line 115) | func NewHandler(s storage, b blockchain, token string, shard byte, hotWa...
function RegisterHandlers (line 543) | func RegisterHandlers(mux *http.ServeMux, h *Handler) {
function generateAddress (line 559) | func generateAddress(
function getAddresses (line 635) | func getAddresses(ctx context.Context, userID string, dbConn storage) (G...
function isValidCommentLen (line 656) | func isValidCommentLen(comment string) bool {
function isValidCurrency (line 660) | func isValidCurrency(cur string) bool {
function convertWithdrawal (line 667) | func convertWithdrawal(w WithdrawalRequest) (core.WithdrawalRequest, err...
function convertTonServiceWithdrawal (line 712) | func convertTonServiceWithdrawal(s storage, w ServiceTonWithdrawalReques...
function convertJettonServiceWithdrawal (line 730) | func convertJettonServiceWithdrawal(s storage, w ServiceJettonWithdrawal...
function convertIncome (line 754) | func convertIncome(dbConn storage, totalIncomes []core.TotalIncome) GetI...
function convertOneIncome (line 784) | func convertOneIncome(dbConn storage, currency string, oneIncome core.Ex...
function convertHistory (line 810) | func convertHistory(dbConn storage, currency string, incomes []core.Exte...
function validateAddress (line 821) | func validateAddress(addr string) (core.Address, bool, error) {
type storage (line 840) | type storage interface
type blockchain (line 859) | type blockchain interface
FILE: api/middleware.go
function recoverMiddleware (line 13) | func recoverMiddleware(next func(http.ResponseWriter, *http.Request)) fu...
function authMiddleware (line 27) | func authMiddleware(next func(http.ResponseWriter, *http.Request)) func(...
function get (line 37) | func get(next func(http.ResponseWriter, *http.Request)) func(http.Respon...
function post (line 47) | func post(next func(http.ResponseWriter, *http.Request)) func(http.Respo...
function checkToken (line 57) | func checkToken(req *http.Request, token string) bool {
function writeHttpError (line 71) | func writeHttpError(resp http.ResponseWriter, status int, comment string) {
FILE: audit/log.go
type Severity (line 10) | type Severity
constant Error (line 13) | Error Severity = "ERROR"
constant Warning (line 14) | Warning Severity = "WARNING"
constant Info (line 15) | Info Severity = "INFO"
type message (line 18) | type message struct
function pushLog (line 23) | func pushLog(m message) {
function LogTX (line 37) | func LogTX(severity Severity, location string, hash []byte, text string) {
function Log (line 44) | func Log(severity Severity, location, event, text string) {
FILE: blockchain/blockchain.go
type Connection (line 30) | type Connection struct
method WaitForBlock (line 35) | func (c *Connection) WaitForBlock(seqno uint32) ton.APIClientWrapped {
method FindLastTransactionByInMsgHash (line 39) | func (c *Connection) FindLastTransactionByInMsgHash(ctx context.Contex...
method FindLastTransactionByOutMsgHash (line 44) | func (c *Connection) FindLastTransactionByOutMsgHash(ctx context.Conte...
method GenerateDefaultWallet (line 154) | func (c *Connection) GenerateDefaultWallet(seed string, isHighload boo...
method GenerateSubWallet (line 173) | func (c *Connection) GenerateSubWallet(seed string, shard byte, startS...
method GetJettonWalletAddress (line 196) | func (c *Connection) GetJettonWalletAddress(
method GetJettonBalanceByOwner (line 218) | func (c *Connection) GetJettonBalanceByOwner(
method DnsResolveSmc (line 234) | func (c *Connection) DnsResolveSmc(
method GenerateDepositJettonWalletForProxy (line 291) | func (c *Connection) GenerateDepositJettonWalletForProxy(
method getContract (line 328) | func (c *Connection) getContract(ctx context.Context, addr *address.Ad...
method GetJettonBalance (line 414) | func (c *Connection) GetJettonBalance(ctx context.Context, address cor...
method GetLastJettonBalance (line 437) | func (c *Connection) GetLastJettonBalance(ctx context.Context, address...
method GetAccountCurrentState (line 451) | func (c *Connection) GetAccountCurrentState(ctx context.Context, addre...
method DeployTonWallet (line 479) | func (c *Connection) DeployTonWallet(ctx context.Context, wallet *wall...
method GetTransactionIDsFromBlock (line 500) | func (c *Connection) GetTransactionIDsFromBlock(ctx context.Context, b...
method GetTransactionFromBlock (line 527) | func (c *Connection) GetTransactionFromBlock(ctx context.Context, bloc...
method getCurrentNodeTime (line 539) | func (c *Connection) getCurrentNodeTime(ctx context.Context) (time.Tim...
method CheckTime (line 552) | func (c *Connection) CheckTime(ctx context.Context, cutoff time.Durati...
method WaitStatus (line 571) | func (c *Connection) WaitStatus(ctx context.Context, addr *address.Add...
method GetAccount (line 594) | func (c *Connection) GetAccount(ctx context.Context, block *ton.BlockI...
method SendExternalMessage (line 606) | func (c *Connection) SendExternalMessage(ctx context.Context, msg *tlb...
method RunGetMethod (line 613) | func (c *Connection) RunGetMethod(ctx context.Context, block *ton.Bloc...
method ListTransactions (line 629) | func (c *Connection) ListTransactions(ctx context.Context, addr *addre...
method Client (line 633) | func (c *Connection) Client() ton.LiteClient {
method CurrentMasterchainInfo (line 637) | func (c *Connection) CurrentMasterchainInfo(ctx context.Context) (*ton...
method GetMasterchainInfo (line 641) | func (c *Connection) GetMasterchainInfo(ctx context.Context) (*ton.Blo...
method SendExternalMessageWaitTransaction (line 645) | func (c *Connection) SendExternalMessageWaitTransaction(ctx context.Co...
type contract (line 49) | type contract struct
function NewConnection (line 56) | func NewConnection(addr, key string, rateLimit int) (*Connection, error) {
function getConfigData (line 111) | func getConfigData(ctx context.Context, api ton.APIClientWrapped) (*addr...
function getWalletRecord (line 258) | func getWalletRecord(d *dns.Domain) *address.Address {
function getJettonWalletAddressByTVM (line 365) | func getJettonWalletAddressByTVM(
function newEmulator (line 402) | func newEmulator(code, data *boc.Cell) (*tvm.Emulator, error) {
function inShard (line 535) | func inShard(addr core.Address, shard byte) bool {
function getBlockchainConfig (line 649) | func getBlockchainConfig(ctx context.Context, client ton.LiteClient, blo...
FILE: blockchain/blockchain_test.go
function connect (line 24) | func connect(t *testing.T) *Connection {
function getSeed (line 40) | func getSeed() string {
function Test_NewConnection (line 48) | func Test_NewConnection(t *testing.T) {
function Test_GenerateDefaultWallet (line 52) | func Test_GenerateDefaultWallet(t *testing.T) {
function Test_GenerateSubWallet (line 77) | func Test_GenerateSubWallet(t *testing.T) {
function Test_GetJettonWalletAddress (line 96) | func Test_GetJettonWalletAddress(t *testing.T) {
function Test_GenerateJettonWalletAddressForProxy (line 120) | func Test_GenerateJettonWalletAddressForProxy(t *testing.T) {
function Test_GetJettonBalance (line 159) | func Test_GetJettonBalance(t *testing.T) {
function Test_GetAccountCurrentState (line 185) | func Test_GetAccountCurrentState(t *testing.T) {
function Test_DeployTonWallet (line 205) | func Test_DeployTonWallet(t *testing.T) {
function Test_GetTransactionIDsFromBlock (line 273) | func Test_GetTransactionIDsFromBlock(t *testing.T) {
function Test_GetTransactionFromBlock (line 287) | func Test_GetTransactionFromBlock(t *testing.T) {
function Test_CheckTime (line 313) | func Test_CheckTime(t *testing.T) {
function Test_WaitStatus (line 333) | func Test_WaitStatus(t *testing.T) {
function Test_GetAccount (line 347) | func Test_GetAccount(t *testing.T) {
function Test_RunGetMethod (line 363) | func Test_RunGetMethod(t *testing.T) {
function Test_NextBlock (line 379) | func Test_NextBlock(t *testing.T) {
function Test_Stop (line 394) | func Test_Stop(t *testing.T) {
FILE: blockchain/limited_client.go
type limitedLiteClient (line 12) | type limitedLiteClient struct
method QueryLiteserver (line 24) | func (w *limitedLiteClient) QueryLiteserver(ctx context.Context, paylo...
method StickyContext (line 32) | func (w *limitedLiteClient) StickyContext(ctx context.Context) context...
method StickyNodeID (line 36) | func (w *limitedLiteClient) StickyNodeID(ctx context.Context) uint32 {
method StickyContextNextNode (line 40) | func (w *limitedLiteClient) StickyContextNextNode(ctx context.Context)...
method StickyContextNextNodeBalanced (line 44) | func (w *limitedLiteClient) StickyContextNextNodeBalanced(ctx context....
function newLimitedClient (line 17) | func newLimitedClient(lc ton.LiteClient, rateLimit int) *limitedLiteClie...
FILE: blockchain/shard_tracker.go
constant ErrBlockNotApplied (line 14) | ErrBlockNotApplied = "block is not applied"
constant ErrBlockNotInDB (line 15) | ErrBlockNotInDB = "code 651"
type ShardTracker (line 17) | type ShardTracker struct
method NextBlock (line 44) | func (s *ShardTracker) NextBlock() (core.ShardBlockHeader, bool, error) {
method Stop (line 71) | func (s *ShardTracker) Stop() {
method getNext (line 75) | func (s *ShardTracker) getNext() *core.ShardBlockHeader {
method getNextMasterBlockID (line 84) | func (s *ShardTracker) getNextMasterBlockID(ctx context.Context) (*ton...
method loadShardBlocksBatch (line 104) | func (s *ShardTracker) loadShardBlocksBatch(masterBlockID *ton.BlockID...
method getShardBlocksRecursively (line 139) | func (s *ShardTracker) getShardBlocksRecursively(i *ton.BlockIDExt, ba...
function NewShardTracker (line 29) | func NewShardTracker(shard byte, startBlock *ton.BlockIDExt, connection ...
function isInShard (line 181) | func isInShard(blockShardPrefix uint64, shard byte) bool {
function filterByShard (line 194) | func filterByShard(headers []*ton.BlockIDExt, shard byte) *ton.BlockIDExt {
function convertBlockToShardHeader (line 204) | func convertBlockToShardHeader(block *tlb.Block, info *ton.BlockIDExt, s...
method getShardBlocksHeader (line 221) | func (c *Connection) getShardBlocksHeader(ctx context.Context, shardBloc...
function isNotReadyError (line 240) | func isNotReadyError(err error) bool {
FILE: cmd/processor/main.go
function main (line 25) | func main() {
FILE: cmd/testutil/http.go
type Client (line 18) | type Client struct
method InitDeposits (line 37) | func (s *Client) InitDeposits(host string) (map[string][]string, error) {
method GetAllAddresses (line 69) | func (s *Client) GetAllAddresses(host string) (api.GetAddressesRespons...
method SendWithdrawal (line 101) | func (s *Client) SendWithdrawal(host, currency, destination string, am...
method GetNewAddress (line 150) | func (s *Client) GetNewAddress(host, currency string) (string, error) {
method GetWithdrawalStatus (line 196) | func (s *Client) GetWithdrawalStatus(host string, id int64) (api.Withd...
function NewClient (line 26) | func NewClient(urlA, urlB, token, userID string) *Client {
FILE: cmd/testutil/main.go
constant depositsQty (line 21) | depositsQty = 10
constant tonWithdrawAmount (line 22) | tonWithdrawAmount = 550_000_000
constant jettonWithdrawAmount (line 23) | jettonWithdrawAmount = 550_000_000
constant tonMinCutoff (line 24) | tonMinCutoff = 10_000_000_000
function main (line 27) | func main() {
FILE: cmd/testutil/utils.go
type PayerProcessor (line 19) | type PayerProcessor struct
method Start (line 183) | func (p *PayerProcessor) Start() {
method balanceMonitor (line 191) | func (p *PayerProcessor) balanceMonitor() {
method updateHotWalletBalances (line 238) | func (p *PayerProcessor) updateHotWalletBalances(ctx context.Context) ...
method getDepositBalance (line 274) | func (p *PayerProcessor) getDepositBalance(ctx context.Context, d Depo...
method startPayments (line 289) | func (p *PayerProcessor) startPayments(side PaymentSide) {
method checkBalances (line 315) | func (p *PayerProcessor) checkBalances(side PaymentSide) bool {
method withdrawToDeposits (line 336) | func (p *PayerProcessor) withdrawToDeposits(fromSide PaymentSide) ([]i...
method waitWithdrawals (line 388) | func (p *PayerProcessor) waitWithdrawals(fromSide PaymentSide, ids []i...
method validateWithdrawals (line 423) | func (p *PayerProcessor) validateWithdrawals(ctx context.Context, from...
method loadTXs (line 496) | func (p *PayerProcessor) loadTXs(ctx context.Context, lastTxID TxID, a...
constant TonLossForJettonExternalWithdrawal (line 32) | TonLossForJettonExternalWithdrawal int64 = 58_376_225
constant TonLossForJettonInternalWithdrawal (line 33) | TonLossForJettonInternalWithdrawal int64 = 55_306_226
constant TonLossForTonExternalWithdrawal (line 34) | TonLossForTonExternalWithdrawal int64 = 10_232_080
constant TonLossForTonInternalWithdrawal (line 35) | TonLossForTonInternalWithdrawal int64 = 8_034_998
type PaymentSide (line 38) | type PaymentSide
constant SideA (line 41) | SideA PaymentSide = "A"
constant SideB (line 42) | SideB PaymentSide = "B"
type TxID (line 45) | type TxID struct
type Deposit (line 50) | type Deposit struct
type hotBalances (line 56) | type hotBalances struct
method ReadBalance (line 69) | func (h *hotBalances) ReadBalance(walletSide PaymentSide, currency str...
method WriteBalance (line 90) | func (h *hotBalances) WriteBalance(walletSide PaymentSide, currency st...
function newHotBalances (line 62) | func newHotBalances() *hotBalances {
function NewPayerProcessor (line 104) | func NewPayerProcessor(
function convertDeposits (line 157) | func convertDeposits(bcClient *blockchain.Connection, deposits map[strin...
type withdrawal (line 330) | type withdrawal struct
function checkDoubleSpending (line 485) | func checkDoubleSpending(newUUIDs []uuid.UUID, knownUUIDs map[uuid.UUID]...
function parseTX (line 531) | func parseTX(tx *tlb.Transaction) ([]withdrawal, []uuid.UUID, error) {
function compareWithdrawals (line 579) | func compareWithdrawals(all, target []withdrawal) []withdrawal {
function getLastTxID (line 596) | func getLastTxID(ctx context.Context, bcClient *blockchain.Connection, a...
function predictLoss (line 611) | func predictLoss(currency string) float64 {
FILE: cmd/testwebhook/main.go
function main (line 11) | func main() {
function getNotification (line 20) | func getNotification(resp http.ResponseWriter, req *http.Request) {
function checkToken (line 35) | func checkToken(req *http.Request, token string) {
FILE: config/config.go
constant MaxJettonForwardTonAmount (line 16) | MaxJettonForwardTonAmount = 20_000_000
constant JettonProxyContractCode (line 36) | JettonProxyContractCode = "B5EE9C72410102010037000114FF00F4A413F4BCF2C80...
constant MaxCommentLength (line 38) | MaxCommentLength = 1000
type Jetton (line 68) | type Jetton struct
type Cutoffs (line 75) | type Cutoffs struct
function GetConfig (line 82) | func GetConfig() {
function parseJettonString (line 115) | func parseJettonString(s string) map[string]Jetton {
function parseTonString (line 158) | func parseTonString(s string) Cutoffs {
FILE: core/block_scanner.go
type BlockScanner (line 23) | type BlockScanner struct
method Start (line 87) | func (s *BlockScanner) Start() {
method Stop (line 108) | func (s *BlockScanner) Stop() {
method processBlock (line 112) | func (s *BlockScanner) processBlock(ctx context.Context, block ShardBl...
method pushNotifications (line 134) | func (s *BlockScanner) pushNotifications(e BlockEvents) error {
method pushNotification (line 157) | func (s *BlockScanner) pushNotification(
method filterTXs (line 198) | func (s *BlockScanner) filterTXs(
method processTXs (line 247) | func (s *BlockScanner) processTXs(
method processTonHotWalletTXs (line 281) | func (s *BlockScanner) processTonHotWalletTXs(txs transactions) (Event...
method processTonDepositWalletTXs (line 313) | func (s *BlockScanner) processTonDepositWalletTXs(txs transactions) (E...
method processJettonDepositWalletTXs (line 364) | func (s *BlockScanner) processJettonDepositWalletTXs(
method calculateJettonAmounts (line 413) | func (s *BlockScanner) calculateJettonAmounts(
method failedWithdrawals (line 590) | func (s *BlockScanner) failedWithdrawals(inMap map[Address]struct{}, o...
method processTonHotWalletExternalInMsg (line 630) | func (s *BlockScanner) processTonHotWalletExternalInMsg(tx *tlb.Transa...
method processTonHotWalletProxyMsg (line 719) | func (s *BlockScanner) processTonHotWalletProxyMsg(msg *tlb.InternalMe...
method processTonHotWalletInternalInMsg (line 753) | func (s *BlockScanner) processTonHotWalletInternalInMsg(tx *tlb.Transa...
method processTonDepositWalletExternalInMsg (line 820) | func (s *BlockScanner) processTonDepositWalletExternalInMsg(tx *tlb.Tr...
method processTonDepositWalletInternalInMsg (line 866) | func (s *BlockScanner) processTonDepositWalletInternalInMsg(tx *tlb.Tr...
method processJettonDepositOutMsgs (line 906) | func (s *BlockScanner) processJettonDepositOutMsgs(tx *tlb.Transaction...
method processJettonDepositInMsg (line 993) | func (s *BlockScanner) processJettonDepositInMsg(tx *tlb.Transaction) ...
type transactions (line 32) | type transactions struct
type jettonTransferNotificationMsg (line 38) | type jettonTransferNotificationMsg struct
type JettonTransferMsg (line 44) | type JettonTransferMsg struct
type HighLoadWalletExtMsgInfo (line 50) | type HighLoadWalletExtMsgInfo struct
type incomeNotification (line 56) | type incomeNotification struct
function NewBlockScanner (line 66) | func NewBlockScanner(
function checkTxForSuccess (line 228) | func checkTxForSuccess(tx *tlb.Transaction) (bool, error) {
function convertUnknownJettonTxs (line 442) | func convertUnknownJettonTxs(txs []*tlb.Transaction, addr Address, amoun...
function decodeJettonTransferNotification (line 466) | func decodeJettonTransferNotification(msg *tlb.InternalMessage) (jettonT...
function DecodeJettonTransfer (line 492) | func DecodeJettonTransfer(msg *tlb.InternalMessage) (JettonTransferMsg, ...
function decodeJettonExcesses (line 521) | func decodeJettonExcesses(msg *tlb.InternalMessage) (int64, error) {
function parseExternalMessage (line 540) | func parseExternalMessage(msg *tlb.ExternalMessage) (
function getHighLoadWalletExtMsgInfo (line 605) | func getHighLoadWalletExtMsgInfo(extMsg *tlb.ExternalMessage) (HighLoadW...
FILE: core/models.go
constant TonSymbol (line 20) | TonSymbol = "TON"
constant DefaultWorkchain (line 21) | DefaultWorkchain = 0
constant SideHotWallet (line 27) | SideHotWallet IncomeSide = "hot_wallet"
constant SideDeposit (line 28) | SideDeposit IncomeSide = "deposit"
constant ServiceWithdrawalEvent (line 34) | ServiceWithdrawalEvent EventName = "service withdrawal"
constant InternalWithdrawalEvent (line 35) | InternalWithdrawalEvent EventName = "internal withdrawal"
constant ExternalWithdrawalEvent (line 36) | ExternalWithdrawalEvent EventName = "external withdrawal"
constant InitEvent (line 37) | InitEvent EventName = "initialization"
type WalletType (line 40) | type WalletType
constant TonHotWallet (line 43) | TonHotWallet WalletType = "ton_hot"
constant JettonHotWallet (line 44) | JettonHotWallet WalletType = "jetton_hot"
constant TonDepositWallet (line 45) | TonDepositWallet WalletType = "ton_deposit"
constant JettonDepositWallet (line 46) | JettonDepositWallet WalletType = "jetton_deposit"
constant JettonOwner (line 47) | JettonOwner WalletType = "owner"
type WithdrawalStatus (line 50) | type WithdrawalStatus
constant PendingStatus (line 53) | PendingStatus WithdrawalStatus = "pending"
constant ProcessingStatus (line 54) | ProcessingStatus WithdrawalStatus = "processing"
constant ProcessedStatus (line 55) | ProcessedStatus WithdrawalStatus = "processed"
constant FailedStatus (line 56) | FailedStatus WithdrawalStatus = "failed"
type Address (line 64) | type Address
method Scan (line 67) | func (a *Address) Scan(src interface{}) error {
method Value (line 80) | func (a Address) Value() (driver.Value, error) {
method ToTonutilsAddressStd (line 85) | func (a Address) ToTonutilsAddressStd(flags byte) *address.Address {
method ToUserFormat (line 90) | func (a Address) ToUserFormat() string {
method ToBytes (line 97) | func (a Address) ToBytes() []byte {
function TonutilsAddressToUserFormat (line 101) | func TonutilsAddressToUserFormat(addr *address.Address) string {
function AddressFromBytes (line 107) | func AddressFromBytes(data []byte) (Address, error) {
function AddressFromTonutilsAddress (line 116) | func AddressFromTonutilsAddress(addr *address.Address) (Address, error) {
function AddressMustFromTonutilsAddress (line 126) | func AddressMustFromTonutilsAddress(addr *address.Address) Address {
type AddressInfo (line 134) | type AddressInfo struct
type JettonWallet (line 140) | type JettonWallet struct
type OwnerWallet (line 145) | type OwnerWallet struct
type WalletData (line 150) | type WalletData struct
type WithdrawalRequest (line 158) | type WithdrawalRequest struct
type WithdrawalData (line 170) | type WithdrawalData struct
type ServiceWithdrawalRequest (line 177) | type ServiceWithdrawalRequest struct
type ServiceWithdrawalTask (line 182) | type ServiceWithdrawalTask struct
type ExternalWithdrawalTask (line 189) | type ExternalWithdrawalTask struct
type InternalWithdrawal (line 199) | type InternalWithdrawal struct
type SendingConfirmation (line 208) | type SendingConfirmation struct
type ExternalWithdrawal (line 214) | type ExternalWithdrawal struct
type JettonWithdrawalConfirmation (line 225) | type JettonWithdrawalConfirmation struct
type InternalIncome (line 229) | type InternalIncome struct
type ExternalIncome (line 240) | type ExternalIncome struct
type Events (line 251) | type Events struct
method Append (line 260) | func (e *Events) Append(ae Events) {
type BlockEvents (line 269) | type BlockEvents struct
type InternalWithdrawalTask (line 274) | type InternalWithdrawalTask struct
type TotalIncome (line 281) | type TotalIncome struct
type TotalWithdrawalsAmount (line 287) | type TotalWithdrawalsAmount struct
function NewCoins (line 294) | func NewCoins(int *big.Int) Coins {
function ZeroCoins (line 298) | func ZeroCoins() Coins {
type ShardBlockHeader (line 304) | type ShardBlockHeader struct
type storage (line 313) | type storage interface
type blockchain (line 338) | type blockchain interface
type blocksTracker (line 350) | type blocksTracker interface
type Notificator (line 355) | type Notificator interface
FILE: core/proxy.go
type JettonProxy (line 16) | type JettonProxy struct
method Address (line 63) | func (p *JettonProxy) Address() *address.Address {
method StateInit (line 68) | func (p *JettonProxy) StateInit() *tlb.StateInit {
method BuildMessage (line 73) | func (p *JettonProxy) BuildMessage(destination *address.Address, body ...
function NewJettonProxy (line 23) | func NewJettonProxy(subwalletId uint32, owner *address.Address) (*Jetton...
function buildJettonProxyStateInit (line 42) | func buildJettonProxyStateInit(subwalletId uint32, owner *address.Addres...
FILE: core/wallets.go
type Wallets (line 22) | type Wallets struct
function InitWallets (line 32) | func InitWallets(
function initTonHotWallet (line 79) | func initTonHotWallet(
function initJettonHotWallet (line 143) | func initJettonHotWallet(
function buildComment (line 199) | func buildComment(comment string) *cell.Cell {
function LoadComment (line 207) | func LoadComment(cell *cell.Cell) string {
function WithdrawTONs (line 226) | func WithdrawTONs(ctx context.Context, from, to *wallet.Wallet, comment ...
function WithdrawJettons (line 246) | func WithdrawJettons(
function MakeJettonTransferMessage (line 278) | func MakeJettonTransferMessage(
function decodeBinaryComment (line 318) | func decodeBinaryComment(comment string) (*cell.Cell, error) {
function BuildTonWithdrawalMessage (line 344) | func BuildTonWithdrawalMessage(t ExternalWithdrawalTask) *wallet.Message {
function BuildJettonWithdrawalMessage (line 371) | func BuildJettonWithdrawalMessage(
function BuildJettonProxyWithdrawalMessage (line 399) | func BuildJettonProxyWithdrawalMessage(
function buildJettonProxyServiceTonWithdrawalMessage (line 434) | func buildJettonProxyServiceTonWithdrawalMessage(
function buildTonFillMessage (line 457) | func buildTonFillMessage(
FILE: core/withdrawal_processor.go
type WithdrawalsProcessor (line 20) | type WithdrawalsProcessor struct
method Start (line 64) | func (p *WithdrawalsProcessor) Start() {
method Stop (line 71) | func (p *WithdrawalsProcessor) Stop() {
method startWithdrawalsProcessor (line 75) | func (p *WithdrawalsProcessor) startWithdrawalsProcessor() {
method buildWithdrawalMessages (line 130) | func (p *WithdrawalsProcessor) buildWithdrawalMessages(ctx context.Con...
method getHotWalletBalances (line 222) | func (p *WithdrawalsProcessor) getHotWalletBalances(ctx context.Contex...
method buildJettonInternalWithdrawalMessage (line 257) | func (p *WithdrawalsProcessor) buildJettonInternalWithdrawalMessage(
method buildServiceWithdrawalMessage (line 292) | func (p *WithdrawalsProcessor) buildServiceWithdrawalMessage(
method buildServiceFilling (line 315) | func (p *WithdrawalsProcessor) buildServiceFilling(
method buildServiceTonWithdrawal (line 356) | func (p *WithdrawalsProcessor) buildServiceTonWithdrawal(
method buildServiceJettonWithdrawal (line 385) | func (p *WithdrawalsProcessor) buildServiceJettonWithdrawal(
method buildExternalWithdrawalMessage (line 444) | func (p *WithdrawalsProcessor) buildExternalWithdrawalMessage(wt Exter...
method startExpirationProcessor (line 452) | func (p *WithdrawalsProcessor) startExpirationProcessor() {
method startInternalTonWithdrawalsProcessor (line 471) | func (p *WithdrawalsProcessor) startInternalTonWithdrawalsProcessor() {
method withdrawTONsFromDeposit (line 509) | func (p *WithdrawalsProcessor) withdrawTONsFromDeposit(ctx context.Con...
method serviceWithdrawJettons (line 545) | func (p *WithdrawalsProcessor) serviceWithdrawJettons(ctx context.Cont...
method waitSync (line 584) | func (p *WithdrawalsProcessor) waitSync() {
method makeColdWalletWithdrawals (line 604) | func (p *WithdrawalsProcessor) makeColdWalletWithdrawals(ctx context.C...
type internalWithdrawal (line 29) | type internalWithdrawal struct
type serviceWithdrawal (line 34) | type serviceWithdrawal struct
type withdrawals (line 40) | type withdrawals struct
function NewWithdrawalsProcessor (line 47) | func NewWithdrawalsProcessor(
function decreaseBalances (line 240) | func decreaseBalances(balances map[string]*big.Int, currency string, amo...
FILE: db/db.go
type Connection (line 24) | type Connection struct
method GetWalletType (line 58) | func (c *Connection) GetWalletType(address core.Address) (core.WalletT...
method GetUserID (line 63) | func (c *Connection) GetUserID(address core.Address) (string, bool) {
method GetOwner (line 69) | func (c *Connection) GetOwner(address core.Address) *core.Address {
method GetWalletTypeByTonutilsAddress (line 77) | func (c *Connection) GetWalletTypeByTonutilsAddress(address *address.A...
method GetLastSubwalletID (line 88) | func (c *Connection) GetLastSubwalletID(ctx context.Context) (uint32, ...
method SaveTonWallet (line 97) | func (c *Connection) SaveTonWallet(ctx context.Context, walletData cor...
method GetJettonWallet (line 118) | func (c *Connection) GetJettonWallet(ctx context.Context, address core...
method SaveJettonWallet (line 136) | func (c *Connection) SaveJettonWallet(
method GetTonWalletsAddresses (line 201) | func (c *Connection) GetTonWalletsAddresses(
method GetJettonOwnersAddresses (line 236) | func (c *Connection) GetJettonOwnersAddresses(
method LoadAddressBook (line 272) | func (c *Connection) LoadAddressBook(ctx context.Context) error {
method saveInternalIncome (line 363) | func (c *Connection) saveInternalIncome(ctx context.Context, tx pgx.Tx...
method SaveWithdrawalRequest (line 403) | func (c *Connection) SaveWithdrawalRequest(ctx context.Context, w core...
method SaveServiceWithdrawalRequest (line 435) | func (c *Connection) SaveServiceWithdrawalRequest(ctx context.Context,...
method UpdateServiceWithdrawalRequest (line 454) | func (c *Connection) UpdateServiceWithdrawalRequest(
method IsWithdrawalRequestUnique (line 474) | func (c *Connection) IsWithdrawalRequestUnique(ctx context.Context, w ...
method GetExternalWithdrawalTasks (line 493) | func (c *Connection) GetExternalWithdrawalTasks(ctx context.Context, l...
method GetServiceHotWithdrawalTasks (line 527) | func (c *Connection) GetServiceHotWithdrawalTasks(ctx context.Context,...
method GetServiceDepositWithdrawalTasks (line 559) | func (c *Connection) GetServiceDepositWithdrawalTasks(ctx context.Cont...
method SaveInternalWithdrawalTask (line 659) | func (c *Connection) SaveInternalWithdrawalTask(
method SaveParsedBlockData (line 681) | func (c *Connection) SaveParsedBlockData(ctx context.Context, events c...
method GetTonInternalWithdrawalTasks (line 731) | func (c *Connection) GetTonInternalWithdrawalTasks(ctx context.Context...
method GetJettonInternalWithdrawalTasks (line 775) | func (c *Connection) GetJettonInternalWithdrawalTasks(
method CreateExternalWithdrawals (line 931) | func (c *Connection) CreateExternalWithdrawals(
method GetTonHotWalletAddress (line 969) | func (c *Connection) GetTonHotWalletAddress(ctx context.Context) (core...
method GetLastSavedBlockID (line 982) | func (c *Connection) GetLastSavedBlockID(ctx context.Context) (*ton.Bl...
method SetExpired (line 1010) | func (c *Connection) SetExpired(ctx context.Context) error {
method IsActualBlockData (line 1069) | func (c *Connection) IsActualBlockData(ctx context.Context) (bool, int...
method IsInProgressInternalWithdrawalRequest (line 1087) | func (c *Connection) IsInProgressInternalWithdrawalRequest(
method GetExternalWithdrawalStatus (line 1118) | func (c *Connection) GetExternalWithdrawalStatus(ctx context.Context, ...
method GetIncome (line 1170) | func (c *Connection) GetIncome(
method GetIncomeHistory (line 1227) | func (c *Connection) GetIncomeHistory(
method GetIncomeByTx (line 1297) | func (c *Connection) GetIncomeByTx(
method GetTotalWithdrawalAmounts (line 1346) | func (c *Connection) GetTotalWithdrawalAmounts(ctx context.Context, cu...
type addressBook (line 29) | type addressBook struct
method get (line 34) | func (ab *addressBook) get(address core.Address) (core.AddressInfo, bo...
method put (line 41) | func (ab *addressBook) put(address core.Address, t core.AddressInfo) {
function NewConnection (line 47) | func NewConnection(URI string) (*Connection, error) {
function stripInvalidUTF8 (line 324) | func stripInvalidUTF8(s string) string {
function saveExternalIncome (line 337) | func saveExternalIncome(ctx context.Context, tx pgx.Tx, inc core.Externa...
function saveBlock (line 591) | func saveBlock(ctx context.Context, tx pgx.Tx, block core.ShardBlockHead...
function updateInternalWithdrawal (line 610) | func updateInternalWithdrawal(ctx context.Context, tx pgx.Tx, w core.Int...
function applyJettonWithdrawalConfirmation (line 827) | func applyJettonWithdrawalConfirmation(
function updateExternalWithdrawal (line 841) | func updateExternalWithdrawal(ctx context.Context, tx pgx.Tx, w core.Ext...
function applySendingConfirmations (line 907) | func applySendingConfirmations(ctx context.Context, tx pgx.Tx, w core.Se...
FILE: db/db_test.go
function execMultiStatement (line 15) | func execMultiStatement(c *Connection, ctx context.Context, query string...
function migrateUp (line 35) | func migrateUp(c *Connection, t *testing.T, source string) error {
function migrateDown (line 56) | func migrateDown(c *Connection, t *testing.T) {
function init (line 67) | func init() {
function connect (line 74) | func connect(t *testing.T) *Connection {
function Test_NewConnection (line 82) | func Test_NewConnection(t *testing.T) {
function Test_GetTonInternalWithdrawalTasks (line 86) | func Test_GetTonInternalWithdrawalTasks(t *testing.T) {
function Test_GetJettonInternalWithdrawalTasks (line 111) | func Test_GetJettonInternalWithdrawalTasks(t *testing.T) {
function Test_GetJettonInternalWithdrawalTasksForbidden (line 139) | func Test_GetJettonInternalWithdrawalTasksForbidden(t *testing.T) {
function Test_SetExpired (line 167) | func Test_SetExpired(t *testing.T) {
function TestStripInvalidUTF8 (line 260) | func TestStripInvalidUTF8(t *testing.T) {
FILE: deploy/db/01_init.up.sql
type payments (line 5) | CREATE TABLE IF NOT EXISTS payments.ton_wallets
type ton_wallets_address_index (line 14) | CREATE INDEX IF NOT EXISTS ton_wallets_address_index
type ton_wallets_type_index (line 17) | CREATE INDEX IF NOT EXISTS ton_wallets_type_index
type ton_wallets_user_id_index (line 20) | CREATE INDEX IF NOT EXISTS ton_wallets_user_id_index
type payments (line 23) | CREATE TABLE IF NOT EXISTS payments.jetton_wallets
type jetton_wallets_subwallet_id_index (line 33) | CREATE INDEX IF NOT EXISTS jetton_wallets_subwallet_id_index
type payments (line 36) | CREATE TABLE IF NOT EXISTS payments.internal_incomes
type internal_incomes_deposit_address_index (line 46) | CREATE INDEX IF NOT EXISTS internal_incomes_deposit_address_index
type payments (line 49) | CREATE TABLE IF NOT EXISTS payments.external_withdrawals
type external_withdrawals_expired_at_index (line 64) | CREATE INDEX IF NOT EXISTS external_withdrawals_expired_at_index
type external_withdrawals_msg_uuid_index (line 67) | CREATE INDEX IF NOT EXISTS external_withdrawals_msg_uuid_index
type external_withdrawals_address_index (line 70) | CREATE INDEX IF NOT EXISTS external_withdrawals_address_index
type external_withdrawals_query_id_index (line 73) | CREATE INDEX IF NOT EXISTS external_withdrawals_query_id_index
type payments (line 76) | CREATE TABLE IF NOT EXISTS payments.withdrawal_requests
type withdrawal_requests_user_id_index (line 95) | CREATE INDEX IF NOT EXISTS withdrawal_requests_user_id_index
type withdrawal_requests_user_query_id_index (line 98) | CREATE INDEX IF NOT EXISTS withdrawal_requests_user_query_id_index
type withdrawal_requests_dest_address_index (line 101) | CREATE INDEX IF NOT EXISTS withdrawal_requests_dest_address_index
type payments (line 104) | CREATE TABLE IF NOT EXISTS payments.external_incomes
type external_incomes_deposit_address_index (line 116) | CREATE INDEX IF NOT EXISTS external_incomes_deposit_address_index
type payments (line 119) | CREATE TABLE IF NOT EXISTS payments.block_data
type block_data_seqno_index (line 130) | CREATE INDEX IF NOT EXISTS block_data_seqno_index
type payments (line 133) | CREATE TABLE IF NOT EXISTS payments.internal_withdrawals
type internal_withdrawals_from_address_index (line 148) | CREATE INDEX IF NOT EXISTS internal_withdrawals_from_address_index
type internal_withdrawals_since_lt_index (line 151) | CREATE INDEX IF NOT EXISTS internal_withdrawals_since_lt_index
type internal_withdrawals_expired_at_index (line 154) | CREATE INDEX IF NOT EXISTS internal_withdrawals_expired_at_index
type payments (line 157) | CREATE TABLE IF NOT EXISTS payments.service_withdrawal_requests
FILE: metrics/metrics.go
type counter (line 10) | type counter struct
method Inc (line 15) | func (c *counter) Inc() {
method Add (line 19) | func (c *counter) Add(n uint64) {
method Print (line 23) | func (c *counter) Print(w io.Writer) error {
type printer (line 34) | type printer interface
FILE: queue/queue.go
type AmqpClient (line 9) | type AmqpClient struct
method declareExchange (line 38) | func (c *AmqpClient) declareExchange(exchangeName string) error {
method Publish (line 56) | func (c *AmqpClient) Publish(payload any) error {
function NewAmqpClient (line 18) | func NewAmqpClient(uri string, enabled bool, queueName string) (*AmqpCli...
FILE: webhook/webhook.go
type Client (line 12) | type Client struct
method Publish (line 33) | func (s *Client) Publish(payload any) error {
function NewWebhookClient (line 19) | func NewWebhookClient(uri string, token string) (*Client, error) {
function send (line 57) | func send(client *http.Client, request *http.Request) error {
Condensed preview — 62 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (662K chars).
[
{
"path": ".github/workflows/go.yml",
"chars": 1941,
"preview": "name: Go\n\non:\n push:\n branches: [ \"master\" ]\n workflow_dispatch: {}\n\njobs:\n build:\n runs-on: ubuntu-24.04\n e"
},
{
"path": "Dockerfile",
"chars": 1337,
"preview": "FROM docker.io/library/golang:1.24-bookworm AS builder\nWORKDIR /build-dir\nCOPY go.mod .\nCOPY go.sum .\nRUN go mod downloa"
},
{
"path": "LICENSE",
"chars": 35149,
"preview": " GNU GENERAL PUBLIC LICENSE\n Version 3, 29 June 2007\n\n Copyright (C) 2007 Free "
},
{
"path": "Makefile",
"chars": 310,
"preview": "VERSION := latest\nGIT_TAG := $(shell git describe --tags --always)\n\nbuild:\n\t@echo \"Building tag: $(GIT_TAG)\"\n\tdocker bui"
},
{
"path": "README.md",
"chars": 20108,
"preview": "# TON payment processor\n[![Based on TON][ton-svg]][ton]\n[![Go](https://github.com/gobicycle/bicycle/actions/workflows/go"
},
{
"path": "api/example.http",
"chars": 2376,
"preview": "###\nPOST {{url}}/v1/address/new\nAuthorization: Bearer {{token}}\nContent-Type: application/json\n\n{\"user_id\": \"TestUser\", "
},
{
"path": "api/handlers.go",
"chars": 27973,
"preview": "package api\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"encoding/hex\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"github.com/gobicycle/bicyc"
},
{
"path": "api/http-client.env.json",
"chars": 55,
"preview": "{\n \"local\": {\n \"url\": \"http://localhost:8081\"\n }\n}"
},
{
"path": "api/middleware.go",
"chars": 2125,
"preview": "package api\n\nimport (\n\t\"crypto/subtle\"\n\t\"encoding/json\"\n\t\"github.com/gobicycle/bicycle/config\"\n\tlog \"github.com/sirupsen"
},
{
"path": "audit/log.go",
"chars": 1059,
"preview": "package audit\n\nimport (\n\t\"fmt\"\n\t\"github.com/gobicycle/bicycle/metrics\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"time\"\n)\n\ntype"
},
{
"path": "blockchain/blockchain.go",
"chars": 19640,
"preview": "package blockchain\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"github.com/gobicycle/bicycle/config\"\n\t\"github.com/gobicycle/b"
},
{
"path": "blockchain/blockchain_test.go",
"chars": 10937,
"preview": "package blockchain\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"github.com/gobicycle/bicycle/core\"\n\t\"github.com/xssnick/tonutils-go/a"
},
{
"path": "blockchain/limited_client.go",
"chars": 1234,
"preview": "package blockchain\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/xssnick/tonutils-go/tl\"\n\t\"github.com/xssnick/tonutils-go/to"
},
{
"path": "blockchain/shard_tracker.go",
"chars": 6863,
"preview": "package blockchain\n\nimport (\n\t\"context\"\n\t\"github.com/gobicycle/bicycle/core\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"github."
},
{
"path": "cmd/processor/main.go",
"chars": 3291,
"preview": "package main\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"github.com/gobicycle/bicycle/api\"\n\t\"github.com/gobicycle/bicycle/bl"
},
{
"path": "cmd/testutil/http.go",
"chars": 5970,
"preview": "package main\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"github.com/gobicycle/bicycle/api\"\n\t\"github.com/gobicycle/bicyc"
},
{
"path": "cmd/testutil/main.go",
"chars": 1866,
"preview": "package main\n\nimport (\n\t\"context\"\n\t\"github.com/gobicycle/bicycle/blockchain\"\n\t\"github.com/gobicycle/bicycle/config\"\n\t\"gi"
},
{
"path": "cmd/testutil/metrics.go",
"chars": 1443,
"preview": "package main\n\nimport (\n\t\"github.com/prometheus/client_golang/prometheus\"\n\t\"github.com/prometheus/client_golang/prometheu"
},
{
"path": "cmd/testutil/utils.go",
"chars": 15868,
"preview": "package main\n\nimport (\n\t\"bytes\"\n\t\"context\"\n\t\"fmt\"\n\t\"github.com/gobicycle/bicycle/blockchain\"\n\t\"github.com/gobicycle/bicy"
},
{
"path": "cmd/testwebhook/main.go",
"chars": 1113,
"preview": "package main\n\nimport (\n\t\"crypto/subtle\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"strings\"\n)\n\nfunc main() {\n\thttp.HandleFunc(\"/webhook\""
},
{
"path": "config/config.go",
"chars": 6517,
"preview": "package config\n\nimport (\n\t\"log\"\n\t\"math/big\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/caarlos0/env/v6\"\n\t\"github.com/shopspring/de"
},
{
"path": "core/block_scanner.go",
"chars": 31501,
"preview": "package core\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"math/big\"\n\t\"sync\"\n\t\"time\"\n\n\t\"github.com/gobicycle/bicycle/audit\"\n\t\"github.com"
},
{
"path": "core/models.go",
"chars": 9853,
"preview": "package core\n\nimport (\n\t\"context\"\n\t\"database/sql/driver\"\n\t\"errors\"\n\t\"fmt\"\n\t\"github.com/gobicycle/bicycle/config\"\n\t\"githu"
},
{
"path": "core/proxy.go",
"chars": 2407,
"preview": "package core\n\nimport (\n\t\"encoding/hex\"\n\t\"fmt\"\n\t\"github.com/gobicycle/bicycle/config\"\n\tlog \"github.com/sirupsen/logrus\"\n\t"
},
{
"path": "core/wallets.go",
"chars": 12300,
"preview": "package core\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"github.com/gobicycle/bicycle/audit\"\n\t\"github.com/gobicycle/bicycle/"
},
{
"path": "core/withdrawal_processor.go",
"chars": 20089,
"preview": "package core\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"math/big\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"time\"\n\n\t\"github.com/gobicycle/bicycle/audi"
},
{
"path": "db/db.go",
"chars": 35487,
"preview": "package db\n\nimport (\n\t\"context\"\n\t\"errors\"\n\t\"fmt\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\t\"unicode/utf8\"\n\n\t\"github.com/gobicycle/bicy"
},
{
"path": "db/db_test.go",
"chars": 6779,
"preview": "package db\n\nimport (\n\t\"context\"\n\t\"encoding/hex\"\n\t\"github.com/gobicycle/bicycle/core\"\n\t\"os\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\""
},
{
"path": "db/tests/get-jetton-internal-withdrawal-tasks/01_data.up.sql",
"chars": 4940,
"preview": "BEGIN;\n\nINSERT INTO payments.external_incomes (\n lt,\n utime,\n deposit_address,\n payer_address,\n amount,\n "
},
{
"path": "db/tests/get-ton-internal-withdrawal-tasks/01_data.up.sql",
"chars": 3782,
"preview": "BEGIN;\n\nINSERT INTO payments.external_incomes (\n lt,\n utime,\n deposit_address,\n payer_address,\n amount,\n "
},
{
"path": "db/tests/set-expired/01_data.up.sql",
"chars": 6932,
"preview": "BEGIN;\n\nINSERT INTO payments.external_withdrawals (\n msg_uuid,\n query_id,\n created_at,\n expired_at,\n proc"
},
{
"path": "deploy/db/01_init.down.sql",
"chars": 490,
"preview": "BEGIN;\n\nDROP TABLE IF EXISTS payments.ton_wallets;\nDROP TABLE IF EXISTS payments.jetton_wallets;\nDROP TABLE IF EXISTS pa"
},
{
"path": "deploy/db/01_init.up.sql",
"chars": 5780,
"preview": "BEGIN;\n\nCREATE SCHEMA IF NOT EXISTS payments;\n\nCREATE TABLE IF NOT EXISTS payments.ton_wallets\n(\n subwallet_id bi"
},
{
"path": "deploy/db/02_create_readonly_user.sh",
"chars": 567,
"preview": "#!/bin/bash\n\nif [ -z \"$POSTGRES_READONLY_PASSWORD\" ]; then\n echo \"Environment variable POSTGRES_READONLY_PASSWORD is no"
},
{
"path": "deploy/grafana/main/dashboards/Payments.json",
"chars": 35209,
"preview": "{\n \"__inputs\": [\n {\n \"name\": \"DS_POSTGRES\",\n \"label\": \"Postgres\",\n \"description\": \"\",\n \"type\": \""
},
{
"path": "deploy/grafana/main/provisioning/dashboards/payments.yml",
"chars": 171,
"preview": "apiVersion: 1\n\nproviders:\n - name: dashboards\n type: file\n updateIntervalSeconds: 30\n options:\n path: /et"
},
{
"path": "deploy/grafana/main/provisioning/datasources/data_sources.yml",
"chars": 696,
"preview": "apiVersion: 1\n\ndatasources:\n - name: Prometheus\n uid: DS_PROMETHEUS\n type: prometheus\n access: proxy\n url: "
},
{
"path": "deploy/grafana/test/dashboards/Processor A.json",
"chars": 35260,
"preview": "{\n \"__inputs\": [\n {\n \"name\": \"DS_POSTGRES_A\",\n \"label\": \"Postgres A\",\n \"description\": \"\",\n \"type"
},
{
"path": "deploy/grafana/test/dashboards/Processor B.json",
"chars": 35260,
"preview": "{\n \"__inputs\": [\n {\n \"name\": \"DS_POSTGRES_B\",\n \"label\": \"Postgres B\",\n \"description\": \"\",\n \"type"
},
{
"path": "deploy/grafana/test/dashboards/Test util.json",
"chars": 14349,
"preview": "{\n \"__inputs\": [\n {\n \"name\": \"DS_PROMETHEUS\",\n \"label\": \"Prometheus\",\n \"description\": \"\",\n \"type"
},
{
"path": "deploy/grafana/test/provisioning/dashboards/payments.yml",
"chars": 171,
"preview": "apiVersion: 1\n\nproviders:\n - name: dashboards\n type: file\n updateIntervalSeconds: 30\n options:\n path: /et"
},
{
"path": "deploy/grafana/test/provisioning/datasources/data_sources.yml",
"chars": 1218,
"preview": "apiVersion: 1\n\ndatasources:\n - name: Prometheus\n uid: DS_PROMETHEUS\n type: prometheus\n access: proxy\n url: "
},
{
"path": "deploy/manual_migrations/0.1.x-0.2.0.sql",
"chars": 280,
"preview": "BEGIN;\n\nALTER TABLE payments.external_incomes\nADD COLUMN IF NOT EXISTS payer_workchain integer;\n\nUPDATE payments.externa"
},
{
"path": "deploy/manual_migrations/0.4.x-0.5.0.sql",
"chars": 288,
"preview": "BEGIN;\n\nALTER TABLE payments.external_withdrawals\n ADD COLUMN IF NOT EXISTS tx_hash bytea;\n\nALTER TABLE payments.with"
},
{
"path": "deploy/prometheus/main/prometheus.yml",
"chars": 133,
"preview": "scrape_configs:\n\n - job_name: audit-metrics\n scrape_interval: 5s\n static_configs:\n - targets: ['payment-proc"
},
{
"path": "deploy/prometheus/test/prometheus.yml",
"chars": 125,
"preview": "scrape_configs:\n\n - job_name: test-utils\n scrape_interval: 5s\n static_configs:\n - targets: ['payment_test:91"
},
{
"path": "docker-compose.yml",
"chars": 2470,
"preview": "version: '3'\n\nservices:\n\n payment-postgres:\n image: postgres:14\n container_name: payment_processor_db\n volumes"
},
{
"path": "docs/api.apib",
"chars": 12362,
"preview": "FORMAT: 1A\n\n# Payment processor API\nThis API describes endpoints of payment processor.\n\n## New address [POST /v1/address"
},
{
"path": "docs/index.html",
"chars": 71159,
"preview": "<!DOCTYPE html><html><head><meta charset=\"utf-8\"><title>Payment processor API</title><link rel=\"stylesheet\" href=\"https:"
},
{
"path": "go.mod",
"chars": 1554,
"preview": "module github.com/gobicycle/bicycle\n\ngo 1.24.0\n\ntoolchain go1.24.11\n\nrequire (\n\tgithub.com/caarlos0/env/v6 v6.10.1\n\tgith"
},
{
"path": "go.sum",
"chars": 21392,
"preview": "github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=\ngithub.com/Masterminds/semver/v"
},
{
"path": "jettons.md",
"chars": 4240,
"preview": "# Jettons compatibility information\n\n**! Be careful with Jettons metadata, the information may not be up-to-date, see th"
},
{
"path": "manual_migrations.md",
"chars": 821,
"preview": "# Manual migrations between versions\n\n## v0.1.x -> v0.2.0\n1. Apply [DB migration](/deploy/manual_migrations/0.1.x-0.2.0."
},
{
"path": "manual_testing_plan.md",
"chars": 24243,
"preview": "## Manual testing plan for v0.5.0\nTemplate:\n-[x] Checked\n- TEST : test description\n- RESULT : expected result\n- COMM"
},
{
"path": "metrics/metrics.go",
"chars": 1121,
"preview": "package metrics\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"strconv\"\n\t\"sync/atomic\"\n)\n\ntype counter struct {\n\tname, description string\n\tcou"
},
{
"path": "queue/queue.go",
"chars": 1441,
"preview": "package queue\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\tamqp \"github.com/rabbitmq/amqp091-go\"\n)\n\ntype AmqpClient struct {\n\tconn"
},
{
"path": "release_notes.md",
"chars": 1550,
"preview": "# v0.5.0 release notes\n\n1. Add `tx_hash` field, `sort_order` parameter to `/v1/deposit/history{?user_id,currency,limit,o"
},
{
"path": "technical_notes.md",
"chars": 22667,
"preview": "# Technical notes\n\n- [Glossary](#Glossary)\n- [Limitations](#Limitations)\n- [API blueprint](/docs/api.apib)\n- [Wallets ge"
},
{
"path": "tests/docker-compose-tests.yml",
"chars": 3727,
"preview": "version: '3'\n\nservices:\n\n payment-postgres-a:\n image: postgres:14\n container_name: payment_processor_db_a\n vol"
},
{
"path": "threat_model.md",
"chars": 9716,
"preview": "### List of possible vulnerabilities\nIn the format:\n- P: problem\n- T: threat\n- S: possible solutions\n- D: decision\n\n####"
},
{
"path": "todo_list.md",
"chars": 3731,
"preview": "## TODO\n- [x] Withdraw TON method\n- [x] Withdraw jetton method\n- [x] Generate new address API method\n- [x] Get addresses"
},
{
"path": "webhook/webhook.go",
"chars": 1553,
"preview": "package webhook\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\tlog \"github.com/sirupsen/logrus\"\n\t\"net/http\"\n\t\"time\"\n)\n\ntype"
}
]
About this extraction
This page contains the full source code of the gobicycle/bicycle GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 62 files (600.6 KB), approximately 168.3k tokens, and a symbol index with 419 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.