Showing preview only (977K chars total). Download the full file or copy to clipboard to get everything.
Repository: Debian/debiman
Branch: main
Commit: ac8f5391b43b
Files: 76
Total size: 941.3 KB
Directory structure:
gitextract_7wrezzui/
├── .github/
│ └── workflows/
│ └── main.yml
├── LICENSE
├── PERFORMANCE.md
├── README.md
├── assets/
│ ├── about.tmpl
│ ├── contents.tmpl
│ ├── faq.tmpl
│ ├── footer.tmpl
│ ├── header.tmpl
│ ├── index.tmpl
│ ├── manpage.tmpl
│ ├── manpageerror.tmpl
│ ├── manpagefooterextra.tmpl
│ ├── notfound.tmpl
│ ├── opensearch.xml
│ ├── pkgindex.tmpl
│ ├── srcpkgindex.tmpl
│ └── style.css
├── bundle.go
├── cmd/
│ ├── debiman/
│ │ ├── download.go
│ │ ├── download_test.go
│ │ ├── getcontents.go
│ │ ├── getpackages.go
│ │ ├── globalview.go
│ │ ├── main.go
│ │ ├── main_test.go
│ │ ├── mtime.go
│ │ ├── mtime_linux.go
│ │ ├── prometheus.go
│ │ ├── render.go
│ │ ├── render_test.go
│ │ ├── renderaux.go
│ │ ├── rendercontents.go
│ │ ├── rendermanpage.go
│ │ ├── rendermanpage_test.go
│ │ ├── renderpkgindex.go
│ │ ├── reuse.go
│ │ ├── reuse_test.go
│ │ └── writeindex.go
│ ├── debiman-auxserver/
│ │ └── auxserver.go
│ ├── debiman-idx2rwmap/
│ │ └── rwmap.go
│ └── debiman-minisrv/
│ └── minisrv.go
├── debiman-auxserver.service
├── example/
│ ├── apache2.conf
│ └── nginx.conf
├── go.mod
├── go.sum
├── goembed.go
├── internal/
│ ├── auxserver/
│ │ ├── auxserver.go
│ │ └── auxserver_test.go
│ ├── bundled/
│ │ ├── GENERATED_bundled.go
│ │ └── inject.go
│ ├── commontmpl/
│ │ └── commontmpl.go
│ ├── convert/
│ │ ├── convert.go
│ │ ├── convert_test.go
│ │ └── mandoc.go
│ ├── manpage/
│ │ ├── meta.go
│ │ └── meta_test.go
│ ├── proto/
│ │ ├── generate.go
│ │ ├── index.pb.go
│ │ └── index.proto
│ ├── recode/
│ │ ├── recode.go
│ │ └── recode_test.go
│ ├── redirect/
│ │ ├── legacy.go
│ │ ├── redirect.go
│ │ └── redirect_test.go
│ ├── sitemap/
│ │ ├── sitemap.go
│ │ └── sitemap_test.go
│ ├── tag/
│ │ └── tag.go
│ └── write/
│ └── atomically.go
└── testdata/
├── i3lock.1
├── i3lock.html
├── refs.1
├── refs.html
└── tinymirror/
├── dists/
│ └── testing/
│ └── InRelease
└── pool/
└── main/
└── i/
└── i3-wm/
└── i3-wm_4.13-1_amd64.deb
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/workflows/main.yml
================================================
name: Push
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build:
name: CI
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
# Run on the latest minor release of Go 1.19:
go-version: ^1.19
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@v2
- name: Ensure all files were formatted as per gofmt
run: |
[ "$(gofmt -l $(find . -name '*.go') 2>&1)" = "" ] || { gofmt -l $(find . -name '*.go'); false; }
- name: Install dependency Debian packages
run: sudo apt-get install debian-archive-keyring
- name: Run tests
run: |
/usr/lib/apt/apt-helper download-file https://people.debian.org/~stapelberg/mandoc-static/mandoc mandoc SHA256:91d1f1a6120d0fcccd7590645dcb852ad83a7ad59523e6fd026a64b1c913a102
/usr/lib/apt/apt-helper download-file https://people.debian.org/~stapelberg/mandoc-static/mandocd mandocd SHA256:9c30e64b5721ae223e18449d4c82b30919cf11e423de160beff4724cefcdd02e
chmod +x mandoc mandocd
PATH=$PWD:$PATH go test ./...
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright {yyyy} {name of copyright owner}
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: PERFORMANCE.md
================================================
# debiman performance
In both cases, a Debian mirror was available via a Gigabit ethernet connection.
## Modern machine
Intel® Core™ i7-6700K (8 x 4.0 GHz), 32 GB DDR4-RAM, Intel SSDSC2BP48
| Debian unstable | all Debian suites
-----------------------|-----------------|-----------------------
contents parsing | <6s | <25s
package parsing | <2s | <20s
xref preparation | <3s | <15s
stat | <2s | <4s
**total incremental** | **<13s** | **<65s**
full man extraction | 5m | 20m
full man rendering | <4m | 12m
**total from scratch** | **<10m** | **34m**
## Dated machine
AMD Opteron™ 23xx (2 x 2.2 GHz), 2 GB RAM, TODO spinning disk
| Debian unstable | all Debian suites
-----------------------|-----------------|-----------------------
contents parsing | <70s | <167s
package parsing | <10s | <35s
xref preparation | (not measured) | <80s
stat | (not measured) | <60s
**total incremental** | **<140s** | **<10m** (TODO)
full man extraction | TODO | TODO
full man rendering | TODO | 2h
**total from scratch** | TODO | TODO
================================================
FILE: README.md
================================================
# debiman
[](https://github.com/Debian/debiman/actions/workflows/main.yml)
[](https://goreportcard.com/report/github.com/Debian/debiman)
<img src="https://debian.github.io/debiman/debiman-logo.svg" width="300" height="280" align="right" alt="debiman logo">
## Goals
debiman makes (Debian) manpages accessible in a web browser. Its goals are, in order:
1. **completeness**: all manpages in Debian should be available.
2. **visually appealing** and **convenient**: reading manpages should be fun, convenience features (e.g. permalinks, URL redirects, easy navigation) should be available
3. **speed**: manpages should be quick to load, new manpages should be quickly ingested, the program should run quickly for pleasant development
Currently, there is one known bug with regards to completeness ([#12](https://github.com/Debian/debiman/issues/12)).
With regards to speed, debiman can process all manpages of Debian unstable in **less than 10 minutes** on a modern machine. Incremental updates complete in **less than 15 seconds**. For more details, see [PERFORMANCE.md](https://github.com/Debian/debiman/blob/master/PERFORMANCE.md).
## Prerequisites
* mandoc
* a local or remote Debian mirror or an apt-cacher-ng running on localhost:3142
* a number of Go packages (which `go get` will automatically get for you, see below)
* pault.ag/go/debian
* pault.ag/go/archive
* github.com/golang/protobuf/proto
* golang.org/x/crypto/openpgp
* golang.org/x/net/html
* golang.org/x/sync/errgroup
* golang.org/x/text
## Architecture overview
debiman works in 4 stages:
1. All Debian packages of all architectures of the specified suites are discovered. The following optimizations are used to reduce the number of packages, and hence the input size/required bandwidth:
1. packages which do not own any files in /usr/share/man (as per the Contents-<arch> archive files) are skipped.
2. each package is downloaded only for 1 of its architectures, as manpages are architecture-independent.
2. Man pages and auxiliary files (e.g. content fragment files which are included by a number of manpages) are extracted from the identified Debian packages.
3. All man pages are rendered into an HTML representation using mandoc(1).
4. An index file for debiman-auxserver (which serves redirects) is written.
Each stage runs concurrently (e.g. Contents and Packages files are
inspected concurrently), but only one stage runs at a time,
e.g. extraction needs to complete before rendering can start.
## Development quick start
### Set up Go
Install the latest supported version of Go from https://go.dev/dl. If you prefer
to install Go from Debian, ensure you get the same version — if you use Debian
stable, you likely need to install from backports.
### Install debiman
To download, compile and install debiman to `~/go/bin`, run:
```
go install github.com/Debian/debiman/cmd/...@main
```
### Run debiman
To synchronize Debian testing to ~/man and render a handful of packages, run:
```
~/go/bin/debiman -serving_dir=~/man -only_render_pkgs=qelectrotech,i3-wm,cron
```
### Test the output
To serve manpages from ~/man on localhost:8089, run:
```
~/go/bin/debiman-minisrv -serving_dir=~/man
```
Note that for a production setup, you should not use debiman-minisrv. Instead,
refer to the web server example configuration files in example/.
### Recompile debiman
To update your debiman installation after making changes to the HTML
templates or code in your `debiman` git working directory, run:
```
go generate github.com/Debian/debiman/...
go install github.com/Debian/debiman/cmd/...
```
## Synchronizing
For https://manpages.debian.org, we run:
```
flock /srv/manpages.debian.org/debiman/exclusive.lock \
nice -n 5 \
ionice -n 7 \
debiman \
-sync_codenames=oldstable,oldstable-backports,stable,stable-backports \
-sync_suites=testing,unstable,experimental \
-serving_dir=/srv/manpages.debian.org/www \
-local_mirror=/srv/mirrors/debian
```
…resulting in the directories wheezy/, wheezy-backports/, jessie/, jessie-backports/, testing/, unstable/ and experimental/ (respectively).
Note that you will *NOT* need to change this command line when a new version of Debian is released.
When interrupted, you can just run debiman again with the same options. It will resume where it left off.
If for some reason you notice corruption or other mistakes in some manpages, just delete the directory in which they are placed, then re-run debiman to download and re-process these pages from scratch.
It is safe to run debiman while you are serving from `-serving_dir`. debiman will swap files atomically using [rename(2)](https://manpages.debian.org/rename(2)).
## Customization
You can copy the `assets/` directory, modify its contents and start
debiman with `-inject_assets` pointed to your directory. Any files whose
name does not end in .tmpl are treated as static files and will be
placed in -serving_dir (compressed and uncompressed).
There are a few requirements for the templates, so that debiman can
re-use rendered manpages (for symlinked manpages):
1. In `assets/manpage.tmpl` and `assets/manpageerror.tmpl`, the string `<a
class="toclink"` is used to find table of content links.
2. `</div>\n</div>\n<div id="footer">` is used to delimit the mandoc output
from the rest of the page.
## interesting test cases
[crontab(5)](https://manpages.debian.org/crontab(5)) is present in multiple Debian versions, multiple languages, multiple sections and multiple conflicting packages. Hence, it showcases all debiman features.
[w3m(1)](https://manpages.debian.org/w3m(1)) has a Japanese translation which is only present in UTF-8 starting with Debian jessie. It also has a German translation starting with Debian stretch.
[qelectrotech(1)](https://manpages.debian.org/qelectrotech(1)) has a French translation in 3 different encodings (none specified, ISO8859-1, UTF-8).
[mysqld(8)](https://manpages.debian.org/mysqld(8)) is present in two conflicting packages: `mariadb-server-core-10.0` and `mysql-server-core-5.6`.
## recommended reading
https://wiki.debian.org/RepositoryFormat
## URLs
The URL schema which debiman uses is `(<suite>/)(<binarypkg/>)<name>(.<section>(.<lang>))`. Any part aside from `name` can be omitted; here are a few examples:
Without suite and binary package:
1. https://manpages.debian.org/i3
2. https://manpages.debian.org/i3.fr
3. https://manpages.debian.org/i3.1
4. https://manpages.debian.org/i3.1.fr
With binary package:
1. https://manpages.debian.org/i3-wm/i3
2. https://manpages.debian.org/i3-wm/i3.fr
3. https://manpages.debian.org/i3-wm/i3.1
4. https://manpages.debian.org/i3-wm/i3.1.fr
With suite:
1. https://manpages.debian.org/testing/i3
2. https://manpages.debian.org/testing/i3.fr
3. https://manpages.debian.org/testing/i3.1
4. https://manpages.debian.org/testing/i3.1.fr
With suite and binary package:
1. https://manpages.debian.org/testing/i3-wm/i3
2. https://manpages.debian.org/testing/i3-wm/i3.fr
3. https://manpages.debian.org/testing/i3-wm/i3.1
4. https://manpages.debian.org/testing/i3-wm/i3.1.fr
================================================
FILE: assets/about.tmpl
================================================
{{ template "header" . }}
<div class="maincontents">
<h1>About</h1>
</div>
{{ template "footer" . }}
================================================
FILE: assets/contents.tmpl
================================================
{{ template "header" . }}
<div class="maincontents">
<h1>Binary packages containing manpages in Debian {{ .Suite }}</h1>
<ul>
{{ range $idx, $dir := .Bins }}
{{ if and (not (HasSuffix $dir ".gz")) (not (HasPrefix $dir ".")) }}
<li><a href="{{ BaseURLPath }}/{{ $.Suite }}/{{ $dir}}/index.html">{{ $dir }}</a></li>
{{ end }}
{{ end }}
</ul>
</div>
{{ template "footer" . }}
================================================
FILE: assets/faq.tmpl
================================================
{{ template "header" . }}
<div class="maincontents">
<h1>FAQ</h1>
</div>
{{ template "footer" . }}
================================================
FILE: assets/footer.tmpl
================================================
</div>
<div id="footer">
{{ if ne .FooterExtra "" }}
<p>{{ .FooterExtra }}</p>
{{ else }}
<p>Page last updated {{ Now }}</p>
{{ end }}
<hr>
<div id="fineprint">
<p>debiman {{ .DebimanVersion }}, see <a href="https://github.com/Debian/debiman/">github.com/Debian/debiman</a></p>
</div>
</div>
================================================
FILE: assets/header.tmpl
================================================
<!DOCTYPE html>
{{ if .Meta -}}
<html lang="{{ .Meta.LanguageTag }}">
{{ else -}}
<html lang="en">
{{ end -}}
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{{ .Title }} — debiman</title>
<style type="text/css">
{{ template "style" }}
</style>
<link rel="search" title="Debian manpages" type="application/opensearchdescription+xml" href="/opensearch.xml">
{{ if and (.HrefLangs) (gt (len .HrefLangs) 1) -}}
{{ range $idx, $man := .HrefLangs -}}
<link rel="alternate" href="/{{ $man.ServingPath }}.html" hreflang="{{ $man.LanguageTag }}">
{{ end -}}
{{ end -}}
</head>
<body>
<div id="header">
<div id="upperheader">
<h1><a href="{{ BaseURLPath }}/">some debiman installation</a></h1>
<div id="searchbox">
<form action="{{ BaseURLPath }}/jump" method="get">
{{ if .Meta -}}
<input type="hidden" name="suite" value="{{ .Meta.Package.Suite }}">
<input type="hidden" name="binarypkg" value="{{ .Meta.Package.Binarypkg }}">
<input type="hidden" name="section" value="{{ .Meta.Section }}">
<input type="hidden" name="language" value="{{ .Meta.Language }}">
{{ end -}}
<input type="text" name="q" placeholder="manpage name" required>
<input type="submit" value="Jump">
</form>
</div>
</div>
<div id="navbar">
<p class="hidecss"><a href="#content">Skip Quicknav</a></p>
<ul>
<li><a href="{{ BaseURLPath }}/">Index</a></li>
</ul>
</div>
<p id="breadcrumbs">
{{- range $i, $b := .Breadcrumbs }}
{{ if eq $b.Link "" }}
/ {{ $b.Text }}
{{ else }}
/ <a href="{{ BaseURLPath }}{{ $b.Link }}">{{ $b.Text }}</a>
{{ end }}
{{ end -}}
</p>
</div>
<div id="content">
================================================
FILE: assets/index.tmpl
================================================
{{ template "header" . }}
<div class="maincontents">
<h1>some debiman installation</h1>
<p>
You’re looking at a complete repository of all manpages contained in
Debian.<br>There are a couple of different ways to use this
repository:
</p>
<ol>
<li>
<form method="GET" action="{{ BaseURLPath }}/jump">
Directly jump to manpage:
<input type="text" name="q" autofocus="autofocus" placeholder="manpage name">
<input type="submit" value="Jump to manpage">
</form>
</li>
<li>
In your browser address bar, type enough characters of manpages.debian.org,<br>
press TAB, enter the manpage name, hit ENTER.
</li>
<li>
Navigate to the manpage’s address, using this URL schema:<br>
<code>/<suite>/<binarypackage>/<manpage>.<section>.<language>.html</code><br>
Any part (except <code><manpage></code>) can be omitted, and you will be redirected according to our best guess.
</li>
<li>
Browse the repository index:
<ul>
{{ range $idx, $suite := .Suites }}
<li>
<a href="{{ BaseURLPath }}/contents-{{ $suite }}.html">Debian {{ $suite }}</a>
</li>
{{ end }}
</ul>
</li>
</ol>
</div>
{{ template "footer" . }}
================================================
FILE: assets/manpage.tmpl
================================================
{{ template "header" . }}
<div class="panels" id="panels">
<div class="panel" role="complementary">
<div class="panel-heading" role="heading">
links
</div>
<div class="panel-body">
<ul class="list-group list-group-flush">
<li class="list-group-item">
<a href="{{ BaseURLPath }}/{{ .Meta.PermaLink }}">language-indep link</a>
</li>
<li class="list-group-item">
<a href="https://tracker.debian.org/pkg/{{ .Meta.Package.Binarypkg }}">package tracker</a>
</li>
<li class="list-group-item">
<a href="{{ BaseURLPath }}/{{ .Meta.RawPath }}">raw man page</a>
</li>
</ul>
</div>
</div>
<div class="panel toc" role="complementary" style="padding-bottom: 0">
<details>
<summary>
table of contents
</summary>
<div class="panel-body">
<ul class="list-group list-group-flush">
{{ range $idx, $heading := .TOC }}
<li class="list-group-item">
<a class="toclink" href="{{ FragmentLink $heading }}" title="{{ $heading }}">{{ $heading }}</a>
</li>
{{ end }}
</ul>
</div>
</details>
</div>
<div class="panel otherversions" role="complementary">
<div class="panel-heading" role="heading">
other versions
</div>
<div class="panel-body">
<ul class="list-group list-group-flush">
{{ range $idx, $man := .Suites }}
<li class="list-group-item
{{- if eq $man.Package.Suite $.Meta.Package.Suite }} active{{- end -}}
">
<a href="{{ BaseURLPath }}/{{ $man.ServingPath }}.html">{{ $man.Package.Suite }}</a> <span class="pkgversion" title="{{ $man.Package.Version }}">{{ $man.Package.Version }}</span>
</li>
{{ end }}
</ul>
</div>
</div>
{{ if gt (len .Langs) 1 }}
<div class="panel otherlangs" role="complementary">
<div class="panel-heading" role="heading">
other languages
</div>
<div class="panel-body">
<ul class="list-group list-group-flush">
{{ range $idx, $man := .Langs }}
<li class="list-group-item
{{- if eq $man.Language $.Meta.Language }} active{{- end -}}
">
<a href="{{ BaseURLPath }}/{{ $man.ServingPath }}.html" title="{{ EnglishLang $man.LanguageTag }} ({{ $man.Language }})">{{ DisplayLang $man.LanguageTag }}</a>
{{ if (index $.Ambiguous $man) }}
<span class="pkgname">{{ $man.Package.Binarypkg }}</span>
{{ end }}
</li>
{{ end }}
</ul>
</div>
</div>
{{ end }}
{{ if gt (len .Sections) 1 }}
<div class="panel" role="complementary">
<div class="panel-heading" role="heading">
other sections
</div>
<div class="panel-body">
<ul class="list-group list-group-flush">
{{ range $idx, $man := .Sections }}
<li class="list-group-item
{{- if eq $man.Section $.Meta.Section }} active{{- end -}}
">
<a href="{{ BaseURLPath }}/{{ $man.ServingPath }}.html">{{ $man.Section }} (<span title="{{ LongSection $man.MainSection }}">{{ ShortSection $man.MainSection }}</span>)</a>
</li>
{{ end }}
</ul>
</div>
</div>
{{ end }}
{{ if gt (len .Bins) 1 }}
<div class="panel" role="complementary">
<div class="panel-heading" role="heading">
conflicting packages
</div>
<div class="panel-body">
<ul class="list-group list-group-flush">
{{ range $idx, $man := .Bins }}
<li class="list-group-item
{{- if eq $man.Package.Binarypkg $.Meta.Package.Binarypkg }} active{{- end -}}
">
<a href="{{ BaseURLPath }}/{{ $man.ServingPath }}.html">{{ $man.Package.Binarypkg }}</a>
</li>
{{ end }}
</ul>
</div>
</div>
{{ end }}
</div>
<div class="maincontent">
<p class="paneljump"><a href="#panels">Scroll to navigation</a></p>
{{ .Content }}
</div>
{{ template "footer" . }}
<script type="application/ld+json">
{{ .Breadcrumbs.ToJSON }}
</script>
================================================
FILE: assets/manpageerror.tmpl
================================================
{{ template "header" . }}
<div class="panels" id="panels">
<div class="panel" role="complementary">
<div class="panel-heading" role="heading">
links
</div>
<div class="panel-body">
<ul class="list-group list-group-flush">
<li class="list-group-item">
<a href="{{ BaseURLPath }}/{{ .Meta.PermaLink }}">language-indep link</a>
</li>
<li class="list-group-item">
<a href="https://tracker.debian.org/pkg/{{ .Meta.Package.Binarypkg }}">package tracker</a>
</li>
<li class="list-group-item">
<a href="{{ BaseURLPath }}/{{ .Meta.RawPath }}">raw man page</a>
</li>
</ul>
</div>
</div>
<div class="panel toc" role="complementary" style="padding-bottom: 0">
<details>
<summary>
table of contents
</summary>
<div class="panel-body">
<ul class="list-group list-group-flush">
{{ range $idx, $heading := .TOC }}
<li class="list-group-item">
<a class="toclink" href="{{ FragmentLink $heading }}" title="{{ $heading }}">{{ $heading }}</a>
</li>
{{ end }}
</ul>
</div>
</details>
</div>
<div class="panel otherversions" role="complementary">
<div class="panel-heading" role="heading">
other versions
</div>
<div class="panel-body">
<ul class="list-group list-group-flush">
{{ range $idx, $man := .Suites }}
<li class="list-group-item
{{- if eq $man.Package.Suite $.Meta.Package.Suite }} active{{- end -}}
">
<a href="{{ BaseURLPath }}/{{ $man.ServingPath }}.html">{{ $man.Package.Suite }}</a> <span class="pkgversion" title="{{ $man.Package.Version }}">{{ $man.Package.Version }}</span>
</li>
{{ end }}
</ul>
</div>
</div>
{{ if gt (len .Langs) 1 }}
<div class="panel otherlangs" role="complementary">
<div class="panel-heading" role="heading">
other languages
</div>
<div class="panel-body">
<ul class="list-group list-group-flush">
{{ range $idx, $man := .Langs }}
<li class="list-group-item
{{- if eq $man.Language $.Meta.Language }} active{{- end -}}
">
<a href="{{ BaseURLPath }}/{{ $man.ServingPath }}.html" title="{{ EnglishLang $man.LanguageTag }} ({{ $man.Language }})">{{ DisplayLang $man.LanguageTag }}</a>
{{ if (index $.Ambiguous $man) }}
<span class="pkgname">{{ $man.Package.Binarypkg }}</span>
{{ end }}
</li>
{{ end }}
</ul>
</div>
</div>
{{ end }}
{{ if gt (len .Sections) 1 }}
<div class="panel" role="complementary">
<div class="panel-heading" role="heading">
other sections
</div>
<div class="panel-body">
<ul class="list-group list-group-flush">
{{ range $idx, $man := .Sections }}
<li class="list-group-item
{{- if eq $man.Section $.Meta.Section }} active{{- end -}}
">
<a href="{{ BaseURLPath }}/{{ $man.ServingPath }}.html">{{ $man.Section }} (<span title="{{ LongSection $man.MainSection }}">{{ ShortSection $man.MainSection }}</span>)</a>
</li>
{{ end }}
</ul>
</div>
</div>
{{ end }}
{{ if gt (len .Bins) 1 }}
<div class="panel" role="complementary">
<div class="panel-heading" role="heading">
conflicting packages
</div>
<div class="panel-body">
<ul class="list-group list-group-flush">
{{ range $idx, $man := .Bins }}
<li class="list-group-item
{{- if eq $man.Package.Binarypkg $.Meta.Package.Binarypkg }} active{{- end -}}
">
<a href="{{ BaseURLPath }}/{{ $man.ServingPath }}.html">{{ $man.Package.Binarypkg }}</a>
</li>
{{ end }}
</ul>
</div>
</div>
{{ end }}
</div>
<div class="maincontent">
<p>
Sorry, the manpage could not be rendered!
</p>
<p>
Error message: {{ .Error }}
</p>
</div>
{{ template "footer" . }}
================================================
FILE: assets/manpagefooterextra.tmpl
================================================
<table>
<tr>
<td>
Source file:
</td>
<td>
{{ .SourceFile }} (from <a href="http://snapshot.debian.org/package/{{ .Meta.Package.Sourcepkg }}/{{ .Meta.Package.Version }}/">{{ .Meta.Package.Binarypkg }} {{ .Meta.Package.Version }}</a>)
</td>
</tr>
<tr>
<td>
Source last updated:
</td>
<td>
{{ Iso8601 .LastUpdated }}
</td>
</tr>
<tr>
<td>
Converted to HTML:
</td>
<td>
{{ Iso8601 .Converted }}
</td>
</tr>
</table>
================================================
FILE: assets/notfound.tmpl
================================================
{{ template "header" . }}
<div class="maincontents">
{{ if or (ne .BestChoice.Suite "") (eq .Manpage "index") }}
<p>
Sorry, I could not find the specific manpage version you requested! Possibly it is no longer in Debian?
</p>
{{ else }}
<p>
Sorry, the manpage “{{ .Manpage }}” was not found! Did you spell it correctly?
</p>
{{ end }}
{{ if ne .BestChoice.Suite "" }}
<p>
Could I maybe offer you the manpage <a href="{{ .BestChoice.ServingPath ".html" }}">{{ .BestChoice.ServingPath ".html" }}</a> instead?
</p>
{{ end }}
</div>
{{ template "footer" . }}
================================================
FILE: assets/opensearch.xml
================================================
<?xml version="1.0"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
<ShortName>some debiman installation</ShortName>
<Description>some debiman installation</Description>
<Url type="text/html" method="get" template="http://manpages.debian.org/jump?q={searchTerms}"/>
<Query role="example" searchTerms="wget"/>
<Url type="application/x-suggestions+json" method="GET" template="http://debiman.localhost/suggest?q={searchTerms}" />
</OpenSearchDescription>
================================================
FILE: assets/pkgindex.tmpl
================================================
{{ template "header" . }}
<div class="maincontents">
<h1>Manpages of <a href="https://tracker.debian.org/pkg/{{ .First.Package.Binarypkg }}">{{ .First.Package.Binarypkg }}</a> in Debian {{ .First.Package.Suite }}</h1>
<ul>
{{ range $idx, $fn := .Mans }}
{{ with $m := index $.ManpageByName $fn }}
<li>
<a href="{{ BaseURLPath }}/{{ $m.ServingPath }}.html">{{ $m.Name }}({{ $m.Section }})
{{ if ne $m.Language "en" }}
(<span title="{{ EnglishLang $m.LanguageTag }} ({{ $m.Language }})">{{ DisplayLang $m.LanguageTag }}</span>)
{{ end }}
</a>
</li>
{{ end }}
{{ end }}
</ul>
</div>
{{ template "footer" . }}
================================================
FILE: assets/srcpkgindex.tmpl
================================================
{{ template "header" . }}
<div class="maincontents">
<h1>Manpages of <a href="https://tracker.debian.org/pkg/{{ .Src }}">src:{{ .Src }}</a> in Debian {{ .First.Package.Suite }}</h1>
<ul>
{{ range $idx, $fn := .Mans }}
{{ with $m := index $.ManpageByName $fn }}
<li>
<a href="{{ BaseURLPath }}/{{ $m.ServingPath }}.html">{{ $m.Name }}({{ $m.Section }})
{{ if ne $m.Language "en" }}
(<span title="{{ EnglishLang $m.LanguageTag }} ({{ $m.Language }})">{{ DisplayLang $m.LanguageTag }}</span>)
{{ end }}
</a>
</li>
{{ end }}
{{ end }}
</ul>
</div>
{{ template "footer" . }}
================================================
FILE: assets/style.css
================================================
@font-face {
font-family: 'Inconsolata';
src: local('Inconsolata'), url({{ BaseURLPath }}/Inconsolata.woff2) format('woff2'), url({{ BaseURLPath }}/Inconsolata.woff) format('woff');
}
@font-face {
font-family: 'Roboto';
font-style: normal;
font-weight: 400;
src: local('Roboto'), local('Roboto Regular'), local('Roboto-Regular'), url({{ BaseURLPath }}/Roboto-Regular.woff2) format('woff2'), url({{ BaseURLPath }}/Roboto-Regular.woff) format('woff');
}
@font-face {
font-family: 'Roboto';
font-style: normal;
font-weight: 700;
/* TODO: is local('Roboto Bold') really correct? */
src: local('Roboto Bold'), local('Roboto-Bold'), url({{ BaseURLPath }}/Roboto-Bold.woff2) format('woff2'), url({{ BaseURLPath }}/Roboto-Bold.woff) format('woff');
}
body {
color: #000;
background-color: white;
font-family: 'Roboto', sans-serif;
font-size: 100%;
line-height: 1.2;
letter-spacing: 0.15px;
margin: 0;
padding: 0;
}
body > div#header {
padding: 0 10px 0 52px;
}
#logo {
position: absolute;
top: 0;
left: 0;
border-left: 1px solid transparent;
border-right: 1px solid transparent;
border-bottom: 1px solid transparent;
width: 50px;
height: 5.07em;
min-height: 65px;
}
#logo a {
display: block;
height: 100%;
}
#logo img {
margin-top: 5px;
position: absolute;
bottom: 0.3em;
overflow: auto;
border: 0;
}
#header h1 {
font-size: 100%;
margin: 0;
}
p.section {
margin: 0;
padding: 0 5px 0 5px;
font-family: monospace;
font-size: 13px;
line-height: 16px;
color: white;
letter-spacing: 0.08em;
position: absolute;
top: 0px;
left: 52px;
background-color: #c70036;
}
p.section a {
color: white;
text-decoration: none;
}
.hidecss {
display: none;
}
#searchbox {
text-align:left;
line-height: 1;
margin: 0 10px 0 0.5em;
padding: 1px 0 1px 0;
position: absolute;
top: 0;
right: 0;
font-size: .75em;
}
#navbar ul {
margin: 0;
padding: 0;
overflow: hidden;
}
#navbar li {
list-style: none;
float: left;
}
#navbar a {
display: block;
color: #0035c7;
text-decoration: none;
border-left: 1px solid transparent;
border-right: 1px solid transparent;
}
#navbar a:hover
, #navbar a:visited:hover {
text-decoration: underline;
}
a:link {
color: #0035c7;
}
a:visited {
color: #54638c;
}
#breadcrumbs {
line-height: 2;
min-height: 20px;
margin: 0;
padding: 0;
font-size: 0.75em;
border-bottom: 1px solid #d2d3d7;
}
#breadcrumbs:before {
margin-left: 0.5em;
margin-right: 0.5em;
}
#content {
margin: 0 10px 0 52px;
display: flex;
flex-direction: row;
word-wrap: break-word;
}
.paneljump {
background-color: #d70751;
padding: 0.5em;
border-radius: 3px;
margin-right: .5em;
display: none;
}
.paneljump a,
.paneljump a:visited,
.paneljump a:hover,
.paneljump a:focus {
color: white;
}
@media all and (max-width: 800px) {
#content {
flex-direction: column;
margin: 0.5em;
}
.paneljump {
display: block;
}
}
.panels {
display: block;
order: 2;
}
.maincontent {
width: 100%;
max-width: 80ch;
order: 1;
}
body > div#footer {
border: 1px solid #dfdfe0;
border-left: 0;
border-right: 0;
background-color: #f5f6f7;
padding: 1em;
margin: 1em 10px 0 52px;
font-size: 0.75em;
line-height: 1.5em;
}
hr {
border-top: 1px solid #d2d3d7;
border-bottom: 1px solid white;
border-left: 0;
border-right: 0;
margin: 1.4375em 0 1.5em 0;
height: 0;
background-color: #bbb;
}
#content p {
padding-left: 1em;
}
/* from tracker.debian.org */
a, a:hover, a:focus, a:visited {
color: #0530D7;
text-decoration: none;
}
/* Panel styles */
.panel {
padding: 15px;
margin-bottom: 20px;
background-color: #ffffff;
border: 1px solid #dddddd;
border-radius: 4px;
-webkit-box-shadow: 0 1px 1px rgba(0, 0, 0, 0.05);
box-shadow: 0 1px 1px rgba(0, 0, 0, 0.05);
}
.panel-heading, .panel details {
margin: -15px -15px 0px;
background-color: #d70751;
border-bottom: 1px solid #dddddd;
border-top-right-radius: 3px;
border-top-left-radius: 3px;
}
.panel-heading, .panel summary {
padding: 5px 5px;
font-size: 17.5px;
font-weight: 500;
color: #ffffff;
outline-style: none;
}
.panel summary {
padding-left: 7px;
}
summary, details {
display: block;
}
.panel details ul {
margin: 0;
}
.panel-footer {
padding: 5px 5px;
margin: 15px -15px -15px;
background-color: #f5f5f5;
border-top: 1px solid #dddddd;
border-bottom-right-radius: 3px;
border-bottom-left-radius: 3px;
}
.panel-info {
border-color: #bce8f1;
}
.panel-info .panel-heading {
color: #3a87ad;
background-color: #d9edf7;
border-color: #bce8f1;
}
.list-group {
padding-left: 0;
margin-bottom: 20px;
background-color: #ffffff;
}
.list-group-item {
position: relative;
display: block;
padding: 5px 5px 5px 5px;
margin-bottom: -1px;
border: 1px solid #dddddd;
}
.list-group-item > .list-item-key {
min-width: 27%;
display: inline-block;
}
.list-group-item > .list-item-key.versions-repository {
min-width: 40%;
}
.list-group-item > .list-item-key.versioned-links-version {
min-width: 40%
}
.versioned-links-icon {
margin-right: 2px;
}
.versioned-links-icon a {
color: black;
}
.versioned-links-icon a:hover {
color: blue;
}
.versioned-links-icon-inactive {
opacity: 0.5;
}
.list-group-item:first-child {
border-top-right-radius: 4px;
border-top-left-radius: 4px;
}
.list-group-item:last-child {
margin-bottom: 0;
border-bottom-right-radius: 4px;
border-bottom-left-radius: 4px;
}
.list-group-item-heading {
margin-top: 0;
margin-bottom: 5px;
}
.list-group-item-text {
margin-bottom: 0;
line-height: 1.3;
}
.list-group-item:hover {
background-color: #efefef;
}
.list-group-item.active a {
z-index: 2;
}
.list-group-item.active {
background-color: #efefef;
}
.list-group-flush {
margin: 15px -15px -15px;
}
.panel .list-group-flush {
margin-top: -1px;
}
.list-group-flush .list-group-item {
border-width: 1px 0;
}
.list-group-flush .list-group-item:first-child {
border-top-right-radius: 0;
border-top-left-radius: 0;
}
.list-group-flush .list-group-item:last-child {
border-bottom: 0;
}
/* end of tracker.debian.org */
.panel {
float: right;
clear: right;
font-family: 'Roboto';
min-width: 200px;
}
.toc {
/* Limit the content’s width */
width: 200px;
}
.toc li {
font-size: 98%;
letter-spacing: 0.02em;
display: flex;
}
.otherversions {
/* Limit the content’s width */
width: 200px;
}
.otherversions li,
.otherlangs li {
display: flex;
}
.otherversions a,
.otherlangs a {
flex-shrink: 0;
}
.pkgversion,
.pkgname,
.toc a {
text-overflow: ellipsis;
overflow: hidden;
white-space: nowrap;
}
.pkgversion,
.pkgname {
margin-left: auto;
padding-left: 1em;
}
/* mandoc styles */
.mandoc, .mandoc pre, .mandoc code {
font-family: 'Inconsolata', monospace;
font-size: 1.04rem;
}
.mandoc pre {
white-space: pre-wrap;
}
.mandoc {
margin-right: 45px;
/* Required so that table.head and table.foot can take up 100% of what remains after floating the panels. */
overflow: hidden;
margin-top: .5em;
}
table.head, table.foot {
width: 100%;
}
.head-vol {
text-align: center;
}
.head-rtitle {
text-align: right;
}
/* TODO(later): get rid of .spacer once a new-enough mandoc is in Debian */
.spacer, .Pp {
min-height: 1em;
}
pre {
margin-left: 2em;
}
.anchor {
margin-left: .25em;
visibility: hidden;
}
h1:hover .anchor,
h2:hover .anchor,
h3:hover .anchor,
h4:hover .anchor,
h5:hover .anchor,
h6:hover .anchor {
visibility: visible;
}
h1, h2, h3, h4, h5, h6 {
letter-spacing: .07em;
margin-top: 1.5em;
margin-bottom: .35em;
}
h1 {
font-size: 150%;
}
h2 {
font-size: 125%;
}
@media print {
#header, #footer, .panel, .anchor, .paneljump {
display: none;
}
#content {
margin: 0;
}
.mandoc {
margin: 0;
}
}
/* from mandoc.css */
/* Displays and lists. */
.Bd { }
.Bd-indent { margin-left: 3.8em; }
.Bl-bullet { list-style-type: disc;
padding-left: 1em; }
.Bl-bullet > li { }
.Bl-dash { list-style-type: none;
padding-left: 0em; }
.Bl-dash > li:before {
content: "\2014 "; }
.Bl-item { list-style-type: none;
padding-left: 0em; }
.Bl-item > li { }
.Bl-compact > li {
margin-top: 0em; }
.Bl-enum { padding-left: 2em; }
.Bl-enum > li { }
.Bl-compact > li {
margin-top: 0em; }
.Bl-diag { }
.Bl-diag > dt {
font-style: normal;
font-weight: bold; }
.Bl-diag > dd {
margin-left: 0em; }
.Bl-hang { }
.Bl-hang > dt { }
.Bl-hang > dd {
margin-left: 5.5em; }
.Bl-inset { }
.Bl-inset > dt { }
.Bl-inset > dd {
margin-left: 0em; }
.Bl-ohang { }
.Bl-ohang > dt { }
.Bl-ohang > dd {
margin-left: 0em; }
.Bl-tag { margin-left: 5.5em; }
.Bl-tag > dt {
float: left;
margin-top: 0em;
margin-left: -5.5em;
padding-right: 1.2em;
vertical-align: top; }
.Bl-tag > dd {
clear: both;
width: 100%;
margin-top: 0em;
margin-left: 0em;
vertical-align: top;
overflow: auto; }
.Bl-compact > dt {
margin-top: 0em; }
.Bl-column { }
.Bl-column > tbody > tr { }
.Bl-column > tbody > tr > td {
margin-top: 1em; }
.Bl-compact > tbody > tr > td {
margin-top: 0em; }
.Rs { font-style: normal;
font-weight: normal; }
.RsA { }
.RsB { font-style: italic;
font-weight: normal; }
.RsC { }
.RsD { }
.RsI { font-style: italic;
font-weight: normal; }
.RsJ { font-style: italic;
font-weight: normal; }
.RsN { }
.RsO { }
.RsP { }
.RsQ { }
.RsR { }
.RsT { text-decoration: underline; }
.RsU { }
.RsV { }
.eqn { }
.tbl { }
.HP { margin-left: 3.8em;
text-indent: -3.8em; }
/* Semantic markup for command line utilities. */
table.Nm { }
code.Nm { font-style: normal;
font-weight: bold;
font-family: inherit; }
.Fl { font-style: normal;
font-weight: bold;
font-family: inherit; }
.Cm { font-style: normal;
font-weight: bold;
font-family: inherit; }
.Ar { font-style: italic;
font-weight: normal; }
.Op { display: inline; }
.Ic { font-style: normal;
font-weight: bold;
font-family: inherit; }
.Ev { font-style: normal;
font-weight: normal;
font-family: monospace; }
.Pa { font-style: italic;
font-weight: normal; }
/* Semantic markup for function libraries. */
.Lb { }
code.In { font-style: normal;
font-weight: bold;
font-family: inherit; }
a.In { }
.Fd { font-style: normal;
font-weight: bold;
font-family: inherit; }
.Ft { font-style: italic;
font-weight: normal; }
.Fn { font-style: normal;
font-weight: bold;
font-family: inherit; }
.Fa { font-style: italic;
font-weight: normal; }
.Vt { font-style: italic;
font-weight: normal; }
.Va { font-style: italic;
font-weight: normal; }
.Dv { font-style: normal;
font-weight: normal;
font-family: monospace; }
.Er { font-style: normal;
font-weight: normal;
font-family: monospace; }
/* Various semantic markup. */
.An { }
.Lk { }
.Mt { }
.Cd { font-style: normal;
font-weight: bold;
font-family: inherit; }
.Ad { font-style: italic;
font-weight: normal; }
.Ms { font-style: normal;
font-weight: bold; }
.St { }
.Ux { }
/* Physical markup. */
.Bf { display: inline; }
.No { font-style: normal;
font-weight: normal; }
.Em { font-style: italic;
font-weight: normal; }
.Sy { font-style: normal;
font-weight: bold; }
.Li { font-style: normal;
font-weight: normal;
font-family: monospace; }
================================================
FILE: bundle.go
================================================
package bundle
//go:generate sh -c "go run goembed.go -package bundled -var assets assets/header.tmpl assets/footer.tmpl assets/style.css assets/manpage.tmpl assets/manpageerror.tmpl assets/manpagefooterextra.tmpl assets/contents.tmpl assets/pkgindex.tmpl assets/srcpkgindex.tmpl assets/index.tmpl assets/faq.tmpl assets/notfound.tmpl assets/Inconsolata.woff assets/Inconsolata.woff2 assets/opensearch.xml assets/Roboto-Bold.woff assets/Roboto-Bold.woff2 assets/Roboto-Regular.woff assets/Roboto-Regular.woff2 > internal/bundled/GENERATED_bundled.go"
================================================
FILE: cmd/debiman/download.go
================================================
package main
import (
"archive/tar"
"bufio"
"bytes"
"compress/gzip"
"fmt"
"io"
"io/ioutil"
"log"
"os"
"path/filepath"
"sort"
"strings"
"sync/atomic"
"unicode/utf8"
"golang.org/x/net/context"
"golang.org/x/sync/errgroup"
"github.com/Debian/debiman/internal/manpage"
"github.com/Debian/debiman/internal/recode"
"github.com/Debian/debiman/internal/write"
"pault.ag/go/archive"
"pault.ag/go/debian/control"
"pault.ag/go/debian/deb"
"pault.ag/go/debian/version"
)
// canSkip returns true if the package is present in the same (or a
// newer) version on disk already.
func canSkip(p pkgEntry, vPath string) bool {
v, err := ioutil.ReadFile(vPath)
if err != nil {
return false
}
vCurrent, err := version.Parse(string(v))
if err != nil {
log.Printf("Warning: could not parse current package version from %q: %v", vPath, err)
return false
}
return version.Compare(vCurrent, p.version) >= 0
}
type contentByBinarypkg []*contentEntry
func (p contentByBinarypkg) Len() int { return len(p) }
func (p contentByBinarypkg) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
func (p contentByBinarypkg) Less(i, j int) bool { return p[i].binarypkg < p[j].binarypkg }
// findClosestFile returns a manpage struct for name, if name exists in the same suite.
// TODO(stapelberg): resolve multiple matches: consider dependencies of src
func findClosestFile(logger *log.Logger, p pkgEntry, src, name string, contentByPath map[string][]*contentEntry) string {
logger.Printf("findClosestFile(src=%q, name=%q)", src, name)
c, ok := contentByPath[strings.TrimPrefix(name, "/usr/share/man/")]
if !ok {
return ""
}
// Ensure we only consider choices within the same suite.
filtered := make([]*contentEntry, 0, len(c))
for _, e := range c {
if e.suite != p.suite {
continue
}
filtered = append(filtered, e)
}
c = filtered
// We still have more than one choice. In case the candidate is in
// the same package as the source link, we take it.
if len(c) > 1 {
var last *contentEntry
cnt := 0
for _, e := range c {
if e.binarypkg != p.binarypkg {
continue
}
last = e
if cnt++; cnt > 1 {
break
}
}
if cnt == 1 {
c = []*contentEntry{last}
}
// We can’t make a 100% correct choice, but we can at least
// make a deterministic choice. The user will see the
// conflicting packages in the navigation panel to ultimately
// resolve the situation, if necessary.
sort.Sort(contentByBinarypkg(c))
}
if len(c) == 0 {
return ""
}
m, err := manpage.FromManPath(strings.TrimPrefix(name, "/usr/share/man/"), &manpage.PkgMeta{
Binarypkg: c[0].binarypkg,
Suite: c[0].suite,
})
logger.Printf("parsing %q as man: %v", name, err)
if err == nil {
return m.ServingPath() + ".gz"
}
return ""
}
func findFile(logger *log.Logger, src, name string, contentByPath map[string][]*contentEntry) (string, string, bool) {
// TODO(later): why is "/"+ in front of src necessary?
searchPath := []string{
"/" + filepath.Dir(src), // “.”
// To prefer manpages in the same language, add “..”, e.g.:
// /usr/share/man/fr/man7/bash-builtins.7 references
// man1/bash.1, which should be taken from
// /usr/share/man/fr/man1/bash.1 instead of
// /usr/share/man/man1/bash.1.
"/" + filepath.Dir(src) + "/..",
"/usr/share/man",
}
logger.Printf("searching reference so=%q", name)
for _, search := range searchPath {
var check string
if filepath.IsAbs(name) {
check = filepath.Clean(name)
} else {
check = filepath.Join(search, name)
}
// Some references include the .gz suffix, some don’t.
if !strings.HasSuffix(check, ".gz") {
check = check + ".gz"
}
c, ok := contentByPath[strings.TrimPrefix(check, "/usr/share/man/")]
if !ok {
log.Printf("%q does not exist", check)
continue
}
sort.Sort(contentByBinarypkg(c))
m, err := manpage.FromManPath(strings.TrimPrefix(check, "/usr/share/man/"), &manpage.PkgMeta{
Binarypkg: c[0].binarypkg,
Suite: c[0].suite,
})
logger.Printf("parsing %q as man: %v", check, err)
if err == nil {
return m.ServingPath() + ".gz", "", true
}
// TODO(later): try to resolve this reference intelligently, i.e. consider installability to narrow down the list of candidates. add a testcase with all cases that we have in all Debian suites currently
return c[0].suite + "/" + c[0].binarypkg + "/aux" + check, check, true
}
return name, "", false
}
func soElim(logger *log.Logger, src string, r io.Reader, w io.Writer, contentByPath map[string][]*contentEntry) ([]string, error) {
var refs []string
scanner := bufio.NewScanner(r)
for scanner.Scan() {
line := scanner.Text()
if !strings.HasPrefix(line, ".so ") {
fmt.Fprintln(w, line)
continue
}
so := strings.TrimSpace(line[len(".so "):])
resolved, ref, ok := findFile(logger, src, so, contentByPath)
if !ok {
// Omitting .so lines which cannot be found is consistent
// with what man(1) and other online man viewers do.
logger.Printf("WARNING: could not find .so referenced file %q, omitting the .so line", so)
continue
}
fmt.Fprintf(w, ".so %s\n", resolved)
if ref != "" {
refs = append(refs, ref)
}
}
return refs, scanner.Err()
}
func writeManpage(logger *log.Logger, src, dest string, r io.Reader, m *manpage.Meta, contentByPath map[string][]*contentEntry) ([]string, error) {
var refs []string
content, err := ioutil.ReadAll(r)
if err != nil {
return nil, err
}
if !utf8.Valid(content) {
content, err = ioutil.ReadAll(recode.Reader(bytes.NewReader(content), m.Language))
if err != nil {
return nil, err
}
}
err = write.Atomically(dest, true, func(w io.Writer) error {
var err error
refs, err = soElim(logger, src, bytes.NewReader(content), w, contentByPath)
return err
})
return refs, err
}
func createAlternativesLinks(logger *log.Logger, p pkgEntry, gv globalView) (map[string]bool, error) {
refs := make(map[string]bool)
key := p.suite + "/" + p.binarypkg
if len(gv.alternatives[key]) == 0 {
return nil, nil
}
logger.Printf("creating %d links for binary package %q", len(gv.alternatives[key]), p.binarypkg)
for _, link := range gv.alternatives[key] {
if !strings.HasPrefix(link.from, "/usr/share/man/") {
continue
}
m, err := manpage.FromManPath(strings.TrimPrefix(link.from, "/usr/share/man/"), &manpage.PkgMeta{
Binarypkg: p.binarypkg,
Suite: p.suite,
})
if err != nil {
logger.Printf("WARNING: file name %q (underneath /usr/share/man) cannot be parsed: %v", link.from, err)
continue
}
resolved := link.to
if !strings.HasSuffix(resolved, ".gz") {
resolved = resolved + ".gz"
}
destsp := findClosestFile(logger, p, link.from, resolved, gv.contentByPath)
if destsp == "" {
// Try to extract the resolved file as non-manpage
// file. If the resolved file does not live in this
// package, this will result in a dangling symlink.
refs[resolved] = true
destsp = filepath.Join(filepath.Dir(m.ServingPath()), "aux", resolved)
logger.Printf("WARNING: possibly dangling symlink %q -> %q, setting to %q", link.from, link.to, destsp)
}
// TODO(stapelberg): add a unit test for this entire function
// TODO(stapelberg): ganeti has an interesting twist: their manpages live outside of usr/share/man, and they only have symlinks. in this case, we should extract the file to aux/ and then mangle the symlink dest. problem: manpages actually are in a separate package (ganeti-2.15) and use an absolute symlink (/etc/ganeti/share), which is not shipped with the package.
rel, err := filepath.Rel(filepath.Dir(m.ServingPath()), destsp)
if err != nil {
logger.Printf("WARNING: %v", err)
continue
}
if err := os.MkdirAll(filepath.Dir(m.ServingPath()), 0755); err != nil {
return refs, err
}
if err := os.Symlink(rel, m.ServingPath()+".gz"); err != nil {
if os.IsExist(err) {
continue
}
return refs, err
}
}
return refs, nil
}
func downloadPkg(ar *archive.Downloader, p pkgEntry, gv globalView) error {
vPath := filepath.Join(*servingDir, p.suite, p.binarypkg, "VERSION")
logger := log.New(os.Stderr, p.suite+"/"+p.binarypkg+": ", log.LstdFlags)
if !*forceReextract && canSkip(p, vPath) {
// Even when skipping the package, the alternatives data we get from
// piuparts might have changed, see issue #119.
if _, err := createAlternativesLinks(logger, p, gv); err != nil {
return err
}
return nil
}
tmp, err := ar.TempFile(control.FileHash{
Filename: p.filename,
Algorithm: "sha256",
Hash: fmt.Sprintf("%x", p.sha256),
})
if err != nil {
return fmt.Errorf("archive download: %v", err)
}
defer os.Remove(tmp.Name())
defer tmp.Close()
if _, err := tmp.Seek(0, os.SEEK_SET); err != nil {
return err
}
allRefs := make(map[string]bool)
d, err := deb.Load(tmp, p.filename)
if err != nil {
return fmt.Errorf("loading %q: %v", p.filename, err)
}
for {
header, err := d.Data.Next()
if err == io.EOF {
break
}
if err != nil {
return err
}
if header.Typeflag != tar.TypeReg &&
header.Typeflag != tar.TypeRegA &&
header.Typeflag != tar.TypeSymlink &&
header.Typeflag != tar.TypeLink {
continue
}
if header.FileInfo().IsDir() {
continue
}
if !strings.HasPrefix(header.Name, "./usr/share/man/") {
continue
}
destdir := filepath.Join(*servingDir, p.suite, p.binarypkg)
if err := os.MkdirAll(destdir, 0755); err != nil {
return err
}
// TODO: return m?
m, err := manpage.FromManPath(strings.TrimPrefix(header.Name, "./usr/share/man/"), &manpage.PkgMeta{
Binarypkg: p.binarypkg,
Suite: p.suite,
})
if err != nil {
logger.Printf("WARNING: file name %q (underneath /usr/share/man) cannot be parsed: %v", header.Name, err)
continue
}
destPath := filepath.Join(*servingDir, m.ServingPath()+".gz")
if header.Typeflag == tar.TypeLink {
d, err := manpage.FromManPath(strings.TrimPrefix(header.Linkname, "./usr/share/man/"), &manpage.PkgMeta{
Binarypkg: p.binarypkg,
Suite: p.suite,
})
if err != nil {
logger.Printf("WARNING: hard link name %q (underneath /usr/share/man) cannot be parsed: %v", header.Linkname, err)
continue
}
if err := os.Link(filepath.Join(*servingDir, d.ServingPath()+".gz"), m.ServingPath()+".gz"); err != nil {
if os.IsExist(err) {
continue
}
return err
}
continue
}
if header.Typeflag == tar.TypeSymlink {
// filepath.Join calls filepath.Abs
resolved := filepath.Join(filepath.Dir(strings.TrimPrefix(header.Name, ".")), header.Linkname)
if !strings.HasSuffix(resolved, ".gz") {
resolved = resolved + ".gz"
}
destsp := findClosestFile(logger, p, header.Name, resolved, gv.contentByPath)
if destsp == "" {
// Try to extract the resolved file as non-manpage
// file. If the resolved file does not live in this
// package, this will result in a dangling symlink.
allRefs[resolved] = true
destsp = filepath.Join(filepath.Dir(m.ServingPath()), "aux", resolved)
logger.Printf("WARNING: possibly dangling symlink %q -> %q", header.Name, header.Linkname)
}
// TODO(stapelberg): add a unit test for this entire function
// TODO(stapelberg): ganeti has an interesting twist: their manpages live outside of usr/share/man, and they only have symlinks. in this case, we should extract the file to aux/ and then mangle the symlink dest. problem: manpages actually are in a separate package (ganeti-2.15) and use an absolute symlink (/etc/ganeti/share), which is not shipped with the package.
rel, err := filepath.Rel(filepath.Dir(m.ServingPath()), destsp)
if err != nil {
logger.Printf("WARNING: %v", err)
continue
}
if err := os.Symlink(rel, destPath); err != nil {
if os.IsExist(err) {
continue
}
return err
}
if err := maybeSetLinkMtime(destPath, header.ModTime); err != nil {
return err
}
continue
}
r := io.Reader(d.Data)
var gzr *gzip.Reader
if strings.HasSuffix(header.Name, ".gz") {
gzr, err = gzip.NewReader(d.Data)
if err != nil {
return err
}
r = gzr
}
refs, err := writeManpage(logger, header.Name, destPath, r, m, gv.contentByPath)
if err != nil {
return err
}
if err := os.Chtimes(destPath, header.ModTime, header.ModTime); err != nil {
return err
}
if gzr != nil {
if err := gzr.Close(); err != nil {
return err
}
}
for _, r := range refs {
allRefs[r] = true
}
}
// Create all symlinks for slave alternatives.
refs, err := createAlternativesLinks(logger, p, gv)
if err != nil {
return err
}
for r := range refs {
allRefs[r] = true
}
// Extract all non-manpage files which were referenced via .so
// statements, if any.
if len(allRefs) > 0 {
if _, err := tmp.Seek(0, os.SEEK_SET); err != nil {
return err
}
d, err = deb.Load(tmp, p.filename)
if err != nil {
return err
}
for {
header, err := d.Data.Next()
if err == io.EOF {
break
}
if err != nil {
return err
}
if header.Typeflag != tar.TypeReg &&
header.Typeflag != tar.TypeRegA &&
header.Typeflag != tar.TypeSymlink {
continue
}
if header.FileInfo().IsDir() {
continue
}
if !allRefs[strings.TrimPrefix(header.Name, ".")] {
continue
}
destPath := filepath.Join(*servingDir, p.suite, p.binarypkg, "aux", header.Name)
logger.Printf("extracting referenced non-manpage file %q to %q", header.Name, destPath)
if err := os.MkdirAll(filepath.Dir(destPath), 0755); err != nil {
return err
}
if err := write.Atomically(destPath, false, func(w io.Writer) error {
_, err := io.Copy(w, d.Data)
return err
}); err != nil {
return err
}
}
}
if err := ioutil.WriteFile(vPath, []byte(p.version.String()), 0644); err != nil {
if os.IsNotExist(err) {
// If the directory does not exist, we did not extract any
// manpages. Since Contents files are not precise (they
// might lag behind), this can happen occasionally.
return nil
}
return fmt.Errorf("Writing version file %q: %v", vPath, err)
}
atomic.AddUint64(&gv.stats.PackagesExtracted, 1)
return nil
}
func parallelDownload(ar *archive.Downloader, gv globalView) error {
eg, ctx := errgroup.WithContext(context.Background())
downloadChan := make(chan pkgEntry)
// TODO: flag for parallelism level
for i := 0; i < 10; i++ {
eg.Go(func() error {
for p := range downloadChan {
if err := downloadPkg(ar, p, gv); err != nil {
return fmt.Errorf("downloading %s/src:%s %v: %v", p.suite, p.source, p.version, err)
}
}
return nil
})
}
for _, p := range gv.pkgs {
select {
case downloadChan <- *p:
case <-ctx.Done():
break
}
}
close(downloadChan)
return eg.Wait()
}
================================================
FILE: cmd/debiman/download_test.go
================================================
package main
import (
"bytes"
"log"
"os"
"strings"
"testing"
)
func TestWriteManpage(t *testing.T) {
table := []struct {
src string
manpage string
want string
wantRefs []string
pkg pkgEntry
contentByPath map[string][]*contentEntry
}{
{
src: "/usr/share/man/man1/noref.1",
manpage: "no ref in here\n",
want: "no ref in here\n",
wantRefs: nil,
pkg: pkgEntry{},
contentByPath: make(map[string][]*contentEntry),
},
{
src: "/usr/share/man/man1/unresolved.1",
manpage: ".so notfound.1\n",
want: "",
wantRefs: nil,
pkg: pkgEntry{},
contentByPath: make(map[string][]*contentEntry),
},
{
src: "/usr/share/man/man1/samepkg.1",
manpage: ".so man1/samepkg.1\n",
want: ".so jessie/bash/samepkg.1.en.gz\n",
wantRefs: nil,
pkg: pkgEntry{
binarypkg: "bash",
suite: "jessie",
},
contentByPath: map[string][]*contentEntry{
"man1/samepkg.1.gz": []*contentEntry{
&contentEntry{
binarypkg: "bash",
suite: "jessie",
},
},
},
},
{
src: "/usr/share/man/man1/samepkgaux.1",
manpage: ".so man1/samepkgaux.inc\n",
want: ".so jessie/bash/aux/usr/share/man/man1/samepkgaux.inc.gz\n",
wantRefs: []string{
"/usr/share/man/man1/samepkgaux.inc.gz",
},
pkg: pkgEntry{
binarypkg: "bash",
suite: "jessie",
},
contentByPath: map[string][]*contentEntry{
"man1/samepkgaux.inc.gz": []*contentEntry{
&contentEntry{
binarypkg: "bash",
suite: "jessie",
},
},
},
},
{
src: "/usr/share/man/man1/samedir.1",
manpage: ".so samedir.inc\n",
want: ".so jessie/bash/aux/usr/share/man/man1/samedir.inc.gz\n",
wantRefs: []string{
"/usr/share/man/man1/samedir.inc.gz",
},
pkg: pkgEntry{
binarypkg: "bash",
suite: "jessie",
},
contentByPath: map[string][]*contentEntry{
"man1/samedir.inc.gz": []*contentEntry{
&contentEntry{
binarypkg: "bash",
suite: "jessie",
},
},
},
},
// example for an absolute path: isdnutils-base/isdnctrl.8.en.gz uses .so /usr/share/man/man8/.isdnctrl_conf.8
{
src: "/usr/share/man/man1/absolute.1",
manpage: ".so /usr/share/man/man8/absolute.8\n",
want: ".so jessie/extra/absolute.8.en.gz\n",
wantRefs: nil,
pkg: pkgEntry{
binarypkg: "bash",
suite: "jessie",
},
contentByPath: map[string][]*contentEntry{
"man8/absolute.8.gz": []*contentEntry{
&contentEntry{
binarypkg: "extra",
suite: "jessie",
},
},
},
},
{
src: "/usr/share/man/man1/absolutenotfound.1",
manpage: ".so /usr/share/man/man8/absolute.8\n",
want: "",
wantRefs: nil,
pkg: pkgEntry{
binarypkg: "bash",
suite: "jessie",
},
contentByPath: map[string][]*contentEntry{},
},
{
src: "/usr/share/man/fr/man7/bash-builtins.7",
manpage: ".so man1/bash.1\n",
want: ".so jessie/manpages-fr-extra/bash.1.fr.gz\n",
wantRefs: nil,
pkg: pkgEntry{
binarypkg: "manpages-fr-extra",
suite: "jessie",
},
contentByPath: map[string][]*contentEntry{
"man1/bash.1.gz": []*contentEntry{
&contentEntry{
binarypkg: "bash",
suite: "jessie",
},
},
"fr/man1/bash.1.gz": []*contentEntry{
&contentEntry{
binarypkg: "manpages-fr-extra",
suite: "jessie",
},
},
},
},
}
for _, entry := range table {
entry := entry // capture
t.Run(entry.src, func(t *testing.T) {
t.Parallel()
r := strings.NewReader(entry.manpage)
var buf bytes.Buffer
logger := log.New(os.Stderr, "", log.LstdFlags)
refs, err := soElim(logger, entry.src, r, &buf, entry.contentByPath)
if err != nil {
t.Fatal(err)
}
if got, want := buf.String(), entry.want; got != want {
t.Fatalf("Unexpected soElim() result: got %q, want %q", got, want)
}
if got, want := len(refs), len(entry.wantRefs); got != want {
t.Fatalf("Unexpected number of soElim() ref results: got %d, want %d", got, want)
}
for i := 0; i < len(refs); i++ {
if got, want := refs[i], entry.wantRefs[i]; got != want {
t.Fatalf("soElim() ref differs in entry %d: got %q, want %q", i, got, want)
}
}
})
}
}
================================================
FILE: cmd/debiman/getcontents.go
================================================
package main
import (
"bufio"
"bytes"
"fmt"
"io"
"log"
"os"
"golang.org/x/sync/errgroup"
"pault.ag/go/archive"
"pault.ag/go/debian/control"
)
type contentEntry struct {
suite string
arch string
binarypkg string
filename string
}
var manPrefix = []byte("usr/share/man/")
func parseContentsEntry(scanner *bufio.Scanner) ([]*contentEntry, error) {
for scanner.Scan() {
text := scanner.Bytes()
if !bytes.HasPrefix(text, manPrefix) {
continue
}
idx := bytes.LastIndex(text, []byte{' '})
if idx == -1 {
continue
}
parts := bytes.Split(text[idx:], []byte{','})
entries := make([]*contentEntry, 0, len(parts))
for _, part := range parts {
idx2 := bytes.LastIndex(part, []byte{'/'})
if idx2 == -1 {
continue
}
entries = append(entries, &contentEntry{
binarypkg: string(part[idx2+1:]),
filename: string(bytes.TrimSpace(text[len(manPrefix):idx])),
})
}
if len(entries) > 0 {
return entries, nil
}
}
if err := scanner.Err(); err != nil {
return nil, err
}
return nil, io.EOF
}
func getContents(ar *archive.Downloader, suite string, component string, archs []string, hashByFilename map[string]*control.SHA256FileHash) ([]*contentEntry, error) {
files := make([]*os.File, len(archs))
scanners := make([]*bufio.Scanner, len(archs))
contents := make([][]*contentEntry, len(archs))
advance := make([]bool, len(archs))
exhausted := make([]bool, len(archs))
var eg errgroup.Group
for idx, arch := range archs {
idx := idx // copy
arch := arch // copy
eg.Go(func() error {
path := component + "/Contents-" + arch + ".gz"
fh, ok := hashByFilename[path]
if !ok {
return fmt.Errorf("ERROR: expected path %q not found in Release file", path)
}
log.Printf("getting %q (hash %v)", suite+"/"+path, fh.Hash)
fh.Filename = "dists/" + suite + "/" + fh.Filename
r, err := ar.TempFile(fh.FileHash)
if err != nil {
return err
}
files[idx] = r
scanners[idx] = bufio.NewScanner(r)
// Some packages have excessively large fields, see e.g.:
// https://bugs.debian.org/942487
scanners[idx].Buffer(nil, 512*1024)
contents[idx], err = parseContentsEntry(scanners[idx])
if err != nil {
if err == io.EOF {
exhausted[idx] = true
return nil
}
return err
}
advance[idx] = false
return nil
})
}
defer func() {
for _, f := range files {
if f != nil {
os.Remove(f.Name())
f.Close()
}
}
}()
if err := eg.Wait(); err != nil {
return nil, err
}
var entries []*contentEntry
for {
for idx, move := range advance {
if !move {
continue
}
var err error
contents[idx], err = parseContentsEntry(scanners[idx])
if err != nil {
if err == io.EOF {
exhausted[idx] = true
} else {
return nil, err
}
}
}
// TODO: unit test for edge cases: can this loop indefinitely or can packages be skipped here?
if done(exhausted) {
break
}
// find the filename which is the least advanced in the sort order
var lowest int
var sum int
for idx := range archs {
sum += len(contents[idx])
if exhausted[idx] {
continue
}
if len(contents[lowest]) == 0 || contents[idx][0].filename < contents[lowest][0].filename {
lowest = idx
}
}
for idx := range advance {
advance[idx] = !exhausted[idx] && contents[lowest][0].filename == contents[idx][0].filename
}
binarypkgs := make(map[string]string, sum)
for idx := range archs {
if !advance[idx] {
continue
}
for _, e := range contents[idx] {
// first arch (amd64) wins
if _, ok := binarypkgs[e.binarypkg]; !ok {
binarypkgs[e.binarypkg] = archs[idx]
}
}
}
for pkg, arch := range binarypkgs {
entries = append(entries, &contentEntry{
binarypkg: pkg,
arch: arch,
filename: contents[lowest][0].filename,
suite: suite,
})
}
}
return entries, nil
}
func getAllContents(ar *archive.Downloader, suite string, release *archive.Release, hashByFilename map[string]*control.SHA256FileHash) ([]*contentEntry, error) {
// We skip archAll, because there is no Contents-all file. The
// contents of Architecture: all packages are included in the
// architecture-specific Contents-* files.
var components = [...]string{"main", "contrib"}
parts := make([][]*contentEntry, len(components))
var sum int
for idx, component := range components {
archs := make([]string, len(release.Architectures))
for idx, arch := range release.Architectures {
archs[idx] = arch.String()
}
part, err := getContents(ar, suite, component, archs, hashByFilename)
if err != nil {
return nil, err
}
parts[idx] = part
sum += len(part)
}
results := make([]*contentEntry, 0, sum)
for _, part := range parts {
results = append(results, part...)
}
return results, nil
}
================================================
FILE: cmd/debiman/getpackages.go
================================================
package main
import (
"bufio"
"bytes"
"encoding/hex"
"fmt"
"io"
"log"
"os"
"strconv"
"strings"
"golang.org/x/sync/errgroup"
"github.com/Debian/debiman/internal/manpage"
"pault.ag/go/archive"
"pault.ag/go/debian/control"
"pault.ag/go/debian/version"
)
type pkgEntry struct {
source string
suite string
binarypkg string
arch string
filename string
version version.Version
sha256 []byte
bytes int64
replaces []string
}
// TODO(later): containsMans could be a map[string]bool, if only all
// Debian packages would ship their manpages in all
// architectures. Example of a package which is doing it wrong:
// “inventor-clients”, which only contains manpages in i386.
//
// In theory, /usr/share must contain the same files across
// architectures: the file-system hierarchy standard (FHS) specifies
// that /usr/share is reserved for architecture independent files, see
// http://www.pathname.com/fhs/pub/fhs-2.3.html#USRSHAREARCHITECTUREINDEPENDENTDATA
// TODO(later): find out which packages are affected and file bugs
func buildContainsMains(content []*contentEntry, links map[string][]link) map[string]map[string]bool {
containsMans := make(map[string]map[string]bool)
for _, entry := range content {
if _, ok := containsMans[entry.binarypkg]; !ok {
containsMans[entry.binarypkg] = make(map[string]bool)
}
containsMans[entry.binarypkg][entry.arch] = true
}
for key := range links {
// key is e.g. “testing/vim-nox”
idx := strings.Index(key, "/")
binarypkg := key[idx+1:]
if containsMans[binarypkg] == nil {
containsMans[binarypkg] = map[string]bool{mostPopularArchitecture: true}
}
}
log.Printf("%d content entries, %d packages\n", len(content), len(containsMans))
return containsMans
}
var emptyVersion version.Version
var (
prefixPackage = []byte("Package")
prefixSource = []byte("Source")
prefixVersion = []byte("Version")
prefixFilename = []byte("Filename")
prefixSize = []byte("Size")
prefixSHA256 = []byte("SHA256")
prefixReplaces = []byte("Replaces")
)
func parsePackageParagraph(scanner *bufio.Scanner, arch string, containsMans map[string]map[string]bool) (pkgEntry, error) {
var entry pkgEntry
for scanner.Scan() {
text := scanner.Bytes()
if len(text) == 0 {
entry = pkgEntry{}
continue
}
idx := bytes.IndexByte(text, ':')
if idx == -1 {
continue
}
key := text[:idx]
if bytes.Equal(key, prefixPackage) {
entry.binarypkg = string(text[idx+2:])
} else if bytes.Equal(key, prefixSource) {
entry.source = string(text[idx+2:])
} else if bytes.Equal(key, prefixVersion) {
v, err := version.Parse(string(text[idx+2:]))
if err != nil {
return entry, err
}
entry.version = v
} else if bytes.Equal(key, prefixFilename) {
entry.filename = string(text[idx+2:])
} else if bytes.Equal(key, prefixSize) {
i, err := strconv.ParseInt(string(text[idx+2:]), 0, 64)
if err != nil {
return entry, err
}
entry.bytes = i
} else if bytes.Equal(key, prefixSHA256) {
h := make([]byte, hex.DecodedLen(len(text[idx+2:])))
n, err := hex.Decode(h, text[idx+2:])
if err != nil {
return entry, err
}
entry.sha256 = h[:n]
} else if bytes.Equal(key, prefixReplaces) {
// e.g. Replaces: systemd (<< 224-2)
pkgs := strings.Split(string(text[idx+2:]), ",")
for _, pkg := range pkgs {
if idx := strings.Index(pkg, " "); idx > -1 {
pkg = pkg[:idx]
}
entry.replaces = append(entry.replaces, pkg)
}
}
if entry.binarypkg != "" &&
entry.version != emptyVersion &&
entry.filename != "" &&
entry.bytes > 0 &&
entry.sha256 != nil {
if !containsMans[entry.binarypkg][arch] {
entry = pkgEntry{}
continue
}
if entry.source == "" {
entry.source = entry.binarypkg
}
idx := strings.Index(entry.source, " ")
if idx > -1 {
entry.source = entry.source[:idx]
}
return entry, nil
}
}
if err := scanner.Err(); err != nil {
return entry, err
}
entry = pkgEntry{}
return entry, io.EOF
}
func less(a, b pkgEntry) bool {
if a.source == b.source {
return a.binarypkg < b.binarypkg
}
return a.source < b.source
}
func done(exhausted []bool) bool {
for idx := range exhausted {
if !exhausted[idx] {
return false
}
}
return true
}
func getPackages(ar *archive.Downloader, rd *archive.ReleaseDownloader, suite string, component string, archs []string, hashByFilename map[string]*control.SHA256FileHash, containsMans map[string]map[string]bool) ([]*pkgEntry, map[string]*manpage.PkgMeta, error) {
files := make([]*os.File, len(archs))
scanners := make([]*bufio.Scanner, len(archs))
pkgs := make([]pkgEntry, len(archs))
advance := make([]bool, len(archs))
exhausted := make([]bool, len(archs))
var eg errgroup.Group
for idx, arch := range archs {
idx := idx // copy
arch := arch // copy
eg.Go(func() error {
// Prefer gzip over xz because gzip uncompresses faster.
path := component + "/binary-" + arch + "/Packages.gz"
fh, ok := hashByFilename[path]
if !ok {
path = component + "/binary-" + arch + "/Packages.xz"
fh, ok = hashByFilename[path]
if !ok {
return fmt.Errorf("ERROR: expected path %q not found in Release file", path)
}
}
log.Printf("getting %q (hash %v)", suite+"/"+path, fh.Hash)
r, err := rd.TempFile(fh.FileHash)
if err != nil {
return err
}
files[idx] = r
scanners[idx] = bufio.NewScanner(r)
// Some packages have excessively large fields, see e.g.:
// https://bugs.debian.org/942487
scanners[idx].Buffer(nil, 512*1024)
advance[idx] = true
return nil
})
}
defer func() {
for _, f := range files {
if f != nil {
os.Remove(f.Name())
f.Close()
}
}
}()
if err := eg.Wait(); err != nil {
return nil, nil, err
}
byVersion := make(map[string]*pkgEntry)
for {
for idx, move := range advance {
if !move {
continue
}
arch := archs[idx]
p, err := parsePackageParagraph(scanners[idx], arch, containsMans)
if err != nil {
if err == io.EOF {
exhausted[idx] = true
} else {
return nil, nil, err
}
}
p.arch = arch
p.suite = suite
pkgs[idx] = p
}
// TODO: unit test for edge cases: can this loop indefinitely or can packages be skipped here?
if done(exhausted) {
break
}
// find the package which is the least advanced in the sort order
lowest := -1
for idx := range archs {
if exhausted[idx] {
continue
}
if lowest == -1 || less(pkgs[idx], pkgs[lowest]) {
lowest = idx
}
}
for idx := range advance {
advance[idx] = !exhausted[idx] && !less(pkgs[lowest], pkgs[idx])
}
// find the best architecture for that package
var newest *pkgEntry
for idx := range archs {
if exhausted[idx] {
continue
}
if less(pkgs[lowest], pkgs[idx]) {
continue
}
if newest == nil || version.Compare(pkgs[idx].version, newest.version) > 0 {
newest = &(pkgs[idx])
}
}
key := suite + "/" + newest.binarypkg
if v, ok := byVersion[key]; ok && version.Compare(v.version, newest.version) > 0 {
continue
}
var best *pkgEntry
for idx, p := range pkgs {
if exhausted[idx] {
continue
}
if less(pkgs[lowest], pkgs[idx]) {
continue
}
if p.version != newest.version {
continue
}
if p.arch == mostPopularArchitecture {
best = &(pkgs[idx])
break
}
}
if best == nil {
for idx, p := range pkgs {
if exhausted[idx] {
continue
}
if less(pkgs[lowest], pkgs[idx]) {
continue
}
if p.version != newest.version {
continue
}
best = &(pkgs[idx])
break
}
}
entry := *best // copy
byVersion[key] = &entry
}
result := make([]*pkgEntry, 0, len(byVersion))
latestVersion := make(map[string]*manpage.PkgMeta, len(byVersion))
for key, p := range byVersion {
result = append(result, p)
latestVersion[key] = &manpage.PkgMeta{
Replaces: p.replaces,
Component: component,
Filename: p.filename,
Sourcepkg: p.source,
Binarypkg: p.binarypkg,
Suite: p.suite,
Version: p.version,
}
}
return result, latestVersion, nil
}
func getAllPackages(ar *archive.Downloader, rd *archive.ReleaseDownloader, suite string, release *archive.Release, hashByFilename map[string]*control.SHA256FileHash, containsMans map[string]map[string]bool) ([]*pkgEntry, map[string]*manpage.PkgMeta, error) {
var components = [...]string{"main", "contrib"}
partsp := make([][]*pkgEntry, len(components))
partsl := make([]map[string]*manpage.PkgMeta, len(components))
latestVersion := make(map[string]*manpage.PkgMeta)
var sum int
for idx, component := range components {
archs := make([]string, len(release.Architectures))
for idx, arch := range release.Architectures {
archs[idx] = arch.String()
}
partp, partl, err := getPackages(ar, rd, suite, component, archs, hashByFilename, containsMans)
if err != nil {
return nil, nil, err
}
partsp[idx] = partp
partsl[idx] = partl
sum += len(partp)
}
results := make([]*pkgEntry, 0, sum)
for idx := range partsp {
results = append(results, partsp[idx]...)
for key, value := range partsl[idx] {
latestVersion[key] = value
}
}
return results, latestVersion, nil
}
================================================
FILE: cmd/debiman/globalview.go
================================================
package main
import (
"compress/gzip"
"encoding/json"
"fmt"
"io/ioutil"
"log"
"os"
"path/filepath"
"strings"
"time"
"golang.org/x/sync/errgroup"
"github.com/Debian/debiman/internal/manpage"
"pault.ag/go/archive"
"pault.ag/go/debian/control"
)
// mostPopularArchitecture is used as preferred architecture when we
// need to pick an arbitrary architecture. The rationale is that
// downloading the package for the most popular architecture has the
// least bad influence on the mirror server’s caches.
const mostPopularArchitecture = "amd64"
type stats struct {
PackagesExtracted uint64
PackagesDeleted uint64
ManpagesRendered uint64
ManpageBytes uint64
HTMLBytes uint64
IndexBytes uint64
}
type link struct {
from string
to string
}
type globalView struct {
// pkgs contains all binary packages we know of.
pkgs []*pkgEntry
// suites contains the Debian suites that we know of. Can either be a codename or a suite,
// depending on the values of -sync_codenames and -sync_suites.
// e.g. “stretch” (codename) or “stable” (suite)
suites map[string]bool
// idxSuites maps codename, suite and command-line argument to suite (as in
// suites).
// e.g. map[oldoldstable:wheezy wheezy:wheezy]
idxSuites map[string]string
// contentByPath maps from paths underneath /usr/share/man to a contentEntry.
contentByPath map[string][]*contentEntry
// xref maps from manpage.Meta.Name (e.g. “w3m” or “systemd.service”) to
// the corresponding manpage.Meta.
xref map[string][]*manpage.Meta
// alternatives maps from Debian binary package to a slice of
// links (from→to pairs).
alternatives map[string][]link
stats *stats
start time.Time
}
type distributionIdentifier int
const (
fromCodename = iota
fromSuite
)
type distribution struct {
name string
identifier distributionIdentifier
}
// distributions returns a list of all distributions (either codenames
// [e.g. wheezy, jessie] or suites [e.g. testing, unstable]) from the
// -sync_codenames and -sync_suites flags.
func distributions(codenames []string, suites []string) []distribution {
distributions := make([]distribution, 0, len(codenames)+len(suites))
for _, e := range codenames {
e = strings.TrimSpace(e)
if e == "" {
continue
}
distributions = append(distributions, distribution{
name: e,
identifier: fromCodename})
}
for _, e := range suites {
e = strings.TrimSpace(e)
if e == "" {
continue
}
distributions = append(distributions, distribution{
name: e,
identifier: fromSuite})
}
return distributions
}
func parseAlternativesFile(fn, prefix string) (map[string][]link, error) {
f, err := os.Open(fn)
if err != nil {
return nil, err
}
defer f.Close()
r, err := gzip.NewReader(f)
if err != nil {
return nil, err
}
defer r.Close()
res := make(map[string][]link)
dec := json.NewDecoder(r)
// read open bracket
if _, err := dec.Token(); err != nil {
return nil, err
}
for dec.More() {
var m struct {
Binpackage string
From string
To string
}
if err := dec.Decode(&m); err != nil {
return nil, err
}
log.Printf("adding from %q to %q to pkg %q", m.From, m.To, m.Binpackage)
key := prefix + "/" + m.Binpackage
res[key] = append(res[key], link{
from: m.From,
to: m.To,
})
}
return res, nil
}
func parseAlternativesDir(dir string) (map[string][]link, error) {
if dir == "" {
return map[string][]link{}, nil
}
infos, err := ioutil.ReadDir(dir)
if err != nil {
return nil, err
}
results := make([]map[string][]link, len(infos))
var eg errgroup.Group
for idx, fi := range infos {
idx, fi := idx, fi // copy
eg.Go(func() error {
suite := strings.TrimSuffix(fi.Name(), ".json.gz")
res, err := parseAlternativesFile(filepath.Join(dir, fi.Name()), suite)
results[idx] = res
return err
})
}
if err := eg.Wait(); err != nil {
return nil, err
}
// Merge all subresults into one map. This is non-destructive
// because the keys are prefixed by the Debian suite, which is
// derived from the filename and hence unique.
merged := make(map[string][]link)
for idx := range infos {
for key, val := range results[idx] {
merged[key] = val
}
}
return merged, nil
}
func markPresent(latestVersion map[string]*manpage.PkgMeta, xref map[string][]*manpage.Meta, filename string, key string) error {
if _, ok := latestVersion[key]; !ok {
return fmt.Errorf("Could not determine latest version")
}
m, err := manpage.FromManPath(strings.TrimPrefix(filename, "usr/share/man/"), latestVersion[key])
if err != nil {
return fmt.Errorf("Trying to interpret path %q: %v", filename, err)
}
// NOTE(stapelberg): this additional verification step
// is necessary because manpages such as the French
// manpage for qelectrotech(1) are present in multiple
// encodings. manpageFromManPath ignores encodings, so
// if we didn’t filter, we would end up with what
// looks like duplicates.
present := false
for _, x := range xref[m.Name] {
if x.ServingPath() == m.ServingPath() {
present = true
break
}
}
if !present {
xref[m.Name] = append(xref[m.Name], m)
}
return nil
}
func buildGlobalView(ar *archive.Downloader, dists []distribution, alternativesDir string, start time.Time) (globalView, error) {
var stats stats
res := globalView{
suites: make(map[string]bool, len(dists)),
idxSuites: make(map[string]string, len(dists)),
contentByPath: make(map[string][]*contentEntry),
xref: make(map[string][]*manpage.Meta),
stats: &stats,
start: start,
}
var err error
res.alternatives, err = parseAlternativesDir(alternativesDir)
if err != nil {
return res, err
}
for _, dist := range dists {
release, rd, err := ar.Release(dist.name)
if err != nil {
return res, err
}
var suite string
if dist.identifier == fromCodename {
suite = release.Codename // e.g. “stretch”
} else {
suite = release.Suite // e.g. “stable”
}
res.suites[suite] = true
res.idxSuites[release.Suite] = suite
res.idxSuites[release.Codename] = suite
res.idxSuites[dist.name] = suite
hashByFilename := make(map[string]*control.SHA256FileHash, len(release.SHA256))
for idx, fh := range release.SHA256 {
// fh.Filename contains e.g. “non-free/source/Sources”
hashByFilename[fh.Filename] = &(release.SHA256[idx])
}
content, err := getAllContents(ar, suite, release, hashByFilename)
if err != nil {
return res, err
}
for _, c := range content {
res.contentByPath[c.filename] = append(res.contentByPath[c.filename], c)
}
var latestVersion map[string]*manpage.PkgMeta
{
// Collect package download work units
var pkgs []*pkgEntry
var err error
pkgs, latestVersion, err = getAllPackages(ar, rd, suite, release, hashByFilename, buildContainsMains(content, res.alternatives))
if err != nil {
return res, err
}
log.Printf("Adding %d packages from suite %q", len(pkgs), suite)
res.pkgs = append(res.pkgs, pkgs...)
}
knownIssues := make(map[string][]error)
// Build a global view of all the manpages (required for cross-referencing).
// TODO(issue): edge case: packages which got renamed between releases
for _, c := range content {
key := c.suite + "/" + c.binarypkg
if err := markPresent(latestVersion, res.xref, c.filename, key); err != nil {
knownIssues[key] = append(knownIssues[key], err)
}
}
for key, links := range res.alternatives {
for _, link := range links {
log.Printf("key=%q, link=%v, latest = %v", key, link, latestVersion[key])
if err := markPresent(latestVersion, res.xref, strings.TrimPrefix(link.from, "/"), key); err != nil {
knownIssues[key] = append(knownIssues[key], err)
}
}
}
for key, errors := range knownIssues {
// TODO: write these to a known-issues file, parse bug numbers from an auxiliary file
log.Printf("package %q has errors: %v", key, errors)
}
}
return res, nil
}
================================================
FILE: cmd/debiman/main.go
================================================
package main
import (
"flag"
"fmt"
"io"
"log"
"net/http"
"os"
"path/filepath"
"strings"
"time"
"golang.org/x/crypto/openpgp"
_ "net/http/pprof"
"github.com/Debian/debiman/internal/bundled"
"github.com/Debian/debiman/internal/commontmpl"
"github.com/Debian/debiman/internal/write"
"pault.ag/go/archive"
)
var (
servingDir = flag.String("serving_dir",
"/srv/man",
"Directory in which to place the manpages which should be served")
indexPath = flag.String("index",
"<serving_dir>/auxserver.idx",
"Path to an auxserver index to generate")
syncCodenames = flag.String("sync_codenames",
"",
"Debian codenames to synchronize (e.g. wheezy, jessie, …)")
syncSuites = flag.String("sync_suites",
"testing",
"Debian suites to synchronize (e.g. testing, unstable)")
onlyRender = flag.String("only_render_pkgs",
"",
"If non-empty, a comma-separated whitelist of packages to render (for developing)")
forceRerender = flag.Bool("force_rerender",
false,
"Forces all manpages to be re-rendered, even if they are up to date")
forceReextract = flag.Bool("force_reextract",
false,
"Forces all manpages to be re-extracted, even if there is no newer package version")
remoteMirror = flag.String("remote_mirror",
"http://localhost:3142/deb.debian.org/",
"URL of a Debian mirror to fetch packages from. localhost:3142 is provided by apt-cacher-ng")
localMirror = flag.String("local_mirror",
"",
"If non-empty, a file system path to a Debian mirror, e.g. /srv/mirrors/debian on DSA-maintained machines")
injectAssets = flag.String("inject_assets",
"",
"If non-empty, a file system path to a directory containing assets to overwrite")
alternativesDir = flag.String("alternatives_dir",
"",
"If non-empty, a directory containing JSON-encoded lists of slave alternative links, named after the suite (e.g. sid.json.gz, testing.json.gz, etc.)")
keyring = flag.String("keyring",
"",
"If non-empty, the specified GPG public keyring will be used for validating archive signatures instead of "+archive.DebianArchiveKeyring)
showVersion = flag.Bool("version",
false,
"Show debiman version and exit")
)
// use go build -ldflags "-X main.debimanVersion=<version>" to set the version
var debimanVersion = "HEAD"
// TODO: handle deleted packages, i.e. packages which are present on
// disk but not in pkgs
// TODO(later): add memory usage estimates to the big structures, set
// parallelism level according to available memory on the system
func logic() error {
start := time.Now()
ar := &archive.Downloader{
Parallel: 10,
MaxTransientRetries: 3,
Mirror: *remoteMirror + "/debian",
LocalMirror: *localMirror,
}
if *keyring != "" {
f, err := os.Open(*keyring)
if err != nil {
return fmt.Errorf("loading -keyring: %v", err)
}
defer f.Close()
ar.Keyring, err = openpgp.ReadKeyRing(f)
if err != nil {
return fmt.Errorf("ReadKeyRing(%s): %v", *keyring, err)
}
}
// Stage 1: all Debian packages of all architectures of the
// specified suites are discovered.
globalView, err := buildGlobalView(ar, distributions(
strings.Split(*syncCodenames, ","),
strings.Split(*syncSuites, ",")),
*alternativesDir,
start)
if err != nil {
return fmt.Errorf("gathering packages: %v", err)
}
log.Printf("gathered packages of all suites, total %d packages", len(globalView.pkgs))
// Stage 2: man pages and auxiliary files (e.g. content fragment
// files which are included by a number of manpages) are extracted
// from the identified Debian packages.
if err := parallelDownload(ar, globalView); err != nil {
return fmt.Errorf("extracting manpages: %v", err)
}
log.Printf("Extracted all manpages, now rendering")
// Stage 3: all man pages are rendered into an HTML representation
// using mandoc(1), directory index files are rendered, contents
// files are rendered.
if err := renderAll(globalView); err != nil {
return fmt.Errorf("rendering manpages: %v", err)
}
log.Printf("Rendered all manpages, writing index")
// Stage 4: write the index only after all rendering is complete,
// otherwise debiman-auxserver might serve redirects to pages
// which cannot be served yet.
path := strings.Replace(*indexPath, "<serving_dir>", *servingDir, -1)
log.Printf("Writing debiman-auxserver index to %q", path)
if err := writeIndex(path, globalView); err != nil {
return fmt.Errorf("writing index: %v", err)
}
if err := renderAux(*servingDir, globalView); err != nil {
return fmt.Errorf("rendering aux files: %v", err)
}
fmt.Printf("total number of packages: %d\n", len(globalView.pkgs))
fmt.Printf("packages extracted: %d\n", globalView.stats.PackagesExtracted)
fmt.Printf("packages deleted: %d\n", globalView.stats.PackagesDeleted)
fmt.Printf("manpages rendered: %d\n", globalView.stats.ManpagesRendered)
fmt.Printf("total manpage bytes: %d\n", globalView.stats.ManpageBytes)
fmt.Printf("total HTML bytes: %d\n", globalView.stats.HTMLBytes)
fmt.Printf("auxserver index bytes: %d\n", globalView.stats.IndexBytes)
fmt.Printf("wall-clock runtime (s): %d\n", int(time.Now().Sub(start).Seconds()))
return write.Atomically(filepath.Join(*servingDir, "metrics.txt"), false, func(w io.Writer) error {
if err := writeMetrics(w, globalView, start); err != nil {
return fmt.Errorf("writing metrics: %v", err)
}
return nil
})
}
func main() {
flag.Parse()
log.SetFlags(log.LstdFlags | log.Lshortfile)
if *showVersion {
fmt.Printf("debiman %s\n", debimanVersion)
return
}
if *injectAssets != "" {
if err := bundled.Inject(*injectAssets); err != nil {
log.Fatal(err)
}
commonTmpls = commontmpl.MustParseCommonTmpls()
contentsTmpl = mustParseContentsTmpl()
pkgindexTmpl = mustParsePkgindexTmpl()
srcpkgindexTmpl = mustParseSrcPkgindexTmpl()
indexTmpl = mustParseIndexTmpl()
faqTmpl = mustParseFaqTmpl()
aboutTmpl = mustParseAboutTmpl()
manpageTmpl = mustParseManpageTmpl()
manpageerrorTmpl = mustParseManpageerrorTmpl()
manpagefooterextraTmpl = mustParseManpagefooterextraTmpl()
}
// All of our .so references are relative to *servingDir. For
// mandoc(1) to find the files, we need to change the working
// directory now.
//
// We turn *servingDir into an absolute path so that it still refers to the
// same location even after our os.Chdir() call (see issue #152).
abs, err := filepath.Abs(*servingDir)
if err != nil {
log.Fatal(err)
}
*servingDir = abs
if err := os.Chdir(*servingDir); err != nil {
log.Fatal(err)
}
go http.ListenAndServe(":4414", nil)
if err := logic(); err != nil {
log.Fatal(err)
}
}
================================================
FILE: cmd/debiman/main_test.go
================================================
package main
import (
"flag"
"io/ioutil"
"os"
"testing"
)
func TestEndToEnd(t *testing.T) {
dir, err := ioutil.TempDir("", "debiman")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(dir)
flag.Set("serving_dir", dir)
flag.Set("local_mirror", "../../testdata/tinymirror")
if err := logic(); err != nil {
t.Fatal(err)
}
}
================================================
FILE: cmd/debiman/mtime.go
================================================
//go:build !linux
// +build !linux
package main
import "time"
func maybeSetLinkMtime(destPath string, t time.Time) error {
return nil
}
================================================
FILE: cmd/debiman/mtime_linux.go
================================================
//go:build linux
// +build linux
package main
import (
"os"
"path/filepath"
"time"
"golang.org/x/sys/unix"
)
func maybeSetLinkMtime(destPath string, t time.Time) error {
ts := unix.NsecToTimespec(t.UnixNano())
dir, err := os.Open(filepath.Dir(destPath))
if err != nil {
return err
}
defer dir.Close()
return unix.UtimesNanoAt(int(dir.Fd()), destPath, []unix.Timespec{ts, ts}, unix.AT_SYMLINK_NOFOLLOW)
}
================================================
FILE: cmd/debiman/prometheus.go
================================================
package main
import (
"html/template"
"io"
"time"
)
const metricsTmplContent = `
# HELP packages_total The total number of Debian binary packages processed.
# TYPE packages_total gauge
packages_total {{ .Packages }}
# HELP packages_extracted Number of Debian binary packages from which manpages were extracted.
# TYPE packages_extracted gauge
packages_extracted {{ .Stats.PackagesExtracted }}
# HELP packages_deleted Number of Debian binary packages deleted because they were no longer present.
# TYPE packages_deleted gauge
packages_deleted {{ .Stats.PackagesDeleted }}
# HELP manpages_rendered Number of manpages rendered to HTML
# TYPE manpages_rendered gauge
manpages_rendered {{ .Stats.ManpagesRendered }}
# HELP manpage_bytes Total number of bytes used by manpages (by format).
# TYPE manpage_bytes gauge
manpage_bytes{format="man"} {{ .Stats.ManpageBytes }}
manpage_bytes{format="html"} {{ .Stats.HTMLBytes }}
# HELP index_bytes Total number of bytes used for the auxserver index.
# TYPE index_bytes gauge
index_bytes {{ .Stats.IndexBytes }}
# HELP runtime Wall-clock runtime in seconds.
# TYPE runtime gauge
runtime {{ .Seconds }}
# HELP last_successful_run Last successful run in seconds since the epoch.
# TYPE last_successful_run gauge
last_successful_run {{ .LastSuccessfulRun }}
`
var metricsTmpl = template.Must(template.New("metrics").Parse(metricsTmplContent))
func writeMetrics(w io.Writer, gv globalView, start time.Time) error {
now := time.Now()
return metricsTmpl.Execute(w, struct {
Packages int
Stats *stats
Now time.Time
Seconds int
LastSuccessfulRun int64
}{
Packages: len(gv.pkgs),
Stats: gv.stats,
Now: now,
Seconds: int(now.Sub(start).Seconds()),
LastSuccessfulRun: now.Unix(),
})
}
================================================
FILE: cmd/debiman/render.go
================================================
package main
import (
"compress/gzip"
"encoding/json"
"flag"
"fmt"
"html/template"
"io"
"io/ioutil"
"log"
"os"
"path/filepath"
"sort"
"strings"
"sync"
"sync/atomic"
"syscall"
"time"
"github.com/Debian/debiman/internal/commontmpl"
"github.com/Debian/debiman/internal/convert"
"github.com/Debian/debiman/internal/manpage"
"github.com/Debian/debiman/internal/sitemap"
"github.com/Debian/debiman/internal/write"
"golang.org/x/net/context"
"golang.org/x/sync/errgroup"
)
var (
manwalkConcurrency = flag.Int("concurrency_manwalk",
1000, // below the default 1024 open file descriptor limit
"Concurrency level for walking through binary package man directories (ulimit -n must be higher!)")
renderConcurrency = flag.Int("concurrency_render",
5,
"Concurrency level for rendering manpages using mandoc")
gzipLevel = flag.Int("gzip",
9,
"gzip compression level to use for compressing HTML versions of manpages. defaults to 9 to keep network traffic minimal, but useful to reduce for development/disaster recovery (level 1 results in a 2x speedup!)")
baseURL = flag.String("base_url",
"https://manpages.debian.org",
"Base URL (without trailing slash) to the site. Used where absolute URLs are required, e.g. sitemaps.")
)
type breadcrumb struct {
Link string
Text string
}
type breadcrumbs []breadcrumb
func (b breadcrumbs) ToJSON() template.HTML {
type item struct {
Type string `json:"@type"`
ID string `json:"@id"`
Name string `json:"name"`
}
type listItem struct {
Type string `json:"@type"`
Position int `json:"position"`
Item item `json:"item"`
}
type breadcrumbList struct {
Context string `json:"@context"`
Type string `json:"@type"`
Elements []listItem `json:"itemListElement"`
}
l := breadcrumbList{
Context: "http://schema.org",
Type: "BreadcrumbList",
Elements: make([]listItem, len(b)),
}
for idx, br := range b {
l.Elements[idx] = listItem{
Type: "ListItem",
Position: idx + 1,
Item: item{
Type: "Thing",
ID: br.Link,
Name: br.Text,
},
}
}
jsonb, err := json.Marshal(l)
if err != nil {
log.Fatal(err)
}
return template.HTML(jsonb)
}
var commonTmpls = commontmpl.MustParseCommonTmpls()
type renderingMode int
const (
regularFiles renderingMode = iota
symlinks
)
// listManpages lists all files in dir (non-recursively) and returns a map from
// filename (within dir) to *manpage.Meta.
func listManpages(dir string) (map[string]*manpage.Meta, error) {
manpageByName := make(map[string]*manpage.Meta)
files, err := os.Open(dir)
if err != nil {
return nil, err
}
defer files.Close()
var predictedEOF bool
for {
if predictedEOF {
break
}
names, err := files.Readdirnames(2048)
if err != nil {
if err == io.EOF {
break
} else {
// We avoid an additional stat syscalls for each
// binary package directory by just optimistically
// calling readdir and handling the ENOTDIR error.
if sce, ok := err.(*os.SyscallError); ok && sce.Err == syscall.ENOTDIR {
return nil, nil
}
return nil, err
}
}
// When len(names) < 2048 the next Readdirnames() call will
// result in io.EOF and can be skipped to reduce getdents(2)
// syscalls by half.
predictedEOF = len(names) < 2048
for _, fn := range names {
if !strings.HasSuffix(fn, ".gz") ||
strings.HasSuffix(fn, ".html.gz") {
continue
}
full := filepath.Join(dir, fn)
m, err := manpage.FromServingPath(*servingDir, full)
if err != nil {
// If we run into this case, our code cannot correctly
// interpret the result of ServingPath().
log.Printf("BUG: cannot parse manpage from serving path %q: %v", full, err)
continue
}
manpageByName[fn] = m
}
}
return manpageByName, nil
}
func renderDirectoryIndex(dir string, newestModTime time.Time) error {
st, err := os.Stat(filepath.Join(dir, "index.html.gz"))
if !*forceRerender && err == nil && st.ModTime().After(newestModTime) {
return nil
}
manpageByName, err := listManpages(dir)
if err != nil {
return err
}
if len(manpageByName) == 0 {
log.Printf("WARNING: empty directory %q, not generating package index", dir)
return nil
}
return renderPkgindex(filepath.Join(dir, "index.html.gz"), manpageByName)
}
// walkManContents walks over all entries in dir and, depending on mode, does:
// 1. send a renderJob for each regular file
// 2. send a renderJob for each symlink
func walkManContents(ctx context.Context, renderChan chan<- renderJob, dir string, mode renderingMode, gv globalView, newestModTime time.Time) (time.Time, error) {
// the invariant is: each file ending in .gz must have a corresponding .html.gz file
// the .html.gz must have a modtime that is >= the modtime of the .gz file
files, err := os.Open(dir)
if err != nil {
return newestModTime, err
}
defer files.Close()
var predictedEOF bool
for {
if predictedEOF {
break
}
names, err := files.Readdirnames(2048)
if err != nil {
if err == io.EOF {
break
} else {
// We avoid an additional stat syscalls for each
// binary package directory by just optimistically
// calling readdir and handling the ENOTDIR error.
if sce, ok := err.(*os.SyscallError); ok && sce.Err == syscall.ENOTDIR {
return newestModTime, nil
}
return newestModTime, err
}
}
// When len(names) < 2048 the next Readdirnames() call will
// result in io.EOF and can be skipped to reduce getdents(2)
// syscalls by half.
predictedEOF = len(names) < 2048
for _, fn := range names {
if !strings.HasSuffix(fn, ".gz") ||
strings.HasSuffix(fn, ".html.gz") {
continue
}
full := filepath.Join(dir, fn)
st, err := os.Lstat(full)
if err != nil {
continue
}
if st.ModTime().After(newestModTime) {
newestModTime = st.ModTime()
}
symlink := st.Mode()&os.ModeSymlink != 0
if !symlink {
atomic.AddUint64(&gv.stats.ManpageBytes, uint64(st.Size()))
}
if mode == regularFiles && symlink ||
mode == symlinks && !symlink {
continue
}
n := strings.TrimSuffix(fn, ".gz") + ".html.gz"
htmlst, err := os.Stat(filepath.Join(dir, n))
if err == nil {
atomic.AddUint64(&gv.stats.HTMLBytes, uint64(htmlst.Size()))
}
if err != nil || *forceRerender || htmlst.ModTime().Before(st.ModTime()) {
m, err := manpage.FromServingPath(*servingDir, full)
if err != nil {
// If we run into this case, our code cannot correctly
// interpret the result of ServingPath().
log.Printf("BUG: cannot parse manpage from serving path %q: %v", full, err)
continue
}
versions := gv.xref[m.Name]
// Replace m with its corresponding entry in versions
// so that rendermanpage() can use pointer equality to
// efficiently skip entries.
for _, v := range versions {
if v.ServingPath() == m.ServingPath() {
m = v
break
}
}
// Render dependent manpages first to properly resume
// in case debiman is interrupted.
for _, v := range versions {
if v == m || *forceRerender {
continue
}
vfull := filepath.Join(*servingDir, v.RawPath())
vfn := filepath.Join(*servingDir, v.ServingPath()+".html.gz")
vhtmlst, err := os.Stat(vfn)
if err == nil && vhtmlst.ModTime().After(gv.start) {
// The variant was already re-rendered with this globalView.
continue
}
vst, err := os.Stat(vfull)
if err != nil {
log.Printf("WARNING: stat %q: %v", vfull, err)
continue
}
vreuse := ""
if vhtmlst != nil && vhtmlst.ModTime().After(vst.ModTime()) {
vreuse = vfn
}
log.Printf("%s invalidated by %s", vfn, full)
select {
case renderChan <- renderJob{
dest: vfn,
src: vfull,
meta: v,
versions: versions,
xref: gv.xref,
modTime: vst.ModTime(),
reuse: vreuse,
}:
case <-ctx.Done():
break
}
}
var reuse string
if symlink {
link, err := os.Readlink(full)
if err == nil {
resolved := filepath.Join(dir, link)
reuse = strings.TrimSuffix(resolved, ".gz") + ".html.gz"
}
}
select {
case renderChan <- renderJob{
dest: filepath.Join(dir, n),
src: full,
meta: m,
versions: versions,
xref: gv.xref,
modTime: st.ModTime(),
reuse: reuse,
}:
case <-ctx.Done():
break
}
}
}
}
return newestModTime, nil
}
func walkContents(ctx context.Context, renderChan chan<- renderJob, whitelist map[string]bool, gv globalView) error {
sitemaps := make(map[string]time.Time)
suitedirs, err := ioutil.ReadDir(*servingDir)
if err != nil {
return err
}
for _, sfi := range suitedirs {
if !sfi.IsDir() {
continue
}
if !gv.suites[sfi.Name()] {
continue
}
bins, err := os.Open(filepath.Join(*servingDir, sfi.Name()))
if err != nil {
return err
}
defer bins.Close()
// 20000 is the order of magnitude of binary packages
// (containing manpages) in any given Debian suite, so that is
// a good value to start with.
sitemapEntries := make(map[string]time.Time, 20000)
var sitemapEntriesMu sync.RWMutex
for {
names, err := bins.Readdirnames(*manwalkConcurrency)
if err != nil {
if err == io.EOF {
break
} else {
return err
}
}
var wg errgroup.Group
for _, bfn := range names {
if whitelist != nil && !whitelist[bfn] {
continue
}
if bfn == "sourcesWithManpages.txt.gz" ||
bfn == "index.html.gz" ||
bfn == "sitemap.xml.gz" ||
bfn == ".nobackup" {
continue
}
bfn := bfn // copy
dir := filepath.Join(*servingDir, sfi.Name(), bfn)
wg.Go(func() error {
// Iterating through the same directory in all
// modes increases the chance for the dirents to
// still be cached. This is important for machines
// like manziarly.debian.org, which do not have
// enough RAM to keep all dirents cached over the
// runtime of this code path.
var newestModTime time.Time
var err error
// Render all regular files first
newestModTime, err = walkManContents(ctx, renderChan, dir, regularFiles, gv, newestModTime)
if err != nil {
return err
}
// then render all symlinks, re-using the rendered fragments
newestModTime, err = walkManContents(ctx, renderChan, dir, symlinks, gv, newestModTime)
if err != nil {
return err
}
// and finally render the package index files which need to
// consider both regular files and symlinks.
if err := renderDirectoryIndex(dir, newestModTime); err != nil {
return err
}
if !newestModTime.IsZero() {
sitemapEntriesMu.Lock()
defer sitemapEntriesMu.Unlock()
sitemapEntries[bfn] = newestModTime
}
return nil
})
}
if err := wg.Wait(); err != nil {
return err
}
}
bins.Close()
sitemapPath := filepath.Join(*servingDir, sfi.Name(), "sitemap.xml.gz")
if err := write.Atomically(sitemapPath, true, func(w io.Writer) error {
return sitemap.WriteTo(w, *baseURL+"/"+sfi.Name(), sitemapEntries)
}); err != nil {
return err
}
st, err := os.Stat(sitemapPath)
if err == nil {
sitemaps[sfi.Name()] = st.ModTime()
}
}
return write.Atomically(filepath.Join(*servingDir, "sitemapindex.xml.gz"), true, func(w io.Writer) error {
return sitemap.WriteIndexTo(w, *baseURL, sitemaps)
})
}
func writeSourceIndex(gv globalView, newestForSource map[string]time.Time) error {
// Partition by suite for reduced memory usage and better locality of file
// system access
for suite := range gv.suites {
binariesBySource := make(map[string][]string)
for _, p := range gv.pkgs {
if p.suite != suite {
continue
}
binariesBySource[p.source] = append(binariesBySource[p.source], p.binarypkg)
}
for src, binaries := range binariesBySource {
srcDir := filepath.Join(*servingDir, suite, "src:"+src)
// skip if current index file is more recent than newestForSource
st, err := os.Stat(filepath.Join(srcDir, "index.html.gz"))
if !*forceRerender && err == nil && st.ModTime().After(newestForSource[src]) {
continue
}
// Aggregate manpages of all binary packages for this source package
manpages := make(map[string]*manpage.Meta)
for _, binary := range binaries {
m, err := listManpages(filepath.Join(*servingDir, suite, binary))
if err != nil {
if os.IsNotExist(err) {
continue // The package might not contain any manpages.
}
return err
}
for k, v := range m {
manpages[k] = v
}
}
if len(manpages) == 0 {
continue // The entire source package does not contain any manpages.
}
if err := os.MkdirAll(srcDir, 0755); err != nil {
return err
}
if err := renderSrcPkgindex(filepath.Join(srcDir, "index.html.gz"), src, manpages); err != nil {
return err
}
}
}
return nil
}
func writeSourcesWithManpages(gv globalView) error {
for suite := range gv.suites {
hasManpages := make(map[string]bool)
for _, p := range gv.pkgs {
if p.suite != suite {
continue
}
hasManpages[p.source] = true
}
sourcesWithManpages := make([]string, 0, len(hasManpages))
for source := range hasManpages {
sourcesWithManpages = append(sourcesWithManpages, source)
}
sort.Strings(sourcesWithManpages)
dest := filepath.Join(*servingDir, suite, "sourcesWithManpages.txt.gz")
if err := os.MkdirAll(filepath.Dir(dest), 0755); err != nil {
return err
}
if err := write.Atomically(dest, true, func(w io.Writer) error {
for _, source := range sourcesWithManpages {
if _, err := fmt.Fprintln(w, source); err != nil {
return err
}
}
return nil
}); err != nil {
return err
}
}
return nil
}
func renderAll(gv globalView) error {
log.Printf("Preparing inverted maps")
sourceByBinary := make(map[string]string, len(gv.pkgs))
newestForSource := make(map[string]time.Time)
for _, p := range gv.pkgs {
sourceByBinary[p.suite+"/"+p.binarypkg] = p.source
newestForSource[p.source] = time.Time{}
}
log.Printf("%d sourceByBinary entries, %d newestForSource entries", len(sourceByBinary), len(newestForSource))
eg, ctx := errgroup.WithContext(context.Background())
renderChan := make(chan renderJob)
for i := 0; i < *renderConcurrency; i++ {
eg.Go(func() error {
converter, err := convert.NewProcess()
if err != nil {
return err
}
defer converter.Kill()
// NOTE(stapelberg): gzip’s decompression phase takes the same
// time, regardless of compression level. Hence, we invest the
// maximum CPU time once to achieve the best compression.
gzipw, err := gzip.NewWriterLevel(nil, *gzipLevel)
if err != nil {
return err
}
for r := range renderChan {
n, err := rendermanpage(gzipw, converter, r)
if err != nil {
// rendermanpage writes an error page if rendering
// failed, any returned error is severe (e.g. file
// system full) and should lead to termination.
return err
}
atomic.AddUint64(&gv.stats.HTMLBytes, n)
atomic.AddUint64(&gv.stats.ManpagesRendered, 1)
}
return nil
})
}
var whitelist map[string]bool
if *onlyRender != "" {
whitelist = make(map[string]bool)
log.Printf("Restricting rendering to the following binary packages:")
for _, e := range strings.Split(strings.TrimSpace(*onlyRender), ",") {
whitelist[e] = true
log.Printf(" %q", e)
}
log.Printf("(total: %d whitelist entries)", len(whitelist))
}
if err := walkContents(ctx, renderChan, whitelist, gv); err != nil {
return err
}
close(renderChan)
if err := eg.Wait(); err != nil {
return err
}
if err := writeSourceIndex(gv, newestForSource); err != nil {
return fmt.Errorf("writing source index: %v", err)
}
if err := writeSourcesWithManpages(gv); err != nil {
return fmt.Errorf("writing sourcesWithManpages: %v", err)
}
suitedirs, err := ioutil.ReadDir(*servingDir)
if err != nil {
return err
}
for _, sfi := range suitedirs {
if !sfi.IsDir() {
continue
}
if !gv.suites[sfi.Name()] {
continue
}
bins, err := os.Open(filepath.Join(*servingDir, sfi.Name()))
if err != nil {
return err
}
defer bins.Close()
names, err := bins.Readdirnames(-1)
if err != nil {
return err
}
if err := renderContents(filepath.Join(*servingDir, fmt.Sprintf("contents-%s.html.gz", sfi.Name())), sfi.Name(), names); err != nil {
return err
}
bins.Close()
}
return nil
}
================================================
FILE: cmd/debiman/render_test.go
================================================
package main
import (
"bytes"
"fmt"
"strings"
"testing"
"github.com/Debian/debiman/internal/manpage"
)
func TestBreadcrumbsToJSON(t *testing.T) {
const breadcrumbsJSON = `{"@context":"http://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"item":{"@type":"Thing","@id":"/contents-jessie.html","name":"jessie"}},{"@type":"ListItem","position":2,"item":{"@type":"Thing","@id":"/jessie/i3-wm/index.html","name":"i3-wm"}},{"@type":"ListItem","position":3,"item":{"@type":"Thing","@id":"","name":"i3(1)"}}]}`
const Suite = "jessie"
const Binarypkg = "i3-wm"
b := breadcrumbs{
{fmt.Sprintf("/contents-%s.html", Suite), Suite},
{fmt.Sprintf("/%s/%s/index.html", Suite, Binarypkg), Binarypkg},
{"", "i3(1)"},
}
if got, want := string(b.ToJSON()), breadcrumbsJSON; got != want {
fmt.Printf("%s\n", got)
t.Fatalf("unexpected breadcrumbs JSON: got %q, want %q", got, want)
}
}
// Ensure that section names containing unsafe characters like colons
// are properly handled (and do not result in ZgotmplZ values) on pages
// like https://manpages.debian.org/trixie/foot/foot.ini.5.en.html
func TestFragmentLinkWithColon(t *testing.T) {
var buf bytes.Buffer
err := manpageTmpl.ExecuteTemplate(&buf, "manpage", manpagePrepData{
Meta: &manpage.Meta{
Name: "test",
Section: "1",
Package: &manpage.PkgMeta{
Suite: "testing",
Binarypkg: "test",
},
},
TOC: []string{"SECTION: main"},
})
if err != nil {
t.Fatal(err)
}
if strings.Contains(buf.String(), "ZgotmplZ") {
t.Fatal("ZgotmplZ in output")
}
}
================================================
FILE: cmd/debiman/renderaux.go
================================================
package main
import (
"fmt"
"html/template"
"io"
"path/filepath"
"sort"
"strings"
"github.com/Debian/debiman/internal/bundled"
"github.com/Debian/debiman/internal/manpage"
"github.com/Debian/debiman/internal/write"
)
var indexTmpl = mustParseIndexTmpl()
func mustParseIndexTmpl() *template.Template {
return template.Must(template.Must(commonTmpls.Clone()).New("index").Parse(bundled.Asset("index.tmpl")))
}
var faqTmpl = mustParseFaqTmpl()
func mustParseFaqTmpl() *template.Template {
return template.Must(template.Must(commonTmpls.Clone()).New("faq").Parse(bundled.Asset("faq.tmpl")))
}
var aboutTmpl = mustParseAboutTmpl()
func mustParseAboutTmpl() *template.Template {
return template.Must(template.Must(commonTmpls.Clone()).New("about").Parse(bundled.Asset("about.tmpl")))
}
type bySuiteStr []string
func (p bySuiteStr) Len() int { return len(p) }
func (p bySuiteStr) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
func (p bySuiteStr) Less(i, j int) bool {
orderi, oki := sortOrder[p[i]]
orderj, okj := sortOrder[p[j]]
if !oki || !okj {
panic(fmt.Sprintf("either %q or %q is an unknown suite. known: %+v", p[i], p[j], sortOrder))
}
return orderi < orderj
}
func renderAux(destDir string, gv globalView) error {
suites := make([]string, 0, len(gv.suites))
for suite := range gv.suites {
suites = append(suites, suite)
}
sort.Stable(bySuiteStr(suites))
if err := write.Atomically(filepath.Join(destDir, "index.html.gz"), true, func(w io.Writer) error {
return indexTmpl.Execute(w, struct {
Title string
DebimanVersion string
Breadcrumbs breadcrumbs
FooterExtra string
Suites []string
Meta *manpage.Meta
HrefLangs []*manpage.Meta
}{
Title: "index",
Suites: suites,
DebimanVersion: debimanVersion,
})
}); err != nil {
return err
}
if err := write.Atomically(filepath.Join(destDir, "faq.html.gz"), true, func(w io.Writer) error {
return faqTmpl.Execute(w, struct {
Title string
DebimanVersion string
Breadcrumbs breadcrumbs
FooterExtra string
Meta *manpage.Meta
HrefLangs []*manpage.Meta
}{
Title: "FAQ",
DebimanVersion: debimanVersion,
})
}); err != nil {
return err
}
if err := write.Atomically(filepath.Join(destDir, "about.html.gz"), true, func(w io.Writer) error {
return aboutTmpl.Execute(w, struct {
Title string
DebimanVersion string
Breadcrumbs breadcrumbs
FooterExtra string
Meta *manpage.Meta
HrefLangs []*manpage.Meta
}{
Title: "About",
DebimanVersion: debimanVersion,
})
}); err != nil {
return err
}
for name, content := range bundled.AssetsFiltered(func(fn string) bool {
return !strings.HasSuffix(fn, ".tmpl") && !strings.HasSuffix(fn, "style.css")
}) {
if err := write.Atomically(filepath.Join(destDir, filepath.Base(name)+".gz"), true, func(w io.Writer) error {
_, err := io.WriteString(w, content)
return err
}); err != nil {
return err
}
if err := write.Atomically(filepath.Join(destDir, filepath.Base(name)), false, func(w io.Writer) error {
_, err := io.WriteString(w, content)
return err
}); err != nil {
return err
}
}
return nil
}
================================================
FILE: cmd/debiman/rendercontents.go
================================================
package main
import (
"fmt"
"html/template"
"io"
"os"
"path/filepath"
"sort"
"github.com/Debian/debiman/internal/bundled"
"github.com/Debian/debiman/internal/manpage"
"github.com/Debian/debiman/internal/write"
)
var contentsTmpl = mustParseContentsTmpl()
func mustParseContentsTmpl() *template.Template {
return template.Must(template.Must(commonTmpls.Clone()).New("contents").Parse(bundled.Asset("contents.tmpl")))
}
func renderContents(dest, suite string, bins []string) error {
sort.Strings(bins)
if err := write.Atomically(dest, true, func(w io.Writer) error {
return contentsTmpl.Execute(w, struct {
Title string
DebimanVersion string
Breadcrumbs breadcrumbs
FooterExtra string
Bins []string
Suite string
Meta *manpage.Meta
HrefLangs []*manpage.Meta
}{
Title: fmt.Sprintf("Contents of Debian %s", suite),
DebimanVersion: debimanVersion,
Breadcrumbs: breadcrumbs{
{fmt.Sprintf("/contents-%s.html", suite), suite},
{"", "Contents"},
},
Bins: bins,
Suite: suite,
})
}); err != nil {
return err
}
destPath := filepath.Join(*servingDir, suite, "index.html.gz")
link := fmt.Sprintf("../contents-%s.html.gz", suite)
if err := os.Symlink(link, destPath); err != nil && !os.IsExist(err) {
return err
}
return nil
}
================================================
FILE: cmd/debiman/rendermanpage.go
================================================
package main
import (
"bytes"
"compress/gzip"
"errors"
"fmt"
"html/template"
"io"
"log"
"net/url"
"os"
"path/filepath"
"sort"
"strings"
"time"
"github.com/Debian/debiman/internal/bundled"
"github.com/Debian/debiman/internal/commontmpl"
"github.com/Debian/debiman/internal/convert"
"github.com/Debian/debiman/internal/manpage"
"github.com/Debian/debiman/internal/write"
"golang.org/x/text/language"
)
const iso8601Format = "2006-01-02T15:04:05Z"
// TODO(later): move this list to a package within pault.ag/debian/?
var releaseList = []string{
"buzz",
"rex",
"bo",
"hamm",
"slink",
"potato",
"woody",
"sarge",
"etch",
"lenny",
"squeeze",
"wheezy",
"wheezy-backports",
"jessie",
"jessie-backports",
"stretch",
"stretch-backports",
"buster",
"buster-backports",
"bullseye",
"bullseye-backports",
"bookworm",
"bookworm-backports",
"trixie",
"trixie-backports",
"forky",
"forky-backports",
}
var sortOrder = make(map[string]int)
func init() {
for idx, r := range releaseList {
sortOrder[r] = idx
}
sortOrder["testing"] = sortOrder["forky"]
sortOrder["unstable"] = len(releaseList)
sortOrder["experimental"] = sortOrder["unstable"] + 1
}
// stapelberg came up with the following abbreviations:
var shortSections = map[string]string{
"1": "progs",
"2": "syscalls",
"3": "libfuncs",
"4": "files",
"5": "formats",
"6": "games",
"7": "misc",
"8": "sysadmin",
"9": "kernel",
}
// taken from man(1)
var longSections = map[string]string{
"1": "Executable programs or shell commands",
"2": "System calls (functions provided by the kernel)",
"3": "Library calls (functions within program libraries)",
"4": "Special files (usually found in /dev)",
"5": "File formats and conventions eg /etc/passwd",
"6": "Games",
"7": "Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7)",
"8": "System administration commands (usually only for root)",
"9": "Kernel routines [Non standard]",
}
var manpageTmpl = mustParseManpageTmpl()
func mustParseManpageTmpl() *template.Template {
return template.Must(template.Must(commonTmpls.Clone()).New("manpage").
Funcs(map[string]interface{}{
"ShortSection": func(section string) string {
return shortSections[section]
},
"LongSection": func(section string) string {
return longSections[section]
},
"FragmentLink": func(fragment string) template.URL {
u := url.URL{Fragment: strings.Replace(fragment, " ", "_", -1)}
return template.URL(u.String())
},
}).
Parse(bundled.Asset("manpage.tmpl")))
}
var manpageerrorTmpl = mustParseManpageerrorTmpl()
func mustParseManpageerrorTmpl() *template.Template {
return template.Must(template.Must(commonTmpls.Clone()).New("manpage-error").
Funcs(map[string]interface{}{
"ShortSection": func(section string) string {
return shortSections[section]
},
"LongSection": func(section string) string {
return longSections[section]
},
"FragmentLink": func(fragment string) string {
u := url.URL{Fragment: strings.Replace(fragment, " ", "_", -1)}
return u.String()
},
}).
Parse(bundled.Asset("manpageerror.tmpl")))
}
var manpagefooterextraTmpl = mustParseManpagefooterextraTmpl()
func mustParseManpagefooterextraTmpl() *template.Template {
return template.Must(template.Must(commonTmpls.Clone()).New("manpage-footerextra").
Funcs(map[string]interface{}{
"Iso8601": func(t time.Time) string {
return t.UTC().Format(iso8601Format)
},
}).
Parse(bundled.Asset("manpagefooterextra.tmpl")))
}
func convertFile(converter *convert.Process, src string, resolve func(ref string) string) (doc string, toc []string, err error) {
f, err := os.Open(src)
if err != nil {
return "", nil, err
}
defer f.Close()
r, err := gzip.NewReader(f)
if err != nil {
if err == io.EOF {
// TODO: better representation of an empty manpage
return "This space intentionally left blank.", nil, nil
}
return "", nil, err
}
defer r.Close()
out, toc, err := converter.ToHTML(r, resolve)
if err != nil {
return "", nil, fmt.Errorf("convert(%q): %v", src, err)
}
return out, toc, nil
}
type byPkgAndLanguage struct {
opts []*manpage.Meta
currentpkg string
}
func (p byPkgAndLanguage) Len() int { return len(p.opts) }
func (p byPkgAndLanguage) Swap(i, j int) { p.opts[i], p.opts[j] = p.opts[j], p.opts[i] }
func (p byPkgAndLanguage) Less(i, j int) bool {
// prefer manpages from the same package
if p.opts[i].Package.Binarypkg != p.opts[j].Package.Binarypkg {
if p.opts[i].Package.Binarypkg == p.currentpkg {
return true
}
}
return p.opts[i].Language < p.opts[j].Language
}
// bestLanguageMatch returns the best manpage out of options (coming
// from current) based on text/language’s matching.
func bestLanguageMatch(current *manpage.Meta, options []*manpage.Meta) *manpage.Meta {
sort.Stable(byPkgAndLanguage{options, current.Package.Binarypkg})
if options[0].Language != "en" {
for i := 1; i < len(options); i++ {
if options[i].Language == "en" {
options = append([]*manpage.Meta{options[i]}, options...)
break
}
}
}
tags := make([]language.Tag, len(options))
for idx, m := range options {
tags[idx] = m.LanguageTag
}
// NOTE(stapelberg): it would be even better to match on the
// user’s Accept-Language HTTP header here, but that is
// incompatible with the processing model of pre-generating
// all manpages.
// TODO(stapelberg): to fix the above, we could have
// client-side javascript which queries the redirector and
// improves cross-references.
matcher := language.NewMatcher(tags)
_, idx, _ := matcher.Match(current.LanguageTag)
return options[idx]
}
type byLanguage []*manpage.Meta
func (p byLanguage) Len() int { return len(p) }
func (p byLanguage) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
func (p byLanguage) Less(i, j int) bool { return p[i].Language < p[j].Language }
type renderJob struct {
dest string
src string
meta *manpage.Meta
versions []*manpage.Meta
xref map[string][]*manpage.Meta
modTime time.Time
reuse string
}
var notYetRenderedSentinel = errors.New("Not yet rendered")
type manpagePrepData struct {
Title string
DebimanVersion string
Breadcrumbs breadcrumbs
FooterExtra template.HTML
Suites []*manpage.Meta
Versions []*manpage.Meta
Sections []*manpage.Meta
Bins []*manpage.Meta
Langs []*manpage.Meta
HrefLangs []*manpage.Meta
Meta *manpage.Meta
TOC []string
Ambiguous map[*manpage.Meta]bool
Content template.HTML
Error error
}
type bySuite []*manpage.Meta
func (p bySuite) Len() int { return len(p) }
func (p bySuite) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
func (p bySuite) Less(i, j int) bool {
orderi, oki := sortOrder[p[i].Package.Suite]
orderj, okj := sortOrder[p[j].Package.Suite]
if !oki || !okj {
panic(fmt.Sprintf("either %q or %q is an unknown suite. known: %+v", p[i].Package.Suite, p[j].Package.Suite, sortOrder))
}
return orderi < orderj
}
type byMainSection []*manpage.Meta
func (p byMainSection) Len() int { return len(p) }
func (p byMainSection) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
func (p byMainSection) Less(i, j int) bool { return p[i].MainSection() < p[j].MainSection() }
type byBinarypkg []*manpage.Meta
func (p byBinarypkg) Len() int { return len(p) }
func (p byBinarypkg) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
func (p byBinarypkg) Less(i, j int) bool { return p[i].Package.Binarypkg < p[j].Package.Binarypkg }
func rendermanpageprep(converter *convert.Process, job renderJob) (*template.Template, manpagePrepData, error) {
meta := job.meta // for convenience
// TODO(issue): document fundamental limitation: “other languages” is imprecise: e.g. crontab(1) — are the languages for package:systemd-cron or for package:cron?
// TODO(later): to boost confidence in detecting cross-references, can we add to testdata the entire list of man page names from debian to have a good test?
// TODO(later): add plain-text version
var (
content string
toc []string
renderErr = notYetRenderedSentinel
)
if job.reuse != "" {
content, toc, renderErr = reuse(job.reuse)
if renderErr != nil {
log.Printf("WARNING: re-using %q failed: %v", job.reuse, renderErr)
}
}
if renderErr != nil {
content, toc, renderErr = convertFile(converter, job.src, func(ref string) string {
idx := strings.LastIndex(ref, "(")
if idx == -1 {
return ""
}
section := ref[idx+1 : len(ref)-1]
name := ref[:idx]
related, ok := job.xref[name]
if !ok {
return ""
}
filtered := make([]*manpage.Meta, 0, len(related))
for _, r := range related {
if r.MainSection() != section {
continue
}
if r.Package.Suite != meta.Package.Suite {
continue
}
filtered = append(filtered, r)
}
if len(filtered) == 0 {
return ""
}
return commontmpl.BaseURLPath() + "/" + bestLanguageMatch(meta, filtered).ServingPath() + ".html"
})
}
log.Printf("rendering %q", job.dest)
suites := make([]*manpage.Meta, 0, len(job.versions))
for _, v := range job.versions {
if !v.Package.SameBinary(meta.Package) {
continue
}
if v.Section != meta.Section {
continue
}
// TODO(later): allow switching to a different suite even if
// switching requires a language-change. we should indicate
// this in the UI.
if v.Language != meta.Language {
continue
}
suites = append(suites, v)
}
sort.Stable(bySuite(suites))
bySection := make(map[string][]*manpage.Meta)
for _, v := range job.versions {
if v.Package.Suite != meta.Package.Suite {
continue
}
bySection[v.Section] = append(bySection[v.Section], v)
}
sections := make([]*manpage.Meta, 0, len(bySection))
for _, all := range bySection {
sections = append(sections, bestLanguageMatch(meta, all))
}
sort.Stable(byMainSection(sections))
conflicting := make(map[string]bool)
bins := make([]*manpage.Meta, 0, len(job.versions))
for _, v := range job.versions {
if v.Section != meta.Section {
continue
}
if v.Package.Suite != meta.Package.Suite {
continue
}
// We require a strict match for the language when determining
// conflicting packages, because otherwise the packages might
// be augmenting, not conflicting: crontab(1) is present in
// cron, but its translations are shipped e.g. in
// manpages-fr-extra.
if v.Language != meta.Language {
continue
}
if v.Package.Binarypkg != meta.Package.Binarypkg {
conflicting[v.Package.Binarypkg] = true
}
bins = append(bins, v)
}
sort.Stable(byBinarypkg(bins))
ambiguous := make(map[*manpage.Meta]bool)
byLang := make(map[string][]*manpage.Meta)
for _, v := range job.versions {
if v.Section != meta.Section {
continue
}
if v.Package.Suite != meta.Package.Suite {
continue
}
if conflicting[v.Package.Binarypkg] {
continue
}
byLang[v.Language] = append(byLang[v.Language], v)
}
langs := make([]*manpage.Meta, 0, len(byLang))
hrefLangs := make([]*manpage.Meta, 0, len(byLang))
for _, all := range byLang {
for _, e := range all {
langs = append(langs, e)
if len(all) > 1 {
ambiguous[e] = true
}
// hreflang consists only of language and region,
// scripts are not supported.
if !strings.Contains(e.Language, "@") {
hrefLangs = append(hrefLangs, e)
}
}
}
// Sort alphabetically by the locale names (e.g. zh_TW).
sort.Sort(byLanguage(langs))
sort.Sort(byLanguage(hrefLangs))
t := manpageTmpl
title := fmt.Sprintf("%s(%s) — %s — Debian %s", meta.Name, meta.Section, meta.Package.Binarypkg, meta.Package.Suite)
shorttitle := fmt.Sprintf("%s(%s)", meta.Name, meta.Section)
if renderErr != nil {
t = manpageerrorTmpl
title = "Error: " + title
}
var footerExtra bytes.Buffer
if err := manpagefooterextraTmpl.Execute(&footerExtra, struct {
SourceFile string
LastUpdated time.Time
Converted time.Time
Meta *manpage.Meta
}{
SourceFile: filepath.Base(job.src),
LastUpdated: job.modTime,
Converted: time.Now(),
Meta: meta,
}); err != nil {
return nil, manpagePrepData{}, err
}
return t, manpagePrepData{
Title: title,
DebimanVersion: debimanVersion,
Breadcrumbs: breadcrumbs{
{fmt.Sprintf("/contents-%s.html", meta.Package.Suite), meta.Package.Suite},
{fmt.Sprintf("/%s/%s/index.html", meta.Package.Suite, meta.Package.Binarypkg), meta.Package.Binarypkg},
{"", shorttitle},
},
FooterExtra: template.HTML(footerExtra.String()),
Suites: suites,
Versions: job.versions,
Sections: sections,
Bins: bins,
Langs: langs,
HrefLangs: hrefLangs,
Meta: meta,
TOC: toc,
Ambiguous: ambiguous,
Content: template.HTML(content),
Error: renderErr,
}, nil
}
type countingWriter int64
func (c *countingWriter) Write(p []byte) (n int, err error) {
*c += countingWriter(len(p))
return len(p), nil
}
func rendermanpage(gzipw *gzip.Writer, converter *convert.Process, job renderJob) (uint64, error) {
t, data, err := rendermanpageprep(converter, job)
if err != nil {
return 0, err
}
var written countingWriter
if err := write.AtomicallyWithGz(job.dest, gzipw, func(w io.Writer) error {
return t.Execute(io.MultiWriter(w, &written), data)
}); err != nil {
return 0, err
}
return uint64(written), nil
}
================================================
FILE: cmd/debiman/rendermanpage_test.go
================================================
package main
import (
"compress/gzip"
"io/ioutil"
"os"
"testing"
"time"
"github.com/Debian/debiman/internal/convert"
"github.com/Debian/debiman/internal/manpage"
)
func mustParseFromServingPath(t *testing.T, path string) *manpage.Meta {
m, err := manpage.FromServingPath("/srv/man", path)
if err != nil {
t.Fatal(err)
}
return m
}
func TestBestLanguageMatch(t *testing.T) {
table := []struct {
current *manpage.Meta
options []*manpage.Meta
wantServingPath string
}{
{
current: mustParseFromServingPath(t, "testing/cron/crontab.1.fr"),
options: []*manpage.Meta{
mustParseFromServingPath(t, "testing/systemd-cron/crontab.5.fr"),
mustParseFromServingPath(t, "testing/cron/crontab.5.fr"),
mustParseFromServingPath(t, "testing/cron/crontab.5.en"),
},
wantServingPath: "testing/cron/crontab.5.fr",
},
}
for _, entry := range table {
entry := entry // capture
t.Run(entry.wantServingPath, func(t *testing.T) {
t.Parallel()
best := bestLanguageMatch(entry.current, entry.options)
if got, want := best.ServingPath(), entry.wantServingPath; got != want {
t.Fatalf("Unexpected best language match: got %q, want %q", got, want)
}
})
}
}
func TestPrep(t *testing.T) {
const manContents = `.SH foobar
baz
.SH qux
`
f, err := ioutil.TempFile("", "debiman-test")
if err != nil {
t.Fatal(err)
}
defer os.Remove(f.Name())
gzipw := gzip.NewWriter(f)
if _, err := gzipw.Write([]byte(manContents)); err != nil {
t.Fatal(err)
}
if err := gzipw.Close(); err != nil {
t.Fatal(err)
}
if err := f.Close(); err != nil {
t.Fatal(err)
}
manpagesFrExtra5 := mustParseFromServingPath(t, "jessie/manpages-fr-extra/crontab.5.fr")
manpagesFrExtra1 := mustParseFromServingPath(t, "jessie/manpages-fr-extra/crontab.1.fr")
manpagesJa := mustParseFromServingPath(t, "jessie/manpages-ja/crontab.5.ja")
systemdCron := mustParseFromServingPath(t, "jessie/systemd-cron/crontab.5.en")
cron := mustParseFromServingPath(t, "jessie/cron/crontab.5.en")
bcronRun := mustParseFromServingPath(t, "jessie/bcron-run/crontab.5.en")
// Pretend crontab.5.en moved to manpages-fr-systemd for testing issue #27
manpagesFrSystemd := mustParseFromServingPath(t, "testing/manpages-fr-systemd/crontab.5.fr")
manpagesFrSystemd.Package.Replaces = []string{"manpages-fr-extra"}
converter, err := convert.NewProcess()
if err != nil {
t.Fatal(err)
}
defer converter.Kill()
_, data, err := rendermanpageprep(converter, renderJob{
dest: f.Name(),
src: f.Name(),
meta: manpagesFrExtra5,
versions: []*manpage.Meta{
manpagesFrExtra5,
manpagesFrExtra1,
manpagesJa,
systemdCron,
cron,
bcronRun,
manpagesFrSystemd,
},
xref: map[string][]*manpage.Meta{
"crontab": []*manpage.Meta{
manpagesFrExtra5,
manpagesFrExtra1,
manpagesJa,
systemdCron,
cron,
bcronRun,
manpagesFrSystemd,
},
},
modTime: time.Now(),
})
if err != nil {
t.Fatal(err)
}
t.Run("versions", func(t *testing.T) {
wantSuites := []*manpage.Meta{
manpagesFrExtra5,
manpagesFrSystemd,
}
if got, want := len(data.Suites), len(wantSuites); got != want {
t.Fatalf("unexpected number of data.Suites: got %d, want %d", got, want)
}
for i := 0; i < len(data.Suites); i++ {
if got, want := data.Suites[i], wantSuites[i]; got != want {
t.Fatalf("unexpected entry in data.Suites: got %v, want %v", got, want)
}
}
})
t.Run("lang", func(t *testing.T) {
wantLang := []*manpage.Meta{
systemdCron,
cron,
bcronRun,
manpagesFrExtra5,
manpagesJa,
}
if got, want := len(data.Langs), len(wantLang); got != want {
t.Fatalf("unexpected number of data.Langs: got %d, want %d", got, want)
}
for i := 0; i < len(data.Langs); i++ {
if got, want := data.Langs[i], wantLang[i]; got != want {
t.Fatalf("unexpected entry in data.Langs: got %v, want %v", got, want)
}
}
})
t.Run("section", func(t *testing.T) {
wantSections := []*manpage.Meta{
manpagesFrExtra1,
manpagesFrExtra5,
}
if got, want := len(data.Sections), len(wantSections); got != want {
t.Fatalf("unexpected number of data.Sections: got %d, want %d", got, want)
}
for i := 0; i < len(data.Sections); i++ {
if got, want := data.Sections[i], wantSections[i]; got != want {
t.Fatalf("unexpected entry in data.Sections: got %v, want %v", got, want)
}
}
})
t.Run("ambiguous", func(t *testing.T) {
wantAmbiguous := map[*manpage.Meta]bool{
systemdCron: true,
cron: true,
bcronRun: true,
}
if got, want := len(data.Ambiguous), len(wantAmbiguous); got != want {
t.Fatalf("unexpected number of data.Ambiguous: got %d, want %d", got, want)
}
for want := range wantAmbiguous {
if _, ok := data.Ambiguous[want]; !ok {
t.Fatalf("data.Ambiguous unexpectedly does not contain key %v", want)
}
}
})
}
================================================
FILE: cmd/debiman/renderpkgindex.go
================================================
package main
import (
"fmt"
"html/template"
"io"
"sort"
"github.com/Debian/debiman/internal/bundled"
"github.com/Debian/debiman/internal/manpage"
"github.com/Debian/debiman/internal/write"
)
var pkgindexTmpl = mustParsePkgindexTmpl()
var srcpkgindexTmpl = mustParseSrcPkgindexTmpl()
func mustParsePkgindexTmpl() *template.Template {
return template.Must(template.Must(commonTmpls.Clone()).New("pkgindex").Parse(bundled.Asset("pkgindex.tmpl")))
}
func mustParseSrcPkgindexTmpl() *template.Template {
return template.Must(template.Must(commonTmpls.Clone()).New("srcpkgindex").Parse(bundled.Asset("srcpkgindex.tmpl")))
}
func renderPkgindex(dest string, manpageByName map[string]*manpage.Meta) error {
var first *manpage.Meta
for _, m := range manpageByName {
first = m
break
}
mans := make([]string, 0, len(manpageByName))
for n := range manpageByName {
mans = append(mans, n)
}
sort.Strings(mans)
return write.Atomically(dest, true, func(w io.Writer) error {
return pkgindexTmpl.Execute(w, struct {
Title string
DebimanVersion string
Breadcrumbs breadcrumbs
FooterExtra string
First *manpage.Meta
Meta *manpage.Meta
ManpageByName map[string]*manpage.Meta
Mans []string
HrefLangs []*manpage.Meta
}{
Title: fmt.Sprintf("Manpages of %s in Debian %s", first.Package.Binarypkg, first.Package.Suite),
DebimanVersion: debimanVersion,
Breadcrumbs: breadcrumbs{
{fmt.Sprintf("/contents-%s.html", first.Package.Suite), first.Package.Suite},
{fmt.Sprintf("/%s/%s/index.html", first.Package.Suite, first.Package.Binarypkg), first.Package.Binarypkg},
{"", "Contents"},
},
First: first,
Meta: first,
ManpageByName: manpageByName,
Mans: mans,
})
})
}
func renderSrcPkgindex(dest string, src string, manpageByName map[string]*manpage.Meta) error {
var first *manpage.Meta
for _, m := range manpageByName {
first = m
break
}
mans := make([]string, 0, len(manpageByName))
for n := range manpageByName {
mans = append(mans, n)
}
sort.Strings(mans)
return write.Atomically(dest, true, func(w io.Writer) error {
return srcpkgindexTmpl.Execute(w, struct {
Title string
DebimanVersion string
Breadcrumbs breadcrumbs
FooterExtra string
First *manpage.Meta
Meta *manpage.Meta
ManpageByName map[string]*manpage.Meta
Mans []string
HrefLangs []*manpage.Meta
Src string
}{
Title: fmt.Sprintf("Manpages of src:%s in Debian %s", src, first.Package.Suite),
DebimanVersion: debimanVersion,
Breadcrumbs: breadcrumbs{
{fmt.Sprintf("/contents-%s.html", first.Package.Suite), first.Package.Suite},
{fmt.Sprintf("/%s/src:%s/index.html", first.Package.Suite, src), "src:" + src},
{"", "Contents"},
},
First: first,
Meta: first,
ManpageByName: manpageByName,
Mans: mans,
Src: src,
})
})
}
================================================
FILE: cmd/debiman/reuse.go
================================================
package main
import (
"bufio"
"bytes"
"compress/gzip"
"os"
)
var mandocDivB = []byte(`<div class="mandoc">`)
var tocLinkPrefix = []byte(` <a class="toclink"`)
var footerB = []byte(`<div id="footer">`)
func reuse(src string) (doc string, toc []string, err error) {
f, err := os.Open(src)
if err != nil {
return "", nil, err
}
defer f.Close()
r, err := gzip.NewReader(f)
if err != nil {
return "", nil, err
}
defer r.Close()
var (
buf bytes.Buffer
inManpage bool
)
scanner := bufio.NewScanner(r)
for scanner.Scan() {
b := scanner.Bytes()
if bytes.Equal(b, mandocDivB) {
inManpage = true
}
if bytes.Equal(b, footerB) {
all := buf.Bytes()
all = bytes.TrimSuffix(all, []byte("</div>\n</div>\n"))
return string(bytes.TrimSpace(all)), toc, nil
}
if inManpage {
if _, err := buf.Write(b); err != nil {
return "", nil, err
}
if _, err := buf.Write([]byte{'\n'}); err != nil {
return "", nil, err
}
} else if bytes.HasPrefix(b, tocLinkPrefix) {
entry := bytes.TrimSuffix(b, []byte("</a>"))
off := bytes.Index(entry, []byte{'>'})
if off > -1 {
toc = append(toc, string(entry[off+1:]))
}
}
}
return buf.String(), nil, scanner.Err()
}
================================================
FILE: cmd/debiman/reuse_test.go
================================================
package main
import (
"compress/gzip"
"io/ioutil"
"os"
"strings"
"testing"
"time"
"github.com/Debian/debiman/internal/convert"
"github.com/Debian/debiman/internal/manpage"
)
func TestReuse(t *testing.T) {
const manContents = `.SH foobar
baz
.SH qux
`
f, err := ioutil.TempFile("", "debiman-test")
if err != nil {
t.Fatal(err)
}
defer os.Remove(f.Name())
gzipw := gzip.NewWriter(f)
if _, err := gzipw.Write([]byte(manContents)); err != nil {
t.Fatal(err)
}
if err := gzipw.Close(); err != nil {
t.Fatal(err)
}
if err := f.Close(); err != nil {
t.Fatal(err)
}
meta := &manpage.Meta{
Name: "test",
Section: "1",
Language: "en",
Package: &manpage.PkgMeta{
Binarypkg: "test",
Suite: "jessie",
},
}
converter, err := convert.NewProcess()
if err != nil {
t.Fatal(err)
}
defer converter.Kill()
gzipw, err = gzip.NewWriterLevel(nil, gzip.BestCompression)
if err != nil {
t.Fatal(err)
}
if _, err := rendermanpage(gzipw, converter, renderJob{
dest: f.Name(),
src: f.Name(),
meta: meta,
versions: []*manpage.Meta{meta},
xref: map[string][]*manpage.Meta{
"test": []*manpage.Meta{meta},
},
modTime: time.Now(),
}); err != nil {
t.Fatal(err)
}
docWant, tocWant, err := converter.ToHTML(strings.NewReader(manContents), nil)
if err != nil {
t.Fatal(err)
}
docGot, tocGot, err := reuse(f.Name())
if err != nil {
t.Fatal(err)
}
docGot = strings.TrimSpace(docGot)
docWant = strings.TrimSpace(docWant)
if docGot != docWant {
t.Fatalf("Unexpected HTML fragment: got %q, want %q", docGot, docWant)
}
if got, want := len(tocGot), len(tocWant); got != want {
t.Fatalf("Unexpected table of contents length: got %d, want %d", got, want)
}
for n := 0; n < len(tocGot); n++ {
if got, want := tocGot[n], tocWant[n]; got != want {
t.Fatalf("Unexpected table of contents element %d: got %q, want %q", n, got, want)
}
}
}
================================================
FILE: cmd/debiman/writeindex.go
================================================
package main
import (
"io"
"sync/atomic"
pb "github.com/Debian/debiman/internal/proto"
"github.com/Debian/debiman/internal/write"
"github.com/golang/protobuf/proto"
)
// writeIndex serializes an index for the redirect package (used in
// debiman-auxserver) to dest.
func writeIndex(dest string, gv globalView) error {
idx := &pb.Index{
Entry: make([]*pb.IndexEntry, 0, len(gv.xref)),
}
langs := make(map[string]bool)
sections := make(map[string]bool)
for _, x := range gv.xref {
for _, m := range x {
idx.Entry = append(idx.Entry, &pb.IndexEntry{
Name: m.Name,
Suite: m.Package.Suite,
Binarypkg: m.Package.Binarypkg,
Section: m.Section,
Language: m.Language,
})
langs[m.Language] = true
sections[m.Section] = true
sections[m.MainSection()] = true
}
}
for lang := range langs {
idx.Language = append(idx.Language, lang)
}
for section := range sections {
idx.Section = append(idx.Section, section)
}
idx.Suite = gv.idxSuites
idxb, err := proto.Marshal(idx)
if err != nil {
return err
}
return write.Atomically(dest, false, func(w io.Writer) error {
_, err := w.Write(idxb)
if err != nil {
return err
}
atomic.AddUint64(&gv.stats.IndexBytes, uint64(len(idxb)))
return nil
})
}
================================================
FILE: cmd/debiman-auxserver/auxserver.go
================================================
// auxserver serves HTTP redirects and cookie handlers.
package main
import (
"flag"
"html/template"
"log"
"net/http"
"os"
"os/signal"
"runtime/debug"
"syscall"
"github.com/Debian/debiman/internal/auxserver"
"github.com/Debian/debiman/internal/bundled"
"github.com/Debian/debiman/internal/commontmpl"
"github.com/Debian/debiman/internal/redirect"
)
var (
indexPath = flag.String("index",
"/srv/man/auxserver.idx",
"Path to an auxserver index generated by debiman")
listenAddr = flag.String("listen",
"localhost:2431",
"host:port address to listen on")
injectAssets = flag.String("inject_assets",
"",
"If non-empty, a file system path to a directory containing assets to overwrite")
baseURL = flag.String("base_url",
"https://manpages.debian.org",
"Base URL (without trailing slash) to the site. Used where absolute URLs are required, e.g. sitemaps.")
)
// use go build -ldflags "-X main.debimanVersion=<version>" to set the version
var debimanVersion = "HEAD"
func main() {
flag.Parse()
log.Printf("debiman auxserver loading index from %q", *indexPath)
if *injectAssets != "" {
if err := bundled.Inject(*injectAssets); err != nil {
log.Fatal(err)
}
}
idx, err := redirect.IndexFromProto(*indexPath)
if err != nil {
log.Fatal(err)
}
commonTmpls := commontmpl.MustParseCommonTmpls()
notFoundTmpl := template.Must(commonTmpls.New("notfound").Parse(bundled.Asset("notfound.tmpl")))
server := auxserver.NewServer(idx, notFoundTmpl, debimanVersion)
c := make(chan os.Signal, 1)
signal.Notify(c, syscall.SIGHUP)
go func() {
for _ = range c {
log.Printf("SIGHUP received, trying to reload index")
newidx, err := redirect.IndexFromProto(*indexPath)
if err != nil {
log.Printf("Could not load new index from %q: %v", *indexPath, err)
continue
}
log.Printf("Loaded %d manpage entries, %d suites, %d languages from new index %q",
len(newidx.Entries), len(newidx.Suites), len(newidx.Langs), *indexPath)
if err := server.SwapIndex(newidx); err != nil {
log.Printf("Swapping index failed: %v", err)
continue
}
log.Printf("Index swapped")
// Force the garbage collector to return all unused memory to the
// operating system. Even though, on Linux, unused memory can
// apparently be reclaimed by the kernel, preemptively returning the
// memory is less confusing for sysadmins who aren’t intimately
// familiar with Go’s memory model.
debug.FreeOSMemory()
}
}()
basePath := commontmpl.BaseURLPath()
mux := http.NewServeMux()
mux.HandleFunc("/jump", server.HandleJump)
mux.HandleFunc("/suggest", server.HandleSuggest)
mux.HandleFunc("/", server.HandleRedirect)
http.Handle("/", http.StripPrefix(basePath, mux))
log.Printf("Loaded %d manpage entries, %d suites, %d languages from index %q",
len(idx.Entries), len(idx.Suites), len(idx.Langs), *indexPath)
log.Printf("Starting HTTP listener on %q", *listenAddr)
log.Fatal(http.ListenAndServe(*listenAddr, nil))
}
================================================
FILE: cmd/debiman-idx2rwmap/rwmap.go
================================================
// idx2rwmap converts an auxserver index into a file that can be used
// as an Apache RewriteMap. The resulting file contains all possible
// URLs under which manpages can be reached. For a 30MB auxserver
// index, the resulting rwmap is 1.6GB.
//
// The -concurrency option determines how many shards are created in
// -output_dir. To sort and combine the individual shards, use:
//
// LC_ALL=C sort output.* > /srv/man/rwmap.txt
//
// Usually, the resulting file is then converted to DBM so that Apache
// can quickly look up keys:
//
// httxt2dbm -i /srv/man/rwmap.txt -o /srv/man/rwmap.dbm
package main
import (
"bufio"
"flag"
"fmt"
"log"
"os"
"path/filepath"
"runtime"
"strconv"
"strings"
"sync"
"github.com/Debian/debiman/internal/redirect"
)
var (
indexPath = flag.String("index",
"/srv/man/auxserver.idx",
"Path to an auxserver index generated by debiman")
concurrency = flag.Int("concurrency",
0,
"Number of output files to create in parallel. Defaults to the number of logical CPUs.")
outputDir = flag.String("output_dir",
"",
"Directory in which to store the output.n (with n = 0 to -concurrency) files. Defaults to the working directory")
)
type oncePrinter struct {
printed map[string]bool
w *bufio.Writer
idx redirect.Index
variants []redirect.IndexEntry
}
func (op oncePrinter) mustPrint(key string, template redirect.IndexEntry) {
if op.printed[key] {
return
}
filtered := op.idx.Narrow("", template, redirect.IndexEntry{}, op.variants)
if _, err := op.w.WriteString(key); err != nil {
log.Fatal(err)
}
if err := op.w.WriteByte(' '); err != nil {
log.Fatal(err)
}
if _, err := op.w.WriteString(filtered[0].ServingPath(".html")); err != nil {
log.Fatal(err)
}
if err := op.w.WriteByte('\n'); err != nil {
log.Fatal(err)
}
op.printed[key] = true
}
func printAll(bufw *bufio.Writer, idx redirect.Index, name string) {
variants := idx.Entries[name]
op := oncePrinter{
printed: make(map[string]bool),
w: bufw,
idx: idx,
variants: variants,
}
for _, v := range variants {
suites := []string{v.Suite}
for name, rewrite := range idx.Suites {
if rewrite == v.Suite {
suites = append(suites, name)
}
}
lcName := strings.ToLower(v.Name)
// case 01
op.mustPrint(fmt.Sprintf("/%s", lcName),
redirect.IndexEntry{})
// case 02
op.mustPrint(fmt.Sprintf("/%s.%s", lcName, v.Language),
redirect.IndexEntry{Language: v.Language})
// case 03
op.mustPrint(fmt.Sprintf("/%s.%s", lcName, v.Section),
redirect.IndexEntry{Section: v.Section})
// case 03
op.mustPrint(fmt.Sprintf("/%s.%s", lcName, v.Section[:1]),
redirect.IndexEntry{Section: v.Section[:1]})
// FreeBSD-style case 03
op.mustPrint(fmt.Sprintf("/%s/%s", lcName, v.Section),
redirect.IndexEntry{Section: v.Section})
// FreeBSD-style case 03
op.mustPrint(fmt.Sprintf("/%s/%s", lcName, v.Section[:1]),
redirect.IndexEntry{Section: v.Section[:1]})
// case 04
op.mustPrint(fmt.Sprintf("/%s.%s.%s", lcName, v.Section, v.Language),
redirect.IndexEntry{Language: v.Language, Section: v.Section})
// case 04
op.mustPrint(fmt.Sprintf("/%s.%s.%s", lcName, v.Section[:1], v.Language),
redirect.IndexEntry{Language: v.Language, Section: v.Section[:1]})
// case 05
op.mustPrint(fmt.Sprintf("/%s/%s", v.Binarypkg, lcName),
redirect.IndexEntry{Binarypkg: v.Binarypkg})
// case 06
op.mustPrint(fmt.Sprintf("/%s/%s.%s", v.Binarypkg, lcName, v.Language),
redirect.IndexEntry{Language: v.Language, Binarypkg: v.Binarypkg})
// case 07
op.mustPrint(fmt.Sprintf("/%s/%s.%s", v.Binarypkg, lcName, v.Section),
redirect.IndexEntry{Binarypkg: v.Binarypkg, Section: v.Section})
// case 07
op.mustPrint(fmt.Sprintf("/%s/%s.%s", v.Binarypkg, lcName, v.Section[:1]),
redirect.IndexEntry{Binarypkg: v.Binarypkg, Section: v.Section[:1]})
// case 08
op.mustPrint(fmt.Sprintf("/%s/%s.%s.%s", v.Binarypkg, lcName, v.Section, v.Language),
redirect.IndexEntry{Language: v.Language, Section: v.Section, Binarypkg: v.Binarypkg})
// case 08
op.mustPrint(fmt.Sprintf("/%s/%s.%s.%s", v.Binarypkg, lcName, v.Section[:1], v.Language),
redirect.IndexEntry{Language: v.Language, Section: v.Section[:1], Binarypkg: v.Binarypkg})
for _, suite := range suites {
// case 09
op.mustPrint(fmt.Sprintf("/%s/%s", suite, lcName),
redirect.IndexEntry{Suite: v.Suite})
// case 10
op.mustPrint(fmt.Sprintf("/%s/%s.%s", suite, lcName, v.Language),
redirect.IndexEntry{Language: v.Language, Suite: v.Suite})
// case 11
op.mustPrint(fmt.Sprintf("/%s/%s.%s", suite, lcName, v.Section),
redirect.IndexEntry{Section: v.Section, Suite: v.Suite})
// case 11
op.mustPrint(fmt.Sprintf("/%s/%s.%s", suite, lcName, v.Section[:1]),
redirect.IndexEntry{Section: v.Section, Suite: v.Suite})
// case 12
op.mustPrint(fmt.Sprintf("/%s/%s.%s.%s", suite, lcName, v.Section, v.Language),
redirect.IndexEntry{Language: v.Language, Section: v.Section, Suite: v.Suite})
// case 12
op.mustPrint(fmt.Sprintf("/%s/%s.%s.%s", suite, lcName, v.Section[:1], v.Language),
redirect.IndexEntry{Language: v.Language, Section: v.Section[:1], Suite: v.Suite})
// case 13
op.mustPrint(fmt.Sprintf("/%s/%s/%s", suite, v.Binarypkg, lcName),
redirect.IndexEntry{Binarypkg: v.Binarypkg, Suite: v.Suite})
// case 14
op.mustPrint(fmt.Sprintf("/%s/%s/%s.%s", suite, v.Binarypkg, lcName, v.Language),
redirect.IndexEntry{Language: v.Language, Binarypkg: v.Binarypkg, Suite: v.Suite})
// case 15
op.mustPrint(fmt.Sprintf("/%s/%s/%s.%s", suite, v.Binarypkg, lcName, v.Section),
redirect.IndexEntry{Section: v.Section, Binarypkg: v.Binarypkg, Suite: v.Suite})
// case 15
op.mustPrint(fmt.Sprintf("/%s/%s/%s.%s", suite, v.Binarypkg, lcName, v.Section[:1]),
redirect.IndexEntry{Section: v.Section[:1], Binarypkg: v.Binarypkg, Suite: v.Suite})
// case 16
op.mustPrint(fmt.Sprintf("/%s/%s/%s.%s.%s", suite, v.Binarypkg, lcName, v.Section, v.Language),
redirect.IndexEntry{Language: v.Language, Binarypkg: v.Binarypkg, Section: v.Section, Suite: v.Suite})
// case 16
op.mustPrint(fmt.Sprintf("/%s/%s/%s.%s.%s", suite, v.Binarypkg, lcName, v.Section[:1], v.Language),
redirect.IndexEntry{Language: v.Language, Binarypkg: v.Binarypkg, Section: v.Section[:1], Suite: v.Suite})
}
}
}
func main() {
flag.Parse()
idx, err := redirect.IndexFromProto(*indexPath)
if err != nil {
log.Fatal(err)
}
log.Printf("Loaded %d index entries from %q", len(idx.Entries), *indexPath)
work := make(chan string)
var wg sync.WaitGroup
workers := *concurrency
if workers <= 0 {
workers = runtime.NumCPU()
}
for i := 0; i < workers; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
f, err := os.Create(filepath.Join(*outputDir, "output."+strconv.Itoa(i)))
if err != nil {
log.Fatal(err)
}
defer f.Close()
bufw := bufio.NewWriter(f)
for name := range work {
printAll(bufw, idx, name)
}
if err := bufw.Flush(); err != nil {
log.Fatal(err)
}
}(i)
}
for name, _ := range idx.Entries {
work <- name
}
close(work)
wg.Wait()
}
================================================
FILE: cmd/debiman-minisrv/minisrv.go
================================================
// minisrv serves a manpage repository for development purposes (not
// production!).
package main
import (
"compress/gzip"
"errors"
"flag"
"html/template"
"io"
"log"
"mime"
"net/http"
"os"
"path/filepath"
"strings"
"github.com/Debian/debiman/internal/auxserver"
"github.com/Debian/debiman/internal/bundled"
"github.com/Debian/debiman/internal/commontmpl"
"github.com/Debian/debiman/internal/redirect"
)
var (
servingDir = flag.String("serving_dir",
"/srv/man",
"Directory from which manpages should be served")
listenAddr = flag.String("listen",
"localhost:8089",
"host:port on which to serve manpages")
)
// use go build -ldflags "-X main.debimanVersion=<version>" to set the version
var debimanVersion = "HEAD"
var fileNotFound = errors.New("File not found")
func serveFile(w http.ResponseWriter, r *http.Request) error {
compressed := false
path := filepath.Join(*servingDir, r.URL.Path)
if r.URL.Path == "/" {
path = filepath.Join(path, "index.html")
}
f, err := os.Open(path)
if err != nil {
if os.IsNotExist(err) {
// Try with .gz suffix
compressed = true
f, err = os.Open(path + ".gz")
if err != nil && os.IsNotExist(err) {
return fileNotFound
}
}
if err != nil {
return err
}
}
defer f.Close()
ctype := mime.TypeByExtension(filepath.Ext(path))
if ctype == "" {
ctype = "text/html"
}
w.Header().Set("Content-Type", ctype)
rd := io.Reader(f)
if compressed {
gzipr, err := gzip.NewReader(f)
if err != nil {
return err
}
rd = gzipr
defer gzipr.Close()
}
_, err = io.Copy(w, rd)
return err
}
func main() {
flag.Parse()
idx, err := redirect.IndexFromProto(filepath.Join(*servingDir, "auxserver.idx"))
if err != nil {
log.Fatalf("Could not load auxserver index: %v", err)
}
commonTmpls := commontmpl.MustParseCommonTmpls()
notFoundTmpl := template.Must(commonTmpls.New("notfound").Parse(bundled.Asset("notfound.tmpl")))
server := auxserver.NewServer(idx, notFoundTmpl, debimanVersion)
http.HandleFunc("/jump", server.HandleJump)
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
// Similarly to http.ServeFile, deny requests containing .. as
// a precaution. The server will usually be running on
// localhost, but might be exposed to the internet for testing
// temporarily.
if strings.Contains(r.URL.Path, "..") {
http.Error(w, "invalid URL path", http.StatusBadRequest)
log.Printf("Error: invalid URL path %q", r.URL.Path)
return
}
// Check if the path refers to an existing file (possibly compressed)
err := serveFile(w, r)
if err != nil && err != fileNotFound {
http.Error(w, err.Error(), http.StatusInternalServerError)
log.Printf("Error: %v", err)
return
}
if err == nil {
return
}
server.HandleRedirect(w, r)
})
log.Printf("Serving manpages from %q on %q", *servingDir, *listenAddr)
log.Fatal(http.ListenAndServe(*listenAddr, nil))
}
================================================
FILE: debiman-auxserver.service
================================================
[Unit]
Description=debiman auxiliary service endpoints
[Service]
Restart=always
StartLimitInterval=0
User=nobody
Group=nogroup
ExecStart=/usr/bin/debiman-auxserver
# Provide a separate /tmp to the process.
PrivateTmp=true
# Provide all system files read-only to the process.
ProtectSystem=strict
[Install]
WantedBy=multi-user.target
================================================
FILE: example/apache2.conf
================================================
# requires apache2 ≥ 2.4.13 for ap_expr evaluation in ErrorDocument
# required apache2 modules: proxy_http, deflate, expires, headers
<VirtualHost *:3080>
ServerName man.localhost
ServerAdmin webmaster@localhost
DocumentRoot /srv/man
LogLevel alert
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
# Add gzip to the Accept-Encoding to prevent apache from serving an
# HTTP 406 Not Acceptable response. We keep the original
# Accept-Encoding value and later on use mod_deflate to uncompress if
# necessary.
RequestHeader set Accept-Encoding "expr=gzip,%{req:Accept-Encoding}" early
ExpiresActive On
ExpiresDefault "access plus 1 hours"
<Files ~ "^rwmap">
Order allow,deny
Deny from all
</Files>
<Location /auxserver/>
ProxyPass "http://localhost:2431/"
ProxyPassReverse "http://localhost:2431/"
</Location>
ErrorDocument 404 /auxserver/%{REQUEST_URI}?%{QUERY_STRING}
<Directory /srv/man>
Require all granted
# To set the correct Content-Type (e.g. text/html).
RemoveType .gz
AddEncoding gzip gz
FilterDeclare gzip CONTENT_SET
FilterProvider gzip inflate "%{req:Accept-Encoding} !~ /gzip,.*gzip/"
FilterChain gzip
Options +Multiviews
</Directory>
</VirtualHost>
# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
================================================
FILE: example/nginx.conf
================================================
# tested with nginx 1.10.2
server {
listen 127.0.0.2:80;
root /srv/man;
expires 1h;
location / {
# We cannot use try_files because then gzip_static always will
# not be effective anymore.
rewrite ^/?$ /index.html;
# We only have gzip-compressed files:
gzip_static always;
# Uncompress files for clients which do not support gzip:
gunzip on;
# Anything which cannot be served directly from disk is handled
# by auxserver:
error_page 404 = @auxserver;
}
location @auxserver {
proxy_pass http://localhost:2431;
}
}
================================================
FILE: go.mod
================================================
module github.com/Debian/debiman
go 1.24.0
require (
github.com/golang/protobuf v1.5.4
golang.org/x/crypto v0.45.0
golang.org/x/net v0.47.0
golang.org/x/sync v0.18.0
golang.org/x/sys v0.38.0
golang.org/x/text v0.31.0
pault.ag/go/archive v0.0.0-20200912011324-7149510a39c7
pault.ag/go/debian v0.18.0
)
require (
github.com/kjk/lzma v0.0.0-20161016003348-3fd93898850d // indirect
github.com/klauspost/compress v1.18.0 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
google.golang.org/protobuf v1.33.0 // indirect
pault.ag/go/blobstore v0.0.0-20180314122834-d6d187c5a029 // indirect
pault.ag/go/topsort v0.1.1 // indirect
)
================================================
FILE: go.sum
================================================
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/kjk/lzma v0.0.0-20161016003348-3fd93898850d h1:RnWZeH8N8KXfbwMTex/KKMYMj0FJRCF6tQubUuQ02GM=
github.com/kjk/lzma v0.0.0-20161016003348-3fd93898850d/go.mod h1:phT/jsRPBAEqjAibu1BurrabCBNTYiVI+zbmyCZJY6Q=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 h1:nIPpBwaJSVYIxUFsDv3M8ofmx9yWTog9BfvIu0q41lo=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8/go.mod h1:HUYIGzjTL3rfEspMxjDjgmT5uz5wzYJKVo23qUhYTos=
golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=
golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=
golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=
golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=
golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=
golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=
golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
pault.ag/go/archive v0.0.0-20200912011324-7149510a39c7 h1:h/7d1wHt/irYLjUc7YxnuQIdnSRYZv6CTDRnFrRnZqQ=
pault.ag/go/archive v0.0.0-20200912011324-7149510a39c7/go.mod h1:lhAivGV0NOEp/nlrrBFmyZ0YCp45xsn3b8ksEE0r4lM=
pault.ag/go/blobstore v0.0.0-20180314122834-d6d187c5a029 h1:cNq0RBAXx0QlORNPsT78f04j6OtI8LKxXrKN/Uo1IQQ=
pault.ag/go/blobstore v0.0.0-20180314122834-d6d187c5a029/go.mod h1:t2UsqOkWrvOEVHHesR/BDhxYKvww36r/JOdFe0nNObk=
pault.ag/go/debian v0.18.0 h1:nr0iiyOU5QlG1VPnhZLNhnCcHx58kukvBJp+dvaM6CQ=
pault.ag/go/debian v0.18.0/go.mod h1:JFl0XWRCv9hWBrB5MDDZjA5GSEs1X3zcFK/9kCNIUmE=
pault.ag/go/topsort v0.1.1 h1:L0QnhUly6LmTv0e3DEzbN2q6/FGgAcQvaEw65S53Bg4=
pault.ag/go/topsort v0.1.1/go.mod h1:r1kc/L0/FZ3HhjezBIPaNVhkqv8L0UJ9bxRuHRVZ0q4=
================================================
FILE: goembed.go
================================================
// copied from https://github.com/dsymonds/goembed/ with pull requests applied
//go:build ignore
// +build ignore
// goembed generates a Go source file from an input file.
package main
import (
"bytes"
"compress/gzip"
"flag"
"fmt"
"io"
"io/ioutil"
"log"
"os"
"text/template"
)
var (
packageFlag = flag.String("package", "", "Go package name")
varFlag = flag.String("var", "", "Go var name")
gzipFlag = flag.Bool("gzip", false, "Whether to gzip contents")
)
func main() {
flag.Parse()
fmt.Printf("package %s\n\n", *packageFlag)
if flag.NArg() > 0 {
fmt.Println("// Table of contents")
fmt.Printf("var %v = map[string]string{\n", *varFlag)
for i, filename := range flag.Args() {
fmt.Printf("\t%#v: %s_%d,\n", filename, *varFlag, i)
}
fmt.Println("}")
// Using a separate variable for each []byte, instead of
// combining them into a single map literal, enables a storage
// optimization: the compiler places the data directly in the
// program's noptrdata section instead of the heap.
for i, filename := range flag.Args() {
if err := oneVar(fmt.Sprintf("%s_%d", *varFlag, i), filename); err != nil {
log.Fatal(err)
}
}
} else {
if err := oneVarReader(*varFlag, os.Stdin); err != nil {
log.Fatal(err)
}
}
}
func oneVar(varName, filename string) error {
f, err := os.Open(filename)
if err != nil {
return err
}
defer f.Close()
return oneVarReader(varName, f)
}
func oneVarReader(varName string, r io.Reader) error {
raw, err := ioutil.ReadAll(r)
if err != nil {
return err
}
// Generate []byte(<big string constant>) instead of []byte{<list of byte values>}.
// The latter causes a memory explosion in the compiler (60 MB of input chews over 9 GB RAM).
// Doing a string conversion avoids some of that, but incurs a slight startup cost.
if !*gzipFlag {
fmt.Printf(`var %s = "`, varName)
} else {
var buf bytes.Buffer
gzw, _ := gzip.NewWriterLevel(&buf, gzip.BestCompression)
if _, err := gzw.Write(raw); err != nil {
return err
}
if err := gzw.Close(); err != nil {
return err
}
gz := buf.Bytes()
if err := gzipPrologue.Execute(os.Stdout, varName); err != nil {
return err
}
fmt.Printf("var %s string // set in init\n\n", varName)
fmt.Printf(`var %s_gzip = "`, varName)
raw = gz
}
for _, b := range raw {
fmt.Printf("\\x%02x", b)
}
fmt.Println(`"`)
return nil
}
var gzipPrologue = template.Must(template.New("").Parse(`
import (
"bytes"
"compress/gzip"
"io/ioutil"
)
func init() {
r, err := gzip.NewReader(bytes.NewReader({{.}}_gzip))
if err != nil {
panic(err)
}
defer r.Close()
{{.}}, err = ioutil.ReadAll(r)
if err != nil {
panic(err)
}
}
`))
================================================
FILE: internal/auxserver/auxserver.go
================================================
package auxserver
import (
"bytes"
"encoding/json"
"fmt"
"html/template"
"io"
"net/http"
"net/url"
"sort"
"strings"
"sync"
"github.com/Debian/debiman/internal/commontmpl"
"github.com/Debian/debiman/internal/manpage"
"github.com/Debian/debiman/internal/redirect"
)
type Server struct {
idx redirect.Index
idxMu sync.RWMutex
notFoundTmpl *template.Template
debimanVersion string
sortedNames []string
}
func NewServer(idx redirect.Index, notFoundTmpl *template.Template, debimanVersion string) *Server {
s := &Server{
idx: idx,
notFoundTmpl: notFoundTmpl,
debimanVersion: debimanVersion,
}
s.prepareSuggest()
return s
}
// prepareSuggest sets sortedNames to a sorted slice of
// <name>.<section> strings found in idx.
func (s *Server) prepareSuggest() {
names := make(map[string]bool)
for name, entries := range s.idx.Entries {
for _, entry := range entries {
names[name+"."+entry.Section] = true
}
}
result := make([]string, 0, len(names))
for name := range names {
result = append(result, name)
}
sort.Strings(result)
s.sortedNames = result
}
func (s *Server) SwapIndex(idx redirect.Index) error {
u, err := url.Parse("/i3")
if err != nil {
return err
}
redir, err := idx.Redirect(&http.Request{
URL: u,
})
if err != nil {
return fmt.Errorf("idx.Redirect: %v", err)
}
if !strings.HasSuffix(redir, "i3.1.en.html") {
return fmt.Errorf("Redirect(/i3) does not lead to i3.1.en.html: got %q", redir)
}
s.idxMu.Lock()
defer s.idxMu.Unlock()
s.idx = idx
s.prepareSuggest()
return nil
}
func (s *Server) redirect(r *http.Request) (string, error) {
s.idxMu.RLock()
defer s.idxMu.RUnlock()
return s.idx.Redirect(r)
}
func (s *Server) HandleRedirect(w http.ResponseWriter, r *http.Request) {
redir, err := s.redirect(r)
if err != nil {
if nf, ok := err.(*redirect.NotFoundError); ok {
var buf bytes.Buffer
err = s.notFoundTmpl.Execute(&buf, struct {
Title string
DebimanVersion string
Breadcrumbs []string // incorrect type, but empty anyway
FooterExtra string
Manpage string
BestChoice redirect.IndexEntry
Meta *manpage.Meta
HrefLangs []*manpage.Meta
}{
Title: "Not Found",
DebimanVersion: s.debimanVersion,
Manpage: nf.Manpage,
BestChoice: nf.BestChoice,
})
if err == nil {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(http.StatusNotFound)
io.Copy(w, &buf)
return
}
/* fallthrough */
}
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
// StatusTemporaryRedirect (HTTP 307) means subsequent requests
// should use the old URI, which is what we want — the redirect
// target will likely change in the future.
http.Redirect(w, r, commontmpl.BaseURLPath()+redir, http.StatusTemporaryRedirect)
}
func (s *Server) HandleJump(w http.ResponseWriter, r *http.Request) {
q := r.FormValue("q")
if strings.TrimSpace(q) == "" {
http.Error(w, "No q= query parameter specified", http.StatusBadRequest)
return
}
r.URL.Path = "/" + strings.TrimPrefix(q, commontmpl.BaseURLPath())
s.HandleRedirect(w, r)
}
func (s *Server) suggest(q string) []string {
s.idxMu.RLock()
defer s.idxMu.RUnlock()
i := sort.Search(len(s.sortedNames), func(i int) bool {
return s.sortedNames[i] >= q
})
var result []string
for i < len(s.sortedNames) {
if strings.HasPrefix(s.sortedNames[i], q) {
result = append(result, s.sortedNames[i])
} else {
break
}
i++
}
if len(result) > 10 {
result = result[:10]
}
return result
}
func (s *Server) HandleSuggest(w http.ResponseWriter, r *http.Request) {
q := r.FormValue("q")
if strings.TrimSpace(q) == "" {
http.Error(w, "No q= query parameter specified", http.StatusBadRequest)
return
}
r.URL.Path = "/" + q
completions := s.suggest(q)
var buf bytes.Buffer
if err := json.NewEncoder(&buf).Encode([]interface{}{
q,
completions,
}); err != nil {
http.Error(w, fmt.Sprintf("encoding response: %v", err), http.StatusInternalServerError)
return
}
io.Copy(w, &buf)
}
================================================
FILE: internal/auxserver/auxserver_test.go
================================================
package auxserver
import (
"net/http"
"net/url"
"reflect"
"testing"
"github.com/Debian/debiman/internal/redirect"
)
var i3OnlyIdx = redirect.Index{
Entries: map[string][]redirect.IndexEntry{
"i3": []redirect.IndexEntry{
{
Name: "i3",
Suite: "jessie",
Binarypkg: "i3-wm",
Section: "1",
Language: "en",
},
},
},
Suites: map[string]string{
"jessie": "jessie",
},
Langs: map[string]bool{
"en": true,
},
Sections: map[string]bool{
"1": true,
},
}
func mustRedirectI3(t *testing.T, s *Server) {
u, err := url.Parse("/i3")
if err != nil {
t.Fatal(err)
}
redir, err := s.redirect(&http.Request{URL: u})
if err != nil {
t.Fatal(err)
}
if got, want := redir, "/jessie/i3-wm/i3.1.en.html"; got != want {
t.Fatalf("Unexpected redirect for i3: got %q, want %q", got, want)
}
}
func TestIndexSwapSucceed(t *testing.T) {
t.Parallel()
u, err := url.Parse("/w3m")
if err != nil {
t.Fatal(err)
}
s := NewServer(i3OnlyIdx, nil, "")
mustRedirectI3(t, s)
redir, err := s.redirect(&http.Request{URL: u})
if err == nil {
t.Fatal("redirect(/w3m) unexpectedly succeeded")
}
updatedIdx := redirect.Index{
Entries: map[string][]redirect.IndexEntry{
"i3": []redirect.IndexEntry{
{
Name: "i3",
Suite: "jessie",
Binarypkg: "i3-wm",
Section: "1",
Language: "en",
},
},
"w3m": []redirect.IndexEntry{
{
Name: "w3m",
Suite: "jessie",
Binarypkg: "w3m",
Section: "1",
Language: "en",
},
},
},
Suites: map[string]string{
"jessie": "jessie",
},
Langs: map[string]bool{
"en": true,
},
Sections: map[string]bool{
"1": true,
},
}
if err := s.SwapIndex(updatedIdx); err != nil {
t.Fatal(err)
}
mustRedirectI3(t, s)
redir, err = s.redirect(&http.Request{URL: u})
if err != nil {
t.Fatal(err)
}
if got, want := redir, "/jessie/w3m/w3m.1.en.html"; got != want {
t.Fatalf("Unexpected redirect for w3m: got %q, want %q", got, want)
}
}
func TestIndexSwapFail(t *testing.T) {
t.Parallel()
emptyIdx := redirect.Index{}
s := NewServer(i3OnlyIdx, nil, "")
mustRedirectI3(t, s)
if err := s.SwapIndex(emptyIdx); err == nil {
t.Fatal("SwapIndex(emptyIdx) unexpectedly succeeded")
}
mustRedirectI3(t, s)
}
func TestSuggest(t *testing.T) {
s := NewServer(i3OnlyIdx, nil, "")
for _, entry := range []struct {
query string
want []string
}{
{
query: "i",
want: []string{"i3.1"},
},
{
query: "a",
want: nil,
},
} {
if got, want := s.suggest(entry.query), entry.want; !reflect.DeepEqual(got, want) {
t.Fatalf("unexpected result: got %v, want %v", got, want)
}
}
}
func BenchmarkSuggest(b *testing.B) {
// TODO: load representative index
s := NewServer(i3OnlyIdx, nil, "")
b.ResetTimer()
for i := 0; i < b.N; i++ {
// TODO: run sub benchmarks for a few search terms
s.suggest("i")
}
}
================================================
FILE: internal/bundled/GENERATED_bundled.go
================================================
package bundled
// Table of contents
var assets = map[string]string{
"assets/header.tmpl": assets_0,
"assets/footer.tmpl": assets_1,
"assets/style.css": assets_2,
"assets/manpage.tmpl": assets_3,
"assets/manpageerror.tmpl": assets_4,
"assets/manpagefooterextra.tmpl": assets_5,
"assets/contents.tmpl": assets_6,
"assets/pkgindex.tmpl": assets_7,
"assets/srcpkgindex.tmpl": assets_8,
"assets/index.tmpl": assets_9,
"assets/faq.tmpl": assets_10,
"assets/notfound.tmpl": assets_11,
"assets/Inconsolata.woff": assets_12,
"assets/Inconsolata.woff2": assets_13,
"assets/opensearch.xml": assets_14,
"assets/Roboto-Bold.woff": assets_15,
"assets/Roboto-Bold.woff2": assets_16,
"assets/Roboto-Regular.woff": assets_17,
"assets/Roboto-Regular.woff2": assets_18,
}
var assets_0 = "\x3c\x21\x44\x4f\x43\x54\x59\x50\x45\x20\x68\x74\x6d\x6c\x3e\x0a\x7b\x7b\x20\x69\x66\x20\x2e\x4d\x65\x74\x61\x20\x2d\x7d\x7d\x0a\x3c\x68\x74\x6d\x6c\x20\x6c\x61\x6e\x67\x3d\x22\x7b\x7b\x20\x2e\x4d\x65\x74\x61\x2e\x4c\x61\x6e\x67\x75\x61\x67\x65\x54\x61\x67\x20\x7d\x7d\x22\x3e\x0a\x7b\x7b\x20\x65\x6c\x73\x65\x20\x2d\x7d\x7d\x0a\x3c\x68\x74\x6d\x6c\x20\x6c\x61\x6e\x67\x3d\x22\x65\x6e\x22\x3e\x0a\x7b\x7b\x20\x65\x6e\x64\x20\x2d\x7d\x7d\x0a\x3c\x68\x65\x61\x64\x3e\x0a\x3c\x6d\x65\x74\x61\x20\x63\x68\x61\x72\x73\x65\x74\x3d\x22\x55\x54\x46\x2d\x38\x22\x3e\x0a\x3c\x6d\x65\x74\x61\x20\x6e\x61\x6d\x65\x3d\x22\x76\x69\x65\x77\x70\x6f\x72\x74\x22\x20\x63\x6f\x6e\x74\x65\x6e\x74\x3d\x22\x77\x69\x64\x74\x68\x3d\x64\x65\x76\x69\x63\x65\x2d\x77\x69\x64\x74\x68\x2c\x20\x69\x6e\x69\x74\x69\x61\x6c\x2d\x73\x63\x61\x6c\x65\x3d\x31\x2e\x30\x22\x3e\x0a\x3c\x74\x69\x74\x6c\x65\x3e\x7b\x7b\x20\x2e\x54\x69\x74\x6c\x65\x20\x7d\x7d\x20\xe2\x80\x94\x20\x64\x65\x62\x69\x6d\x61\x6e\x3c\x2f\x74\x69\x74\x6c\x65\x3e\x0a\x3c\x73\x74\x79\x6c\x65\x20\x74\x79\x70\x65\x3d\x22\x74\x65\x78\x74\x2f\x63\x73\x73\x22\x3e\x0a\x7b\x7b\x20\x74\x65\x6d\x70\x6c\x61\x74\x65\x20\x22\x73\x74\x79\x6c\x65\x22\x20\x7d\x7d\x0a\x3c\x2f\x73\x74\x79\x6c\x65\x3e\x0a\x3c\x6c\x69\x6e\x6b\x20\x72\x65\x6c\x3d\x22\x73\x65\x61\x72\x63\x68\x22\x20\x74\x69\x74\x6c\x65\x3d\x22\x44\x65\x62\x69\x61\x6e\x20\x6d\x61\x6e\x70\x61\x67\x65\x73\x22\x20\x74\x79\x70\x65\x3d\x22\x61\x70\x70\x6c\x69\x63\x61\x74\x69\x6f\x6e\x2f\x6f\x70\x65\x6e\x73\x65\x61\x72\x63\x68\x64\x65\x73\x63\x72\x69\x70\x74\x69\x6f\x6e\x2b\x78\x6d\x6c\x22\x20\x68\x72\x65\x66\x3d\x22\x2f\x6f\x70\x65\x6e\x73\x65\x61\x72\x63\x68\x2e\x78\x6d\x6c\x22\x3e\x0a\x7b\x7b\x20\x69\x66\x20\x61\x6e\x64\x20\x28\x2e\x48\x72\x65\x66\x4c\x61\x6e\x67\x73\x29\x20\x28\x67\x74\x20\x28\x6c\x65\x6e\x20\x2e\x48\x72\x65\x66\x4c\x61\x6e\x67\x73\x29\x20\x31\x29\x20\x2d\x7d\x7d\x0a\x7b\x7b\x20\x72\x61\x6e\x67\x65\x20\x24\x69\x64\x78\x2c\x20\x24\x6d\x61\x6e\x20\x3a\x3d\x20\x2e\x48\x72\x65\x66\x4c\x61\x6e\x67\x73\x20\x2d\x7d\x7d\x0a\x3c\x6c\x69\x6e\x6b\x20\x72\x65\x6c\x3d\x22\x61\x6c\x74\x65\x72\x6e\x61\x74\x65\x22\x20\x68\x72\x65\x66\x3d\x22\x2f\x7b\x7b\x20\x24\x6d\x61\x6e\x2e\x53\x65\x72\x76\x69\x6e\x67\x50\x61\x74\x68\x20\x7d\x7d\x2e\x68\x74\x6d\x6c\x22\x20\x68\x72\x65\x66\x6c\x61\x6e\x67\x3d\x22\x7b\x7b\x20\x24\x6d\x61\x6e\x2e\x4c\x61\x6e\x67\x75\x61\x67\x65\x54\x61\x67\x20\x7d\x7d\x22\x3e\x0a\x7b\x7b\x20\x65\x6e\x64\x20\x2d\x7d\x7d\x0a\x7b\x7b\x20\x65\x6e\x64\x20\x2d\x7d\x7d\x0a\x3c\x2f\x68\x65\x61\x64\x3e\x0a\x3c\x62\x6f\x64\x79\x3e\x0a\x3c\x64\x69\x76\x20\x69\x64\x3d\x22\x68\x65\x61\x64\x65\x72\x22\x3e\x0a\x20\x20\x20\x3c\x64\x69\x76\x20\x69\x64\x3d\x22\x75\x70\x70\x65\x72\x68\x65\x61\x64\x65\x72\x22\x3e\x0a\x20\x20\x3c\x68\x31\x3e\x3c\x61\x20\x68\x72\x65\x66\x3d\x22\x7b\x7b\x20\x42\x61\x73\x65\x55\x52\x4c\x50\x61\x74\x68\x20\x7d\x7d\x2f\x22\x3e\x73\x6f\x6d\x65\x20\x64\x65\x62\x69\x6d\x61\x6e\x20\x69\x6e\x73\x74\x61\x6c\x6c\x61\x74\x69\x6f\x6e\x3c\x2f\x61\x3e\x3c\x2f\x68\x31\x3e\x0a\x20\x20\x3c\x64\x69\x76\x20\x69\x64\x3d\x22\x73\x65\x61\x72\x63\x68\x62\x6f\x78\x22\x3e\x0a\x20\x20\x20\x20\x3c\x66\x6f\x72\x6d\x20\x61\x63\x74\x69\x6f\x6e\x3d\x22\x7b\x7b\x20\x42\x61\x73\x65\x55\x52\x4c\x50\x61\x74\x68\x20\x7d\x7d\x2f\x6a\x75\x6d\x70\x22\x20\x6d\x65\x74\x68\x6f\x64\x3d\x22\x67\x65\x74\x22\x3e\x0a\x20\x20\x20\x20\x20\x20\x7b\x7b\x20\x69\x66\x20\x2e\x4d\x65\x74\x61\x20\x2d\x7d\x7d\x0a\x20\x20\x20\x20\x20\x20\x3c\x69\x6e\x70\x75\x74\x20\x74\x79\x70\x65\x3d\x22\x68\x69\x64\x64\x65\x6e\x22\x20\x6e\x61\x6d\x65\x3d\x22\x73\x75\x69\x74\x65\x22\x20\x76\x61\x6c\x75\x65\x3d\x22\x7b\x7b\x20\x2e\x4d\x65\x74\x61\x2e\x50\x61\x63\x6b\x61\x67\x65\x2e\x53\x75\x69\x74\x65\x20\x7d\x7d\x22\x3e\x0a\x20\x20\x20\x20\x20\x20\x3c\x69\x6e\x70\x75\x74\x20\x74\x79\x70\x65\x3d\x22\x68\x69\x64\x64\x65\x6e\x22\x20\x6e\x61\x6d\x65\x3d\x22\x62\x69\x6e\x61\x72\x79\x70\x6b\x67\x22\x20\x76\x61\x6c\x75\x65\x3d\x22\x7b\x7b\x20\x2e\x4d\x65\x74\x61\x2e\x50\x61\x63\x6b\x61\x67\x65\x2e\x42\x69\x6e\x61\x72\x79\x70\x6b\x67\x20\x7d\x7d\x22\x3e\x0a\x20\x20\x20\x20\x20\x20\x3c\x69\x6e\x70\x75\x74\x20\x74\x79\x70\x65\x3d\x22\x68\x69\x64\x64\x65\x6e\x22\x20\x6e\x61\x6d\x65\x3d\x22\x73\x65\x63\x74\x69\x6f\x6e\x22\x20\x76\x61\x6c\x75\x65\x3d\x22\x7b\x7b\x20\x2e\x4d\x65\x74\x61\x2e\x53\x65\x63\x74\x69\x6f\x6e\x20\x7d\x7d\x22\x3e\x0a\x20\x20\x20\x20\x20\x20\x3c\x69\x6e\x70\x75\x74\x20\x74\x79\x70\x65\x3d\x22\x68\x69\x64\x64\x65\x6e\x22\x20\x6e\x61\x6d\x65\x3d\x22\x6c\x61\x6e\x67\x75\x61\x67\x65\x22\x20\x76\x61\x6c\x75\x65\x3d\x22\x7b\x7b\x20\x2e\x4d\x65\x74\x61\x2e\x4c\x61\x6e\x67\x75\x61\x67\x65\x20\x7d\x7d\x22\x3e\x0a\x20\x20\x20\x20\x20\x20\x7b\x7b\x20\x65\x6e\x64\x20\x2d\x7d\x7d\x0a\x20\x20\x20\x20\x20\x20\x3c\x69\x6e\x70\x75\x74\x20\x74\x79\x70\x65\x3d\x22\x74\x65\x78\x74\x22\x20\x6e\x61\x6d\x65\x3d\x22\x71\x22\x20\x70\x6c\x61\x63\x65\x68\x6f\x6c\x64\x65\x72\x3d\x22\x6d\x61\x6e\x70\x61\x67\x65\x20\x6e\x61\x6d\x65\x22\x20\x72\x65\x71\x75\x69\x72\x65\x64\x3e\x0a\x20\x20\x20\x20\x20\x20\x3c\x69\x6e\x70\x75\x74\x20\x74\x79\x70\x65\x3d\x22\x73\x75\x62\x6d\x69\x74\x22\x20\x76\x61\x6c\x75\x65\x3d\x22\x4a\x75\x6d\x70\x22\x3e\x0a\x20\x20\x20\x20\x3c\x2f\x66\x6f\x72\x6d\x3e\x0a\x20\x20\x3c\x2f\x64\x69\x76\x3e\x0a\x20\x3c\x2f\x64\x69\x76\x3e\x0a\x3c\x64\x69\x76\x20\x69\x64\x3d\x22\x6e\x61\x76\x62\x61\x72\x22\x3e\x0a\x3c\x70\x20\x63\x6c\x61\x73\x73\x3d\x22\x68\x69\x64\x65\x63\x73\x73\x22\x3e\x3c\x61\x20\x68\x72\x65\x66\x3d\x22\x23\x63\x6f\x6e\x74\x65\x6e\x74\x22\x3e\x53\x6b\x69\x70\x20\x51\x75\x69\x63\x6b\x6e\x61\x76\x3c\x2f\x61\x3e\x3c\x2f\x70\x3e\x0a\x3c\x75\x6c\x3e\x0a\x20\x20\x20\x3c\x6c\x69\x3e\x3c\x61\x20\x68\x72\x65\x66\x3d\x22\x7b\x7b\x20\x42\x61\x73\x65\x55\x52\x4c\x50\x61\x74\x68\x20\x7d\x7d\x2f\x22\x3e\x49\x6e\x64\x65\x78\x3c\x2f\x61\x3e\x3c\x2f\x6c\x69\x3e\x0a\x3c\x2f\x75\x6c\x3e\x0a\x3c\x2f\x64\x69\x76\x3e\x0a\x20\x20\x20\x3c\x70\x20\x69\x64\x3d\x22\x62\x72\x65\x61\x64\x63\x72\x75\x6d\x62\x73\x22\x3e\x26\x6e\x62\x73\x70\x3b\x0a\x20\x20\x20\x20\x20\x7b\x7b\x2d\x20\x72\x61\x6e\x67\x65\x20\x24\x69\x2c\x20\x24\x62\x20\x3a\x3d\x20\x2e\x42\x72\x65\x61\x64\x63\x72\x75\x6d\x62\x73\x20\x7d\x7d\x0a\x20\x20\x20\x20\x20\x7b\x7b\x20\x69\x66\x20\x65\x71\x20\x24\x62\x2e\x4c\x69\x6e\x6b\x20\x22\x22\x20\x7d\x7d\x0a\x20\x20\x20\x20\x20\x26\x23\x78\x32\x46\x3b\x20\x7b\x7b\x20\x24\x62\x2e\x54\x65\x78\x74\x20\x7d\x7d\x0a\x20\x20\x20\x20\x20\x7b\x7b\x20\x65\x6c\x73\x65\x20\x7d\x7d\x0a\x20\x20\x20\x20\x20\x26\x23\x78\x32\x46\x3b\x20\x3c\x61\x20\x68\x72\x65\x66\x3d\x22\x7b\x7b\x20\x42\x61\x73\x65\x55\x52\x4c\x50\x61\x74\x68\x20\x7d\x7d\x7b\x7b\x20\x24\x62\x2e\x4c\x69\x6e\x6b\x20\x7d\x7d\x22\x3e\x7b\x7b\x20\x24\x62\x2e\x54\x65\x78\x74\x20\x7d\x7d\x3c\x2f\x61\x3e\x0a\x20\x20\x20\x20\x20\x7b\x7b\x20\x65\x6e\x64\x20\x7d\x7d\x0a\x20\x20\x20\x20\x20\x7b\x7b\x20\x65\x6e\x64\x20\x2d\x7d\x7d\x0a\x20\x20\x20\x3c\x2f\x70\x3e\x0a\x3c\x2f\x64\x69\x76\x3e\x0a\x3c\x64\x69\x76\x20\x69\x64\x3d\x22\x63\x6f\x6e\x74\x65\x6e\x74\x22\x3e\x0a"
var assets_1 = "\x3c\x2f\x64\x69\x76\x3e\x0a\x3c\x64\x69\x76\x20\x69\x64\x3d\x22\x66\x6f\x6f\x74\x65\x72\x22\x3e\x0a\x7b\x7b\x20\x69\x66\x20\x6e\x65\x20\x2e\x46\x6f\x6f\x74\x65\x72\x45\x78\x74\x72\x61\x20\x22\x22\x20\x7d\x7d\x0a\x3c\x70\x3e\x7b\x7b\x20\x2e\x46\x6f\x6f\x74\x65\x72\x45\x78\x74\x72\x61\x20\x7d\x7d\x3c\x2f\x70\x3e\x0a\x7b\x7b\x20\x65\x6c\x73\x65\x20\x7d\x7d\x0a\x3c\x70\x3e\x50\x61\x67\x65\x20\x6c\x61\x73\x74\x20\x75\x70\x64\x61\x74\x65\x64\x20\x7b\x7b\x20\x4e\x6f\x77\x20\x7d\x7d\x3c\x2f\x70\x3e\x0a\x7b\x7b\x20\x65\x6e\x64\x20\x7d\x7d\x0a\x3c\x68\x72\x3e\x0a\x3c\x64\x69\x76\x20\x69\x64\x3d\x22\x66\x69\x6e\x65\x70\x72\x69\x6e\x74\x22\x3e\x0a\x3c\x70\x3e\x64\x65\x62\x69\x6d\x61\x6e\x20\x7b\x7b\x20\x2e\x44\x65\x62\x69\x6d\x61\x6e\x56\x65\x72\x73\x69\x6f\x6e\x20\x7d\x7d\x2c\x20\x73\x65\x65\x20\x3c\x61\x20\x68\x72\x65\x66\x3d\x22\x68\x74\x74\x70\x73\x3a\x2f\x2f\x67\x69\x74\x68\x75\x62\x2e\x63\x6f\x6d\x2f\x44\x65\x62\x69\x61\x6e\x2f\x64\x65\x62\x69\x6d\x61\x6e\x2f\x22\x3e\x67\x69\x74\x68\x75\x62\x2e\x63\x6f\x6d\x2f\x44\x65\x62\x69\x61\x6e\x2f\x64\x65\x62\x69\x6d\x61\x6e\x3c\x2f\x61\x3e\x3c\x2f\x70\x3e\x0a\x3c\x2f\x64\x69\x76\x3e\x0a\x3c\x2f\x64\x69\x76\x3e\x0a"
var assets_2 = "\x40\x66\x6f\x6e\x74\x2d\x66\x61\x63\x65\x20\x7b\x0a\x20\x20\x66\x6f\x6e\x74\x2d\x66\x61\x6d\x69\x6c\x79\x3a\x20\x27\x49\x6e\x63\x6f\x6e\x73\x6f\x6c\x61\x74\x61\x27\x3b\x0a\x20\x20\x73\x72\x63\x3a\x20\x6c\x6f\x63\x61\x6c\x28\x27\x49\x6e\x63\x6f\x6e\x73\x6f\x6c\x61\x74\x61\x27\x29\x2c\x20\x75\x72\x6c\x28\x7b\x7b\x20\x42\x61\x73\x65\x55\x52\x4c\x50\x61\x74\x68\x20\x7d\x7d\x2f\x49\x6e\x63\x6f\x6e\x73\x6f\x6c\x61\x74\x61\x2e\x77\x6f\x66\x66\x32\x29\x20\x66\x6f\x72\x6d\x61\x74\x28\x27\x77\x6f\x66\x66\x32\x27\x29\x2c\x20\x75\x72\x6c\x28\x7b\x7b\x20\x42\x61\x73\x65\x55\x52\x4c\x50\x61\x74\x68\x20\x7d\x7d\x2f\x49\x6e\x63\x6f\x6e\x73\x6f\x6c\x61\x74\x61\x2e\x77\x6f\x66\x66\x29\x20\x66\x6f\x72\x6d\x61\x74\x28\x27\x77\x6f\x66\x66\x27\x29\x3b\x0a\x7d\x0a\x0a\x40\x66\x6f\x6e\x74\x2d\x66\x61\x63\x65\x20\x7b\x0a\x20\x20\x66\x6f\x6e\x74\x2d\x66\x61\x6d\x69\x6c\x79\x3a\x20\x27\x52\x6f\x62\x6f\x74\x6f\x27\x3b\x0a\x20\x20\x66\x6f\x6e\x74\x2d\x73\x74\x79\x6c\x65\x3a\x20\x6e\x6f\x72\x6d\x61\x6c\x3b\x0a\x20\x20\x66\x6f\x6e\x74\x2d\x77\x65\x69\x67\x68\x74\x3a\x20\x34\x30\x30\x3b\x0a\x20\x20\x73\x72\x63\x3a\x20\x6c\x6f\x63\x61\x6c\x28\x27\x52\x6f\x62\x6f\x74\x6f\x27\x29\x2c\x20\x6c\x6f\x63\x61\x6c\x28\x27\x52\x6f\x62\x6f\x74\x6f\x20\x52\x65\x67\x75\x6c\x61\x72\x27\x29\x2c\x20\x6c\x6f\x63\x61\x6c\x28\x27\x52\x6f\x62\x6f\x74\x6f\x2d\x52\x65\x67\x75\x6c\x61\x72\x27\x29\x2c\x20\x75\x72\x6c\x28\x7b\x7b\x20\x42\x61\x73\x65\x55\x52\x4c\x50\x61\x74\x68\x20\x7d\x7d\x2f\x52\x6f\x62\x6f\x74\x6f\x2d\x52\x65\x67\x75\x6c\x61\x72\x2e\x77\x6f\x66\x66\x32\x29\x20\x66\x6f\x72\x6d\x61\x74\x28\x27\x77\x6f\x66\x66\x32\x27\x29\x2c\x20\x75\x72\x6c\x28\x7b\x7b\x20\x42\x61\x73\x65\x55\x52\x4c\x50\x61\x74\x68\x20\x7d\x7d\x2f\x52\x6f\x62\x6f\x74\x6f\x2d\x52\x65\x67\x75\x6c\x61\x72\x2e\x77\x6f\x66\x66\x29\x20\x66\x6f\x72\x6d\x61\x74\x28\x27\x77\x6f\x66\x66\x27\x29\x3b\x0a\x7d\x0a\x0a\x40\x66\x6f\x6e\x74\x2d\x66\x61\x63\x65\x20\x7b\x0a\x20\x20\x66\x6f\x6e\x74\x2d\x66\x61\x6d\x69\x6c\x79\x3a\x20\x27\x52\x6f\x62\x6f\x74\x6f\x27\x3b\x0a\x20\x20\x66\x6f\x6e\x74\x2d\x73\x74\x79\x6c\x65\x3a\x20\x6e\x6f\x72\x6d\x61\x6c\x3b\x0a\x20\x20\x66\x6f\x6e\x74\x2d\x77\x65\x69\x67\x68\x74\x3a\x20\x37\x30\x30\x3b\x0a\x20\x20\x2f\x2a\x20\x54\x4f\x44\x4f\x3a\x20\x69\x73\x20\x6c\x6f\x63\x61\x6c\x28\x27\x52\x6f\x62\x6f\x74\x6f\x20\x42\x6f\x6c\x64\x27\x29\x20\x72\x65\x61\x6c\x6c\x79\x20\x63\x6f\x72\x72\x65\x63\x74\x3f\x20\x2a\x2f\x0a\x20\x20\x73\x72\x63\x3a\x20\x6c\x6f\x63\x61\x6c\x28\x27\x52\x6f\x62\x6f\x74\x6f\x20\x42\x6f\x6c\x64\x27\x29\x2c\x20\x6c\x6f\x63\x61\x6c\x28\x27\x52\x6f\x62\x6f\x74\x6f\x2d\x42\x6f\x6c\x64\x27\x29\x2c\x20\x75\x72\x6c\x28\x7b\x7b\x20\x42\x61\x73\x65\x55\x52\x4c\x50\x61\x74\x68\x20\x7d\x7d\x2f\x52\x6f\x62\x6f\x74\x6f\x2d\x42\x6f\x6c\x64\x2e\x77\x6f\x66\x66\x32\x29\x20\x66\x6f\x72\x6d\x61\x74\x28\x27\x77\x6f\x66\x66\x32\x27\x29\x2c\x20\x75\x72\x6c\x28\x7b\x7b\x20\x42\x61\x73\x65\x55\x52\x4c\x50\x61\x74\x68\x20\x7d\x7d\x2f\x52\x6f\x62\x6f\x74\x6f\x2d\x42\x6f\x6c\x64\x2e\x77\x6f\x66\x66\x29\x20\x66\x6f\x72\x6d\x61\x74\x28\x27\x77\x6f\x66\x66\x27\x29\x3b\x0a\x7d\x0a\x0a\x62\x6f\x64\x79\x20\x7b\x0a\x09\x63\x6f\x6c\x6f\x72\x3a\x20\x23\x30\x30\x30\x3b\x0a\x09\x62\x61\x63\x6b\x67\x72\x6f\x75\x6e\x64\x2d\x63\x6f\x6c\x6f\x72\x3a\x20\x77\x68\x69\x74\x65\x3b\x0a\x09\x66\x6f\x6e\x74\x2d\x66\x61\x6d\x69\x6c\x79\x3a\x20\x27\x52\x6f\x62\x6f\x74\x6f\x27\x2c\x20\x73\x61\x6e\x73\x2d\x73\x65\x72\x69\x66\x3b\x0a\x09\x66\x6f\x6e\x74\x2d\x73\x69\x7a\x65\x3a\x20\x31\x30\x30\x25\x3b\x0a\x09\x6c\x69\x6e\x65\x2d\x68\x65\x69\x67\x68\x74\x3a\x20\x31\x2e\x32\x3b\x0a\x09\x6c\x65\x74\x74\x65\x72\x2d\x73\x70\x61\x63\x69\x6e\x67\x3a\x20\x30\x2e\x31\x35\x70\x78\x3b\x0a\x09\x6d\x61\x72\x67\x69\x6e\x3a\x20\x30\x3b\x0a\x09\x70\x61\x64\x64\x69\x6e\x67\x3a\x20\x30\x3b\x0a\x7d\x0a\x0a\x23\x68\x65\x61\x64\x65\x72\x20\x7b\x0a\x09\x70\x61\x64\x64\x69\x6e\x67\x3a\x20\x30\x20\x31\x30\x70\x78\x20\x30\x20\x35\x32\x70\x78\x3b\x0a\x7d\x0a\x0a\x23\x6c\x6f\x67\x6f\x20\x7b\x0a\x09\x70\x6f\x73\x69\x74\x69\x6f\x6e\x3a\x20\x61\x62\x73\x6f\x6c\x75\x74\x65\x3b\x0a\x09\x74\x6f\x70\x3a\x20\x30\x3b\x0a\x09\x6c\x65\x66\x74\x3a\x20\x30\x3b\x0a\x09\x62\x6f\x72\x64\x65\x72\x2d\x6c\x65\x66\x74\x3a\x20\x31\x70\x78\x20\x73\x6f\x6c\x69\x64\x20\x74\x72\x61\x6e\x73\x70\x61\x72\x65\x6e\x74\x3b\x0a\x09\x62\x6f\x72\x64\x65\x72\x2d\x72\x69\x67\x68\x74\x3a\x20\x31\x70\x78\x20\x73\x6f\x6c\x69\x64\x20\x74\x72\x61\x6e\x73\x70\x61\x72\x65\x6e\x74\x3b\x0a\x09\x62\x6f\x72\x64\x65\x72\x2d\x62\x6f\x74\x74\x6f\x6d\x3a\x20\x31\x70\x78\x20\x73\x6f\x6c\x69\x64\x20\x74\x72\x61\x6e\x73\x70\x61\x72\x65\x6e\x74\x3b\x0a\x09\x77\x69\x64\x74\x68\x3a\x20\x35\x30\x70\x78\x3b\x0a\x09\x68\x65\x69\x67\x68\x74\x3a\x20\x35\x2e\x30\x37\x65\x6d\x3b\x0a\x09\x6d\x69\x6e\x2d\x68\x65\x69\x67\x68\x74\x3a\x20\x36\x35\x70\x78\x3b\x0a\x7d\x0a\x0a\x23\x6c\x6f\x67\x6f\x20\x61\x20\x7b\x0a\x09\x64\x69\x73\x70\x6c\x61\x79\x3a\x20\x62\x6c\x6f\x63\x6b\x3b\x0a\x09\x68\x65\x69\x67\x68\x74\x3a\x20\x31\x30\x30\x25\x3b\x0a\x7d\x0a\x0a\x23\x6c\x6f\x67\x6f\x20\x69\x6d\x67\x20\x7b\x0a\x09\x6d\x61\x72\x67\x69\x6e\x2d\x74\x6f\x70\x3a\x20\x35\x70\x78\x3b\x0a\x09\x70\x6f\x73\x69\x74\x69\x6f\x6e\x3a\x20\x61\x62\x73\x6f\x6c\x75\x74\x65\x3b\x0a\x09\x62\x6f\x74\x74\x6f\x6d\x3a\x20\x30\x2e\x33\x65\x6d\x3b\x0a\x09\x6f\x76\x65\x72\x66\x6c\x6f\x77\x3a\x20\x61\x75\x74\x6f\x3b\x0a\x09\x62\x6f\x72\x64\x65\x72\x3a\x20\x30\x3b\x0a\x7d\x0a\x0a\x23\x68\x65\x61\x64\x65\x72\x20\x68\x31\x20\x7b\x0a\x20\x20\x20\x20\x66\x6f\x6e\x74\x2d\x73\x69\x7a\x65\x3a\x20\x31\x30\x30\x25\x3b\x0a\x20\x20\x20\x20\x6d\x61\x72\x67\x69\x6e\x3a\x20\x30\x3b\x0a\x7d\x0a\x0a\x70\x2e\x73\x65\x63\x74\x69\x6f\x6e\x20\x7b\x0a\x09\x6d\x61\x72\x67\x69\x6e\x3a\x20\x30\x3b\x0a\x09\x70\x61\x64\x64\x69\x6e\x67\x3a\x20\x30\x20\x35\x70\x78\x20\x30\x20\x35\x70\x78\x3b\x0a\x09\x66\x6f\x6e\x74\x2d\x66\x61\x6d\x69\x6c\x79\x3a\x20\x6d\x6f\x6e\x6f\x73\x70\x61\x63\x65\x3b\x0a\x09\x66\x6f\x6e\x74\x2d\x73\x69\x7a\x65\x3a\x20\x31\x33\x70\x78\x3b\x0a\x09\x6c\x69\x6e\x65\x2d\x68\x65\x69\x67\x68\x74\x3a\x20\x31\x36\x70\x78\x3b\x0a\x09\x63\x6f\x6c\x6f\x72\x3a\x20\x77\x68\x69\x74\x65\x3b\x0a\x09\x6c\x65\x74\x74\x65\x72\x2d\x73\x70\x61\x63\x69\x6e\x67\x3a\x20\x30\x2e\x30\x38\x65\x6d\x3b\x0a\x09\x70\x6f\x73\x69\x74\x69\x6f\x6e\x3a\x20\x61\x62\x73\x6f\x6c\x75\x74\x65\x3b\x0a\x09\x74\x6f\x70\x3a\x20\x30\x70\x78\x3b\x0a\x09\x6c\x65\x66\x74\x3a\x20\x35\x32\x70\x78\x3b\x0a\x09\x62\x61\x63\x6b\x67\x72\x6f\x75\x6e\x64\x2d\x63\x6f\x6c\x6f\x72\x3a\x20\x23\x63\x37\x30\x30\x33\x36\x3b\x0a\x7d\x0a\x0a\x70\x2e\x73\x65\x63\x74\x69\x6f\x6e\x20\x61\x20\x7b\x0a\x09\x63\x6f\x6c\x6f\x72\x3a\x20\x77\x68\x69\x74\x65\x3b\x0a\x09\x74\x65\x78\x74\x2d\x64\x65\x63\x6f\x72\x61\x74\x69\x6f\x6e\x3a\x20\x6e\x6f\x6e\x65\x3b\x0a\x7d\x0a\x0a\x2e\x68\x69\x64\x65\x63\x73\x73\x20\x7b\x0a\x09\x64\x69\x73\x70\x6c\x61\x79\x3a\x20\x6e\x6f\x6e\x65\x3b\x0a\x7d\x0a\x0a\x23\x73\x65\x61\x72\x63\x68\x62\x6f\x78\x20\x7b\x0a\x09\x74\x65\x78\x74\x2d\x61\x6c\x69\x67\x6e\x3a\x6c\x65\x66\x74\x3b\x0a\x09\x6c\x69\x6e\x65\x2d\x68\x65\x69\x67\x68\x74\x3a\x20\x31\x3b\x0a\x09\x6d\x61\x72\x67\x69\x6e\x3a\x20\x30\x20\x31\x30\x70\x78\x20\x30\x20\x30\x2e\x35\x65\x6d\x3b\x0a\x09\x70\x61\x64\x64\x69\x6e\x67\x3a\x20\x31\x70\x78\x20\x30\x20\x31\x70\x78\x20\x30\x3b\x0a\x09\x70\x6f\x73\x69\x74\x69\x6f\x6e\x3a\x20\x61\x62\x73\x6f\x6c\x75\x74\x65\x3b\x0a\x09\x74\x6f\x70\x3a\x20\x30\x3b\x0a\x09\x72\x69\x67\x68\x74\x3a\x20\x30\x3b\x0a\x09\x66\x6f\x6e\x74\x2d\x73\x69\x7a\x65\x3a\x20\x2e\x37\x35\x65\x6d\x3b\x0a\x7d\x0a\x0a\x23\x6e\x61\x76\x62\x61\x72\x20\x75\x6c\x20\x7b\x0a\x09\x6d\x61\x72\x67\x69\x6e\x3a\x20\x30\x3b\x0a\x09\x70\x61\x64\x64\x69\x6e\x67\x3a\x20\x30\x3b\x0a\x09\x6f\x76\x65\x72\x66\x6c\x6f\x77\x3a\x20\x68\x69\x64\x64\x65\x6e\x3b\x0a\x7d\x0a\x0a\x23\x6e\x61\x76\x62\x61\x72\x20\x6c\x69\x20\x7b\x0a\x09\x6c\x69\x73\x74\x2d\x73\x74\x79\x6c\x65\x3a\x20\x6e\x6f\x6e\x65\x3b\x0a\x09\x66\x6c\x6f\x61\x74\x3a\x20\x6c\x65\x66\x74\x3b\x0a\x7d\x0a\x0a\x23\x6e\x61\x76\x62\x61\x72\x20\x61\x20\x7b\x0a\x09\x64\x69\x73\x70\x6c\x61\x79\x3a\x20\x62\x6c\x6f\x63\x6b\x3b\x0a\x09\x63\x6f\x6c\x6f\x72\x3a\x20\x23\x30\x30\x33\x35\x63\x37\x3b\x0a\x09\x74\x65\x78\x74\x2d\x64\x65\x63\x6f\x72\x61\x74\x69\x6f\x6e\x3a\x20\x6e\x6f\x6e\x65\x3b\x0a\x09\x62\x6f\x72\x64\x65\x72\x2d\x6c\x65\x66\x74\x3a\x20\x31\x70\x78\x20\x73\x6f\x6c\x69\x64\x20\x74\x72\x61\x6e\x73\x70\x61\x72\x65\x6e\x74\x3b\x0a\x09\x62\x6f\x72\x64\x65\x72\x2d\x72\x69\x67\x68\x74\x3a\x20\x31\x70\x78\x20\x73\x6f\x6c\x69\x64\x20\x74\x72\x61\x6e\x73\x70\x61\x72\x65\x6e\x74\x3b\x0a\x7d\x0a\x0a\x23\x6e\x61\x76\x62\x61\x72\x20\x61\x3a\x68\x6f\x76\x65\x72\x0a\x2c\x20\x23\x6e\x61\x76\x62\x61\x72\x20\x61\x3a\x76\x69\x73\x69\x74\x65\x64\x3a\x68\x6f\x76\x65\x72\x20\x7b\x0a\x09\x74\x65\x78\x74\x2d\x64\x65\x63\x6f\x72\x61\x74\x69\x6f\x6e\x3a\x20\x75\x6e\x64\x65\x72\x6c\x69\x6e\x65\x3b\x0a\x7d\x0a\x0a\x61\x3a\x6c\x69\x6e\x6b\x20\x7b\x0a\x09\x63\x6f\x6c\x6f\x72\x3a\x20\x23\x30\x30\x33\x35\x63\x37\x3b\x0a\x7d\x0a\x0a\x61\x3a\x76\x69\x73\x69\x74\x65\x64\x20\x7b\x0a\x09\x63\x6f\x6c\x6f\x72\x3a\x20\x23\x35\x34\x36\x33\x38\x63\x3b\x0a\x7d\x0a\x0a\x23\x62\x72\x65\x61\x64\x63\x72\x75\x6d\x62\x73\x20\x7b\x0a\x09\x6c\x69\x6e\x65\x2d\x68\x65\x69\x67\x68\x74\x3a\x20\x32\x3b\x0a\x09\x6d\x69\x6e\x2d\x68\x65\x69\x67\x68\x74\x3a\x20\x32\x30\x70\x78\x3b\x0a\x09\x6d\x61\x72\x67\x69\x6e\x3a\x20\x30\x3b\x0a\x09\x70\x61\x64\x64\x69\x6e\x67\x3a\x20\x30\x3b\x0a\x09\x66\x6f\x6e\x74\x2d\x73\x69\x7a\x65\x3a\x20\x30\x2e\x37\x35\x65\x6d\x3b\x0a\x09\x62\x6f\x72\x64\x65\x72\x2d\x62\x6f\x74\x74\x6f\x6d\x3a\x20\x31\x70\x78\x20\x73\x6f\x6c\x69\x64\x20\x23\x64\x32\x64\x33\x64\x37\x3b\x0a\x7d\x0a\x0a\x23\x62\x72\x65\x61\x64\x63\x72\x75\x6d\x62\x73\x3a\x62\x65\x66\x6f\x72\x65\x20\x7b\x0a\x09\x6d\x61\x72\x67\x69\x6e\x2d\x6c\x65\x66\x74\x3a\x20\x30\x2e\x35\x65\x6d\x3b\x0a\x09\x6d\x61\x72\x67\x69\x6e\x2d\x72\x69\x67\x68\x74\x3a\x20\x30\x2e\x35\x65\x6d\x3b\x0a\x7d\x0a\x0a\x23\x63\x6f\x6e\x74\x65\x6e\x74\x20\x7b\x0a\x20\x20\x20\x20\x6d\x61\x72\x67\x69\x6e\x3a\x20\x30\x20\x31\x30\x70\x78\x20\x30\x20\x35\x32\x70\x78\x3b\x0a\x20\x20\x20\x20\x64\x69\x73\x70\x6c\x61\x79\x3a\x20\x66\x6c\x65\x78\x3b\x0a\x20\x20\x20\x20\x66\x6c\x65\x78\x2d\x64\x69\x72\x65\
gitextract_7wrezzui/
├── .github/
│ └── workflows/
│ └── main.yml
├── LICENSE
├── PERFORMANCE.md
├── README.md
├── assets/
│ ├── about.tmpl
│ ├── contents.tmpl
│ ├── faq.tmpl
│ ├── footer.tmpl
│ ├── header.tmpl
│ ├── index.tmpl
│ ├── manpage.tmpl
│ ├── manpageerror.tmpl
│ ├── manpagefooterextra.tmpl
│ ├── notfound.tmpl
│ ├── opensearch.xml
│ ├── pkgindex.tmpl
│ ├── srcpkgindex.tmpl
│ └── style.css
├── bundle.go
├── cmd/
│ ├── debiman/
│ │ ├── download.go
│ │ ├── download_test.go
│ │ ├── getcontents.go
│ │ ├── getpackages.go
│ │ ├── globalview.go
│ │ ├── main.go
│ │ ├── main_test.go
│ │ ├── mtime.go
│ │ ├── mtime_linux.go
│ │ ├── prometheus.go
│ │ ├── render.go
│ │ ├── render_test.go
│ │ ├── renderaux.go
│ │ ├── rendercontents.go
│ │ ├── rendermanpage.go
│ │ ├── rendermanpage_test.go
│ │ ├── renderpkgindex.go
│ │ ├── reuse.go
│ │ ├── reuse_test.go
│ │ └── writeindex.go
│ ├── debiman-auxserver/
│ │ └── auxserver.go
│ ├── debiman-idx2rwmap/
│ │ └── rwmap.go
│ └── debiman-minisrv/
│ └── minisrv.go
├── debiman-auxserver.service
├── example/
│ ├── apache2.conf
│ └── nginx.conf
├── go.mod
├── go.sum
├── goembed.go
├── internal/
│ ├── auxserver/
│ │ ├── auxserver.go
│ │ └── auxserver_test.go
│ ├── bundled/
│ │ ├── GENERATED_bundled.go
│ │ └── inject.go
│ ├── commontmpl/
│ │ └── commontmpl.go
│ ├── convert/
│ │ ├── convert.go
│ │ ├── convert_test.go
│ │ └── mandoc.go
│ ├── manpage/
│ │ ├── meta.go
│ │ └── meta_test.go
│ ├── proto/
│ │ ├── generate.go
│ │ ├── index.pb.go
│ │ └── index.proto
│ ├── recode/
│ │ ├── recode.go
│ │ └── recode_test.go
│ ├── redirect/
│ │ ├── legacy.go
│ │ ├── redirect.go
│ │ └── redirect_test.go
│ ├── sitemap/
│ │ ├── sitemap.go
│ │ └── sitemap_test.go
│ ├── tag/
│ │ └── tag.go
│ └── write/
│ └── atomically.go
└── testdata/
├── i3lock.1
├── i3lock.html
├── refs.1
├── refs.html
└── tinymirror/
├── dists/
│ └── testing/
│ └── InRelease
└── pool/
└── main/
└── i/
└── i3-wm/
└── i3-wm_4.13-1_amd64.deb
SYMBOL INDEX (258 symbols across 43 files)
FILE: cmd/debiman-auxserver/auxserver.go
function main (line 41) | func main() {
FILE: cmd/debiman-idx2rwmap/rwmap.go
type oncePrinter (line 46) | type oncePrinter struct
method mustPrint (line 53) | func (op oncePrinter) mustPrint(key string, template redirect.IndexEnt...
function printAll (line 73) | func printAll(bufw *bufio.Writer, idx redirect.Index, name string) {
function main (line 201) | func main() {
FILE: cmd/debiman-minisrv/minisrv.go
function serveFile (line 39) | func serveFile(w http.ResponseWriter, r *http.Request) error {
function main (line 81) | func main() {
FILE: cmd/debiman/download.go
function canSkip (line 34) | func canSkip(p pkgEntry, vPath string) bool {
type contentByBinarypkg (line 49) | type contentByBinarypkg
method Len (line 51) | func (p contentByBinarypkg) Len() int { return len(p) }
method Swap (line 52) | func (p contentByBinarypkg) Swap(i, j int) { p[i], p[j] = p[j], p...
method Less (line 53) | func (p contentByBinarypkg) Less(i, j int) bool { return p[i].binarypk...
function findClosestFile (line 57) | func findClosestFile(logger *log.Logger, p pkgEntry, src, name string, c...
function findFile (line 115) | func findFile(logger *log.Logger, src, name string, contentByPath map[st...
function soElim (line 163) | func soElim(logger *log.Logger, src string, r io.Reader, w io.Writer, co...
function writeManpage (line 190) | func writeManpage(logger *log.Logger, src, dest string, r io.Reader, m *...
function createAlternativesLinks (line 210) | func createAlternativesLinks(logger *log.Logger, p pkgEntry, gv globalVi...
function downloadPkg (line 268) | func downloadPkg(ar *archive.Downloader, p pkgEntry, gv globalView) error {
function parallelDownload (line 496) | func parallelDownload(ar *archive.Downloader, gv globalView) error {
FILE: cmd/debiman/download_test.go
function TestWriteManpage (line 11) | func TestWriteManpage(t *testing.T) {
FILE: cmd/debiman/getcontents.go
type contentEntry (line 17) | type contentEntry struct
function parseContentsEntry (line 26) | func parseContentsEntry(scanner *bufio.Scanner) ([]*contentEntry, error) {
function getContents (line 61) | func getContents(ar *archive.Downloader, suite string, component string,...
function getAllContents (line 179) | func getAllContents(ar *archive.Downloader, suite string, release *archi...
FILE: cmd/debiman/getpackages.go
type pkgEntry (line 23) | type pkgEntry struct
function buildContainsMains (line 45) | func buildContainsMains(content []*contentEntry, links map[string][]link...
function parsePackageParagraph (line 76) | func parsePackageParagraph(scanner *bufio.Scanner, arch string, contains...
function less (line 152) | func less(a, b pkgEntry) bool {
function done (line 159) | func done(exhausted []bool) bool {
function getPackages (line 168) | func getPackages(ar *archive.Downloader, rd *archive.ReleaseDownloader, ...
function getAllPackages (line 329) | func getAllPackages(ar *archive.Downloader, rd *archive.ReleaseDownloade...
FILE: cmd/debiman/globalview.go
constant mostPopularArchitecture (line 26) | mostPopularArchitecture = "amd64"
type stats (line 28) | type stats struct
type link (line 37) | type link struct
type globalView (line 42) | type globalView struct
type distributionIdentifier (line 71) | type distributionIdentifier
constant fromCodename (line 74) | fromCodename = iota
constant fromSuite (line 75) | fromSuite
type distribution (line 78) | type distribution struct
function distributions (line 86) | func distributions(codenames []string, suites []string) []distribution {
function parseAlternativesFile (line 109) | func parseAlternativesFile(fn, prefix string) (map[string][]link, error) {
function parseAlternativesDir (line 145) | func parseAlternativesDir(dir string) (map[string][]link, error) {
function markPresent (line 179) | func markPresent(latestVersion map[string]*manpage.PkgMeta, xref map[str...
function buildGlobalView (line 206) | func buildGlobalView(ar *archive.Downloader, dists []distribution, alter...
FILE: cmd/debiman/main.go
function logic (line 87) | func logic() error {
function main (line 170) | func main() {
FILE: cmd/debiman/main_test.go
function TestEndToEnd (line 10) | func TestEndToEnd(t *testing.T) {
FILE: cmd/debiman/mtime.go
function maybeSetLinkMtime (line 8) | func maybeSetLinkMtime(destPath string, t time.Time) error {
FILE: cmd/debiman/mtime_linux.go
function maybeSetLinkMtime (line 14) | func maybeSetLinkMtime(destPath string, t time.Time) error {
FILE: cmd/debiman/prometheus.go
constant metricsTmplContent (line 9) | metricsTmplContent = `
function writeMetrics (line 46) | func writeMetrics(w io.Writer, gv globalView, start time.Time) error {
FILE: cmd/debiman/render.go
type breadcrumb (line 48) | type breadcrumb struct
type breadcrumbs (line 53) | type breadcrumbs
method ToJSON (line 55) | func (b breadcrumbs) ToJSON() template.HTML {
type renderingMode (line 96) | type renderingMode
constant regularFiles (line 99) | regularFiles renderingMode = iota
constant symlinks (line 100) | symlinks
function listManpages (line 105) | func listManpages(dir string) (map[string]*manpage.Meta, error) {
function renderDirectoryIndex (line 161) | func renderDirectoryIndex(dir string, newestModTime time.Time) error {
function walkManContents (line 183) | func walkManContents(ctx context.Context, renderChan chan<- renderJob, d...
function walkContents (line 342) | func walkContents(ctx context.Context, renderChan chan<- renderJob, whit...
function writeSourceIndex (line 451) | func writeSourceIndex(gv globalView, newestForSource map[string]time.Tim...
function writeSourcesWithManpages (line 500) | func writeSourcesWithManpages(gv globalView) error {
function renderAll (line 532) | func renderAll(gv globalView) error {
FILE: cmd/debiman/render_test.go
function TestBreadcrumbsToJSON (line 12) | func TestBreadcrumbsToJSON(t *testing.T) {
function TestFragmentLinkWithColon (line 31) | func TestFragmentLinkWithColon(t *testing.T) {
FILE: cmd/debiman/renderaux.go
function mustParseIndexTmpl (line 18) | func mustParseIndexTmpl() *template.Template {
function mustParseFaqTmpl (line 24) | func mustParseFaqTmpl() *template.Template {
function mustParseAboutTmpl (line 30) | func mustParseAboutTmpl() *template.Template {
type bySuiteStr (line 34) | type bySuiteStr
method Len (line 36) | func (p bySuiteStr) Len() int { return len(p) }
method Swap (line 37) | func (p bySuiteStr) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
method Less (line 38) | func (p bySuiteStr) Less(i, j int) bool {
function renderAux (line 47) | func renderAux(destDir string, gv globalView) error {
FILE: cmd/debiman/rendercontents.go
function mustParseContentsTmpl (line 18) | func mustParseContentsTmpl() *template.Template {
function renderContents (line 22) | func renderContents(dest, suite string, bins []string) error {
FILE: cmd/debiman/rendermanpage.go
constant iso8601Format (line 26) | iso8601Format = "2006-01-02T15:04:05Z"
function init (line 60) | func init() {
function mustParseManpageTmpl (line 97) | func mustParseManpageTmpl() *template.Template {
function mustParseManpageerrorTmpl (line 116) | func mustParseManpageerrorTmpl() *template.Template {
function mustParseManpagefooterextraTmpl (line 135) | func mustParseManpagefooterextraTmpl() *template.Template {
function convertFile (line 145) | func convertFile(converter *convert.Process, src string, resolve func(re...
type byPkgAndLanguage (line 167) | type byPkgAndLanguage struct
method Len (line 172) | func (p byPkgAndLanguage) Len() int { return len(p.opts) }
method Swap (line 173) | func (p byPkgAndLanguage) Swap(i, j int) { p.opts[i], p.opts[j] = p.op...
method Less (line 174) | func (p byPkgAndLanguage) Less(i, j int) bool {
function bestLanguageMatch (line 186) | func bestLanguageMatch(current *manpage.Meta, options []*manpage.Meta) *...
type byLanguage (line 217) | type byLanguage
method Len (line 219) | func (p byLanguage) Len() int { return len(p) }
method Swap (line 220) | func (p byLanguage) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
method Less (line 221) | func (p byLanguage) Less(i, j int) bool { return p[i].Language < p[j]....
type renderJob (line 223) | type renderJob struct
type manpagePrepData (line 235) | type manpagePrepData struct
type bySuite (line 253) | type bySuite
method Len (line 255) | func (p bySuite) Len() int { return len(p) }
method Swap (line 256) | func (p bySuite) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
method Less (line 257) | func (p bySuite) Less(i, j int) bool {
type byMainSection (line 266) | type byMainSection
method Len (line 268) | func (p byMainSection) Len() int { return len(p) }
method Swap (line 269) | func (p byMainSection) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
method Less (line 270) | func (p byMainSection) Less(i, j int) bool { return p[i].MainSection()...
type byBinarypkg (line 272) | type byBinarypkg
method Len (line 274) | func (p byBinarypkg) Len() int { return len(p) }
method Swap (line 275) | func (p byBinarypkg) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
method Less (line 276) | func (p byBinarypkg) Less(i, j int) bool { return p[i].Package.Binaryp...
function rendermanpageprep (line 278) | func rendermanpageprep(converter *convert.Process, job renderJob) (*temp...
type countingWriter (line 466) | type countingWriter
method Write (line 468) | func (c *countingWriter) Write(p []byte) (n int, err error) {
function rendermanpage (line 473) | func rendermanpage(gzipw *gzip.Writer, converter *convert.Process, job r...
FILE: cmd/debiman/rendermanpage_test.go
function mustParseFromServingPath (line 14) | func mustParseFromServingPath(t *testing.T, path string) *manpage.Meta {
function TestBestLanguageMatch (line 22) | func TestBestLanguageMatch(t *testing.T) {
function TestPrep (line 51) | func TestPrep(t *testing.T) {
FILE: cmd/debiman/renderpkgindex.go
function mustParsePkgindexTmpl (line 17) | func mustParsePkgindexTmpl() *template.Template {
function mustParseSrcPkgindexTmpl (line 21) | func mustParseSrcPkgindexTmpl() *template.Template {
function renderPkgindex (line 25) | func renderPkgindex(dest string, manpageByName map[string]*manpage.Meta)...
function renderSrcPkgindex (line 65) | func renderSrcPkgindex(dest string, src string, manpageByName map[string...
FILE: cmd/debiman/reuse.go
function reuse (line 14) | func reuse(src string) (doc string, toc []string, err error) {
FILE: cmd/debiman/reuse_test.go
function TestReuse (line 15) | func TestReuse(t *testing.T) {
FILE: cmd/debiman/writeindex.go
function writeIndex (line 14) | func writeIndex(dest string, gv globalView) error {
FILE: goembed.go
function main (line 27) | func main() {
function oneVar (line 56) | func oneVar(varName, filename string) error {
function oneVarReader (line 65) | func oneVarReader(varName string, r io.Reader) error {
FILE: internal/auxserver/auxserver.go
type Server (line 20) | type Server struct
method prepareSuggest (line 40) | func (s *Server) prepareSuggest() {
method SwapIndex (line 56) | func (s *Server) SwapIndex(idx redirect.Index) error {
method redirect (line 77) | func (s *Server) redirect(r *http.Request) (string, error) {
method HandleRedirect (line 83) | func (s *Server) HandleRedirect(w http.ResponseWriter, r *http.Request) {
method HandleJump (line 122) | func (s *Server) HandleJump(w http.ResponseWriter, r *http.Request) {
method suggest (line 133) | func (s *Server) suggest(q string) []string {
method HandleSuggest (line 156) | func (s *Server) HandleSuggest(w http.ResponseWriter, r *http.Request) {
function NewServer (line 28) | func NewServer(idx redirect.Index, notFoundTmpl *template.Template, debi...
FILE: internal/auxserver/auxserver_test.go
function mustRedirectI3 (line 35) | func mustRedirectI3(t *testing.T, s *Server) {
function TestIndexSwapSucceed (line 49) | func TestIndexSwapSucceed(t *testing.T) {
function TestIndexSwapFail (line 113) | func TestIndexSwapFail(t *testing.T) {
function TestSuggest (line 128) | func TestSuggest(t *testing.T) {
function BenchmarkSuggest (line 149) | func BenchmarkSuggest(b *testing.B) {
FILE: internal/bundled/inject.go
function Inject (line 14) | func Inject(dir string) error {
function Asset (line 46) | func Asset(basename string) string {
function AssetsFiltered (line 50) | func AssetsFiltered(cb func(string) bool) map[string]string {
FILE: internal/commontmpl/commontmpl.go
constant iso8601Format (line 18) | iso8601Format = "2006-01-02T15:04:05Z"
function BaseURLPath (line 33) | func BaseURLPath() string {
function MustParseCommonTmpls (line 44) | func MustParseCommonTmpls() *template.Template {
FILE: internal/convert/convert.go
function recurse (line 23) | func recurse(n *html.Node, f func(c *html.Node) error) error {
type ref (line 38) | type ref struct
type byStart (line 43) | type byStart
method Len (line 45) | func (p byStart) Len() int {
method Less (line 48) | func (p byStart) Less(i, j int) bool {
method Swap (line 51) | func (p byStart) Swap(i, j int) {
function findXrefs (line 55) | func findXrefs(txt string) [][]int {
function xrefMatches (line 84) | func xrefMatches(txt string, resolve func(ref string) string) []ref {
function findUrls (line 101) | func findUrls(txt string) [][]int {
function urlMatches (line 168) | func urlMatches(txt string) []ref {
function xref (line 184) | func xref(txt string, resolve func(ref string) string) []*html.Node {
function plaintext (line 242) | func plaintext(n *html.Node) string {
function headTable (line 253) | func headTable(n *html.Node) bool {
function replaceId (line 262) | func replaceId(n *html.Node, id string) {
function stripAttr (line 275) | func stripAttr(n *html.Node, key, val string) {
function postprocess (line 286) | func postprocess(resolve func(ref string) string, n *html.Node, toc *[]s...
method ToHTML (line 407) | func (p *Process) ToHTML(r io.Reader, resolve func(ref string) string) (...
FILE: internal/convert/convert_test.go
function diff (line 17) | func diff(got, want string) ([]byte, error) {
function TestToHTML (line 51) | func TestToHTML(t *testing.T) {
function cmpElems (line 101) | func cmpElems(input *html.Node, got []*html.Node, want []*html.Node) err...
function TestXref (line 143) | func TestXref(t *testing.T) {
function TestHref (line 243) | func TestHref(t *testing.T) {
function TestXrefHref (line 295) | func TestXrefHref(t *testing.T) {
function formattedXrefInput (line 326) | func formattedXrefInput() *html.Node {
function TestFormattedXref (line 360) | func TestFormattedXref(t *testing.T) {
function BenchmarkXref (line 412) | func BenchmarkXref(b *testing.B) {
function TestXrefHrefExclude (line 419) | func TestXrefHrefExclude(t *testing.T) {
function TestXrefHrefExcludeDot (line 454) | func TestXrefHrefExcludeDot(t *testing.T) {
FILE: internal/convert/mandoc.go
type Process (line 19) | type Process struct
method Kill (line 30) | func (p *Process) Kill() error {
method initMandoc (line 38) | func (p *Process) initMandoc() error {
method mandoc (line 89) | func (p *Process) mandoc(r io.Reader) (stdout string, stderr string, e...
method mandocFork (line 106) | func (p *Process) mandocFork(r io.Reader) (stdout string, stderr strin...
method mandocUnix (line 118) | func (p *Process) mandocUnix(r io.Reader) (stdout string, stderr strin...
function NewProcess (line 25) | func NewProcess() (*Process, error) {
FILE: internal/manpage/meta.go
type PkgMeta (line 14) | type PkgMeta struct
method SameBinary (line 29) | func (p *PkgMeta) SameBinary(o *PkgMeta) bool {
type Meta (line 46) | type Meta struct
method String (line 168) | func (m *Meta) String() string {
method ServingPath (line 172) | func (m *Meta) ServingPath() string {
method RawPath (line 179) | func (m *Meta) RawPath() string {
method PermaLink (line 183) | func (m *Meta) PermaLink() string {
method MainSection (line 187) | func (m *Meta) MainSection() string {
function FromManPath (line 67) | func FromManPath(path string, p *PkgMeta) (*Meta, error) {
function FromServingPath (line 126) | func FromServingPath(servingDir, path string) (*Meta, error) {
FILE: internal/manpage/meta_test.go
function TestManpageFromManPath (line 9) | func TestManpageFromManPath(t *testing.T) {
function TestLanguageTag (line 76) | func TestLanguageTag(t *testing.T) {
FILE: internal/proto/index.pb.go
constant _ (line 32) | _ = proto1.ProtoPackageIsVersion2
type IndexEntry (line 34) | type IndexEntry struct
method Reset (line 42) | func (m *IndexEntry) Reset() { *m = IndexEntry{} }
method String (line 43) | func (m *IndexEntry) String() string { return proto1.Compac...
method ProtoMessage (line 44) | func (*IndexEntry) ProtoMessage() {}
method Descriptor (line 45) | func (*IndexEntry) Descriptor() ([]byte, []int) { return fileDescripto...
method GetName (line 47) | func (m *IndexEntry) GetName() string {
method GetSuite (line 54) | func (m *IndexEntry) GetSuite() string {
method GetBinarypkg (line 61) | func (m *IndexEntry) GetBinarypkg() string {
method GetSection (line 68) | func (m *IndexEntry) GetSection() string {
method GetLanguage (line 75) | func (m *IndexEntry) GetLanguage() string {
type Index (line 82) | type Index struct
method Reset (line 89) | func (m *Index) Reset() { *m = Index{} }
method String (line 90) | func (m *Index) String() string { return proto1.CompactText...
method ProtoMessage (line 91) | func (*Index) ProtoMessage() {}
method Descriptor (line 92) | func (*Index) Descriptor() ([]byte, []int) { return fileDescriptor0, [...
method GetEntry (line 94) | func (m *Index) GetEntry() []*IndexEntry {
method GetLanguage (line 101) | func (m *Index) GetLanguage() []string {
method GetSuite (line 108) | func (m *Index) GetSuite() map[string]string {
method GetSection (line 115) | func (m *Index) GetSection() []string {
function init (line 122) | func init() {
function init (line 127) | func init() { proto1.RegisterFile("index.proto", fileDescriptor0) }
FILE: internal/recode/recode.go
function Reader (line 52) | func Reader(r io.Reader, lang string) io.Reader {
FILE: internal/recode/recode_test.go
function readGzipped (line 12) | func readGzipped(fn string) ([]byte, error) {
function TestRecode (line 33) | func TestRecode(t *testing.T) {
FILE: internal/redirect/legacy.go
method splitLegacy (line 5) | func (i *Index) splitLegacy(path string) (suite string, binarypkg string...
FILE: internal/redirect/redirect.go
type IndexEntry (line 18) | type IndexEntry struct
method ServingPath (line 26) | func (e IndexEntry) ServingPath(suffix string) string {
type Index (line 30) | type Index struct
method split (line 76) | func (i Index) split(path string) (suite string, binarypkg string, nam...
method Narrow (line 185) | func (i Index) Narrow(acceptLang string, template, ref IndexEntry, ent...
method Redirect (line 359) | func (i Index) Redirect(r *http.Request) (string, error) {
constant defaultSuite (line 38) | defaultSuite = "trixie"
constant defaultLanguage (line 39) | defaultLanguage = "en"
function bestLanguageMatch (line 42) | func bestLanguageMatch(t []language.Tag, options []IndexEntry) IndexEntry {
type byMainSection (line 138) | type byMainSection
method Len (line 140) | func (p byMainSection) Len() int { return len(p) }
method Swap (line 141) | func (p byMainSection) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
method Less (line 142) | func (p byMainSection) Less(i, j int) bool {
function searchOrder (line 158) | func searchOrder(sections []string) map[string]int {
type bySection (line 166) | type bySection
method Len (line 168) | func (p bySection) Len() int { return len(p) }
method Swap (line 169) | func (p bySection) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
method Less (line 170) | func (p bySection) Less(i, j int) bool {
type NotFoundError (line 350) | type NotFoundError struct
method Error (line 355) | func (e *NotFoundError) Error() string {
function IndexFromProto (line 444) | func IndexFromProto(path string) (Index, error) {
FILE: internal/redirect/redirect_test.go
function TestNotIndexed (line 217) | func TestNotIndexed(t *testing.T) {
function TestNotFoundWrongSuite (line 227) | func TestNotFoundWrongSuite(t *testing.T) {
function TestNotFoundFullySpecified (line 254) | func TestNotFoundFullySpecified(t *testing.T) {
function TestUnderspecified (line 276) | func TestUnderspecified(t *testing.T) {
function TestLegacyManpagesDebianOrgRedirects (line 408) | func TestLegacyManpagesDebianOrgRedirects(t *testing.T) {
function TestManFreeBSDRedirects (line 477) | func TestManFreeBSDRedirects(t *testing.T) {
function TestAcceptLanguage (line 508) | func TestAcceptLanguage(t *testing.T) {
function TestFormExtra (line 559) | func TestFormExtra(t *testing.T) {
function TestRawManpageRedirect (line 598) | func TestRawManpageRedirect(t *testing.T) {
function TestBlankRedirect (line 629) | func TestBlankRedirect(t *testing.T) {
function TestCronSection (line 662) | func TestCronSection(t *testing.T) {
FILE: internal/sitemap/sitemap.go
type url (line 11) | type url struct
type sitemap (line 17) | type sitemap struct
constant sitemapDateFormat (line 23) | sitemapDateFormat = "2006-01-02"
function WriteTo (line 25) | func WriteTo(w io.Writer, baseUrl string, contents map[string]time.Time)...
function WriteIndexTo (line 63) | func WriteIndexTo(w io.Writer, baseUrl string, contents map[string]time....
FILE: internal/sitemap/sitemap_test.go
function TestSitemap (line 9) | func TestSitemap(t *testing.T) {
function TestSitemapIndex (line 25) | func TestSitemapIndex(t *testing.T) {
FILE: internal/tag/tag.go
function FromLocale (line 25) | func FromLocale(l string) (language.Tag, error) {
FILE: internal/write/atomically.go
function tempDir (line 12) | func tempDir(dest string) string {
function Atomically (line 23) | func Atomically(dest string, compress bool, write func(w io.Writer) erro...
function AtomicallyWithGz (line 73) | func AtomicallyWithGz(dest string, gzipw *gzip.Writer, write func(w io.W...
Condensed preview — 76 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (1,170K chars).
[
{
"path": ".github/workflows/main.yml",
"chars": 1163,
"preview": "name: Push\n\non:\n push:\n branches: [ main ]\n pull_request:\n branches: [ main ]\n\njobs:\n\n build:\n name: CI\n "
},
{
"path": "LICENSE",
"chars": 11357,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "PERFORMANCE.md",
"chars": 1312,
"preview": "# debiman performance\n\nIn both cases, a Debian mirror was available via a Gigabit ethernet connection.\n\n## Modern machin"
},
{
"path": "README.md",
"chars": 7282,
"preview": "# debiman\n\n[](https://github."
},
{
"path": "assets/about.tmpl",
"chars": 105,
"preview": "{{ template \"header\" . }}\n\n<div class=\"maincontents\">\n\n<h1>About</h1>\n\n</div>\n\n{{ template \"footer\" . }}\n"
},
{
"path": "assets/contents.tmpl",
"chars": 380,
"preview": "{{ template \"header\" . }}\n\n<div class=\"maincontents\">\n\n<h1>Binary packages containing manpages in Debian {{ .Suite }}</h"
},
{
"path": "assets/faq.tmpl",
"chars": 103,
"preview": "{{ template \"header\" . }}\n\n<div class=\"maincontents\">\n\n<h1>FAQ</h1>\n\n</div>\n\n{{ template \"footer\" . }}\n"
},
{
"path": "assets/footer.tmpl",
"chars": 292,
"preview": "</div>\n<div id=\"footer\">\n{{ if ne .FooterExtra \"\" }}\n<p>{{ .FooterExtra }}</p>\n{{ else }}\n<p>Page last updated {{ Now }}"
},
{
"path": "assets/header.tmpl",
"chars": 1735,
"preview": "<!DOCTYPE html>\n{{ if .Meta -}}\n<html lang=\"{{ .Meta.LanguageTag }}\">\n{{ else -}}\n<html lang=\"en\">\n{{ end -}}\n<head>\n<me"
},
{
"path": "assets/index.tmpl",
"chars": 1242,
"preview": "{{ template \"header\" . }}\n\n<div class=\"maincontents\">\n\n<h1>some debiman installation</h1>\n\n<p>\n You’re looking at a com"
},
{
"path": "assets/manpage.tmpl",
"chars": 3405,
"preview": "{{ template \"header\" . }}\n\n<div class=\"panels\" id=\"panels\">\n<div class=\"panel\" role=\"complementary\">\n<div class=\"panel-h"
},
{
"path": "assets/manpageerror.tmpl",
"chars": 3343,
"preview": "{{ template \"header\" . }}\n\n<div class=\"panels\" id=\"panels\">\n<div class=\"panel\" role=\"complementary\">\n<div class=\"panel-h"
},
{
"path": "assets/manpagefooterextra.tmpl",
"chars": 413,
"preview": "<table>\n<tr>\n<td>\nSource file:\n</td>\n<td>\n{{ .SourceFile }} (from <a href=\"http://snapshot.debian.org/package/{{ .Meta.P"
},
{
"path": "assets/notfound.tmpl",
"chars": 560,
"preview": "{{ template \"header\" . }}\n\n<div class=\"maincontents\">\n\n{{ if or (ne .BestChoice.Suite \"\") (eq .Manpage \"index\") }}\n<p>\nS"
},
{
"path": "assets/opensearch.xml",
"chars": 481,
"preview": "<?xml version=\"1.0\"?>\n<OpenSearchDescription xmlns=\"http://a9.com/-/spec/opensearch/1.1/\">\n <ShortName>some debiman inst"
},
{
"path": "assets/pkgindex.tmpl",
"chars": 635,
"preview": "{{ template \"header\" . }}\n\n<div class=\"maincontents\">\n\n<h1>Manpages of <a href=\"https://tracker.debian.org/pkg/{{ .First"
},
{
"path": "assets/srcpkgindex.tmpl",
"chars": 597,
"preview": "{{ template \"header\" . }}\n\n<div class=\"maincontents\">\n\n<h1>Manpages of <a href=\"https://tracker.debian.org/pkg/{{ .Src }"
},
{
"path": "assets/style.css",
"chars": 11393,
"preview": "@font-face {\n font-family: 'Inconsolata';\n src: local('Inconsolata'), url({{ BaseURLPath }}/Inconsolata.woff2) format("
},
{
"path": "bundle.go",
"chars": 552,
"preview": "package bundle\n\n//go:generate sh -c \"go run goembed.go -package bundled -var assets assets/header.tmpl assets/footer.tmp"
},
{
"path": "cmd/debiman/download.go",
"chars": 14754,
"preview": "package main\n\nimport (\n\t\"archive/tar\"\n\t\"bufio\"\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"log\"\n\t\"os\"\n\t\"path/f"
},
{
"path": "cmd/debiman/download_test.go",
"chars": 4409,
"preview": "package main\n\nimport (\n\t\"bytes\"\n\t\"log\"\n\t\"os\"\n\t\"strings\"\n\t\"testing\"\n)\n\nfunc TestWriteManpage(t *testing.T) {\n\ttable := []"
},
{
"path": "cmd/debiman/getcontents.go",
"chars": 4838,
"preview": "package main\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\t\"os\"\n\n\t\"golang.org/x/sync/errgroup\"\n\n\t\"pault.ag/go/archive"
},
{
"path": "cmd/debiman/getpackages.go",
"chars": 9235,
"preview": "package main\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"encoding/hex\"\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\t\"os\"\n\t\"strconv\"\n\t\"strings\"\n\n\t\"golang.org/x"
},
{
"path": "cmd/debiman/globalview.go",
"chars": 7978,
"preview": "package main\n\nimport (\n\t\"compress/gzip\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"log\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\""
},
{
"path": "cmd/debiman/main.go",
"chars": 6668,
"preview": "package main\n\nimport (\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\t\"net/http\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"time\"\n\n\t\"golang.org/x"
},
{
"path": "cmd/debiman/main_test.go",
"chars": 339,
"preview": "package main\n\nimport (\n\t\"flag\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"testing\"\n)\n\nfunc TestEndToEnd(t *testing.T) {\n\tdir, err := ioutil.Te"
},
{
"path": "cmd/debiman/mtime.go",
"chars": 140,
"preview": "//go:build !linux\n// +build !linux\n\npackage main\n\nimport \"time\"\n\nfunc maybeSetLinkMtime(destPath string, t time.Time) er"
},
{
"path": "cmd/debiman/mtime_linux.go",
"chars": 420,
"preview": "//go:build linux\n// +build linux\n\npackage main\n\nimport (\n\t\"os\"\n\t\"path/filepath\"\n\t\"time\"\n\n\t\"golang.org/x/sys/unix\"\n)\n\nfun"
},
{
"path": "cmd/debiman/prometheus.go",
"chars": 1839,
"preview": "package main\n\nimport (\n\t\"html/template\"\n\t\"io\"\n\t\"time\"\n)\n\nconst metricsTmplContent = `\n# HELP packages_total The total nu"
},
{
"path": "cmd/debiman/render.go",
"chars": 16594,
"preview": "package main\n\nimport (\n\t\"compress/gzip\"\n\t\"encoding/json\"\n\t\"flag\"\n\t\"fmt\"\n\t\"html/template\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"log\"\n\t\"os\""
},
{
"path": "cmd/debiman/render_test.go",
"chars": 1591,
"preview": "package main\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"github.com/Debian/debiman/internal/manpage\"\n)\n\nfunc Test"
},
{
"path": "cmd/debiman/renderaux.go",
"chars": 3279,
"preview": "package main\n\nimport (\n\t\"fmt\"\n\t\"html/template\"\n\t\"io\"\n\t\"path/filepath\"\n\t\"sort\"\n\t\"strings\"\n\n\t\"github.com/Debian/debiman/in"
},
{
"path": "cmd/debiman/rendercontents.go",
"chars": 1356,
"preview": "package main\n\nimport (\n\t\"fmt\"\n\t\"html/template\"\n\t\"io\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"sort\"\n\n\t\"github.com/Debian/debiman/interna"
},
{
"path": "cmd/debiman/rendermanpage.go",
"chars": 13439,
"preview": "package main\n\nimport (\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"errors\"\n\t\"fmt\"\n\t\"html/template\"\n\t\"io\"\n\t\"log\"\n\t\"net/url\"\n\t\"os\"\n\t\"path/"
},
{
"path": "cmd/debiman/rendermanpage_test.go",
"chars": 4881,
"preview": "package main\n\nimport (\n\t\"compress/gzip\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/Debian/debiman/internal/conv"
},
{
"path": "cmd/debiman/renderpkgindex.go",
"chars": 3033,
"preview": "package main\n\nimport (\n\t\"fmt\"\n\t\"html/template\"\n\t\"io\"\n\t\"sort\"\n\n\t\"github.com/Debian/debiman/internal/bundled\"\n\t\"github.com"
},
{
"path": "cmd/debiman/reuse.go",
"chars": 1225,
"preview": "package main\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"os\"\n)\n\nvar mandocDivB = []byte(`<div class=\"mandoc\">`)\nvar t"
},
{
"path": "cmd/debiman/reuse_test.go",
"chars": 1932,
"preview": "package main\n\nimport (\n\t\"compress/gzip\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"strings\"\n\t\"testing\"\n\t\"time\"\n\n\t\"github.com/Debian/debiman/in"
},
{
"path": "cmd/debiman/writeindex.go",
"chars": 1276,
"preview": "package main\n\nimport (\n\t\"io\"\n\t\"sync/atomic\"\n\n\tpb \"github.com/Debian/debiman/internal/proto\"\n\t\"github.com/Debian/debiman/"
},
{
"path": "cmd/debiman-auxserver/auxserver.go",
"chars": 2996,
"preview": "// auxserver serves HTTP redirects and cookie handlers.\npackage main\n\nimport (\n\t\"flag\"\n\t\"html/template\"\n\t\"log\"\n\t\"net/htt"
},
{
"path": "cmd/debiman-idx2rwmap/rwmap.go",
"chars": 7182,
"preview": "// idx2rwmap converts an auxserver index into a file that can be used\n// as an Apache RewriteMap. The resulting file con"
},
{
"path": "cmd/debiman-minisrv/minisrv.go",
"chars": 2930,
"preview": "// minisrv serves a manpage repository for development purposes (not\n// production!).\npackage main\n\nimport (\n\t\"compress/"
},
{
"path": "debiman-auxserver.service",
"chars": 334,
"preview": "[Unit]\nDescription=debiman auxiliary service endpoints\n\n[Service]\nRestart=always\nStartLimitInterval=0\nUser=nobody\nGroup="
},
{
"path": "example/apache2.conf",
"chars": 1297,
"preview": "# requires apache2 ≥ 2.4.13 for ap_expr evaluation in ErrorDocument\n# required apache2 modules: proxy_http, deflate, exp"
},
{
"path": "example/nginx.conf",
"chars": 548,
"preview": "# tested with nginx 1.10.2\nserver {\n\tlisten 127.0.0.2:80;\n\n\troot /srv/man;\n\n\texpires 1h;\n\n\tlocation / {\n\t\t# We cannot us"
},
{
"path": "go.mod",
"chars": 667,
"preview": "module github.com/Debian/debiman\n\ngo 1.24.0\n\nrequire (\n\tgithub.com/golang/protobuf v1.5.4\n\tgolang.org/x/crypto v0.45.0\n\t"
},
{
"path": "go.sum",
"chars": 2846,
"preview": "github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=\ngithub.com/golang/protobuf v1.5.4/go.m"
},
{
"path": "goembed.go",
"chars": 2695,
"preview": "// copied from https://github.com/dsymonds/goembed/ with pull requests applied\n\n//go:build ignore\n// +build ignore\n\n// g"
},
{
"path": "internal/auxserver/auxserver.go",
"chars": 4202,
"preview": "package auxserver\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"html/template\"\n\t\"io\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"sort\"\n\t\"str"
},
{
"path": "internal/auxserver/auxserver_test.go",
"chars": 2931,
"preview": "package auxserver\n\nimport (\n\t\"net/http\"\n\t\"net/url\"\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/Debian/debiman/internal/redirect\""
},
{
"path": "internal/bundled/GENERATED_bundled.go",
"chars": 698395,
"preview": "package bundled\n\n// Table of contents\nvar assets = map[string]string{\n\t\"assets/header.tmpl\": assets_0,\n\t\"ass"
},
{
"path": "internal/bundled/inject.go",
"chars": 1370,
"preview": "package bundled\n\nimport (\n\t\"io/ioutil\"\n\t\"log\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n)\n\n// Inject overwrites bundled assets w"
},
{
"path": "internal/commontmpl/commontmpl.go",
"chars": 2080,
"preview": "package commontmpl\n\nimport (\n\t\"flag\"\n\t\"html/template\"\n\t\"log\"\n\t\"net/url\"\n\t\"strings\"\n\t\"sync\"\n\t\"time\"\n\n\t\"golang.org/x/text/"
},
{
"path": "internal/convert/convert.go",
"chars": 9543,
"preview": "package convert\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/url\"\n\t\"sort\"\n\t\"strings\"\n\n\t\"golang.org/x/net/html\"\n)\n\nvar heading ="
},
{
"path": "internal/convert/convert_test.go",
"chars": 10350,
"preview": "package convert\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"log\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\t\"testing\"\n\n\t\"golang.org"
},
{
"path": "internal/convert/mandoc.go",
"chars": 3655,
"preview": "package convert\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"log\"\n\t\"net\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\t\"syscall\"\n\n\t\"gol"
},
{
"path": "internal/manpage/meta.go",
"chars": 4824,
"preview": "package manpage\n\nimport (\n\t\"fmt\"\n\t\"path/filepath\"\n\t\"regexp\"\n\t\"strings\"\n\n\t\"github.com/Debian/debiman/internal/tag\"\n\t\"gola"
},
{
"path": "internal/manpage/meta_test.go",
"chars": 3092,
"preview": "package manpage\n\nimport (\n\t\"testing\"\n\n\t\"golang.org/x/text/language\"\n)\n\nfunc TestManpageFromManPath(t *testing.T) {\n\ttabl"
},
{
"path": "internal/proto/generate.go",
"chars": 59,
"preview": "package proto\n\n//go:generate protoc index.proto --go_out=.\n"
},
{
"path": "internal/proto/index.pb.go",
"chars": 4785,
"preview": "// Code generated by protoc-gen-go.\n// source: index.proto\n// DO NOT EDIT!\n\n/*\nPackage proto is a generated protocol buf"
},
{
"path": "internal/proto/index.proto",
"chars": 313,
"preview": "syntax = \"proto3\";\n\npackage proto;\n\nmessage IndexEntry {\n string name = 1;\n string suite = 2;\n string binarypkg = 3;\n"
},
{
"path": "internal/recode/recode.go",
"chars": 1726,
"preview": "package recode\n\nimport (\n\t\"io\"\n\n\t\"golang.org/x/text/encoding\"\n\t\"golang.org/x/text/encoding/charmap\"\n\t\"golang.org/x/text/"
},
{
"path": "internal/recode/recode_test.go",
"chars": 1513,
"preview": "package recode\n\nimport (\n\t\"bytes\"\n\t\"compress/gzip\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"testing\"\n\t\"unicode/utf8\"\n)\n\nfunc readGzipped(fn "
},
{
"path": "internal/redirect/legacy.go",
"chars": 784,
"preview": "package redirect\n\nimport \"strings\"\n\nfunc (i *Index) splitLegacy(path string) (suite string, binarypkg string, name strin"
},
{
"path": "internal/redirect/redirect.go",
"chars": 12075,
"preview": "package redirect\n\nimport (\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"log\"\n\t\"net/http\"\n\t\"path/filepath\"\n\t\"sort\"\n\t\"strings\"\n\n\tpb \"github.com/D"
},
{
"path": "internal/redirect/redirect_test.go",
"chars": 21248,
"preview": "package redirect\n\nimport (\n\t\"net/http\"\n\t\"net/url\"\n\t\"testing\"\n)\n\nvar testIdx = Index{\n\tLangs: map[string]bool{\n\t\t\"en\": tr"
},
{
"path": "internal/sitemap/sitemap.go",
"chars": 2308,
"preview": "package sitemap\n\nimport (\n\t\"encoding/xml\"\n\t\"fmt\"\n\t\"io\"\n\t\"sort\"\n\t\"time\"\n)\n\ntype url struct {\n\tXMLName xml.Name `xml:\"url\""
},
{
"path": "internal/sitemap/sitemap_test.go",
"chars": 1214,
"preview": "package sitemap\n\nimport (\n\t\"bytes\"\n\t\"testing\"\n\t\"time\"\n)\n\nfunc TestSitemap(t *testing.T) {\n\tconst want = `<?xml version=\""
},
{
"path": "internal/tag/tag.go",
"chars": 1318,
"preview": "package tag\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\n\t\"golang.org/x/text/language\"\n)\n\n// see https://wiki.openoffice.org/wiki/Locale"
},
{
"path": "internal/write/atomically.go",
"chars": 2071,
"preview": "package write\n\nimport (\n\t\"bufio\"\n\t\"compress/gzip\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"path/filepath\"\n)\n\nfunc tempDir(dest string)"
},
{
"path": "testdata/i3lock.1",
"chars": 1381,
"preview": ".de Vb \\\" Begin verbatim text\n.ft CW\n.nf\n.ne \\\\$1\n..\n.de Ve \\\" End verbatim text\n.ft R\n.fi\n..\n\n.TH i3lock 1 \"JANUARY 201"
},
{
"path": "testdata/i3lock.html",
"chars": 2863,
"preview": "<div class=\"mandoc\">\n<table class=\"head\">\n <tbody><tr>\n <td class=\"head-ltitle\"><a href=\"testing/i3lock/i3lock.1.C\">"
},
{
"path": "testdata/refs.1",
"chars": 604,
"preview": ".TH REFS 1 \"2016-12-23\" \"debiman\"\n.SH NAME\nrefs \\- test file\n\n.SH SEE ALSO\n.\\\" Using .UR results in a <a> element (most "
},
{
"path": "testdata/refs.html",
"chars": 1408,
"preview": "<div class=\"mandoc\">\n<table class=\"head\">\n <tbody><tr>\n <td class=\"head-ltitle\">REFS(1)</td>\n <td class=\"head-vol"
},
{
"path": "testdata/tinymirror/dists/testing/InRelease",
"chars": 733,
"preview": "Origin: Debian\nLabel: Debian\nSuite: testing\nCodename: stretch\nChangelogs: http://metadata.ftp-master.debian.org/changelo"
}
]
// ... and 1 more files (download for full content)
About this extraction
This page contains the full source code of the Debian/debiman GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 76 files (941.3 KB), approximately 475.9k tokens, and a symbol index with 258 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.