Showing preview only (486K chars total). Download the full file or copy to clipboard to get everything.
Repository: jugglerchris/rust-html2text
Branch: main
Commit: 1f57ca869d10
Files: 37
Total size: 466.7 KB
Directory structure:
gitextract_ep3xu7e5/
├── .circleci/
│ └── config.yml
├── .github/
│ ├── dependabot.yml
│ └── workflows/
│ ├── ci.yml
│ └── jekyll-gh-pages.yml
├── .gitignore
├── CHANGELOG.md
├── Cargo.toml
├── LICENSE
├── README.md
├── benches/
│ └── tables.rs
├── examples/
│ └── html2term.rs
├── html2text-cli/
│ ├── Cargo.toml
│ ├── README.md
│ └── src/
│ └── main.rs
├── html2text-web-demo/
│ ├── .cargo/
│ │ └── config.toml
│ ├── .gitignore
│ ├── Cargo.toml
│ ├── Trunk.toml
│ ├── index.html
│ └── src/
│ └── lib.rs
├── pages/
│ ├── .gitignore
│ ├── _config.yml
│ ├── _includes/
│ │ └── head.html
│ ├── assets/
│ │ ├── demo-main.js
│ │ └── demo.css
│ └── index.markdown
├── rust.yml
└── src/
├── ansi_colours.rs
├── css/
│ ├── parser.rs
│ └── types.rs
├── css.rs
├── lib.rs
├── macros.rs
├── markup5ever_rcdom.rs
├── render/
│ ├── mod.rs
│ └── text_renderer.rs
└── tests.rs
================================================
FILE CONTENTS
================================================
================================================
FILE: .circleci/config.yml
================================================
version: 2.1
orbs:
win: circleci/windows@2.2.0
jobs:
build-stable:
docker:
- image: cimg/rust:1.85.1
steps:
- checkout
- run: cargo --version
- run: cargo build --workspace
- run: cargo test
- run:
name: Install tools
command: |
rustup component add rustfmt clippy
- run:
name: Check formatting
command: |
cargo fmt --all -- --check --color=auto
- run:
name: Clippy
command: |
cargo clippy --all-features
build-css:
docker:
- image: cimg/rust:1.90
steps:
- checkout
- run: cargo --version
- run: cargo build --features=css,css_ext,xml --workspace
- run: cargo test --features=css,css_ext,xml
build-1-85:
docker:
- image: cimg/rust:1.85
steps:
- checkout
- run: cargo --version
- run: cargo build --features=css
- run: cargo test --features=css
build-windows:
executor:
name: win/default
size: medium
shell: bash.exe
environment:
PATHk
steps:
- checkout
- run:
name: Install Rust
command: |
curl https://static.rust-lang.org/rustup/dist/x86_64-pc-windows-msvc/rustup-init.exe --output rustup-init.exe
./rustup-init.exe -y
- run:
name: Update PATH and cargo config
command: |
echo "[net]" >> $USERPROFILE/.cargo/config
echo "git-fetch-with-cli = true" >> $USERPROFILE/.cargo/config
echo 'export PATH=$USERPROFILE/.cargo/bin:$PATH' >> $BASH_ENV
- run:
name: Build
command: |
cargo build
- run:
name: Tests
command: |
cargo test
workflows:
version: 2
build:
jobs:
- "build-stable"
- "build-css"
- "build-1-85"
- "build-windows"
================================================
FILE: .github/dependabot.yml
================================================
version: 2
updates:
- package-ecosystem: "cargo"
directory: "/"
schedule:
interval: "weekly"
day: "friday"
rebase-strategy: "disabled"
================================================
FILE: .github/workflows/ci.yml
================================================
name: CI
on:
pull_request:
push:
branches:
- main
env:
CARGO_TERM_COLOR: always
jobs:
test-action:
name: Check semver compatibility
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v2
- name: Install stable toolchain
uses: actions-rs/toolchain@v1
with:
toolchain: stable
profile: minimal
override: true
- name: Check semver
uses: obi1kenobi/cargo-semver-checks-action@v2
with:
version-tag-prefix: ''
================================================
FILE: .github/workflows/jekyll-gh-pages.yml
================================================
# Sample workflow for building and deploying a Jekyll site to GitHub Pages
name: Build and deploy demo site
on:
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write
# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
group: "pages"
cancel-in-progress: false
jobs:
# Build job:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Pages
uses: actions/configure-pages@v5
- name: Install trunk
run: cargo install trunk --version=0.21.13
- name: Install WASM rust target
run: rustup target add wasm32-unknown-unknown
- name: Build WASM module
run: trunk build
working-directory: ./html2text-web-demo
- name: Copy WASM assets
run: cp html2text-web-demo/dist/html2text-web-demo{.js,_bg.wasm} ./pages/assets/
- name: Build with Jekyll
uses: actions/jekyll-build-pages@v1
with:
source: ./pages
destination: ./_site
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
# Deployment job
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
================================================
FILE: .gitignore
================================================
target
================================================
FILE: CHANGELOG.md
================================================
# Changelog
Possible log types:
- `[added]` for new features.
- `[changed]` for changes in existing functionality.
- `[deprecated]` for once-stable features removed in upcoming releases.
- `[removed]` for deprecated features removed in this release.
- `[fixed]` for any bug fixes.
- `[security]` to invite users to upgrade in case of vulnerabilities.
### 0.17.1
- [added] Add support for XHTML (for the cases where it doesn't quite behave
like HTML).
### 0.17.0
- [changed] Split `html2text` example into `html2text-cli` crate
- [fixed] A possible panic when syntax-highlighting
- [changed] Update html5ever to 0.39.0
### 0.16.7
- [added] Support `<b>` tags as bold (thanks amir)
- [changed] Update html5ever to 0.38.0 (thanks mtorromeo)
### 0.16.6
- [changed] Update html5ever and tendril dependencies.
### 0.16.5
- [fixed] Fix a subtract with underflow with rowspans and empty rows (thanks
mdierksen)
### 0.16.4
- [fixed] Further fix for RcDom::serialize() when there is a `<doctype>`.
### 0.16.3
- [fixed] RcDom::serialize() panicked.
- [changed] Bumped html5ever dependency
- [fixed] Fixed a subtraction underflow in the `html2term` example.
### 0.16.2
- [fixed] Removed spurious `dbg!()` accidentally left in.
### 0.16.1
- [added] Add `Config::empty_img_mode()` to configure how images with no alt text
are handled.
### 0.16.0
- [changed] Updated MSRV to 1.85.
- [fixed] Fix a panic in debug mode (subtraction underflow) with some table/rowspan
edge cases (thanks mtorromeo)
### 0.15.5
- [fixed] Fix an assertion and some missing styles with rowspan cells in rich mode.
### 0.15.4
- [added] Support handling `rowspan` in tables.
### 0.15.3
- [fixed] Parse `<noscript>` tags as if scripting is disabled (thanks craigchiang)
- [fixed] Treat inline `<svg>` as images (thanks craigchiang)
### 0.15.2
- [fixed] Fix whitespace-only tables being shown as "─" (thanks fuelen)
- [fixed] Fix wrapping with non-breaking spaces, zero-width spaces, and
some similar equivalents.
### 0.15.1
- [added] CSS: Support basic attribute selectors (`div[attr="bar"]`).
- [changed] Various improvements to syntax highlighting:
- It uses the priority of the `x-syntax` rule.
- Now supported on non-`<pre>` elements.
- No longer strips contained tags when highlighting
- Compatible with `display: x-raw-dom` extension (e.g. to colour the HTML)
- [fixed] With `pad_block_width` enabled, do a better job of padding blocks.
In particular, the padding gets the block's background colour (when CSS etc.
are being used).
### 0.15.0
- [added] Syntax highlighting support for `<pre>` blocks
(`Config::register_highlighter` and CSS `x-syntax: foo`)
- [changed] CSS extensions (until now `display: x-raw-dom`, and only if the
`css_ext` Cargo feature is enabled) are now only available in agent and user CSS.
This is a breaking change, but is not likely to affect many users.
### 0.14.4
- [added] `RcDom::serialize`, and expose a few more of the `RcDom` types.
- [added] Online [demo page](https://jugglerchris.github.io/rust-html2text/)
### 0.14.3
- [changed] Updated dependencies, including html5ever 0.31.
### 0.14.2
- [fixed] An issue with multiple verions of markup5ever being included.
(thanks anna-hope)
### 0.14.1
- [fixed] An issue with `FragmentStart`s being lost (thanks toiletbril)
- [fixed] An infinite loop if tabs inside `<pre>` wrapped past the width
(thanks nshp)
### 0.14.0
- [changed] Various small refactors (thanks sftse)
- [changed] `Config::rich()` no longer includes decorations around `<em>` etc. -
use `Config::rich().do_decorate()` to get the old behaviour.
- [fixed] Remove unnecessary empty lines at the start of lists (thanks russellbanks)
- [added] New CSS support: `::before`/`::after` and `content: "string"`, which is now
used for simple decorations. With CSS enabled, this allows for customising
the display of `<em>foo</em>` without writing a decorator.
- [added] Add support for `<h5>` and `<h6>` (thanks noahbaculi)
- [changed] Link footnotes are now configurable independently of the decorator, and on
by default for `config::plain()` but can be enabled or disabled with
`config.link_footnotes(true/false)`. The footnote references (e.g. `[1]`) are added
in the main renderer, and the actual footnotes are written in a default implementation
of `TextDecorator::finalise()` so can be customised.
### 0.13.6
- [fixed] Fixed issue parsing CSS rules with known rules but unknown values,
which caused parsing to stop instead of just skipping the unkown rule.
### 0.13.5
- [added] CSS support for `:nth-child()` (not yet with the `of foo`).
- [added] Non-standard `display: x-raw-dom` for debugging (with `css_ext`
feature flag).
- [fixed] An issue which could (apparently rarely) miss out some output depending on wrapping
- [fixed] CSS parsing stopped when it hit an at-rule.
- [added] Add `--show-css` option to `html2text` example for debugging what rules were parsed.
- [added] Add poor-man's inspect mode to `html2term` - `I` to enable/disable, and arrows to navigate
around the DOM. Implemented using `:nth-child` and `x-raw-dom`.
### 0.13.4
- [fixed] Fix a debug assertion from a double-counted length increment
(thanks JadedBlueeyes).
### 0.13.3
- [fixed] Handle some obsolete `bgcolor=...` attributes.
- [added] html2text example has `--show-render` to help debugging render issues.
- [changed] Some error handling and other tidyups (thanks sftse)
### 0.13.2
- [fixed] Fixed errors when building with Rust 1.72.
### 0.13.1
- [added] html2text now has --show-dom
- [fixed] Support background CSS property (for colour)
- [fixed] Some edge cases with CSS styles on whitespace
### 0.13.0
- [added] Support CSS white-space: pre-wrap (and normal, pre).
### 0.13.0-alpha.2
- [changed] Updated html5ever and markup5ever crate versions. This has meant
updating the MSRV, which is now set to 1.72.
- [fixed] Add `Config::no_link_wrapping()` (thanks JadeBlueEyes)
- [fixed] Fix panic with empty table inside a list (thanks sftse)
- [changed] Top level convenience functions (`from_read` etc.) now return
`Result<..>` instead of panicking (thanks sftse)
- [fixed] Fix panic with very large HTML `colspan` (thanks pycui)
- [changed] CSS updates:
- Separate user agent, author, and user CSS layers
- Improve the style precedence between layers and implement specificity.
### 0.13.0-alpha.1
- [fixed] Table rows with colours would disappear. (thanks tkapias)
### 0.13.0-alpha.0
- [changed] Replaced LightningCSS with a smaller CSS parser. There is a chance
that some CSS edge cases which no longer work; if so this would be a bug.
- [removed] Some previously `pub` items and methods which are either internal
implementation details or considered redundant have been removed or made
private (thanks sftse). Please open an issue for anything removed that was
being used.
Of note, `RenderTree::render_plain()` and `RenderTree::render_rich()` have
been removed. Replace code like:
```rust
let text = html2text::parse(html)?
.render_plain(80)?
.into_string()?;
```
with:
```rust
let text = html2text::config::plain()
.render_to_string(html2text::parse(html)?)?
```
- [changed] Some names moved out of `text_renderer` module, so some `use` statements
may need updating.
- [changed] Replace some `unwrap()` with improved patterns (thanks sftse).
- [changed] Updated some dependencies
### 0.12.5
- [changed] Updated some dependencies
- [added] The `html2text` example now has `--ignore-css-colour`, which ignores CSS
colour information but still uses `display: none`, for example.
- [added] The `html2text` example now has `--only-css` option, to not use
default colours when CSS colours are being used.
- [fixed] Make the dummy `dashmap` depenency optional so it's not included
unnecessarily when CSS isn't enabled (thanks xmakro)
### 0.12.4
- [changed] Update the previous `max-height: 0` to also look at `height: 0` and require
`overflow: hidden` as well.
This helps with a hack some e-mail senders use for e-mail previews. (thanks tkapias)
### 0.12.3
- [changed] Treat `max-height: 0` as if it's `display: none` when CSS is enabled.
This helps with a hack some e-mail senders use for e-mail previews. (thanks tkapias)
### 0.12.2
- [changed] Bump version of lightningcss dependency to fix build failures.
### 0.12.1
- [fixed] Fix a case where Err(TooNarrow) was returned unnecessarily. (thanks sftse)
- [added] Add new rendering options `Config::raw_mode()` and
`Config::no_table_borders()` (thanks sftse)
- [changed] Formatting, clippy and other tidy-ups (thanks sftse)
- [changed] Cargo fmt now enforced in CI
### 0.12.0
- [changed] Updated termion dev-dependency
- [added] Support `<sup>` HTML elements
- [added] Export `RcDom` publically. It was already returned by a pub function.
- [added] Update handling of width overflow:
With `Config::allow_width_overflow()`, prefer returning output wider
than requested, instead of returning `Err(TooNarrow)`.
`Config::min_wrap_width()` sets the minimum text wrap width (default
3). The minimum width (before overflow or `TooNarrow`) is now
handled more cleanly.
- [added] CSS: use color/bgcolor attributes on elements.
### 0.11.0
- [fixed] CSS: rules marked !important were ignored.
- [changed] html\_trace feature now uses the `log` crate.
- [changed] Bumped MSRV to 1.63 (matching Debian stable) due to some dependencies.
### 0.10.3
- [fixed] A panic on some unlucky text wrapping coincidences.
- [fixed] Use dep:backtrace in Cargo.toml to avoid implicit feature.
### 0.10.2
- [fixed] CSS: Ignore transparent colours.
### 0.10.1
- [fixed] `max_width` was not working with some render methods.
### 0.10.0
- [added] Simple support for `<i>`, `<ins>`, and `<del>` (thanks sgtatham)
- [added] Added background-color support
- [fixed] CSS support didn't work in some places, such as `<td>` elements.
- [added] Add support for `style` attributes.
- [added] Styles apply to table borders
- [changed] Update some dependencies
- [fixed] Fix a few places which caused excess blank lines or empty tables
### 0.9.4
- [changed] Updated the termion dev-dependency to 2.0.
### 0.9.3
- [changed] Added cargo categories and update to 2021 edition.
### 0.9.2
- [fixed] CSS didn't work inside `<ul>` or `<ol>`.
- [added] Add methods to get and use the intermediate HTML DOM and RenderTree
from Config.
- [fixed] Removed some clones which are no longer necessary now that Box<FnOnce>
works.
### 0.9.1
- [fixed] Various documentation issues (thanks sgtatham)
- [changed] CSS color rules now work for elements other than span.
### 0.9.0
- [changed] `Config::add_css` now returns `Result` instead of panicking on
CSS parse errors. Errors from parsing document CSS are ignored.
- [added] Support `<font color=...>` when CSS is enabled.
- [added] `Config::max_wrap_width()` to wrap text to a norrower width than
the overal size available.
- [added] Add --wrap-width and --css options to html2text example.
### 0.8.0
- [added] CSS: Support more extensive selectors
- [changed] CSS handling defaults to off; use `Config::use_doc_css()`
or `Config::add_css` to use CSS.
### 0.7.1
- [added] Now recognised CSS `display:none`
- [added] Can now add extra CSS rules via `Config::add_css`.
- [changed] StyleData::coloured is no longer public.
### 0.7.0
- [changed] Remove some noisy stderr output when encoutering control chars
(thanks sftse)
- [added] A builder-based config API.
- [changed] Updated MSRV to 1.60
- [fixed] Fixed #88: panic when a width of zero passed in (thanks bingen13)
- [fixed] Fixed #90: Fixed a divide-by-zero panic with colspan=0 (thanks mtorromeo)
- [added] Add very basic CSS colour support (under the css feature flag)
- [changed] Removed ansi\_colours feature (from\_read\_coloured is always available)
- [changed] Overhauled error handling. Internally (and in the lower level
API) errors (mainly "TooNarrow") are passed around with `Result`. Fixed
some panics and infinite loops. (Thanks WIZeaz for fuzzing)
### 0.6.0
- [changed] Improve layout of tables thanks to sftse:
- Table column size estimates have been improved when the source HTML has a lot
of unnecessary whitespace.
- Move the URL footnotes out to the top level, also improving layout of tables
containing links.
- [changed] Some APIs have slightly changed as part of the table improvements,
though most users should not be affeted.
### 0.5.1
- [fixed] Some tables were rendered too wide.
### 0.5.0
- [changed] Rich Image annotations now include the src attirbute (thanks spencerwi).
### 0.4.5
- [fixed] Preserve empty lines in pre blocks (thanks kpagacz).
### 0.4.4
- [fixed] Fix some panics when enumerated lists are in tables (thanks sfts).
- [fixed] Impove table size estimation to include links.
### 0.4.3
- [changed] MSRV is now 1.56.
- [fixed] Fix some panics when very large widths are used with tables.
### 0.4.2
- [changed] Moved the rcdom module directly into src/
### 0.4.1 (unpublished)
- [changed] rcdom now vendored as a module.
### 0.4.0 (unpublished)
- [changed] Update html5ever to v0.26.
- [changed] MSRV is now 1.49.
### 0.3.1
- [changed] Update the build badges to reflect the updated CI configuration.
### 0.3.0
- [added] New experimental `from_read_coloured()` (under `ansi_colours` feature).
- [added] Add `into_tagged_strings` and `tagged_strings` methods to `TaggedLine`
(thanks Robin Krahl)
- [added] Add `width` method to `TaggedString` (thanks Robin Krahl)
- [changed] Keep annotations in `TextRenderer::into_lines` (thanks Robin Krahl)
- [fixed] Add colon to reference style link (thanks zakaluka)
- [added] Allow text decorators to customise block prefix strings (thanks SardineFish)
- [fixed] Fixed some problems rendering some complicated tables, including a panic
and near-infinite loops.
- [changed] Tables which are too wide to possibly render in the given width are now
arranged vertically instead (with `///`) lines.
- [changed] A number of small table rendering improvements.
- [changed] MSRV is now 1.41.
### 0.2.1
- [added] New entry points - split HTML parsing from rendering the output,
thanks Robin Krahl.
- [fixed] Decorators weren't being used for preformatted text.
### 0.2.0
- [added] Support `<s>` strikeout text.
### 0.1.14 (2020-08-07)
- [fixed] A table with an `id` attribute on `<tbody>` would be hidden.
### 0.1.13 (2020-07-21)
- [changed] Run cargo fmt (thanks crunchyjesus)
- [added] CHANGELOG.md
- [fixed] Some text near a fragment start (`id="foo"` attribute) could be
lost if it needed breaking across lines.
- [added] Experimentally add dependabot configuration.
================================================
FILE: Cargo.toml
================================================
[package]
name = "html2text"
description = "Render HTML as plain text."
readme = "README.md"
documentation = "https://docs.rs/html2text/"
categories = ["text-processing"]
keywords = ["html", "text"]
version.workspace = true
repository.workspace = true
license.workspace = true
edition.workspace = true
rust-version.workspace = true
[workspace]
members = [
".",
"html2text-cli",
]
[workspace.package]
version = "0.17.1"
repository = "https://github.com/jugglerchris/rust-html2text/"
license = "MIT"
authors = ["Chris Emerson <github@mail.nosreme.org>"]
edition = "2024"
rust-version = "1.85"
[dependencies]
html5ever = "0.39.0"
tendril = "0.5"
unicode-width = "0.2"
backtrace = { version = "0.3", optional=true }
thiserror = "2.0.0"
log = { version = "0.4.20", optional = true }
nom = { version = "8.0.0", optional = true }
xml5ever = { version = "0.39.0", optional = true }
[features]
html_trace = ["dep:log"]
html_trace_bt = ["html_trace", "dep:backtrace"]
default = []
css = [ "dep:nom" ]
css_ext = ["css"]
xml = ["dep:xml5ever"]
[[example]]
name = "html2term"
path = "examples/html2term.rs"
[dev-dependencies]
env_logger = "0.11.6"
argparse = "0.2.2"
log = "0.4.20"
[target.'cfg(unix)'.dev-dependencies]
termion = "4.0"
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2016 Chris Emerson
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
[](https://app.circleci.com/pipelines/github/jugglerchris/rust-html2text?filter=all)
# html2text
html2text is a [Rust](http://www.rust-lang.org/) crate which converts HTML to
plain text (as in Rust `String`) or text spans with annotations like colours,
e.g. optionally using CSS. See [the online demo](https://jugglerchris.github.io/rust-html2text/)
for examples of the output.
`cargo install html2text-cli` will install the `html2text` CLI wrapper.
It makes use of the [Servo project](https://github.com/servo/servo)'s HTML
parser, [html5ever](https://github.com/servo/html5ever/), using the DOM to
generate text (which can optionally include annotations for some features such
as hyperlinks).
The project aims to do a reasonable job of rendering reasonable HTML in a
terminal or other places where HTML needs to be converted to text (for
example the text/plain fallback in HTML e-mails).
With feature flags enabled (see below) some CSS/colour support is available.
## Examples
The simple functions like `from_read()` return formatted text (in various
formats including plain text).
```rust
use html2text::from_read;
let html = b"
<ul>
<li>Item one</li>
<li>Item two</li>
<li>Item three</li>
</ul>";
assert_eq!(from_read(&html[..], 20).unwrap(),
"\
* Item one
* Item two
* Item three
");
```
A lower level API gives a bit more control. This give the same result (except for
returning errors as Result instead of panicking):
```rust
use html2text::config;
let html = b"
<ul>
<li>Item one</li>
<li>Item two</li>
<li>Item three</li>
</ul>";
assert_eq!(
config::plain()
.string_from_read(&html[..], 20)
.unwrap(),
"\
* Item one
* Item two
* Item three
");
```
A couple of simple demonstration programs are included as examples:
### html2text
The simplest example uses `from_read` to convert HTML on stdin into plain
text:
```sh
$ cargo run --example html2text < foo.html
[...]
```
### html2term
A very simple example of using the rich interface (`from_read_rich`) for a
slightly interactive console HTML viewer is provided as `html2term`.
```sh
$ cargo run --example html2term foo.html
[...]
```
Note that this example takes the HTML file as a parameter so that it can
read keys from stdin.
## Cargo Features
|Feature| Description|
|-------|------------|
|css | Limited handling of CSS, adding Coloured nodes to the render tree. |
|css\_ext| Some CSS extensions (see below for details) |
|xml | Support XHTML (for cases where a document may be parsed differently than as HTML). |
|html\_trace| Add verbose internal logging (not recommended) |
|html\_trace\_bt| Add backtraces to the verbose internal logging |
### CSS support
When the `css` feature is enabled, some simple CSS handling is available.
Style rules are taken from:
* If `Config::use_doc_css()` is called, then style from the document:
* `<style>` elements
* Inline `style` attributes (`<div style="...">`)
* `<font color=...>`
* Independently of `use_doc_css`, extra rules can be added with `Config::add_css(...)`
The following CSS features are implemented:
* Basic selector matching (including child and descendents, classes and element
types).
* Attribute selectors including `[foo]` and `[foo="bar"]`
* `nth-child` pseudo-class.
* CSS colors (`color`/`background-color`) will add
`Coloured(...)`/`BgColoured(...)` nodes to the render tree.
* Rules with `display: none` will cause matching elements to be removed from
the render tree. The same is true if `overflow: hidden` or `overflow-y: hidden` and
the `height` or `max-height` are 0.
* `content: "text"` inside a `::before`/`::after`
* `white-space` values "normal", "pre", and "prewrap"
The CSS handling is expected to improve in future (PRs welcome), but not to a full-
blown browser style system, which would be overkill for terminal output.
There are two ways to make use of the colours:
* Use `from_read_rich()` or one of its variants. One of the annotations you may get
back is `Colour(..)`.
* Use `from_read_coloured()`. This is similar to `from_read()`, but you provide
a function to add terminal colours (or other styling) based on the same
RichAnnotations. See examples/html2text.rs for an example using termion.
### CSS extensions (`css\_ext` Cargo feature)
The following CSS extensions are implemented:
* `x-syntax: foo;`
- Syntax-highlight with language "foo". A highlighter needs to be registered
for that language with `config::register_highlighter()` - see the
`html2text-cli` for an example using `syntect`.
* `display: x-raw-dom;`
- Show the HTML elements instead of rendering them. (Useful for debugging, along
with something like `:nth-child(...)` to select a particular node)
### XML/XHTML support
In some rare cases, parsing an XHTML document as HTML behaves differently. For example:
```xml
<?xml version="1.0"?>
<html xmlns="http://www.w3.org/1999/xhtml">
<body>
<h1/>
<p>Not Heading</p>
</body>
</html>
```
HTML does not have the self-closing `<h1/>` form, so this would parse as an
unclosed `<h1>` element, containing the following `<p>`, but in XHTML this is parsed as an empty `<h1>` followed by a `<p>`.
With the `xml` Cargo feature enabled, by default documents are parsed as XHTML
if they start with the `<?xml?>` declaration and HTML otherwise. This can be
configured with `Config::xml_mode()`.
================================================
FILE: benches/tables.rs
================================================
#![feature(test)]
extern crate html2text;
extern crate test;
use ::test::Bencher;
use html2text::from_read;
fn make_html(content: &str) -> String {
String::from("<html>") + content + "</html>"
}
fn make_tab(cell: &str, rows: usize, cols: usize) -> String {
let mut result = String::from("<table>");
for _ in 0..rows {
result.push_str("<tr>");
for _ in 0..cols {
result.push_str("<td>");
result.push_str(cell);
result.push_str("</td>");
}
result.push_str("</tr>");
}
result
}
#[bench]
fn bench_empty(b: &mut Bencher) {
b.iter(|| from_read(make_html("").as_bytes(), 80));
}
#[bench]
fn bench_tab_1_1(b: &mut Bencher) {
b.iter(|| from_read(make_html(&make_tab("cell", 1, 1)).as_bytes(), 80));
}
#[bench]
fn bench_tab_2_2(b: &mut Bencher) {
b.iter(|| from_read(make_html(&make_tab("cell", 2, 2)).as_bytes(), 80));
}
#[bench]
fn bench_tab_3_3(b: &mut Bencher) {
b.iter(|| from_read(make_html(&make_tab("cell", 3, 3)).as_bytes(), 80));
}
#[bench]
fn bench_tab_4_4(b: &mut Bencher) {
b.iter(|| from_read(make_html(&make_tab("cell", 4, 4)).as_bytes(), 80));
}
#[bench]
fn bench_tab_5_5(b: &mut Bencher) {
b.iter(|| from_read(make_html(&make_tab("cell", 5, 5)).as_bytes(), 80));
}
#[bench]
fn bench_tab_6_6(b: &mut Bencher) {
b.iter(|| from_read(make_html(&make_tab("cell", 6, 6)).as_bytes(), 80));
}
// Try a table with `depth` nested tables each with `rows` rows and `cols` columns.
fn bench_tab_depth(b: &mut Bencher, content: &str, depth: usize, rows: usize, cols: usize) {
let mut t = String::from(content);
for _ in 0..depth {
t = make_tab(&t, rows, cols);
}
let html = make_html(&t);
b.iter(|| from_read(html.as_bytes(), 80));
}
#[bench]
fn bench_tab_2_1_depth_2(b: &mut Bencher) {
bench_tab_depth(b, "cell", 2, 2, 1);
}
#[bench]
fn bench_tab_3_1_depth_2(b: &mut Bencher) {
bench_tab_depth(b, "cell", 2, 3, 1);
}
#[bench]
fn bench_tab_4_1_depth_2(b: &mut Bencher) {
bench_tab_depth(b, "cell", 2, 4, 1);
}
#[bench]
fn bench_tab_1_2_depth_2(b: &mut Bencher) {
bench_tab_depth(b, "cell", 2, 1, 2);
}
#[bench]
fn bench_tab_1_3_depth_2(b: &mut Bencher) {
bench_tab_depth(b, "cell", 2, 1, 3);
}
#[bench]
fn bench_tab_1_4_depth_2(b: &mut Bencher) {
bench_tab_depth(b, "cell", 2, 1, 4);
}
#[bench]
fn bench_tab_2_depth_2(b: &mut Bencher) {
bench_tab_depth(b, "cell", 2, 2, 2);
}
/*
#[bench]
fn bench_tab_2_depth_3(b: &mut Bencher) {
bench_tab_depth(b, "cell", 3, 2, 2);
}
#[bench]
fn bench_tab_2_depth_4(b: &mut Bencher) {
bench_tab_depth(b, "cell", 4, 2, 2);
}
#[bench]
fn bench_tab_2_depth_5(b: &mut Bencher) {
bench_tab_depth(b, "cell", 5, 2, 2);
}
*/
================================================
FILE: examples/html2term.rs
================================================
#[cfg(unix)]
extern crate argparse;
#[cfg(unix)]
extern crate unicode_width;
#[cfg(unix)]
mod top {
#[cfg(feature = "css")]
use argparse::StoreFalse;
use argparse::{ArgumentParser, Store};
use html2text::render::{RichAnnotation, TaggedLine, TaggedLineElement};
use std::collections::HashMap;
use std::io::{self, Write};
use termion::cursor::Goto;
use termion::event::Key;
use termion::input::TermRead;
use termion::raw::IntoRawMode;
use termion::screen::IntoAlternateScreen;
use unicode_width::UnicodeWidthStr;
fn to_style(tag: &[RichAnnotation]) -> String {
use termion::color::*;
let mut style = String::new();
for ann in tag {
match *ann {
RichAnnotation::Default => (),
RichAnnotation::Link(_) => {
style.push_str(&format!("{}", termion::style::Underline));
}
RichAnnotation::Image(_) => {
style.push_str(&format!("{}", Fg(LightBlue)));
}
RichAnnotation::Emphasis => {
style.push_str(&format!("{}", Fg(LightGreen)));
}
RichAnnotation::Strong => {
style.push_str(&format!("{}", Fg(LightGreen)));
}
RichAnnotation::Strikeout => (),
RichAnnotation::Code => {
style.push_str(&format!("{}", Fg(LightYellow)));
}
RichAnnotation::Preformat(is_cont) => {
if is_cont {
style.push_str(&format!("{}", Fg(LightMagenta)));
} else {
style.push_str(&format!("{}", Fg(Magenta)));
}
}
RichAnnotation::Colour(col) => {
style.push_str(&format!("{}", Fg(Rgb(col.r, col.g, col.b))));
}
RichAnnotation::BgColour(col) => {
style.push_str(&format!("{}", Bg(Rgb(col.r, col.g, col.b))));
}
_ => todo!(),
}
}
style
}
struct LinkMap {
lines: Vec<Vec<Option<String>>>, // lines[y][x] => Some(URL) or None
}
impl LinkMap {
pub fn link_at(&self, x: usize, y: usize) -> Option<&str> {
if let Some(linevec) = self.lines.get(y) {
if let Some(Some(text)) = linevec.get(x) {
return Some(text);
}
}
None
}
}
fn link_from_tag(tag: &[RichAnnotation]) -> Option<String> {
let mut link = None;
for annotation in tag {
if let RichAnnotation::Link(text) = annotation {
link = Some(text.clone());
}
}
link
}
fn find_links(lines: &[TaggedLine<Vec<RichAnnotation>>]) -> LinkMap {
let mut map = Vec::new();
for line in lines {
let mut linevec = Vec::new();
for ts in line.tagged_strings() {
let link = link_from_tag(&ts.tag);
for _ in 0..UnicodeWidthStr::width(ts.s.as_str()) {
linevec.push(link.clone());
}
}
map.push(linevec);
}
LinkMap { lines: map }
}
struct FragMap {
start_xy: HashMap<String, (usize, usize)>,
}
fn find_frags(lines: &[TaggedLine<Vec<RichAnnotation>>]) -> FragMap {
use self::TaggedLineElement::*;
let mut map = HashMap::new();
for (y, line) in lines.iter().enumerate() {
let mut x = 0;
for tli in line.iter() {
match tli {
FragmentStart(fragname) => {
map.insert(fragname.to_string(), (x, y));
}
Str(ts) => {
x += UnicodeWidthStr::width(ts.s.as_str());
}
}
}
}
FragMap { start_xy: map }
}
struct Options {
#[cfg(feature = "css")]
use_css: bool,
}
impl Options {
fn new() -> Options {
Options {
#[cfg(feature = "css")]
use_css: true,
}
}
}
pub fn main() {
let mut filename = String::new();
#[allow(unused_mut)]
let mut options = Options::new();
{
let mut ap = ArgumentParser::new();
ap.refer(&mut filename)
.add_argument("filename", Store, "Set HTML filename");
#[cfg(feature = "css")]
ap.refer(&mut options.use_css)
.add_option(&["--no-css"], StoreFalse, "Disable CSS");
ap.parse_args_or_exit();
}
let (width, height) = termion::terminal_size().unwrap();
let (width, height) = (width as usize, height as usize);
let mut file = std::fs::File::open(filename).expect("Tried to open file");
let dom = html2text::config::plain()
.parse_html(&mut file)
.expect("Failed to parse HTML");
let mut keys = io::stdin().keys();
// top_y is the (0-based) index of the document line shown at
// the top of the visible screen.
let mut top_y = 0;
// doc_x and doc_y are the logical (0-based) x and y of the
// cursor position within the document.
let mut doc_x = 0;
let mut doc_y = 0;
let mut screen = io::stdout()
.into_raw_mode()
.unwrap()
.into_alternate_screen()
.unwrap();
let mut annotated = rerender(&dom, &[], width, &options);
let link_map = find_links(&annotated);
let frag_map = find_frags(&annotated);
let mut inspect_path = vec![];
loop {
// max_y is the largest (0-based) index of a real document line.
let max_y = annotated.len().saturating_sub(1);
// Sanity-check the current screen position. max_y should
// be small enough that no blank lines beyond the end of
// the document are visible on screen (except when the
// document is shorter than a screenful); large enough
// that the cursor isn't off the bottom of the visible
// screen; and small enough that the cursor isn't off the
// top.
if max_y >= height - 1 {
top_y = std::cmp::min(top_y, max_y - (height - 1));
}
if doc_y >= height - 1 {
top_y = std::cmp::max(top_y, doc_y - (height - 1));
}
top_y = std::cmp::min(top_y, doc_y);
let opt_url = link_map.link_at(doc_x, doc_y);
let mut vis_y_limit = std::cmp::min(top_y + height, max_y + 1);
if !inspect_path.is_empty() {
vis_y_limit -= 1;
}
write!(screen, "{}", termion::clear::All).unwrap();
for (i, line) in annotated[top_y..vis_y_limit].iter().enumerate() {
write!(screen, "{}", Goto(1, i as u16 + 1)).unwrap();
for ts in line.tagged_strings() {
let style = to_style(&ts.tag);
let link = link_from_tag(&ts.tag);
match (opt_url, link) {
(Some(ref t1), Some(ref t2)) if t1 == t2 => {
write!(screen, "{}", termion::style::Invert).unwrap();
}
_ => (),
}
write!(screen, "{}{}{}", style, ts.s, termion::style::Reset).unwrap();
}
}
if !inspect_path.is_empty() {
let mut pth = String::from("top ");
let mut node = dom.document.clone();
for &idx in &inspect_path {
node = node.nth_child(idx).unwrap();
pth.push_str(&format!("> {}", node.element_name().unwrap()));
}
write!(
screen,
"{}{}{:?}",
Goto(1, vis_y_limit as u16),
pth,
&inspect_path
)
.unwrap();
}
// 1-based screen coordinates
let cursor_x = (doc_x + 1) as u16;
let cursor_y = (doc_y - top_y + 1) as u16;
write!(screen, "{}", Goto(cursor_x, cursor_y)).unwrap();
screen.flush().unwrap();
if let Some(Ok(k)) = keys.next() {
match k {
Key::Char('q') => break,
Key::Char('j') | Key::Down => {
if inspect_path.is_empty() {
if doc_y < max_y {
doc_y += 1;
}
} else {
*inspect_path.last_mut().unwrap() += 1;
if dom.get_node_by_path(&inspect_path).is_none() {
// No next node - undo.
*inspect_path.last_mut().unwrap() -= 1;
} else {
annotated = rerender(&dom, &inspect_path, width, &options);
}
}
}
Key::Char('k') | Key::Up => {
if inspect_path.is_empty() {
doc_y = doc_y.saturating_sub(1);
} else if *inspect_path.last().unwrap() > 1 {
*inspect_path.last_mut().unwrap() -= 1;
annotated = rerender(&dom, &inspect_path, width, &options);
}
}
Key::Char('h') | Key::Left => {
if inspect_path.is_empty() {
doc_x = doc_x.saturating_sub(1);
} else if inspect_path.len() > 1 {
inspect_path.pop();
annotated = rerender(&dom, &inspect_path, width, &options);
}
}
Key::Char('l') | Key::Right => {
if inspect_path.is_empty() {
if doc_x + 1 < width {
doc_x += 1;
}
} else {
inspect_path.push(1);
if dom.get_node_by_path(&inspect_path).is_none() {
inspect_path.pop();
} else {
annotated = rerender(&dom, &inspect_path, width, &options);
}
}
}
Key::Char(' ') | Key::PageDown => {
// Ideally, move both the cursor and the top
// visible line down by a whole page
doc_y += height;
top_y += height;
// But bound the cursor within the document
doc_y = std::cmp::min(doc_y, max_y);
// And the standard bounds checking for top_y
// will take care of the rest of the special
// cases.
}
Key::PageUp => {
// Ideally, move both the cursor and the top
// visible line up by a whole page. But bound
// both at zero.
doc_y = std::cmp::max(doc_y, height) - height;
top_y = std::cmp::max(top_y, height) - height;
}
Key::Home => {
doc_y = 0;
}
Key::End => {
doc_y = max_y;
}
Key::Char('\t') => {}
Key::Char('\r') | Key::Char('\n') => {
if let Some(url) = opt_url {
if let Some(u) = url.strip_prefix('#') {
let start = frag_map.start_xy.get(u);
if let Some((x, y)) = start {
doc_x = *x;
doc_y = *y;
}
}
}
}
#[cfg(feature = "css_ext")]
Key::Char('I') => {
// Enter/leave inspect mode
if inspect_path.is_empty() {
inspect_path.push(1);
} else {
inspect_path.clear();
}
annotated = rerender(&dom, &inspect_path, width, &options);
}
_ => {}
}
}
}
}
fn rerender(
dom: &html2text::RcDom,
inspect_path: &[usize],
width: usize,
#[allow(unused)] options: &Options,
) -> Vec<TaggedLine<Vec<RichAnnotation>>> {
let config = html2text::config::rich();
#[cfg(feature = "css")]
let config = if options.use_css {
config
.use_doc_css()
.add_agent_css(
r#"
img {
color: #77f;
}
"#,
)
.unwrap()
} else {
config
};
if inspect_path.is_empty() {
let render_tree = config
.dom_to_render_tree(dom)
.expect("Failed to build render tree");
config
.render_to_lines(render_tree, width)
.expect("Failed to render")
} else {
#[cfg(feature = "css_ext")]
{
let mut path_selector = String::new();
for &idx in &inspect_path[1..] {
path_selector.push_str(&format!(" > :nth-child({})", idx));
}
let config = config
.add_agent_css(
&(format!(
r#"
html {} {{
color: white !important;
background-color: black !important;
display: x-raw-dom;
}}
"#,
path_selector
)),
)
.expect("Invalid CSS");
let render_tree = config
.dom_to_render_tree(dom)
.expect("Failed to build render tree");
config
.render_to_lines(render_tree, width)
.expect("Failed to render")
}
#[cfg(not(feature = "css_ext"))]
unreachable!()
}
}
}
#[cfg(not(unix))]
mod top {
pub fn main() {}
}
fn main() {
top::main()
}
================================================
FILE: html2text-cli/Cargo.toml
================================================
[package]
name = "html2text-cli"
description = "Render HTML as plain text."
readme = "README.md"
categories = ["text-processing"]
keywords = ["html", "text"]
default-run = "html2text"
version.workspace = true
repository.workspace = true
license.workspace = true
edition.workspace = true
rust-version.workspace = true
[[bin]]
name = "html2text"
path = "src/main.rs"
[dependencies]
html2text = { version = "0.17.0", path = ".." }
argparse = "0.2.2"
log = "0.4.20"
syntect = "5.2.0"
[target.'cfg(unix)'.dependencies]
termion = "4.0"
[features]
default = ["css"]
css = ["html2text/css"]
css_ext = ["html2text/css_ext"]
html_trace = ["html2text/html_trace"]
html_trace_btr = ["html2text/html_trace_bt"]
xml = ["html2text/xml"]
================================================
FILE: html2text-cli/README.md
================================================
# html2text
A command-line tool to convert HTML into text in various forms. Supports some
CSS including colours with default features.
The default, which gives text with ASCII annotations similar to (but not
exactly) markdown for things like `<strong>` elements and lists, and table
formatting. It defaults to reading from standard input and writing to standard
output.
```sh
html2text foo.html
```
For a more fancy terminal output, including ANSI colours, syntax highlighting
using syntect (`css_ext` cargo feature required) and added CSS rules:
```sh
html2text --colour --css --syntax --agent-css 'pre.rust { x-syntax: rs; }' < file_with_rust_code.html
```
================================================
FILE: html2text-cli/src/main.rs
================================================
extern crate argparse;
extern crate html2text;
use argparse::{ArgumentParser, Store, StoreOption, StoreTrue};
use html2text::config::{self, Config};
use html2text::render::{TextDecorator, TrivialDecorator};
use log::trace;
use std::io;
use std::io::Write;
#[cfg(unix)]
use html2text::render::RichAnnotation;
#[cfg(unix)]
fn default_colour_map(
annotations: &[RichAnnotation],
s: &str,
use_css_colours: bool,
no_default_colours: bool,
) -> String {
use RichAnnotation::*;
use termion::color::*;
// Explicit CSS colours override any other colours
let mut have_explicit_colour = no_default_colours;
let mut start = Vec::new();
let mut finish = Vec::new();
trace!("default_colour_map: str={s}, annotations={annotations:?}");
for annotation in annotations.iter() {
match annotation {
Default => {}
Link(_) => {
start.push(format!("{}", termion::style::Underline));
finish.push(format!("{}", termion::style::Reset));
}
Image(_) => {
if !have_explicit_colour {
start.push(format!("{}", Fg(Blue)));
finish.push(format!("{}", Fg(Reset)));
}
}
Emphasis => {
start.push(format!("{}", termion::style::Bold));
finish.push(format!("{}", termion::style::Reset));
}
Strong => {
if !have_explicit_colour {
start.push(format!("{}", Fg(LightYellow)));
finish.push(format!("{}", Fg(Reset)));
}
}
Strikeout => {
if !have_explicit_colour {
start.push(format!("{}", Fg(LightBlack)));
finish.push(format!("{}", Fg(Reset)));
}
}
Code => {
if !have_explicit_colour {
start.push(format!("{}", Fg(Blue)));
finish.push(format!("{}", Fg(Reset)));
}
}
Preformat(_) => {
if !have_explicit_colour {
start.push(format!("{}", Fg(Blue)));
finish.push(format!("{}", Fg(Reset)));
}
}
Colour(c) => {
if use_css_colours {
start.push(format!("{}", Fg(Rgb(c.r, c.g, c.b))));
finish.push(format!("{}", Fg(Reset)));
have_explicit_colour = true;
}
}
BgColour(c) => {
if use_css_colours {
start.push(format!("{}", Bg(Rgb(c.r, c.g, c.b))));
finish.push(format!("{}", Bg(Reset)));
}
}
_ => {}
}
}
// Reverse the finish sequences
finish.reverse();
let mut result = start.join("");
result.push_str(s);
for s in finish {
result.push_str(&s);
}
trace!("default_colour_map: output={result}");
result
}
#[cfg(feature = "css_ext")]
fn do_syntect_highlight<'t>(text: &'t str, language: &str) -> Vec<(html2text::TextStyle, &'t str)> {
use html2text::{Colour, TextStyle};
use syntect::{
easy::HighlightLines, highlighting::ThemeSet, parsing::SyntaxSet, util::LinesWithEndings,
};
let ps = SyntaxSet::load_defaults_newlines();
let ts = ThemeSet::load_defaults();
let syntax = ps.find_syntax_by_extension(&language).unwrap();
let mut h = HighlightLines::new(syntax, &ts.themes["Solarized (dark)"]);
let mut results = Vec::new();
for line in LinesWithEndings::from(&text) {
let ranges: Vec<(syntect::highlighting::Style, &str)> =
h.highlight_line(line, &ps).unwrap();
fn convert(c: syntect::highlighting::Color) -> Colour {
Colour {
r: c.r,
g: c.g,
b: c.b,
}
}
for (sty, text) in ranges {
results.push((
TextStyle::colours(convert(sty.foreground), convert(sty.background)),
text,
));
}
}
results
}
fn update_config<T: TextDecorator>(mut config: Config<T>, flags: &Flags) -> Config<T> {
if let Some(wrap_width) = flags.wrap_width {
config = config.max_wrap_width(wrap_width);
}
#[cfg(feature = "css")]
if flags.use_css {
config = config.use_doc_css();
}
#[cfg(feature = "css")]
if !flags.agent_css.is_empty() {
config = config.add_agent_css(&flags.agent_css).expect("Invalid CSS");
}
#[cfg(feature = "css_ext")]
if flags.syntax_highlight {
config = config
.register_highlighter("rs", Box::new(|text| do_syntect_highlight(text, "rs")))
.register_highlighter("html", Box::new(|text| do_syntect_highlight(text, "html")));
}
match (flags.link_footnotes, flags.no_link_footnotes) {
(true, true) => {
eprintln!("Error: can't specify both --link-footnotes and --no-link-footnotes");
std::process::exit(1);
}
(true, false) => config = config.link_footnotes(true),
(false, true) => config = config.link_footnotes(false),
(false, false) => {}
};
if flags.pad_width {
config = config.pad_block_width();
}
config
}
fn translate<R>(input: R, flags: Flags, literal: bool) -> String
where
R: io::Read,
{
#[cfg(unix)]
{
if flags.use_colour {
let conf = config::rich();
let conf = update_config(conf, &flags);
#[cfg(feature = "css_ext")]
let conf = if flags.show_dom {
conf.add_agent_css("body { display: x-raw-dom !important; }")
.unwrap()
} else {
conf
};
#[cfg(feature = "css")]
let use_css_colours = !flags.ignore_css_colours;
#[cfg(not(feature = "css"))]
let use_css_colours = false;
#[cfg(feature = "css")]
let use_only_css = flags.use_only_css;
#[cfg(not(feature = "css"))]
let use_only_css = false;
return conf
.coloured(input, flags.width, move |anns, s| {
default_colour_map(anns, s, use_css_colours, use_only_css)
})
.unwrap();
}
}
#[cfg(feature = "css")]
{
if flags.show_css {
let conf = config::plain();
let conf = update_config(conf, &flags);
let dom = conf.parse_html(input).unwrap();
return html2text::dom_to_parsed_style(&dom).expect("Parsing CSS");
}
}
if flags.show_dom {
let conf = config::plain();
let conf = update_config(conf, &flags);
let dom = conf.parse_html(input).unwrap();
dom.as_dom_string()
} else if flags.show_render {
let conf = config::plain();
let conf = update_config(conf, &flags);
let dom = conf.parse_html(input).unwrap();
let rendertree = conf.dom_to_render_tree(&dom).unwrap();
rendertree.to_string()
} else if literal {
let conf = config::with_decorator(TrivialDecorator::new());
let conf = update_config(conf, &flags);
conf.string_from_read(input, flags.width).unwrap()
} else {
let conf = config::plain();
let conf = update_config(conf, &flags);
conf.string_from_read(input, flags.width).unwrap()
}
}
#[derive(Debug)]
struct Flags {
width: usize,
wrap_width: Option<usize>,
#[allow(unused)]
use_colour: bool,
#[cfg(feature = "css")]
use_css: bool,
#[cfg(feature = "css")]
ignore_css_colours: bool,
#[cfg(feature = "css")]
use_only_css: bool,
show_dom: bool,
show_render: bool,
#[cfg(feature = "css")]
show_css: bool,
pad_width: bool,
link_footnotes: bool,
no_link_footnotes: bool,
#[cfg(feature = "css_ext")]
syntax_highlight: bool,
#[cfg(feature = "css")]
agent_css: String,
}
fn main() {
#[cfg(feature = "html_trace")]
env_logger::init();
let mut infile: Option<String> = None;
let mut outfile: Option<String> = None;
let mut flags = Flags {
width: 80,
wrap_width: None,
use_colour: false,
#[cfg(feature = "css")]
use_css: false,
#[cfg(feature = "css")]
ignore_css_colours: false,
#[cfg(feature = "css")]
use_only_css: false,
show_dom: false,
show_render: false,
#[cfg(feature = "css")]
show_css: false,
#[cfg(feature = "css")]
agent_css: Default::default(),
pad_width: false,
link_footnotes: false,
no_link_footnotes: false,
#[cfg(feature = "css_ext")]
syntax_highlight: false,
};
let mut literal: bool = false;
{
let mut ap = ArgumentParser::new();
ap.refer(&mut infile).add_argument(
"infile",
StoreOption,
"Input HTML file (default is standard input)",
);
ap.refer(&mut flags.width).add_option(
&["-w", "--width"],
Store,
"Column width to format to (default is 80)",
);
ap.refer(&mut flags.wrap_width).add_option(
&["-W", "--wrap-width"],
StoreOption,
"Maximum text wrap width (default same as width)",
);
ap.refer(&mut outfile).add_option(
&["-o", "--output"],
StoreOption,
"Output file (default is standard output)",
);
ap.refer(&mut literal).add_option(
&["-L", "--literal"],
StoreTrue,
"Output only literal text (no decorations)",
);
ap.refer(&mut flags.pad_width).add_option(
&["--pad-width"],
StoreTrue,
"Pad blocks to their full width",
);
ap.refer(&mut flags.link_footnotes).add_option(
&["--link-footnotes"],
StoreTrue,
"Enable link footnotes",
);
ap.refer(&mut flags.no_link_footnotes).add_option(
&["--no-link-footnotes"],
StoreTrue,
"Enable link footnotes",
);
#[cfg(unix)]
ap.refer(&mut flags.use_colour).add_option(
&["--colour"],
StoreTrue,
"Use ANSI terminal colours",
);
#[cfg(feature = "css")]
ap.refer(&mut flags.use_css)
.add_option(&["--css"], StoreTrue, "Enable CSS");
#[cfg(feature = "css")]
ap.refer(&mut flags.ignore_css_colours)
.add_option(&["--ignore-css-colour"], StoreTrue, "With --css, ignore CSS colour information (still hides elements with e.g. display: none)");
#[cfg(feature = "css")]
ap.refer(&mut flags.use_only_css).add_option(
&["--only-css"],
StoreTrue,
"Don't use default non-CSS colours",
);
ap.refer(&mut flags.show_dom).add_option(
&["--show-dom"],
StoreTrue,
"Show the parsed HTML DOM instead of rendered output",
);
ap.refer(&mut flags.show_render).add_option(
&["--show-render"],
StoreTrue,
"Show the computed render tree instead of the rendered output",
);
#[cfg(feature = "css")]
ap.refer(&mut flags.show_css).add_option(
&["--show-css"],
StoreTrue,
"Show the parsed CSS instead of rendered output",
);
#[cfg(feature = "css")]
ap.refer(&mut flags.agent_css).add_option(
&["--agent-css"],
Store,
"Add some CSS rules (to the agent spreadsheet)",
);
#[cfg(feature = "css_ext")]
ap.refer(&mut flags.syntax_highlight).add_option(
&["--syntax"],
StoreTrue,
"Enable syntax highlighting of <pre> blocks.",
);
ap.parse_args_or_exit();
}
let data = match infile {
None => {
let stdin = io::stdin();
translate(&mut stdin.lock(), flags, literal)
}
Some(name) => {
let mut file = std::fs::File::open(name).expect("Tried to open file");
translate(&mut file, flags, literal)
}
};
match outfile {
None => {
print!("{}", data);
}
Some(name) => {
let mut file = std::fs::File::create(name).expect("Tried to create file");
write!(file, "{}", data).unwrap();
}
};
}
================================================
FILE: html2text-web-demo/.cargo/config.toml
================================================
[build]
target = "wasm32-wasip2"
================================================
FILE: html2text-web-demo/.gitignore
================================================
dist
================================================
FILE: html2text-web-demo/Cargo.toml
================================================
[package]
name = "html2text-web-demo"
version = "0.1.0"
edition = "2021"
[dependencies]
html2text = { path = "..", features = ["css"] }
ratzilla = "0.0.6"
wasm-bindgen = "0.2.100"
[lib]
crate-type = ["cdylib", "rlib"]
================================================
FILE: html2text-web-demo/Trunk.toml
================================================
trunk-version = "^0.21.13"
[build]
public_url = "/rust-html2text/"
release = true
filehash = false
inject_scripts = true #false
offline = false #true
frozen = true
minify = "on_release"
================================================
FILE: html2text-web-demo/index.html
================================================
<!doctype html>
<html>
<head>
<style>
body {
margin: 0;
}
pre {
margin: 0;
}
#main, #lib {
background-color: black;
}
#input_html {
width: 100%;
height: 300px;
}
</style>
<script>
function update_html() {
const text = document.getElementById("input_html").value;
const css = document.getElementById("conf_css").checked;
const colour = document.getElementById("conf_colour").checked;
let conf = wasmBindings.Config.new();
console.log("CSS:", css);
if (css) {
conf.use_css();
}
if (colour) {
conf.use_colour();
}
wasmBindings.format_html(conf, text);
}
window.addEventListener("TrunkApplicationStarted", update_html);
</script>
</head>
<body>
<h1>Html2text demo</h1>
<input type="checkbox" id="conf_css" checked=true onchange="update_html()">CSS</input>
<br>
<input type="checkbox" id="conf_colour" checked=true onchange="update_html()">Colour</input>
<br>
<textarea id="input_html" onchange="update_html()" oninput="update_html()"><html>
<style>
.green {
color: #4f4;
}
</style>
<body>
<h1>Hi there</h1>
<p>This is some simple text</p>
<ol>
<li>Item one</li>
<li><s>Item two</s></li>
<li class="green">Item three</li>
</ol>
<table>
<tr><th>Heading 1</th><th>Heading 2</th><th>Heading 3</th></tr>
<tr><td>Data 1</td><td>Data 2</td><td>Data 3</td></tr>
<tr><td colspan=3>Hello there</td></tr>
</table>
</body></html>
</textarea>
<br>
<button onclick="update_html()">Update</button>
<div id="lib"></div>
</body>
</html>
================================================
FILE: html2text-web-demo/src/lib.rs
================================================
use wasm_bindgen::prelude::wasm_bindgen;
use ratzilla::ratatui::{
style::{Color, Style, Stylize},
text::{Text, Line, Span},
widgets::{Block, Paragraph},
Frame,
Terminal,
};
use html2text::{
config::ImageRenderMode,
render::TextDecorator,
};
use ratzilla::DomBackend;
#[derive(Default)]
#[wasm_bindgen]
pub struct Config {
css: bool,
colour: bool,
user_css: Option<String>,
agent_css: Option<String>,
pad_block_width: bool,
wrap_width: Option<usize>,
allow_overflow: bool,
min_wrap_width: Option<usize>,
raw_mode: bool,
no_borders: bool,
no_link_wrap: bool,
unicode_so: bool,
do_decorate: bool,
link_footnotes: bool,
image_mode: ImageRenderMode,
}
#[wasm_bindgen]
impl Config {
pub fn new() -> Self {
Config {
..Default::default()
}
}
pub fn use_colour(&mut self) {
self.colour = true;
}
pub fn use_css(&mut self) {
self.css = true;
}
pub fn add_user_css(&mut self, css: String) {
if css.trim().is_empty() {
self.user_css = None;
} else {
self.user_css = Some(css);
}
}
pub fn add_agent_css(&mut self, css: String) {
if css.trim().is_empty() {
self.agent_css = None;
} else {
self.agent_css = Some(css);
}
}
pub fn pad_block_width(&mut self) {
self.pad_block_width = true;
}
pub fn max_wrap_width(&mut self, width: usize) {
self.wrap_width = Some(width);
}
pub fn allow_overflow(&mut self) {
self.allow_overflow = true;
}
pub fn min_wrap_width(&mut self, width: usize) {
self.min_wrap_width = Some(width);
}
pub fn raw_mode(&mut self) {
self.raw_mode = true;
}
pub fn no_borders(&mut self) {
self.no_borders = true;
}
pub fn no_link_wrap(&mut self) {
self.no_link_wrap = true;
}
pub fn unicode_so(&mut self) {
self.unicode_so = true;
}
pub fn do_decorate(&mut self) {
self.do_decorate = true;
}
pub fn link_footnotes(&mut self, value: bool) {
self.link_footnotes = value;
}
pub fn image_mode(&mut self, value: &str) {
match value {
"ignore" => self.image_mode = ImageRenderMode::IgnoreEmpty,
"always" => self.image_mode = ImageRenderMode::ShowAlways,
"replace" => self.image_mode = ImageRenderMode::Replace("XX"),
"filename" => self.image_mode = ImageRenderMode::Filename,
_ => self.image_mode = ImageRenderMode::IgnoreEmpty,
}
}
fn update_conf<D: TextDecorator>(&self, conf: html2text::config::Config<D>) -> Result<html2text::config::Config<D>, String> {
let mut conf = if self.css {
conf.use_doc_css()
} else {
conf
};
if let Some(user_css) = &self.user_css {
conf = conf.add_css(user_css).map_err(|e| format!("{}", e))?;
}
if let Some(agent_css) = &self.agent_css {
conf = conf.add_agent_css(agent_css).map_err(|e| format!("{}", e))?;
}
if self.pad_block_width {
conf = conf.pad_block_width();
}
if let Some(width) = self.wrap_width {
conf = conf.max_wrap_width(width);
}
if self.allow_overflow {
conf = conf.allow_width_overflow();
}
if let Some(width) = self.min_wrap_width {
conf = conf.min_wrap_width(width);
}
if self.raw_mode {
conf = conf.raw_mode(true);
}
if self.no_borders {
conf = conf.no_table_borders();
}
if self.no_link_wrap {
conf = conf.no_link_wrapping();
}
if self.unicode_so {
conf = conf.unicode_strikeout(true);
}
if self.do_decorate {
conf = conf.do_decorate();
}
conf = conf.link_footnotes(self.link_footnotes);
if self.image_mode != ImageRenderMode::IgnoreEmpty {
conf = conf.empty_img_mode(self.image_mode);
}
Ok(conf
.unicode_strikeout(false))
}
}
fn do_render_colour(f: &mut Frame, config: &Config, input: &[u8]) -> Result<(), String> {
let area = f.area();
let conf = config.update_conf(html2text::config::rich())?;
let lines = conf.lines_from_read(input, area.width as usize - 2).unwrap();
let mut out = Text::default();
for line in lines {
let mut term_line = Line::default();
for piece in line.tagged_strings() {
let span = Span::from(dbg!(piece.s.clone()));
let mut style = Style::new();
for attr in &piece.tag {
use html2text::render::RichAnnotation::*;
match attr {
Default | Link(_) | Image(_) | Code | Preformat(_) => {}
Emphasis => {
style = style.italic();
}
Strong => {
style = style.bold();
}
Strikeout => {
style = style.crossed_out();
}
Colour(col) => {
style = style.fg(Color::Rgb(col.r, col.g, col.b));
}
BgColour(col) => {
style = style.bg(Color::Rgb(col.r, col.g, col.b));
}
_ => {}
}
}
term_line.push_span(span.style(style));
}
out.push_line(term_line);
}
f.render_widget(
Paragraph::new(out).block(Block::bordered().title("HTML").border_style(Color::Yellow)),
f.area());
Ok(())
}
#[wasm_bindgen]
pub fn format_html(config: Config, input: &str) -> Result<(), String> {
let backend = DomBackend::new_by_id("lib").unwrap();
let mut terminal = Terminal::new(backend).unwrap();
let inp = input.to_string();
terminal.draw(move |f| {
if config.colour {
do_render_colour(f, &config, inp.as_bytes()).unwrap();
} else {
let area = f.area();
let conf = config.update_conf(html2text::config::plain()).unwrap();
let output = conf.string_from_read(inp.as_bytes(), area.width as usize).unwrap();
f.render_widget(
Paragraph::new(output),
f.area());
}
}).map_err(|e| format!("{e}"))?;
Ok(())
}
================================================
FILE: pages/.gitignore
================================================
_site
================================================
FILE: pages/_config.yml
================================================
lsi: false
safe: true
source: .
incremental: false
baseurl: "/rust-html2text"
gist:
noscript: false
theme: minima
github_username: jugglerchris
================================================
FILE: pages/_includes/head.html
================================================
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
{%- seo -%}
<link rel="stylesheet" href="{{ "/assets/main.css" | relative_url }}">
{%- feed_meta -%}
{%- if jekyll.environment == 'production' and site.google_analytics -%}
{%- include google-analytics.html -%}
{%- endif -%}
{%- if page.h2t_wasm -%}
<link rel=modulepreload src="{{"/assets/html2text-web-demo.js" | relative_url }}">
<link rel=preload src="{{"/assets/html2text-web-demo-bg.wasm" | relative_url }}" as="fetch" type="application/wasm" crossorigin="">
<link rel="stylesheet" href="{{ "/assets/demo.css" | relative_url }}">
{%- endif -%}
{%- if page.h2t_js -%}
<script src="{{ page.h2t_js | relative_url }}">
</script>
{%- endif -%}
</head>
================================================
FILE: pages/assets/demo-main.js
================================================
const img_modes = {
"ignore": "IgnoreEmpty",
"always": "ShowAlways",
"replace": "Replace(\"XX\")",
"filename": "Filename",
};
const controls = [
// Element id, (conf, value) -> "rust code"
["conf_css", (conf, value) => { conf.use_css(); return ".use_doc_css()"; }],
["conf_user_css", (conf, value) => { conf.add_user_css(value); return `.add_css(r#"{value}"#)`; }],
["conf_agent_css", (conf, value) => { conf.add_agent_css(value); return `.add_agent_css(r#"{value}"#)`; }],
["conf_pad_block_width", (conf, value) => { conf.bad_block_width(); return `.pad_block_width()`; }],
["conf_wrap_width", (conf, value) => { conf.max_wrap_width(value); return `.max_wrap_width({value})`; }],
["conf_allow_overflow", (conf, value) => { conf.allow_overflow(); return `.allow_width_overflow()`; }],
["conf_min_wrap_width", (conf, value) => { conf.min_wrap_width(value); return `.min_wrap_width({value})`; }],
["conf_raw", (conf, value) => { conf.raw_mode(); return `.raw_mode(true)`; }],
["conf_no_borders", (conf, value) => { conf.no_borders(); return `.no_table_borders(true)`; }],
["conf_no_link_wrap", (conf, value) => { conf.no_link_wrap(); return `.no_link_wrapping(true)`; }],
["conf_unicode_so", (conf, value) => { conf.unicode_so(); return `.unicode_strikeout(true)`; }],
["conf_do_decorate", (conf, value) => { conf.do_decorate(); return `.do_decorate(true)`; }],
["conf_link_footnotes", (conf, value) => { conf.link_footnotes(value); return `.link_footnotes({value})`; }],
["conf_img_mode", (conf, value) => { conf.image_mode(value); return '.empty_img_mode(ImageRenderMode::' + img_modes[value] + ")"; }],
];
function update_html() {
const text = document.getElementById("input_html").value;
const colour = document.getElementById("conf_colour").checked;
const raw = document.getElementById("conf_raw").checked;
const no_borders = document.getElementById("conf_no_borders").checked;
const no_link_wrap = document.getElementById("conf_no_link_wrap").checked;
const unicode_so = document.getElementById("conf_unicode_so").checked;
const do_decorate = document.getElementById("conf_do_decorate").checked;
const link_footnotes = document.getElementById("conf_link_footnotes").checked;
let rust_code = "";
let conf = wasmBindings.Config.new();
if (colour) {
rust_code += "let config = html2text::config::rich()";
conf.use_colour();
} else {
rust_code += "let config = html2text::config::plain()";
}
for (const conf_desc of controls) {
const elt_id = conf_desc[0];
const handler = conf_desc[1];
const elt = document.getElementById(elt_id);
if (elt.type == "checkbox") {
if (elt.checked) {
let codefrag = handler(conf, elt.checked);
if (codefrag) {
rust_code += "\n " + codefrag;
}
}
} else {
if (elt.value) {
let codefrag = handler(conf, elt.value);
if (codefrag) {
rust_code += "\n " + codefrag;
}
}
}
}
rust_code += ";\n";
if (colour) {
rust_code += `
let lines = conf.lines_from_read(input, width);
for line in lines {
for ts in line.tagged_strings() {
// examine tags for each text span for colours etc.
}
}
`;
} else {
rust_code += `
let text = conf.string_from_read(input, width);
`;
}
let tn = document.createTextNode(rust_code);
document.getElementById("rust-code").replaceChildren(tn);
wasmBindings.format_html(conf, text);
}
function start() {
const confItems = document.querySelectorAll("input");
confItems.forEach((elt) => {
elt.addEventListener("change", update_html);
});
const selectItems = document.querySelectorAll("select");
selectItems.forEach((elt) => {
elt.addEventListener("change", update_html);
});
// Do the first render
update_html();
}
window.addEventListener("TrunkApplicationStarted", start);
================================================
FILE: pages/assets/demo.css
================================================
#lib {
background-color: black;
height: 30em;
overflow: scroll;
}
#input_html {
height: 300px;
width: 95%;
overflow: scroll;
}
#lib pre {
margin: 0;
padding: 0;
overflow: hidden;
background-color: black;
border: 0px;
}
.warning {
color: red;
}
.warning::before {
content: "⚠️";
}
div.wrapper {
max-width: 100%;
}
@media screen and (min-width: 1000px) {
#h2tmain {
display: grid;
gap: 10px;
grid-template-columns: 1fr 1fr;
}
#lib_container {
grid-column: 1;
min-width: 45%;
}
#input_container {
grid-column: 1;
grid-row-start: 2;
min-width: 45%;
}
#configtable {
grid-column: 2;
grid-row-start: 1;
grid-row-end: 3;
min-width: 45%;
}
#rust-code-pre {
grid-column: 2;
min-width: 45%;
}
}
================================================
FILE: pages/index.markdown
================================================
---
# Feel free to add content and custom Front Matter to this file.
# To modify the layout, see https://jekyllrb.com/docs/themes/#overriding-theme-defaults
title: html2text API demo
layout: home
# Local additions
h2t_wasm: true
h2t_js: "/assets/demo-main.js"
---
<noscript>
<h2 class="warning">This demo page requires javascript (and WASM) to work.</h2>
</noscript>
An online demonstration of the
[`html2text`](https://github.com/jugglerchris/rust-html2text) Rust crate. Edit
the HTML below and see how `html2text` converts it for text or terminal
display.
This demo uses `html2text` compiled to WASM, which can run in any modern
browser, with [ratzilla](https://github.com/orhun/ratzilla) for the web-based
terminal output.
<div id="h2tmain" markdown="1">
<div id="lib_container" markdown="1">
## Output
The html2text output is updated here:
<div id="lib"></div>
</div>
<div id="input_container">
<h2>Input HTML</h2>
<p>Edit the HTML here - the output will update live.</p>
<textarea id="input_html" onchange="update_html()" oninput="update_html()">
<html>
<style>
.green {
color: #4f4;
}
</style>
<body>
<h1>Hi there</h1>
<p>This is some simple text with a <a href="https://github.com/jugglerchris/html2text/">link to github</a></p>
<ol>
<li>Item one</li>
<li><s>Item two</s></li>
<li class="green">Item three</li>
</ol>
<table>
<tr><th>Heading 1</th><th>Heading 2</th><th>Heading 3</th></tr>
<tr><td>Data 1</td><td>Data 2</td><td>Data 3</td></tr>
<tr><td colspan=3>Hello there</td></tr>
</table>
</body></html>
</textarea>
</div>
<div id="configtable" markdown="1">
## Configuration
The following are the configuration settings (accessible via [`html2text::config`](https://docs.rs/html2text/latest/html2text/config/struct.Config.html)).
| <input type="checkbox" id="conf_colour" checked=true>Use Rich output | The [`rich`](https://docs.rs/html2text/latest/html2text/config/fn.rich.html) mode returns spans with attributes (like hyperlinks, emphasis, or colours). When disabled ([`plain`](https://docs.rs/html2text/latest/html2text/config/fn.plain.html)), the output is a plain `String` (possibly with formatting depending on other settings, e.g. table borders or `**markdown-style**` characters added). Rich output adds extra information (annotations) to allow, for example, using terminal colours and other features for a nicer TUI. |
| <input type="checkbox" id="conf_css" checked=true>use_doc_css | Parse CSS from the HTML document (css) |
| <input type="text" id="conf_user_css">User CSS | Add user stylesheet rules (css) |
| <input type="text" id="conf_agent_css">Agent CSS | Add browser stylesheet rules (css) |
| <input type="checkbox" id="conf_pad_block_width">Pad block width | Pad blocks to the width with spaces |
| <input type="number" id="conf_wrap_width">Text wrap width | Wrap text to this width even if overall width is wider |
| <input type="checkbox" id="conf_allow_overflow">Allow width overflow | Allow text to be too wide in extreme cases instead of returning an error |
| <input type="number" id="conf_min_wrap_width">Minimum wrap width | Set the minimum number of columns to use for text blocks. |
| <input type="checkbox" id="conf_raw">Raw mode | Render contents of tables as if they were just text. Implies `no_table_borders` |
| <input type="checkbox" id="conf_no_borders">Don't render table borders | Tables are shown without borders |
| <input type="checkbox" id="conf_no_link_wrap">Don't wrap URLs at the end | Some terminals handle long URLs better if not pre-wrapped |
| <input type="checkbox" id="conf_unicode_so">Use Unicode combining characters for strikeout | This allows crossed out text without terminal codes, but some environments don't render them correctly (e.g. offset). |
| <input type="checkbox" id="conf_do_decorate">Add markdown-like decoration | Add characters, e.g. `*` around `<em>` text even with plain decorators. |
| <input type="checkbox" id="conf_link_footnotes">URL footnotes | Add numbered list of URLs at the end of the output |
| <select name="imgmode" id="conf_img_mode"><option value="">--Select mode--</option><option value="ignore">IgnoreEmpty</option><option value="always">ShowAlways</option><option value="replace">Replace(...)</option><option value="filename">Filename</option></select> | Configure how images with no `alt` text are handled |
</div>
<div id="rust-code-pre" markdown="1">
## Rust API configuration
The code below shows how to use the currently selected settings in the Rust API.
<pre><code id="rust-code"></code></pre>
</div>
<script type="module">
import init, * as bindings from '/rust-html2text/assets/html2text-web-demo.js';
const wasm = await init({ module_or_path: '/rust-html2text/assets/html2text-web-demo_bg.wasm' });
window.wasmBindings = bindings;
dispatchEvent(new CustomEvent("TrunkApplicationStarted", {detail: {wasm}}));
</script>
================================================
FILE: rust.yml
================================================
name: Rust
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
env:
CARGO_TERM_COLOR: always
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Build
run: cargo build --verbose
- name: Run tests
run: cargo test --verbose
================================================
FILE: src/ansi_colours.rs
================================================
//! Convenience helper for producing coloured terminal output.
//!
//! This optional helper applies terminal colours (or other effects which
//! can be achieved using inline characters sent to the terminal such as
//! underlining in some terminals).
use crate::RichAnnotation;
use std::io;
/// Reads HTML from `input`, and returns text wrapped to `width` columns.
///
/// The text is returned as a `Vec<TaggedLine<_>>`; the annotations are vectors
/// of `RichAnnotation`. The "outer" annotation comes first in the `Vec`.
///
/// The function `colour_map` is given a slice of `RichAnnotation` and should
/// return a pair of static strings which should be inserted before/after a text
/// span with that annotation; for example a string which sets text colour
/// and a string which sets the colour back to the default.
pub fn from_read_coloured<R, FMap>(
input: R,
width: usize,
colour_map: FMap,
) -> Result<String, crate::Error>
where
R: io::Read,
FMap: Fn(&[RichAnnotation], &str) -> String,
{
super::config::rich().coloured(input, width, colour_map)
}
================================================
FILE: src/css/parser.rs
================================================
//! Parsing for the subset of CSS used in html2text.
use std::{borrow::Cow, ops::Deref, str::FromStr};
use nom::{
AsChar, IResult, Parser,
branch::alt,
bytes::complete::{tag, take_until},
character::complete::{self, digit0, digit1},
combinator::{fail, map, opt, recognize},
error::ErrorKind,
multi::{many0, many1, separated_list0},
};
#[derive(Debug, PartialEq)]
pub(crate) enum Colour {
Rgb(u8, u8, u8),
}
impl From<Colour> for crate::Colour {
fn from(value: Colour) -> Self {
match value {
Colour::Rgb(r, g, b) => crate::Colour { r, g, b },
}
}
}
#[derive(Debug, PartialEq)]
pub(crate) enum LengthUnit {
// Absolute units
In,
Cm,
Mm,
Pt,
Pc,
Px,
// Relative units
Em,
Ex,
}
#[derive(Debug, PartialEq)]
pub(crate) enum Height {
#[allow(unused)]
Auto,
// If the length is 0, the unit will be Px
Length(f32, LengthUnit),
}
#[derive(Debug, PartialEq)]
pub(crate) enum Overflow {
Visible,
Hidden,
Scroll,
Auto,
}
#[non_exhaustive]
#[derive(Debug, PartialEq)]
pub(crate) enum Display {
None,
Other,
#[cfg(feature = "css_ext")]
RawDom,
}
#[derive(Debug, PartialEq)]
#[non_exhaustive]
pub(crate) enum Decl {
Color {
value: Colour,
},
BackgroundColor {
value: Colour,
},
Height {
value: Height,
},
MaxHeight {
value: Height,
},
Overflow {
value: Overflow,
},
OverflowY {
value: Overflow,
},
Display {
value: Display,
},
WhiteSpace {
value: WhiteSpace,
},
Content {
text: String,
},
#[cfg(feature = "css_ext")]
XSyntax {
language: String,
},
Unknown {
name: PropertyName,
// value: Vec<Token>,
},
}
// Tokens as defined in the CSS Syntax Module Level 3
#[allow(unused)]
#[derive(Debug, PartialEq)]
enum Token<'s> {
/// Plain identifier
Ident(Cow<'s, str>),
/// Start of a function call: <ident>(
Function(Cow<'s, str>),
/// @<ident>
AtKeyword(Cow<'s, str>),
/// #abcd12
Hash(Cow<'s, str>),
/// Quoted (double or single) string
String(Cow<'s, str>),
/// <bad-string-token>
BadString(Cow<'s, str>),
/// URL
Url(Cow<'s, str>),
/// <bad-url-token>
BadUrl(Cow<'s, str>),
/// Delim character
Delim(char),
/// Number
Number(Cow<'s, str>),
/// Dimension (number, unit)
Dimension(Cow<'s, str>, Cow<'s, str>),
/// Percentage
Percentage(Cow<'s, str>),
/// Whitespace
//Whitespace(Cow<'s, str>),
/// CDO (<!--)
#[allow(clippy::upper_case_acronyms)]
CDO,
/// CDC (-->)
#[allow(clippy::upper_case_acronyms)]
CDC,
/// Colon
Colon,
/// Semicolon
Semicolon,
/// Comma
Comma,
/// [-token
OpenSquare,
/// ]-token
CloseSquare,
/// (-token
OpenRound,
/// )-token
CloseRound,
/// {-token
OpenBrace,
/// }-token
CloseBrace,
}
// A raw, uninterpreted declaration value.
#[derive(Debug, PartialEq)]
struct RawValue<'s> {
tokens: Vec<Token<'s>>,
important: bool,
}
#[derive(Debug, PartialEq)]
pub(crate) struct Declaration {
pub data: Decl,
pub important: Importance,
}
use crate::css::styles_from_properties;
use super::{PseudoElement, Selector, SelectorComponent, StyleDecl, WhiteSpace, types::Importance};
#[derive(Debug, PartialEq)]
pub(crate) struct RuleSet {
pub selectors: Vec<Selector>,
pub declarations: Vec<Declaration>,
}
#[derive(Debug, PartialEq)]
pub(crate) struct PropertyName(String);
fn match_comment(text: &str) -> IResult<&str, ()> {
let (rest, _) = tag("/*")(text)?;
let (rest, _) = take_until("*/")(rest)?;
map(tag("*/"), |_t| ()).parse(rest)
}
fn match_whitespace_item(text: &str) -> IResult<&str, ()> {
alt((map(complete::one_of(" \t\r\n\x0c"), |_c| ()), match_comment)).parse(text)
}
fn skip_optional_whitespace(text: &str) -> IResult<&str, ()> {
map(many0(match_whitespace_item), |_res| ()).parse(text)
}
fn nmstart_char(s: &str) -> IResult<&str, char> {
let mut iter = s.chars();
match iter.next() {
Some(c) => match c {
'_' | 'a'..='z' | 'A'..='Z' => Ok((iter.as_str(), c.to_ascii_lowercase())),
_ => IResult::Err(nom::Err::Error(nom::error::Error::new(s, ErrorKind::Fail))),
},
None => fail().parse(s),
}
}
fn is_ident_start(c: char) -> bool {
matches!(c, 'a'..='z' | 'A'..='Z' | '_' | '\u{0081}'..)
}
fn is_digit(c: char) -> bool {
c.is_ascii_digit()
}
fn nmchar_char(s: &str) -> IResult<&str, char> {
let mut iter = s.chars();
match iter.next() {
Some(c) => match c {
'_' | 'a'..='z' | 'A'..='Z' | '0'..='9' | '-' => {
Ok((iter.as_str(), c.to_ascii_lowercase()))
}
_ => IResult::Err(nom::Err::Error(nom::error::Error::new(s, ErrorKind::Fail))),
},
None => fail().parse(s),
}
}
fn ident_escape(s: &str) -> IResult<&str, char> {
let (rest, _) = tag("\\")(s)?;
let mut chars = rest.char_indices();
match chars.next() {
None => {
// EOF: return replacement char
Ok((rest, '\u{fffd}'))
}
Some((i, c)) if c.is_hex_digit() => {
// Option 1: up to 6 hex digits.
let start_idx = i;
let mut end_idx = i + 1;
for (nexti, nextc) in chars {
if nextc.is_hex_digit() && nexti - start_idx < 6 {
continue;
} else {
end_idx = nexti;
break;
}
}
let val = u32::from_str_radix(&rest[start_idx..end_idx], 16).unwrap();
Ok((&rest[end_idx..], char::from_u32(val).unwrap_or('\u{fffd}')))
}
Some((_i, c)) => {
let bytes = c.len_utf8();
Ok((&rest[bytes..], c))
}
}
}
fn nmstart(text: &str) -> IResult<&str, char> {
alt((nmstart_char, ident_escape)).parse(text)
}
fn nmchar(text: &str) -> IResult<&str, char> {
alt((nmchar_char, ident_escape)).parse(text)
}
fn parse_ident(text: &str) -> IResult<&str, String> {
let (rest, _) = skip_optional_whitespace(text)?;
let mut name = Vec::new();
let (rest, dash) = opt(tag("-")).parse(rest)?;
if dash.is_some() {
name.push('-');
}
let (rest, start) = nmstart(rest)?;
name.push(start);
let (rest, chars) = many0(nmchar).parse(rest)?;
name.extend(chars);
Ok((rest, name.into_iter().collect()))
}
fn parse_identstring(text: &str) -> IResult<&str, String> {
let (rest, _) = skip_optional_whitespace(text)?;
let (rest, name) = many1(nmchar).parse(rest)?;
Ok((rest, name.into_iter().collect()))
}
fn parse_property_name(text: &str) -> IResult<&str, PropertyName> {
parse_ident(text).map(|(r, s)| (r, PropertyName(s)))
}
// For now ignore whitespace
fn parse_token(text: &str) -> IResult<&str, Token<'_>> {
let (rest, _) = skip_optional_whitespace(text)?;
let mut chars = rest.chars();
match chars.next() {
None => fail().parse(rest),
Some('"') | Some('\'') => parse_string_token(rest),
Some('#') => match parse_identstring(&rest[1..]) {
Ok((rest, id)) => Ok((rest, Token::Hash(id.into()))),
Err(_) => Ok((rest, Token::Delim('#'))),
},
Some(';') => Ok((&rest[1..], Token::Semicolon)),
Some('(') => Ok((&rest[1..], Token::OpenRound)),
Some(')') => Ok((&rest[1..], Token::CloseRound)),
Some('+') => match parse_numeric_token(&rest[1..]) {
Ok(result) => Ok(result),
Err(_) => Ok((&rest[1..], Token::Delim('+'))),
},
Some(',') => Ok((&rest[1..], Token::Comma)),
Some('-') => {
if let Ok((rest_n, tok)) = parse_numeric_token(rest) {
return Ok((rest_n, tok));
}
if let Some(rest_cdc) = rest.strip_prefix("-->") {
return Ok((rest_cdc, Token::CDC));
}
if let Ok((rest_id, token)) = parse_ident_like(rest) {
return Ok((rest_id, token));
}
Ok((&rest[1..], Token::Delim('-')))
}
Some('.') => {
if let Ok((rest_n, tok)) = parse_numeric_token(rest) {
return Ok((rest_n, tok));
}
Ok((&rest[1..], Token::Delim('.')))
}
Some(':') => Ok((&rest[1..], Token::Colon)),
Some('<') => {
if let Some(rest_cdo) = rest.strip_prefix("<!--") {
return Ok((rest_cdo, Token::CDO));
}
Ok((&rest[1..], Token::Delim('<')))
}
Some('@') => {
if let Ok((rest_id, id)) = parse_ident(rest) {
return Ok((rest_id, Token::AtKeyword(id.into())));
}
Ok((&rest[1..], Token::Delim('@')))
}
Some('[') => Ok((&rest[1..], Token::OpenSquare)),
Some('\\') => {
if let Ok((rest_id, token)) = parse_ident_like(rest) {
Ok((rest_id, token))
} else {
Ok((&rest[1..], Token::Delim('\\')))
}
}
Some(']') => Ok((&rest[1..], Token::CloseSquare)),
Some('{') => Ok((&rest[1..], Token::OpenBrace)),
Some('}') => Ok((&rest[1..], Token::CloseBrace)),
Some(c) if is_ident_start(c) => parse_ident_like(rest),
Some(c) if is_digit(c) => parse_numeric_token(rest),
Some('!') => Ok((&rest[1..], Token::Delim('!'))),
Some(c) => {
let num_bytes = c.len_utf8();
Ok((&rest[num_bytes..], Token::Delim(c)))
}
}
}
fn parse_token_not_semicolon(text: &str) -> IResult<&str, Token<'_>> {
let (rest, token) = parse_token(text)?;
if token == Token::Semicolon {
fail().parse(text)
} else {
Ok((rest, token))
}
}
fn parse_value(text: &str) -> IResult<&str, RawValue<'_>> {
let (rest, mut tokens) = many0(parse_token_not_semicolon).parse(text)?;
let mut important = false;
if let [.., Token::Delim('!'), Token::Ident(x)] = &tokens[..] {
if x == "important" {
tokens.pop();
tokens.pop();
important = true;
}
}
Ok((rest, RawValue { tokens, important }))
}
pub(crate) fn parse_color_attribute(
text: &str,
) -> Result<Colour, nom::Err<nom::error::Error<&'static str>>> {
let (_rest, value) = parse_value(text).map_err(|_| empty_fail())?;
parse_color(&value.tokens).or_else(|e| parse_faulty_color(e, text))
}
fn parse_color_part(text: &str, index: std::ops::Range<usize>) -> Option<u8> {
u8::from_str_radix(text.get(index)?, 16).ok()
}
// Both Firefox and Chromium accept "00aabb" as a bgcolor - I'm not sure this has ever been legal,
// but regrettably I've had e-mails which were unreadable without doing this.
fn parse_faulty_color(
e: nom::Err<nom::error::Error<&'static str>>,
text: &str,
) -> Result<Colour, nom::Err<nom::error::Error<&'static str>>> {
let text = text.trim();
let r = parse_color_part(text, 0..2);
let g = parse_color_part(text, 2..4);
let b = parse_color_part(text, 4..6);
if let (Some(r), Some(g), Some(b)) = (r, g, b) {
return Ok(Colour::Rgb(r, g, b));
}
Err(e)
}
pub(crate) fn parse_declaration(text: &str) -> IResult<&str, Option<Declaration>> {
let (rest, (prop, _ws1, _colon, _ws2, value)) = (
parse_property_name,
skip_optional_whitespace,
tag(":"),
skip_optional_whitespace,
parse_value,
)
.parse(text)?;
let decl = match prop.0.as_str() {
"background-color" => {
if let Ok(value) = parse_color(&value.tokens) {
Decl::BackgroundColor { value }
} else {
Decl::Unknown { name: prop }
}
}
"background" => match parse_background_color(&value) {
Ok(Some(value)) => Decl::BackgroundColor { value },
_ => Decl::Unknown { name: prop },
},
"color" => {
if let Ok(value) = parse_color(&value.tokens) {
Decl::Color { value }
} else {
Decl::Unknown { name: prop }
}
}
"height" => {
if let Ok(value) = parse_height(&value) {
Decl::Height { value }
} else {
Decl::Unknown { name: prop }
}
}
"max-height" => {
if let Ok(value) = parse_height(&value) {
Decl::MaxHeight { value }
} else {
Decl::Unknown { name: prop }
}
}
"overflow" => {
if let Ok(value) = parse_overflow(&value) {
Decl::Overflow { value }
} else {
Decl::Unknown { name: prop }
}
}
"overflow-y" => {
if let Ok(value) = parse_overflow(&value) {
Decl::OverflowY { value }
} else {
Decl::Unknown { name: prop }
}
}
"display" => {
if let Ok(value) = parse_display(&value) {
Decl::Display { value }
} else {
Decl::Unknown { name: prop }
}
}
"white-space" => {
if let Ok(value) = parse_white_space(&value) {
Decl::WhiteSpace { value }
} else {
Decl::Unknown { name: prop }
}
}
"content" => {
if let Ok(value) = parse_content(&value) {
Decl::Content { text: value }
} else {
Decl::Unknown { name: prop }
}
}
#[cfg(feature = "css_ext")]
"x-syntax" => {
if let Ok(value) = parse_syntax(&value) {
Decl::XSyntax { language: value }
} else {
Decl::Unknown { name: prop }
}
}
_ => Decl::Unknown {
name: prop,
// value: /*value*/"".into(),
},
};
Ok((
rest,
Some(Declaration {
data: decl,
important: if value.important {
Importance::Important
} else {
Importance::Default
},
}),
))
}
fn empty_fail() -> nom::Err<nom::error::Error<&'static str>> {
nom::Err::Error(nom::error::Error::new("", ErrorKind::Fail))
}
fn parse_color(tokens: &[Token]) -> Result<Colour, nom::Err<nom::error::Error<&'static str>>> {
let fail_error = empty_fail();
if tokens.is_empty() {
return Err(fail_error);
}
match tokens {
[Token::Ident(c)] => {
let colour = match c.deref() {
"aqua" => Colour::Rgb(0, 0xff, 0xff),
"black" => Colour::Rgb(0, 0, 0),
"blue" => Colour::Rgb(0, 0, 0xff),
"fuchsia" => Colour::Rgb(0xff, 0, 0xff),
"gray" => Colour::Rgb(0x80, 0x80, 0x80),
"green" => Colour::Rgb(0, 0x80, 0),
"lime" => Colour::Rgb(0, 0xff, 0),
"maroon" => Colour::Rgb(0x80, 0, 0),
"navy" => Colour::Rgb(0, 0, 0x80),
"olive" => Colour::Rgb(0x80, 0x80, 0),
"orange" => Colour::Rgb(0xff, 0xa5, 0),
"purple" => Colour::Rgb(0x80, 0, 0x80),
"red" => Colour::Rgb(0xff, 0, 0),
"silver" => Colour::Rgb(0xc0, 0xc0, 0xc0),
"teal" => Colour::Rgb(0, 0x80, 0x80),
"white" => Colour::Rgb(0xff, 0xff, 0xff),
"yellow" => Colour::Rgb(0xff, 0xff, 0),
_ => {
return Err(empty_fail());
}
};
Ok(colour)
}
[Token::Function(name), rgb_args @ .., Token::CloseRound] => {
use Token::*;
match name.deref() {
"rgb" => match rgb_args {
[Number(r), Comma, Number(g), Comma, Number(b)] => {
let r = r.parse().map_err(|_e| empty_fail())?;
let g = g.parse().map_err(|_e| empty_fail())?;
let b = b.parse().map_err(|_e| empty_fail())?;
Ok(Colour::Rgb(r, g, b))
}
_ => Err(empty_fail()),
},
_ => Err(empty_fail()),
}
}
[Token::Hash(s)] => {
if s.len() == 3 {
let v = u32::from_str_radix(s, 16).map_err(|_| fail_error)?;
let r = ((v >> 8) & 0xf) as u8 * 0x11;
let g = ((v >> 4) & 0xf) as u8 * 0x11;
let b = (v & 0xf) as u8 * 0x11;
Ok(Colour::Rgb(r, g, b))
} else if s.len() == 6 {
let v = u32::from_str_radix(s, 16).map_err(|_| fail_error)?;
let r = ((v >> 16) & 0xff) as u8;
let g = ((v >> 8) & 0xff) as u8;
let b = (v & 0xff) as u8;
Ok(Colour::Rgb(r, g, b))
} else {
Err(fail_error)
}
}
_ => Err(fail_error),
}
}
// Parse background: value, extracting only the colour (if present).
fn parse_background_color(
value: &RawValue,
) -> Result<Option<Colour>, nom::Err<nom::error::Error<&'static str>>> {
let tokens = if let Some(last) = value.tokens.rsplit(|tok| *tok == Token::Comma).next() {
last
} else {
return Err(empty_fail());
};
match parse_color(tokens) {
Ok(col) => Ok(Some(col)),
Err(_) => Ok(None),
}
}
fn parse_integer(text: &str) -> IResult<&str, f32> {
let (rest, digits) = digit1(text)?;
Ok((rest, <f32 as FromStr>::from_str(digits).unwrap()))
}
fn parse_decimal(text: &str) -> IResult<&str, f32> {
let (rest, valstr) = recognize((digit0, tag("."), digit1)).parse(text)?;
Ok((rest, <f32 as FromStr>::from_str(valstr).unwrap()))
}
fn parse_number(text: &str) -> IResult<&str, f32> {
let (rest, _) = skip_optional_whitespace(text)?;
let (rest, (sign, val)) = (
opt(alt((tag("-"), tag("+")))),
alt((parse_integer, parse_decimal)),
)
.parse(rest)?;
Ok((
rest,
match sign {
Some("-") => -val,
None | Some("+") => val,
_ => unreachable!(),
},
))
}
fn parse_numeric_token(text: &str) -> IResult<&str, Token<'_>> {
let (rest, num) = recognize(parse_number).parse(text)?;
let match_pct: IResult<_, _> = tag("%")(rest);
if let Ok((rest_p, _)) = match_pct {
return Ok((rest_p, Token::Percentage(num.into())));
}
match parse_ident(rest) {
Ok((rest_id, dim)) => Ok((rest_id, Token::Dimension(num.into(), dim.into()))),
Err(_) => Ok((rest, Token::Number(num.into()))),
}
}
fn parse_ident_like(text: &str) -> IResult<&str, Token<'_>> {
let (rest, ident) = parse_ident(text)?;
// If the next character is '(', then it's a function token
let match_bracket: IResult<_, _> = tag("(")(rest);
match match_bracket {
Ok((rest_f, _)) => Ok((rest_f, Token::Function(ident.into()))),
Err(_) => Ok((rest, Token::Ident(ident.into()))),
}
}
fn parse_string_token(text: &str) -> IResult<&str, Token<'_>> {
let mut chars = text.char_indices();
let mut s = String::new();
let end_char = chars.next().unwrap().1;
if !(end_char == '"' || end_char == '\'') {
return fail().parse(text);
}
loop {
match chars.next() {
None => return Ok(("", Token::String(s.into()))),
Some((i, c)) if c == end_char => {
return Ok((&text[i + 1..], Token::String(s.into())));
}
Some((i, '\n')) => {
return Ok((&text[i..], Token::BadString(s.into())));
}
Some((i, '\\')) => {
match chars.next() {
None => {
// Backslash at end
return Ok((&text[i + 1..], Token::String(s.into())));
}
Some((_i, '\n')) => {} // Eat the newline
Some((_i, c)) => {
s.push(c);
}
}
}
Some((_, c)) => {
s.push(c);
}
}
}
}
// Parse a string as in `parse_string_token`, but fail on
// bad strings.
fn parse_quoted_string(text: &str) -> IResult<&str, String> {
let (rest, result) = parse_string_token(text)?;
match result {
Token::String(s) => Ok((rest, s.to_string())),
_ => fail().parse("Invalid string"),
}
}
/*
fn parse_unit(text: &str) -> IResult<&str, LengthUnit> {
let (rest, word) = alpha0(text)?;
Ok((rest, match word {
_ => {
return fail().parse(text);
}
}))
}
*/
fn parse_height(value: &RawValue) -> Result<Height, nom::Err<nom::error::Error<&'static str>>> {
match value.tokens[..] {
[Token::Dimension(ref n, ref unit)] => {
let (_, num) = parse_number(n).map_err(|_e| empty_fail())?;
let unit = match unit.deref() {
"in" => LengthUnit::In,
"cm" => LengthUnit::Cm,
"mm" => LengthUnit::Mm,
"pt" => LengthUnit::Pt,
"pc" => LengthUnit::Pc,
"px" => LengthUnit::Px,
"em" => LengthUnit::Em,
"ex" => LengthUnit::Ex,
_ => {
return Err(empty_fail());
}
};
Ok(Height::Length(num, unit))
}
[Token::Number(ref n)] => {
let (_, n) = parse_number(n).map_err(|_e| empty_fail())?;
if n == 0.0 {
Ok(Height::Length(0.0, LengthUnit::Px))
} else {
Err(empty_fail())
}
}
_ => Err(empty_fail()),
}
}
fn parse_overflow(value: &RawValue) -> Result<Overflow, nom::Err<nom::error::Error<&'static str>>> {
for tok in &value.tokens {
if let Token::Ident(word) = tok {
match word.deref() {
"visible" => {
return Ok(Overflow::Visible);
}
"hidden" => {
return Ok(Overflow::Hidden);
}
"scroll" => {
return Ok(Overflow::Scroll);
}
"auto" => {
return Ok(Overflow::Auto);
}
_ => {}
}
}
}
Err(empty_fail())
}
fn parse_display(value: &RawValue) -> Result<Display, nom::Err<nom::error::Error<&'static str>>> {
for tok in &value.tokens {
if let Token::Ident(word) = tok {
#[allow(clippy::single_match)]
match word.deref() {
"none" => return Ok(Display::None),
#[cfg(feature = "css_ext")]
"x-raw-dom" => return Ok(Display::RawDom),
_ => (),
}
}
}
Ok(Display::Other)
}
#[cfg(feature = "css_ext")]
fn parse_syntax(value: &RawValue) -> Result<String, nom::Err<nom::error::Error<&'static str>>> {
if let Some(Token::Ident(word)) = value.tokens.first() {
return Ok(word.to_string());
}
Err(empty_fail())
}
fn parse_white_space(
value: &RawValue,
) -> Result<WhiteSpace, nom::Err<nom::error::Error<&'static str>>> {
for tok in &value.tokens {
if let Token::Ident(word) = tok {
match word.deref() {
"normal" => return Ok(WhiteSpace::Normal),
"pre" => return Ok(WhiteSpace::Pre),
"pre-wrap" => return Ok(WhiteSpace::PreWrap),
_ => (),
}
}
}
Ok(WhiteSpace::Normal)
}
// Parse content - currently only support a single string.
fn parse_content(value: &RawValue) -> Result<String, nom::Err<nom::error::Error<&'static str>>> {
let mut result = String::new();
for tok in &value.tokens {
if let Token::String(word) = tok {
result.push_str(word);
} else {
return Err(empty_fail());
}
}
Ok(result)
}
pub(crate) fn parse_rules(text: &str) -> IResult<&str, Vec<Declaration>> {
separated_list0((tag(";"), skip_optional_whitespace), parse_declaration)
.parse(text)
.map(|(rest, v)| (rest, v.into_iter().flatten().collect()))
}
fn parse_class(text: &str) -> IResult<&str, SelectorComponent> {
let (rest, _) = tag(".")(text)?;
let (rest, classname) = parse_ident(rest)?;
Ok((rest, SelectorComponent::Class(classname)))
}
fn parse_attr(text: &str) -> IResult<&str, SelectorComponent> {
alt((
map((tag("["), parse_ident, tag("]")), |(_, name, _)| {
SelectorComponent::Attr {
name,
value: None,
op: super::AttrOperator::Present,
}
}),
map(
(
tag("["),
parse_ident,
tag("="),
parse_quoted_string,
tag("]"),
),
|(_, name, _, value, _)| SelectorComponent::Attr {
name,
value: Some(value),
op: super::AttrOperator::Equal,
},
),
map(
(tag("["), parse_ident, tag("="), parse_ident, tag("]")),
|(_, name, _, value, _)| SelectorComponent::Attr {
name,
value: Some(value),
op: super::AttrOperator::Equal,
},
),
))
.parse(text)
}
#[derive(Eq, PartialEq, Copy, Clone)]
enum Sign {
Plus,
Neg,
}
impl Sign {
fn val(&self) -> i32 {
match self {
Sign::Plus => 1,
Sign::Neg => -1,
}
}
}
fn opt_sign(text: &str) -> IResult<&str, Sign> {
match text.chars().next() {
Some('-') => Ok((&text[1..], Sign::Neg)),
Some('+') => Ok((&text[1..], Sign::Plus)),
_ => Ok((text, Sign::Plus)),
}
}
fn sign(text: &str) -> IResult<&str, Sign> {
match text.chars().next() {
Some('-') => Ok((&text[1..], Sign::Neg)),
Some('+') => Ok((&text[1..], Sign::Plus)),
_ => fail().parse(text),
}
}
fn parse_nth_child_args(text: &str) -> IResult<&str, SelectorComponent> {
let (rest, _) = tag("(")(text)?;
let (rest, _) = skip_optional_whitespace(rest)?;
let (rest, (a, b)) = alt((
map(tag("even"), |_| (2, 0)),
map(tag("odd"), |_| (2, 1)),
// The case where both a and b are specified
map(
(
opt_sign,
opt(digit1),
tag("n"),
skip_optional_whitespace,
sign,
digit1,
),
|(a_sign, a_opt_val, _, _, b_sign, b_val)| {
let a =
<i32 as FromStr>::from_str(a_opt_val.unwrap_or("1")).unwrap() * a_sign.val();
let b = <i32 as FromStr>::from_str(b_val).unwrap() * b_sign.val();
(a, b)
},
),
// Just a
map(
(opt_sign, opt(digit1), tag("n")),
|(a_sign, a_opt_val, _)| {
let a =
<i32 as FromStr>::from_str(a_opt_val.unwrap_or("1")).unwrap() * a_sign.val();
(a, 0)
},
),
// Just b
map((opt_sign, digit1), |(b_sign, b_val)| {
let b = <i32 as FromStr>::from_str(b_val).unwrap() * b_sign.val();
(0, b)
}),
))
.parse(rest)?;
let (rest, _) = (skip_optional_whitespace, tag(")")).parse(rest)?;
let sel = Selector {
components: vec![SelectorComponent::Star],
..Default::default()
};
Ok((rest, SelectorComponent::NthChild { a, b, sel }))
}
fn parse_pseudo_class(text: &str) -> IResult<&str, SelectorComponent> {
let (rest, _) = tag(":")(text)?;
let (rest, pseudo_classname) = parse_ident(rest)?;
match pseudo_classname.as_str() {
"nth-child" => {
let (rest, component) = parse_nth_child_args(rest)?;
Ok((rest, component))
}
_ => fail().parse(text),
}
}
fn parse_hash(text: &str) -> IResult<&str, SelectorComponent> {
let (rest, _) = tag("#")(text)?;
let (rest, word) = parse_identstring(rest)?;
Ok((rest, SelectorComponent::Hash(word)))
}
// Match some (not zero) whitespace
fn parse_ws(text: &str) -> IResult<&str, ()> {
map(many1(match_whitespace_item), |_| ()).parse(text)
}
fn parse_simple_selector_component(text: &str) -> IResult<&str, SelectorComponent> {
alt((
map(
(skip_optional_whitespace, tag(">"), skip_optional_whitespace),
|_| SelectorComponent::CombChild,
),
map(
(skip_optional_whitespace, tag("*"), skip_optional_whitespace),
|_| SelectorComponent::Star,
),
map(parse_ws, |_| SelectorComponent::CombDescendant),
parse_class,
parse_attr,
parse_hash,
map(parse_ident, SelectorComponent::Element),
parse_pseudo_class,
))
.parse(text)
}
fn parse_selector_with_element(text: &str) -> IResult<&str, Vec<SelectorComponent>> {
let (rest, ident) = parse_ident(text)?;
let (rest, extras) = many0(parse_simple_selector_component).parse(rest)?;
let mut result = vec![SelectorComponent::Element(ident)];
result.extend(extras);
Ok((rest, result))
}
fn parse_selector_without_element(text: &str) -> IResult<&str, Vec<SelectorComponent>> {
many1(parse_simple_selector_component).parse(text)
}
pub(crate) fn parse_pseudo_element(text: &str) -> IResult<&str, Option<PseudoElement>> {
opt(alt((
map(tag("::before"), |_| PseudoElement::Before),
map(tag("::after"), |_| PseudoElement::After),
)))
.parse(text)
}
pub(crate) fn parse_selector(text: &str) -> IResult<&str, Selector> {
let (rest, mut components) = alt((
parse_selector_with_element,
parse_selector_without_element,
fail(),
))
.parse(text)?;
// Reverse. Also remove any leading/trailing CombDescendant, as leading/trailing whitespace
// shouldn't count as a descendent combinator.
if let Some(&SelectorComponent::CombDescendant) = components.last() {
components.pop();
}
components.reverse();
if let Some(&SelectorComponent::CombDescendant) = components.last() {
components.pop();
}
let (rest, pseudo_element) = parse_pseudo_element(rest)?;
Ok((
rest,
Selector {
components,
pseudo_element,
},
))
}
fn parse_ruleset(text: &str) -> IResult<&str, RuleSet> {
let (rest, _) = skip_optional_whitespace(text)?;
let (rest, selectors) =
separated_list0((tag(","), skip_optional_whitespace), parse_selector).parse(rest)?;
let (rest, (_ws1, _bra, _ws2, declarations, _ws3, _optsemi, _ws4, _ket, _ws5)) = (
skip_optional_whitespace,
tag("{"),
skip_optional_whitespace,
parse_rules,
skip_optional_whitespace,
opt(tag(";")),
skip_optional_whitespace,
tag("}"),
skip_optional_whitespace,
)
.parse(rest)?;
Ok((
rest,
RuleSet {
selectors,
declarations,
},
))
}
fn skip_to_end_of_statement(text: &str) -> IResult<&str, ()> {
let mut rest = text;
let mut bra_stack = vec![];
loop {
let (remain, tok) = match parse_token(rest) {
Ok(res) => res,
Err(_) => return Ok((rest, ())),
};
match &tok {
Token::Ident(..)
| Token::AtKeyword(_)
| Token::Hash(_)
| Token::String(_)
| Token::BadString(_)
| Token::Url(_)
| Token::BadUrl(_)
| Token::Delim(_)
| Token::Number(_)
| Token::Dimension(_, _)
| Token::Percentage(_)
| Token::Colon
| Token::Comma => (),
Token::Function(_) | Token::OpenRound => {
bra_stack.push(Token::CloseRound);
}
Token::CDO => {
bra_stack.push(Token::CDC);
}
Token::OpenSquare => {
bra_stack.push(Token::CloseSquare);
}
Token::OpenBrace => {
bra_stack.push(Token::CloseBrace);
}
Token::Semicolon => {
if bra_stack.is_empty() {
return Ok((remain, ()));
}
}
Token::CloseBrace if bra_stack.is_empty() => {
// The stack is empty, so don't include the closing brace.
return Ok((rest, ()));
}
// Standard closing brackets
Token::CDC | Token::CloseSquare | Token::CloseRound | Token::CloseBrace => {
if bra_stack.last() == Some(&tok) {
bra_stack.pop();
if tok == Token::CloseBrace && bra_stack.is_empty() {
// The rule lasted until the end of the next block;
// eat this closing brace.
return Ok((remain, ()));
}
} else {
// Unbalanced brackets
return fail().parse(rest);
}
}
}
rest = remain;
}
}
fn parse_at_rule(text: &str) -> IResult<&str, ()> {
let (rest, _) = (
skip_optional_whitespace,
tag("@"),
skip_optional_whitespace,
parse_ident,
)
.parse(text)?;
skip_to_end_of_statement(rest)
}
fn parse_statement(text: &str) -> IResult<&str, Option<RuleSet>> {
alt((map(parse_ruleset, Some), map(parse_at_rule, |_| None))).parse(text)
}
pub(crate) fn parse_stylesheet(text: &str) -> IResult<&str, Vec<RuleSet>> {
let (rest, items) = many0(parse_statement).parse(text)?;
Ok((rest, items.into_iter().flatten().collect()))
}
pub(crate) fn parse_style_attribute(text: &str) -> crate::Result<Vec<StyleDecl>> {
html_trace_quiet!("Parsing inline style: {text}");
let (_rest, decls) = parse_rules(text).map_err(|_| crate::Error::CssParseError)?;
let styles = styles_from_properties(&decls, false);
html_trace_quiet!("Parsed inline style: {:?}", styles);
Ok(styles)
}
#[cfg(test)]
mod test {
use crate::css::{
AttrOperator, PseudoElement, SelectorComponent,
parser::{Height, Importance, LengthUnit, RuleSet, Selector},
};
use super::{Colour, Decl, Declaration, Overflow, PropertyName};
#[test]
fn test_parse_decl() {
assert_eq!(
super::parse_declaration("foo:bar;"),
Ok((
";",
Some(Declaration {
data: Decl::Unknown {
name: PropertyName("foo".into()),
// value: "bar".into()
},
important: Importance::Default,
})
))
);
}
#[test]
fn test_parse_overflow() {
assert_eq!(
super::parse_rules("overflow: hidden; overflow-y: scroll"),
Ok((
"",
vec![
Declaration {
data: Decl::Overflow {
value: Overflow::Hidden
},
important: Importance::Default,
},
Declaration {
data: Decl::OverflowY {
value: Overflow::Scroll
},
important: Importance::Default,
},
]
))
);
}
#[test]
fn test_parse_color() {
assert_eq!(
super::parse_rules("color: #123; color: #abcdef"),
Ok((
"",
vec![
Declaration {
data: Decl::Color {
value: Colour::Rgb(0x11, 0x22, 0x33)
},
important: Importance::Default,
},
Declaration {
data: Decl::Color {
value: Colour::Rgb(0xab, 0xcd, 0xef)
},
important: Importance::Default,
},
]
))
);
assert_eq!(
super::parse_rules("color: inherit"),
Ok((
"",
vec![Declaration {
data: Decl::Unknown {
name: PropertyName("color".into()),
},
important: Importance::Default,
},]
))
);
}
#[test]
fn test_parse_height() {
assert_eq!(
super::parse_rules("height: 0; max-height: 100cm"),
Ok((
"",
vec![
Declaration {
data: Decl::Height {
value: Height::Length(0.0, LengthUnit::Px),
},
important: Importance::Default,
},
Declaration {
data: Decl::MaxHeight {
value: Height::Length(100.0, LengthUnit::Cm),
},
important: Importance::Default,
},
]
))
);
}
#[test]
fn test_parse_empty_ss() {
assert_eq!(super::parse_stylesheet(""), Ok(("", vec![])));
}
#[test]
fn test_parse_ss_col() {
assert_eq!(
super::parse_stylesheet(
"
foo {
color: #112233;
}
"
),
Ok((
"",
vec![RuleSet {
selectors: vec![Selector {
components: vec![SelectorComponent::Element("foo".into()),],
..Default::default()
},],
declarations: vec![Declaration {
data: Decl::Color {
value: Colour::Rgb(0x11, 0x22, 0x33)
},
important: Importance::Default,
},],
}]
))
);
}
#[test]
fn test_parse_class() {
assert_eq!(
super::parse_stylesheet(
"
.foo {
color: #112233;
background-color: #332211 !important;
}
"
),
Ok((
"",
vec![RuleSet {
selectors: vec![Selector {
components: vec![SelectorComponent::Class("foo".into()),],
..Default::default()
},],
declarations: vec![
Declaration {
data: Decl::Color {
value: Colour::Rgb(0x11, 0x22, 0x33)
},
important: Importance::Default,
},
Declaration {
data: Decl::BackgroundColor {
value: Colour::Rgb(0x33, 0x22, 0x11)
},
important: Importance::Important,
},
],
}]
))
);
}
#[test]
fn test_parse_at_rules() {
assert_eq!(
super::parse_stylesheet(
"
@media paper {
}
@blah asldfkjasfda;
@nested { lkasjfd alkdsjfa sldkfjas ( alksjdasfd ) [ alskdjfalskdf] }
@keyframes foo {
0% { transform: translateY(0); }
50% { opacity:0.8; }
100% { }
}
.foo {
color: #112233;
background-color: #332211 !important;
}
"
),
Ok((
"",
vec![RuleSet {
selectors: vec![Selector {
components: vec![SelectorComponent::Class("foo".into()),],
..Default::default()
},],
declarations: vec![
Declaration {
data: Decl::Color {
value: Colour::Rgb(0x11, 0x22, 0x33)
},
important: Importance::Default,
},
Declaration {
data: Decl::BackgroundColor {
value: Colour::Rgb(0x33, 0x22, 0x11)
},
important: Importance::Important,
},
],
}]
))
);
}
#[test]
fn test_parse_named_colour() {
assert_eq!(
super::parse_declaration("color: white"),
Ok((
"",
Some(Declaration {
data: Decl::Color {
value: Colour::Rgb(0xff, 0xff, 0xff)
},
important: Importance::Default,
})
))
);
}
#[test]
fn test_parse_colour_func() {
assert_eq!(
super::parse_declaration("color: rgb(1, 2, 3)"),
Ok((
"",
Some(Declaration {
data: Decl::Color {
value: Colour::Rgb(1, 2, 3)
},
important: Importance::Default,
})
))
);
}
#[test]
fn test_parse_multi_selector() {
assert_eq!(
super::parse_stylesheet(
"
.foo a .foo-bar2 { color: inherit; }
.foo a.foo-bar .foo-bar2 { color: #112233; }
"
),
Ok((
"",
vec![
RuleSet {
selectors: vec![Selector {
components: vec![
SelectorComponent::Class("foo-bar2".into()),
SelectorComponent::CombDescendant,
SelectorComponent::Element("a".into()),
SelectorComponent::CombDescendant,
SelectorComponent::Class("foo".into()),
],
..Default::default()
},],
declarations: vec![Declaration {
data: Decl::Unknown {
name: PropertyName("color".into()),
},
important: Importance::Default,
},],
},
RuleSet {
selectors: vec![Selector {
components: vec![
SelectorComponent::Class("foo-bar2".into()),
SelectorComponent::CombDescendant,
SelectorComponent::Class("foo-bar".into()),
SelectorComponent::Element("a".into()),
SelectorComponent::CombDescendant,
SelectorComponent::Class("foo".into()),
],
..Default::default()
},],
declarations: vec![Declaration {
data: Decl::Color {
value: Colour::Rgb(0x11, 0x22, 0x33)
},
important: Importance::Default,
},],
}
]
))
);
}
#[test]
fn test_parse_comma_selector() {
assert_eq!(
super::parse_stylesheet(
"
.foo a, p { color: #112233; }
"
),
Ok((
"",
vec![RuleSet {
selectors: vec![
Selector {
components: vec![
SelectorComponent::Element("a".into()),
SelectorComponent::CombDescendant,
SelectorComponent::Class("foo".into()),
],
..Default::default()
},
Selector {
components: vec![SelectorComponent::Element("p".into()),],
..Default::default()
},
],
declarations: vec![Declaration {
data: Decl::Color {
value: Colour::Rgb(0x11, 0x22, 0x33)
},
important: Importance::Default,
},],
},]
))
);
}
#[test]
fn test_parse_before_after() {
assert_eq!(
super::parse_stylesheet(
"
.foo a::before, p::after { color: #112233; }
"
),
Ok((
"",
vec![RuleSet {
selectors: vec![
Selector {
components: vec![
SelectorComponent::Element("a".into()),
SelectorComponent::CombDescendant,
SelectorComponent::Class("foo".into()),
],
pseudo_element: Some(PseudoElement::Before),
},
Selector {
components: vec![SelectorComponent::Element("p".into()),],
pseudo_element: Some(PseudoElement::After),
},
],
declarations: vec![Declaration {
data: Decl::Color {
value: Colour::Rgb(0x11, 0x22, 0x33)
},
important: Importance::Default,
},],
},]
))
);
}
#[test]
fn test_parse_content() {
assert_eq!(
super::parse_stylesheet(
r#"
.foo a::before, p::after { content: "blah*#"; }
"#
),
Ok((
"",
vec![RuleSet {
selectors: vec![
Selector {
components: vec![
SelectorComponent::Element("a".into()),
SelectorComponent::CombDescendant,
SelectorComponent::Class("foo".into()),
],
pseudo_element: Some(PseudoElement::Before),
},
Selector {
components: vec![SelectorComponent::Element("p".into()),],
pseudo_element: Some(PseudoElement::After),
},
],
declarations: vec![Declaration {
data: Decl::Content {
text: "blah*#".into()
},
important: Importance::Default,
},],
},]
))
);
}
#[test]
fn test_background() {
assert_eq!(
super::parse_declaration("background: white"),
Ok((
"",
Some(Declaration {
data: Decl::BackgroundColor {
value: Colour::Rgb(0xff, 0xff, 0xff)
},
important: Importance::Default,
})
))
);
assert_eq!(
super::parse_declaration("background: url('blah'), white"),
Ok((
"",
Some(Declaration {
data: Decl::BackgroundColor {
value: Colour::Rgb(0xff, 0xff, 0xff)
},
important: Importance::Default,
})
))
);
assert_eq!(
super::parse_declaration("background: url('blah'), foo"),
Ok((
"",
Some(Declaration {
data: Decl::Unknown {
name: PropertyName("background".into()),
},
important: Importance::Default,
})
))
);
}
#[test]
fn test_nth_child() {
use SelectorComponent::NthChild;
let (_, sel_all) = super::parse_selector("*").unwrap();
assert_eq!(
super::parse_selector(":nth-child(even)").unwrap(),
(
"",
Selector {
components: vec![NthChild {
a: 2,
b: 0,
sel: sel_all.clone()
}],
..Default::default()
}
)
);
assert_eq!(
super::parse_selector(":nth-child(odd)").unwrap(),
(
"",
Selector {
components: vec![NthChild {
a: 2,
b: 1,
sel: sel_all.clone()
}],
..Default::default()
}
)
);
assert_eq!(
super::parse_selector(":nth-child(17)").unwrap(),
(
"",
Selector {
components: vec![NthChild {
a: 0,
b: 17,
sel: sel_all.clone()
}],
..Default::default()
}
)
);
assert_eq!(
super::parse_selector(":nth-child(17n)").unwrap(),
(
"",
Selector {
components: vec![NthChild {
a: 17,
b: 0,
sel: sel_all.clone()
}],
..Default::default()
}
)
);
assert_eq!(
super::parse_selector(":nth-child(10n-1)").unwrap(),
(
"",
Selector {
components: vec![NthChild {
a: 10,
b: -1,
sel: sel_all.clone()
}],
..Default::default()
}
)
);
assert_eq!(
super::parse_selector(":nth-child(10n+9)").unwrap(),
(
"",
Selector {
components: vec![NthChild {
a: 10,
b: 9,
sel: sel_all.clone()
}],
..Default::default()
}
)
);
assert_eq!(
super::parse_selector(":nth-child(-n+3)").unwrap(),
(
"",
Selector {
components: vec![NthChild {
a: -1,
b: 3,
sel: sel_all.clone()
}],
..Default::default()
}
)
);
assert_eq!(
super::parse_selector(":nth-child(n)").unwrap(),
(
"",
Selector {
components: vec![NthChild {
a: 1,
b: 0,
sel: sel_all.clone()
}],
..Default::default()
}
)
);
assert_eq!(
super::parse_selector(":nth-child(+n)").unwrap(),
(
"",
Selector {
components: vec![NthChild {
a: 1,
b: 0,
sel: sel_all.clone()
}],
..Default::default()
}
)
);
assert_eq!(
super::parse_selector(":nth-child(-n)").unwrap(),
(
"",
Selector {
components: vec![NthChild {
a: -1,
b: 0,
sel: sel_all.clone()
}],
..Default::default()
}
)
);
}
#[test]
fn test_attr() {
use AttrOperator::*;
use SelectorComponent::{Attr, Class, Element};
assert_eq!(
super::parse_selector("[foo]").unwrap(),
(
"",
Selector {
components: vec![Attr {
name: "foo".into(),
value: None,
op: Present,
}],
..Default::default()
}
)
);
assert_eq!(
super::parse_selector("[foo=bar]").unwrap(),
(
"",
Selector {
components: vec![Attr {
name: "foo".into(),
value: Some("bar".into()),
op: Equal,
}],
..Default::default()
}
)
);
assert_eq!(
super::parse_selector("[foo='some string']").unwrap(),
(
"",
Selector {
components: vec![Attr {
name: "foo".into(),
value: Some("some string".into()),
op: Equal,
}],
..Default::default()
}
)
);
assert_eq!(
super::parse_selector("x.y[foo='some string']").unwrap(),
(
"",
Selector {
components: vec![
Attr {
name: "foo".into(),
value: Some("some string".into()),
op: Equal,
},
Class("y".into()),
Element("x".into()),
],
..Default::default()
}
)
);
}
}
================================================
FILE: src/css/types.rs
================================================
#[derive(Copy, Clone, PartialEq, Eq, Debug)]
pub(crate) enum Importance {
Default,
Important,
}
================================================
FILE: src/css.rs
================================================
//! Some basic CSS support.
use std::ops::Deref;
use std::rc::Rc;
#[cfg(feature = "css")]
mod parser;
pub(crate) mod types;
#[cfg(feature = "css")]
use crate::{Colour, Result, WhiteSpace};
#[cfg(feature = "css")]
use parser::parse_style_attribute;
use types::Importance;
use crate::{
ComputedStyle, Specificity, StyleOrigin,
markup5ever_rcdom::{
Handle,
NodeData::{self, Comment, Document, Element},
},
};
#[derive(Debug, Clone, PartialEq, Eq)]
/// Attribute seletor operations
pub(crate) enum AttrOperator {
#[allow(unused)]
Present, // foo[href]
#[allow(unused)]
Equal, // foo[href="foo"]
}
#[derive(Debug, Clone, PartialEq, Eq)]
#[allow(unused)]
pub(crate) enum SelectorComponent {
Class(String),
Element(String),
Hash(String),
Star,
CombChild,
CombDescendant,
NthChild {
/* An + B [of sel] */
a: i32,
b: i32,
sel: Selector,
},
Attr {
name: String,
value: Option<String>,
op: AttrOperator,
// TODO: other comparisions like $=
// TODO: case sensitivity flags
},
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) enum PseudoElement {
Before,
After,
}
impl std::fmt::Display for SelectorComponent {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
SelectorComponent::Class(name) => write!(f, ".{}", name),
SelectorComponent::Element(name) => write!(f, "{}", name),
SelectorComponent::Hash(val) => write!(f, "#{}", val),
SelectorComponent::Star => write!(f, " * "),
SelectorComponent::CombChild => write!(f, " > "),
SelectorComponent::CombDescendant => write!(f, " "),
SelectorComponent::NthChild { a, b, .. } => write!(f, ":nth-child({}n+{})", a, b),
SelectorComponent::Attr { name, value, op } => match op {
AttrOperator::Present => write!(f, "[{name}]"),
AttrOperator::Equal => write!(
f,
"[{name} = \"{}\"]",
value
.as_ref()
.expect("Missing value for attribute equality comparison")
),
},
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Default)]
pub(crate) struct Selector {
// List of components, right first so we match from the leaf.
pub(crate) components: Vec<SelectorComponent>,
pub(crate) pseudo_element: Option<PseudoElement>,
}
impl std::fmt::Display for Selector {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
for comp in self.components.iter().rev() {
comp.fmt(f)?;
}
match self.pseudo_element {
Some(PseudoElement::Before) => write!(f, "::before")?,
Some(PseudoElement::After) => write!(f, "::after")?,
None => (),
}
Ok(())
}
}
impl Selector {
fn do_matches(comps: &[SelectorComponent], node: &Handle) -> bool {
match comps.first() {
None => true,
Some(comp) => match comp {
SelectorComponent::Class(class) => match &node.data {
Document
| NodeData::Doctype { .. }
| NodeData::Text { .. }
| Comment { .. }
| NodeData::ProcessingInstruction { .. } => false,
Element { attrs, .. } => {
let attrs = attrs.borrow();
for attr in attrs.iter() {
if &attr.name.local == "class" {
for cls in attr.value.split_whitespace() {
if cls == class {
return Self::do_matches(&comps[1..], node);
}
}
}
}
false
}
},
SelectorComponent::Attr { name, value, op } => match &node.data {
Document
| NodeData::Doctype { .. }
| NodeData::Text { .. }
| Comment { .. }
| NodeData::ProcessingInstruction { .. } => false,
Element { attrs, .. } => {
let attrs = attrs.borrow();
for attr in attrs.iter() {
if &attr.name.local == name {
match op {
AttrOperator::Present => {
return Self::do_matches(&comps[1..], node);
}
AttrOperator::Equal => {
if &*attr.value
== value
.as_ref()
.expect("No value in Attr equality comparison")
{
return Self::do_matches(&comps[1..], node);
} else {
return false;
}
}
}
}
}
false
}
},
SelectorComponent::Hash(hash) => {
if let Element { attrs, .. } = &node.data {
let attrs = attrs.borrow();
for attr in attrs.iter() {
if &attr.name.local == "id" && &*attr.value == hash {
return Self::do_matches(&comps[1..], node);
}
}
}
false
}
SelectorComponent::Element(name) => match &node.data {
Element { name: eltname, .. } if name == eltname.expanded().local.deref() => {
Self::do_matches(&comps[1..], node)
}
_ => false,
},
SelectorComponent::Star => Self::do_matches(&comps[1..], node),
SelectorComponent::CombChild => match node.get_parent() {
Some(parent) => Self::do_matches(&comps[1..], &parent),
_ => false,
},
SelectorComponent::CombDescendant => match node.get_parent() {
Some(parent) => {
Self::do_matches(&comps[1..], &parent) || Self::do_matches(comps, &parent)
}
_ => false,
},
SelectorComponent::NthChild { a, b, sel } => {
let parent = match node.get_parent() {
Some(parent) => parent,
_ => {
return false;
}
};
let mut idx = 0i32;
for child in parent.children.borrow().iter() {
if let Element { .. } = child.data {
if sel.matches(child) {
idx += 1;
if Rc::ptr_eq(child, node) {
break;
}
} else if Rc::ptr_eq(child, node) {
return false;
}
}
}
if idx == 0 {
// The child wasn't found(?)
return false;
}
/* The selector matches if idx == a*n + b, where
* n >= 0
*/
let idx_offset = idx - b;
if *a == 0 {
return idx_offset == 0 && Self::do_matches(&comps[1..], node);
}
if (idx_offset % a) != 0 {
// Not a multiple
return false;
}
let n = idx_offset / a;
n >= 0 && Self::do_matches(&comps[1..], node)
}
},
}
}
fn matches(&self, node: &Handle) -> bool {
Self::do_matches(&self.components, node)
}
fn specificity(&self) -> Specificity {
let mut result: Specificity = Default::default();
for component in &self.components {
match component {
SelectorComponent::Class(_) | SelectorComponent::Attr { .. } => {
result.class += 1;
}
SelectorComponent::Element(_) => {
result.typ += 1;
}
SelectorComponent::Hash(_) => {
result.id += 1;
}
SelectorComponent::Star => {}
SelectorComponent::CombChild => {}
SelectorComponent::CombDescendant => {}
SelectorComponent::NthChild { sel, .. } => {
result.class += 1;
result += &sel.specificity();
}
}
}
result
}
}
#[cfg(feature = "css")]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum Display {
/// display: none
None,
#[cfg(feature = "css_ext")]
/// Show node as HTML DOM
ExtRawDom,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct PseudoContent {
/// content: "foo"
pub(crate) text: String,
}
#[cfg(feature = "css_ext")]
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct SyntaxInfo {
/// Highlight language
pub(crate) language: String,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) enum Style {
#[cfg(feature = "css")]
Colour(Colour),
#[cfg(feature = "css")]
BgColour(Colour),
#[cfg(feature = "css")]
Display(Display),
#[cfg(feature = "css")]
WhiteSpace(WhiteSpace),
Content(PseudoContent),
#[cfg(feature = "css_ext")]
Syntax(SyntaxInfo),
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct StyleDecl {
pub(crate) style: Style,
pub(crate) importance: Importance,
}
impl std::fmt::Display for StyleDecl {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match &self.style {
#[cfg(feature = "css")]
Style::Colour(col) => write!(f, "color: {}", col)?,
#[cfg(feature = "css")]
Style::BgColour(col) => write!(f, "background-color: {}", col)?,
#[cfg(feature = "css")]
Style::Display(Display::None) => write!(f, "display: none")?,
#[cfg(feature = "css_ext")]
Style::Display(Display::ExtRawDom) => write!(f, "display: x-raw-dom")?,
#[cfg(feature = "css")]
Style::WhiteSpace(ws) => match ws {
WhiteSpace::Normal => write!(f, "white-space: normal")?,
WhiteSpace::Pre => write!(f, "white-space: pre")?,
WhiteSpace::PreWrap => write!(f, "white-space: pre-wrap")?,
},
Style::Content(content) => write!(f, "content: \"{}\"", content.text)?,
#[cfg(feature = "css_ext")]
Style::Syntax(syntax_info) => write!(f, "x-syntax: {}", syntax_info.language)?,
}
match self.importance {
Importance::Default => (),
Importance::Important => write!(f, " !important")?,
}
Ok(())
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct Ruleset {
pub(crate) selector: Selector,
pub(crate) styles: Vec<StyleDecl>,
}
impl std::fmt::Display for Ruleset {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
writeln!(f, " {} {{", self.selector)?;
for decl in &self.styles {
writeln!(f, " {}", decl)?;
}
writeln!(f, " }}")?;
Ok(())
}
}
/// Stylesheet data which can be used while building the render tree.
#[derive(Clone, Default, Debug, PartialEq, Eq)]
pub(crate) struct StyleData {
agent_rules: Vec<Ruleset>,
user_rules: Vec<Ruleset>,
author_rules: Vec<Ruleset>,
}
#[cfg(feature = "css")]
fn styles_from_properties(
decls: &[parser::Declaration],
_allow_extensions: bool,
) -> Vec<StyleDecl> {
let mut styles = Vec::new();
html_trace_quiet!("styles:from_properties2: {decls:?}");
let mut overflow_hidden = false;
let mut height_zero = false;
for decl in decls {
html_trace_quiet!("styles:from_properties2: {decl:?}");
match &decl.data {
parser::Decl::Unknown { .. } => {}
parser::Decl::Color {
value: parser::Colour::Rgb(r, g, b),
} => {
styles.push(StyleDecl {
style: Style::Colour(Colour {
r: *r,
g: *g,
b: *b,
}),
importance: decl.important,
});
}
parser::Decl::BackgroundColor {
value: parser::Colour::Rgb(r, g, b),
} => {
styles.push(StyleDecl {
style: Style::BgColour(Colour {
r: *r,
g: *g,
b: *b,
}),
importance: decl.important,
});
}
parser::Decl::Height { value } => match value {
parser::Height::Auto => (),
parser::Height::Length(l, _) => {
if *l == 0.0 {
height_zero = true;
}
}
},
parser::Decl::MaxHeight { value } => match value {
parser::Height::Auto => (),
parser::Height::Length(l, _) => {
if *l == 0.0 {
height_zero = true;
}
}
},
parser::Decl::Overflow {
value: parser::Overflow::Hidden,
}
| parser::Decl::OverflowY {
value: parser::Overflow::Hidden,
} => {
overflow_hidden = true;
}
parser::Decl::Overflow { .. } | parser::Decl::OverflowY { .. } => {}
parser::Decl::Display { value } => match value {
parser::Display::None => {
styles.push(StyleDecl {
style: Style::Display(Display::None),
importance: decl.important,
});
}
#[cfg(feature = "css_ext")]
parser::Display::RawDom => {
if !_allow_extensions {
continue;
}
styles.push(StyleDecl {
style: Style::Display(Display::ExtRawDom),
importance: decl.important,
});
}
_ => (),
},
parser::Decl::WhiteSpace { value } => {
styles.push(StyleDecl {
style: Style::WhiteSpace(*value),
importance: decl.important,
});
}
parser::Decl::Content { text } => {
styles.push(StyleDecl {
style: Style::Content(PseudoContent { text: text.clone() }),
importance: decl.important,
});
}
#[cfg(feature = "css_ext")]
parser::Decl::XSyntax { language } => {
if !_allow_extensions {
continue;
}
styles.push(StyleDecl {
style: Style::Syntax(SyntaxInfo {
language: language.clone(),
}),
importance: decl.important,
});
} /*
_ => {
html_trace_quiet!("CSS: Unhandled property {:?}", decl);
}
*/
}
}
// If the height is set to zero and overflow hidden, treat as display: none
if height_zero && overflow_hidden {
styles.push(StyleDecl {
style: Style::Display(Display::None),
importance: Importance::Default,
});
}
styles
}
impl StyleData {
#[cfg(feature = "css")]
/// Add some CSS source to be included. The source will be parsed
/// and the relevant and supported features extracted.
fn do_add_css(css: &str, rules: &mut Vec<Ruleset>, allow_extensions: bool) -> Result<()> {
let (_, ss) = parser::parse_stylesheet(css).map_err(|_| crate::Error::CssParseError)?;
for rule in ss {
let styles = styles_from_properties(&rule.declarations, allow_extensions);
if !styles.is_empty() {
for selector in rule.selectors {
let ruleset = Ruleset {
selector,
styles: styles.clone(),
};
html_trace_quiet!("Adding ruleset {ruleset:?}");
rules.push(ruleset);
}
}
}
Ok(())
}
pub(crate) fn add_agent_rules(&mut self, rules: &[Ruleset]) {
for rule in rules {
self.agent_rules.push(rule.clone());
}
}
#[cfg(feature = "css")]
/// Add some CSS source to be included as part of the user agent ("browser") CSS rules.
pub fn add_agent_css(&mut self, css: &str) -> Result<()> {
Self::do_add_css(css, &mut self.agent_rules, true)
}
#[cfg(feature = "css")]
/// Add some CSS source to be included as part of the user CSS rules.
pub fn add_user_css(&mut self, css: &str) -> Result<()> {
Self::do_add_css(css, &mut self.user_rules, true)
}
#[cfg(feature = "css")]
/// Add some CSS source to be included as part of the document/author CSS rules.
pub fn add_author_css(&mut self, css: &str) -> Result<()> {
Self::do_add_css(css, &mut self.author_rules, false)
}
#[cfg(feature = "css")]
/// Merge style data from other into this one.
/// Data on other takes precedence.
pub fn merge(&mut self, other: Self) {
self.agent_rules.extend(other.agent_rules);
self.user_rules.extend(other.user_rules);
self.author_rules.extend(other.author_rules);
}
pub(crate) fn computed_style(
&self,
parent_style: &ComputedStyle,
handle: &Handle,
_use_doc_css: bool,
) -> ComputedStyle {
let mut result = parent_style.inherit();
for (origin, ruleset) in [
(StyleOrigin::Agent, &self.agent_rules),
(StyleOrigin::User, &self.user_rules),
(StyleOrigin::Author, &self.author_rules),
] {
for rule in ruleset {
if rule.selector.matches(handle) {
for style in rule.styles.iter() {
Self::merge_computed_style(
&mut result,
style.importance == Importance::Important,
origin,
rule.selector.specificity(),
rule.selector.pseudo_element.as_ref(),
style,
);
}
}
}
}
#[cfg(feature = "css")]
if _use_doc_css {
// Now look for a style attribute
if let Element { attrs, .. } = &handle.data {
let borrowed = attrs.borrow();
for attr in borrowed.iter() {
if &attr.name.local == "style" {
let rules = parse_style_attribute(&attr.value).unwrap_or_default();
for style in rules {
Self::merge_computed_style(
&mut result,
false,
StyleOrigin::Author,
Specificity::inline(),
None,
&style,
);
}
} else if &*attr.name.local == "color" {
if let Ok(colour) = parser::parse_color_attribute(&attr.value) {
Self::merge_computed_style(
&mut result,
false,
StyleOrigin::Author,
Specificity::inline(),
None,
&StyleDecl {
style: Style::Colour(colour.into()),
importance: Importance::Default,
},
);
}
} else if &*attr.name.local == "bgcolor" {
if let Ok(colour) = parser::parse_color_attribute(&attr.value) {
Self::merge_computed_style(
&mut result,
false,
StyleOrigin::Author,
Specificity::inline(),
None,
&StyleDecl {
style: Style::BgColour(colour.into()),
importance: Importance::Default,
},
);
}
}
}
}
}
result
}
fn merge_computed_style(
result: &mut ComputedStyle,
important: bool,
origin: StyleOrigin,
specificity: Specificity,
pseudo_selectors: Option<&PseudoElement>,
style: &StyleDecl,
) {
let result_target = match pseudo_selectors {
None => result,
Some(PseudoElement::Before) => {
// TODO: ideally we should inherit from the parent; however we haven't finished
// computing the parent yet.
result.content_before.get_or_insert_with(Default::default)
}
Some(PseudoElement::After) => result.content_after.get_or_insert_with(Default::default),
};
// The increasing priority is:
// * agent
// * user
// * author
// * author !important
// * user !important
// * agent !important
// Since we view in the order agent, user, author, we always want to
// replace the value if we haven't yet seen an !important rule, and
// never afterwards.
match style.style {
#[cfg(feature = "css")]
Style::Colour(col) => {
result_target
.colour
.maybe_update(important, origin, specificity, col);
}
#[cfg(feature = "css")]
Style::BgColour(col) => {
result_target
.bg_colour
.maybe_update(important, origin, specificity, col);
}
#[cfg(feature = "css")]
Style::Display(disp) => {
// We don't have a "not DisplayNone" - we might need to fix this.
result_target
.display
.maybe_update(important, origin, specificity, disp);
}
#[cfg(feature = "css")]
Style::WhiteSpace(ws) => {
result_target
.white_space
.maybe_update(important, origin, specificity, ws);
}
Style::Content(ref content) => {
result_target
.content
.maybe_update(important, origin, specificity, content.clone());
}
#[cfg(feature = "css_ext")]
Style::Syntax(ref syntax_info) => {
result_target.syntax.maybe_update(
important,
origin,
specificity,
syntax_info.clone(),
);
}
}
}
}
impl std::fmt::Display for StyleData {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
if !self.agent_rules.is_empty() {
writeln!(f, "Agent rules:")?;
for ruleset in &self.agent_rules {
ruleset.fmt(f)?;
}
}
if !self.user_rules.is_empty() {
writeln!(f, "User rules:")?;
for ruleset in &self.user_rules {
ruleset.fmt(f)?;
}
}
if !self.author_rules.is_empty() {
writeln!(f, "Author rules:")?;
for ruleset in &self.author_rules {
ruleset.fmt(f)?;
}
}
Ok(())
}
}
#[cfg(feature = "css")]
pub(crate) mod dom_extract {
use std::io::Write;
use crate::{expanded_name, local_name, ns};
use crate::{
Result, TreeMapResult,
markup5ever_rcdom::{
Handle,
NodeData::{self, Comment, Document, Element},
},
tree_map_reduce,
};
use super::StyleData;
fn pending<F>(handle: Handle, f: F) -> TreeMapResult<'static, (), Handle, Vec<String>>
where
F: Fn(&mut (), Vec<Vec<String>>) -> Result<Option<Vec<String>>> + 'static,
{
TreeMapResult::PendingChildren {
children: handle.children.borrow().clone(),
cons: Box::new(f),
prefn: None,
postfn: None,
}
}
fn combine_vecs(vecs: Vec<Vec<String>>) -> Vec<String> {
let mut it = vecs.into_iter();
let first = it.next();
match first {
None => Vec::new(),
Some(mut first) => {
for v in it {
first.extend(v.into_iter());
}
first
}
}
}
fn extract_style_nodes<T: Write>(
handle: Handle,
_err_out: &mut T,
) -> TreeMapResult<'static, (), Handle, Vec<String>> {
use TreeMapResult::*;
match handle.clone().data {
Document => pending(handle, |&mut (), cs| Ok(Some(combine_vecs(cs)))),
Comment { .. } => Nothing,
Element { ref name, .. } => {
match name.expanded() {
expanded_name!(html "style") => {
let mut result = String::new();
// Assume just a flat text node
for child in handle.children.borrow().iter() {
if let NodeData::Text { ref contents } = child.data {
result += &contents.borrow();
}
}
Finished(vec![result])
}
_ => pending(handle, |_, cs| Ok(Some(combine_vecs(cs)))),
}
}
NodeData::Text {
contents: ref _tstr,
} => Nothing,
_ => {
// NodeData doesn't have a Debug impl.
Nothing
}
}
}
/// Extract stylesheet data from document.
pub(crate) fn dom_to_stylesheet<T: Write>(
handle: Handle,
err_out: &mut T,
) -> Result<StyleData> {
let styles = tree_map_reduce(&mut (), handle, |_, handle| {
Ok(extract_style_nodes(handle, err_out))
})?;
let mut result = StyleData::default();
if let Some(styles) = styles {
for css in styles {
// Ignore CSS parse errors.
let _ = result.add_author_css(&css);
}
}
Ok(result)
}
}
#[cfg(feature = "css")]
#[cfg(test)]
mod tests {
use crate::Specificity;
use super::parser::parse_selector;
#[test]
fn test_specificity() {
let sel_id1 = parse_selector("#foo").unwrap().1;
assert_eq!(
sel_id1.specificity(),
Specificity {
id: 1,
..Default::default()
}
);
let sel_cl3 = parse_selector(".foo .bar .baz").unwrap().1;
assert_eq!(
sel_cl3.specificity(),
Specificity {
class: 3,
..Default::default()
}
);
assert!(sel_id1.specificity() > sel_cl3.specificity());
}
}
================================================
FILE: src/lib.rs
================================================
//! Convert HTML to text formats.
//!
//! This crate renders HTML into a text format, wrapped to a specified width.
//! This can either be plain text or with extra annotations to (for example)
//! show in a terminal which supports colours.
//!
//! # Examples
//!
//! ```rust
//! # use html2text::from_read;
//! let html = b"
//! <ul>
//! <li>Item one</li>
//! <li>Item two</li>
//! <li>Item three</li>
//! </ul>";
//! assert_eq!(from_read(&html[..], 20).unwrap(),
//! "\
//! * Item one
//! * Item two
//! * Item three
//! ");
//! ```
//! A couple of simple demonstration programs are included as examples:
//!
//! ### html2text
//!
//! The simplest example uses `from_read` to convert HTML on stdin into plain
//! text:
//!
//! ```sh
//! $ cargo run --example html2text < foo.html
//! [...]
//! ```
//!
//! ### html2term
//!
//! A very simple example of using the rich interface (`from_read_rich`) for a
//! slightly interactive console HTML viewer is provided as `html2term`.
//!
//! ```sh
//! $ cargo run --example html2term foo.html
//! [...]
//! ```
//!
//! Note that this example takes the HTML file as a parameter so that it can
//! read keys from stdin.
//!
#![deny(missing_docs)]
// Check code in README.md
#[cfg(doctest)]
#[doc = include_str!("../README.md")]
struct ReadMe;
#[macro_use]
mod macros;
pub mod css;
pub mod render;
/// Extra methods on chars for dealing with special cases with wrapping and whitespace.
trait WhitespaceExt {
/// Returns whether this character always takes space. This is true for non-whitespace and
/// non-breaking spaces.
fn always_takes_space(&self) -> bool;
/// Returns true if a word before this character is allowed. This includes most whitespace
/// (but not non-breaking space).
fn is_wordbreak_point(&self) -> bool;
}
impl WhitespaceExt for char {
fn always_takes_space(&self) -> bool {
match *self {
'\u{A0}' => true,
c if !c.is_whitespace() => true,
_ => false,
}
}
fn is_wordbreak_point(&self) -> bool {
match *self {
'\u{00A0}' => false,
'\u{200b}' => true,
c if c.is_whitespace() => true,
_ => false,
}
}
}
/// Extra methods for strings
trait StrExt {
/// Trims leading/trailing whitespace expect for hard spaces.
fn trim_collapsible_ws(&self) -> &str;
}
impl StrExt for str {
fn trim_collapsible_ws(&self) -> &str {
self.trim_matches(|c: char| !c.always_takes_space())
}
}
#[cfg(feature = "css_ext")]
/// Text style information.
#[derive(Clone, Debug)]
#[non_exhaustive]
pub struct TextStyle {
/// The foreground colour
pub fg_colour: Colour,
/// The background colour, or None.
pub bg_colour: Option<Colour>,
}
#[cfg(feature = "css_ext")]
impl TextStyle {
/// Create a TextStyle from foreground and background colours.
pub fn colours(fg_colour: Colour, bg_colour: Colour) -> Self {
TextStyle {
fg_colour,
bg_colour: Some(bg_colour),
}
}
/// Create a TextStyle using only a foreground colour.
pub fn foreground(fg_colour: Colour) -> Self {
TextStyle {
fg_colour,
bg_colour: None,
}
}
}
#[cfg(feature = "css_ext")]
/// Syntax highlighter function.
///
/// Takes a string corresponding to some text to be highlighted, and returns
/// spans with sub-strs of that text with associated colours.
pub type SyntaxHighlighter = Box<dyn for<'a> Fn(&'a str) -> Vec<(TextStyle, &'a str)>>;
use markup5ever_rcdom::Node;
use render::text_renderer::{
RenderLine, RenderOptions, RichAnnotation, SubRenderer, TaggedLine, TextRenderer,
};
use render::{Renderer, TextDecorator, TrivialDecorator};
use html5ever::driver::ParseOpts;
use html5ever::parse_document;
use html5ever::tree_builder::TreeBuilderOpts;
mod markup5ever_rcdom;
pub use html5ever::{expanded_name, local_name, namespace_url, ns};
pub use markup5ever_rcdom::{
Handle,
NodeData::{Comment, Document, Element},
RcDom,
};
use std::cell::{Cell, RefCell};
use std::cmp::{max, min};
use std::collections::{BTreeSet, HashMap};
#[cfg(feature = "css_ext")]
use std::ops::Range;
use std::rc::Rc;
use unicode_width::UnicodeWidthStr;
use std::io;
use std::io::Write;
use std::iter::{once, repeat};
#[derive(Debug, Copy, Clone, Default, PartialEq, Eq)]
pub(crate) enum WhiteSpace {
#[default]
Normal,
// NoWrap,
Pre,
#[allow(unused)]
PreWrap,
// PreLine,
// BreakSpaces,
}
impl WhiteSpace {
pub fn preserve_whitespace(&self) -> bool {
match self {
WhiteSpace::Normal => false,
WhiteSpace::Pre | WhiteSpace::PreWrap => true,
}
}
#[allow(unused)]
pub fn do_wrap(&self) -> bool {
match self {
WhiteSpace::Normal | WhiteSpace::PreWrap => true,
WhiteSpace::Pre => false,
}
}
}
/// An RGB colour value
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
pub struct Colour {
/// Red value
pub r: u8,
/// Green value
pub g: u8,
/// Blue value
pub b: u8,
}
impl std::fmt::Display for Colour {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "#{:02x}{:02x}{:02x}", self.r, self.g, self.b)
}
}
#[derive(Debug, Copy, Clone, PartialEq, Eq, Default, PartialOrd)]
pub(crate) enum StyleOrigin {
#[default]
None,
Agent,
#[allow(unused)]
User,
#[allow(unused)]
Author,
}
#[derive(Debug, Copy, Clone, PartialEq, Eq, Default)]
pub(crate) struct Specificity {
inline: bool,
id: u16,
class: u16,
typ: u16,
}
impl Specificity {
#[cfg(feature = "css")]
fn inline() -> Self {
Specificity {
inline: true,
id: 0,
class: 0,
typ: 0,
}
}
}
impl std::ops::Add<&Specificity> for &Specificity {
type Output = Specificity;
fn add(self, rhs: &Specificity) -> Self::Output {
Specificity {
inline: self.inline || rhs.inline,
id: self.id + rhs.id,
class: self.class + rhs.class,
typ: self.typ + rhs.typ,
}
}
}
impl std::ops::AddAssign<&Specificity> for Specificity {
fn add_assign(&mut self, rhs: &Specificity) {
self.inline = self.inline || rhs.inline;
self.id += rhs.id;
self.class += rhs.class;
self.typ += rhs.typ;
}
}
impl PartialOrd for Specificity {
fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
match self.inline.partial_cmp(&other.inline) {
Some(core::cmp::Ordering::Equal) => {}
ord => return ord,
}
match self.id.partial_cmp(&other.id) {
Some(core::cmp::Ordering::Equal) => {}
ord => return ord,
}
match self.class.partial_cmp(&other.class) {
Some(core::cmp::Ordering::Equal) => {}
ord => return ord,
}
self.typ.partial_cmp(&other.typ)
}
}
#[derive(Clone, Copy, Debug)]
pub(crate) struct WithSpec<T> {
val: Option<T>,
origin: StyleOrigin,
specificity: Specificity,
important: bool,
}
impl<T: Clone> WithSpec<T> {
pub(crate) fn maybe_update(
&mut self,
important: bool,
origin: StyleOrigin,
specificity: Specificity,
val: T,
) {
if self.val.is_some() {
// We already have a value, so need to check.
if self.important && !important {
// important takes priority over not important.
return;
}
// importance is the same. Next is checking the origin.
{
use StyleOrigin::*;
match (self.origin, origin) {
(Agent, Agent) | (User, User) | (Author, Author) => {
// They're the same so continue the comparison
}
(mine, theirs) => {
if (important && theirs > mine) || (!important && mine > theirs) {
return;
}
}
}
}
// We're now from the same origin an importance
if specificity < self.specificity {
return;
}
}
self.val = Some(val);
self.origin = origin;
self.specificity = specificity;
self.important = important;
}
pub fn val(&self) -> Option<&T> {
self.val.as_ref()
}
}
impl<T> Default for WithSpec<T> {
fn default() -> Self {
WithSpec {
val: None,
origin: StyleOrigin::None,
specificity: Default::default(),
important: false,
}
}
}
#[derive(Debug, Clone, Default)]
pub(crate) struct ComputedStyle {
#[cfg(feature = "css")]
/// The computed foreground colour, if any
pub(crate) colour: WithSpec<Colour>,
#[cfg(feature = "css")]
/// The computed background colour, if any
pub(crate) bg_colour: WithSpec<Colour>,
#[cfg(feature = "css")]
/// If set, indicates whether `display: none` or something equivalent applies
pub(crate) display: WithSpec<css::Display>,
/// The CSS white-space property
pub(crate) white_space: WithSpec<WhiteSpace>,
/// The CSS content property
pub(crate) content: WithSpec<css::PseudoContent>,
#[cfg(feature = "css_ext")]
pub(crate) syntax: WithSpec<css::SyntaxInfo>,
/// The CSS content property for ::before
pub(crate) content_before: Option<Box<ComputedStyle>>,
/// The CSS content property for ::after
pub(crate) content_after: Option<Box<ComputedStyle>>,
/// A non-CSS flag indicating we're inside a <pre>.
pub(crate) internal_pre: bool,
}
impl ComputedStyle {
/// Return the style data inherited by children.
pub(crate) fn inherit(&self) -> Self {
// TODO: clear fields that shouldn't be inherited
self.clone()
}
}
/// Errors from reading or rendering HTML
#[derive(thiserror::Error, Debug)]
#[non_exhaustive]
pub enum Error {
/// The output width was too narrow to render to.
#[error("Output width not wide enough.")]
TooNarrow,
/// CSS parse error
#[error("Invalid CSS")]
CssParseError,
/// An general error was encountered.
#[error("Unknown failure")]
Fail,
/// An I/O error
#[error("I/O error")]
IoError(#[from] io::Error),
}
impl PartialEq for Error {
fn eq(&self, other: &Error) -> bool {
use Error::*;
match (self, other) {
(TooNarrow, TooNarrow) => true,
#[cfg(feature = "css")]
(CssParseError, CssParseError) => true,
(Fail, Fail) => true,
_ => false,
}
}
}
impl Eq for Error {}
type Result<T> = std::result::Result<T, Error>;
const MIN_WIDTH: usize = 3;
/// Size information/estimate
#[derive(Debug, Copy, Clone, Default)]
struct SizeEstimate {
size: usize, // Rough overall size
min_width: usize, // The narrowest possible
// The use is specific to the node type.
prefix_size: usize,
}
impl SizeEstimate {
/// Combine two estimates into one (add size and take the largest
/// min width)
fn add(self, other: SizeEstimate) -> SizeEstimate {
let min_width = max(self.min_width, other.min_width);
SizeEstimate {
size: self.size + other.size,
min_width,
prefix_size: 0,
}
}
/// Combine two estimates into one which need to be side by side.
/// The min widths are added.
fn add_hor(self, other: SizeEstimate) -> SizeEstimate {
SizeEstimate {
size: self.size + other.size,
min_width: self.min_width + other.min_width,
prefix_size: 0,
}
}
/// Combine two estimates into one (take max of each)
fn max(self, other: SizeEstimate) -> SizeEstimate {
SizeEstimate {
size: max(self.size, other.size),
min_width: max(self.min_width, other.min_width),
prefix_size: 0,
}
}
}
#[derive(Clone, Debug)]
/// Render tree table cell
struct RenderTableCell {
colspan: usize,
rowspan: usize,
content: Vec<RenderNode>,
size_estimate: Cell<Option<SizeEstimate>>,
col_width: Option<usize>, // Actual width to use
x_pos: Option<usize>, // X location
style: ComputedStyle,
is_dummy: bool,
}
impl RenderTableCell {
/// Calculate or return the estimate size of the cell
fn get_size_estimate(&self) -> SizeEstimate {
let Some(size) = self.size_estimate.get() else {
let size = self
.content
.iter()
.map(|node| node.get_size_estimate())
.fold(Default::default(), SizeEstimate::add);
self.size_estimate.set(Some(size));
return size;
};
size
}
/// Make a placeholder cell to cover for a cell above with
/// larger rowspan.
pub fn dummy(colspan: usize) -> Self {
RenderTableCell {
colspan,
rowspan: 1,
content: Default::default(),
size_estimate: Cell::new(Some(SizeEstimate::default())),
col_width: None,
x_pos: None,
style: Default::default(),
is_dummy: true,
}
}
}
#[derive(Clone, Debug)]
/// Render tree table row
struct RenderTableRow {
cells: Vec<RenderTableCell>,
col_sizes: Option<Vec<usize>>,
style: ComputedStyle,
}
impl RenderTableRow {
/// Return a mutable iterator over the cells.
fn cells(&self) -> std::slice::Iter<'_, RenderTableCell> {
self.cells.iter()
}
/// Return a mutable iterator over the cells.
fn cells_mut(&mut self) -> std::slice::IterMut<'_, RenderTableCell> {
self.cells.iter_mut()
}
/// Return an iterator which returns cells by values (removing
/// them from the row).
fn cells_drain(&mut self) -> impl Iterator<Item = RenderTableCell> + use<> {
std::mem::take(&mut self.cells).into_iter()
}
/// Count the number of cells in the row.
/// Takes into account colspan.
fn num_cells(&self) -> usize {
self.cells.iter().map(|cell| cell.colspan.max(1)).sum()
}
/// Return the contained cells as RenderNodes, annotated with their
/// widths if available. Skips cells with no width allocated.
fn into_cells(self, vertical: bool) -> Vec<RenderNode> {
let mut result = Vec::new();
let mut colno = 0;
let col_sizes = self.col_sizes.unwrap();
let mut x_pos = 0;
for mut cell in self.cells {
let colspan = cell.colspan;
let col_width = if vertical {
col_sizes[colno]
} else {
col_sizes[colno..colno + cell.colspan].iter().sum::<usize>()
};
// Skip any zero-width columns
if col_width > 0 {
let this_col_width = col_width + cell.colspan - 1;
cell.col_width = Some(this_col_width);
cell.x_pos = Some(x_pos);
x_pos += this_col_width + 1;
let style = cell.style.clone();
result.push(RenderNode::new_styled(
RenderNodeInfo::TableCell(cell),
style,
));
}
colno += colspan;
}
result
}
}
#[derive(Clone, Debug)]
/// A representation of a table render tree with metadata.
struct RenderTable {
rows: Vec<RenderTableRow>,
num_columns: usize,
size_estimate: Cell<Option<SizeEstimate>>,
}
impl RenderTable {
/// Create a new RenderTable with the given rows
fn new(mut rows: Vec<RenderTableRow>) -> RenderTable {
// We later on want to allocate a vector sized by the column count,
// but occasionally we see something like colspan="1000000000". We
// handle this by remapping the column ids to the smallest values
// possible.
//
// Tables with no explicit colspan will be unchanged, but if there
// are multiple columns each covered by a single <td> on every row,
// they will be collapsed into a single column. For example:
//
// <td><td colspan=1000><td>
// <td colspan=1000><td><td>
//
// becomes the equivalent:
// <td><td colspan=2><td>
// <td colspan=2><td><td>
// This will include 0 and the index after the last colspan.
let mut col_positions = BTreeSet::new();
// Cells which have a rowspan > 1 from previous rows.
// Each element is (rows_left, colpos, colspan)
// Before each row, the overhangs are in reverse order so that
// they can be popped off.
let mut overhang_cells: Vec<(usize, usize, usize)> = Vec::new();
let mut next_overhang_cells = Vec::new();
col_positions.insert(0);
for row in &mut rows {
let mut col = 0;
let mut new_cells = Vec::new();
for cell in row.cells_drain() {
while let Some(hanging) = overhang_cells.last() {
if hanging.1 <= col {
new_cells.push(RenderTableCell::dummy(hanging.2));
col += hanging.2;
col_positions.insert(col);
let mut used = overhang_cells.pop().unwrap();
if used.0 > 1 {
used.0 -= 1;
next_overhang_cells.push(used);
}
} else {
break;
}
}
if cell.rowspan > 1 {
next_overhang_cells.push((cell.rowspan - 1, col, cell.colspan));
}
col += cell.colspan;
col_positions.insert(col);
new_cells.push(cell);
}
// Handle remaining overhanging cells
while let Some(mut hanging) = overhang_cells.pop() {
new_cells.push(RenderTableCell::dummy(hanging.2));
col += hanging.2;
col_positions.insert(col);
if hanging.0 > 1 {
hanging.0 -= 1;
next_overhang_cells.push(hanging);
}
}
row.cells = new_cells;
overhang_cells = std::mem::take(&mut next_overhang_cells);
overhang_cells.reverse();
}
let colmap: HashMap<_, _> = col_positions
.into_iter()
.enumerate()
.map(|(i, pos)| (pos, i))
.collect();
for row in &mut rows {
let mut pos = 0;
let mut mapped_pos = 0;
for cell in row.cells_mut() {
let nextpos = pos + cell.colspan.max(1);
let next_mapped_pos = *colmap.get(&nextpos).unwrap();
cell.colspan = next_mapped_pos - mapped_pos;
pos = nextpos;
mapped_pos = next_mapped_pos;
}
}
let num_columns = rows.iter().map(|r| r.num_cells()).max().unwrap_or(0);
RenderTable {
rows,
num_columns,
size_estimate: Cell::new(None),
}
}
/// Return an iterator over the rows.
fn rows(&self) -> std::slice::Iter<'_, RenderTableRow> {
self.rows.iter()
}
/// Consume this and return a `Vec<RenderNode>` containing the children;
/// the children know the column sizes required.
fn into_rows(self, col_sizes: Vec<usize>, vert: bool) -> Vec<RenderNode> {
self.rows
.into_iter()
.map(|mut tr| {
tr.col_sizes = Some(col_sizes.clone());
let style = tr.style.clone();
RenderNode::new_styled(RenderNodeInfo::TableRow(tr, vert), style)
})
.collect()
}
fn calc_size_estimate(&self, _context: &HtmlContext) -> SizeEstimate {
if self.num_columns == 0 {
let result = SizeEstimate {
size: 0,
min_width: 0,
prefix_size: 0,
};
self.size_estimate.set(Some(result));
return result;
}
let mut sizes: Vec<SizeEstimate> = vec![Default::default(); self.num_columns];
// For now, a simple estimate based on adding up sub-parts.
for row in self.rows() {
let mut colno = 0usize;
for cell in row.cells() {
let cellsize = cell.get_size_estimate();
for colnum in 0..cell.colspan {
sizes[colno + colnum].size += cellsize.size / cell.colspan;
sizes[colno + colnum].min_width = max(
sizes[colno + colnum].min_width,
cellsize.min_width / cell.colspan,
);
}
colno += cell.colspan;
}
}
let size = sizes.iter().map(|s| s.size).sum::<usize>() + self.num_columns.saturating_sub(1);
let min_width = sizes.iter().map(|s| s.min_width).sum::<usize>() + self.num_columns - 1;
let result = SizeEstimate {
size,
min_width,
prefix_size: 0,
};
self.size_estimate.set(Some(result));
result
}
}
/// The node-specific information distilled from the DOM.
#[derive(Clone, Debug)]
#[non_exhaustive]
enum RenderNodeInfo {
/// Some text.
Text(String),
/// A group of nodes collected together.
Container(Vec<RenderNode>),
/// A link with contained nodes
Link(String, Vec<RenderNode>),
/// An emphasised region
Em(Vec<RenderNode>),
/// A strong region
Strong(Vec<RenderNode>),
/// A struck out region
Strikeout(Vec<RenderNode>),
/// A code region
Code(Vec<RenderNode>),
/// An image (src, title)
Img(String, String),
/// An inline SVG (title)
Svg(String),
/// A block element with children
Block(Vec<RenderNode>),
/// A header (h1, h2, ...) with children
Header(usize, Vec<RenderNode>),
/// A Div element with children
Div(Vec<RenderNode>),
/// A blockquote
BlockQuote(Vec<RenderNode>),
/// An unordered list
Ul(Vec<RenderNode>),
/// An ordered list
Ol(i64, Vec<RenderNode>),
/// A description list (containing Dt or Dd)
Dl(Vec<RenderNode>),
/// A term (from a `<dt>`)
Dt(Vec<RenderNode>),
/// A definition (from a `<dl>`)
Dd(Vec<RenderNode>),
/// A line break
Break,
/// A table
Table(RenderTable),
/// A set of table rows (from either `<thead>` or `<tbody>`
TableBody(Vec<RenderTableRow>),
/// Table row (must only appear within a table body)
/// If the boolean is true, then the cells are drawn vertically
/// instead of horizontally (because of space).
TableRow(RenderTableRow, bool),
/// Table cell (must only appear within a table row)
TableCell(RenderTableCell),
/// Start of a named HTML fragment
FragStart(String),
/// A list item
ListItem(Vec<RenderNode>),
/// Superscript text
Sup(Vec<RenderNode>),
}
/// Common fields from a node.
#[derive(Clone, Debug)]
struct RenderNode {
size_estimate: Cell<Option<SizeEstimate>>,
info: RenderNodeInfo,
style: ComputedStyle,
}
impl RenderNode {
/// Create a node from the RenderNodeInfo.
fn new(info: RenderNodeInfo) -> RenderNode {
RenderNode {
size_estimate: Cell::new(None),
info,
style: Default::default(),
}
}
/// Create a node from the RenderNodeInfo.
fn new_styled(info: RenderNodeInfo, style: ComputedStyle) -> RenderNode {
RenderNode {
size_estimate: Cell::new(None),
info,
style,
}
}
/// Get a size estimate
fn get_size_estimate(&self) -> SizeEstimate {
self.size_estimate.get().unwrap()
}
/// Calculate the size of this node.
fn calc_size_estimate<D: TextDecorator>(
&self,
context: &HtmlContext,
decorator: &D,
) -> SizeEstimate {
// If it's already calculated, then just return the answer.
if let Some(s) = self.size_estimate.get() {
return s;
};
use RenderNodeInfo::*;
let recurse = |node: &RenderNode| node.calc_size_estimate(context, decorator);
// Otherwise, make an estimate.
let estimate = match self.info {
Text(ref t) | Img(_, ref t) | Svg(ref t) => {
use unicode_width::UnicodeWidthChar;
let mut len = 0;
let mut in_whitespace = false;
for c in t.trim_collapsible_ws().chars() {
let is_collapsible_ws = !c.always_takes_space();
if !is_collapsible_ws {
len += UnicodeWidthChar::width(c).unwrap_or(0);
// Count the preceding whitespace as one.
if in_whitespace {
len += 1;
}
}
in_whitespace = is_collapsible_ws;
}
// Add one for preceding whitespace, unless the node is otherwise empty.
if let Some(true) = t.chars().next().map(|c| !c.always_takes_space()) {
if len > 0 {
len += 1;
}
}
if let Img(_, _) = self.info {
len += 2;
}
SizeEstimate {
size: len,
min_width: len.min(context.min_wrap_width),
prefix_size: 0,
}
}
Container(ref v) | Em(ref v) | Strong(ref v) | Strikeout(ref v) | Code(ref v)
| Block(ref v) | Div(ref v) | Dl(ref v) | Dt(ref v) | ListItem(ref v) | Sup(ref v) => v
.iter()
.map(recurse)
.fold(Default::default(), SizeEstimate::add),
Link(ref _target, ref v) => v
.iter()
.map(recurse)
.fold(Default::default(), SizeEstimate::add)
.add(SizeEstimate {
size: 5,
min_width: 5,
prefix_size: 0,
}),
Dd(ref v) | BlockQuote(ref v) | Ul(ref v) => {
let prefix = match self.info {
Dd(_) => " ".into(),
BlockQuote(_) => decorator.quote_prefix(),
Ul(_) => decorator.unordered_item_prefix(),
_ => unreachable!(),
};
let prefix_width = UnicodeWidthStr::width(prefix.as_str());
let mut size = v
.iter()
.map(recurse)
.fold(Default::default(), SizeEstimate::add)
.add_hor(SizeEstimate {
size: prefix_width,
min_width: prefix_width,
prefix_size: 0,
});
size.prefix_size = prefix_width;
size
}
Ol(i, ref v) => {
let prefix_size = calc_ol_prefix_size(i, v.len(), decorator);
let mut result = v
.iter()
.map(recurse)
.fold(Default::default(), SizeEstimate::add)
.add_hor(SizeEstimate {
size: prefix_size,
min_width: prefix_size,
prefix_size: 0,
});
result.prefix_size = prefix_size;
result
}
Header(level, ref v) => {
let prefix_size = decorator.header_prefix(level).len();
let mut size = v
.iter()
.map(recurse)
.fold(Default::default(), SizeEstimate::add)
.add_hor(SizeEstimate {
size: prefix_size,
min_width: prefix_size,
prefix_size: 0,
});
size.prefix_size = prefix_size;
size
}
Break => SizeEstimate {
size: 1,
min_width: 1,
prefix_size: 0,
},
Table(ref t) => t.calc_size_estimate(context),
TableRow(..) | TableBody(_) | TableCell(_) => unimplemented!(),
FragStart(_) => Default::default(),
};
self.size_estimate.set(Some(estimate));
estimate
}
/// Return true if this node is definitely empty. This is used to quickly
/// remove e.g. links with no anchor text in most cases, but can't recurse
/// and look more deeply.
fn is_shallow_empty(&self) -> bool {
use RenderNodeInfo::*;
// Otherwise, make an estimate.
match self.info {
Text(ref t) | Img(_, ref t) | Svg(ref t) => {
let len = t.trim().len();
len == 0
}
Container(ref v)
| Link(_, ref v)
| Em(ref v)
| Strong(ref v)
| Strikeout(ref v)
| Code(ref v)
| Block(ref v)
| ListItem(ref v)
| Div(ref v)
| BlockQuote(ref v)
| Dl(ref v)
| Dt(ref v)
| Dd(ref v)
| Ul(ref v)
| Ol(_, ref v)
| Sup(ref v) => v.is_empty(),
Header(_level, ref v) => v.is_empty(),
Break => true,
Table(ref _t) => false,
TableRow(..) | TableBody(_) | TableCell(_) => false,
FragStart(_) => true,
}
}
fn wr
gitextract_ep3xu7e5/
├── .circleci/
│ └── config.yml
├── .github/
│ ├── dependabot.yml
│ └── workflows/
│ ├── ci.yml
│ └── jekyll-gh-pages.yml
├── .gitignore
├── CHANGELOG.md
├── Cargo.toml
├── LICENSE
├── README.md
├── benches/
│ └── tables.rs
├── examples/
│ └── html2term.rs
├── html2text-cli/
│ ├── Cargo.toml
│ ├── README.md
│ └── src/
│ └── main.rs
├── html2text-web-demo/
│ ├── .cargo/
│ │ └── config.toml
│ ├── .gitignore
│ ├── Cargo.toml
│ ├── Trunk.toml
│ ├── index.html
│ └── src/
│ └── lib.rs
├── pages/
│ ├── .gitignore
│ ├── _config.yml
│ ├── _includes/
│ │ └── head.html
│ ├── assets/
│ │ ├── demo-main.js
│ │ └── demo.css
│ └── index.markdown
├── rust.yml
└── src/
├── ansi_colours.rs
├── css/
│ ├── parser.rs
│ └── types.rs
├── css.rs
├── lib.rs
├── macros.rs
├── markup5ever_rcdom.rs
├── render/
│ ├── mod.rs
│ └── text_renderer.rs
└── tests.rs
SYMBOL INDEX (837 symbols across 15 files)
FILE: benches/tables.rs
function make_html (line 9) | fn make_html(content: &str) -> String {
function make_tab (line 13) | fn make_tab(cell: &str, rows: usize, cols: usize) -> String {
function bench_empty (line 28) | fn bench_empty(b: &mut Bencher) {
function bench_tab_1_1 (line 33) | fn bench_tab_1_1(b: &mut Bencher) {
function bench_tab_2_2 (line 37) | fn bench_tab_2_2(b: &mut Bencher) {
function bench_tab_3_3 (line 41) | fn bench_tab_3_3(b: &mut Bencher) {
function bench_tab_4_4 (line 45) | fn bench_tab_4_4(b: &mut Bencher) {
function bench_tab_5_5 (line 49) | fn bench_tab_5_5(b: &mut Bencher) {
function bench_tab_6_6 (line 53) | fn bench_tab_6_6(b: &mut Bencher) {
function bench_tab_depth (line 57) | fn bench_tab_depth(b: &mut Bencher, content: &str, depth: usize, rows: u...
function bench_tab_2_1_depth_2 (line 66) | fn bench_tab_2_1_depth_2(b: &mut Bencher) {
function bench_tab_3_1_depth_2 (line 70) | fn bench_tab_3_1_depth_2(b: &mut Bencher) {
function bench_tab_4_1_depth_2 (line 74) | fn bench_tab_4_1_depth_2(b: &mut Bencher) {
function bench_tab_1_2_depth_2 (line 78) | fn bench_tab_1_2_depth_2(b: &mut Bencher) {
function bench_tab_1_3_depth_2 (line 82) | fn bench_tab_1_3_depth_2(b: &mut Bencher) {
function bench_tab_1_4_depth_2 (line 86) | fn bench_tab_1_4_depth_2(b: &mut Bencher) {
function bench_tab_2_depth_2 (line 90) | fn bench_tab_2_depth_2(b: &mut Bencher) {
FILE: examples/html2term.rs
function to_style (line 20) | fn to_style(tag: &[RichAnnotation]) -> String {
type LinkMap (line 62) | struct LinkMap {
method link_at (line 67) | pub fn link_at(&self, x: usize, y: usize) -> Option<&str> {
function link_from_tag (line 77) | fn link_from_tag(tag: &[RichAnnotation]) -> Option<String> {
function find_links (line 87) | fn find_links(lines: &[TaggedLine<Vec<RichAnnotation>>]) -> LinkMap {
type FragMap (line 104) | struct FragMap {
function find_frags (line 108) | fn find_frags(lines: &[TaggedLine<Vec<RichAnnotation>>]) -> FragMap {
type Options (line 128) | struct Options {
method new (line 134) | fn new() -> Options {
function main (line 142) | pub fn main() {
function rerender (line 353) | fn rerender(
function main (line 418) | pub fn main() {}
function main (line 421) | fn main() {
FILE: html2text-cli/src/main.rs
function default_colour_map (line 13) | fn default_colour_map(
function do_syntect_highlight (line 95) | fn do_syntect_highlight<'t>(text: &'t str, language: &str) -> Vec<(html2...
function update_config (line 129) | fn update_config<T: TextDecorator>(mut config: Config<T>, flags: &Flags)...
function translate (line 162) | fn translate<R>(input: R, flags: Flags, literal: bool) -> String
type Flags (line 225) | struct Flags {
function main (line 249) | fn main() {
FILE: html2text-web-demo/src/lib.rs
type Config (line 19) | pub struct Config {
method new (line 39) | pub fn new() -> Self {
method use_colour (line 45) | pub fn use_colour(&mut self) {
method use_css (line 49) | pub fn use_css(&mut self) {
method add_user_css (line 53) | pub fn add_user_css(&mut self, css: String) {
method add_agent_css (line 61) | pub fn add_agent_css(&mut self, css: String) {
method pad_block_width (line 69) | pub fn pad_block_width(&mut self) {
method max_wrap_width (line 73) | pub fn max_wrap_width(&mut self, width: usize) {
method allow_overflow (line 77) | pub fn allow_overflow(&mut self) {
method min_wrap_width (line 81) | pub fn min_wrap_width(&mut self, width: usize) {
method raw_mode (line 84) | pub fn raw_mode(&mut self) {
method no_borders (line 87) | pub fn no_borders(&mut self) {
method no_link_wrap (line 90) | pub fn no_link_wrap(&mut self) {
method unicode_so (line 93) | pub fn unicode_so(&mut self) {
method do_decorate (line 96) | pub fn do_decorate(&mut self) {
method link_footnotes (line 99) | pub fn link_footnotes(&mut self, value: bool) {
method image_mode (line 103) | pub fn image_mode(&mut self, value: &str) {
method update_conf (line 113) | fn update_conf<D: TextDecorator>(&self, conf: html2text::config::Confi...
function do_render_colour (line 161) | fn do_render_colour(f: &mut Frame, config: &Config, input: &[u8]) -> Res...
function format_html (line 206) | pub fn format_html(config: Config, input: &str) -> Result<(), String> {
FILE: pages/assets/demo-main.js
function update_html (line 25) | function update_html() {
function start (line 88) | function start() {
FILE: src/ansi_colours.rs
function from_read_coloured (line 19) | pub fn from_read_coloured<R, FMap>(
FILE: src/css.rs
type AttrOperator (line 26) | pub(crate) enum AttrOperator {
type SelectorComponent (line 35) | pub(crate) enum SelectorComponent {
method fmt (line 64) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
type PseudoElement (line 58) | pub(crate) enum PseudoElement {
type Selector (line 88) | pub(crate) struct Selector {
method fmt (line 95) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
method do_matches (line 109) | fn do_matches(comps: &[SelectorComponent], node: &Handle) -> bool {
method matches (line 233) | fn matches(&self, node: &Handle) -> bool {
method specificity (line 236) | fn specificity(&self) -> Specificity {
type Display (line 266) | pub(crate) enum Display {
type PseudoContent (line 275) | pub(crate) struct PseudoContent {
type SyntaxInfo (line 282) | pub(crate) struct SyntaxInfo {
type Style (line 288) | pub(crate) enum Style {
type StyleDecl (line 303) | pub(crate) struct StyleDecl {
method fmt (line 309) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
type Ruleset (line 338) | pub(crate) struct Ruleset {
method fmt (line 344) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
type StyleData (line 356) | pub(crate) struct StyleData {
method do_add_css (line 487) | fn do_add_css(css: &str, rules: &mut Vec<Ruleset>, allow_extensions: b...
method add_agent_rules (line 506) | pub(crate) fn add_agent_rules(&mut self, rules: &[Ruleset]) {
method add_agent_css (line 514) | pub fn add_agent_css(&mut self, css: &str) -> Result<()> {
method add_user_css (line 520) | pub fn add_user_css(&mut self, css: &str) -> Result<()> {
method add_author_css (line 526) | pub fn add_author_css(&mut self, css: &str) -> Result<()> {
method merge (line 533) | pub fn merge(&mut self, other: Self) {
method computed_style (line 539) | pub(crate) fn computed_style(
method merge_computed_style (line 622) | fn merge_computed_style(
method fmt (line 694) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
function styles_from_properties (line 363) | fn styles_from_properties(
function pending (line 734) | fn pending<F>(handle: Handle, f: F) -> TreeMapResult<'static, (), Handle...
function combine_vecs (line 746) | fn combine_vecs(vecs: Vec<Vec<String>>) -> Vec<String> {
function extract_style_nodes (line 760) | fn extract_style_nodes<T: Write>(
function dom_to_stylesheet (line 795) | pub(crate) fn dom_to_stylesheet<T: Write>(
function test_specificity (line 822) | fn test_specificity() {
FILE: src/css/parser.rs
type Colour (line 16) | pub(crate) enum Colour {
function from (line 21) | fn from(value: Colour) -> Self {
type LengthUnit (line 29) | pub(crate) enum LengthUnit {
type Height (line 43) | pub(crate) enum Height {
type Overflow (line 51) | pub(crate) enum Overflow {
type Display (line 60) | pub(crate) enum Display {
type Decl (line 69) | pub(crate) enum Decl {
type Token (line 110) | enum Token<'s> {
type RawValue (line 165) | struct RawValue<'s> {
type Declaration (line 171) | pub(crate) struct Declaration {
type RuleSet (line 181) | pub(crate) struct RuleSet {
type PropertyName (line 187) | pub(crate) struct PropertyName(String);
function match_comment (line 189) | fn match_comment(text: &str) -> IResult<&str, ()> {
function match_whitespace_item (line 195) | fn match_whitespace_item(text: &str) -> IResult<&str, ()> {
function skip_optional_whitespace (line 199) | fn skip_optional_whitespace(text: &str) -> IResult<&str, ()> {
function nmstart_char (line 203) | fn nmstart_char(s: &str) -> IResult<&str, char> {
function is_ident_start (line 214) | fn is_ident_start(c: char) -> bool {
function is_digit (line 218) | fn is_digit(c: char) -> bool {
function nmchar_char (line 222) | fn nmchar_char(s: &str) -> IResult<&str, char> {
function ident_escape (line 235) | fn ident_escape(s: &str) -> IResult<&str, char> {
function nmstart (line 266) | fn nmstart(text: &str) -> IResult<&str, char> {
function nmchar (line 270) | fn nmchar(text: &str) -> IResult<&str, char> {
function parse_ident (line 274) | fn parse_ident(text: &str) -> IResult<&str, String> {
function parse_identstring (line 290) | fn parse_identstring(text: &str) -> IResult<&str, String> {
function parse_property_name (line 297) | fn parse_property_name(text: &str) -> IResult<&str, PropertyName> {
function parse_token (line 302) | fn parse_token(text: &str) -> IResult<&str, Token<'_>> {
function parse_token_not_semicolon (line 372) | fn parse_token_not_semicolon(text: &str) -> IResult<&str, Token<'_>> {
function parse_value (line 381) | fn parse_value(text: &str) -> IResult<&str, RawValue<'_>> {
function parse_color_attribute (line 394) | pub(crate) fn parse_color_attribute(
function parse_color_part (line 401) | fn parse_color_part(text: &str, index: std::ops::Range<usize>) -> Option...
function parse_faulty_color (line 407) | fn parse_faulty_color(
function parse_declaration (line 421) | pub(crate) fn parse_declaration(text: &str) -> IResult<&str, Option<Decl...
function empty_fail (line 524) | fn empty_fail() -> nom::Err<nom::error::Error<&'static str>> {
function parse_color (line 528) | fn parse_color(tokens: &[Token]) -> Result<Colour, nom::Err<nom::error::...
function parse_background_color (line 596) | fn parse_background_color(
function parse_integer (line 611) | fn parse_integer(text: &str) -> IResult<&str, f32> {
function parse_decimal (line 616) | fn parse_decimal(text: &str) -> IResult<&str, f32> {
function parse_number (line 621) | fn parse_number(text: &str) -> IResult<&str, f32> {
function parse_numeric_token (line 638) | fn parse_numeric_token(text: &str) -> IResult<&str, Token<'_>> {
function parse_ident_like (line 650) | fn parse_ident_like(text: &str) -> IResult<&str, Token<'_>> {
function parse_string_token (line 660) | fn parse_string_token(text: &str) -> IResult<&str, Token<'_>> {
function parse_quoted_string (line 698) | fn parse_quoted_string(text: &str) -> IResult<&str, String> {
function parse_height (line 718) | fn parse_height(value: &RawValue) -> Result<Height, nom::Err<nom::error:...
function parse_overflow (line 749) | fn parse_overflow(value: &RawValue) -> Result<Overflow, nom::Err<nom::er...
function parse_display (line 772) | fn parse_display(value: &RawValue) -> Result<Display, nom::Err<nom::erro...
function parse_syntax (line 788) | fn parse_syntax(value: &RawValue) -> Result<String, nom::Err<nom::error:...
function parse_white_space (line 795) | fn parse_white_space(
function parse_content (line 812) | fn parse_content(value: &RawValue) -> Result<String, nom::Err<nom::error...
function parse_rules (line 824) | pub(crate) fn parse_rules(text: &str) -> IResult<&str, Vec<Declaration>> {
function parse_class (line 830) | fn parse_class(text: &str) -> IResult<&str, SelectorComponent> {
function parse_attr (line 836) | fn parse_attr(text: &str) -> IResult<&str, SelectorComponent> {
type Sign (line 872) | enum Sign {
method val (line 878) | fn val(&self) -> i32 {
function opt_sign (line 886) | fn opt_sign(text: &str) -> IResult<&str, Sign> {
function sign (line 893) | fn sign(text: &str) -> IResult<&str, Sign> {
function parse_nth_child_args (line 901) | fn parse_nth_child_args(text: &str) -> IResult<&str, SelectorComponent> {
function parse_pseudo_class (line 951) | fn parse_pseudo_class(text: &str) -> IResult<&str, SelectorComponent> {
function parse_hash (line 963) | fn parse_hash(text: &str) -> IResult<&str, SelectorComponent> {
function parse_ws (line 970) | fn parse_ws(text: &str) -> IResult<&str, ()> {
function parse_simple_selector_component (line 974) | fn parse_simple_selector_component(text: &str) -> IResult<&str, Selector...
function parse_selector_with_element (line 994) | fn parse_selector_with_element(text: &str) -> IResult<&str, Vec<Selector...
function parse_selector_without_element (line 1002) | fn parse_selector_without_element(text: &str) -> IResult<&str, Vec<Selec...
function parse_pseudo_element (line 1006) | pub(crate) fn parse_pseudo_element(text: &str) -> IResult<&str, Option<P...
function parse_selector (line 1014) | pub(crate) fn parse_selector(text: &str) -> IResult<&str, Selector> {
function parse_ruleset (line 1041) | fn parse_ruleset(text: &str) -> IResult<&str, RuleSet> {
function skip_to_end_of_statement (line 1066) | fn skip_to_end_of_statement(text: &str) -> IResult<&str, ()> {
function parse_at_rule (line 1131) | fn parse_at_rule(text: &str) -> IResult<&str, ()> {
function parse_statement (line 1143) | fn parse_statement(text: &str) -> IResult<&str, Option<RuleSet>> {
function parse_stylesheet (line 1147) | pub(crate) fn parse_stylesheet(text: &str) -> IResult<&str, Vec<RuleSet>> {
function parse_style_attribute (line 1152) | pub(crate) fn parse_style_attribute(text: &str) -> crate::Result<Vec<Sty...
function test_parse_decl (line 1171) | fn test_parse_decl() {
function test_parse_overflow (line 1188) | fn test_parse_overflow() {
function test_parse_color (line 1212) | fn test_parse_color() {
function test_parse_height (line 1248) | fn test_parse_height() {
function test_parse_empty_ss (line 1272) | fn test_parse_empty_ss() {
function test_parse_ss_col (line 1277) | fn test_parse_ss_col() {
function test_parse_class (line 1305) | fn test_parse_class() {
function test_parse_at_rules (line 1342) | fn test_parse_at_rules() {
function test_parse_named_colour (line 1393) | fn test_parse_named_colour() {
function test_parse_colour_func (line 1409) | fn test_parse_colour_func() {
function test_parse_multi_selector (line 1425) | fn test_parse_multi_selector() {
function test_parse_comma_selector (line 1479) | fn test_parse_comma_selector() {
function test_parse_before_after (line 1515) | fn test_parse_before_after() {
function test_parse_content (line 1551) | fn test_parse_content() {
function test_background (line 1587) | fn test_background() {
function test_nth_child (line 1627) | fn test_nth_child() {
function test_attr (line 1772) | fn test_attr() {
FILE: src/css/types.rs
type Importance (line 2) | pub(crate) enum Importance {
FILE: src/lib.rs
type ReadMe (line 55) | struct ReadMe;
type WhitespaceExt (line 64) | trait WhitespaceExt {
method always_takes_space (line 67) | fn always_takes_space(&self) -> bool;
method is_wordbreak_point (line 71) | fn is_wordbreak_point(&self) -> bool;
method always_takes_space (line 75) | fn always_takes_space(&self) -> bool {
method is_wordbreak_point (line 83) | fn is_wordbreak_point(&self) -> bool {
type StrExt (line 94) | trait StrExt {
method trim_collapsible_ws (line 96) | fn trim_collapsible_ws(&self) -> &str;
method trim_collapsible_ws (line 100) | fn trim_collapsible_ws(&self) -> &str {
type TextStyle (line 109) | pub struct TextStyle {
method colours (line 119) | pub fn colours(fg_colour: Colour, bg_colour: Colour) -> Self {
method foreground (line 127) | pub fn foreground(fg_colour: Colour) -> Self {
type SyntaxHighlighter (line 140) | pub type SyntaxHighlighter = Box<dyn for<'a> Fn(&'a str) -> Vec<(TextSty...
type WhiteSpace (line 172) | pub(crate) enum WhiteSpace {
method preserve_whitespace (line 184) | pub fn preserve_whitespace(&self) -> bool {
method do_wrap (line 191) | pub fn do_wrap(&self) -> bool {
type Colour (line 201) | pub struct Colour {
method fmt (line 211) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
type StyleOrigin (line 217) | pub(crate) enum StyleOrigin {
type Specificity (line 228) | pub(crate) struct Specificity {
method inline (line 237) | fn inline() -> Self {
method add_assign (line 261) | fn add_assign(&mut self, rhs: &Specificity) {
type Output (line 248) | type Output = Specificity;
function add (line 250) | fn add(self, rhs: &Specificity) -> Self::Output {
method partial_cmp (line 270) | fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
type WithSpec (line 288) | pub(crate) struct WithSpec<T> {
function maybe_update (line 295) | pub(crate) fn maybe_update(
function val (line 333) | pub fn val(&self) -> Option<&T> {
method default (line 339) | fn default() -> Self {
type ComputedStyle (line 350) | pub(crate) struct ComputedStyle {
method inherit (line 378) | pub(crate) fn inherit(&self) -> Self {
type Error (line 387) | pub enum Error {
method eq (line 403) | fn eq(&self, other: &Error) -> bool {
type Result (line 417) | type Result<T> = std::result::Result<T, Error>;
constant MIN_WIDTH (line 419) | const MIN_WIDTH: usize = 3;
type SizeEstimate (line 423) | struct SizeEstimate {
method add (line 434) | fn add(self, other: SizeEstimate) -> SizeEstimate {
method add_hor (line 444) | fn add_hor(self, other: SizeEstimate) -> SizeEstimate {
method max (line 453) | fn max(self, other: SizeEstimate) -> SizeEstimate {
type RenderTableCell (line 464) | struct RenderTableCell {
method get_size_estimate (line 477) | fn get_size_estimate(&self) -> SizeEstimate {
method dummy (line 492) | pub fn dummy(colspan: usize) -> Self {
type RenderTableRow (line 508) | struct RenderTableRow {
method cells (line 516) | fn cells(&self) -> std::slice::Iter<'_, RenderTableCell> {
method cells_mut (line 520) | fn cells_mut(&mut self) -> std::slice::IterMut<'_, RenderTableCell> {
method cells_drain (line 525) | fn cells_drain(&mut self) -> impl Iterator<Item = RenderTableCell> + u...
method num_cells (line 530) | fn num_cells(&self) -> usize {
method into_cells (line 536) | fn into_cells(self, vertical: bool) -> Vec<RenderNode> {
type RenderTable (line 568) | struct RenderTable {
method new (line 576) | fn new(mut rows: Vec<RenderTableRow>) -> RenderTable {
method rows (line 671) | fn rows(&self) -> std::slice::Iter<'_, RenderTableRow> {
method into_rows (line 677) | fn into_rows(self, col_sizes: Vec<usize>, vert: bool) -> Vec<RenderNod...
method calc_size_estimate (line 688) | fn calc_size_estimate(&self, _context: &HtmlContext) -> SizeEstimate {
type RenderNodeInfo (line 730) | enum RenderNodeInfo {
type RenderNode (line 789) | struct RenderNode {
method new (line 797) | fn new(info: RenderNodeInfo) -> RenderNode {
method new_styled (line 806) | fn new_styled(info: RenderNodeInfo, style: ComputedStyle) -> RenderNode {
method get_size_estimate (line 815) | fn get_size_estimate(&self) -> SizeEstimate {
method calc_size_estimate (line 820) | fn calc_size_estimate<D: TextDecorator>(
method is_shallow_empty (line 945) | fn is_shallow_empty(&self) -> bool {
method write_container (line 979) | fn write_container(
method write_style (line 992) | fn write_style(
method write_self (line 1023) | fn write_self(
function precalc_size_estimate (line 1130) | fn precalc_size_estimate<'a, D: TextDecorator>(
function table_to_render_tree (line 1193) | fn table_to_render_tree<'a, T: Write>(
function tbody_to_render_tree (line 1219) | fn tbody_to_render_tree<'a, T: Write>(
function tr_to_render_tree (line 1272) | fn tr_to_render_tree<'a, T: Write>(
function td_to_render_tree (line 1305) | fn td_to_render_tree<'a, T: Write>(
type ResultReducer (line 1345) | type ResultReducer<'a, C, R> = dyn FnOnce(&mut C, Vec<R>) -> Result<Opti...
type ChildPreFn (line 1348) | type ChildPreFn<C, N> = dyn Fn(&mut C, &N) -> Result<()>;
type ChildPostFn (line 1353) | type ChildPostFn<C, R> = dyn Fn(&mut C, &R) -> Result<()>;
type TreeMapResult (line 1356) | enum TreeMapResult<'a, C, N, R> {
function tree_map_reduce (line 1371) | fn tree_map_reduce<'a, C, N, R, M>(
type HighlighterMap (line 1452) | struct HighlighterMap {
method get (line 1458) | pub fn get(&self, name: &str) -> Option<Rc<SyntaxHighlighter>> {
method insert (line 1462) | fn insert(&mut self, name: impl Into<String>, f: Rc<SyntaxHighlighter>) {
method fmt (line 1469) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
method eq (line 1478) | fn eq(&self, _other: &Self) -> bool {
type HtmlContext (line 1487) | struct HtmlContext {
type RenderInput (line 1511) | struct RenderInput {
method new (line 1522) | fn new(handle: Handle, parent_style: Rc<ComputedStyle>) -> Self {
method set_syntax_info (line 1533) | fn set_syntax_info(&self, full_text: &str, highlighted: Vec<(TextStyle...
method children (line 1564) | fn children(&self) -> Vec<RenderInput> {
method do_extract_text (line 1623) | fn do_extract_text(
method extract_raw_text (line 1650) | fn extract_raw_text(&self) -> String {
function dom_to_render_tree_with_context (line 1661) | fn dom_to_render_tree_with_context<T: Write>(
function dom_to_parsed_style (line 1688) | pub fn dom_to_parsed_style(dom: &RcDom) -> Result<String> {
function pending (line 1694) | fn pending<F>(
function pending_noempty (line 1709) | fn pending_noempty<F>(
type ChildPosition (line 1738) | enum ChildPosition {
function insert_child (line 1745) | fn insert_child(
function process_dom_node (line 1822) | fn process_dom_node<T: Write>(
function render_tree_to_string (line 2297) | fn render_tree_to_string<T: Write, D: TextDecorator>(
function pending2 (line 2324) | fn pending2<
type PushedStyleInfo (line 2346) | struct PushedStyleInfo {
method apply (line 2354) | fn apply<D: TextDecorator>(render: &mut TextRenderer<D>, style: &Compu...
method unwind (line 2379) | fn unwind<D: TextDecorator>(self, renderer: &mut TextRenderer<D>) {
function do_render_node (line 2395) | fn do_render_node<T: Write, D: TextDecorator>(
function render_table_tree (line 2675) | fn render_table_tree<T: Write, D: TextDecorator>(
function render_table_row (line 2782) | fn render_table_row<T: Write, D: TextDecorator>(
function render_table_row_vert (line 2816) | fn render_table_row_vert<T: Write, D: TextDecorator>(
function render_table_cell (line 2843) | fn render_table_cell<T: Write, D: TextDecorator>(
type ImageRenderMode (line 2878) | pub enum ImageRenderMode {
type XmlMode (line 2894) | pub enum XmlMode {
type Config (line 2906) | pub struct Config<D: TextDecorator> {
function make_context (line 2935) | pub(crate) fn make_context(&self) -> HtmlContext {
function do_parse (line 2960) | pub(crate) fn do_parse<R>(&self, context: &mut HtmlContext, input: R) ->...
function parse_html (line 2997) | pub fn parse_html<R: io::Read>(&self, mut input: R) -> Result<super::RcD...
function parse_xml (line 3013) | pub fn parse_xml<R: io::Read>(&self, mut input: R) -> Result<super::RcDo...
function dom_to_render_tree (line 3022) | pub fn dom_to_render_tree(&self, dom: &super::RcDom) -> Result<RenderTre...
function render_to_string (line 3034) | pub fn render_to_string(&self, render_tree: RenderTree, width: usize) ->...
function render_to_lines (line 3049) | pub fn render_to_lines(
function string_from_read (line 3065) | pub fn string_from_read<R: std::io::Read>(self, input: R, width: usize) ...
function lines_from_read (line 3078) | pub fn lines_from_read<R: std::io::Read>(
function add_css (line 3092) | pub fn add_css(mut self, css: &str) -> Result<Self> {
function add_agent_css (line 3100) | pub fn add_agent_css(mut self, css: &str) -> Result<Self> {
function use_doc_css (line 3107) | pub fn use_doc_css(mut self) -> Self {
function pad_block_width (line 3113) | pub fn pad_block_width(mut self) -> Self {
function max_wrap_width (line 3121) | pub fn max_wrap_width(mut self, wrap_width: usize) -> Self {
function allow_width_overflow (line 3130) | pub fn allow_width_overflow(mut self) -> Self {
function min_wrap_width (line 3140) | pub fn min_wrap_width(mut self, min_wrap_width: usize) -> Self {
function raw_mode (line 3148) | pub fn raw_mode(mut self, raw: bool) -> Self {
function no_table_borders (line 3155) | pub fn no_table_borders(mut self) -> Self {
function no_link_wrapping (line 3160) | pub fn no_link_wrapping(mut self) -> Self {
function unicode_strikeout (line 3166) | pub fn unicode_strikeout(mut self, use_unicode: bool) -> Self {
function make_surround_rule (line 3172) | fn make_surround_rule(element: &str, after: bool, content: &str) -> Rule...
function do_decorate (line 3192) | pub fn do_decorate(mut self) -> Self {
function link_footnotes (line 3209) | pub fn link_footnotes(mut self, include_footnotes: bool) -> Self {
function empty_img_mode (line 3215) | pub fn empty_img_mode(mut self, img_mode: ImageRenderMode) -> Self {
function xml_mode (line 3222) | pub fn xml_mode(mut self, xml_mode: XmlMode) -> Self {
function register_highlighter (line 3232) | pub fn register_highlighter(
function coloured (line 3249) | pub fn coloured<R, FMap>(self, input: R, width: usize, colour_map: FMap)...
function render_coloured (line 3262) | pub fn render_coloured<FMap>(
function rich (line 3285) | pub fn rich() -> Config<RichDecorator> {
function plain (line 3290) | pub fn plain() -> Config<PlainDecorator> {
function plain_no_decorate (line 3297) | pub fn plain_no_decorate() -> Config<PlainDecorator> {
function with_decorator (line 3302) | pub fn with_decorator<D: TextDecorator>(decorator: D) -> Config<D> {
type RenderTree (line 3331) | pub struct RenderTree(RenderNode);
method fmt (line 3334) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
method render_with_context (line 3342) | fn render_with_context<D: TextDecorator>(
type RenderedText (line 3371) | struct RenderedText<D: TextDecorator>(SubRenderer<D>);
function into_string (line 3375) | fn into_string(self) -> render::Result<String> {
function into_lines (line 3381) | fn into_lines(self) -> Result<Vec<TaggedLine<Vec<D::Annotation>>>> {
function parse (line 3392) | pub fn parse(input: impl io::Read) -> Result<RenderTree> {
function from_read_with_decorator (line 3399) | pub fn from_read_with_decorator<R, D>(input: R, width: usize, decorator:...
function from_read (line 3409) | pub fn from_read<R>(input: R, width: usize) -> Result<String>
function from_read_rich (line 3420) | pub fn from_read_rich<R>(input: R, width: usize) -> Result<Vec<TaggedLin...
function calc_ol_prefix_size (line 3434) | fn calc_ol_prefix_size<D: TextDecorator>(start: i64, num_items: usize, d...
FILE: src/macros.rs
function nop (line 10) | pub fn nop() {}
FILE: src/markup5ever_rcdom.rs
type NodeData (line 63) | pub enum NodeData {
type Node (line 112) | pub struct Node {
method new (line 123) | pub fn new(data: NodeData) -> Rc<Self> {
method get_parent (line 131) | pub fn get_parent(&self) -> Option<Rc<Self>> {
method nth_child (line 143) | pub fn nth_child(&self, idx: usize) -> Option<Rc<Self>> {
method element_name (line 157) | pub fn element_name(&self) -> Option<String> {
method serialize (line 166) | pub fn serialize(self: &Rc<Self>, writer: impl io::Write) -> io::Resul...
method fmt (line 199) | fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
method drop (line 180) | fn drop(&mut self) {
type Handle (line 208) | pub type Handle = Rc<Node>;
type WeakHandle (line 211) | pub type WeakHandle = Weak<Node>;
function append (line 214) | fn append(new_parent: &Handle, child: Handle) {
function get_parent_and_index (line 222) | fn get_parent_and_index(target: &Handle) -> Option<(Handle, usize)> {
function append_to_existing_text (line 239) | fn append_to_existing_text(prev: &Handle, text: &str) -> bool {
function remove_from_parent (line 249) | fn remove_from_parent(target: &Handle) {
type RcDom (line 257) | pub struct RcDom {
method add_node_to_string (line 269) | fn add_node_to_string(s: &mut String, node: &Handle, indent: usize) {
method as_dom_string (line 300) | pub fn as_dom_string(&self) -> String {
method node_as_dom_string (line 307) | pub fn node_as_dom_string(node: &Handle) -> String {
method serialize (line 314) | pub fn serialize(&self, mut writer: impl io::Write) -> io::Result<()> {
method get_node_by_path (line 333) | pub fn get_node_by_path(&self, path: &[usize]) -> Option<Handle> {
type Output (line 343) | type Output = Self;
type ElemName (line 345) | type ElemName<'a> = ExpandedName<'a>;
method finish (line 346) | fn finish(self) -> Self {
type Handle (line 350) | type Handle = Handle;
method parse_error (line 352) | fn parse_error(&self, msg: Cow<'static, str>) {
method get_document (line 356) | fn get_document(&self) -> Handle {
method get_template_contents (line 360) | fn get_template_contents(&self, target: &Handle) -> Handle {
method set_quirks_mode (line 376) | fn set_quirks_mode(&self, mode: QuirksMode) {
method same_node (line 380) | fn same_node(&self, x: &Handle, y: &Handle) -> bool {
method elem_name (line 384) | fn elem_name<'a>(&self, target: &'a Handle) -> ExpandedName<'a> {
method create_element (line 391) | fn create_element(&self, name: QualName, attrs: Vec<Attribute>, flags: E...
method create_comment (line 404) | fn create_comment(&self, text: StrTendril) -> Handle {
method create_pi (line 408) | fn create_pi(&self, target: StrTendril, data: StrTendril) -> Handle {
method append (line 415) | fn append(&self, parent: &Handle, child: NodeOrText<Handle>) {
method append_before_sibling (line 436) | fn append_before_sibling(&self, sibling: &Handle, child: NodeOrText<Hand...
method append_based_on_parent_node (line 471) | fn append_based_on_parent_node(
method append_doctype_to_document (line 488) | fn append_doctype_to_document(
method add_attrs_if_missing (line 504) | fn add_attrs_if_missing(&self, target: &Handle, attrs: Vec<Attribute>) {
method remove_from_parent (line 522) | fn remove_from_parent(&self, target: &Handle) {
method reparent_children (line 526) | fn reparent_children(&self, node: &Handle, new_parent: &Handle) {
method is_mathml_annotation_xml_integration_point (line 539) | fn is_mathml_annotation_xml_integration_point(&self, target: &Handle) ->...
method default (line 553) | fn default() -> RcDom {
type SerializeOp (line 562) | enum SerializeOp {
type SerializableHandle (line 567) | pub struct SerializableHandle(Handle);
method from (line 570) | fn from(h: Handle) -> SerializableHandle {
method serialize (line 576) | fn serialize<S>(&self, serializer: &mut S, traversal_scope: TraversalSco...
FILE: src/render/mod.rs
type Result (line 14) | pub(crate) type Result<T> = std::result::Result<T, TooNarrow>;
type TooNarrow (line 17) | pub(crate) struct TooNarrow;
function from (line 20) | fn from(_: TooNarrow) -> crate::Error {
type Renderer (line 26) | pub(crate) trait Renderer {
method add_empty_line (line 28) | fn add_empty_line(&mut self) -> Result<()>;
method new_sub_renderer (line 31) | fn new_sub_renderer(&self, width: usize) -> Result<Self>
method start_block (line 36) | fn start_block(&mut self) -> Result<()>;
method start_table (line 39) | fn start_table(&mut self) -> Result<()>;
method end_block (line 42) | fn end_block(&mut self);
method new_line (line 45) | fn new_line(&mut self) -> Result<()>;
method new_line_hard (line 48) | fn new_line_hard(&mut self) -> Result<()>;
method add_horizontal_border (line 51) | fn add_horizontal_border(&mut self) -> Result<()>;
method add_horizontal_border_width (line 54) | fn add_horizontal_border_width(
method push_preformat (line 63) | fn push_preformat(&mut self);
method pop_preformat (line 66) | fn pop_preformat(&mut self);
method push_ws (line 69) | fn push_ws(&mut self, ws: WhiteSpace);
method pop_ws (line 72) | fn pop_ws(&mut self);
method add_inline_text (line 76) | fn add_inline_text(&mut self, text: &str) -> Result<()>;
method width (line 79) | fn width(&self) -> usize;
method append_subrender (line 83) | fn append_subrender<'a, I>(&mut self, other: Self, prefixes: I) -> Res...
method append_columns_with_borders (line 91) | fn append_columns_with_borders<I>(&mut self, cols: I, collapse: bool) ...
method append_vert_row (line 98) | fn append_vert_row<I>(&mut self, cols: I) -> Result<()>
method empty (line 104) | fn empty(&self) -> bool;
method start_link (line 109) | fn start_link(&mut self, target: &str) -> Result<()>;
method end_link (line 112) | fn end_link(&mut self) -> Result<()>;
method start_emphasis (line 115) | fn start_emphasis(&mut self) -> Result<()>;
method end_emphasis (line 118) | fn end_emphasis(&mut self) -> Result<()>;
method start_strong (line 121) | fn start_strong(&mut self) -> Result<()>;
method end_strong (line 124) | fn end_strong(&mut self) -> Result<()>;
method start_strikeout (line 127) | fn start_strikeout(&mut self) -> Result<()>;
method end_strikeout (line 130) | fn end_strikeout(&mut self) -> Result<()>;
method start_code (line 133) | fn start_code(&mut self) -> Result<()>;
method end_code (line 136) | fn end_code(&mut self) -> Result<()>;
method add_image (line 139) | fn add_image(&mut self, src: &str, title: &str) -> Result<()>;
method header_prefix (line 142) | fn header_prefix(&mut self, level: usize) -> String;
method quote_prefix (line 145) | fn quote_prefix(&mut self) -> String;
method unordered_item_prefix (line 148) | fn unordered_item_prefix(&mut self) -> String;
method ordered_item_prefix (line 151) | fn ordered_item_prefix(&mut self, i: i64) -> String;
method record_frag_start (line 154) | fn record_frag_start(&mut self, fragname: &str);
method push_colour (line 158) | fn push_colour(&mut self, colour: Colour);
method pop_colour (line 162) | fn pop_colour(&mut self);
method push_bgcolour (line 166) | fn push_bgcolour(&mut self, colour: Colour);
method pop_bgcolour (line 170) | fn pop_bgcolour(&mut self);
method start_superscript (line 173) | fn start_superscript(&mut self) -> Result<()>;
method end_superscript (line 176) | fn end_superscript(&mut self) -> Result<()>;
FILE: src/render/text_renderer.rs
type TextRenderer (line 26) | pub(crate) struct TextRenderer<D: TextDecorator> {
type Target (line 32) | type Target = SubRenderer<D>;
method deref (line 33) | fn deref(&self) -> &Self::Target {
method deref_mut (line 38) | fn deref_mut(&mut self) -> &mut Self::Target {
function new (line 47) | pub fn new(subrenderer: SubRenderer<D>) -> TextRenderer<D> {
function start_link (line 57) | pub fn start_link(&mut self, target: &str) -> Result<()> {
function end_link (line 63) | pub fn end_link(&mut self) -> Result<()> {
function push (line 74) | pub fn push(&mut self, builder: SubRenderer<D>) {
function pop (line 80) | pub fn pop(&mut self) -> SubRenderer<D> {
function into_inner (line 88) | pub fn into_inner(mut self) -> (SubRenderer<D>, Vec<String>) {
type TaggedString (line 101) | pub struct TaggedString<T> {
function width (line 115) | pub fn width(&self) -> usize {
function map_tag (line 123) | pub fn map_tag<U>(self, f: &impl Fn(T) -> U) -> TaggedString<U> {
type TaggedLineElement (line 134) | pub enum TaggedLineElement<T> {
function has_content (line 145) | fn has_content(&self) -> bool {
function map_tag (line 154) | pub fn map_tag<U>(self, f: &impl Fn(T) -> U) -> TaggedLineElement<U> {
function width (line 162) | pub fn width(&self) -> usize {
function as_str (line 170) | pub fn as_str(&self) -> &str {
type TaggedLine (line 180) | pub struct TaggedLine<T> {
method default (line 186) | fn default() -> Self {
function new (line 193) | pub fn new() -> TaggedLine<T> {
function from_string (line 201) | pub fn from_string(s: String, tag: &T) -> TaggedLine<T> {
function to_string (line 214) | fn to_string(&self) -> String {
function is_empty (line 225) | fn is_empty(&self) -> bool {
function push_str (line 235) | fn push_str(&mut self, ts: TaggedString<T>) {
function push (line 251) | fn push(&mut self, tle: TaggedLineElement<T>) {
function push_ws (line 262) | fn push_ws(&mut self, len: usize, tag: &T) {
function insert_front (line 271) | fn insert_front(&mut self, ts: TaggedString<T>) {
function push_char (line 287) | fn push_char(&mut self, c: char, tag: &T) {
function consume (line 306) | fn consume(&mut self, tl: &mut TaggedLine<T>) {
function map_tag (line 314) | pub fn map_tag<U>(self, f: &impl Fn(T) -> U) -> TaggedLine<U> {
function remove_items (line 322) | fn remove_items(&mut self) -> impl Iterator<Item = TaggedLineElement<T>>...
function chars (line 328) | pub fn chars(&self) -> impl Iterator<Item = char> + '_ {
function iter (line 341) | pub fn iter(&self) -> impl Iterator<Item = &TaggedLineElement<T>> + '_ {
function tagged_strings (line 346) | pub fn tagged_strings(&self) -> impl Iterator<Item = &TaggedString<T>> {
function into_tagged_strings (line 355) | fn into_tagged_strings(self) -> impl Iterator<Item = TaggedString<T>> {
function width (line 363) | fn width(&self) -> usize {
function pad_to (line 371) | fn pad_to(&mut self, width: usize, tag: &T) {
function remove_leading_whitespace (line 379) | fn remove_leading_whitespace(&mut self) {
function remove_trailing_spaces (line 415) | fn remove_trailing_spaces(&mut self) {
type Item (line 436) | type Item = TaggedLineElement<T>;
type IntoIter (line 437) | type IntoIter = <Vec<TaggedLineElement<T>> as IntoIterator>::IntoIter;
method into_iter (line 439) | fn into_iter(self) -> Self::IntoIter {
type WithWhiteSpace (line 447) | struct WithWhiteSpace<T>(T, WhiteSpace);
type WrappedBlock (line 452) | struct WrappedBlock<T> {
function new (line 467) | pub fn new(
function flush_word (line 488) | fn flush_word(&mut self) -> Result<()> {
function flush_word_hard_wrap (line 587) | fn flush_word_hard_wrap(&mut self) -> Result<()> {
function flush_line (line 644) | fn flush_line(&mut self) {
function force_flush_line (line 650) | fn force_flush_line(&mut self) {
function flush (line 663) | fn flush(&mut self) -> Result<()> {
function take_trailing_fragments (line 687) | pub fn take_trailing_fragments(&mut self) -> Vec<TaggedLineElement<T>> {
function into_lines (line 700) | pub fn into_lines(mut self) -> Result<Vec<TaggedLine<T>>> {
function add_text (line 706) | fn add_text(
function add_element (line 814) | fn add_element(&mut self, elt: TaggedLineElement<T>) {
function text_len (line 819) | fn text_len(&self) -> usize {
function is_empty (line 823) | fn is_empty(&self) -> bool {
type TextDecorator (line 842) | pub trait TextDecorator {
method decorate_link_start (line 848) | fn decorate_link_start(&mut self, url: &str) -> (String, Self::Annotat...
method decorate_link_end (line 851) | fn decorate_link_end(&mut self) -> String;
method decorate_em_start (line 854) | fn decorate_em_start(&self) -> (String, Self::Annotation);
method decorate_em_end (line 857) | fn decorate_em_end(&self) -> String;
method decorate_strong_start (line 860) | fn decorate_strong_start(&self) -> (String, Self::Annotation);
method decorate_strong_end (line 863) | fn decorate_strong_end(&self) -> String;
method decorate_strikeout_start (line 866) | fn decorate_strikeout_start(&self) -> (String, Self::Annotation);
method decorate_strikeout_end (line 869) | fn decorate_strikeout_end(&self) -> String;
method decorate_code_start (line 872) | fn decorate_code_start(&self) -> (String, Self::Annotation);
method decorate_code_end (line 875) | fn decorate_code_end(&self) -> String;
method decorate_preformat_first (line 878) | fn decorate_preformat_first(&self) -> Self::Annotation;
method decorate_preformat_cont (line 882) | fn decorate_preformat_cont(&self) -> Self::Annotation;
method decorate_image (line 885) | fn decorate_image(&mut self, src: &str, title: &str) -> (String, Self:...
method header_prefix (line 888) | fn header_prefix(&self, level: usize) -> String;
method quote_prefix (line 891) | fn quote_prefix(&self) -> String;
method unordered_item_prefix (line 894) | fn unordered_item_prefix(&self) -> String;
method ordered_item_prefix (line 897) | fn ordered_item_prefix(&self, i: i64) -> String;
method make_subblock_decorator (line 901) | fn make_subblock_decorator(&self) -> Self;
method push_colour (line 904) | fn push_colour(&mut self, _: Colour) -> Option<Self::Annotation> {
method pop_colour (line 909) | fn pop_colour(&mut self) -> bool {
method push_bgcolour (line 914) | fn push_bgcolour(&mut self, _: Colour) -> Option<Self::Annotation> {
method pop_bgcolour (line 919) | fn pop_bgcolour(&mut self) -> bool {
method decorate_superscript_start (line 924) | fn decorate_superscript_start(&self) -> (String, Self::Annotation) {
method decorate_superscript_end (line 929) | fn decorate_superscript_end(&self) -> String {
method finalise (line 936) | fn finalise(&mut self, urls: Vec<String>) -> Vec<TaggedLine<Self::Anno...
type Annotation (line 2304) | type Annotation = ();
method decorate_link_start (line 2306) | fn decorate_link_start(&mut self, _url: &str) -> (String, Self::Annota...
method decorate_link_end (line 2310) | fn decorate_link_end(&mut self) -> String {
method decorate_em_start (line 2314) | fn decorate_em_start(&self) -> (String, Self::Annotation) {
method decorate_em_end (line 2318) | fn decorate_em_end(&self) -> String {
method decorate_strong_start (line 2322) | fn decorate_strong_start(&self) -> (String, Self::Annotation) {
method decorate_strong_end (line 2326) | fn decorate_strong_end(&self) -> String {
method decorate_strikeout_start (line 2330) | fn decorate_strikeout_start(&self) -> (String, Self::Annotation) {
method decorate_strikeout_end (line 2334) | fn decorate_strikeout_end(&self) -> String {
method decorate_code_start (line 2338) | fn decorate_code_start(&self) -> (String, Self::Annotation) {
method decorate_code_end (line 2342) | fn decorate_code_end(&self) -> String {
method decorate_preformat_first (line 2346) | fn decorate_preformat_first(&self) -> Self::Annotation {}
method decorate_preformat_cont (line 2347) | fn decorate_preformat_cont(&self) -> Self::Annotation {}
method decorate_image (line 2349) | fn decorate_image(&mut self, _src: &str, title: &str) -> (String, Self...
method header_prefix (line 2353) | fn header_prefix(&self, level: usize) -> String {
method quote_prefix (line 2357) | fn quote_prefix(&self) -> String {
method unordered_item_prefix (line 2361) | fn unordered_item_prefix(&self) -> String {
method ordered_item_prefix (line 2365) | fn ordered_item_prefix(&self, i: i64) -> String {
method make_subblock_decorator (line 2369) | fn make_subblock_decorator(&self) -> Self {
type Annotation (line 2388) | type Annotation = ();
method decorate_link_start (line 2390) | fn decorate_link_start(&mut self, _url: &str) -> (String, Self::Annota...
method decorate_link_end (line 2394) | fn decorate_link_end(&mut self) -> String {
method decorate_em_start (line 2398) | fn decorate_em_start(&self) -> (String, Self::Annotation) {
method decorate_em_end (line 2402) | fn decorate_em_end(&self) -> String {
method decorate_strong_start (line 2406) | fn decorate_strong_start(&self) -> (String, Self::Annotation) {
method decorate_strong_end (line 2410) | fn decorate_strong_end(&self) -> String {
method decorate_strikeout_start (line 2414) | fn decorate_strikeout_start(&self) -> (String, Self::Annotation) {
method decorate_strikeout_end (line 2418) | fn decorate_strikeout_end(&self) -> String {
method decorate_code_start (line 2422) | fn decorate_code_start(&self) -> (String, Self::Annotation) {
method decorate_code_end (line 2426) | fn decorate_code_end(&self) -> String {
method decorate_preformat_first (line 2430) | fn decorate_preformat_first(&self) -> Self::Annotation {}
method decorate_preformat_cont (line 2431) | fn decorate_preformat_cont(&self) -> Self::Annotation {}
method decorate_image (line 2433) | fn decorate_image(&mut self, _src: &str, title: &str) -> (String, Self...
method header_prefix (line 2438) | fn header_prefix(&self, _level: usize) -> String {
method quote_prefix (line 2442) | fn quote_prefix(&self) -> String {
method unordered_item_prefix (line 2446) | fn unordered_item_prefix(&self) -> String {
method ordered_item_prefix (line 2450) | fn ordered_item_prefix(&self, _i: i64) -> String {
method make_subblock_decorator (line 2454) | fn make_subblock_decorator(&self) -> Self {
type Annotation (line 2501) | type Annotation = RichAnnotation;
method decorate_link_start (line 2503) | fn decorate_link_start(&mut self, url: &str) -> (String, Self::Annotat...
method decorate_link_end (line 2507) | fn decorate_link_end(&mut self) -> String {
method decorate_em_start (line 2511) | fn decorate_em_start(&self) -> (String, Self::Annotation) {
method decorate_em_end (line 2515) | fn decorate_em_end(&self) -> String {
method decorate_strong_start (line 2519) | fn decorate_strong_start(&self) -> (String, Self::Annotation) {
method decorate_strong_end (line 2523) | fn decorate_strong_end(&self) -> String {
method decorate_strikeout_start (line 2527) | fn decorate_strikeout_start(&self) -> (String, Self::Annotation) {
method decorate_strikeout_end (line 2531) | fn decorate_strikeout_end(&self) -> String {
method decorate_code_start (line 2535) | fn decorate_code_start(&self) -> (String, Self::Annotation) {
method decorate_code_end (line 2539) | fn decorate_code_end(&self) -> String {
method decorate_preformat_first (line 2543) | fn decorate_preformat_first(&self) -> Self::Annotation {
method decorate_preformat_cont (line 2547) | fn decorate_preformat_cont(&self) -> Self::Annotation {
method decorate_image (line 2551) | fn decorate_image(&mut self, src: &str, title: &str) -> (String, Self:...
method header_prefix (line 2555) | fn header_prefix(&self, level: usize) -> String {
method quote_prefix (line 2559) | fn quote_prefix(&self) -> String {
method unordered_item_prefix (line 2563) | fn unordered_item_prefix(&self) -> String {
method ordered_item_prefix (line 2567) | fn ordered_item_prefix(&self, i: i64) -> String {
method make_subblock_decorator (line 2571) | fn make_subblock_decorator(&self) -> Self {
method push_colour (line 2575) | fn push_colour(&mut self, colour: Colour) -> Option<Self::Annotation> {
method pop_colour (line 2579) | fn pop_colour(&mut self) -> bool {
method push_bgcolour (line 2583) | fn push_bgcolour(&mut self, colour: Colour) -> Option<Self::Annotation> {
method pop_bgcolour (line 2587) | fn pop_bgcolour(&mut self) -> bool {
type BorderSegHoriz (line 948) | enum BorderSegHoriz {
method chop_left (line 978) | pub fn chop_left(&mut self) {
method chop_right (line 1008) | pub fn chop_right(&mut self) {
type BorderHoriz (line 1043) | pub(crate) struct BorderHoriz<T> {
function new (line 1055) | pub fn new(width: usize, tag: T) -> Self {
function new_type (line 1064) | fn new_type(width: usize, linetype: BorderSegHoriz, tag: T) -> Self {
function stretch_to (line 1073) | fn stretch_to(&mut self, width: usize) {
function join_above (line 1081) | fn join_above(&mut self, x: usize) {
function join_below (line 1100) | fn join_below(&mut self, x: usize) {
function merge_from_below (line 1119) | fn merge_from_below(&mut self, other: &BorderHoriz<T>, pos: usize) {
function merge_from_above (line 1132) | fn merge_from_above(&mut self, other: &BorderHoriz<T>, pos: usize) {
function to_vertical_lines_above (line 1146) | fn to_vertical_lines_above(&self) -> String {
function add_text_span (line 1159) | fn add_text_span(&mut self, pos: usize, t: TaggedLineElement<T>)
function seg_to_char (line 1176) | fn seg_to_char(seg: BorderSegHoriz) -> char {
function to_string (line 1195) | fn to_string(&self) -> String {
function extend_to (line 1215) | fn extend_to(&mut self, len: usize) {
function into_tagged_line (line 1223) | fn into_tagged_line(self) -> TaggedLine<T> {
type RenderLine (line 1260) | pub(crate) enum RenderLine<T> {
function to_string (line 1270) | fn to_string(&self) -> String {
function into_tagged_line (line 1279) | pub fn into_tagged_line(self) -> TaggedLine<T> {
function has_content (line 1288) | fn has_content(&self) -> bool {
type SubRenderer (line 1298) | pub(crate) struct SubRenderer<D: TextDecorator> {
function fmt (line 1325) | fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
function fmt (line 1338) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
type RenderOptions (line 1367) | pub(crate) struct RenderOptions {
method default (line 1403) | fn default() -> Self {
function get_wrapping_or_insert (line 1418) | fn get_wrapping_or_insert<'w, D: TextDecorator>(
function finalise (line 1440) | pub fn finalise(&mut self, links: Vec<String>) -> Vec<TaggedLine<D::Anno...
function new (line 1449) | pub fn new(width: usize, options: RenderOptions, decorator: D) -> SubRen...
function add_line (line 1469) | fn add_line(&mut self, mut line: RenderLine<Vec<D::Annotation>>) {
function extend_lines (line 1501) | fn extend_lines(&mut self, lines: impl IntoIterator<Item = RenderLine<Ve...
function flush_wrapping (line 1508) | fn flush_wrapping(&mut self) -> Result<()> {
function flush_all (line 1520) | fn flush_all(&mut self) -> Result<()> {
function into_string (line 1526) | pub fn into_string(mut self) -> Result<String> {
function to_string (line 1540) | fn to_string(&self) -> String {
function fmt_links (line 1550) | pub fn fmt_links(&mut self, mut links: Vec<TaggedLine<D::Annotation>>) {
function into_lines (line 1597) | pub fn into_lines(mut self) -> Result<LinkedList<RenderLine<Vec<D::Annot...
function add_horizontal_line (line 1602) | fn add_horizontal_line(&mut self, line: BorderHoriz<Vec<D::Annotation>>)...
function width_minus (line 1608) | pub fn width_minus(&self, prefix_len: usize, min_width: usize) -> Result...
function ws_mode (line 1616) | fn ws_mode(&self) -> WhiteSpace {
function filter_text_strikeout (line 1621) | fn filter_text_strikeout(s: &str) -> Option<String> {
type LineSet (line 1636) | struct LineSet<A> {
function fmt (line 1648) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
function cell_height (line 1659) | fn cell_height(&self) -> usize {
method add_empty_line (line 1675) | fn add_empty_line(&mut self) -> Result<()> {
method new_sub_renderer (line 1685) | fn new_sub_renderer(&self, width: usize) -> Result<Self> {
method start_block (line 1696) | fn start_block(&mut self) -> Result<()> {
method start_table (line 1707) | fn start_table(&mut self) -> Result<()> {
method new_line (line 1712) | fn new_line(&mut self) -> Result<()> {
method new_line_hard (line 1716) | fn new_line_hard(&mut self) -> Result<()> {
method add_horizontal_border (line 1729) | fn add_horizontal_border(&mut self) -> Result<()> {
method add_horizontal_border_width (line 1738) | fn add_horizontal_border_width(&mut self, width: usize) -> Result<()> {
method push_ws (line 1747) | fn push_ws(&mut self, ws: WhiteSpace) {
method pop_ws (line 1751) | fn pop_ws(&mut self) {
method push_preformat (line 1755) | fn push_preformat(&mut self) {
method pop_preformat (line 1759) | fn pop_preformat(&mut self) {
method end_block (line 1764) | fn end_block(&mut self) {
method add_inline_text (line 1768) | fn add_inline_text(&mut self, text: &str) -> Result<()> {
method width (line 1816) | fn width(&self) -> usize {
method append_subrender (line 1820) | fn append_subrender<'a, I>(&mut self, other: Self, prefixes: I) -> Resul...
method append_columns_with_borders (line 1862) | fn append_columns_with_borders<I>(&mut self, cols: I, collapse: bool) ->...
method append_vert_row (line 2096) | fn append_vert_row<I>(&mut self, cols: I) -> Result<()>
method empty (line 2125) | fn empty(&self) -> bool {
method start_link (line 2134) | fn start_link(&mut self, target: &str) -> Result<()> {
method end_link (line 2139) | fn end_link(&mut self) -> Result<()> {
method start_emphasis (line 2145) | fn start_emphasis(&mut self) -> Result<()> {
method end_emphasis (line 2150) | fn end_emphasis(&mut self) -> Result<()> {
method start_strong (line 2156) | fn start_strong(&mut self) -> Result<()> {
method end_strong (line 2161) | fn end_strong(&mut self) -> Result<()> {
method start_strikeout (line 2167) | fn start_strikeout(&mut self) -> Result<()> {
method end_strikeout (line 2176) | fn end_strikeout(&mut self) -> Result<()> {
method start_code (line 2187) | fn start_code(&mut self) -> Result<()> {
method end_code (line 2193) | fn end_code(&mut self) -> Result<()> {
method add_image (line 2199) | fn add_image(&mut self, src: &str, title: &str) -> Result<()> {
method header_prefix (line 2222) | fn header_prefix(&mut self, level: usize) -> String {
method quote_prefix (line 2226) | fn quote_prefix(&mut self) -> String {
method unordered_item_prefix (line 2230) | fn unordered_item_prefix(&mut self) -> String {
method ordered_item_prefix (line 2234) | fn ordered_item_prefix(&mut self, i: i64) -> String {
method record_frag_start (line 2238) | fn record_frag_start(&mut self, fragname: &str) {
method push_colour (line 2251) | fn push_colour(&mut self, colour: Colour) {
method pop_colour (line 2258) | fn pop_colour(&mut self) {
method push_bgcolour (line 2264) | fn push_bgcolour(&mut self, colour: Colour) {
method pop_bgcolour (line 2270) | fn pop_bgcolour(&mut self) {
method start_superscript (line 2276) | fn start_superscript(&mut self) -> Result<()> {
method end_superscript (line 2282) | fn end_superscript(&mut self) -> Result<()> {
type PlainDecorator (line 2293) | pub struct PlainDecorator {}
method new (line 2298) | pub fn new() -> PlainDecorator {
type TrivialDecorator (line 2377) | pub struct TrivialDecorator {}
method new (line 2382) | pub fn new() -> TrivialDecorator {
type RichDecorator (line 2462) | pub struct RichDecorator {}
method new (line 2495) | pub fn new() -> RichDecorator {
type RichAnnotation (line 2468) | pub enum RichAnnotation {
FILE: src/tests.rs
function test_html (line 21) | fn test_html(input: &[u8], expected: &str, width: usize) {
function test_html_conf (line 26) | fn test_html_conf<F>(input: &[u8], expected: &str, width: usize, conf: F)
function test_html_conf_dec (line 36) | fn test_html_conf_dec<D: TextDecorator, F>(
function test_html_maxwrap (line 51) | fn test_html_maxwrap(input: &[u8], expected: &str, width: usize, wrap_wi...
function test_html_css (line 57) | fn test_html_css(input: &[u8], expected: &str, width: usize) {
function test_colour_map (line 65) | fn test_colour_map(annotations: &[RichAnnotation], s: &str) -> String {
function test_html_coloured_conf (line 130) | fn test_html_coloured_conf<F>(input: &[u8], expected: &str, width: usize...
function test_html_coloured (line 141) | fn test_html_coloured(input: &[u8], expected: &str, width: usize) {
function test_html_err_conf (line 145) | fn test_html_err_conf<F>(input: &[u8], expected: Error, width: usize, co...
function test_html_err (line 159) | fn test_html_err(input: &[u8], expected: Error, width: usize) {
function test_html_style (line 165) | fn test_html_style(input: &[u8], style: &str, expected: &str, width: usi...
function test_html_decorator (line 175) | fn test_html_decorator<D>(input: &[u8], expected: &str, width: usize, de...
function test_html_conf_rendertree (line 185) | fn test_html_conf_rendertree<F>(input: &[u8], conf: F, expected: crate::...
function test_xml (line 199) | fn test_xml(input: &[u8], expected: &str, width: usize) {
function test_table (line 209) | fn test_table() {
function test_table2 (line 229) | fn test_table2() {
function test_thead (line 256) | fn test_thead() {
function test_colspan (line 287) | fn test_colspan() {
function test_colspan_zero (line 319) | fn test_colspan_zero() {
function test_colspan_large (line 351) | fn test_colspan_large() {
function test_colspan_larger (line 383) | fn test_colspan_larger() {
function test_para (line 415) | fn test_para() {
function test_para2 (line 420) | fn test_para2() {
function test_blockquote (line 425) | fn test_blockquote() {
function test_ul (line 443) | fn test_ul() {
function test_ol1 (line 462) | fn test_ol1() {
function test_ol2 (line 481) | fn test_ol2() {
function test_ol_start (line 513) | fn test_ol_start() {
function test_ol_start_9 (line 529) | fn test_ol_start_9() {
function test_ol_start_neg (line 545) | fn test_ol_start_neg() {
function test_strip_nl (line 563) | fn test_strip_nl() {
function test_strip_nl2 (line 577) | fn test_strip_nl2() {
function test_strip_nl_tbl (line 593) | fn test_strip_nl_tbl() {
function test_unknown_element (line 616) | fn test_unknown_element() {
function test_strip_nl_tbl_p (line 641) | fn test_strip_nl_tbl_p() {
function test_pre (line 664) | fn test_pre() {
function test_link (line 684) | fn test_link() {
function test_link2 (line 696) | fn test_link2() {
function test_link3 (line 709) | fn test_link3() {
function test_link_wrap (line 722) | fn test_link_wrap() {
function test_links_footnotes (line 737) | fn test_links_footnotes() {
function test_links_footnotes_trivial (line 761) | fn test_links_footnotes_trivial() {
function test_links_footnotes_rich (line 788) | fn test_links_footnotes_rich() {
function test_wrap (line 815) | fn test_wrap() {
function test_wrap2 (line 829) | fn test_wrap2() {
function test_wrap3 (line 844) | fn test_wrap3() {
function test_wrap4 (line 857) | fn test_wrap4() {
function test_wrap_max (line 872) | fn test_wrap_max() {
function test_wrap_max2 (line 896) | fn test_wrap_max2() {
function test_wrap_nbsp_unicode (line 926) | fn test_wrap_nbsp_unicode() {
function test_wrap_nbsp_ent (line 938) | fn test_wrap_nbsp_ent() {
function test_dowrap_unicode (line 950) | fn test_dowrap_unicode() {
function test_dowrap_wbr (line 962) | fn test_dowrap_wbr() {
function test_nested_ul (line 974) | fn test_nested_ul() {
function test_nested_ol (line 1000) | fn test_nested_ol() {
function test_wrap_word_boundaries (line 1026) | fn test_wrap_word_boundaries() {
function test_div (line 1064) | fn test_div() {
function test_img_alt (line 1085) | fn test_img_alt() {
function test_img_noalt (line 1094) | fn test_img_noalt() {
function test_img_nosrc (line 1128) | fn test_img_nosrc() {
function test_svg (line 1133) | fn test_svg() {
function test_noscript (line 1148) | fn test_noscript() {
function test_br (line 1158) | fn test_br() {
function test_br2 (line 1163) | fn test_br2() {
function test_br3 (line 1168) | fn test_br3() {
function test_subblock (line 1173) | fn test_subblock() {
function test_controlchar (line 1195) | fn test_controlchar() {
function test_nested_table_1 (line 1202) | fn test_nested_table_1() {
function test_nested_table_2 (line 1254) | fn test_nested_table_2() {
function test_h1 (line 1288) | fn test_h1() {
function test_h3 (line 1303) | fn test_h3() {
function test_pre2 (line 1319) | fn test_pre2() {
function test_multi_pre (line 1331) | fn test_multi_pre() {
function test_pre_span (line 1376) | fn test_pre_span() {
function test_pre_tab (line 1393) | fn test_pre_tab() {
function test_pre_tab2 (line 1406) | fn test_pre_tab2() {
function test_pre_tab3 (line 1436) | fn test_pre_tab3() {
function test_em_strong (line 1566) | fn test_em_strong() {
function test_b_tag (line 1578) | fn test_b_tag() {
function test_nbsp_indent (line 1590) | fn test_nbsp_indent() {
function test_deeply_nested (line 1608) | fn test_deeply_nested() {
function test_deeply_nested_table (line 1617) | fn test_deeply_nested_table() {
function test_table_no_id (line 1646) | fn test_table_no_id() {
function test_table_cell_id (line 1663) | fn test_table_cell_id() {
function test_table_row_id (line 1680) | fn test_table_row_id() {
function test_table_table_id (line 1697) | fn test_table_table_id() {
function test_table_tbody_id (line 1714) | fn test_table_tbody_id() {
function test_header_width (line 1733) | fn test_header_width() {
function test_trivial_decorator (line 1761) | fn test_trivial_decorator() {
function test_issue_16 (line 1782) | fn test_issue_16() {
function test_pre_br (line 1787) | fn test_pre_br() {
function test_pre_emptyline (line 1798) | fn test_pre_emptyline() {
function test_link_id_longline (line 1803) | fn test_link_id_longline() {
function test_dl (line 1816) | fn test_dl() {
function test_s (line 1827) | fn test_s() {
function test_multi_parse (line 1836) | fn test_multi_parse() {
function test_read_rich (line 1859) | fn test_read_rich() {
function test_read_rich_nodecorate (line 1870) | fn test_read_rich_nodecorate() {
function test_read_custom (line 1881) | fn test_read_custom() {
function test_pre_rich (line 1892) | fn test_pre_rich() {
function test_finalise (line 1931) | fn test_finalise() {
function test_empty_rows (line 2029) | fn test_empty_rows() {
function test_empty_cols (line 2057) | fn test_empty_cols() {
function test_empty_table (line 2097) | fn test_empty_table() {
function test_table_empty_single_row (line 2108) | fn test_table_empty_single_row() {
function test_table_empty_single_row_empty_cell (line 2119) | fn test_table_empty_single_row_empty_cell() {
function test_table_empty_single_row_ws_cell (line 2130) | fn test_table_empty_single_row_ws_cell() {
function test_renderer_zero_width (line 2149) | fn test_renderer_zero_width() {
function test_ul_tiny_table (line 2159) | fn test_ul_tiny_table() {
function test_issue_54_oob (line 2172) | fn test_issue_54_oob() {
function test_table_vertical_rows (line 2210) | fn test_table_vertical_rows() {
constant MULTILINE_CELLS (line 2233) | const MULTILINE_CELLS: &[u8] = b"<table><tr>
function test_table_without_borders (line 2250) | fn test_table_without_borders() {
function test_table_raw_mode (line 2265) | fn test_table_raw_mode() {
function test_unicode (line 2281) | fn test_unicode() {
function test_list_in_table (line 2298) | fn test_list_in_table() {
function test_max_width (line 2334) | fn test_max_width() {
function test_preserving_empty_newlines_in_pre_blocks (line 2342) | fn test_preserving_empty_newlines_in_pre_blocks() {
function test_links_outside_table (line 2355) | fn test_links_outside_table() {
function test_narrow_width_nested (line 2379) | fn test_narrow_width_nested() {
function test_issue_93_x (line 2399) | fn test_issue_93_x() {
function test_superscript (line 2442) | fn test_superscript() {
function test_header_overflow (line 2448) | fn test_header_overflow() {
function test_blockquote_overflow (line 2465) | fn test_blockquote_overflow() {
function test_ul_overflow (line 2482) | fn test_ul_overflow() {
function test_ol_overflow (line 2499) | fn test_ol_overflow() {
function test_dd_overflow (line 2516) | fn test_dd_overflow() {
function test_overflow_wide_char (line 2533) | fn test_overflow_wide_char() {
function test_table_too_narrow (line 2543) | fn test_table_too_narrow() {
function test_empty_table_in_list (line 2558) | fn test_empty_table_in_list() {
function test_silly_colspan (line 2572) | fn test_silly_colspan() {
function test_rowspan1 (line 2590) | fn test_rowspan1() {
function test_rowspan2_emptyrow (line 2615) | fn test_rowspan2_emptyrow() {
function test_rowspan3_shortrow (line 2642) | fn test_rowspan3_shortrow() {
function test_rowspan_underflow (line 2672) | fn test_rowspan_underflow() {
function test_issue_187 (line 2690) | fn test_issue_187() {
function get_lines (line 2695) | fn get_lines(html: &[u8], width: usize) -> Vec<TaggedLine<Vec<()>>> {
function frag_simple (line 2700) | fn frag_simple() {
function frag_list (line 2718) | fn frag_list() {
function test_serialise_full (line 2748) | fn test_serialise_full() {
function test_disp_none (line 2763) | fn test_disp_none() {
function test_selector_elementname (line 2795) | fn test_selector_elementname() {
function test_selector_aoc (line 2813) | fn test_selector_aoc() {
function test_coloured_a (line 2846) | fn test_coloured_a() {
function test_bgcoloured (line 2863) | fn test_bgcoloured() {
function test_bgcoloured2 (line 2881) | fn test_bgcoloured2() {
function test_bgcoloured3 (line 2899) | fn test_bgcoloured3() {
function test_coloured_element (line 2916) | fn test_coloured_element() {
function test_color_attr (line 2933) | fn test_color_attr() {
function test_css_lists (line 2945) | fn test_css_lists() {
function test_coloured_multi (line 2983) | fn test_coloured_multi() {
function test_coloured_important (line 3019) | fn test_coloured_important() {
function test_wrap_word_boundaries (line 3055) | fn test_wrap_word_boundaries() {
function test_max_height_0 (line 3078) | fn test_max_height_0() {
function test_height_0 (line 3090) | fn test_height_0() {
function test_selector_hash (line 3102) | fn test_selector_hash() {
function test_selector_child_desc (line 3137) | fn test_selector_child_desc() {
function test_colour_row (line 3160) | fn test_colour_row() {
function test_css_levels (line 3192) | fn test_css_levels() {
function test_pre_wrap (line 3258) | fn test_pre_wrap() {
function test_nth_child (line 3321) | fn test_nth_child() {
function test_before_after (line 3445) | fn test_before_after() {
function test_wrap_nbsp_style (line 3466) | fn test_wrap_nbsp_style() {
function test_issue_252 (line 3478) | fn test_issue_252() {
function test_xml1 (line 3490) | fn test_xml1() {
function t (line 3545) | fn t(s: &str) -> RenderNode {
function t_s (line 3549) | fn t_s(s: &str, st: ComputedStyle) -> RenderNode {
function em (line 3553) | fn em(ns: &[RenderNode]) -> RenderNode {
function c (line 3557) | fn c(ns: &[RenderNode]) -> RenderNode {
function b_s (line 3571) | fn b_s(ns: &[RenderNode], st: ComputedStyle) -> RenderNode {
function d (line 3582) | fn d<T: Default>() -> T {
function st (line 3586) | fn st() -> ComputedStyle {
type TestAddStyle (line 3590) | trait TestAddStyle {
method fg (line 3591) | fn fg(self, r: u8, g: u8, b: u8) -> Self;
method pre (line 3592) | fn pre(self) -> Self;
method fg (line 3595) | fn fg(mut self, r: u8, g: u8, b: u8) -> Self {
method pre (line 3604) | fn pre(mut self) -> Self {
function render_1 (line 3617) | fn render_1() {
function syntax_all_blue (line 3625) | fn syntax_all_blue(text: &str) -> Vec<(TextStyle, &str)> {
function syntax_pre_em (line 3629) | fn syntax_pre_em() {
Condensed preview — 37 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (501K chars).
[
{
"path": ".circleci/config.yml",
"chars": 1927,
"preview": "version: 2.1\n\norbs:\n win: circleci/windows@2.2.0\n\njobs:\n build-stable:\n docker:\n - image: cimg/rust:1.85.1\n "
},
{
"path": ".github/dependabot.yml",
"chars": 149,
"preview": "version: 2\nupdates:\n- package-ecosystem: \"cargo\"\n directory: \"/\"\n schedule:\n interval: \"weekly\"\n day: \"friday\"\n "
},
{
"path": ".github/workflows/ci.yml",
"chars": 562,
"preview": "name: CI\n\non:\n pull_request:\n push:\n branches:\n - main\n\nenv:\n CARGO_TERM_COLOR: always\n\njobs:\n test-action:\n"
},
{
"path": ".github/workflows/jekyll-gh-pages.yml",
"chars": 1702,
"preview": "# Sample workflow for building and deploying a Jekyll site to GitHub Pages\nname: Build and deploy demo site\n\non:\n # All"
},
{
"path": ".gitignore",
"chars": 7,
"preview": "target\n"
},
{
"path": "CHANGELOG.md",
"chars": 14785,
"preview": "# Changelog\n\nPossible log types:\n\n- `[added]` for new features.\n- `[changed]` for changes in existing functionality.\n- `"
},
{
"path": "Cargo.toml",
"chars": 1234,
"preview": "[package]\nname = \"html2text\"\ndescription = \"Render HTML as plain text.\"\nreadme = \"README.md\"\ndocumentation = \"https://do"
},
{
"path": "LICENSE",
"chars": 1071,
"preview": "MIT License\n\nCopyright (c) 2016 Chris Emerson\n\nPermission is hereby granted, free of charge, to any person obtaining a c"
},
{
"path": "README.md",
"chars": 5551,
"preview": "[](https://app.circleci."
},
{
"path": "benches/tables.rs",
"chars": 2735,
"preview": "#![feature(test)]\nextern crate html2text;\nextern crate test;\n\nuse ::test::Bencher;\n\nuse html2text::from_read;\n\nfn make_h"
},
{
"path": "examples/html2term.rs",
"chars": 15353,
"preview": "#[cfg(unix)]\nextern crate argparse;\n#[cfg(unix)]\nextern crate unicode_width;\n#[cfg(unix)]\nmod top {\n #[cfg(feature = "
},
{
"path": "html2text-cli/Cargo.toml",
"chars": 726,
"preview": "[package]\nname = \"html2text-cli\"\ndescription = \"Render HTML as plain text.\"\nreadme = \"README.md\"\ncategories = [\"text-pro"
},
{
"path": "html2text-cli/README.md",
"chars": 665,
"preview": "# html2text\n\nA command-line tool to convert HTML into text in various forms. Supports some\nCSS including colours with d"
},
{
"path": "html2text-cli/src/main.rs",
"chars": 12727,
"preview": "extern crate argparse;\nextern crate html2text;\nuse argparse::{ArgumentParser, Store, StoreOption, StoreTrue};\nuse html2t"
},
{
"path": "html2text-web-demo/.cargo/config.toml",
"chars": 33,
"preview": "[build]\ntarget = \"wasm32-wasip2\"\n"
},
{
"path": "html2text-web-demo/.gitignore",
"chars": 5,
"preview": "dist\n"
},
{
"path": "html2text-web-demo/Cargo.toml",
"chars": 220,
"preview": "[package]\nname = \"html2text-web-demo\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n[dependencies]\nhtml2text = { path = \"..\", feat"
},
{
"path": "html2text-web-demo/Trunk.toml",
"chars": 187,
"preview": "trunk-version = \"^0.21.13\"\n\n[build]\npublic_url = \"/rust-html2text/\"\nrelease = true\nfilehash = false\ninject_scripts = tru"
},
{
"path": "html2text-web-demo/index.html",
"chars": 1876,
"preview": "<!doctype html>\n<html>\n <head>\n<style>\nbody {\n margin: 0;\n}\npre {\n margin: 0;\n}\n#main, #lib {\n background-co"
},
{
"path": "html2text-web-demo/src/lib.rs",
"chars": 6570,
"preview": "use wasm_bindgen::prelude::wasm_bindgen;\n\nuse ratzilla::ratatui::{\n style::{Color, Style, Stylize},\n text::{Text, "
},
{
"path": "pages/.gitignore",
"chars": 6,
"preview": "_site\n"
},
{
"path": "pages/_config.yml",
"chars": 148,
"preview": "lsi: false\nsafe: true\nsource: .\nincremental: false\nbaseurl: \"/rust-html2text\"\ngist:\n noscript: false\n\ntheme: minima\n\ngi"
},
{
"path": "pages/_includes/head.html",
"chars": 844,
"preview": "<head>\n <meta charset=\"utf-8\">\n <meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge\">\n <meta name=\"viewport\" content="
},
{
"path": "pages/assets/demo-main.js",
"chars": 4128,
"preview": "const img_modes = {\n \"ignore\": \"IgnoreEmpty\",\n \"always\": \"ShowAlways\",\n \"replace\": \"Replace(\\\"XX\\\")\",\n \"file"
},
{
"path": "pages/assets/demo.css",
"chars": 897,
"preview": "#lib {\n background-color: black;\n height: 30em;\n overflow: scroll;\n}\n#input_html {\n height: 300px;\n width"
},
{
"path": "pages/index.markdown",
"chars": 4896,
"preview": "---\n# Feel free to add content and custom Front Matter to this file.\n# To modify the layout, see https://jekyllrb.com/do"
},
{
"path": "rust.yml",
"chars": 316,
"preview": "name: Rust\n\non:\n push:\n branches: [ master ]\n pull_request:\n branches: [ master ]\n\nenv:\n CARGO_TERM_COLOR: alwa"
},
{
"path": "src/ansi_colours.rs",
"chars": 1084,
"preview": "//! Convenience helper for producing coloured terminal output.\n//!\n//! This optional helper applies terminal colours (or"
},
{
"path": "src/css/parser.rs",
"chars": 55991,
"preview": "//! Parsing for the subset of CSS used in html2text.\n\nuse std::{borrow::Cow, ops::Deref, str::FromStr};\n\nuse nom::{\n "
},
{
"path": "src/css/types.rs",
"chars": 104,
"preview": "#[derive(Copy, Clone, PartialEq, Eq, Debug)]\npub(crate) enum Importance {\n Default,\n Important,\n}\n"
},
{
"path": "src/css.rs",
"chars": 29242,
"preview": "//! Some basic CSS support.\nuse std::ops::Deref;\nuse std::rc::Rc;\n\n#[cfg(feature = \"css\")]\nmod parser;\npub(crate) mod ty"
},
{
"path": "src/lib.rs",
"chars": 122523,
"preview": "//! Convert HTML to text formats.\n//!\n//! This crate renders HTML into a text format, wrapped to a specified width.\n//! "
},
{
"path": "src/macros.rs",
"chars": 1782,
"preview": "#[cfg(feature = \"html_trace_bt\")]\nextern crate backtrace;\n\n/* This is to work around a false positive for the clippy war"
},
{
"path": "src/markup5ever_rcdom.rs",
"chars": 19736,
"preview": "// Copyright 2014-2017 The html5ever Project Developers. See the\n// COPYRIGHT file at the top-level directory of this di"
},
{
"path": "src/render/mod.rs",
"chars": 5453,
"preview": "//! Module containing the `Renderer` interface for constructing a\n//! particular text output.\n\nuse crate::Colour;\nuse cr"
},
{
"path": "src/render/text_renderer.rs",
"chars": 84946,
"preview": "//! Implementations of the `Renderer` trait.\n//!\n//! This module implements helpers and concrete types for rendering fro"
},
{
"path": "src/tests.rs",
"chars": 77670,
"preview": "use std::str;\n\nuse crate::config::Config;\nuse crate::render::TaggedLineElement;\nuse crate::render::text_renderer::{Plain"
}
]
About this extraction
This page contains the full source code of the jugglerchris/rust-html2text GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 37 files (466.7 KB), approximately 115.2k tokens, and a symbol index with 837 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.