When `layout=False`: Adds spaces where the difference between the `x1` of one character and the `x0` of the next is greater than `x_tolerance`. (If `x_tolerance_ratio` is not `None`, the extractor uses a dynamic `x_tolerance` equal to `x_tolerance_ratio * previous_character[\"size\"]`.) Adds newline characters where the difference between the `doctop` of one character and the `doctop` of the next is greater than `y_tolerance`.
When `layout=True` (*experimental feature*): Attempts to mimic the structural layout of the text on the page(s), using `x_density` and `y_density` to determine the minimum number of characters/newlines per \"point,\" the PDF unit of measurement. Passing `line_dir_render=\"ttb\"/\"btt\"/\"ltr\"/\"rtl\"` and/or `char_dir_render=\"ttb\"/\"btt\"/\"ltr\"/\"rtl\"` will output the the lines/characters in a different direction than the default. All remaining `**kwargs` are passed to `.extract_words(...)` (see below), the first step in calculating the layout.
split_at_punctuation='!\"&\\'()*+,.:;<=>?@[\\]^\\`\\{\\|\\}~'. Unless you set `expand_ligatures=False`, ligatures such as `fi` will be expanded into their constituent letters (e.g., `fi`). Passing `return_chars=True` will add, to each word dictionary, a list of its constituent characters, as a list in the `\"chars\"` field.|\n|`.extract_text_lines(layout=False, strip=True, return_chars=True, **kwargs)`|*Experimental feature* that returns a list of dictionaries representing the lines of text on the page. The `strip` parameter works analogously to Python's `str.strip()` method, and returns `text` attributes without their surrounding whitespace. (Only relevant when `layout = True`.) Setting `return_chars` to `False` will exclude the individual character objects from the returned text-line dicts. The remaining `**kwargs` are those you would pass to `.extract_text(layout=True, ...)`.|\n|`.search(pattern, regex=True, case=True, main_group=0, return_groups=True, return_chars=True, layout=False, **kwargs)`|*Experimental feature* that allows you to search a page's text, returning a list of all instances that match the query. For each instance, the response dictionary object contains the matching text, any regex group matches, the bounding box coordinates, and the char objects themselves. `pattern` can be a compiled regular expression, an uncompiled regular expression, or a non-regex string. If `regex` is `False`, the pattern is treated as a non-regex string. If `case` is `False`, the search is performed in a case-insensitive manner. Setting `main_group` restricts the results to a specific regex group within the `pattern` (default of `0` means the entire match). Setting `return_groups` and/or `return_chars` to `False` will exclude the lists of the matched regex groups and/or characters from being added (as `\"groups\"` and `\"chars\"` to the return dicts). The `layout` parameter operates as it does for `.extract_text(...)`. The remaining `**kwargs` are those you would pass to `.extract_text(layout=True, ...)`. __Note__: Zero-width and all-whitespace matches are discarded, because they (generally) have no explicit position on the page. |\n|`.dedupe_chars(tolerance=1, extra_attrs=(\"fontname\", \"size\"))`| Returns a version of the page with duplicate chars — those sharing the same text, positioning (within `tolerance` x/y), and `extra_attrs` as other characters — removed. (See [Issue #71](https://github.com/jsvine/pdfplumber/issues/71) to understand the motivation.)|\n\n## Extracting tables\n\n`pdfplumber`'s approach to table detection borrows heavily from [Anssi Nurminen's master's thesis](https://trepo.tuni.fi/bitstream/handle/123456789/21520/Nurminen.pdf?sequence=3), and is inspired by [Tabula](https://github.com/tabulapdf/tabula-extractor/issues/16). It works like this:\n\n1. For any given PDF page, find the lines that are (a) explicitly defined and/or (b) implied by the alignment of words on the page.\n2. Merge overlapping, or nearly-overlapping, lines.\n3. Find the intersections of all those lines.\n4. Find the most granular set of rectangles (i.e., cells) that use these intersections as their vertices.\n5. Group contiguous cells into tables. \n\n### Table-extraction methods\n\n`pdfplumber.Page` objects can call the following table methods:\n\n| Method | Description |\n|--------|-------------|\n|`.find_tables(table_settings={})`|Returns a list of `Table` objects. The `Table` object provides access to the `.cells`, `.rows`, `.columns`, and `.bbox` properties, as well as the `.extract(x_tolerance=3, y_tolerance=3)` method.|\n|`.find_table(table_settings={})`|Similar to `.find_tables(...)`, but returns the *largest* table on the page, as a `Table` object. If multiple tables have the same size — as measured by the number of cells — this method returns the table closest to the top of the page.|\n|`.extract_tables(table_settings={})`|Returns the text extracted from *all* tables found on the page, represented as a list of lists of lists, with the structure `table -> row -> cell`.|\n|`.extract_table(table_settings={})`|Returns the text extracted from the *largest* table on the page (see `.find_table(...)` above), represented as a list of lists, with the structure `row -> cell`.|\n|`.debug_tablefinder(table_settings={})`|Returns an instance of the `TableFinder` class, with access to the `.edges`, `.intersections`, `.cells`, and `.tables` properties.|\n\nFor example:\n\n```python\npdf = pdfplumber.open(\"path/to/my.pdf\")\npage = pdf.pages[0]\npage.extract_table()\n```\n\n[Click here for a more detailed example.](examples/notebooks/extract-table-ca-warn-report.ipynb)\n\n### Table-extraction settings\n\nBy default, `extract_tables` uses the page's vertical and horizontal lines (or rectangle edges) as cell-separators. But the method is highly customizable via the `table_settings` argument. The possible settings, and their defaults:\n\n```python\n{\n \"vertical_strategy\": \"lines\", \n \"horizontal_strategy\": \"lines\",\n \"explicit_vertical_lines\": [],\n \"explicit_horizontal_lines\": [],\n \"snap_tolerance\": 3,\n \"snap_x_tolerance\": 3,\n \"snap_y_tolerance\": 3,\n \"join_tolerance\": 3,\n \"join_x_tolerance\": 3,\n \"join_y_tolerance\": 3,\n \"edge_min_length\": 3,\n \"edge_min_length_prefilter\": 1,\n \"min_words_vertical\": 3,\n \"min_words_horizontal\": 1,\n \"intersection_tolerance\": 3,\n \"intersection_x_tolerance\": 3,\n \"intersection_y_tolerance\": 3,\n \"text_tolerance\": 3,\n \"text_x_tolerance\": 3,\n \"text_y_tolerance\": 3,\n \"text_*\": …, # See below\n}\n```\n\n| Setting | Description |\n|---------|-------------|\n|`\"vertical_strategy\"`| Either `\"lines\"`, `\"lines_strict\"`, `\"text\"`, or `\"explicit\"`. See explanation below.|\n|`\"horizontal_strategy\"`| Either `\"lines\"`, `\"lines_strict\"`, `\"text\"`, or `\"explicit\"`. See explanation below.|\n|`\"explicit_vertical_lines\"`| A list of vertical lines that explicitly demarcate cells in the table. Can be used in combination with any of the strategies above. Items in the list should be either numbers — indicating the `x` coordinate of a line the full height of the page — or `line`/`rect`/`curve` objects.|\n|`\"explicit_horizontal_lines\"`| A list of horizontal lines that explicitly demarcate cells in the table. Can be used in combination with any of the strategies above. Items in the list should be either numbers — indicating the `y` coordinate of a line the full height of the page — or `line`/`rect`/`curve` objects.|\n|`\"snap_tolerance\"`, `\"snap_x_tolerance\"`, `\"snap_y_tolerance\"`| Parallel lines within `snap_tolerance` points will be \"snapped\" to the same horizontal or vertical position.|\n|`\"join_tolerance\"`, `\"join_x_tolerance\"`, `\"join_y_tolerance\"`| Line segments on the same infinite line, and whose ends are within `join_tolerance` of one another, will be \"joined\" into a single line segment.|\n|`\"edge_min_length\"`| Edges shorter than `edge_min_length` will be discarded before attempting to reconstruct the table.|\n|`\"edge_min_length_prefilter\"`| Edges shorter than `edge_min_length_prefilter` will be discarded during initial edge filtering from the page. Lowering this value (e.g., to `0.5`) can help capture small dashed lines that might otherwise be filtered out.|\n|`\"min_words_vertical\"`| When using `\"vertical_strategy\": \"text\"`, at least `min_words_vertical` words must share the same alignment.|\n|`\"min_words_horizontal\"`| When using `\"horizontal_strategy\": \"text\"`, at least `min_words_horizontal` words must share the same alignment.|\n|`\"intersection_tolerance\"`, `\"intersection_x_tolerance\"`, `\"intersection_y_tolerance\"`| When combining edges into cells, orthogonal edges must be within `intersection_tolerance` points to be considered intersecting.|\n|`\"text_*\"`| All settings prefixed with `text_` are then used when extracting text from each discovered table. All possible arguments to `Page.extract_text(...)` are also valid here.|\n|`\"text_x_tolerance\"`, `\"text_y_tolerance\"`| These `text_`-prefixed settings *also* apply to the table-identification algorithm when the `text` strategy is used. I.e., when that algorithm searches for words, it will expect the individual letters in each word to be no more than `text_x_tolerance`/`text_y_tolerance` points apart.|\n\n### Table-extraction strategies\n\nBoth `vertical_strategy` and `horizontal_strategy` accept the following options:\n\n| Strategy | Description | \n|----------|-------------|\n| `\"lines\"` | Use the page's graphical lines — including the sides of rectangle objects — as the borders of potential table-cells. |\n| `\"lines_strict\"` | Use the page's graphical lines — but *not* the sides of rectangle objects — as the borders of potential table-cells. |\n| `\"text\"` | For `vertical_strategy`: Deduce the (imaginary) lines that connect the left, right, or center of words on the page, and use those lines as the borders of potential table-cells. For `horizontal_strategy`, the same but using the tops of words. |\n| `\"explicit\"` | Only use the lines explicitly defined in `explicit_vertical_lines` / `explicit_horizontal_lines`. |\n\n### Notes\n\n- Often it's helpful to crop a page — `Page.crop(bounding_box)` — before trying to extract the table.\n\n- Table extraction for `pdfplumber` was radically redesigned for `v0.5.0`, and introduced breaking changes.\n\n\n## Extracting form values\n\nSometimes PDF files can contain forms that include inputs that people can fill out and save. While values in form fields appear like other text in a PDF file, form data is handled differently. If you want the gory details, see page 671 of this [specification](https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstandards/pdfreference1.7old.pdf).\n\n`pdfplumber` doesn't have an interface for working with form data, but you can access it using `pdfplumber`'s wrappers around `pdfminer`.\n\nFor example, this snippet will retrieve form field names and values and store them in a dictionary.\n\n```python\nimport pdfplumber\nfrom pdfplumber.utils.pdfinternals import resolve_and_decode, resolve\n\npdf = pdfplumber.open(\"document_with_form.pdf\")\n\ndef parse_field_helper(form_data, field, prefix=None):\n \"\"\" appends any PDF AcroForm field/value pairs in `field` to provided `form_data` list\n\n if `field` has child fields, those will be parsed recursively.\n \"\"\"\n resolved_field = field.resolve()\n field_name = '.'.join(filter(lambda x: x, [prefix, resolve_and_decode(resolved_field.get(\"T\"))]))\n if \"Kids\" in resolved_field:\n for kid_field in resolved_field[\"Kids\"]:\n parse_field_helper(form_data, kid_field, prefix=field_name)\n if \"T\" in resolved_field or \"TU\" in resolved_field:\n # \"T\" is a field-name, but it's sometimes absent.\n # \"TU\" is the \"alternate field name\" and is often more human-readable\n # your PDF may have one, the other, or both.\n alternate_field_name = resolve_and_decode(resolved_field.get(\"TU\")) if resolved_field.get(\"TU\") else None\n field_value = resolve_and_decode(resolved_field[\"V\"]) if 'V' in resolved_field else None\n form_data.append([field_name, alternate_field_name, field_value])\n\n\nform_data = []\nfields = resolve(resolve(pdf.doc.catalog[\"AcroForm\"])[\"Fields\"])\nfor field in fields:\n parse_field_helper(form_data, field)\n```\n\nOnce you run this script, `form_data` is a list containing a three-element tuple for each form element. For instance, a PDF form with a city and state field might look like this.\n```\n[\n ['STATE.0', 'enter STATE', 'CA'],\n ['section 2 accident infoRmation.1.0',\n 'enter city of accident',\n 'SAN FRANCISCO']\n]\n```\n\n*Thanks to [@jeremybmerrill](https://github.com/jeremybmerrill) for helping to maintain the form-parsing code above.*\n\n## Demonstrations\n\n- [Using `extract_table` on a California Worker Adjustment and Retraining Notification (WARN) report](examples/notebooks/extract-table-ca-warn-report.ipynb). Demonstrates basic visual debugging and table extraction.\n- [Using `extract_table` on the FBI's National Instant Criminal Background Check System PDFs](examples/notebooks/extract-table-nics.ipynb). Demonstrates how to use visual debugging to find optimal table extraction settings. Also demonstrates `Page.crop(...)` and `Page.extract_text(...).`\n- [Inspecting and visualizing `curve` objects](examples/notebooks/ag-energy-roundup-curves.ipynb).\n- [Extracting fixed-width data from a San Jose PD firearm search report](examples/notebooks/san-jose-pd-firearm-report.ipynb), an example of using `Page.extract_text(...)`.\n\n## Comparison to other libraries\n\nSeveral other Python libraries help users to extract information from PDFs. As a broad overview, `pdfplumber` distinguishes itself from other PDF processing libraries by combining these features:\n\n- Easy access to detailed information about each PDF object\n- Higher-level, customizable methods for extracting text and tables\n- Tightly integrated visual debugging\n- Other useful utility functions, such as filtering objects via a crop-box\n\nIt's also helpful to know what features `pdfplumber` does __not__ provide:\n\n- PDF *generation*\n- PDF *modification*\n- Optical character recognition (OCR)\n- Strong support for extracting tables from OCR'ed documents\n\n### Specific comparisons\n\n- [`pdfminer.six`](https://github.com/pdfminer/pdfminer.six) provides the foundation for `pdfplumber`. It primarily focuses on parsing PDFs, analyzing PDF layouts and object positioning, and extracting text. It does not provide tools for table extraction or visual debugging. License: [MIT](https://github.com/pdfminer/pdfminer.six?tab=MIT-1-ov-file).\n\n- [`PyPDF2`](https://github.com/mstamy2/PyPDF2) is a pure-Python library \"capable of splitting, merging, cropping, and transforming the pages of PDF files. It can also add custom data, viewing options, and passwords to PDF files.\" It can extract page text, but does not provide easy access to shape objects (rectangles, lines, etc.), table-extraction, or visually debugging tools. License: [BSD](https://github.com/py-pdf/pypdf?tab=License-1-ov-file#readme).\n\n- [`pymupdf`](https://pymupdf.readthedocs.io/) is substantially faster than `pdfminer.six` (and thus also `pdfplumber`) and can generate and modify PDFs, but the library requires installation of non-Python software (MuPDF). It also does not enable easy access to shape objects (rectangles, lines, etc.), and does not provide table-extraction or visual debugging tools. License: [AGPL](https://pymupdf.readthedocs.io/en/latest/about.html#license-and-copyright).\n\n- [`camelot`](https://github.com/camelot-dev/camelot), [`tabula-py`](https://github.com/chezou/tabula-py), and [`pdftables`](https://github.com/drj11/pdftables) all focus primarily on extracting tables. In some cases, they may be better suited to the particular tables you are trying to extract. License: [MIT](https://github.com/camelot-dev/camelot?tab=MIT-1-ov-file#readme) (`camelot`), [MIT](https://github.com/chezou/tabula-py?tab=MIT-1-ov-file#readme) (`tabula-py`), [BSD](https://github.com/drj11/pdftables?tab=BSD-2-Clause-1-ov-file#readme) (`pdftables`).\n\n\n## Acknowledgments / Contributors\n\nMany thanks to the following users who've contributed ideas, features, and fixes:\n\n- [Jacob Fenton](https://github.com/jsfenfen)\n- [Dan Nguyen](https://github.com/dannguyen)\n- [Jeff Barrera](https://github.com/jeffbarrera)\n- [Bob Lannon](https://github.com/boblannon)\n- [Dustin Tindall](https://github.com/dustindall)\n- [@yevgnen](https://github.com/Yevgnen)\n- [@meldonization](https://github.com/meldonization)\n- [Oisín Moran](https://github.com/OisinMoran)\n- [Samkit Jain](https://github.com/samkit-jain)\n- [Francisco Aranda](https://github.com/frascuchon)\n- [Kwok-kuen Cheung](https://github.com/cheungpat)\n- [Marco](https://github.com/ubmarco)\n- [Idan David](https://github.com/idan-david)\n- [@xv44586](https://github.com/xv44586)\n- [Alexander Regueiro](https://github.com/alexreg)\n- [Daniel Peña](https://github.com/trifling)\n- [@bobluda](https://github.com/bobluda)\n- [@ramcdona](https://github.com/ramcdona)\n- [@johnhuge](https://github.com/johnhuge)\n- [Jhonatan Lopes](https://github.com/jhonatan-lopes)\n- [Ethan Corey](https://github.com/ethanscorey)\n- [Shannon Shen](https://github.com/lolipopshock)\n- [Matsumoto Toshi](https://github.com/toshi1127)\n- [John West](https://github.com/jwestwsj)\n- [David Huggins-Daines](https://github.com/dhdaines)\n- [Jeremy B. Merrill](https://github.com/jeremybmerrill)\n- [Echedey Luis](https://github.com/echedey-ls)\n- [Andy Friedman](https://github.com/afriedman412)\n- [Aron Weiler](https://github.com/aronweiler)\n- [Quentin André](https://github.com/QuentinAndre11)\n- [Léo Roux](https://github.com/leorouxx)\n- [@wodny](https://github.com/wodny)\n- [Michal Stolarczyk](https://github.com/stolarczyk)\n- [Brandon Roberts](https://github.com/brandonrobertz)\n- [@ennamarie19](https://github.com/ennamarie19)\n- [Anton Ilin](https://github.com/bronislav)\n\n## Contributing\n\nPull requests are welcome, but please submit a proposal issue first, as the library is in active development.\n\nCurrent maintainers:\n\n- [Jeremy Singer-Vine](https://github.com/jsvine)\n- [Samkit Jain](https://github.com/samkit-jain)\n"
},
{
"path": "codecov.yml",
"content": "codecov:\n branch: stable\n"
},
{
"path": "docs/colors.md",
"content": "# Colors\n\nIn the PDF specification, as well as in `pdfplumber`, most graphical objects can have two color attributes:\n\n- `stroking_color`: The color of the object's outline\n- `non_stroking_color`: The color of the object's interior, or \"fill\"\n\nIn the PDF specification, colors have both a \"color space\" and a \"color value\".\n\n## Color Spaces\n\nValid color spaces are grouped into three categories:\n\n- Device color spaces\n - `DeviceGray`\n - `DeviceRGB`\n - `DeviceCMYK`\n- CIE-based color spaces\n - `CalGray`\n - `CalRGB`\n - `Lab`\n - `ICCBased`\n- Special color spaces\n - `Indexed`\n - `Pattern`\n - `Separation`\n - `DeviceN`\n\nTo read more about the differences between those color spaces, see section 4.5 [here](https://ghostscript.com/~robin/pdf_reference17.pdf).\n\n`pdfplumber` aims to expose those color spaces as `scs` (stroking color space) and `ncs` (non-stroking color space), represented as a __string__.\n\n__Caveat__: The only information `pdfplumber` can __currently__ expose is the non-stroking color space for `char` objects. The rest (stroking color space for `char` objects and either color space for the other types of objects) will require a pull request to `pdfminer.six`.\n\n## Color Values\n\nThe color value determines *what specific color* in the color space should be used. With the exception of the \"special color spaces,\" these color values are specified as a series of numbers. For `DeviceRGB`, for example, the color values are three numbers, representing the intensities of red, green, and blue.\n\nIn `pdfplumber`, those color values are exposed as `stroking_color` and `non_stroking_color`, represented as a __tuple of numbers__.\n\nThe pattern specified by the `Pattern` color space is exposed via the `non_stroking_pattern` and `stroking_pattern` attributes.\n"
},
{
"path": "docs/repairing.md",
"content": "# Repairing Malformed PDFs\n\nMany parsing issues can be traced back to malformed PDFs.\n\nMalformed PDFs can often be [fixed via Ghostscript](https://superuser.com/questions/278562/how-can-i-fix-repair-a-corrupted-pdf-file).\n\n`pdfplumber` lets you automatically run those repairs, in several ways:\n\n- `pdfplumber.open(..., repair=True)` will repair your PDF on the fly (but not save the repaired version to disk).\n- `pdfplumber.repair(path_to_pdf)` will return a `BytesIO` object holding the bytes of a repaired version of the original file.\n- `pdfplumber.repair(path_to_pdf, outfile=\"path/to/repaired.pdf\")` will write a repaired version of the original file to the indicated `outfile` path.\n\n## Custom parameters\n\n- `gs_path=...`: You can pass a custom path for the Ghostscript executable, helpful in case `pdfplumber` is unable to auto-detect your copy of Ghostscript.\n"
},
{
"path": "docs/structure.md",
"content": "# Structure Tree\n\nSince PDF 1.3 it is possible for a PDF to contain logical structure,\ncontained in a *structure tree*. In conjunction with PDF 1.2 [marked\ncontent sections](#marked-content-sections) this forms the basis of\nTagged PDF and other accessibility features.\n\nUnfortunately, since all of these standards are optional and variably\nimplemented in PDF authoring tools, and are frequently not enabled by\ndefault, it is not possible to rely on them to extract the structure\nof a PDF and associated content. Nonetheless they can be useful as\nfeatures for a heuristic or machine-learning based system, or for\nextracting particular structures such as tables.\n\nSince `pdfplumber`'s API is page-based, the structure is available for\na particular page, using the `structure_tree` attribute:\n\n with pdfplumber.open(pdffile) as pdf:\n for element in pdf.pages[0].structure_tree:\n print(element[\"type\"], element[\"mcids\"])\n for child in element.children:\n print(child[\"type\"], child[\"mcids\"])\n\nThe `type` field contains the type of the structure element - the\nstandard structure types can be seen in section 10.7.3 of [the PDF 1.7\nreference\ndocument](https://ghostscript.com/~robin/pdf_reference17.pdf#page=898),\nbut usually they are rather HTML-like, if created by a recent PDF\nauthoring tool (notably, older tools may simply produce `P` for\neverything).\n\nThe `mcids` field contains the list of marked content section IDs\ncorresponding to this element.\n\nThe `lang` field is often present as well, and contains a language\ncode for the text content, e.g. `\"EN-US\"` or `\"FR-CA\"`.\n\nThe `alt_text` field will be present if the author has helpfully added\nalternate text to an image. In some cases, `actual_text` may also be\npresent.\n\nThere are also various attributes that may be in the `attributes`\nfield. Some of these are quite useful indeed, such as ``BBox` which\ngives you the bounding box of a `Table`, `Figure`, or `Image`. You\ncan see a full list of these [in the PDF\nspec](https://ghostscript.com/~robin/pdf_reference17.pdf#page=916).\nNote that the `BBox` is in PDF coordinate space with the origin at the\nbottom left of the page. To convert it to `pdfplumber`'s space you\ncan do, for example:\n\n x0, y0, x1, y1 = element['attributes']['BBox']\n top = page.height - y1\n bottom = page.height - y0\n doctop = page.initial_doctop + top\n bbox = (x0, top, x1, bottom)\n\nIt is also possible to get the structure tree for the entire document.\nIn this case, because marked content IDs are specific to a given page,\neach element will also have a `page_number` attribute, which is the\nnumber of the page containing (partially or completely) this element,\nindexed from 1 (for consistency with `pdfplumber.Page`).\n\nYou can also access the underlying `PDFStructTree` object for more\nflexibility, including visual debugging. For instance to plot the\nbounding boxes of the contents of all of the `TD` elements on the\nfirst page of a document:\n\n page = pdf.pages[0]\n stree = PDFStructTree(pdf, page)\n img = page.to_image()\n img.draw_rects(stree.element_bbox(td) for td in table.find_all(\"TD\"))\n\nThe `find_all` method works rather like the same method in\n[BeautifulSoup](https://beautiful-soup-4.readthedocs.io/en/latest/#searching-the-tree) -\nit takes an element name, a regular expression, or a matching\nfunction.\n"
},
{
"path": "pdfplumber/__init__.py",
"content": "__all__ = [\n \"__version__\",\n \"utils\",\n \"pdfminer\",\n \"open\",\n \"repair\",\n \"set_debug\",\n]\n\nimport pdfminer\nimport pdfminer.pdftypes\n\nfrom . import utils\nfrom ._version import __version__\nfrom .pdf import PDF\nfrom .repair import repair\n\nopen = PDF.open\n"
},
{
"path": "pdfplumber/_typing.py",
"content": "from typing import Any, Dict, Iterable, List, Literal, Sequence, Tuple, Union\n\nT_seq = Sequence\nT_num = Union[int, float]\nT_point = Tuple[T_num, T_num]\nT_bbox = Tuple[T_num, T_num, T_num, T_num]\nT_obj = Dict[str, Any]\nT_obj_list = List[T_obj]\nT_obj_iter = Iterable[T_obj]\nT_dir = Union[Literal[\"ltr\"], Literal[\"rtl\"], Literal[\"ttb\"], Literal[\"btt\"]]\n"
},
{
"path": "pdfplumber/_version.py",
"content": "version_info = (0, 11, 9)\n__version__ = \".\".join(map(str, version_info))\n"
},
{
"path": "pdfplumber/cli.py",
"content": "#!/usr/bin/env python\nimport argparse\nimport json\nimport sys\nfrom collections import defaultdict, deque\nfrom itertools import chain\nfrom typing import Any, DefaultDict, Dict, List\n\nfrom .pdf import PDF\n\nif len(sys.argv) == 1:\n sys.argv.append(\"--help\")\n\n\ndef parse_page_spec(p_str: str) -> List[int]:\n if \"-\" in p_str:\n start, end = map(int, p_str.split(\"-\"))\n return list(range(start, end + 1))\n else:\n return [int(p_str)]\n\n\ndef parse_args(args_raw: List[str]) -> argparse.Namespace:\n parser = argparse.ArgumentParser(\"pdfplumber\")\n\n parser.add_argument(\"infile\", nargs=\"?\", type=argparse.FileType(\"rb\"))\n group = parser.add_mutually_exclusive_group()\n group.add_argument(\n \"--structure\",\n help=\"Write the structure tree as JSON. \"\n \"All other arguments except --pages, --laparams, and --indent will be ignored\",\n action=\"store_true\",\n )\n group.add_argument(\n \"--structure-text\",\n help=\"Write the structure tree as JSON including text contents. \"\n \"All other arguments except --pages, --laparams, and --indent will be ignored\",\n action=\"store_true\",\n )\n\n parser.add_argument(\"--format\", choices=[\"csv\", \"json\", \"text\"], default=\"csv\")\n\n parser.add_argument(\"--types\", nargs=\"+\")\n\n parser.add_argument(\n \"--include-attrs\",\n nargs=\"+\",\n help=\"Include *only* these object attributes in output.\",\n )\n\n parser.add_argument(\n \"--exclude-attrs\",\n nargs=\"+\",\n help=\"Exclude these object attributes from output.\",\n )\n\n parser.add_argument(\"--laparams\", type=json.loads)\n\n parser.add_argument(\"--precision\", type=int)\n\n parser.add_argument(\"--pages\", nargs=\"+\", type=parse_page_spec)\n\n parser.add_argument(\n \"--indent\", type=int, help=\"Indent level for JSON pretty-printing.\"\n )\n\n args = parser.parse_args(args_raw)\n if args.pages is not None:\n args.pages = list(chain(*args.pages))\n return args\n\n\ndef add_text_to_mcids(pdf: PDF, data: List[Dict[str, Any]]) -> None:\n page_contents: DefaultDict[int, Any] = defaultdict(lambda: defaultdict(str))\n for page in pdf.pages:\n text_contents = page_contents[page.page_number]\n for c in page.chars:\n mcid = c.get(\"mcid\")\n if mcid is None:\n continue\n text_contents[mcid] += c[\"text\"]\n d = deque(data)\n while d:\n el = d.popleft()\n if \"children\" in el:\n d.extend(el[\"children\"])\n pageno = el.get(\"page_number\")\n if pageno is None:\n continue\n text_contents = page_contents[pageno]\n if \"mcids\" in el:\n el[\"text\"] = [text_contents[mcid] for mcid in el[\"mcids\"]]\n\n\ndef main(args_raw: List[str] = sys.argv[1:]) -> None:\n args = parse_args(args_raw)\n\n with PDF.open(args.infile, pages=args.pages, laparams=args.laparams) as pdf:\n if args.structure:\n print(json.dumps(pdf.structure_tree, indent=args.indent))\n elif args.structure_text:\n tree = pdf.structure_tree\n add_text_to_mcids(pdf, tree)\n print(json.dumps(tree, indent=args.indent, ensure_ascii=False))\n elif args.format == \"csv\":\n pdf.to_csv(\n sys.stdout,\n args.types,\n precision=args.precision,\n include_attrs=args.include_attrs,\n exclude_attrs=args.exclude_attrs,\n )\n elif args.format == \"text\":\n for page in pdf.pages:\n print(page.extract_text(layout=True))\n else:\n pdf.to_json(\n sys.stdout,\n args.types,\n precision=args.precision,\n include_attrs=args.include_attrs,\n exclude_attrs=args.exclude_attrs,\n indent=args.indent,\n )\n\n\nif __name__ == \"__main__\":\n main()\n"
},
{
"path": "pdfplumber/container.py",
"content": "import csv\nimport json\nfrom io import StringIO\nfrom itertools import chain\nfrom typing import Any, Dict, List, Optional, Set, TextIO\n\nfrom . import utils\nfrom ._typing import T_obj, T_obj_list\nfrom .convert import CSV_COLS_REQUIRED, CSV_COLS_TO_PREPEND, Serializer\n\n\nclass Container(object):\n cached_properties = [\"_rect_edges\", \"_curve_edges\", \"_edges\", \"_objects\"]\n\n @property\n def pages(self) -> Optional[List[Any]]: # pragma: nocover\n raise NotImplementedError\n\n @property\n def objects(self) -> Dict[str, T_obj_list]: # pragma: nocover\n raise NotImplementedError\n\n def to_dict(\n self, object_types: Optional[List[str]] = None\n ) -> Dict[str, Any]: # pragma: nocover\n raise NotImplementedError\n\n def flush_cache(self, properties: Optional[List[str]] = None) -> None:\n props = self.cached_properties if properties is None else properties\n for p in props:\n if hasattr(self, p):\n delattr(self, p)\n\n @property\n def rects(self) -> T_obj_list:\n return self.objects.get(\"rect\", [])\n\n @property\n def lines(self) -> T_obj_list:\n return self.objects.get(\"line\", [])\n\n @property\n def curves(self) -> T_obj_list:\n return self.objects.get(\"curve\", [])\n\n @property\n def images(self) -> T_obj_list:\n return self.objects.get(\"image\", [])\n\n @property\n def chars(self) -> T_obj_list:\n return self.objects.get(\"char\", [])\n\n @property\n def textboxverticals(self) -> T_obj_list:\n return self.objects.get(\"textboxvertical\", [])\n\n @property\n def textboxhorizontals(self) -> T_obj_list:\n return self.objects.get(\"textboxhorizontal\", [])\n\n @property\n def textlineverticals(self) -> T_obj_list:\n return self.objects.get(\"textlinevertical\", [])\n\n @property\n def textlinehorizontals(self) -> T_obj_list:\n return self.objects.get(\"textlinehorizontal\", [])\n\n @property\n def rect_edges(self) -> T_obj_list:\n if hasattr(self, \"_rect_edges\"):\n return self._rect_edges\n rect_edges_gen = (utils.rect_to_edges(r) for r in self.rects)\n self._rect_edges: T_obj_list = list(chain(*rect_edges_gen))\n return self._rect_edges\n\n @property\n def curve_edges(self) -> T_obj_list:\n if hasattr(self, \"_curve_edges\"):\n return self._curve_edges\n curve_edges_gen = (utils.curve_to_edges(r) for r in self.curves)\n self._curve_edges: T_obj_list = list(chain(*curve_edges_gen))\n return self._curve_edges\n\n @property\n def edges(self) -> T_obj_list:\n if hasattr(self, \"_edges\"):\n return self._edges\n line_edges = list(map(utils.line_to_edge, self.lines))\n self._edges: T_obj_list = line_edges + self.rect_edges + self.curve_edges\n return self._edges\n\n @property\n def horizontal_edges(self) -> T_obj_list:\n def test(x: T_obj) -> bool:\n return bool(x[\"orientation\"] == \"h\")\n\n return list(filter(test, self.edges))\n\n @property\n def vertical_edges(self) -> T_obj_list:\n def test(x: T_obj) -> bool:\n return bool(x[\"orientation\"] == \"v\")\n\n return list(filter(test, self.edges))\n\n def to_json(\n self,\n stream: Optional[TextIO] = None,\n object_types: Optional[List[str]] = None,\n include_attrs: Optional[List[str]] = None,\n exclude_attrs: Optional[List[str]] = None,\n precision: Optional[int] = None,\n indent: Optional[int] = None,\n ) -> Optional[str]:\n\n data = self.to_dict(object_types)\n\n serialized = Serializer(\n precision=precision,\n include_attrs=include_attrs,\n exclude_attrs=exclude_attrs,\n ).serialize(data)\n\n if stream is None:\n return json.dumps(serialized, indent=indent)\n else:\n json.dump(serialized, stream, indent=indent)\n return None\n\n def to_csv(\n self,\n stream: Optional[TextIO] = None,\n object_types: Optional[List[str]] = None,\n precision: Optional[int] = None,\n include_attrs: Optional[List[str]] = None,\n exclude_attrs: Optional[List[str]] = None,\n ) -> Optional[str]:\n if stream is None:\n stream = StringIO()\n to_string = True\n else:\n to_string = False\n\n if object_types is None:\n object_types = list(self.objects.keys()) + [\"annot\"]\n\n serialized = []\n fields: Set[str] = set()\n\n pages = [self] if self.pages is None else self.pages\n\n serializer = Serializer(\n precision=precision,\n include_attrs=include_attrs,\n exclude_attrs=exclude_attrs,\n )\n for page in pages:\n for t in object_types:\n objs = getattr(page, t + \"s\")\n if len(objs):\n serialized += serializer.serialize(objs)\n new_keys = [k for k, v in objs[0].items() if type(v) is not dict]\n fields = fields.union(set(new_keys))\n\n non_req_cols = CSV_COLS_TO_PREPEND + list(\n sorted(set(fields) - set(CSV_COLS_REQUIRED + CSV_COLS_TO_PREPEND))\n )\n\n cols = CSV_COLS_REQUIRED + list(filter(serializer.attr_filter, non_req_cols))\n\n w = csv.DictWriter(\n stream,\n fieldnames=cols,\n extrasaction=\"ignore\",\n quoting=csv.QUOTE_MINIMAL,\n escapechar=\"\\\\\",\n )\n w.writeheader()\n w.writerows(serialized)\n\n if to_string:\n stream.seek(0)\n return stream.read()\n else:\n return None\n"
},
{
"path": "pdfplumber/convert.py",
"content": "import base64\nfrom typing import Any, Callable, Dict, List, Optional, Tuple\n\nfrom pdfminer.psparser import PSLiteral\n\nfrom .utils import decode_text\n\nENCODINGS_TO_TRY = [\n \"utf-8\",\n \"latin-1\",\n \"utf-16\",\n \"utf-16le\",\n]\n\nCSV_COLS_REQUIRED = [\n \"object_type\",\n]\n\nCSV_COLS_TO_PREPEND = [\n \"page_number\",\n \"x0\",\n \"x1\",\n \"y0\",\n \"y1\",\n \"doctop\",\n \"top\",\n \"bottom\",\n \"width\",\n \"height\",\n]\n\n\ndef get_attr_filter(\n include_attrs: Optional[List[str]] = None, exclude_attrs: Optional[List[str]] = None\n) -> Callable[[str], bool]:\n if include_attrs is not None and exclude_attrs is not None:\n raise ValueError(\n \"Cannot specify `include_attrs` and `exclude_attrs` at the same time.\"\n )\n\n elif include_attrs is not None:\n incl = set(CSV_COLS_REQUIRED + include_attrs)\n return lambda attr: attr in incl\n\n elif exclude_attrs is not None:\n nonexcludable = set(exclude_attrs).intersection(set(CSV_COLS_REQUIRED))\n if len(nonexcludable):\n raise ValueError(\n f\"Cannot exclude these required properties: {list(nonexcludable)}\"\n )\n excl = set(exclude_attrs)\n return lambda attr: attr not in excl\n\n else:\n return lambda attr: True\n\n\ndef to_b64(data: bytes) -> str:\n return base64.b64encode(data).decode(\"ascii\")\n\n\nclass Serializer:\n def __init__(\n self,\n precision: Optional[int] = None,\n include_attrs: Optional[List[str]] = None,\n exclude_attrs: Optional[List[str]] = None,\n ):\n\n self.precision = precision\n self.attr_filter = get_attr_filter(\n include_attrs=include_attrs, exclude_attrs=exclude_attrs\n )\n\n def serialize(self, obj: Any) -> Any:\n if obj is None:\n return None\n\n t = type(obj)\n\n # Basic types don't need to be converted\n if t in (int, str):\n return obj\n\n # Use one of the custom converters, if possible\n fn = getattr(self, f\"do_{t.__name__}\", None)\n if fn is not None:\n return fn(obj)\n\n # Otherwise, just use the string-representation\n else:\n return str(obj)\n\n def do_float(self, x: float) -> float:\n return x if self.precision is None else round(x, self.precision)\n\n def do_bool(self, x: bool) -> int:\n return int(x)\n\n def do_list(self, obj: List[Any]) -> List[Any]:\n return list(self.serialize(x) for x in obj)\n\n def do_tuple(self, obj: Tuple[Any, ...]) -> Tuple[Any, ...]:\n return tuple(self.serialize(x) for x in obj)\n\n def do_dict(self, obj: Dict[str, Any]) -> Dict[str, Any]:\n if \"object_type\" in obj.keys():\n return {k: self.serialize(v) for k, v in obj.items() if self.attr_filter(k)}\n else:\n return {k: self.serialize(v) for k, v in obj.items()}\n\n def do_PDFStream(self, obj: Any) -> Dict[str, Optional[str]]:\n return {\"rawdata\": to_b64(obj.rawdata) if obj.rawdata else None}\n\n def do_PSLiteral(self, obj: PSLiteral) -> str:\n return decode_text(obj.name)\n\n def do_bytes(self, obj: bytes) -> Optional[str]:\n for e in ENCODINGS_TO_TRY:\n try:\n return obj.decode(e)\n except UnicodeDecodeError: # pragma: no cover\n return None\n # If none of the decodings work, raise whatever error\n # decoding with utf-8 causes\n obj.decode(ENCODINGS_TO_TRY[0]) # pragma: no cover\n return None # pragma: no cover\n"
},
{
"path": "pdfplumber/ctm.py",
"content": "import math\nfrom typing import NamedTuple\n\n# For more details, see the PDF Reference, 6th Ed., Section 4.2.2 (\"Common\n# Transformations\")\n\n\nclass CTM(NamedTuple):\n a: float\n b: float\n c: float\n d: float\n e: float\n f: float\n\n @property\n def scale_x(self) -> float:\n return math.sqrt(pow(self.a, 2) + pow(self.b, 2))\n\n @property\n def scale_y(self) -> float:\n return math.sqrt(pow(self.c, 2) + pow(self.d, 2))\n\n @property\n def skew_x(self) -> float:\n return (math.atan2(self.d, self.c) * 180 / math.pi) - 90\n\n @property\n def skew_y(self) -> float:\n return math.atan2(self.b, self.a) * 180 / math.pi\n\n @property\n def translation_x(self) -> float:\n return self.e\n\n @property\n def translation_y(self) -> float:\n return self.f\n"
},
{
"path": "pdfplumber/display.py",
"content": "import pathlib\nfrom io import BufferedReader, BytesIO\nfrom typing import TYPE_CHECKING, Any, List, Optional, Tuple, Union\n\nimport PIL.Image\nimport PIL.ImageDraw\nimport pypdfium2 # type: ignore\n\nfrom . import utils\nfrom ._typing import T_bbox, T_num, T_obj, T_obj_list, T_point, T_seq\nfrom .table import T_table_settings, Table, TableFinder, TableSettings\nfrom .utils.exceptions import MalformedPDFException\n\nif TYPE_CHECKING: # pragma: nocover\n import pandas as pd\n\n from .page import Page\n\n\nclass COLORS:\n RED = (255, 0, 0)\n GREEN = (0, 255, 0)\n BLUE = (0, 0, 255)\n TRANSPARENT = (0, 0, 0, 0)\n\n\nDEFAULT_FILL = COLORS.BLUE + (50,)\nDEFAULT_STROKE = COLORS.RED + (200,)\nDEFAULT_STROKE_WIDTH = 1\nDEFAULT_RESOLUTION = 72\n\nT_color = Union[Tuple[int, int, int], Tuple[int, int, int, int], str]\nT_contains_points = Union[Tuple[T_point, ...], List[T_point], T_obj]\n\n\ndef get_page_image(\n stream: Union[BufferedReader, BytesIO],\n path: Optional[pathlib.Path],\n page_ix: int,\n resolution: Union[int, float],\n password: Optional[str],\n antialias: bool = False,\n) -> PIL.Image.Image:\n\n src: Union[pathlib.Path, BufferedReader, BytesIO]\n\n # If we are working with a file object saved to disk\n if path:\n src = path\n\n # If we instead are working with a BytesIO stream\n else:\n stream.seek(0)\n src = stream\n\n try:\n pdfium_doc = pypdfium2.PdfDocument(src, password=password)\n except pypdfium2.PdfiumError as e:\n raise MalformedPDFException(e)\n\n pdfium_page = pdfium_doc.get_page(page_ix)\n\n img: PIL.Image.Image = pdfium_page.render(\n # Modifiable arguments\n scale=resolution / 72,\n no_smoothtext=not antialias,\n no_smoothpath=not antialias,\n no_smoothimage=not antialias,\n # Non-modifiable arguments\n prefer_bgrx=True,\n ).to_pil()\n pdfium_doc.close()\n\n return img.convert(\"RGB\")\n\n\nclass PageImage:\n def __init__(\n self,\n page: \"Page\",\n original: Optional[PIL.Image.Image] = None,\n resolution: Union[int, float] = DEFAULT_RESOLUTION,\n antialias: bool = False,\n force_mediabox: bool = False,\n ):\n self.page = page\n self.root = page if page.is_original else page.root_page\n self.resolution = resolution\n\n if original is None:\n self.original = get_page_image(\n stream=page.pdf.stream,\n path=page.pdf.path,\n page_ix=page.page_number - 1,\n resolution=resolution,\n antialias=antialias,\n password=page.pdf.password,\n )\n else:\n self.original = original\n\n self.scale = self.original.size[0] / (page.cropbox[2] - page.cropbox[0])\n\n # This value represents the coordinates of the page,\n # in page-unit values, that will be displayed.\n self.bbox = (\n page.bbox\n if page.bbox != page.mediabox\n else (page.mediabox if force_mediabox else page.cropbox)\n )\n\n # If this value is different than the *Page*'s .cropbox\n # (e.g., because the mediabox differs from the cropbox or\n # or because we've used Page.crop(...)), then we'll need to\n # crop the initially-converted image.\n if page.bbox != page.cropbox:\n crop_dims = self._reproject_bbox(page.cropbox)\n bbox_dims = self._reproject_bbox(self.bbox)\n self.original = self.original.crop(\n (\n bbox_dims[0] - crop_dims[0],\n bbox_dims[1] - crop_dims[1],\n bbox_dims[2] - crop_dims[0],\n bbox_dims[3] - crop_dims[1],\n )\n )\n\n self.reset()\n\n def _reproject_bbox(self, bbox: T_bbox) -> Tuple[int, int, int, int]:\n x0, top, x1, bottom = bbox\n _x0, _top = self._reproject((x0, top))\n _x1, _bottom = self._reproject((x1, bottom))\n return (_x0, _top, _x1, _bottom)\n\n def _reproject(self, coord: T_point) -> Tuple[int, int]:\n \"\"\"\n Given an (x0, top) tuple from the *root* coordinate system,\n return an (x0, top) tuple in the *image* coordinate system.\n \"\"\"\n x0, top = coord\n _x0 = (x0 - self.bbox[0]) * self.scale\n _top = (top - self.bbox[1]) * self.scale\n return (int(_x0), int(_top))\n\n def reset(self) -> \"PageImage\":\n self.annotated = PIL.Image.new(\"RGB\", self.original.size)\n self.annotated.paste(self.original)\n self.draw = PIL.ImageDraw.Draw(self.annotated, \"RGBA\")\n return self\n\n def save(\n self,\n dest: Union[str, pathlib.Path, BytesIO],\n format: str = \"PNG\",\n quantize: bool = True,\n colors: int = 256,\n bits: int = 8,\n **kwargs: Any,\n ) -> None:\n if quantize:\n out = self.annotated.quantize(colors, method=PIL.Image.FASTOCTREE).convert(\n \"P\"\n )\n else:\n out = self.annotated\n\n out.save(\n dest,\n format=format,\n bits=bits,\n dpi=(self.resolution, self.resolution),\n **kwargs,\n )\n\n def copy(self) -> \"PageImage\":\n return self.__class__(self.page, self.original)\n\n def draw_line(\n self,\n points_or_obj: T_contains_points,\n stroke: T_color = DEFAULT_STROKE,\n stroke_width: int = DEFAULT_STROKE_WIDTH,\n ) -> \"PageImage\":\n # If passing a raw list of points, use those\n if isinstance(points_or_obj, (tuple, list)):\n points = points_or_obj\n # Else, use the \"pts\" attribute if available\n elif isinstance(points_or_obj, dict) and \"pts\" in points_or_obj:\n points = [(x, y) for x, y in points_or_obj[\"pts\"]]\n # Otherwise, just use ((x0, top), (x1, bottom))\n else:\n obj = points_or_obj\n points = ((obj[\"x0\"], obj[\"top\"]), (obj[\"x1\"], obj[\"bottom\"]))\n\n self.draw.line(\n list(map(self._reproject, points)), fill=stroke, width=stroke_width\n )\n\n return self\n\n def draw_lines(\n self,\n list_of_lines: Union[T_seq[T_contains_points], \"pd.DataFrame\"],\n stroke: T_color = DEFAULT_STROKE,\n stroke_width: int = DEFAULT_STROKE_WIDTH,\n ) -> \"PageImage\":\n for x in utils.to_list(list_of_lines):\n self.draw_line(x, stroke=stroke, stroke_width=stroke_width)\n return self\n\n def draw_vline(\n self,\n location: T_num,\n stroke: T_color = DEFAULT_STROKE,\n stroke_width: int = DEFAULT_STROKE_WIDTH,\n ) -> \"PageImage\":\n points = (location, self.bbox[1], location, self.bbox[3])\n self.draw.line(self._reproject_bbox(points), fill=stroke, width=stroke_width)\n return self\n\n def draw_vlines(\n self,\n locations: Union[List[T_num], \"pd.Series[float]\"],\n stroke: T_color = DEFAULT_STROKE,\n stroke_width: int = DEFAULT_STROKE_WIDTH,\n ) -> \"PageImage\":\n for x in list(locations):\n self.draw_vline(x, stroke=stroke, stroke_width=stroke_width)\n return self\n\n def draw_hline(\n self,\n location: T_num,\n stroke: T_color = DEFAULT_STROKE,\n stroke_width: int = DEFAULT_STROKE_WIDTH,\n ) -> \"PageImage\":\n points = (self.bbox[0], location, self.bbox[2], location)\n self.draw.line(self._reproject_bbox(points), fill=stroke, width=stroke_width)\n return self\n\n def draw_hlines(\n self,\n locations: Union[List[T_num], \"pd.Series[float]\"],\n stroke: T_color = DEFAULT_STROKE,\n stroke_width: int = DEFAULT_STROKE_WIDTH,\n ) -> \"PageImage\":\n for x in list(locations):\n self.draw_hline(x, stroke=stroke, stroke_width=stroke_width)\n return self\n\n def draw_rect(\n self,\n bbox_or_obj: Union[T_bbox, T_obj],\n fill: T_color = DEFAULT_FILL,\n stroke: T_color = DEFAULT_STROKE,\n stroke_width: int = DEFAULT_STROKE_WIDTH,\n ) -> \"PageImage\":\n if isinstance(bbox_or_obj, (tuple, list)):\n bbox = bbox_or_obj\n else:\n obj = bbox_or_obj\n bbox = (obj[\"x0\"], obj[\"top\"], obj[\"x1\"], obj[\"bottom\"])\n\n x0, top, x1, bottom = bbox\n half = stroke_width / 2\n x0 = min(x0 + half, (x0 + x1) / 2)\n top = min(top + half, (top + bottom) / 2)\n x1 = max(x1 - half, (x0 + x1) / 2)\n bottom = max(bottom - half, (top + bottom) / 2)\n\n fill_bbox = self._reproject_bbox((x0, top, x1, bottom))\n self.draw.rectangle(fill_bbox, fill, COLORS.TRANSPARENT)\n\n if stroke_width > 0:\n segments = [\n ((x0, top), (x1, top)), # top\n ((x0, bottom), (x1, bottom)), # bottom\n ((x0, top), (x0, bottom)), # left\n ((x1, top), (x1, bottom)), # right\n ]\n self.draw_lines(segments, stroke=stroke, stroke_width=stroke_width)\n return self\n\n def draw_rects(\n self,\n list_of_rects: Union[List[T_bbox], T_obj_list, \"pd.DataFrame\"],\n fill: T_color = DEFAULT_FILL,\n stroke: T_color = DEFAULT_STROKE,\n stroke_width: int = DEFAULT_STROKE_WIDTH,\n ) -> \"PageImage\":\n for x in utils.to_list(list_of_rects):\n self.draw_rect(x, fill=fill, stroke=stroke, stroke_width=stroke_width)\n return self\n\n def draw_circle(\n self,\n center_or_obj: Union[T_point, T_obj],\n radius: int = 5,\n fill: T_color = DEFAULT_FILL,\n stroke: T_color = DEFAULT_STROKE,\n ) -> \"PageImage\":\n if isinstance(center_or_obj, tuple):\n center = center_or_obj\n else:\n obj = center_or_obj\n center = ((obj[\"x0\"] + obj[\"x1\"]) / 2, (obj[\"top\"] + obj[\"bottom\"]) / 2)\n cx, cy = center\n bbox = (cx - radius, cy - radius, cx + radius, cy + radius)\n self.draw.ellipse(self._reproject_bbox(bbox), fill, stroke)\n return self\n\n def draw_circles(\n self,\n list_of_circles: Union[List[T_point], T_obj_list, \"pd.DataFrame\"],\n radius: int = 5,\n fill: T_color = DEFAULT_FILL,\n stroke: T_color = DEFAULT_STROKE,\n ) -> \"PageImage\":\n for x in utils.to_list(list_of_circles):\n self.draw_circle(x, radius=radius, fill=fill, stroke=stroke)\n return self\n\n def debug_table(\n self,\n table: Table,\n fill: T_color = DEFAULT_FILL,\n stroke: T_color = DEFAULT_STROKE,\n stroke_width: int = 1,\n ) -> \"PageImage\":\n \"\"\"\n Outline all found tables.\n \"\"\"\n self.draw_rects(\n table.cells, fill=fill, stroke=stroke, stroke_width=stroke_width\n )\n return self\n\n def debug_tablefinder(\n self,\n table_settings: Optional[\n Union[TableFinder, TableSettings, T_table_settings]\n ] = None,\n ) -> \"PageImage\":\n if isinstance(table_settings, TableFinder):\n finder = table_settings\n elif table_settings is None or isinstance(\n table_settings, (TableSettings, dict)\n ):\n finder = self.page.debug_tablefinder(table_settings)\n else:\n raise ValueError(\n \"Argument must be instance of TableFinder\"\n \"or a TableFinder settings dict.\"\n )\n\n for table in finder.tables:\n self.debug_table(table)\n\n self.draw_lines(finder.edges, stroke_width=1)\n\n self.draw_circles(\n list(finder.intersections.keys()),\n fill=COLORS.TRANSPARENT,\n stroke=COLORS.BLUE + (200,),\n radius=3,\n )\n return self\n\n def outline_words(\n self,\n stroke: T_color = DEFAULT_STROKE,\n fill: T_color = DEFAULT_FILL,\n stroke_width: int = DEFAULT_STROKE_WIDTH,\n x_tolerance: T_num = utils.DEFAULT_X_TOLERANCE,\n y_tolerance: T_num = utils.DEFAULT_Y_TOLERANCE,\n ) -> \"PageImage\":\n\n words = self.page.extract_words(\n x_tolerance=x_tolerance, y_tolerance=y_tolerance\n )\n self.draw_rects(words, stroke=stroke, fill=fill, stroke_width=stroke_width)\n return self\n\n def outline_chars(\n self,\n stroke: T_color = (255, 0, 0, 255),\n fill: T_color = (255, 0, 0, int(255 / 4)),\n stroke_width: int = DEFAULT_STROKE_WIDTH,\n ) -> \"PageImage\":\n\n self.draw_rects(\n self.page.chars, stroke=stroke, fill=fill, stroke_width=stroke_width\n )\n return self\n\n def _repr_png_(self) -> bytes:\n b = BytesIO()\n self.save(b, \"PNG\")\n return b.getvalue()\n\n def show(self) -> None: # pragma: no cover\n self.annotated.show()\n"
},
{
"path": "pdfplumber/page.py",
"content": "import numbers\nimport re\nfrom functools import lru_cache\nfrom typing import (\n TYPE_CHECKING,\n Any,\n Callable,\n Dict,\n Generator,\n List,\n Optional,\n Pattern,\n Tuple,\n Union,\n)\nfrom unicodedata import normalize as normalize_unicode\nfrom warnings import warn\n\nfrom pdfminer.converter import PDFPageAggregator\nfrom pdfminer.layout import (\n LTChar,\n LTComponent,\n LTContainer,\n LTCurve,\n LTItem,\n LTPage,\n LTTextContainer,\n)\nfrom pdfminer.pdfinterp import PDFPageInterpreter, PDFStackT\nfrom pdfminer.pdfpage import PDFPage\nfrom pdfminer.psparser import PSLiteral\n\nfrom . import utils\nfrom ._typing import T_bbox, T_num, T_obj, T_obj_list\nfrom .container import Container\nfrom .structure import PDFStructTree, StructTreeMissing\nfrom .table import T_table_settings, Table, TableFinder, TableSettings\nfrom .utils import decode_text, resolve_all, resolve_and_decode\nfrom .utils.exceptions import MalformedPDFException, PdfminerException\nfrom .utils.text import TextMap\n\nlt_pat = re.compile(r\"^LT\")\n\nALL_ATTRS = set(\n [\n \"adv\",\n \"height\",\n \"linewidth\",\n \"pts\",\n \"size\",\n \"srcsize\",\n \"width\",\n \"x0\",\n \"x1\",\n \"y0\",\n \"y1\",\n \"bits\",\n \"matrix\",\n \"upright\",\n \"fontname\",\n \"text\",\n \"imagemask\",\n \"colorspace\",\n \"evenodd\",\n \"fill\",\n \"non_stroking_color\",\n \"stroke\",\n \"stroking_color\",\n \"stream\",\n \"name\",\n \"mcid\",\n \"tag\",\n ]\n)\n\n\nif TYPE_CHECKING: # pragma: nocover\n from .display import PageImage\n from .pdf import PDF\n\n# via https://git.ghostscript.com/?p=mupdf.git;a=blob;f=source/pdf/pdf-font.c;h=6322cedf2c26cfb312c0c0878d7aff97b4c7470e;hb=HEAD#l774 # noqa\n\nCP936_FONTNAMES = {\n b\"\\xcb\\xce\\xcc\\xe5\": \"SimSun,Regular\",\n b\"\\xba\\xda\\xcc\\xe5\": \"SimHei,Regular\",\n b\"\\xbf\\xac\\xcc\\xe5_GB2312\": \"SimKai,Regular\",\n b\"\\xb7\\xc2\\xcb\\xce_GB2312\": \"SimFang,Regular\",\n b\"\\xc1\\xa5\\xca\\xe9\": \"SimLi,Regular\",\n}\n\n\ndef fix_fontname_bytes(fontname: bytes) -> str:\n if b\"+\" in fontname:\n split_at = fontname.index(b\"+\") + 1\n prefix, suffix = fontname[:split_at], fontname[split_at:]\n else:\n prefix, suffix = b\"\", fontname\n\n suffix_new = CP936_FONTNAMES.get(suffix, str(suffix)[2:-1])\n return str(prefix)[2:-1] + suffix_new\n\n\ndef tuplify_list_kwargs(kwargs: Dict[str, Any]) -> Dict[str, Any]:\n return {\n key: (tuple(value) if isinstance(value, list) else value)\n for key, value in kwargs.items()\n }\n\n\nclass PDFPageAggregatorWithMarkedContent(PDFPageAggregator):\n \"\"\"Extract layout from a specific page, adding marked-content IDs to\n objects where found.\"\"\"\n\n cur_mcid: Optional[int] = None\n cur_tag: Optional[str] = None\n\n def begin_tag(self, tag: PSLiteral, props: Optional[PDFStackT] = None) -> None:\n \"\"\"Handle beginning of tag, setting current MCID if any.\"\"\"\n self.cur_tag = decode_text(tag.name)\n if isinstance(props, dict) and \"MCID\" in props:\n self.cur_mcid = props[\"MCID\"]\n else:\n self.cur_mcid = None\n\n def end_tag(self) -> None:\n \"\"\"Handle beginning of tag, clearing current MCID.\"\"\"\n self.cur_tag = None\n self.cur_mcid = None\n\n def tag_cur_item(self) -> None:\n \"\"\"Add current MCID to what we hope to be the most recent object created\n by pdfminer.six.\"\"\"\n # This is somewhat hacky and would not be necessary if\n # pdfminer.six supported MCIDs. In reading the code it's\n # clear that the `render_*` methods methods will only ever\n # create one object, but that is far from being guaranteed.\n # Even if pdfminer.six's API would just return the objects it\n # creates, we wouldn't have to do this.\n if self.cur_item._objs:\n cur_obj = self.cur_item._objs[-1]\n cur_obj.mcid = self.cur_mcid # type: ignore\n cur_obj.tag = self.cur_tag # type: ignore\n\n def render_char(self, *args, **kwargs) -> float: # type: ignore\n \"\"\"Hook for rendering characters, adding the `mcid` attribute.\"\"\"\n adv = super().render_char(*args, **kwargs)\n self.tag_cur_item()\n return adv\n\n def render_image(self, *args, **kwargs) -> None: # type: ignore\n \"\"\"Hook for rendering images, adding the `mcid` attribute.\"\"\"\n super().render_image(*args, **kwargs)\n self.tag_cur_item()\n\n def paint_path(self, *args, **kwargs) -> None: # type: ignore\n \"\"\"Hook for rendering lines and curves, adding the `mcid` attribute.\"\"\"\n super().paint_path(*args, **kwargs)\n self.tag_cur_item()\n\n\ndef _normalize_box(box_raw: T_bbox, rotation: T_num = 0) -> T_bbox:\n # Per PDF Reference 3.8.4: \"Note: Although rectangles are\n # conventionally specified by their lower-left and upperright\n # corners, it is acceptable to specify any two diagonally opposite\n # corners.\"\n if not all(isinstance(x, numbers.Number) for x in box_raw): # pragma: nocover\n raise MalformedPDFException(\n f\"Bounding box contains non-number coordinate(s): {box_raw}\"\n )\n x0, x1 = sorted((box_raw[0], box_raw[2]))\n y0, y1 = sorted((box_raw[1], box_raw[3]))\n if rotation in [90, 270]:\n return (y0, x0, y1, x1)\n else:\n return (x0, y0, x1, y1)\n\n\n# PDFs coordinate spaces refer to an origin in the bottom-left of the\n# page; pdfplumber flips this vertically, so that the origin is in the\n# top-left.\ndef _invert_box(box_raw: T_bbox, mb_height: T_num) -> T_bbox:\n x0, y0, x1, y1 = box_raw\n return (x0, mb_height - y1, x1, mb_height - y0)\n\n\nclass Page(Container):\n cached_properties: List[str] = Container.cached_properties + [\"_layout\"]\n is_original: bool = True\n pages = None\n\n def __init__(\n self,\n pdf: \"PDF\",\n page_obj: PDFPage,\n page_number: int,\n initial_doctop: T_num = 0,\n ):\n self.pdf = pdf\n self.root_page = self\n self.page_obj = page_obj\n self.page_number = page_number\n self.initial_doctop = initial_doctop\n\n def get_attr(key: str, default: Any = None) -> Any:\n value = resolve_all(page_obj.attrs.get(key))\n return default if value is None else value\n\n # Per PDF Reference Table 3.27: \"The number of degrees by which the\n # page should be rotated clockwise when displayed or printed. The value\n # must be a multiple of 90. Default value: 0\"\n _rotation = get_attr(\"Rotate\", 0)\n self.rotation = _rotation % 360\n\n mb_raw = _normalize_box(get_attr(\"MediaBox\"), self.rotation)\n mb_height = mb_raw[3] - mb_raw[1]\n\n self.mediabox = _invert_box(mb_raw, mb_height)\n\n for box_name in [\"CropBox\", \"TrimBox\", \"BleedBox\", \"ArtBox\"]:\n if box_name in page_obj.attrs:\n box_normalized = _invert_box(\n _normalize_box(get_attr(box_name), self.rotation), mb_height\n )\n setattr(self, box_name.lower(), box_normalized)\n\n if \"CropBox\" not in page_obj.attrs:\n self.cropbox = self.mediabox\n\n # Page.bbox defaults to self.mediabox, but can be altered by Page.crop(...)\n self.bbox = self.mediabox\n\n # See https://rednafi.com/python/lru_cache_on_methods/\n self.get_textmap = lru_cache()(self._get_textmap)\n\n def close(self) -> None:\n self.flush_cache()\n self.get_textmap.cache_clear()\n\n @property\n def width(self) -> T_num:\n return self.bbox[2] - self.bbox[0]\n\n @property\n def height(self) -> T_num:\n return self.bbox[3] - self.bbox[1]\n\n @property\n def structure_tree(self) -> List[Dict[str, Any]]:\n \"\"\"Return the structure tree for a page, if any.\"\"\"\n try:\n return [elem.to_dict() for elem in PDFStructTree(self.pdf, self)]\n except StructTreeMissing:\n return []\n\n @property\n def layout(self) -> LTPage:\n if hasattr(self, \"_layout\"):\n return self._layout\n device = PDFPageAggregatorWithMarkedContent(\n self.pdf.rsrcmgr,\n pageno=self.page_number,\n laparams=self.pdf.laparams,\n )\n interpreter = PDFPageInterpreter(self.pdf.rsrcmgr, device)\n try:\n interpreter.process_page(self.page_obj)\n except Exception as e:\n raise PdfminerException(e)\n self._layout: LTPage = device.get_result()\n return self._layout\n\n @property\n def annots(self) -> T_obj_list:\n def rotate_point(pt: Tuple[float, float], r: int) -> Tuple[float, float]:\n turns = r // 90\n for i in range(turns):\n x, y = pt\n comp = self.width if i == turns % 2 else self.height\n pt = (y, (comp - x))\n return pt\n\n def parse(annot: T_obj) -> T_obj:\n _a, _b, _c, _d = annot[\"Rect\"]\n pt0 = rotate_point((_a, _b), self.rotation)\n pt1 = rotate_point((_c, _d), self.rotation)\n rh = self.root_page.height\n x0, top, x1, bottom = _invert_box(_normalize_box((*pt0, *pt1)), rh)\n\n a = annot.get(\"A\", {})\n extras = {\n \"uri\": a.get(\"URI\"),\n \"title\": annot.get(\"T\"),\n \"contents\": annot.get(\"Contents\"),\n }\n for k, v in extras.items():\n if v is not None:\n try:\n extras[k] = v.decode(\"utf-8\")\n except UnicodeDecodeError:\n try:\n extras[k] = v.decode(\"utf-16\")\n except UnicodeDecodeError:\n if self.pdf.raise_unicode_errors:\n raise\n warn(\n f\"Could not decode {k} of annotation.\"\n f\" {k} will be missing.\"\n )\n\n parsed = {\n \"page_number\": self.page_number,\n \"object_type\": \"annot\",\n \"x0\": x0,\n \"y0\": rh - bottom,\n \"x1\": x1,\n \"y1\": rh - top,\n \"doctop\": self.initial_doctop + top,\n \"top\": top,\n \"bottom\": bottom,\n \"width\": x1 - x0,\n \"height\": bottom - top,\n }\n parsed.update(extras)\n # Replace the indirect reference to the page dictionary\n # with a pointer to our actual page\n if \"P\" in annot:\n annot[\"P\"] = self\n parsed[\"data\"] = annot\n return parsed\n\n raw = resolve_all(self.page_obj.annots) or []\n parsed = list(map(parse, raw))\n if isinstance(self, CroppedPage):\n return self._crop_fn(parsed)\n else:\n return parsed\n\n @property\n def hyperlinks(self) -> T_obj_list:\n return [a for a in self.annots if a[\"uri\"] is not None]\n\n @property\n def objects(self) -> Dict[str, T_obj_list]:\n if hasattr(self, \"_objects\"):\n return self._objects\n self._objects: Dict[str, T_obj_list] = self.parse_objects()\n return self._objects\n\n def point2coord(self, pt: Tuple[T_num, T_num]) -> Tuple[T_num, T_num]:\n # See note below re. #1181 and mediabox-adjustment reversions\n return (self.mediabox[0] + pt[0], self.mediabox[1] + self.height - pt[1])\n\n def process_object(self, obj: LTItem) -> T_obj:\n kind = re.sub(lt_pat, \"\", obj.__class__.__name__).lower()\n\n def process_attr(item: Tuple[str, Any]) -> Optional[Tuple[str, Any]]:\n k, v = item\n if k in ALL_ATTRS:\n res = resolve_all(v)\n return (k, res)\n else:\n return None\n\n attr = dict(filter(None, map(process_attr, obj.__dict__.items())))\n\n attr[\"object_type\"] = kind\n attr[\"page_number\"] = self.page_number\n\n for cs in [\"ncs\", \"scs\"]:\n # Note: As of pdfminer.six v20221105, that library only\n # exposes ncs for LTChars, and neither attribute for\n # other objects. Keeping this code here, though,\n # for ease of addition if color spaces become\n # more available via pdfminer.six\n if hasattr(obj, cs):\n attr[cs] = resolve_and_decode(getattr(obj, cs).name)\n\n if isinstance(obj, (LTChar, LTTextContainer)):\n text = obj.get_text()\n attr[\"text\"] = (\n normalize_unicode(self.pdf.unicode_norm, text)\n if self.pdf.unicode_norm is not None\n else text\n )\n\n if isinstance(obj, LTChar):\n # pdfminer.six (at least as of v20221105) does not\n # directly expose .stroking_color and .non_stroking_color\n # for LTChar objects (unlike, e.g., LTRect objects).\n gs = obj.graphicstate\n attr[\"stroking_color\"] = (\n gs.scolor if isinstance(gs.scolor, tuple) else (gs.scolor,)\n )\n attr[\"non_stroking_color\"] = (\n gs.ncolor if isinstance(gs.ncolor, tuple) else (gs.ncolor,)\n )\n\n # Handle (rare) byte-encoded fontnames\n if isinstance(attr[\"fontname\"], bytes): # pragma: nocover\n attr[\"fontname\"] = fix_fontname_bytes(attr[\"fontname\"])\n\n elif isinstance(obj, (LTCurve,)):\n attr[\"pts\"] = list(map(self.point2coord, attr[\"pts\"]))\n\n # Ignoring typing because type signature for obj.original_path\n # appears to be incorrect\n attr[\"path\"] = [(cmd, *map(self.point2coord, pts)) for cmd, *pts in obj.original_path] # type: ignore # noqa: E501\n\n attr[\"dash\"] = obj.dashing_style\n\n # As noted in #1181, `pdfminer.six` adjusts objects'\n # coordinates relative to the MediaBox:\n # https://github.com/pdfminer/pdfminer.six/blob/1a8bd2f730295b31d6165e4d95fcb5a03793c978/pdfminer/converter.py#L79-L84\n mb_x0, mb_top = self.mediabox[:2]\n\n if \"y0\" in attr:\n attr[\"top\"] = (self.height - attr[\"y1\"]) + mb_top\n attr[\"bottom\"] = (self.height - attr[\"y0\"]) + mb_top\n attr[\"doctop\"] = self.initial_doctop + attr[\"top\"]\n\n if \"x0\" in attr and mb_x0 != 0:\n attr[\"x0\"] = attr[\"x0\"] + mb_x0\n attr[\"x1\"] = attr[\"x1\"] + mb_x0\n\n return attr\n\n def iter_layout_objects(\n self, layout_objects: List[LTComponent]\n ) -> Generator[T_obj, None, None]:\n for obj in layout_objects:\n # If object is, like LTFigure, a higher-level object ...\n if isinstance(obj, LTContainer):\n # and LAParams is passed, process the object itself.\n if self.pdf.laparams is not None:\n yield self.process_object(obj)\n # Regardless, iterate through its children\n yield from self.iter_layout_objects(obj._objs)\n else:\n yield self.process_object(obj)\n\n def parse_objects(self) -> Dict[str, T_obj_list]:\n objects: Dict[str, T_obj_list] = {}\n for obj in self.iter_layout_objects(self.layout._objs):\n kind = obj[\"object_type\"]\n if kind in [\"anno\"]:\n continue\n if objects.get(kind) is None:\n objects[kind] = []\n objects[kind].append(obj)\n return objects\n\n def debug_tablefinder(\n self, table_settings: Optional[T_table_settings] = None\n ) -> TableFinder:\n tset = TableSettings.resolve(table_settings)\n return TableFinder(self, tset)\n\n def find_tables(\n self, table_settings: Optional[T_table_settings] = None\n ) -> List[Table]:\n tset = TableSettings.resolve(table_settings)\n return TableFinder(self, tset).tables\n\n def find_table(\n self, table_settings: Optional[T_table_settings] = None\n ) -> Optional[Table]:\n tset = TableSettings.resolve(table_settings)\n tables = self.find_tables(tset)\n\n if len(tables) == 0:\n return None\n\n # Return the largest table, as measured by number of cells.\n def sorter(x: Table) -> Tuple[int, T_num, T_num]:\n return (-len(x.cells), x.bbox[1], x.bbox[0])\n\n largest = list(sorted(tables, key=sorter))[0]\n\n return largest\n\n def extract_tables(\n self, table_settings: Optional[T_table_settings] = None\n ) -> List[List[List[Optional[str]]]]:\n tset = TableSettings.resolve(table_settings)\n tables = self.find_tables(tset)\n return [table.extract(**(tset.text_settings or {})) for table in tables]\n\n def extract_table(\n self, table_settings: Optional[T_table_settings] = None\n ) -> Optional[List[List[Optional[str]]]]:\n tset = TableSettings.resolve(table_settings)\n table = self.find_table(tset)\n if table is None:\n return None\n else:\n return table.extract(**(tset.text_settings or {}))\n\n def _get_textmap(self, **kwargs: Any) -> TextMap:\n defaults: Dict[str, Any] = dict(\n layout_bbox=self.bbox,\n )\n if \"layout_width_chars\" not in kwargs:\n defaults.update({\"layout_width\": self.width})\n if \"layout_height_chars\" not in kwargs:\n defaults.update({\"layout_height\": self.height})\n full_kwargs: Dict[str, Any] = {**defaults, **kwargs}\n return utils.chars_to_textmap(self.chars, **full_kwargs)\n\n def search(\n self,\n pattern: Union[str, Pattern[str]],\n regex: bool = True,\n case: bool = True,\n main_group: int = 0,\n return_chars: bool = True,\n return_groups: bool = True,\n **kwargs: Any,\n ) -> List[Dict[str, Any]]:\n textmap = self.get_textmap(**tuplify_list_kwargs(kwargs))\n return textmap.search(\n pattern,\n regex=regex,\n case=case,\n main_group=main_group,\n return_chars=return_chars,\n return_groups=return_groups,\n )\n\n def extract_text(self, **kwargs: Any) -> str:\n return self.get_textmap(**tuplify_list_kwargs(kwargs)).as_string\n\n def extract_text_simple(self, **kwargs: Any) -> str:\n return utils.extract_text_simple(self.chars, **kwargs)\n\n def extract_words(self, **kwargs: Any) -> T_obj_list:\n return utils.extract_words(self.chars, **kwargs)\n\n def extract_text_lines(\n self, strip: bool = True, return_chars: bool = True, **kwargs: Any\n ) -> T_obj_list:\n return self.get_textmap(**tuplify_list_kwargs(kwargs)).extract_text_lines(\n strip=strip, return_chars=return_chars\n )\n\n def crop(\n self, bbox: T_bbox, relative: bool = False, strict: bool = True\n ) -> \"CroppedPage\":\n return CroppedPage(self, bbox, relative=relative, strict=strict)\n\n def within_bbox(\n self, bbox: T_bbox, relative: bool = False, strict: bool = True\n ) -> \"CroppedPage\":\n \"\"\"\n Same as .crop, except only includes objects fully within the bbox\n \"\"\"\n return CroppedPage(\n self, bbox, relative=relative, strict=strict, crop_fn=utils.within_bbox\n )\n\n def outside_bbox(\n self, bbox: T_bbox, relative: bool = False, strict: bool = True\n ) -> \"CroppedPage\":\n \"\"\"\n Same as .crop, except only includes objects fully within the bbox\n \"\"\"\n return CroppedPage(\n self, bbox, relative=relative, strict=strict, crop_fn=utils.outside_bbox\n )\n\n def filter(self, test_function: Callable[[T_obj], bool]) -> \"FilteredPage\":\n return FilteredPage(self, test_function)\n\n def dedupe_chars(self, **kwargs: Any) -> \"FilteredPage\":\n \"\"\"\n Removes duplicate chars — those sharing the same text and positioning\n (within `tolerance`) as other characters in the set. Adjust extra_args\n to be more/less restrictive with the properties checked.\n \"\"\"\n p = FilteredPage(self, lambda x: True)\n p._objects = {kind: objs for kind, objs in self.objects.items()}\n p._objects[\"char\"] = utils.dedupe_chars(self.chars, **kwargs)\n return p\n\n def to_image(\n self,\n resolution: Optional[Union[int, float]] = None,\n width: Optional[Union[int, float]] = None,\n height: Optional[Union[int, float]] = None,\n antialias: bool = False,\n force_mediabox: bool = False,\n ) -> \"PageImage\":\n \"\"\"\n You can pass a maximum of 1 of the following:\n - resolution: The desired number pixels per inch. Defaults to 72.\n - width: The desired image width in pixels.\n - height: The desired image width in pixels.\n \"\"\"\n from .display import DEFAULT_RESOLUTION, PageImage\n\n num_specs = sum(x is not None for x in [resolution, width, height])\n if num_specs > 1:\n raise ValueError(\n f\"Only one of these arguments can be provided: resolution, width, height. You provided {num_specs}\" # noqa: E501\n )\n elif width is not None:\n resolution = 72 * width / self.width\n elif height is not None:\n resolution = 72 * height / self.height\n\n return PageImage(\n self,\n resolution=resolution or DEFAULT_RESOLUTION,\n antialias=antialias,\n force_mediabox=force_mediabox,\n )\n\n def to_dict(self, object_types: Optional[List[str]] = None) -> Dict[str, Any]:\n if object_types is None:\n _object_types = list(self.objects.keys()) + [\"annot\"]\n else:\n _object_types = object_types\n d = {\n \"page_number\": self.page_number,\n \"initial_doctop\": self.initial_doctop,\n \"rotation\": self.rotation,\n \"cropbox\": self.cropbox,\n \"mediabox\": self.mediabox,\n \"bbox\": self.bbox,\n \"width\": self.width,\n \"height\": self.height,\n }\n for t in _object_types:\n d[t + \"s\"] = getattr(self, t + \"s\")\n return d\n\n def __repr__(self) -> str:\n return f\"