[
{
"path": ".flake8",
"content": "[flake8]\nignore = E501,W503\n"
},
{
"path": ".github/PULL_REQUEST_TEMPLATE.md",
"content": "*Issue #, if available:*\n\n*Description of changes:*\n\n\nBy submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.\n"
},
{
"path": ".github/workflows/test_pull_request.yml",
"content": "# Controls when the action will run. Triggers the workflow on push or pull request\n# events but only for the main branch and changes in folder src-python\nname: Test-Pull-Request\non:\n pull_request:\n types: [assigned, opened, synchronize, reopened]\n paths:\n - src-python\n\n workflow_dispatch:\n\n# Run the tests\njobs:\n build:\n runs-on: ubuntu-latest\n strategy:\n matrix:\n python-version: [\"3.8\", \"3.9\", \"3.10\", \"3.11\"]\n defaults:\n run:\n working-directory: ./src-python\n\n steps:\n # Checks out the repository\n - uses: actions/checkout@v3\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v4\n with:\n python-version: ${{ matrix.python-version }}\n\n # Install package locally\n - name: Install package\n run: pip install -e .\n\n # Install dev dependencies\n - name: Install dependencies\n run: pip install -r ./extras/dev.txt\n\n # Run tests\n - name: Test\n run: pytest\n"
},
{
"path": ".idea/amazon-textract-response-parser.iml",
"content": "\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n"
},
{
"path": ".idea/inspectionProfiles/profiles_settings.xml",
"content": "\n \n \n \n \n"
},
{
"path": ".idea/misc.xml",
"content": "\n\n \n"
},
{
"path": ".idea/modules.xml",
"content": "\n\n \n \n \n \n \n"
},
{
"path": ".idea/vcs.xml",
"content": "\n\n \n \n \n"
},
{
"path": "CODE_OF_CONDUCT.md",
"content": "## Code of Conduct\nThis project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).\nFor more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact\nopensource-codeofconduct@amazon.com with any additional questions or comments.\n"
},
{
"path": "CONTRIBUTING.md",
"content": "# Contributing Guidelines\n\nThank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional\ndocumentation, we greatly value feedback and contributions from our community.\n\nPlease read through this document before submitting any issues or pull requests to ensure we have all the necessary\ninformation to effectively respond to your bug report or contribution.\n\n\n## Reporting Bugs/Feature Requests\n\nWe welcome you to use the GitHub issue tracker to report bugs or suggest features.\n\nWhen filing an issue, please check [existing open](https://github.com/aws-samples/amazon-textract-response-parser/issues), or [recently closed](https://github.com/aws-samples/amazon-textract-response-parser/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already\nreported the issue. Please try to include as much information as you can. Details like these are incredibly useful:\n\n* A reproducible test case or series of steps\n* The version of our code being used\n* Any modifications you've made relevant to the bug\n* Anything unusual about your environment or deployment\n\n\n## Contributing via Pull Requests\nContributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:\n\n1. You are working against the latest source on the *master* branch.\n2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.\n3. You open an issue to discuss any significant work - we would hate for your time to be wasted.\n\nTo send us a pull request, please:\n\n1. Fork the repository.\n2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.\n3. Ensure local tests pass.\n4. Commit to your fork using clear commit messages.\n5. Send us a pull request, answering any default questions in the pull request interface.\n6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.\n\nGitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and\n[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).\n\n\n## Finding contributions to work on\nLooking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws-samples/amazon-textract-response-parser/labels/help%20wanted) issues is a great place to start.\n\n\n## Code of Conduct\nThis project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).\nFor more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact\nopensource-codeofconduct@amazon.com with any additional questions or comments.\n\n\n## Security issue notifications\nIf you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.\n\n\n## Licensing\n\nSee the [LICENSE](https://github.com/aws-samples/amazon-textract-response-parser/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.\n\nWe may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.\n"
},
{
"path": "LICENSE",
"content": "\n Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n"
},
{
"path": "README.md",
"content": "# Textract Response Parser\n\nYou can use Textract response parser library to easily parse JSON returned by Amazon Textract. The library parses JSON and provides programming language specific constructs to work with different parts of the document. [textractor](https://github.com/aws-samples/amazon-textract-textractor) is an example of a PoC batch processing tool that takes advantage of the Textract response parser library and generates output in multiple formats.\n\n## Python Usage\n\nFor documentation on usage see: [src-python/README.md](src-python/README.md)\n\n## JavaScript/TypeScript Usage\n\nFor documentation on usage see: [src-js/README.md](src-js/README.md)\n\n## C# Usage\n\n### Forms\n\n```csharp\ndocument.Pages.ForEach(page => {\n Console.WriteLine(\"Print Lines and Words:\");\n page.Lines.ForEach(line => {\n Console.WriteLine(\"{0}--{1}\", line.Text, line.Confidence);\n line.Words.ForEach(word => {\n Console.WriteLine(\"{0}--{1}\", word.Text, word.Confidence);\n });\n });\n Console.WriteLine(\"Print Fields:\");\n page.Form.Fields.ForEach(f => {\n Console.WriteLine(\"Field: Key: {0}, Value {1}\", f.Key, f.Value);\n });\n Console.WriteLine(\"Get Field by Key:\");\n var key = \"Phone Number:\";\n var field = page.Form.GetFieldByKey(key);\n if(field != null) {\n Console.WriteLine(\"Field: Key: {0}, Value: {1}\", field.Key, field.Value);\n }\n});\n```\n\n### Tables\n\n```csharp\ndocument.Pages.ForEach(page => {\n page.Tables.ForEach(table => {\n var r = 0;\n table.Rows.ForEach(row => {\n r++;\n var c = 0;\n row.Cells.ForEach(cell => {\n c++;\n Console.WriteLine(\"Table [{0}][{1}] = {2}--{3}\", r, c, cell.Text, cell.Confidence);\n });\n });\n });\n});\n```\n\nCheck out the `src-csharp` folder for instructions on how to run [.NET Core C#](src-csharp/readme.md) samples\n\n## Other Resources\n\n- [Large scale document processing with Amazon Textract - Reference Architecture](https://github.com/aws-samples/amazon-textract-serverless-large-scale-document-processing)\n- [Batch processing tool](https://github.com/aws-samples/amazon-textract-textractor)\n- [Code samples](https://github.com/aws-samples/amazon-textract-code-samples)\n\n## License Summary\n\nThis sample code is made available under the Apache License V2.0 license. See the LICENSE file.\n\n"
},
{
"path": "src-csharp/LICENSE",
"content": "Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of\nthis software and associated documentation files (the \"Software\"), to deal in\nthe Software without restriction, including without limitation the rights to\nuse, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of\nthe Software, and to permit persons to whom the Software is furnished to do so.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS\nFOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR\nCOPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER\nIN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN\nCONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."
},
{
"path": "src-csharp/Program.cs",
"content": "using System;\r\nusing Microsoft.Extensions.Configuration;\r\nusing Amazon.Textract;\r\nusing Amazon.Textract.Model;\r\nusing System.Collections.Generic;\r\nusing System.Threading.Tasks;\r\n\r\nnamespace parser {\r\n\tclass Program {\r\n\t\tconst string BucketName = \"\";\r\n\t\tconst string FormFile = \"employmentapp.png\";\r\n\t\tstatic void Main(string[] args) {\r\n\t\t\tvar textractAnalysisClient = BuildTextractClient();\r\n\t\t\tvar document = PrepareDocument(textractAnalysisClient, \"FORMS\");\r\n\t\t\tdocument.Pages.ForEach(page => {\r\n\t\t\t\tConsole.WriteLine(\"Print Lines and Words:\");\r\n\t\t\t\tpage.Lines.ForEach(line => {\r\n\t\t\t\t\tConsole.WriteLine(\"{0}--{1}\", line.Text, line.Confidence);\r\n\t\t\t\t\tline.Words.ForEach(word => {\r\n\t\t\t\t\t\tConsole.WriteLine(\"{0}--{1}\", word.Text, word.Confidence);\r\n\t\t\t\t\t});\r\n\t\t\t\t});\r\n\t\t\t\tConsole.WriteLine(\"Print Fields:\");\r\n\t\t\t\tpage.Form.Fields.ForEach(f => {\r\n\t\t\t\t\tConsole.WriteLine(\"Field: Key: {0}, Value {1}\", f.Key, f.Value);\r\n\t\t\t\t});\r\n\t\t\t\tConsole.WriteLine(\"Get Field by Key:\");\r\n\t\t\t\tvar key = \"Phone Number:\";\r\n\t\t\t\tvar field = page.Form.GetFieldByKey(key);\r\n\t\t\t\tif(field != null) {\r\n\t\t\t\t\tConsole.WriteLine(\"Field: Key: {0}, Value: {1}\", field.Key, field.Value);\r\n\t\t\t\t}\r\n\t\t\t});\r\n\r\n\t\t\tdocument = PrepareDocument(textractAnalysisClient, \"TABLES\");\r\n\t\t\tdocument.Pages.ForEach(page => {\r\n\t\t\t\tpage.Tables.ForEach(table => {\r\n\t\t\t\t\tvar r = 0;\r\n\t\t\t\t\ttable.Rows.ForEach(row => {\r\n\t\t\t\t\t\tr++;\r\n\t\t\t\t\t\tvar c = 0;\r\n\t\t\t\t\t\trow.Cells.ForEach(cell => {\r\n\t\t\t\t\t\t\tc++;\r\n\t\t\t\t\t\t\tConsole.WriteLine(\"Table [{0}][{1}] = {2}--{3}\", r, c, cell.Text, cell.Confidence);\r\n\t\t\t\t\t\t});\r\n\t\t\t\t\t});\r\n\t\t\t\t});\r\n\t\t\t});\r\n\t\t}\r\n\r\n\t\tstatic TextractDocument PrepareDocument(TextractTextAnalysisService textractAnalysisClient, string type) {\r\n\t\t\tvar task = textractAnalysisClient.StartDocumentAnalysis(BucketName, FormFile, type);\r\n\t\t\tvar jobId = task.Result;\r\n\t\t\ttextractAnalysisClient.WaitForJobCompletion(jobId);\r\n\t\t\tvar results = textractAnalysisClient.GetJobResults(jobId);\r\n\t\t\treturn new TextractDocument(results);\r\n\t\t}\r\n\r\n\t\tstatic TextractTextAnalysisService BuildTextractClient() {\r\n\t\t\tvar builder = new ConfigurationBuilder()\r\n\t\t\t\t.SetBasePath(Environment.CurrentDirectory)\r\n\t\t\t\t.AddJsonFile(\"appsettings.json\", optional: false, reloadOnChange: true)\r\n\t\t\t\t.AddEnvironmentVariables()\r\n\t\t\t\t.Build();\r\n\t\t\tvar awsOptions = builder.GetAWSOptions();\r\n\t\t\treturn new TextractTextAnalysisService(awsOptions.CreateServiceClient());\r\n\t\t}\r\n\t}\r\n\r\n\tpublic class TextractTextAnalysisService {\r\n\t\tprivate IAmazonTextract textract;\r\n\t\tpublic TextractTextAnalysisService(IAmazonTextract textract) {\r\n\t\t\tthis.textract = textract;\r\n\t\t}\r\n\t\tpublic GetDocumentAnalysisResponse GetJobResults(string jobId) {\r\n\t\t\tvar response = this.textract.GetDocumentAnalysisAsync(new GetDocumentAnalysisRequest {\r\n\t\t\t\tJobId = jobId\r\n\t\t\t});\r\n\t\t\tresponse.Wait();\r\n\t\t\treturn response.Result;\r\n\t\t}\r\n\r\n\t\tpublic bool IsJobComplete(string jobId) {\r\n\t\t\tvar response = this.textract.GetDocumentAnalysisAsync(new GetDocumentAnalysisRequest {\r\n\t\t\t\tJobId = jobId\r\n\t\t\t});\r\n\t\t\tresponse.Wait();\r\n\t\t\treturn !response.Result.JobStatus.Equals(\"IN_PROGRESS\");\r\n\t\t}\r\n\r\n\t\tpublic async Task StartDocumentAnalysis(string bucketName, string key, string featureType) {\r\n\t\t\tvar request = new StartDocumentAnalysisRequest();\r\n\t\t\tvar s3Object = new S3Object {\r\n\t\t\t\tBucket = bucketName,\r\n\t\t\t\tName = key\r\n\t\t\t};\r\n\t\t\trequest.DocumentLocation = new DocumentLocation {\r\n\t\t\t\tS3Object = s3Object\r\n\t\t\t};\r\n\t\t\trequest.FeatureTypes = new List { featureType };\r\n\t\t\tvar response = await this.textract.StartDocumentAnalysisAsync(request);\r\n\t\t\treturn response.JobId;\r\n\t\t}\r\n\r\n\t\tpublic void WaitForJobCompletion(string jobId, int delay = 5000) {\r\n\t\t\twhile(!IsJobComplete(jobId)) {\r\n\t\t\t\tthis.Wait(delay);\r\n\t\t\t}\r\n\t\t}\r\n\r\n\t\tprivate void Wait(int delay = 5000) {\r\n\t\t\tTask.Delay(delay).Wait();\r\n\t\t\tConsole.Write(\".\");\r\n\t\t}\r\n\t}\r\n}\r\n"
},
{
"path": "src-csharp/README.md",
"content": "# Usage\n\n## Forms\n\n```csharp\ndocument.Pages.ForEach(page => {\n Console.WriteLine(\"Print Lines and Words:\");\n page.Lines.ForEach(line => {\n Console.WriteLine(\"{0}--{1}\", line.Text, line.Confidence);\n line.Words.ForEach(word => {\n Console.WriteLine(\"{0}--{1}\", word.Text, word.Confidence);\n });\n });\n Console.WriteLine(\"Print Fields:\");\n page.Form.Fields.ForEach(f => {\n Console.WriteLine(\"Field: Key: {0}, Value {1}\", f.Key, f.Value);\n });\n Console.WriteLine(\"Get Field by Key:\");\n var key = \"Phone Number:\";\n var field = page.Form.GetFieldByKey(key);\n if(field != null) {\n Console.WriteLine(\"Field: Key: {0}, Value: {1}\", field.Key, field.Value);\n }\n});\n```\n\n## Tables\n\n```csharp\ndocument.Pages.ForEach(page => {\n page.Tables.ForEach(table => {\n var r = 0;\n table.Rows.ForEach(row => {\n r++;\n var c = 0;\n row.Cells.ForEach(cell => {\n c++;\n Console.WriteLine(\"Table [{0}][{1}] = {2}--{3}\", r, c, cell.Text, cell.Confidence);\n });\n });\n });\n});\n```\n\n# Test\n\n## Prerequisites\n\n- [Install](https://dotnet.microsoft.com/download) .NET Core\n- [Install](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html)\n and\n [Configure](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html)\n AWS CLI\n\nThen\n\n- Download source code to your local machine\n- Run the following at a command line inside the source code folder to execute\n\n```\ndotnet run\n```\n\n# Extra\n\nupload file to S3\n\n```\naws s3 cp test-files/employmentapp.png s3://\n```\n"
},
{
"path": "src-csharp/TextractExtensions.cs",
"content": "using System;\nusing System.Collections.Generic;\n\nnamespace Amazon.Textract.Model {\n\n\tpublic class Word {\n\t\tpublic Word(Block block, List blocks) {\n\t\t\tthis.Block = block;\n\t\t\tthis.Confidence = block.Confidence;\n\t\t\tthis.Geometry = block.Geometry;\n\t\t\tthis.Id = block.Id;\n\t\t\tthis.Text = block == null ? string.Empty : block.Text;\n\t\t}\n\n\t\tpublic Block Block { get; set; }\n\t\tpublic float Confidence { get; set; }\n\t\tpublic Geometry Geometry { get; set; }\n\t\tpublic string Id { get; set; }\n\t\tpublic string Text { get; set; }\n\n\t\tpublic override string ToString() {\n\t\t\treturn Text;\n\t\t}\n\t}\n\n\tpublic class TextractDocument {\n\t\tprivate List blockMap = new List();\n\t\tList> documentPages = new List>();\n\n\t\tpublic TextractDocument(GetDocumentAnalysisResponse response) {\n\t\t\tthis.Pages = new List();\n\t\t\tthis.ResponsePages = new List();\n\t\t\tthis.ResponsePages.Add(response);\n\n\t\t\tthis.ParseDocumentPagesAndBlockMap();\n\t\t\tthis.Parse();\n\t\t}\n\n\t\tprivate void ParseDocumentPagesAndBlockMap() {\n\t\t\tList documentPage = null;\n\t\t\tthis.ResponsePages.ForEach(page => {\n\t\t\t\tpage.Blocks.ForEach(block => {\n\t\t\t\t\tthis.blockMap.Add(block);\n\n\t\t\t\t\tif(block.BlockType == \"PAGE\") {\n\t\t\t\t\t\tif(documentPage != null) {\n\t\t\t\t\t\t\tthis.documentPages.Add(documentPage);\n\t\t\t\t\t\t}\n\t\t\t\t\t\tdocumentPage = new List();\n\t\t\t\t\t\tdocumentPage.Add(block);\n\t\t\t\t\t} else {\n\t\t\t\t\t\tdocumentPage.Add(block);\n\t\t\t\t\t}\n\t\t\t\t});\n\t\t\t});\n\n\t\t\tif(documentPage != null) {\n\t\t\t\tthis.documentPages.Add(documentPage);\n\t\t\t}\n\n\t\t}\n\n\t\tprivate void Parse() {\n\t\t\tthis.documentPages.ForEach(documentPage => {\n\t\t\t\tvar page = new Page(documentPage, this.blockMap);\n\t\t\t\tthis.Pages.Add(page);\n\t\t\t});\n\t\t}\n\n\t\tpublic Block GetBlockById(string blockId) {\n\t\t\treturn this.blockMap.Find(x => x.Id == blockId);\n\t\t}\n\n\t\tpublic List ResponsePages { get; set; }\n\t\tpublic List Pages { get; set; }\n\t\tpublic List> PageBlocks {\n\t\t\tget {\n\t\t\t\treturn this.documentPages;\n\t\t\t}\n\t\t}\n\t}\n\n\tpublic class Table {\n\t\tpublic Table(Block block, List blocks) {\n\t\t\tthis.Block = block;\n\t\t\tthis.Confidence = block.Confidence;\n\t\t\tthis.Geometry = block.Geometry;\n\t\t\tthis.Id = block.Id;\n\t\t\tthis.Rows = new List();\n\t\t\tvar ri = 1;\n\t\t\tvar row = new Row();\n\n\t\t\tvar relationships = block.Relationships;\n\t\t\tif(relationships != null && relationships.Count > 0) {\n\t\t\t\trelationships.ForEach(r => {\n\t\t\t\t\tif(r.Type == \"CHILD\") {\n\t\t\t\t\t\tr.Ids.ForEach(id => {\n\t\t\t\t\t\t\tvar cell = new Cell(blocks.Find(b => b.Id == id), blocks);\n\t\t\t\t\t\t\tif(cell.RowIndex > ri) {\n\t\t\t\t\t\t\t\tthis.Rows.Add(row);\n\t\t\t\t\t\t\t\trow = new Row();\n\t\t\t\t\t\t\t\tri = cell.RowIndex;\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\trow.Cells.Add(cell);\n\t\t\t\t\t\t});\n\t\t\t\t\t\tif(row != null && row.Cells.Count > 0)\n\t\t\t\t\t\t\tthis.Rows.Add(row);\n\t\t\t\t\t}\n\t\t\t\t});\n\t\t\t}\n\t\t}\n\t\tpublic List Rows { get; set; }\n\t\tpublic Block Block { get; set; }\n\t\tpublic float Confidence { get; set; }\n\t\tpublic Geometry Geometry { get; set; }\n\t\tpublic string Id { get; set; }\n\n\t\tpublic override string ToString() {\n\t\t\tvar result = new List();\n\t\t\tresult.Add(string.Format(\"Table{0}===={0}\", Environment.NewLine));\n\t\t\tthis.Rows.ForEach(r => {\n\t\t\t\tresult.Add(string.Format(\"Row{0}===={0}{1}{0}\", Environment.NewLine, r));\n\t\t\t});\n\t\t\treturn string.Join(\"\", result);\n\t\t}\n\t}\n\n\tpublic class SelectionElement {\n\t\tpublic SelectionElement(Block block, List blocks) {\n\t\t\tthis.Confidence = block.Confidence;\n\t\t\tthis.Geometry = block.Geometry;\n\t\t\tthis.Id = block.Id;\n\t\t\tthis.SelectionStatus = block.SelectionStatus;\n\t\t}\n\t\tpublic float Confidence { get; set; }\n\t\tpublic Geometry Geometry { get; set; }\n\t\tpublic string Id { get; set; }\n\t\tpublic string SelectionStatus { get; set; }\n\n\t}\n\n\tpublic class Row {\n\t\tpublic Row() {\n\t\t\tthis.Cells = new List();\n\t\t}\n\t\tpublic List Cells { get; set; }\n\n\t\tpublic override string ToString() {\n\t\t\tvar result = new List();\n\t\t\tthis.Cells.ForEach(c => {\n\t\t\t\tresult.Add(string.Format(\"[{0}]\", c));\n\t\t\t});\n\t\t\treturn string.Join(\"\", result);\n\t\t}\n\t}\n\n\tpublic class Page {\n\t\tpublic Page(List blocks, List blockMap) {\n\t\t\tthis.Blocks = blocks;\n\t\t\tthis.Text = string.Empty;\n\t\t\tthis.Lines = new List();\n\t\t\tthis.Form = new Form();\n\t\t\tthis.Tables = new List
();\n\t\t\tthis.Content = new List();\n\n\t\t\tblocks.ForEach(b => {\n\t\t\t\tif(b.BlockType == \"PAGE\") {\n\t\t\t\t\tthis.Geometry = new NewGeometry(b.Geometry);\n\t\t\t\t\tthis.Id = b.Id;\n\t\t\t\t} else if(b.BlockType == \"LINE\") {\n\t\t\t\t\tvar l = new Line(b, blockMap);\n\t\t\t\t\tthis.Lines.Add(l);\n\t\t\t\t\tthis.Content.Add(l);\n\t\t\t\t\tthis.Text = this.Text + l.Text + Environment.NewLine;\n\t\t\t\t} else if(b.BlockType == \"TABLE\") {\n\t\t\t\t\tvar t = new Table(b, blockMap);\n\t\t\t\t\tthis.Tables.Add(t);\n\t\t\t\t\tthis.Content.Add(t);\n\t\t\t\t} else if(b.BlockType == \"KEY_VALUE_SET\") {\n\t\t\t\t\tif(b.EntityTypes.Contains(\"KEY\")) {\n\t\t\t\t\t\tvar f = new Field(b, blockMap);\n\t\t\t\t\t\tif(f.Key != null) {\n\t\t\t\t\t\t\tthis.Form.AddField(f);\n\t\t\t\t\t\t\tthis.Content.Add(f);\n\t\t\t\t\t\t}\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t});\n\n\t\t}\n\n\t\tpublic List GetLinesInReadingOrder() {\n\t\t\tvar lines = new List();\n\t\t\tvar columns = new List();\n\t\t\tthis.Lines.ForEach(line => {\n\t\t\t\tvar columnFound = false;\n\t\t\t\tfor(var index = 0; index < columns.Count; index++) {\n\t\t\t\t\tvar column = columns[index];\n\t\t\t\t\tvar bb = line.Geometry.BoundingBox;\n\t\t\t\t\tvar bbLeft = bb.Left;\n\t\t\t\t\tvar bbRight = bb.Left + bb.Width;\n\t\t\t\t\tvar bbCentre = bb.Left + (bb.Width / 2);\n\t\t\t\t\tvar columnCentre = column.Left + (column.Right / 2);\n\n\t\t\t\t\tif((bbCentre > column.Left && bbCentre < column.Right) || (columnCentre > bbLeft && columnCentre < bbRight)) {\n\t\t\t\t\t\tlines.Add(new IndexedText { ColumnIndex = index, Text = line.Text });\n\t\t\t\t\t\tcolumnFound = true;\n\t\t\t\t\t\tbreak;\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\tif(!columnFound) {\n\t\t\t\t\tvar bb = line.Geometry.BoundingBox;\n\t\t\t\t\tcolumns.Add(new Column { Left = bb.Left, Right = bb.Left + bb.Width });\n\t\t\t\t\tlines.Add(new IndexedText { ColumnIndex = columns.Count - 1, Text = line.Text });\n\t\t\t\t}\n\t\t\t});\n\t\t\tlines.FindAll(line => line.ColumnIndex == 0).ForEach(line => Console.WriteLine(line));\n\t\t\treturn lines;\n\t\t}\n\n\t\tpublic string GetTextInReadingOrder() {\n\t\t\tvar lines = this.GetLinesInReadingOrder();\n\t\t\tvar text = string.Empty;\n\t\t\tlines.ForEach(line => {\n\t\t\t\ttext = text + line.Text + \"\\n\";\n\t\t\t});\n\t\t\treturn text;\n\t\t}\n\n\n\t\tpublic List Blocks { get; set; }\n\t\tpublic string Text { get; set; }\n\t\tpublic List Lines { get; set; }\n\t\tpublic Form Form { get; set; }\n\t\tpublic List
Tables { get; set; }\n\t\tpublic List Content { get; set; }\n\t\tpublic Geometry Geometry { get; set; }\n\t\tpublic string Id { get; set; }\n\n\t\tpublic override string ToString() {\n\t\t\tvar result = new List();\n\t\t\tresult.Add(string.Format(\"Page{0}===={0}\", Environment.NewLine));\n\t\t\tthis.Content.ForEach(c => {\n\t\t\t\tresult.Add(string.Format(\"{1}{0}\", Environment.NewLine, c));\n\t\t\t});\n\t\t\treturn string.Join(\"\", result);\n\t\t}\n\n\t\tpublic class Column {\n\t\t\tpublic float Left { get; set; }\n\t\t\tpublic float Right { get; set; }\n\n\t\t\tpublic override string ToString() {\n\t\t\t\treturn string.Format(\"Left: {0}, Right :{1}\", this.Left, this.Right);\n\t\t\t}\n\t\t}\n\n\t\tpublic class IndexedText {\n\t\t\tpublic int ColumnIndex { get; set; }\n\t\t\tpublic string Text { get; set; }\n\n\t\t\tpublic override string ToString() {\n\t\t\t\treturn string.Format(\"[{0}] {1}\", this.ColumnIndex, this.Text);\n\t\t\t}\n\t\t}\n\t}\n\n\tpublic class NewGeometry : Geometry {\n\n\t\tpublic NewGeometry(Geometry geometry) : base() {\n\t\t\tthis.BoundingBox = geometry.BoundingBox;\n\t\t\tthis.Polygon = geometry.Polygon;\n\t\t\tvar bb = new NewBoundingBox(this.BoundingBox.Width, this.BoundingBox.Height, this.BoundingBox.Left, this.BoundingBox.Top);\n\t\t\tvar pgs = new List();\n\t\t\tPolygon.ForEach(pg => pgs.Add(new Point {\n\t\t\t\tX = pg.X,\n\t\t\t\tY = pg.Y\n\t\t\t}));\n\n\t\t\tBoundingBox = bb;\n\t\t\tPolygon = pgs;\n\t\t}\n\n\t\tpublic override string ToString() {\n\t\t\treturn string.Format(\"BoundingBox: {0}{1}\", BoundingBox, Environment.NewLine);\n\t\t}\n\n\n\t}\n\n\tpublic class NewBoundingBox : BoundingBox {\n\t\tpublic NewBoundingBox(float width, float height, float left, float top) : base() {\n\t\t\tthis.Width = width;\n\t\t\tthis.Height = height;\n\t\t\tthis.Left = left;\n\t\t\tthis.Top = top;\n\t\t}\n\n\t\tpublic override string ToString() {\n\t\t\treturn string.Format(\"width: {0}, height: {1}, left: {2}, top: {3}\", Width, Height, Left, Top);\n\t\t}\n\t}\n\n\tpublic class Line {\n\t\tpublic Line(Block block, List blocks) {\n\t\t\tthis.Block = block;\n\t\t\tthis.Confidence = block.Confidence;\n\t\t\tthis.Geometry = block.Geometry;\n\t\t\tthis.Id = block.Id;\n\t\t\tthis.Text = block == null ? string.Empty : block.Text;\n\t\t\tthis.Words = new List();\n\n\t\t\tvar relationships = block.Relationships;\n\t\t\tif(relationships != null && relationships.Count > 0) {\n\t\t\t\trelationships.ForEach(r => {\n\t\t\t\t\tif(r.Type == \"CHILD\") {\n\t\t\t\t\t\tr.Ids.ForEach(id => {\n\t\t\t\t\t\t\tthis.Words.Add(new Word(blocks.Find(b => b.BlockType == \"WORD\" && b.Id == id), blocks));\n\t\t\t\t\t\t});\n\t\t\t\t\t}\n\t\t\t\t});\n\t\t\t}\n\t\t}\n\n\t\tpublic float Confidence { get; set; }\n\t\tpublic Geometry Geometry { get; set; }\n\t\tpublic string Id { get; set; }\n\t\tpublic List Words { get; set; }\n\t\tpublic string Text { get; set; }\n\t\tpublic Block Block { get; set; }\n\n\t\tpublic override string ToString() {\n\t\t\treturn string.Format(@\"\n Line{0}===={0}\n {1} {0}\n Words{0}----{0}\n {2}{0}\n ----\n \", Environment.NewLine, this.Text, string.Join(\", \", this.Words));\n\t\t}\n\t}\n\n\tpublic class Form {\n\t\tpublic List Fields { get; set; }\n\t\tprivate Dictionary fieldMap;\n\n\t\tpublic Form() {\n\t\t\tthis.Fields = new List();\n\t\t\tthis.fieldMap = new Dictionary();\n\t\t}\n\n\t\tpublic void AddField(Field field) {\n\t\t\tthis.Fields.Add(field);\n\t\t\tthis.fieldMap.Add(field.Key.ToString(), field);\n\t\t}\n\t\tpublic Field GetFieldByKey(string key) {\n\t\t\treturn this.fieldMap.GetValueOrDefault(key);\n\t\t}\n\n\t\tpublic List SearchFieldsByKey(string key) {\n\t\t\treturn this.Fields.FindAll(f => f.Key.ToString().ToLower().Contains(key.ToLower()));\n\t\t}\n\n\t\tpublic override string ToString() {\n\t\t\treturn string.Join(\"\\n\", this.Fields);\n\t\t}\n\t}\n\n\tpublic class FieldValue {\n\t\tpublic FieldValue(Block block, List children, List blocks) {\n\t\t\tthis.Block = block;\n\t\t\tthis.Confidence = block.Confidence;\n\t\t\tthis.Geometry = block.Geometry;\n\t\t\tthis.Id = block.Id;\n\t\t\tthis.Text = string.Empty;\n\t\t\tthis.Content = new List();\n\n\t\t\tvar words = new List();\n\t\t\tif(children != null && children.Count > 0) {\n\t\t\t\tchildren.ForEach(c => {\n\t\t\t\t\tvar wordBlock = blocks.Find(b => b.Id == c);\n\t\t\t\t\tif(wordBlock.BlockType == \"WORD\") {\n\t\t\t\t\t\tvar w = new Word(wordBlock, blocks);\n\t\t\t\t\t\tthis.Content.Add(w);\n\t\t\t\t\t\twords.Add(w.Text);\n\t\t\t\t\t} else if(wordBlock.BlockType == \"SELECTION_ELEMENT\") {\n\t\t\t\t\t\tvar selection = new SelectionElement(wordBlock, blocks);\n\t\t\t\t\t\tthis.Content.Add(selection);\n\t\t\t\t\t\twords.Add(selection.SelectionStatus);\n\t\t\t\t\t}\n\t\t\t\t});\n\t\t\t}\n\n\t\t\tif(words.Count > 0) {\n\t\t\t\tthis.Text = string.Join(\" \", words);\n\t\t\t}\n\t\t}\n\t\tpublic List Content { get; set; }\n\t\tpublic Block Block { get; set; }\n\t\tpublic float Confidence { get; set; }\n\t\tpublic Geometry Geometry { get; set; }\n\t\tpublic string Id { get; set; }\n\t\tpublic string Text { get; set; }\n\n\t\tpublic override string ToString() {\n\t\t\treturn Text;\n\t\t}\n\t}\n\n\tpublic class FieldKey {\n\t\tpublic FieldKey(Block block, List children, List blocks) {\n\t\t\tthis.Block = block;\n\t\t\tthis.Confidence = block.Confidence;\n\t\t\tthis.Geometry = block.Geometry;\n\t\t\tthis.Id = block.Id;\n\t\t\tthis.Text = string.Empty;\n\t\t\tthis.Content = new List();\n\n\t\t\tvar words = new List();\n\n\t\t\tif(children != null && children.Count > 0) {\n\t\t\t\tchildren.ForEach(c => {\n\t\t\t\t\tvar wordBlock = blocks.Find(b => b.Id == c);\n\t\t\t\t\tif(wordBlock.BlockType == \"WORD\") {\n\t\t\t\t\t\tvar w = new Word(wordBlock, blocks);\n\t\t\t\t\t\tthis.Content.Add(w);\n\t\t\t\t\t\twords.Add(w.Text);\n\t\t\t\t\t}\n\t\t\t\t});\n\t\t\t}\n\n\t\t\tif(words.Count > 0) {\n\t\t\t\tthis.Text = string.Join(\" \", words);\n\t\t\t}\n\n\t\t}\n\t\tpublic List Content { get; set; }\n\t\tpublic Block Block { get; set; }\n\t\tpublic float Confidence { get; set; }\n\t\tpublic Geometry Geometry { get; set; }\n\t\tpublic string Id { get; set; }\n\t\tpublic string Text { get; set; }\n\n\t\tpublic override string ToString() {\n\t\t\treturn Text;\n\t\t}\n\t}\n\n\tpublic class Field {\n\t\tpublic Field(Block block, List blocks) {\n\t\t\tvar relationships = block.Relationships;\n\t\t\tif(relationships != null && relationships.Count > 0) {\n\t\t\t\trelationships.ForEach(r => {\n\t\t\t\t\tif(r.Type == \"CHILD\") {\n\t\t\t\t\t\tthis.Key = new FieldKey(block, r.Ids, blocks);\n\t\t\t\t\t} else if(r.Type == \"VALUE\") {\n\t\t\t\t\t\tr.Ids.ForEach(id => {\n\t\t\t\t\t\t\tvar v = blocks.Find(b => b.Id == id);\n\t\t\t\t\t\t\tif(v.EntityTypes.Contains(\"VALUE\")) {\n\t\t\t\t\t\t\t\tvar vr = v.Relationships;\n\t\t\t\t\t\t\t\tif(vr != null && vr.Count > 0) {\n\t\t\t\t\t\t\t\t\tvr.ForEach(vc => {\n\t\t\t\t\t\t\t\t\t\tif(vc.Type == \"CHILD\") {\n\t\t\t\t\t\t\t\t\t\t\tthis.Value = new FieldValue(v, vc.Ids, blocks);\n\t\t\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t\t\t});\n\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t});\n\t\t\t\t\t}\n\t\t\t\t});\n\t\t\t}\n\t\t}\n\t\tpublic FieldKey Key { get; set; }\n\t\tpublic FieldValue Value { get; set; }\n\n\t\tpublic override string ToString() {\n\t\t\tvar k = this.Key == null ? string.Empty : this.Key.ToString();\n\t\t\tvar v = this.Value == null ? string.Empty : this.Value.ToString();\n\t\t\treturn string.Format(@\"\n {0}Field{0}===={0}\n Key: {1}, Value: {2}\n \", Environment.NewLine, k, v);\n\t\t}\n\t}\n\n\tpublic class Cell {\n\t\tpublic Cell(Block block, List blocks) {\n\t\t\tthis.Block = block;\n\t\t\tthis.ColumnIndex = block.ColumnIndex;\n\t\t\tthis.ColumnSpan = block.ColumnSpan;\n\t\t\tthis.Confidence = block.Confidence;\n\t\t\tthis.Content = new List();\n\t\t\tthis.Geometry = block.Geometry;\n\t\t\tthis.Id = block.Id;\n\t\t\tthis.RowIndex = block.RowIndex;\n\t\t\tthis.RowSpan = block.RowSpan;\n\t\t\tthis.Text = string.Empty;\n\n\t\t\tvar relationships = block.Relationships;\n\t\t\tif(relationships != null && relationships.Count > 0) {\n\t\t\t\trelationships.ForEach(r => {\n\t\t\t\t\tif(r.Type == \"CHILD\") {\n\t\t\t\t\t\tr.Ids.ForEach(id => {\n\t\t\t\t\t\t\tvar rb = blocks.Find(b => b.Id == id);\n\t\t\t\t\t\t\tif(rb.BlockType == \"WORD\") {\n\t\t\t\t\t\t\t\tvar w = new Word(rb, blocks);\n\t\t\t\t\t\t\t\tthis.Content.Add(w);\n\t\t\t\t\t\t\t\tthis.Text = this.Text + w.Text + \" \";\n\t\t\t\t\t\t\t} else if(rb.BlockType == \"SELECTION_ELEMENT\") {\n\t\t\t\t\t\t\t\tvar se = new SelectionElement(rb, blocks);\n\t\t\t\t\t\t\t\tthis.Content.Add(se);\n\t\t\t\t\t\t\t\tthis.Text = this.Text + se.SelectionStatus + \", \";\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t});\n\t\t\t\t\t}\n\n\t\t\t\t});\n\t\t\t}\n\t\t}\n\t\tpublic int RowIndex { get; set; }\n\t\tpublic int RowSpan { get; set; }\n\t\tpublic int ColumnIndex { get; set; }\n\t\tpublic int ColumnSpan { get; set; }\n\t\tpublic List Content { get; set; }\n\t\tpublic Block Block { get; set; }\n\t\tpublic float Confidence { get; set; }\n\t\tpublic Geometry Geometry { get; set; }\n\t\tpublic string Id { get; set; }\n\t\tpublic string Text { get; set; }\n\n\t\tpublic override string ToString() {\n\t\t\treturn this.Text;\n\t\t}\n\t}\n}"
},
{
"path": "src-csharp/appsettings.json",
"content": "{\n \"AWS\": {\n \"Profile\": \"default\",\n \"Region\": \"us-west-2\"\n }\n}\n"
},
{
"path": "src-csharp/parser.csproj",
"content": "\r\n\r\n \r\n Exe\r\n netcoreapp2.2\r\n \r\n\r\n\r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n \r\n\r\n \r\n \r\n Always\r\n \r\n \r\n \r\n\r\n\r\n"
},
{
"path": "src-js/.eslintrc.js",
"content": "module.exports = {\n parser: \"@typescript-eslint/parser\", // Specifies the ESLint parser\n parserOptions: {\n ecmaVersion: 2020, // Allows for the parsing of modern ECMAScript features\n sourceType: \"module\" // Allows for the use of imports\n },\n plugins: [\"@typescript-eslint\", \"prettier\"],\n extends: [\n // \"plugin:@typescript-eslint/recommended\", // Uses the recommended rules from the @typescript-eslint/eslint-plugin\n // \"prettier/@typescript-eslint\", // Uses eslint-config-prettier to disable ESLint rules from @typescript-eslint/eslint-plugin that would conflict with prettier\n // \"plugin:prettier/recommended\", // Enables eslint-plugin-prettier and eslint-config-prettier. This will display prettier errors as ESLint errors. Make sure this is always the last configuration in the extends array.\n \"eslint:recommended\",\n \"plugin:@typescript-eslint/eslint-recommended\",\n \"plugin:@typescript-eslint/recommended\",\n \"prettier\",\n ],\n rules: {\n // Place to specify ESLint rules. Can be used to overwrite rules specified from the extended config\n // e.g. \"@typescript-eslint/explicit-function-return-type\": \"off\",\n \"prettier/prettier\": 2,\n },\n};\n"
},
{
"path": "src-js/.nvmrc",
"content": "lts/hydrogen\n"
},
{
"path": "src-js/.prettierrc.js",
"content": "module.exports = {\n printWidth: 110,\n};\n"
},
{
"path": "src-js/CHANGELOG.md",
"content": "# Changelog\n\n## 0.4.3 (2024-11-19)\n### Changed\n- Bumped dev dependencies (including `cross-spawn`, `lint-staged`, `rollup`) for dependabot/audit\n### Fixed\n- `.html()` no longer fails on empty pages with no content (proposed fix for [this AWS re:Post question](https://repost.aws/questions/QU68wHh5vLSkiDC9Vt4lXXsw))\n\n## 0.4.2 (2024-06-28)\n### Added\n- Filter content by block type in a variety of contexts, with `includeBlockTypes` (allow-list) and `skipBlockTypes` (deny-list) options. These filters are available in the core `iter/listContent()`, `Layout.iter/listItems()` and `LayoutItem.iter/listLayoutChildren()` accessors, but can also be used to hide certain content (like page headers and footers) when you render with `.html({...})`. ([#179](https://github.com/aws-samples/amazon-textract-response-parser/issues/179))\n- Low-level relationship traversal via `iter/listRelatedItemsByRelType()` is now supported from `Page`s (PAGE blocks)\n- New accessor on `SelectionElement.isSelected`, in convenient boolean format (versus the 2-member `.selectionStatus` enumeration)\n- Form `Field.isCheckbox` and `FieldValue.isCheckbox`, check if a K->V field corresponds to a (label)->(checkbox) pair. Also added `{Field/FieldValue}.isSelected` and `.selectionStatus`, which return `null` for non-'checkbox' fields. (Pre-work for [#183](https://github.com/aws-samples/amazon-textract-response-parser/issues/183))\n### Changed\n- `WithContent` mixin options refactored to more closely mirror `IBlockTypeFilterOpts`, because WithContent now aligns to `iter/listRelatedItemsByRelType()` under the hood. This will give us more fine-grained but standardised control of missing and unexpected non-content child block type handling, per item class... But means some warning/error behaviour when parsing Textract JSON might have shifted a little (hopefully for the better).\n- A page's `Layout` no longer keeps any internal list-of-items state, instead referring to the parent `PAGE` block's child relationships directly.\n\n## 0.4.1 (2024-06-04)\n### Added\n- `iter/listRelatedItemsByRelType()` utility methods on all host-linked block wrapper objects, as most common use-cases for `relatedBlockIdsByRelType()` were just to then fetch the parsed wrapper for the retrieved block ID. Hope to further standardise across `childBlockIds`, `relatedBlockIdsByRelType`, and these new methods in a future release - but this might require some breaking changes to drive consistency in the handling of invalid JSONs (with missing block IDs, etc).\n- `iter/listLayoutChildren()` utility methods to generically traverse (nested?) child layout elements. We support generic & recursive access, but today the only known nesting is LAYOUT_LIST->LAYOUT_TEXT.\n### Fixed\n- `html()`, `str()` and `text` representations of page `Layout` no longer duplicate the content of `LAYOUT_TEXT` children under `LAYOUT_LIST` obects. ([#177](https://github.com/aws-samples/amazon-textract-response-parser/issues/177))\n### Deprecated\n- Page `Layout.nItems` is ambiguous: Prefer `.nItemsTotal` for previous behaviour (counting all direct and indirect children) or `.nItemsDirect` to count only top-level layout items, excluding those referenced as children by others.\n\n## 0.4.0 (2024-02-06)\n### Added\n- Load and navigate [Amazon Textract Layout analysis](https://aws.amazon.com/blogs/machine-learning/amazon-textracts-new-layout-feature-introduces-efficiencies-in-general-purpose-and-generative-ai-document-processing-tasks/) data. ([#164](https://github.com/aws-samples/amazon-textract-response-parser/issues/164))\n- Serialize individual elements, pages and documents to semantic markup with `.html()` (for page and document level, currently depends on `Layout` being enabled).\n- Proper support for [table title and footer elements](https://aws.amazon.com/blogs/machine-learning/announcing-enhanced-table-extractions-with-amazon-textract/) (`TABLE_TITLE` and `TABLE_FOOTER`) linked from tables. ([#171](https://github.com/aws-samples/amazon-textract-response-parser/issues/171))\n- Support [signature detection results](https://aws.amazon.com/blogs/machine-learning/detect-signatures-on-documents-or-images-using-the-signatures-feature-in-amazon-textract/) (`SIGNATURE` blocks)\n- More complete exposure of Textract API model constructs and `base.ts` utility functions in external-facing TRP API\n### Changed\n- **(BREAKING)** Previously-exposed `CellBase` class is removed, due to refactoring `Cell` and `MergedCell` to depend more on composable mixins and less on fragile hierarchy of (now internal) `CellBaseGeneric`. Use `Cell | MergedCell` instead for typing.\n- `Page`s now explicitly track parsed objects in their scope by block ID, which reduced state tracking requirements for other objects (like `Line`, `Query`) as we work toward supporting more edit/mutation operations. See `IBlockManager.registerParsedItem()` and `.getItemByBlockId()` for details. This may result in some **minor warning & error behavior changes** when handling invalid or incomplete Textract JSON.\n- Split out `api-models/document` types to better align with library components, and made some minor typing updates.\n### Fixed\n- `Table.nCells` now correctly reflects merged cells (instead of just counting all sub-cells).\n- Support alternative `KEY` and `VALUE` blocks for Forms K-V data, observed in place of the typical `KEY_VALUE_SET` blocks for some test data files (Was this a temporary API issue? A change going forward? 🤷♂️)\n### Deprecated\n- `ApiBlockWrapper` base class is now slated to become internal-only: Please let us know if you have use-cases\n- Various re-exports from `/api-modules/document` sub-module: Prefer importing direct from top-level\n- `ApiAsyncJobOuputInProgress` typo superseded by `ApiAsyncJobOutputInProgress`, but original not yet fully removed\n\n## 0.3.1 (2023-08-28)\n### Fixed\n- Suppress \"content may be truncated\" warnings when API `NextToken` is present but `null` ([#154](https://github.com/aws-samples/amazon-textract-response-parser/issues/154))\n- Fix typed `TABLE_FOOTER` and `TABLE_SECTION_HEADER` EntityType values to match the [API doc](https://docs.aws.amazon.com/textract/latest/dg/API_Block.html) ([#158](https://github.com/aws-samples/amazon-textract-response-parser/issues/158))\n\n## 0.3.0 (2023-07-31)\n### Added\n- **(BREAKING)** `ignoreMerged` and `repeatMultiRowCells` options on `Table` methods are now wrapped into `opts` objects for better future extensibility and clearer user code.\n- Expose the `ignoreMerged` option through `Table.rowAt()`, `Table.iterRows()`, and `Table.listRows()`, to enable navigating table rows ignoring merged cells.\n- Page-level access to [Amazon Textract Queries](https://docs.aws.amazon.com/textract/latest/dg/queryresponse.html) results. (Still assessing compositing architecture for a unified document-level view in future) ([#80](https://github.com/aws-samples/amazon-textract-response-parser/issues/80))\n- Average OCR (text recognition) confidence is now available on form fields (and their keys and values) as well as tables, table rows, and table cells - via `getOcrConfidence()`, with configurable aggregation method (including minimum, mean, etc.).\n- `EntityTypes` for tables and table cells/merged-cells are now accessible through `Table.tableType` property and `Cell.hasEntityTypes()` function - and also added to the underlying API data types. ([#78](https://github.com/aws-samples/amazon-textract-response-parser/issues/78))\n### Changed\n- **(BREAKING)** UMD module output `dist/umd` removed, following deprecation at v0.2.0 and no requests from users to restore it.\n### Fixed\n- Corrected wrongly typed `ApiCellBlock.Relationships` from an array of `ApiChildRelationship` to an optional array of same: This field may be omitted altogether when a cell is detected but has no content.\n- Corrected wrongly typed `ApiKeyValueSetBlock.EntityTypes` data model from `ApiKeyValueEntityType` to an array of same.\n\n## 0.2.2 (2023-06-19)\n### Fixed\n- Removed `browser` field from package.json because front end bundlers like webpack use it, and the (IIFE `dist/browser`) build it pointed to was not appropriate for these build systems. Added `jsdelivr` field in its place to help ensure direct-to-browser CDN imports continue to consume the IIFE build by default. ([Issue #139](https://github.com/aws-samples/amazon-textract-response-parser/issues/139))\n\n## 0.2.1 (2023-05-22)\n### Fixed\n- `.geometry` on Expense result fields is now optional, as the underlying field may not be returned by Amazon Textract in some cases. Typings updated to reflect the fix. ([Issue #102](https://github.com/aws-samples/amazon-textract-response-parser/issues/102))\n\n## 0.2.0 (2022-04-28)\n### Added\n- Initial support for Amazon Textract [identity document APIs](https://docs.aws.amazon.com/textract/latest/dg/how-it-works-identity.html).\n- Document-level Form field access and querying via `TextractDocument.form` in addition to `Page.form`.\n- `Page.pageNumber` to find and return 1-based index of the current page in the parent document.\n- New ES (esnext) module output in `dist/es` and `module` hint in package.json to encourage compatible tools to use this output.\n### Changed\n- Use CommonJS `dist/cjs` as default NPM module format instead of previous UMD `dist/umd`.\n- Separate type declarations into `dist/types` to reduce duplication and build size.\n- Use new [merged table cells](https://aws.amazon.com/about-aws/whats-new/2022/03/amazon-textract-updates-tables-check-detection/) feature by default, rather than classic split cells.\n- Eliminate trailing whitespace previously automatically added to Cell.text\n### Deprecated\n- UMD module output `dist/umd` slated to be removed in a future version: Please let us know if the other format options don't work for you!\n\n## 0.1.2 (2021-12-16)\n### Added\n- Header and footer segmentation utility (by text `LINE`)\n### Changed\n- Significantly improved `inReadingOrder` results for multi-column documents.\n"
},
{
"path": "src-js/README.md",
"content": "# Textract Response Parser for JavaScript/TypeScript\n\nThis library loads [Amazon Textract](https://docs.aws.amazon.com/textract/latest/dg/what-is.html) API response JSONs into structured classes with helper methods, for easier post-processing.\n\nIt's designed to work in both NodeJS and browser environments, and to support projects in either JavaScript or TypeScript.\n\n> ⚠️ **Warning:** If you're migrating from another TRP implementation such as the [Textract Response Parser for Python](https://github.com/aws-samples/amazon-textract-response-parser/tree/master/src-python), please note that the APIs and available features may be substantially different. Please let us know if there's a feature you're missing!\n\n\n## Installation\n\nYou can use TRP in your JavaScript or TypeScript NPM projects:\n\n```sh\n$ npm install amazon-textract-response-parser\n```\n\n```js\n// With CommonJS-style require:\nconst { TextractDocument, TextractIdentity } = require(\"amazon-textract-response-parser\");\n// Or ES-style module imports:\nimport { TextractDocument, TextractExpense } from \"amazon-textract-response-parser\";\n```\n\n...Or link directly in the browser - for example via a CDN like [unpkg](https://unpkg.com/):\n\n```html\n\n\n\n```\n\nTo enable this, the distribution of this library provides multiple builds:\n\n- `dist/cjs` (default `main`), for CommonJS environments like NodeJS - including most front end applications built with tools like React and Webpack.\n- `dist/es` (default `module`), for ES6/ES2015/esnext capable environments.\n- `dist/browser` (default `jsdelivr` and `unpkg`), for linking directly from browser HTML with no module framework (IIFE).\n\nThis means that **deep imports** will depend on your build environment, but are generally discouraged anyway and may not work correctly with TypeScript. Check out the [examples/](examples/README.md) folder on GitHub for some basic starters using the different styles.\n\n\n## Loading data\n\nInitialize a `TextractDocument` (or `TextractExpense`, `TextractIdentity`) by providing the parsed response JSON object from the corresponding [Amazon Textract APIs](https://docs.aws.amazon.com/textract/latest/dg/API_Reference.html) such as [GetDocumentAnalysis](https://docs.aws.amazon.com/textract/latest/dg/API_GetDocumentAnalysis.html), [AnalyzeID](https://docs.aws.amazon.com/textract/latest/dg/API_AnalyzeID.html), or [AnalyzeExpense](https://docs.aws.amazon.com/textract/latest/dg/API_AnalyzeExpense.html). In most cases, providing an **array** of response objects is also supported (for use when a large Amazon Textract response was split/paginated).\n\nFor example, loading a response JSON from file in NodeJS:\n\n```js\nfs.readFile(\"./my-analyze-document-response.json\", (err, resBuffer) => {\n if (err) throw err;\n const doc = new TextractDocument(JSON.parse(resBuffer));\n // ...\n});\n```\n\nIf you're using TypeScript, you may need to **typecast** your input JSON while loading it.\n\n> The `ApiResponsePage` input interface exposed and expected by this module is more constrained than - but functionally compatible with - the output types produced by the [AWS SDK for JavaScript Textract Client](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-textract/index.html).\n\n```typescript\nimport { ApiAnalyzeExpenseResponse } from \"amazon-textract-response-parser\";\nimport { TextractClient, AnalyzeExpenseCommand } from \"@aws-sdk/client-textract\";\nconst textract = new TextractClient({});\n\nasync function main() {\n const textractResponse = await textract.send(\n new AnalyzeExpenseCommand({\n Document: { Bytes: await fs.readFile(\"...\") },\n })\n );\n const expense = new TextractExpense((textractResponse as unknown) as ApiAnalyzeExpenseResponse);\n}\n```\n\nWith your data loaded in to a TRP `TextractDocument` or similar, you're ready to take advantage of the higher-level TRP.js functions to navigate and analyze the result.\n\n\n## Generic document text navigation\n\nIn general, this library avoids directly exposing **arrays** in results (see the *Mutation operations* section below). Instead, you can use:\n\n- `.n***` properties to count items\n- `.list***()` functions to return a copy of the underlying array\n- `.iter***()` functions to iterate through collections, or\n- `.***At***()` functions to fetch a specific item from a collection\n\nFor example:\n\n```typescript\n// Navigate the document hierarchy:\nconsole.log(`Opened doc with ${doc.nPages} pages`);\nconsole.log(\n `The first word of the first line is ${doc.pageNumber(1).lineAtIndex(0).wordAtIndex(0).text}`\n);\n\n// Iterate through content:\nfor (const page of doc.iterPages()) {\n // (In Textract's output order...)\n for (const line of page.iterLines()) {\n for (const word of line.iterWords()) {\n console.log(word.text);\n }\n }\n}\n\n// ...Or get snapshot arrays instead of iterators, if you need:\nconst linesArrsByPage = doc.listPages().map((p) => p.listLines());\n```\n\nThese arrays are in the raw order returned by Amazon Textract, which is not necessarily a logical human reading order - especially for multi-column documents. See the *Layout analysis* and *List text in approximate reading order* sections below for extra content sorting utilities.\n\n\n## Queries\n\nThe results of [Amazon Textract Queries](https://docs.aws.amazon.com/textract/latest/dg/queryresponse.html) are accessible at the page level under `page.queries`. You can `get*` a query by exact question text or alias, or `search*` them by case-insensitive substrings:\n\n```typescript\ndoc.listPages().forEach((page) => {\n // Log a quick human-readable overview of queries & answers:\n console.log(page.queries.str());\n\n // Get a query (and its top result's text) by exact alias:\n const customer = page.queries.getQueryByAlias(\"customer_name\")?.topResult?.text;\n\n // Get possible results of a query from most to least confident:\n const shippingAddrCandidates =\n page.queries.getQueryByAlias(\"shipping_addr\")?.listResultsByConfidence() || [];\n const shippingAddrTopConf = shippingAddrCandidates[0].confidence;\n\n // Seaching matches queries e.g. 'What is the Shipping Address?', 'FIND THE BILLING ADDRESS', etc\n const addrQueries = page.queries.searchQueriesByQuestion(\"address\");\n});\n```\n\n\n## Forms (Key-Value pairs)\n\nAs well as looping through the [form data key-value pairs](https://docs.aws.amazon.com/textract/latest/dg/how-it-works-kvp.html) in the document, you can query fields by key:\n\n```typescript\nconsole.log(doc.form.nFields);\nconst fields = doc.form.listFields();\n\n// Exact match:\nconst addr = doc.form.getFieldByKey(\"Address\").value?.text;\n\n// Search key containing (case-insensitive):\nconst addresses = doc.form.searchFieldsByKey(\"address\");\naddresses.forEach((addrField) => { console.log(addrField.key.text); });\n```\n\nNote that the `Field.confidence`, `FieldKey.confidence` and `FieldValue.confidence` scores reflect confidence of the **key-value structure detection** model. For aggregated OCR confidence of their **actual text**, use `.getOcrConfidence()` instead.\n\nYou can also search form keys at the individual page level, or look up the page number for detected fields:\n\n```typescript\nconst fieldByDoc = doc.form.getFieldByKey(\"Address\");\nconsole.log(`Detected Address on page ${fieldByDoc.parentPage.pageNumber}`);\n\nconst page = doc.pageNumber(1);\nconst fieldByPage = page.form.getFieldByKey(\"Address\");\n```\n\n`field.isCheckbox` is true for fields whose value contain exactly one SelectionElement object: meaning they're a (key=label)->(value=checkbox/radio) pair. For these fields, you can directly use `field.selectionStatus` or `field.isSelected` to look up the value's status. For other (non-checkbox) fields, they'll return `null`.\n\n\n## Tables\n\nThis library's table navigation tools address **[merged cells](https://docs.aws.amazon.com/textract/latest/dg/how-it-works-tables.html) by default**, for convenience.\n\n```typescript\nconsole.log(page.nTables);\nconst table = page.tableAtIndex(0);\n\n// Index cells by row, column, or both:\nconst headerStrs = table.cellsAt(1, null)?.map(cell => cell.text);\nconst firstColCells = table.cellsAt(null, 1);\nconst targetCell = table.cellAt(2, 4);\n\n// Iterate over rows/cells:\nfor (const row of table.iterRows()) {\n for (const cell of row.iterCells()) {\n console.log(cell.text);\n }\n}\n```\n\nFurther configuration arguments can be used to change the treatment of merged cells if needed:\n\n```typescript\n// Iterate over rows repeating any cells spanning multiple rows:\nfor (const row of table.iterRows({repeatMultiRowCells: true})) {}\n\n// Return split sub-cells instead of merged cells when indexing:\nconst firstColCellFragments = table.cellsAt(null, 1, {ignoreMerged: true});\n```\n\nThe `Table.confidence`, `Row.getConfidence()` and `Cell.confidence` scores reflect confidence of the **table structure detection** model. For aggregated OCR confidence of the text contained inside, use `.getOcrConfidence()` instead.\n\nUse `Table.tableType` and `Cell.hasEntityTypes()` to explore the more advanced [entity types](https://docs.aws.amazon.com/textract/latest/dg/how-it-works-tables.html) extracted by Amazon Textract: For example column headers, title cells, footer cells, and summary cells:\n\n```typescript\nimport { ApiTableCellEntityType, ApiTableEntityType } from \"amazon-textract-response-parser\";\n\nconst isSemiStruct = table.tableType === ApiTableEntityType.SemiStructuredTable;\nconst colHeaders = table.rowAt(1).listCells()\n .filter((c) => c.hasEntityTypes(ApiTableCellEntityType.ColumnHeader));\n```\n\nFor [overall table-level title and footer captions](https://aws.amazon.com/blogs/machine-learning/announcing-enhanced-table-extractions-with-amazon-textract/), see `table.listTitles()` and `table.listFooters()`, etc.\n\n\n## Layout analysis\n\n[Layout analysis in Amazon Textract](https://aws.amazon.com/blogs/machine-learning/amazon-textracts-new-layout-feature-introduces-efficiencies-in-general-purpose-and-generative-ai-document-processing-tasks/) detects higher-level semantic components than the core text Lines & Words - like paragraphs and headings. If you enabled this analysis, you can access the results through the `page.layout` collection:\n\n```typescript\n// Loop through content in implied reading order (from Layout API):\npage.layout.listItems().forEach((layItem) => {\n console.log(layItem.blockType); // There are different kinds of Layout Item\n const textLines = layItem.listTextLines(); // All Layout* items can be queried for text LINEs\n const children = layItem.listContent(); // Usually text LINEs, but sometimes other Layout* items\n console.log(layItem.text + \"\\n\"); // ...Or you can just pull up the text\n});\n\n// Filtering by content type is also supported:\nfor (const layItem of page.layout.listItems({\n skipBlockTypes: [\n ApiBlockType.LayoutHeader, ApiBlockType.LayoutFooter, ApiBlockType.LayoutPageNumber\n ],\n})) {\n console.log(layItem.text);\n}\n```\n\nIf Forms and/or Tables analyses were also enabled, you'll be able to traverse from the relevant Layout object types to these more detailed representations. **However,** because these are separate analyses the correspondence may not be 1-to-1 and TRP is having to do some reconciliation under the hood:\n\n```typescript\nimport { ApiBlockType, LayoutKeyValue, LayoutTable } from \"amazon-textract-response-parser\";\n\npage.layout.listItems().forEach((layItem) => {\n if (layItem.blockType === ApiBlockType.LayoutKeyValue) {\n const fields = (layItem as LayoutKeyValue).listFields(); // Probably multiple\n fields.forEach((field) => console.log(field.key.text));\n } else if (layItem.blockType === ApiBlockType.LayoutTable) {\n const tables = (layItem as LayoutTable).listTables(); // Probably just 1\n tables.forEach((table) => console.log(table.nCells));\n }\n});\n```\n\n\n### List text in approximate reading order (with or without `Layout`)\n\nParticularly for multi-column documents, the default output sequence for Amazon Textract `LINE`/`WORD` OCR results will likely not be the overall reading order you'd like. For best performance, enable and use the `Layout` analysis because **layout items are returned in implied reading order** as estimated by the AI service.\n\nAlternatively, TRP.js provides a **client-side heuristic algorithm** that can attempt to sort results without Layout. There are even some configuration parameters exposed to help you tune the results for your particular domain, and test harnesses in the [tests/unit/corpus folder](tests/unit/corpus) to help you experiment via `npm run test:unit`:\n\n```typescript\nimport { ReadingOrderLayoutMode } from \"amazon-textract-response-parser\";\n\n// By default, we automatically use `Layout` when it's available and heuristics when it's not:\nlet textInReadingOrder: string = page.getTextInReadingOrder(); // Just generate text\nlet pseudoParas = page.getLineClustersInReadingOrder();\n\n// You can force use of `Layout` (throwing an error if none available):\nlet layText = page.getTextInReadingOrder({ useLayout: ReadingOrderLayoutMode.RequireLayout });\n// Or fine-tune heuristic parameters:\nlet layParas = page.getLineClustersInReadingOrder({\n colHOverlapThresh = 0.75,\n paraVDistTol = 0.8,\n // ...\n useLayout: ReadingOrderLayoutMode.IgnoreLayout,\n});\n\n// Lines are clustered by \"paragraph\"/layout element:\nfor (const pseudoParagraph of pseudoParas) {\n for (const line of pseudoParagraph) {\n console.log(line.text);\n }\n console.log(); // Print a gap between \"paragraphs\"\n}\n```\n\nWhen configured to use Layout analysis results, these functions should be equivalent to just looping through your `page.layout.iterItems()` to get the text from each one in order.\n\n\n### Render documents to semantic markup/markdown\n\nIf you'd like to use AI/ML models to further post-process your Amazon Textract results, you have a choice between those that take text-only inputs - and \"multi-modal\" models that can also ingest structural information (see for example [this Amazon Comprehend feature](https://aws.amazon.com/about-aws/whats-new/2021/09/amazon-comprehend-extract-entities-native-format/) and [this Amazon SageMaker sample](https://github.com/aws-samples/amazon-textract-transformer-pipeline/tree/main)). While multi-modal models may work best on complex structured documents, the pace of research on text-only Large Language Models has historically been faster (perhaps because plain text data is easier to come by and work with).\n\n**Semantic markup like HTML** provides somewhat of a middle ground where we can try to preserve the layout/form/table/etc structure Amazon Textract extracted, but still provide plain text. This may be particularly useful for working with **Generative Large Language Models** (GenAI/LLMs) like those on [Amazon Bedrock](https://aws.amazon.com/bedrock/).\n\n```typescript\n// Render HTML for individual components:\nconsole.log(page.listTables[0].html());\n\n// ...Or for whole pages/documents:\nconst docHtml = doc.html();\nfs.writeFile(\"./my-doc.html\", docHtml, (err) => {});\n```\n\nSome caveats to be aware of:\n\n- Top-level `Page.html()` and `TextractDocument.html()` currently depend on Layout analysis being enabled, because the Layout results are used to sequence all the elements together.\n- Only HTML is supported currently, but we're keen to add `.markdown()` if there's interest\n\nYou can also **filter out** types of content you don't want to include in your HTML. \n\n```typescript\n// Most commonly, you'll `skip` high-level layout elements like `LayoutHeader`:\nconst docHtml = doc.html({\n skipBlockTypes: [\n ApiBlockType.LayoutHeader, ApiBlockType.LayoutFooter, ApiBlockType.LayoutPageNumber\n ],\n});\n\n// Skipping lower-level blocks is also possible, but can produce weird results:\nconst docHtmlNoCellsOrSelectors = doc.html({\n skipBlockTypes: [ApiBlockType.Cell, ApiBlockType.SelectionElement],\n});\n\n// Allow-listing is also possible, but you should include *everything* relevant:\nconst docTablesHtml = doc.html({\n includeBlockTypes: [\n ApiBlockType.Page,\n ApiBlockType.LayoutTable,\n ApiBlockType.Table,\n ApiBlockType.Cell,\n ApiBlockType.SelectionElement,\n ApiBlockType.Word,\n ],\n});\n```\n\nIf you have feedback about these features, please let us know in the GitHub issues to help prioritise!\n\n\n### Segment headers and footers from main content\n\nThis is another task for which you might find [Textract Layout analysis](https://docs.aws.amazon.com/textract/latest/dg/layoutresponse.html) useful - by looping through layout items and filtering out those of type `LayoutHeader`, `LayoutFooter` and `PageNumber`.\n\nHowever, TRP.js also provides a heuristic function you can try instead:\n\n```typescript\nconst segmented = page.getLinesByLayoutArea(\n true // (Also try to sort lines in reading order)\n);\n\nconsole.log(\"---- HEADER:\")\nconsole.log(segmented.header.map((l) => l.text).join(\"\\n\"));\nconsole.log(\"\\n---- CONTENT:\")\nconsole.log(segmented.content.map((l) => l.text).join(\"\\n\"));\nconsole.log(\"\\n---- FOOTER:\")\nconsole.log(segmented.footer.map((l) => l.text).join(\"\\n\"));\n```\n\n**Note:** Unlike the `*inReadingOrder` APIs, this utility has not yet been updated to use Textract Layout analysis when it's available. That behavior might change in future.\n\n\n### Calculate average skew of page text\n\nCalculating the overall skew of a page can be useful for validation checks: For example to detect and reject a strongly skewed image which might degrade the accuracy of tables, forms, or other downstream analyses.\n\n```typescript\n// Check the average angle/skew of detected text:\nconst skew = page.getModalWordOrientationDegrees();\n```\n\nThis method aggregates the skew to find the most common angle across all content on the page.\n\n\n## Signatures\n\nIf you enabled [signature detection in Amazon Textract](https://aws.amazon.com/blogs/machine-learning/detect-signatures-on-documents-or-images-using-the-signatures-feature-in-amazon-textract/), you can check for signatures at the page level:\n\n```typescript\n// e.g. print number of signatures detected by page:\ndoc.listPages()\n .forEach((page, ix) => { console.log(`${page.nSignatures} signatures on page ${ix+1}`); });\n// ...Or get the position of the first signature on the first page:\nconst bbox = doc.pageNumber(1).listSignatures()[0].geometry.boundingBox;\n```\n\n\n## Expense (invoice and receipt) objects\n\nSince the format of responses for Amazon Textract's [Expense results](https://docs.aws.amazon.com/textract/latest/dg/expensedocuments.html) is very different from the [general document analysis APIs](https://docs.aws.amazon.com/textract/latest/dg/how-it-works-document-layout.html), you can use the separate `TextractExpense` class in this library to process these.\n\n```typescript\nconst expense = new TextractExpense(textractResponse);\n\n// Iterate through content:\nconsole.log(`Found ${expense.nDocs} expense docs in file`);\nconst expenseDoc = [...expense.iterDocs()][0];\nfor (const group of expenseDoc.iterLineItemGroups()) {\n for (const item of group.iterLineItems()) {\n console.log(`Found line item with ${item.nFields} fields`);\n for (const field of item.iterFields()) {\n ...\n }\n }\n}\n\n// Get snapshot arrays instead of iterators, if you need:\nconst summaryFieldsArrByDoc = expense.listDocs().map((doc) => doc.listSummaryFields());\nconst linesArrsByPage = doc.listPages().map((p) => p.listLines())\n\n// Retrieve item fields by their tagged 'type':\nconst vendorNameFields = expenseDoc.searchSummaryFieldsByType(\"VENDOR_NAME\");\nconsole.log(`Found ${vendorNameFields.length} vendor name fields in doc summary`);\nconsole.log(vendorNameFields[0].fieldType.text); // \"VENDOR_NAME\"\nconsole.log(vendorNameFields[0].value.text); // e.g. \"Amazon.com\"\n```\n\n\n## Identity document objects\n\nSimilarly to expenses mentioned above, Amazon Textract offers specific APIs for [identity document analysis](https://docs.aws.amazon.com/textract/latest/dg/how-it-works-identity.html). You can use the separate `TextractIdentity` class in this library to process these.\n\n```typescript\nimport { ApiAnalyzeIdResponse, TextractIdentity } from \"amazon-textract-response-parser\";\nimport { TextractClient, AnalyzeIDCommand } from \"@aws-sdk/client-textract\";\nconst textract = new TextractClient({});\n\nasync function main() {\n const textractResponse = await textract.send(\n new AnalyzeIDCommand({\n Document: { Bytes: await fs.readFile(\"...\") },\n })\n );\n const identity = new TextractIdentity((textractResponse as unknown) as ApiAnalyzeIdResponse);\n}\n```\n\nThe library implements some enumerations of known values (for field types, ID types, and so on) to make processing AnalyzeID responses a little simpler:\n\n```typescript\nimport { IdDocumentType, IdFieldType } from \"amazon-textract-response-parser\";\n\nconst idDoc = identity.getDocAtIndex(0); // (Or iterate, list docs in a result)\n\nif (idDoc.idType === IdDocumentType.Passport) {\n // Fetch fields by known type:\n const passNumField = idDoc.getFieldByType(IdFieldType.DocumentNumber);\n console.log(\n `Passport number ${passNumField.value}, confidence ${passNumField.valueConfidence}%`\n );\n\n} else if (idDoc.idType === IdDocumentType.DrivingLicense) {\n // ...Or list or iterate the document's fields:\n for (const field of idDoc.iterFields()) {\n console.log(`${field.fieldTypeRaw}: ${field.valueRaw}`);\n }\n\n} else {\n // Produce human-readable representations of fields, documents, or whole responses:\n console.log(idDoc.str());\n}\n```\n\n\n## Mutation operations\n\nEasier analysis and querying of Textract results is useful, but what if you want to augment or edit your Textract JSONs with JS/TS Textract Response Parser?\n\nIn general:\n\n- Where the library classes (`TextractDocument`, `Page`, `Word`, etc) offer mutation operations, these should modify the source API JSON object **in-place** and ensure self-consistency.\n- For library classes that are backed by a specific object in the source API JSON, you can access it via the `.dict` property (`word.dict`, `table.dict`, etc) but then *you're* responsible for updating any required references in other objects if making changes there.\n- Any individual-block-level changes you make to the underlying API JSON should be dynamically reflected in the parsed TRP objects (e.g. overriding word text, coordinates, etc)... But changes that affect inter-block relationships are more likely to cause staleness issues.\n\nIn particular for **array properties**, you'll note that TRP generally exposes getters and iterators (such as `table.nRows`, `table.iterRows()`, `table.listRows()`, `table.cellsAt()`) rather than direct access to lists - to avoid implying that arbitrary array mutations (such as `table.rows.pop()`) are properly supported.\n\n\n## Other features and examples\n\nFor more examples on how to use the library, you can refer to the (basic) [examples](examples/) and (more complete) [test](test/) folders on GitHub, and the source code itself. If you have suggestions for additional features that would be useful, please open a GitHub issue!\n\n\n## Development\n\nThe integration tests for this library validate the end-to-end toolchain for calling Amazon Textract and parsing the result, so note that to run the full `npm run test` command:\n\n1. Your environment will need to be configured with a login to AWS (e.g. via the [AWS CLI](https://aws.amazon.com/cli/))\n2. Billable API requests may be made\n\nYou can alternatively run just the local/unit tests via `npm run test:unit`.\n"
},
{
"path": "src-js/bin/reading-order-diagnostic.js",
"content": "/**\n * Basic script to extract and save reading-order text from Amazon Textract JSONs.\n *\n * This script uses the built NodeJS library, so check your build is up-to-date by first running\n * `npm run build`! JSON files are read from IN_FOLDER, parsed with the TRP, and reading-order text\n * files output to the OUT_FOLDER. This can be a helpful tool for debugging issues with (or writing\n * corpus tests for) the 'inReadingOrder' functions.\n */\n/* eslint-disable no-undef */\n/* eslint-disable @typescript-eslint/no-var-requires */\n\n// Node Built-Ins:\nconst fs = require(\"fs\");\nconst path = require(\"path\");\n\n// Local Dependencies:\nconst { TextractDocument } = require(\"../dist/cjs\");\n\nconst IN_FOLDER = \"test/data/corpus\";\nconst OUT_FOLDER = \"test/data/corpus-readingorder\";\n\nif (!fs.existsSync(OUT_FOLDER)) {\n fs.mkdirSync(OUT_FOLDER, { recursive: true });\n}\n\nfs.readdirSync(IN_FOLDER).forEach((file) => {\n let response;\n try {\n response = JSON.parse(fs.readFileSync(path.join(IN_FOLDER, file)));\n } catch (err) {\n console.error(`Skipping ${file} - doesn't look like valid JSON`);\n return;\n }\n const pageTexts = [];\n const doc = new TextractDocument(response);\n doc.listPages().forEach((page, ixPage) => {\n pageTexts.push(\n [\n \"------------------------------------------------\",\n `PAGE ${ixPage + 1}`,\n \"------------------------------------------------\",\n page.getTextInReadingOrder(),\n ].join(\"\\n\")\n );\n });\n\n const outFileRoot = path.join(OUT_FOLDER, file.split(\".\")[0]);\n fs.writeFileSync(`${outFileRoot}.readingorder.txt`, pageTexts.join(\"\\n\\n\\n\"));\n fs.writeFileSync(\n `${outFileRoot}.readingorder.json`,\n JSON.stringify(\n doc\n .listPages()\n .map((page) =>\n page._getLineClustersByColumn().map((col) => col.map((cluster) => cluster.map((line) => line.text)))\n ),\n null,\n 2\n )\n );\n console.log(`Done ${file}`);\n});\nconsole.log(\"All done!\");\n"
},
{
"path": "src-js/examples/README.md",
"content": "# Examples for TRP.js\n\nThis folder contains example projects using the Amazon Textract Response Parser for JavaScript/TypeScript from various different build environments, to help you get started.\n\n> ⚠️ **Note:** While all of the example projects reference local API response JSON files, some also make Amazon Textract API calls by default - so running them may incur (typically very small) charges. See [Amazon Textract Pricing](https://aws.amazon.com/textract/pricing/) for details.\n\n\n## Pre-requisites for running the examples\n\n\n### Local builds of TRP.js\n\nThe projects use the **local build** of the library for pre-publication testing, so you'll need to run `npm run build` in the parent `src-js` folder before they'll work.\n\nTo instead switch to published TRP.js versions (if you're using an example as a skeleton for your own project):\n\n- For NodeJS projects, Replace the package.json relative path in `\"amazon-textract-response-parser\": \"file:../..\"` with a normal version spec like `\"amazon-textract-response-parser\": \"^0.4.3\"`, and re-run `npm install`\n- For browser IIFE projects, edit the `\n \n \n \n \n \n \n
Select an Amazon Textract API response JSON file to analyze