[ { "path": ".github/dependabot.yml", "content": "version: 2\nupdates:\n - package-ecosystem: \"github-actions\"\n directory: \"/\"\n schedule:\n interval: \"weekly\"\n" }, { "path": ".github/workflows/ruby.yml", "content": "# This workflow uses actions that are not certified by GitHub.\n# They are provided by a third-party and are governed by\n# separate terms of service, privacy policy, and support\n# documentation.\n# This workflow will download a prebuilt Ruby version, install dependencies and run tests with Rake\n# For more information see: https://github.com/marketplace/actions/setup-ruby-jruby-and-truffleruby\n\nname: Ruby\n\non:\n push:\n branches: [ \"master\" ]\n pull_request:\n branches: [ \"master\" ]\n\npermissions:\n contents: read\n\njobs:\n test:\n\n runs-on: ubuntu-latest\n strategy:\n matrix:\n ruby-version: ['2.6', '2.7', '3.0', '3.1', '3.2', '3.3']\n\n steps:\n - uses: actions/checkout@v4\n - name: Set up Ruby\n uses: ruby/setup-ruby@v1\n with:\n ruby-version: ${{ matrix.ruby-version }}\n bundler-cache: true # runs 'bundle install' and caches installed gems automatically\n - name: Run tests\n run: bundle exec rake\n" }, { "path": ".gitignore", "content": "*.gem\n*.rbc\n.bundle\n.config\n.yardoc\nGemfile.lock\nInstalledFiles\n_yardoc\ncoverage\ndoc/\nlib/bundler/man\npkg\nrdoc\nspec/reports\ntest/tmp\ntest/version_tmp\ntmp\n" }, { "path": ".travis.yml", "content": "language: ruby\ncache: bundler\nbefore_install:\n - gem update bundler\nrvm:\n - 2.5.8\n - 2.7.2\n - 3.0.0\n" }, { "path": "CHANGELOG.md", "content": "### 5.1.0\n\n* Parse sheets containing namespaces and no 'r' att (@skipchris)\n* Fix Zlib error when loading from string (@myabc)\n* Prevent a SimpleXlsxReader::CellLoadError (no implicit conversion of Integer\n into String) when the casted value (friendly name) is not a string (@tsdbrown)\n* Accidental 25% perfarmance improvement while experimenting with namespace\n support (see #53f5a9).\n\n### 5.0.0\n\n* Change SimpleXlsxReader::Hyperlink to default to the visible cell value\n instead of the hyperlink URL, which in the case of mailto hyperlinks is\n surprising.\n* Fix blank content when parsing docs from string (@codemole)\n\n### 4.0.1\n\n* Fix nil error when handling some inline strings\n\n Inline strings are almost exclusively used by non-Excel XLSX\n implementations, but are valid, and sometimes have nil chunks.\n\n Also, inline strings weren't preserving whitespace if Nokogiri is\n parsing the string in chunks, as it does when encountering escaped\n characters. Fixed.\n\n### 4.0.0\n\n* Fix percentage rounding errors. Previously we were dividing by 100, when we\n actually don't need to, so percentage types were 100x too small. Fixes #21.\n Major bump because workarounds might have been implemented for previous\n incorrect behavior.\n* Fix small oddity in one currency format where round numbers would be cast\n to an integer instead of a float.\n\n### 3.0.1\n\n* Fix parsing \"chunky\" UTF-8 workbooks. Closes issues #39 and #45. See ce67f0d4.\n\n### 3.0.0\n\n* Change the way we typecast cells in the General format. This probably won't\n break anything in your app, but it's a change in behavior that theoretically\n could.\n\n Previously, we were treating cells using General the format as strings, when\n according to the Office XML standard, they should be treated as numbers. We\n now attempt to cast such cells as numbers, and fall back to strings if number\n casting fails.\n\n Thanks @jrodrigosm\n\n### 2.0.1\n\n* Restore ability to parse IO strings (@robbevp)\n* Add Ruby 3.1 and 3.2 to CI (@taichi-ishitani)\n\n### 2.0.0\n\n* SPEED\n * Reimplement internals in terms of a SAX parser\n * Change `SimpleXlsxReader::Sheet#rows` to be a `RowsProxy` that streams `#each`\n* Convenience - use `rows#each(headers: true)` to get header names while enumerating rows\n\n### 1.0.5\n\n* Support string or io input via `SimpleXlsxReader#parse` (@kalsan, @til)\n\n### 1.0.4\n\n* Fix Windows + RubyZip 1.2.1 bug preventing files from being read\n* Add ability to parse hyperlinks\n* Support files exported from Google Docs (@Strnadj)\n\n### 1.0.3\n\nBroken on Ruby 1.9; yanked.\n\n### 1.0.2\n\n* Fix Ruby 1.9.3-specific bug preventing parsing most sheets [middagj, eritiro]\n* Better support for non-excel-generated xlsx files [bwlang]\n * You don't always have a numFmtId column, and that's OK\n * Sometimes 'sharedStrings.xml' can be 'sharedstrings.xml'\n* Fixed parsing times very close to 12/30/1899 [Valeriy Utyaganov]\n* Be more flexible with custom formats using a numFmtId < 164\n\n### 1.0.1\n\n* Add support for the 1904 date system [zilverline]\n\n### 1.0.0\n\nNo changes since 1.0.0.pre. Releasing 1.0.0 since the project has seen a\nfew months of stability in terms of bug fix requests, and the API is not\ngoing to change.\n\n### 1.0.0.pre\n\n* Handle files with blank rows [Brian Hoffman]\n* Preserve seconds when casting datetimes [Rob Newbould]\n* Preserve empty rows (previously would be ommitted)\n* Speed up parsing by ~55%\n\n### 0.9.8\n\n* Rubyzip 1.0 compatability\n\n### 0.9.7\n\n* Fix cell parsing where cells have a type, but no content\n* Add a speed test; parsing performs in linear time, but a relatively\n slow line :/\n\n### 0.9.6\n\n* Fix worksheet indexes when worksheets have been deleted\n\n### 0.9.5\n\n* Fix inlineStr support (broken by formula support commit)\n\n### 0.9.4\n\n* Formula support. Formulas used to cause things to blow up, now they don't!\n* Support number types styled as dates. Previously, the type was honored\n above the style, which is incorrect for dates; date-numbers now parse as\n dates.\n* Error-free parsing of empty sheets\n* Fix custom styles w/ numFmtId == 164. Custom style types are delineated\n starting *at* numFmtId 164, not greater than 164.\n\n### 0.9.3\n\n* Support 1.8.7 (tests pass). Ongoing support will depend on ease.\n\n### 0.9.2\n\n* Support reading files written by ex. simple_xlsx_writer that don't\n specify sheet dimensions explicitly (which Excel does).\n\n### 0.9.1\n\n* Fixed an important parse bug that ignored empty 'Generic' cells\n\n### 0.9.0\n\n* Initial release. 0.9 version number is meant to reflect the near-stable\n public api, yet still prerelease status of the project.\n" }, { "path": "Gemfile", "content": "source 'https://rubygems.org'\n\n# Specify your gem's dependencies in simple_xlsx_reader.gemspec\ngemspec\n" }, { "path": "LICENSE.txt", "content": "Copyright (c) 2013 Woody Peterson\n\nMIT License\n\nPermission is hereby granted, free of charge, to any person obtaining\na copy of this software and associated documentation files (the\n\"Software\"), to deal in the Software without restriction, including\nwithout limitation the rights to use, copy, modify, merge, publish,\ndistribute, sublicense, and/or sell copies of the Software, and to\npermit persons to whom the Software is furnished to do so, subject to\nthe following conditions:\n\nThe above copyright notice and this permission notice shall be\nincluded in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\nNONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE\nLIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION\nOF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION\nWITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE." }, { "path": "README.md", "content": "# SimpleXlsxReader\n\nA [fast](#performance) xlsx reader for Ruby that parses xlsx cell values into\nplain ruby primitives and dates/times.\n\nThis is *not* a rewrite of excel in Ruby. Font styles, for\nexample, are parsed to determine whether a cell is a number or a date,\nthen forgotten. We just want to get the data, and get out!\n\n## Summary (now with stream parsing):\n\n```ruby\ndoc = SimpleXlsxReader.open('/path/to/workbook.xlsx')\ndoc.sheets # => [<#SXR::Sheet>, ...]\ndoc.sheets.first.name # 'Sheet1'\nrows = doc.sheets.first.rows # \nrows.each # an ready to chain or stream\nrows.each {} # Streams the rows to your block\nrows.each(headers: true) {} # Streams row-hashes\nrows.each(headers: {id: /ID/}) {} # finds & maps headers, streams\nrows.slurp # Slurps rows into memory as a 2D array\n```\n\nThat's the gist of it!\n\nSee also the [Document](https://github.com/woahdae/simple_xlsx_reader/blob/2.0.0-pre/lib/simple_xlsx_reader/document.rb) object.\n\n## Why?\n\n### Accurate\n\nThis project was started years ago, primarily because other Ruby xlsx parsers\ndidn't import data with the correct types. Numbers as strings, dates as numbers,\n[hyperlinks](https://github.com/woahdae/simple_xlsx_reader/blob/master/lib/simple_xlsx_reader/hyperlink.rb)\nwith inaccessible URLs, or - subtly buggy - simple dates as DateTime\nobjects. If your app uses a timezone offset, depending on what timezone and\nwhat time of day you load the xlsx file, your dates might end up a day off!\nSimpleXlsxReader understands all these correctly.\n\n### Idiomatic\n\nMany Ruby xlsx parsers seem to be inspired more by Excel than Ruby, frankly.\nSimpleXlsxReader strives to be fairly idiomatic Ruby:\n\n```ruby\n# quick example having fun w/ ruby\ndoc = SimpleXlsxReader.open(file_path) # or SimpleXlsxReader.parse(string_or_io)\ndoc.sheets.first.rows.each(headers: {id: /ID/})\n .with_index.with_object({}) do |(row, index), acc|\n acc[row[:id]] = index\nend\n```\n\n### Now faster\n\nFinally, as of v2.0, SimpleXlsxReader is the fastest and most\nmemory-efficient parser. Previously this project couldn't reasonably load\nanything over ~10k rows. Other parsers could load 100k+ rows, but were still\ntaking ~1gb RSS to do so, even \"streaming,\" which seemed excessive. So a SAX\nimplementation was born. See [performance](#performance) for details.\n\n## Usage\n\n### Streaming\n\nSimpleXlsxReader is performant by default - If you use\n`rows.each {|row| ...}` it will stream the XLSX rows to your block without\nloading either the sheet XML or the full sheet data into memory.\n\nYou can also chain `rows.each` with other Enumerable functions without\ntriggering a slurp, and you have lots of ways to find and map headers while\nstreaming.\n\nIf you had an excel sheet representing this data:\n\n```\n| Hero ID | Hero Name | Location |\n| 13576 | Samus Aran | Planet Zebes |\n| 117 | John Halo | Ring World |\n| 9704133 | Iron Man | Planet Earth |\n```\n\nGet a handle on the rows proxy:\n\n```ruby\nrows = SimpleXlsxReader.open('suited_heroes.xlsx').sheets.first.rows\n```\n\nSimple streaming (kinda boring):\n\n```ruby\nrows.each { |row| ... }\n````\n\nStreaming with headers, and how about a little enumerable chaining:\n\n```ruby\n# Map of hero names by ID: { 117 => 'John Halo', ... }\n\nrows.each(headers: true).with_object({}) do |row, acc|\n acc[row['Hero ID']] = row['Hero Name']\nend\n```\n\nSometimes though you have some junk at the top of your spreadsheet:\n\n```\n| Unofficial Report | | |\n| Dont tell Nintendo | Yes \"John Halo\" I know | |\n| | | |\n| Hero ID | Hero Name | Location |\n| 13576 | Samus Aran | Planet Zebes |\n| 117 | John Halo | Ring World |\n| 9704133 | Iron Man | Planet Earth |\n```\n\nFor this, `headers` can be a hash whose keys replace headers and whose values\nhelp find the correct header row:\n\n```ruby\n# Same map of hero names by ID: { 117 => 'John Halo', ... }\n\nrows.each(headers: {id: /ID/, name: /Name/}).with_object({}) do |row, acc|\n acc[row[:id]] = row[:name]\nend\n```\n\nIf your header-to-attribute mapping is more complicated than key/value, you\ncan do the mapping elsewhere, but use a block to find the header row:\n\n```ruby\n# Example roughly analogous to some production code mapping a single spreadsheet\n# across many objects. Might be a simpler way now that we have the headers-hash\n# feature.\n\nobject_map = { Hero => { id: 'Hero ID', name: 'Hero Name', location: 'Location' } }\n\nHEADERS = ['Hero ID', 'Hero Name', 'Location']\n\nrows.each(headers: ->(row) { (HEADERS & row).any? }) do |row|\n object_map.each_pair do |klass, attribute_map|\n attributes =\n attribute_map.each_pair.with_object({}) do |(key, header), attrs|\n attrs[key] = row[header]\n end\n\n klass.new(attributes)\n end\nend\n```\n\n### Slurping\n\nTo make SimpleXlsxReader rows act like an array, for use with legacy\nSimpleXlsxReader apps or otherwise, we still support slurping the whole array\ninto memory. The good news is even when doing this, the xlsx worksheet & shared\nstring files are never loaded as a (big) Nokogiri doc, so that's nice.\n\nBy default, to prevent accidental slurping, `` will throw an exception\nif you try to access it with array methods like `[]` and `shift` without\nexplicitly slurping first. You can slurp either by calling `rows.slurp` or\nglobally by setting `SimpleXlsxReader.configuration.auto_slurp = true`.\n\nOnce slurped, enumerable methods on `rows` will use the slurped data\n(i.e. not re-parse the sheet), and those Array-like methods will work.\n\nWe don't support all Array methods, just the few we have used in real projects,\nas we transition towards streaming instead.\n\n### Load Errors\n\nBy default, cell load errors (ex. if a date cell contains the string\n'hello') result in a SimpleXlsxReader::CellLoadError.\n\nIf you would like to provide better error feedback to your users, you\ncan set `SimpleXlsxReader.configuration.catch_cell_load_errors =\ntrue`, and load errors will instead be inserted into Sheet#load_errors keyed\nby [rownum, colnum]:\n\n```ruby\n{\n [rownum, colnum] => '[error]'\n}\n```\n\n### Performance\n\nSimpleXlsxReader is (as of this writing) the fastest and most memory efficient\nRuby xlsx parser.\n\nRecent updates here have focused on large spreadsheets with especially\nnon-unique strings in sheets using xlsx' shared strings feature\n(Excel-generated spreadsheets always use this). Other projects have implemented\nstreaming parsers for the sheet data, but currently none stream while loading\nthe shared strings file, which is the second-largest file in an xlsx archive\nand can represent millions of strings in large files.\n\nFor more details, see [my fork of @shkm's excel benchmark project](https://github.com/woahdae/excel-parsing-benchmarks), but here's the summary:\n\n1mb excel file, 10,000 rows of sample \"sales records\" with a fair amount of\nnon-unique strings (ran on an M1 Macbook Pro):\n\n| Gem | Parses/second | RSS Increase | Allocated Mem | Retained Mem | Allocated Objects | Retained Objects |\n|--------------------|---------------|--------------|---------------|--------------|-------------------|------------------|\n| simple_xlsx_reader | 1.13 | 36.94mb | 614.51mb | 1.13kb | 8796275 | 3 |\n| roo | 0.75 | 74.0mb | 164.47mb | 2.18kb | 2128396 | 4 |\n| creek | 0.65 | 107.55mb | 581.38mb | 3.3kb | 7240760 | 16 |\n| xsv | 0.61 | 75.66mb | 2127.42mb | 3.66kb | 5922563 | 10 |\n| rubyxl | 0.27 | 373.52mb | 716.7mb | 2.18kb | 10612577 | 4 |\n\nHere is a benchmark for the \"worst\" file I've seen, a 26mb file whose shared\nstrings represent 10% of the archive (note, MemoryProfiler has too much\noverhead to reasonably measure allocations so that analysis was left off, and\nwe just measure total time for one parse):\n\n| Gem | Time | RSS Increase |\n|--------------------|---------|--------------|\n| simple_xlsx_reader | 28.71s | 148.77mb |\n| roo | 40.25s | 1322.08mb |\n| xsv | 45.82s | 391.27mb |\n| creek | 60.63s | 886.81mb |\n| rubyxl | 238.68s | 9136.3mb |\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n gem 'simple_xlsx_reader'\n\nAnd then execute:\n\n $ bundle\n\nOr install it yourself as:\n\n $ gem install simple_xlsx_reader\n\n## Versioning\n\nThis project follows [semantic versioning 1.0](http://semver.org/spec/v1.0.0.html)\n\n## Contributing\n\nRemember to write tests, think about edge cases, and run the existing\nsuite.\n\nThe full suite contains a performance test that on an M1 MBP runs the final\nlarge file in about five seconds. Check out that test before & after your\nchange to check for performance changes.\n\nThen, the standard stuff:\n\n1. Fork this project\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create new Pull Request\n" }, { "path": "Rakefile", "content": "# frozen_string_literal: true\n\nrequire \"bundler/gem_tasks\"\n\nrequire 'rake/testtask'\nRake::TestTask.new do |t|\n t.pattern = \"test/**/*_test.rb\"\n t.libs << 'test'\nend\n\ntask default: [:test]\n" }, { "path": "lib/simple_xlsx_reader/document.rb", "content": "# frozen_string_literal: true\n\nrequire 'forwardable'\n\nmodule SimpleXlsxReader\n\n ##\n # Main class for the public API. See the README for usage examples,\n # or read the code, it's pretty friendly.\n class Document\n attr_reader :string_or_io\n\n def initialize(legacy_file_path = nil, file_path: nil, string_or_io: nil)\n fail(ArgumentError, 'either file_path or string_or_io must be provided') if legacy_file_path.nil? && file_path.nil? && string_or_io.nil?\n \n @string_or_io = string_or_io || File.new(legacy_file_path || file_path)\n end\n\n def sheets\n @sheets ||= Loader.new(string_or_io).init_sheets\n end\n\n # Expensive because it slurps all the sheets into memory,\n # probably only appropriate for testing\n def to_hash\n sheets.each_with_object({}) { |sheet, acc| acc[sheet.name] = sheet.rows.to_a; }\n end\n\n # `rows` is a RowsProxy that responds to #each\n class Sheet\n extend Forwardable\n\n attr_reader :name, :rows\n\n def_delegators :rows, :load_errors, :slurp\n\n def initialize(name:, sheet_parser:)\n @name = name\n @rows = RowsProxy.new(sheet_parser: sheet_parser)\n end\n\n # Legacy - consider `rows.each(headers: true)` for better performance\n def headers\n rows.slurped![0]\n end\n\n # Legacy - consider `rows` or `rows.each(headers: true)` for better\n # performance\n def data\n rows.slurped![1..-1]\n end\n end\n\n # Waits until we call #each with a block to parse the rows\n class RowsProxy\n include Enumerable\n\n attr_reader :slurped, :load_errors\n\n def initialize(sheet_parser:)\n @sheet_parser = sheet_parser\n @slurped = nil\n @load_errors = {}\n end\n\n # By default, #each streams the rows to the provided block, either as\n # arrays, or as header => cell value pairs if provided a `headers:`\n # argument.\n #\n # `headers` can be:\n #\n # * `true` - simply takes the first row as the header row\n # * block - calls the block with successive rows until the block returns\n # true, which it then uses that row for the headers. All data prior to\n # finding the headers is ignored.\n # * hash - transforms the header row by replacing cells with keys matched\n # by value, ex. `{id: /ID|Identity/, name: /Name/i, date: 'Date'}` would\n # potentially yield the row `{id: 5, name: 'Jane', date: [Date object]}`\n # instead of the headers from the sheet. It would also search for the\n # row that matches at least one header, in case the header row isn't the\n # first.\n #\n # If rows have been slurped, #each will iterate the slurped rows instead.\n #\n # Note, calls to this after slurping will raise if given the `headers:`\n # argument, as that's handled by the sheet parser. If this is important\n # to someone, speak up and we could potentially support it.\n def each(headers: false, &block)\n if slurped?\n raise '#each does not support headers with slurped rows' if headers\n\n slurped.each(&block)\n elsif block_given?\n # It's possible to slurp while yielding to the block, which would\n # null out @sheet_parser, so let's just keep track of it here too\n sheet_parser = @sheet_parser\n @sheet_parser.parse(headers: headers, &block).tap do\n @load_errors = sheet_parser.load_errors\n end\n else\n to_enum(:each, headers: headers)\n end\n end\n\n # Mostly for legacy support, I'm not aware of a use case for doing this\n # when you don't have to.\n #\n # Note that #each will use slurped results if available, and since we're\n # leveraging Enumerable, all the other Enumerable methods will too.\n def slurp\n # possibly release sheet parser from memory on next GC run;\n # untested, but it can hold a lot of stuff, so worth a try\n @slurped ||= to_a.tap { @sheet_parser = nil }\n end\n\n def slurped?\n !!@slurped\n end\n\n def slurped!\n check_slurped\n\n slurped\n end\n\n def [](*args)\n check_slurped\n\n slurped[*args]\n end\n\n def shift(*args)\n check_slurped\n\n slurped.shift(*args)\n end\n\n private\n\n def check_slurped\n slurp if SimpleXlsxReader.configuration.auto_slurp\n return if slurped?\n\n raise 'Called a slurp-y method without explicitly slurping;'\\\n ' use #each or call rows.slurp first'\n end\n end\n end\nend\n" }, { "path": "lib/simple_xlsx_reader/hyperlink.rb", "content": "# frozen_string_literal: true\n\nmodule SimpleXlsxReader\n # We support hyperlinks as a \"type\" even though they're technically\n # represented either as a function or an external reference in the xlsx spec.\n #\n # In practice, hyperlinks are usually a link or a mailto. In the case of a\n # link, we probably want to follow it to download something, but in the case\n # of an email, we probably just want the email and not the mailto. So we\n # represent a hyperlink primarily as it is seen by the user, following the\n # principle of least surprise, but the url is accessible via #url.\n #\n # Microsoft calls the visible part of a hyperlink cell the \"friendly name,\"\n # so we expose that as a method too, in case you want to be explicit about\n # how you're accessing it.\n #\n # See MS documentation on the HYPERLINK function for some background:\n # https://support.office.com/en-us/article/HYPERLINK-function-333c7ce6-c5ae-4164-9c47-7de9b76f577f\n class Hyperlink < String\n attr_reader :friendly_name\n attr_reader :url\n\n def initialize(url, friendly_name = nil)\n @url = url\n @friendly_name = friendly_name&.to_s\n super(@friendly_name || @url)\n end\n end\nend\n" }, { "path": "lib/simple_xlsx_reader/loader/shared_strings_parser.rb", "content": "# frozen_string_literal: true\n\nmodule SimpleXlsxReader\n class Loader\n # For performance reasons, excel uses an optional SpreadsheetML feature\n # that puts all strings in a separate xml file, and then references\n # them by their index in that file.\n #\n # http://msdn.microsoft.com/en-us/library/office/gg278314.aspx\n class SharedStringsParser < Nokogiri::XML::SAX::Document\n def self.parse(file)\n new.tap do |parser|\n Nokogiri::XML::SAX::Parser.new(parser).parse(file)\n end.result\n end\n\n def initialize\n @result = []\n @composite = false\n @extract = false\n end\n\n attr_reader :result\n\n def start_element(name, _attrs = [])\n case name\n when 'si' then @current_string = +\"\" # UTF-8 variant of String.new\n when 't' then @extract = true\n end\n end\n\n def characters(string)\n return unless @extract\n\n @current_string << string\n end\n\n def end_element(name)\n case name\n when 't' then @extract = false\n when 'si' then @result << @current_string\n end\n end\n end\n end\nend\n" }, { "path": "lib/simple_xlsx_reader/loader/sheet_parser.rb", "content": "# frozen_string_literal: true\n\nrequire 'forwardable'\n\nmodule SimpleXlsxReader\n class Loader\n class SheetParser < Nokogiri::XML::SAX::Document\n extend Forwardable\n\n attr_accessor :xrels_file\n attr_accessor :hyperlinks_by_cell\n\n attr_reader :load_errors\n\n def_delegators :@loader, :style_types, :shared_strings, :base_date\n\n def initialize(file_io:, loader:)\n @file_io = file_io\n @loader = loader\n end\n\n def parse(headers: false, &block)\n raise 'parse called without a block; what should this do?'\\\n unless block_given?\n\n @headers = headers\n @each_callback = block\n @load_errors = {}\n @current_row_num = nil\n @last_seen_row_idx = 0\n @url = nil # silence warnings\n @function = nil # silence warnings\n @capture = nil # silence warnings\n @captured = nil # silence warnings\n @dimension = nil # silence warnings\n @column_index = 0\n\n @file_io.rewind # if it's IO from IO.read, we need to rewind it\n\n # In this project this is only used for GUI-made hyperlinks (as opposed\n # to FUNCTION-based hyperlinks). Unfortunately the're needed to parse\n # the spreadsheet, and they come AFTER the sheet data. So, solution is\n # to just stream-parse the file twice, first for the hyperlinks at the\n # bottom of the file, then for the file itself. In the future it would\n # be clever to use grep to extract the xml into its own smaller file.\n if xrels_file\n if xrels_file.grep(/hyperlink/).any?\n xrels_file.rewind\n load_gui_hyperlinks # represented as hyperlinks_by_cell\n end\n @file_io.rewind # we've already parsed this once\n end\n\n Nokogiri::XML::SAX::Parser.new(self).parse(@file_io)\n end\n\n ###\n # SAX document hooks\n\n def start_element_namespace(name, attrs = [], _prefix, _uri, _ns)\n case name\n when 'dimension'\n @dimension = attrs.last.value\n when 'row'\n @current_row_num = attrs.find {|attr| attr.localname == 'r'}&.value&.to_i\n @current_row = Array.new(column_length)\n @column_index = 0\n when 'c'\n attrs = attrs.inject({}) {|acc, attr| acc[attr.localname] = attr.value; acc}\n @cell_name = attrs['r'] || column_number_to_letter(@column_index)\n @type = attrs['t']\n @style = attrs['s'] && style_types[attrs['s'].to_i]\n @column_index += 1\n when 'f' then @function = true\n when 'v', 't' then @capture = true\n end\n end\n\n def characters(string)\n if @function\n # the only \"function\" we support is a hyperlink\n @url = string.slice(/HYPERLINK\\(\"(.*?)\"/, 1)\n end\n\n return unless @capture\n\n captured =\n begin\n SimpleXlsxReader::Loader.cast(\n string, @type, @style,\n url: @url || hyperlinks_by_cell&.[](@cell_name),\n shared_strings: shared_strings,\n base_date: base_date\n )\n rescue StandardError => e\n column, row = @cell_name.match(/([A-Z]+)([0-9]+)/).captures\n col_idx = column_letter_to_number(column) - 1\n row_idx = row.to_i - 1\n\n if !SimpleXlsxReader.configuration.catch_cell_load_errors\n error = CellLoadError.new(\n \"Row #{row_idx}, Col #{col_idx}: #{e.message}\"\n )\n error.set_backtrace(e.backtrace)\n raise error\n else\n @load_errors[[row_idx, col_idx]] = e.message\n\n string\n end\n end\n\n # For some reason I can't figure out in a reasonable timeframe,\n # SAX parsing some workbooks captures separate strings in the same cell\n # when we encounter UTF-8, although I can't get workbooks made in my\n # own version of excel to repro it. Our fix is just to keep building\n # the string in this case, although maybe there's a setting in Nokogiri\n # to make it not do this (looked, couldn't find it).\n #\n # Loading the workbook test/chunky_utf8.xlsx repros the issue.\n @captured = @captured ? @captured + (captured || '') : captured\n end\n\n def end_element_namespace(name, _prefix, _uri)\n case name\n when 'row'\n if @headers == true # ya a little funky\n @headers = @current_row\n elsif @headers.is_a?(Hash)\n test_headers_hash_against_current_row\n # in case there were empty rows before finding the header\n @last_seen_row_idx = @current_row_num - 1\n elsif @headers.respond_to?(:call)\n @headers = @current_row if @headers.call(@current_row)\n # in case there were empty rows before finding the header\n @last_seen_row_idx = @current_row_num - 1\n elsif @headers\n possibly_yield_empty_rows(headers: true)\n yield_row(@current_row, headers: true)\n else\n possibly_yield_empty_rows(headers: false)\n yield_row(@current_row, headers: false)\n end\n\n @last_seen_row_idx += 1\n\n # Note that excel writes a '/worksheet/dimension' node we can get\n # this from, but some libs (ex. simple_xlsx_writer) don't record it.\n # In that case, we assume the data is of uniform column length and\n # store the column name of the last header row we see. Obviously this\n # isn't the most robust strategy, but it likely fits 99% of use cases\n # considering it's not a problem with actual excel docs.\n @dimension = \"A1:#{@cell_name}\" if @dimension.nil?\n when 'v', 't'\n @current_row[cell_idx] = @captured\n @capture = false\n @captured = nil\n when 'f' then @function = false\n when 'c' then @url = nil\n end\n end\n\n ###\n # /End SAX hooks\n\n def test_headers_hash_against_current_row\n found = false\n\n @current_row.each_with_index do |cell, cell_idx|\n @headers.each_pair do |key, search|\n if search.is_a?(String) ? cell == search : cell&.match?(search)\n found = true\n @current_row[cell_idx] = key\n end\n end\n end\n\n @headers = @current_row if found\n end\n\n def possibly_yield_empty_rows(headers:)\n while @current_row_num && @current_row_num > @last_seen_row_idx + 1\n @last_seen_row_idx += 1\n yield_row(Array.new(column_length), headers: headers)\n end\n end\n\n def yield_row(row, headers:)\n if headers\n @each_callback.call(Hash[@headers.zip(row)])\n else\n @each_callback.call(row)\n end\n end\n\n # This sax-parses the whole sheet, just to extract hyperlink refs at the end.\n def load_gui_hyperlinks\n self.hyperlinks_by_cell =\n HyperlinksParser.parse(@file_io, xrels: xrels)\n end\n\n class HyperlinksParser < Nokogiri::XML::SAX::Document\n def initialize(file_io, xrels:)\n @file_io = file_io\n @xrels = xrels\n end\n\n def self.parse(file_io, xrels:)\n new(file_io, xrels: xrels).parse\n end\n\n def parse\n @hyperlinks_by_cell = {}\n Nokogiri::XML::SAX::Parser.new(self).parse(@file_io)\n @hyperlinks_by_cell\n end\n\n def start_element_namespace(name, attrs, _prefix, _uri, _ns)\n case name\n when 'hyperlink'\n attrs = attrs.inject({}) {|acc, attr| acc[attr.localname] = attr.value; acc}\n id = attrs['id'] || attrs['r:id']\n\n @hyperlinks_by_cell[attrs['ref']] =\n @xrels.at_xpath(%(//*[@Id=\"#{id}\"])).attr('Target')\n end\n end\n end\n\n def xrels\n @xrels ||= Nokogiri::XML(xrels_file.read) if xrels_file\n end\n\n def column_length\n return 0 unless @dimension\n\n @column_length ||= column_letter_to_number(last_cell_letter)\n end\n\n def cell_idx\n column_letter_to_number(@cell_name.scan(/[A-Z]+/).first) - 1\n end\n\n ##\n # Returns the last column name, ex. 'E'\n def last_cell_letter\n return unless @dimension\n\n @dimension.scan(/:([A-Z]+)/)&.first&.first || 'A'\n end\n\n # formula fits an exponential factorial function of the form:\n # 'A' = 1\n # 'B' = 2\n # 'Z' = 26\n # 'AA' = 26 * 1 + 1\n # 'AZ' = 26 * 1 + 26\n # 'BA' = 26 * 2 + 1\n # 'ZA' = 26 * 26 + 1\n # 'ZZ' = 26 * 26 + 26\n # 'AAA' = 26 * 26 * 1 + 26 * 1 + 1\n # 'AAZ' = 26 * 26 * 1 + 26 * 1 + 26\n # 'ABA' = 26 * 26 * 1 + 26 * 2 + 1\n # 'BZA' = 26 * 26 * 2 + 26 * 26 + 1\n def column_letter_to_number(column_letter)\n pow = column_letter.length - 1\n result = 0\n column_letter.each_byte do |b|\n result += 26**pow * (b - 64)\n pow -= 1\n end\n result\n end\n\n def column_number_to_letter(n)\n result = []\n loop do\n result.unshift((n % 26 + 65).chr)\n n = (n / 26) - 1\n break if n < 0\n end\n result.join\n end\n end\n end\nend\n" }, { "path": "lib/simple_xlsx_reader/loader/style_types_parser.rb", "content": "# frozen_string_literal: true\n\nmodule SimpleXlsxReader\n class Loader\n StyleTypesParser = Struct.new(:file_io) do\n def self.parse(file_io)\n new(file_io).tap(&:parse).style_types\n end\n\n # Map of non-custom numFmtId to casting symbol\n NumFmtMap = {\n 0 => :string, # General\n 1 => :fixnum, # 0\n 2 => :float, # 0.00\n 3 => :fixnum, # #,##0\n 4 => :float, # #,##0.00\n 5 => :unsupported, # $#,##0_);($#,##0)\n 6 => :unsupported, # $#,##0_);[Red]($#,##0)\n 7 => :unsupported, # $#,##0.00_);($#,##0.00)\n 8 => :unsupported, # $#,##0.00_);[Red]($#,##0.00)\n 9 => :percentage, # 0%\n 10 => :percentage, # 0.00%\n 11 => :bignum, # 0.00E+00\n 12 => :unsupported, # # ?/?\n 13 => :unsupported, # # ??/??\n 14 => :date, # mm-dd-yy\n 15 => :date, # d-mmm-yy\n 16 => :date, # d-mmm\n 17 => :date, # mmm-yy\n 18 => :time, # h:mm AM/PM\n 19 => :time, # h:mm:ss AM/PM\n 20 => :time, # h:mm\n 21 => :time, # h:mm:ss\n 22 => :date_time, # m/d/yy h:mm\n 37 => :unsupported, # #,##0 ;(#,##0)\n 38 => :unsupported, # #,##0 ;[Red](#,##0)\n 39 => :unsupported, # #,##0.00;(#,##0.00)\n 40 => :unsupported, # #,##0.00;[Red](#,##0.00)\n 44 => :float, # some odd currency format ?from Office 2007?\n 45 => :time, # mm:ss\n 46 => :time, # [h]:mm:ss\n 47 => :time, # mmss.0\n 48 => :bignum, # ##0.0E+0\n 49 => :unsupported # @\n }.freeze\n\n def parse\n @xml = Nokogiri::XML(file_io.read).remove_namespaces!\n end\n\n # Excel doesn't record types for some cells, only its display style, so\n # we have to back out the type from that style.\n #\n # Some of these styles can be determined from a known set (see NumFmtMap),\n # while others are 'custom' and we have to make a best guess.\n #\n # This is the array of types corresponding to the styles a spreadsheet\n # uses, and includes both the known style types and the custom styles.\n #\n # Note that the xml sheet cells that use this don't reference the\n # numFmtId, but instead the array index of a style in the stored list of\n # only the styles used in the spreadsheet (which can be either known or\n # custom). Hence this style types array, rather than a map of numFmtId to\n # type.\n def style_types\n @xml.xpath('/styleSheet/cellXfs/xf').map do |xstyle|\n style_type_by_num_fmt_id(\n xstyle.attributes['numFmtId']&.value\n )\n end\n end\n\n # Finds the type we think a style is; For example, fmtId 14 is a date\n # style, so this would return :date.\n #\n # Note, custom styles usually (are supposed to?) have a numFmtId >= 164,\n # but in practice can sometimes be simply out of the usual \"Any Language\"\n # id range that goes up to 49. For example, I have seen a numFmtId of\n # 59 specified as a date. In Thai, 59 is a number format, so this seems\n # like a bad idea, but we try to be flexible and just go with it.\n def style_type_by_num_fmt_id(id)\n return nil if id.nil?\n\n id = id.to_i\n NumFmtMap[id] || custom_style_types[id]\n end\n\n # Map of (numFmtId >= 164) (custom styles) to our best guess at the type\n # ex. {164 => :date_time}\n def custom_style_types\n @custom_style_types ||=\n @xml.xpath('/styleSheet/numFmts/numFmt')\n .each_with_object({}) do |xstyle, acc|\n acc[xstyle.attributes['numFmtId'].value.to_i] =\n determine_custom_style_type(xstyle.attributes['formatCode'].value)\n end\n end\n\n # This is the least deterministic part of reading xlsx files. Due to\n # custom styles, you can't know for sure when a date is a date other than\n # looking at its format and gessing. It's not impossible to guess right,\n # though.\n #\n # http://stackoverflow.com/questions/4948998/determining-if-an-xlsx-cell-is-date-formatted-for-excel-2007-spreadsheets\n def determine_custom_style_type(string)\n return :float if string[0] == '_'\n return :float if string[0] == ' 0'\n\n # Looks for one of ymdhis outside of meta-stuff like [Red]\n return :date_time if string =~ /(^|\\])[^\\[]*[ymdhis]/i\n\n :unsupported\n end\n end\n end\nend\n" }, { "path": "lib/simple_xlsx_reader/loader/workbook_parser.rb", "content": "# frozen_string_literal: true\n\nmodule SimpleXlsxReader\n class Loader\n WorkbookParser = Struct.new(:file_io) do\n def self.parse(file_io)\n parser = new(file_io).tap(&:parse)\n [parser.sheet_toc, parser.base_date]\n end\n\n def parse\n @xml = Nokogiri::XML(file_io.read).remove_namespaces!\n end\n\n # Table of contents for the sheets, ex. {'Authors' => 0, ...}\n def sheet_toc\n @xml.xpath('/workbook/sheets/sheet')\n .each_with_object({}) do |sheet, acc|\n acc[sheet.attributes['name'].value] =\n sheet.attributes['sheetId'].value.to_i - 1 # keep things 0-indexed\n end\n end\n\n ## Returns the base_date from which to calculate dates.\n # Defaults to 1900 (minus two days due to excel quirk), but use 1904 if\n # it's set in the Workbook's workbookPr.\n # http://msdn.microsoft.com/en-us/library/ff530155(v=office.12).aspx\n def base_date\n return DATE_SYSTEM_1900 if @xml.nil?\n\n @xml.xpath('//workbook/workbookPr[@date1904]').each do |workbookPr|\n return DATE_SYSTEM_1904 if workbookPr['date1904'] =~ /true|1/i\n end\n\n DATE_SYSTEM_1900\n end\n end\n end\nend\n" }, { "path": "lib/simple_xlsx_reader/loader.rb", "content": "# frozen_string_literal: true\n\nmodule SimpleXlsxReader\n class Loader < Struct.new(:string_or_io)\n attr_accessor :shared_strings, :sheet_parsers, :sheet_toc, :style_types, :base_date\n\n def init_sheets\n ZipReader.new(\n string_or_io: string_or_io,\n loader: self\n ).read\n\n sheet_toc.each_with_index.map do |(sheet_name, _sheet_number), i|\n # sheet_number is *not* the index into xml.sheet_parsers\n SimpleXlsxReader::Document::Sheet.new(\n name: sheet_name,\n sheet_parser: sheet_parsers[i]\n )\n end\n end\n\n ZipReader = Struct.new(:string_or_io, :loader, keyword_init: true) do\n attr_reader :zip\n\n def initialize(*args)\n super\n @zip = SimpleXlsxReader::Zip.open_buffer(string_or_io)\n end\n\n def read\n entry_at('xl/workbook.xml') do |file_io|\n loader.sheet_toc, loader.base_date = *WorkbookParser.parse(file_io)\n end\n\n entry_at('xl/styles.xml') do |file_io|\n loader.style_types = StyleTypesParser.parse(file_io)\n end\n\n # optional feature used by excel,\n # but not often used by xlsx generation libraries\n if (ss_entry = entry_at('xl/sharedStrings.xml'))\n ss_entry.get_input_stream do |file|\n loader.shared_strings = SharedStringsParser.parse(file)\n end\n else\n loader.shared_strings = []\n end\n\n loader.sheet_parsers = []\n\n # Sometimes there's a zero-index sheet.xml, ex.\n # Google Docs creates:\n # xl/worksheets/sheet.xml\n # xl/worksheets/sheet1.xml\n # xl/worksheets/sheet2.xml\n # While Excel creates:\n # xl/worksheets/sheet1.xml\n # xl/worksheets/sheet2.xml\n add_sheet_parser_at_index(nil)\n\n i = 1\n while(add_sheet_parser_at_index(i)) do\n i += 1\n end\n end\n\n def entry_at(path, &block)\n # Older and newer (post-mid-2021) RubyZip normalizes pathnames,\n # but unfortunately there is a time in between where it doesn't.\n # Rather than require a specific version, let's just be flexible.\n entry =\n zip.find_entry(path) || # *nix-generated\n zip.find_entry(path.tr('/', '\\\\')) || # Windows-generated\n zip.find_entry(path.downcase) || # Sometimes it's lowercase\n zip.find_entry(path.tr('/', '\\\\').downcase) # Sometimes it's lowercase\n\n if block\n entry.get_input_stream(&block)\n else\n entry\n end\n end\n\n def add_sheet_parser_at_index(i)\n sheet_file_name = \"xl/worksheets/sheet#{i}.xml\"\n return unless (entry = entry_at(sheet_file_name))\n\n parser =\n SheetParser.new(\n file_io: entry.get_input_stream,\n loader: loader\n )\n\n relationship_file_name = \"xl/worksheets/_rels/sheet#{i}.xml.rels\"\n if (rel = entry_at(relationship_file_name))\n parser.xrels_file = rel.get_input_stream\n end\n\n loader.sheet_parsers << parser\n end\n end\n\n ##\n # The heart of typecasting. The ruby type is determined either explicitly\n # from the cell xml or implicitly from the cell style, and this\n # method expects that work to have been done already. This, then,\n # takes the type we determined it to be and casts the cell value\n # to that type.\n #\n # types:\n # - s: shared string (see #shared_string)\n # - n: number (cast to a float)\n # - b: boolean\n # - str: string\n # - inlineStr: string\n # - ruby symbol: for when type has been determined by style\n #\n # options:\n # - shared_strings: needed for 's' (shared string) type\n def self.cast(value, type, style, options = {})\n return nil if value.nil? || value.empty?\n\n # Sometimes the type is dictated by the style alone\n if type.nil? ||\n (type == 'n' && %i[date time date_time].include?(style))\n type = style\n end\n\n casted =\n case type\n\n ##\n # There are few built-in types\n ##\n\n when 's' # shared string\n options[:shared_strings][value.to_i]\n when 'n' # number\n value.to_f\n when 'b'\n value.to_i == 1\n when 'str'\n value\n when 'inlineStr'\n value\n\n ##\n # Type can also be determined by a style,\n # detected earlier and cast here by its standardized symbol\n ##\n\n # no type encoded with the the General format defaults to a number type\n when nil, :string\n retval = Integer(value, exception: false)\n retval ||= Float(value, exception: false)\n retval ||= value\n retval\n when :unsupported\n value\n when :fixnum\n value.to_i\n when :float\n value.to_f\n when :percentage\n value.to_f\n # the trickiest. note that all these formats can vary on\n # whether they actually contain a date, time, or datetime.\n when :date, :time, :date_time\n value = Float(value)\n days_since_date_system_start = value.to_i\n fraction_of_24 = value - days_since_date_system_start\n\n # http://stackoverflow.com/questions/10559767/how-to-convert-ms-excel-date-from-float-to-date-format-in-ruby\n date = options.fetch(:base_date, DATE_SYSTEM_1900) + days_since_date_system_start\n\n if fraction_of_24 > 0 # there is a time associated\n seconds = (fraction_of_24 * 86_400).round\n return Time.utc(date.year, date.month, date.day) + seconds\n else\n return date\n end\n when :bignum\n if defined?(BigDecimal)\n BigDecimal(value)\n else\n value.to_f\n end\n\n ##\n # Beats me\n ##\n\n else\n value\n end\n\n if options[:url]\n Hyperlink.new(options[:url], casted)\n else\n casted\n end\n end\n end\nend\n\n" }, { "path": "lib/simple_xlsx_reader/version.rb", "content": "# frozen_string_literal: true\n\nmodule SimpleXlsxReader\n VERSION = '5.1.0'\nend\n" }, { "path": "lib/simple_xlsx_reader.rb", "content": "# frozen_string_literal: true\n\nrequire 'nokogiri'\nrequire 'date'\n\nrequire 'simple_xlsx_reader/version'\nrequire 'simple_xlsx_reader/hyperlink'\nrequire 'simple_xlsx_reader/document'\nrequire 'simple_xlsx_reader/loader'\nrequire 'simple_xlsx_reader/loader/workbook_parser'\nrequire 'simple_xlsx_reader/loader/shared_strings_parser'\nrequire 'simple_xlsx_reader/loader/sheet_parser'\nrequire 'simple_xlsx_reader/loader/style_types_parser'\n\n\n# Rubyzip 1.0 only has different naming, everything else is the same, so let's\n# be flexible so we don't force people into a dependency hell w/ other gems.\nbegin\n # Try loading rubyzip < 1.0\n require 'zip/zip'\n require 'zip/zipfilesystem'\n SimpleXlsxReader::Zip = Zip::ZipFile\nrescue LoadError\n # Try loading rubyzip >= 1.0\n require 'zip'\n require 'zip/filesystem'\n SimpleXlsxReader::Zip = Zip::File\nend\n\nmodule SimpleXlsxReader\n DATE_SYSTEM_1900 = Date.new(1899, 12, 30)\n DATE_SYSTEM_1904 = Date.new(1904, 1, 1)\n\n class CellLoadError < StandardError; end\n\n class << self\n def configuration\n @configuration ||= Struct.new(:catch_cell_load_errors, :auto_slurp).new.tap do |c|\n c.catch_cell_load_errors = false\n c.auto_slurp = false\n end\n end\n\n def open(file_path)\n Document.new(file_path: file_path).tap(&:sheets)\n end\n \n def parse(string_or_io)\n Document.new(string_or_io: string_or_io).tap(&:sheets)\n end\n end\nend\n" }, { "path": "simple_xlsx_reader.gemspec", "content": "# -*- encoding: utf-8 -*-\nlib = File.expand_path('../lib', __FILE__)\n$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)\nrequire 'simple_xlsx_reader/version'\n\nGem::Specification.new do |gem|\n gem.name = \"simple_xlsx_reader\"\n gem.version = SimpleXlsxReader::VERSION\n gem.authors = [\"Woody Peterson\"]\n gem.email = [\"woody.peterson@gmail.com\"]\n gem.description = %q{Read xlsx data the Ruby way}\n gem.summary = %q{Read xlsx data the Ruby way}\n gem.homepage = \"\"\n gem.license = \"MIT\"\n\n gem.add_dependency 'nokogiri'\n gem.add_dependency 'rubyzip'\n\n gem.add_development_dependency 'minitest', '>= 5.0'\n gem.add_development_dependency 'rake'\n gem.add_development_dependency 'pry'\n\n gem.files = `git ls-files`.split($/)\n gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }\n gem.test_files = gem.files.grep(%r{^test/})\n gem.require_paths = [\"lib\"]\nend\n" }, { "path": "test/date1904_test.rb", "content": "# frozen_string_literal: true\n\nrequire_relative 'test_helper'\n\ndescribe SimpleXlsxReader do\n let(:date1904_file) { File.join(File.dirname(__FILE__), 'date1904.xlsx') }\n let(:subject) { SimpleXlsxReader::Document.new(date1904_file) }\n\n it 'supports converting dates with the 1904 date system' do\n _(subject.to_hash).must_equal(\n 'date1904' => [[Date.parse('2014-05-01')]]\n )\n end\nend\n" }, { "path": "test/datetime_test.rb", "content": "# frozen_string_literal: true\n\nrequire_relative 'test_helper'\n\ndescribe SimpleXlsxReader do\n let(:datetimes_file) do\n File.join(\n File.dirname(__FILE__),\n 'datetimes.xlsx'\n )\n end\n\n let(:subject) { SimpleXlsxReader::Document.new(datetimes_file) }\n\n it 'converts date_times with the correct precision' do\n _(subject.to_hash).must_equal(\n 'Datetimes' =>\n [\n [Time.parse('2013-08-19 18:29:59 UTC')],\n [Time.parse('2013-08-19 18:30:00 UTC')],\n [Time.parse('2013-08-19 18:30:01 UTC')],\n [Time.parse('1899-12-30 00:30:00 UTC')]\n ]\n )\n end\nend\n" }, { "path": "test/gdocs_sheet_test.rb", "content": "# frozen_string_literal: true\n\nrequire_relative 'test_helper'\nrequire 'time'\n\ndescribe SimpleXlsxReader do\n let(:one_sheet_file) { File.join(File.dirname(__FILE__), 'gdocs_sheet.xlsx') }\n let(:subject) { SimpleXlsxReader::Document.new(one_sheet_file) }\n\n it 'able to load file from google docs' do\n _(subject.to_hash).must_equal(\n 'List 1' => [['Empty gdocs list 1']],\n 'List 2' => [['Empty gdocs list 2']]\n )\n end\nend\n" }, { "path": "test/lower_case_sharedstrings_test.rb", "content": "# frozen_string_literal: true\n\nrequire_relative 'test_helper'\n\ndescribe SimpleXlsxReader do\n let(:lower_case_shared_strings) do\n File.join(\n File.dirname(__FILE__),\n 'lower_case_sharedstrings.xlsx'\n )\n end\n\n let(:subject) { SimpleXlsxReader::Document.new(lower_case_shared_strings) }\n\n describe '#to_hash' do\n it 'should have the word Well in the first row' do\n _(subject.sheets.first.rows.to_a[0]).must_include('Well')\n end\n end\nend\n" }, { "path": "test/namespaces_and_missing_atts_test.rb", "content": "# frozen_string_literal: true\n\nrequire_relative 'test_helper'\n\ndescribe SimpleXlsxReader do\n # Based on a real-world sheet possibly generated by PowerBI, where the xml\n # has namespacing and rows are missing the 'r' attribute.\n let(:sheet) do\n <<~XML\n \n \n \n \n \n \n Salmon\n \n \n \n \n Trout\n \n \n \n \n \n \n Cat\n \n \n \n \n Dog\n \n \n \n \n \n XML\n end\n\n let(:styles) do\n <<~XML\n \n XML\n end\n\n let(:wonky_file) do\n TestXlsxBuilder.new(\n sheets: [sheet],\n styles: styles\n )\n end\n\n let(:subject) { SimpleXlsxReader::Document.new(wonky_file.archive.path) }\n\n describe '#to_hash' do\n it 'should extract values from namespaced cells missing \"r\" attributes' do\n _(subject.sheets.first.rows.to_a[0]).must_include('Salmon')\n _(subject.sheets.first.rows.to_a[1]).must_include('Dog')\n end\n end\nend\n" }, { "path": "test/performance_test.rb", "content": "# frozen_string_literal: true\n\nrequire_relative 'test_helper'\nrequire 'minitest/benchmark'\n\ndescribe 'SimpleXlsxReader Benchmark' do\n # n is 0-indexed for us, then converted to 1-indexed for excel\n def sheet_with_n_rows(row_count)\n acc = +\"\"\n acc <<\n <<~XML\n \n \n XML\n\n row_count.times.each do |n|\n n += 1\n acc <<\n <<~XML\n \n \n Cell A#{n}\n \n \n 2.4\n \n \n 30687\n \n \n Cell D#{n}\n \n\n \n Cell E#{n}\n \n \n 2.4\n \n \n 30687\n \n \n Cell H#{n}\n \n\n \n Cell I#{n}\n \n \n 2.4\n \n \n 30687\n \n \n Cell L#{n}\n \n \n XML\n end\n\n acc <<\n <<~XML\n \n \n XML\n end\n\n let(:styles) do\n # s='0' above refers to the value of numFmtId at cellXfs index 0,\n # which is in this case 'General' type\n _styles =\n <<-XML\n \n \n \n \n \n \n \n XML\n end\n\n before do\n @xlsxs = {}\n\n # Every new sheet has one more row\n self.class.bench_range.each do |num_rows|\n @xlsxs[num_rows] =\n TestXlsxBuilder.new(\n sheets: [sheet_with_n_rows(num_rows)],\n styles: styles\n ).archive\n end\n end\n\n def self.bench_range\n # Works out to a max just shy of 265k rows, which takes ~20s on my M1 Mac.\n # Second-largest is ~65k rows @ ~5s.\n max = ENV['BIG_PERF_TEST'] ? 265_000 : 66_000\n bench_exp(100, max, 4)\n end\n\n bench_performance_linear 'parses sheets in linear time', 0.999 do |n|\n SimpleXlsxReader.open(@xlsxs[n].path).sheets[0].rows.each(headers: true) {|_row| }\n end\nend\n" }, { "path": "test/shared_strings.xml", "content": "\n \n Cell A1\n \n \n Cell B1\n \n \n My Cell\n \n \n \n \n \n \n \n \n \n \n Cell\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n A2\n \n \n \n \n \n \n \n \n \n \n \n Cell\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n B2\n \n \n \n Cell Fmt\n \n \n ’ When it sees a unicode character (such as the fancy apostrophe starting this sentence), it starts chunking the stream for at least the current node, and we have to keep consuming the characters until we hit the end of the text. We can't assume that the string first given by the SAX callback us is the whole shared string content. It only happens with both unicode *and* really long text.\n \n \n\n" }, { "path": "test/simple_xlsx_reader_test.rb", "content": "# frozen_string_literal: true\n\nrequire_relative 'test_helper'\nrequire 'time'\n\nSXR = SimpleXlsxReader\n\ndescribe SimpleXlsxReader do\n let(:sesame_street_blog_file) do\n File.join(File.dirname(__FILE__), 'sesame_street_blog.xlsx')\n end\n\n let(:document) { SimpleXlsxReader.open(sesame_street_blog_file) }\n\n ##\n # A high-level acceptance test testing misc features such as date/time parsing,\n # hyperlinks (both function and ref kinds), formula dates, emty rows, etc.\n\n let(:sesame_street_blog_file_path) { File.join(File.dirname(__FILE__), 'sesame_street_blog.xlsx') }\n let(:sesame_street_blog_io) { File.new(sesame_street_blog_file_path) }\n let(:sesame_street_blog_string) { IO.read(sesame_street_blog_file_path) }\n\n let(:expected_result) do\n {\n 'Authors' =>\n [\n ['Name', 'Occupation'],\n ['Big Bird', 'Teacher']\n ],\n 'Posts' =>\n [\n ['Author Name', 'Title', 'Body', 'Created At', 'Comment Count', 'URL'],\n ['Big Bird', 'The Number 1', 'The Greatest', Time.parse('2002-01-01 11:00:00 UTC'), 1, SXR::Hyperlink.new('http://www.example.com/hyperlink-function', 'This uses the HYPERLINK() function')],\n ['Big Bird', 'The Number 2', 'Second Best', Time.parse('2002-01-02 14:00:00 UTC'), 2, SXR::Hyperlink.new('http://www.example.com/hyperlink-gui', 'This uses the hyperlink GUI option')],\n ['Big Bird', 'Formula Dates', 'Tricky tricky', Time.parse('2002-01-03 14:00:00 UTC'), 0, nil],\n ['Empty Eagress', nil, 'The title, date, and comment have types, but no values', nil, nil, nil]\n ]\n }\n end\n\n describe SimpleXlsxReader do\n describe 'load from file path' do\n let(:subject) { SimpleXlsxReader.open(sesame_street_blog_file_path) }\n\n it 'reads an xlsx file into a hash of {[sheet name] => [data]}' do\n _(subject.to_hash).must_equal(expected_result)\n end\n end\n\n describe 'load from buffer' do\n let(:subject) { SimpleXlsxReader.parse(sesame_street_blog_io) }\n\n it 'reads an xlsx buffer into a hash of {[sheet name] => [data]}' do\n _(subject.to_hash).must_equal(expected_result)\n end\n end\n\n describe 'load from string' do\n let(:subject) { SimpleXlsxReader.parse(sesame_street_blog_string) }\n\n it 'reads an xlsx string into a hash of {[sheet name] => [data]}' do\n _(subject.to_hash).must_equal(expected_result)\n end\n end\n\n it 'outputs strings in UTF-8 encoding' do\n document = SimpleXlsxReader.parse(sesame_street_blog_io)\n _(document.sheets[0].rows.to_a.flatten.map(&:encoding).uniq)\n .must_equal [Encoding::UTF_8]\n end\n\n it 'can use all our enumerable nicities without slurping' do\n document = SimpleXlsxReader.parse(sesame_street_blog_io)\n\n headers = {\n name: 'Author Name',\n title: 'Title',\n body: 'Body',\n created_at: 'Created At',\n count: /Count/\n }\n\n rows = document.sheets[1].rows\n result =\n rows.each(headers: headers).with_index.with_object({}) do |(row, i), acc|\n acc[i] = row\n end\n\n _(result[0]).must_equal(\n name: 'Big Bird',\n title: 'The Number 1',\n body: 'The Greatest',\n created_at: Time.parse('2002-01-01 11:00:00 UTC'),\n count: 1,\n \"URL\" => 'This uses the HYPERLINK() function'\n )\n\n _(rows.slurped?).must_equal false\n end\n end\n\n ##\n # For more fine-grained unit tests, we sometimes build our own workbook via\n # Nokogiri. TestXlsxBuilder has some defaults, and this let-style lets us\n # concisely override them in nested describe blocks.\n\n let(:shared_strings) { nil }\n let(:styles) { nil }\n let(:sheet) { nil }\n let(:workbook) { nil }\n let(:rels) { nil }\n\n let(:xlsx) do\n TestXlsxBuilder.new(\n shared_strings: shared_strings,\n styles: styles,\n sheets: sheet && [sheet],\n workbook: workbook,\n rels: rels\n )\n end\n\n let(:reader) { SimpleXlsxReader.open(xlsx.archive.path) }\n\n describe 'when parsing escaped characters' do\n let(:escaped_content) do\n '<a href=\"https://www.example.com\">Link A</a> &bull; <a href=\"https://www.example.com\">Link B</a>'\n end\n\n let(:unescaped_content) do\n 'Link A • Link B'\n end\n\n let(:sheet) do\n <<~XML\n \n \n \n \n \n 0\n \n \n #{escaped_content}\n \n \n \n \n XML\n end\n\n let(:shared_strings) do\n <<~XML\n \n \n #{escaped_content}\n \n \n XML\n end\n\n it 'loads correctly using inline strings' do\n _(reader.sheets[0].rows.slurp[0][0]).must_equal(unescaped_content)\n end\n\n it 'loads correctly using shared strings' do\n _(reader.sheets[0].rows.slurp[0][1]).must_equal(unescaped_content)\n end\n end\n\n describe 'Sheet#rows#each(headers: true)' do\n let(:sheet) do\n <<~XML\n \n \n \n \n \n Header 1\n \n \n Header 2\n \n \n \n \n Data 1-A\n \n \n Data 1-B\n \n \n \n \n Data 2-A\n \n \n Data 2-B\n \n \n \n \n XML\n end\n\n it 'yields rows as hashes' do\n acc = []\n\n reader.sheets[0].rows.each(headers: true) do |row|\n acc << row\n end\n\n _(acc).must_equal(\n [\n { 'Header 1' => 'Data 1-A', 'Header 2' => 'Data 1-B' },\n { 'Header 1' => nil, 'Header 2' => nil },\n { 'Header 1' => 'Data 2-A', 'Header 2' => 'Data 2-B' }\n ]\n )\n end\n end\n\n describe 'Sheet#rows#each(headers: ->(row) {...})' do\n let(:sheet) do\n <<~XML\n \n \n \n \n \n a chart or something\n \n \n Rabble rabble\n \n \n \n \n Chatty junk\n \n \n \n \n \n \n \n Header 1\n \n \n Header 2\n \n \n \n \n Data 1-A\n \n \n Data 1-B\n \n \n \n \n Data 2-A\n \n \n Data 2-B\n \n \n \n \n XML\n end\n\n it 'yields rows as hashes' do\n acc = []\n\n finder = ->(row) { row.find {|c| c&.match(/Header/)} }\n reader.sheets[0].rows.each(headers: finder) do |row|\n acc << row\n end\n\n _(acc).must_equal(\n [\n { 'Header 1' => 'Data 1-A', 'Header 2' => 'Data 1-B' },\n { 'Header 1' => nil, 'Header 2' => nil },\n { 'Header 1' => 'Data 2-A', 'Header 2' => 'Data 2-B' }\n ]\n )\n end\n end\n\n describe \"Sheet#rows#each(headers: a_hash)\" do\n let(:sheet) do\n Nokogiri::XML(\n <<~XML\n \n \n \n \n \n a chart or something\n \n \n Rabble rabble\n \n \n Rabble rabble\n \n \n \n \n Chatty junk\n \n \n \n \n \n \n \n \n \n \n ID Number\n \n \n ExacT\n \n \n FOO Name\n \n\n \n \n \n ID 1-A\n \n \n Exact 1-B\n \n \n Name 1-C\n \n \n \n \n ID 2-A\n \n \n Exact 2-B\n \n \n Name 2-C\n \n \n \n \n XML\n )\n end\n\n it 'transforms headers into symbols based on the header map' do\n header_map = {id: /ID/, name: /foo/i, exact: 'ExacT'}\n result = reader.sheets[0].rows.each(headers: header_map).to_a\n\n _(result).must_equal(\n [\n { id: 'ID 1-A', exact: 'Exact 1-B', name: 'Name 1-C' },\n { id: nil, exact: nil, name: nil },\n { id: 'ID 2-A', exact: 'Exact 2-B', name: 'Name 2-C' },\n ]\n )\n end\n\n it 'if a match isnt found, uses un-matched header name' do\n sheet.xpath(\"//*[text() = 'ExacT']\")\n .first.children.first.content = 'not ExacT'\n\n header_map = {id: /ID/, name: /foo/i, exact: 'ExacT'}\n result = reader.sheets[0].rows.each(headers: header_map).to_a\n\n _(result).must_equal(\n [\n { id: 'ID 1-A', 'not ExacT' => 'Exact 1-B', name: 'Name 1-C' },\n { id: nil, 'not ExacT' => nil, name: nil },\n { id: 'ID 2-A', 'not ExacT' => 'Exact 2-B', name: 'Name 2-C' },\n ]\n )\n end\n end\n\n describe 'Sheet#rows[]' do\n it 'raises a RuntimeError if rows not slurped yet' do\n _(-> { reader.sheets[0].rows[1] }).must_raise(RuntimeError)\n end\n\n it 'works if the rows have been slurped' do\n _(reader.sheets[0].rows.tap(&:slurp)[0]).must_equal(\n ['Cell A', 'Cell B', 'Cell C']\n )\n end\n\n it 'works if the config allows auto slurping' do\n SimpleXlsxReader.configuration.auto_slurp = true\n\n _(reader.sheets[0].rows[0]).must_equal(\n ['Cell A', 'Cell B', 'Cell C']\n )\n\n SimpleXlsxReader.configuration.auto_slurp = false\n end\n end\n\n describe 'Sheet#rows#slurp' do\n let(:rows) { reader.sheets[0].rows.tap(&:slurp) }\n\n it 'loads the sheet parser results into memory' do\n _(rows.slurped).must_equal(\n [['Cell A', 'Cell B', 'Cell C']]\n )\n end\n\n it '#each and #map use slurped results' do\n _(rows.map(&:reverse)).must_equal(\n [['Cell C', 'Cell B', 'Cell A']]\n )\n end\n end\n\n describe 'Sheet#rows#each' do\n let(:sheet) do\n <<~XML\n \n \n \n \n \n Header 1\n \n \n Header 2\n \n \n \n \n Data 1-A\n \n \n Data 1-B\n \n \n \n \n Data 2-A\n \n \n Data 2-B\n \n \n \n \n XML\n end\n\n let(:rows) { reader.sheets[0].rows }\n\n it 'with no block, returns an enumerator when not slurped' do\n _(rows.each.class).must_equal Enumerator\n end\n\n it 'with no block, passes on header argument in enumerator' do\n _(rows.each(headers: true).inspect).must_match 'headers: true'\n end\n\n it 'returns an enumerator when slurped' do\n rows.slurp\n _(rows.each.class).must_equal Enumerator\n end\n end\n\n describe 'Sheet#rows#map' do\n let(:sheet) do\n <<~XML\n \n \n \n \n \n Header 1\n \n \n Header 2\n \n \n \n \n Data 1-A\n \n \n Data 1-B\n \n \n \n \n Data 2-A\n \n \n Data 2-B\n \n \n \n \n XML\n end\n\n let(:rows) { reader.sheets[0].rows }\n\n it 'does not slurp' do\n _(rows.map(&:first)).must_equal(\n [\"Header 1\", \"Data 1-A\", nil, \"Data 2-A\"]\n )\n _(rows.slurped?).must_equal false\n end\n end\n\n describe 'Sheet#headers' do\n let(:doc_sheet) { reader.sheets[0] }\n\n it 'raises a RuntimeError if rows not slurped yet' do\n _(-> { doc_sheet.headers }).must_raise(RuntimeError)\n end\n\n it 'returns first row if slurped' do\n _(doc_sheet.tap(&:slurp).headers).must_equal(\n ['Cell A', 'Cell B', 'Cell C']\n )\n end\n\n it 'returns first row if auto_slurp' do\n SimpleXlsxReader.configuration.auto_slurp = true\n\n _(doc_sheet.headers).must_equal(\n ['Cell A', 'Cell B', 'Cell C']\n )\n\n SimpleXlsxReader.configuration.auto_slurp = false\n end\n end\n\n describe SimpleXlsxReader::Loader do\n let(:described_class) { SimpleXlsxReader::Loader }\n\n describe '::cast' do\n it 'reads type s as a shared string' do\n _(described_class.cast('1', 's', nil, shared_strings: %w[a b c]))\n .must_equal 'b'\n end\n\n it 'reads type inlineStr as a string' do\n _(described_class.cast('the value', nil, 'inlineStr'))\n .must_equal 'the value'\n end\n\n it 'reads date styles' do\n _(described_class.cast('41505', nil, :date))\n .must_equal Date.parse('2013-08-19')\n end\n\n it 'reads time styles' do\n _(described_class.cast('41505.77083', nil, :time))\n .must_equal Time.parse('2013-08-19 18:30 UTC')\n end\n\n it 'reads date_time styles' do\n _(described_class.cast('41505.77083', nil, :date_time))\n .must_equal Time.parse('2013-08-19 18:30 UTC')\n end\n\n it 'reads number types styled as dates' do\n _(described_class.cast('41505', 'n', :date))\n .must_equal Date.parse('2013-08-19')\n end\n\n it 'reads number types styled as times' do\n _(described_class.cast('41505.77083', 'n', :time))\n .must_equal Time.parse('2013-08-19 18:30 UTC')\n end\n\n it 'reads less-than-zero complex number types styled as times' do\n _(described_class.cast('6.25E-2', 'n', :time))\n .must_equal Time.parse('1899-12-30 01:30:00 UTC')\n end\n\n it 'reads number types styled as date_times' do\n _(described_class.cast('41505.77083', 'n', :date_time))\n .must_equal Time.parse('2013-08-19 18:30 UTC')\n end\n\n it 'raises when date-styled values are not numerical' do\n _(-> { described_class.cast('14 is not a valid date', nil, :date) })\n .must_raise(ArgumentError)\n end\n\n describe 'with the url option' do\n let(:url) { 'http://www.example.com/hyperlink' }\n it 'creates a hyperlink with a string type' do\n _(described_class.cast('A link', 'str', :string, url: url))\n .must_equal SXR::Hyperlink.new(url, 'A link')\n end\n\n it 'creates a hyperlink with a shared string type' do\n _(described_class.cast('2', 's', nil, shared_strings: %w[a b c], url: url))\n .must_equal SXR::Hyperlink.new(url, 'c')\n end\n\n it 'creates a hyperlink with a fixnum friendly_name' do\n _(described_class.cast('123', nil, :fixnum, url: url))\n .must_equal SXR::Hyperlink.new(url, '123')\n end\n end\n end\n\n describe 'shared_strings' do\n let(:xml) do\n File.open(File.join(File.dirname(__FILE__), 'shared_strings.xml'))\n end\n\n let(:ss) { SimpleXlsxReader::Loader::SharedStringsParser.parse(xml) }\n\n it 'parses strings formatted at the cell level' do\n _(ss[0..2]).must_equal ['Cell A1', 'Cell B1', 'My Cell']\n end\n\n it 'parses strings formatted at the character level' do\n _(ss[3..5]).must_equal ['Cell A2', 'Cell B2', 'Cell Fmt']\n end\n\n it 'parses looong strings containing unicode' do\n _(ss[6]).must_include 'It only happens with both unicode *and* really long text.'\n end\n end\n\n describe 'style_types' do\n let(:xml_file) do\n File.open(File.join(File.dirname(__FILE__), 'styles.xml'))\n end\n\n let(:parser) do\n SimpleXlsxReader::Loader::StyleTypesParser.new(xml_file).tap(&:parse)\n end\n\n it 'reads custom formatted styles (numFmtId >= 164)' do\n _(parser.style_types[1]).must_equal :date_time\n _(parser.custom_style_types[164]).must_equal :date_time\n end\n\n # something I've seen in the wild; don't think it's correct, but let's be flexible.\n it 'reads custom formatted styles given an id < 164, but not explicitly defined in the SpreadsheetML spec' do\n _(parser.style_types[2]).must_equal :date_time\n _(parser.custom_style_types[59]).must_equal :date_time\n end\n end\n\n describe '#last_cell_label' do\n # Note, this is not a valid sheet, since the last cell is actually D1 but\n # the dimension specifies C1. This is just for testing.\n let(:sheet) do\n Nokogiri::XML(\n <<-XML\n \n \n \n \n \n Cell A\n \n \n Cell C\n \n \n Cell D\n \n \n \n \n XML\n ).remove_namespaces!\n end\n\n let(:loader) do\n SimpleXlsxReader::Loader.new(nil).tap do |l|\n l.shared_strings = []\n l.sheet_toc = { 'Sheet1': 0 }\n l.style_types = []\n l.base_date = SimpleXlsxReader::DATE_SYSTEM_1900\n end\n end\n\n let(:sheet_parser) do\n tempfile = Tempfile.new(['sheet', '.xml'])\n tempfile.write(sheet)\n tempfile.rewind\n\n SimpleXlsxReader::Loader::SheetParser.new(\n file_io: tempfile,\n loader: loader\n ).tap { |parser| parser.parse {} }\n end\n\n it 'uses /worksheet/dimension if available' do\n _(sheet_parser.last_cell_letter).must_equal 'C'\n end\n\n it 'uses the last header cell if /worksheet/dimension is missing' do\n sheet.at_xpath('/worksheet/dimension').remove\n _(sheet_parser.last_cell_letter).must_equal 'D'\n end\n\n it 'returns \"A1\" if the dimension is just one cell' do\n sheet.xpath('/worksheet/sheetData/row').remove\n sheet.xpath('/worksheet/dimension').attr('ref', 'A1')\n _(sheet_parser.last_cell_letter).must_equal 'A'\n end\n\n it 'returns nil if the sheet is just one cell, but /worksheet/dimension is missing' do\n sheet.xpath('/worksheet/sheetData/row').remove\n sheet.xpath('/worksheet/dimension').remove\n _(sheet_parser.last_cell_letter).must_be_nil\n end\n end\n\n describe '#column_letter_to_number' do\n let(:subject) { SXR::Loader::SheetParser.new(file_io: nil, loader: nil) }\n\n [\n ['A', 1],\n ['B', 2],\n ['Z', 26],\n ['AA', 27],\n ['AB', 28],\n ['AZ', 52],\n ['BA', 53],\n ['BZ', 78],\n ['ZZ', 702],\n ['AAA', 703],\n ['AAZ', 728],\n ['ABA', 729],\n ['ABZ', 754],\n ['AZZ', 1378],\n ['ZZZ', 18_278]\n ].each do |(letter, number)|\n it \"converts #{letter} to #{number}\" do\n _(subject.column_letter_to_number(letter)).must_equal number\n end\n end\n end\n end\n\n describe 'parse errors' do\n after do\n SimpleXlsxReader.configuration.catch_cell_load_errors = false\n end\n\n let(:sheet) do\n Nokogiri::XML(\n <<-XML\n \n \n \n \n \n 14 is a date style; this is not a date\n \n \n \n \n XML\n ).remove_namespaces!\n end\n\n let(:styles) do\n # s='0' above refers to the value of numFmtId at cellXfs index 0\n Nokogiri::XML(\n <<-XML\n \n \n \n \n \n XML\n ).remove_namespaces!\n end\n\n it 'raises if configuration.catch_cell_load_errors' do\n SimpleXlsxReader.configuration.catch_cell_load_errors = false\n\n _(-> { SimpleXlsxReader.open(xlsx.archive.path).to_hash })\n .must_raise(SimpleXlsxReader::CellLoadError)\n end\n\n it 'records a load error if not configuration.catch_cell_load_errors' do\n SimpleXlsxReader.configuration.catch_cell_load_errors = true\n\n sheet = SimpleXlsxReader.open(xlsx.archive.path).sheets[0].tap(&:slurp)\n _(sheet.load_errors).must_equal(\n [0, 0] => 'invalid value for Float(): \"14 is a date style; this is not a date\"'\n )\n end\n end\n\n describe 'missing numFmtId attributes' do\n let(:sheet) do\n Nokogiri::XML(\n <<-XML\n \n \n \n \n \n some content\n \n \n \n \n XML\n ).remove_namespaces!\n end\n\n let(:styles) do\n Nokogiri::XML(\n <<-XML\n \n\n \n XML\n ).remove_namespaces!\n end\n\n before do\n @row = SimpleXlsxReader.open(xlsx.archive.path).sheets[0].rows.to_a[0]\n end\n\n it 'continues even when cells are missing numFmtId attributes ' do\n _(@row[0]).must_equal 'some content'\n end\n end\n\n describe 'parsing types' do\n let(:sheet) do\n Nokogiri::XML(\n <<-XML\n \n \n \n \n \n Cell A1\n \n\n \n 2.4\n \n \n\n \n 30687\n \n \n\n \n Cell G1\n \n\n \n HYPERLINK(\"http://www.example.com/hyperlink-function\", \"HYPERLINK function\")\n HYPERLINK function\n \n\n \n GUI-made hyperlink\n \n\n \n 1\n \n \n \n\n \n \n \n \n XML\n ).remove_namespaces!\n end\n\n let(:styles) do\n # s='0' above refers to the value of numFmtId at cellXfs index 0,\n # which is in this case 'General' type\n Nokogiri::XML(\n <<-XML\n \n \n \n \n \n \n \n XML\n ).remove_namespaces!\n end\n\n # Although not a \"type\" or \"style\" according to xlsx spec,\n # it sure could/should be, so let's test it with the rest of our\n # typecasting code.\n let(:rels) do\n [\n Nokogiri::XML(\n <<-XML\n \n \n \n XML\n ).remove_namespaces!\n ]\n end\n\n before do\n @row = SimpleXlsxReader.open(xlsx.archive.path).sheets[0].rows.to_a[0]\n end\n\n it \"reads 'Generic' cells as strings\" do\n _(@row[0]).must_equal 'Cell A1'\n end\n\n it \"reads empty 'Generic' cells as nil\" do\n _(@row[1]).must_be_nil\n end\n\n # We could expand on these type tests, but really just a couple\n # demonstrate that it's wired together. Type-specific tests should go\n # on #cast\n\n it 'reads floats' do\n _(@row[2]).must_equal 2.4\n end\n\n it 'reads empty floats as nil' do\n _(@row[3]).must_be_nil\n end\n\n it 'reads dates' do\n _(@row[4]).must_equal Date.parse('Jan 6, 1984')\n end\n\n it 'reads empty date cells as nil' do\n _(@row[5]).must_be_nil\n end\n\n it 'reads strings formatted as inlineStr' do\n _(@row[6]).must_equal 'Cell G1'\n end\n\n it 'reads hyperlinks created via HYPERLINK()' do\n _(@row[7]).must_equal(\n SXR::Hyperlink.new(\n 'http://www.example.com/hyperlink-function', 'HYPERLINK function'\n )\n )\n end\n\n it 'reads hyperlinks created via the GUI' do\n _(@row[8]).must_equal(\n SXR::Hyperlink.new(\n 'http://www.example.com/hyperlink-gui', 'GUI-made hyperlink'\n )\n )\n end\n\n it \"reads 'Generic' cells with numbers as numbers\" do\n _(@row[9]).must_equal 1\n end\n end\n\n describe 'parsing documents with blank rows' do\n let(:sheet) do\n Nokogiri::XML(\n <<-XML\n \n \n \n \n \n a\n \n \n \n \n 1\n \n \n \n \n 2\n \n \n \n \n 3\n \n \n \n \n XML\n ).remove_namespaces!\n end\n\n before do\n @rows = SimpleXlsxReader.open(xlsx.archive.path).sheets[0].rows.to_a\n end\n\n it 'reads row data despite gaps in row numbering' do\n _(@rows).must_equal [\n [nil, nil, nil, nil],\n ['a', nil, nil, nil],\n [nil, nil, nil, nil],\n [nil, 1, nil, nil],\n [nil, nil, 2, nil],\n [nil, nil, nil, nil],\n [nil, nil, nil, 3]\n ]\n end\n end\n\n describe 'parsing documents with non-hyperlinked rels' do\n let(:rels) do\n [\n Nokogiri::XML(\n <<-XML\n \n \n XML\n ).remove_namespaces!\n ]\n end\n\n describe 'when document is opened as path' do\n before do\n @row = SimpleXlsxReader.open(xlsx.archive.path).sheets[0].rows.to_a[0]\n end\n\n it 'reads cell content' do\n _(@row[0]).must_equal 'Cell A'\n end\n end\n\n describe 'when document is parsed as a String' do\n before do\n output = File.binread(xlsx.archive.path)\n @row = SimpleXlsxReader.parse(output).sheets[0].rows.to_a[0]\n end\n\n it 'reads cell content' do\n _(@row[0]).must_equal 'Cell A'\n end\n end\n\n describe 'when document is parsed as StringIO' do\n before do\n stream = StringIO.new(File.binread(xlsx.archive.path), 'rb')\n @row = SimpleXlsxReader.parse(stream).sheets[0].rows.to_a[0]\n stream.close\n end\n\n it 'reads cell content' do\n _(@row[0]).must_equal 'Cell A'\n end\n end\n end\n\n # https://support.microsoft.com/en-us/office/available-number-formats-in-excel-0afe8f52-97db-41f1-b972-4b46e9f1e8d2\n describe 'numeric fields styled as \"General\"' do\n let(:misc_numbers_path) do\n File.join(File.dirname(__FILE__), 'misc_numbers.xlsx')\n end\n\n let(:sheet) { SimpleXlsxReader.open(misc_numbers_path).sheets[0] }\n\n it 'reads medium sized integers as integers' do\n _(sheet.rows.slurp[1][0]).must_equal 98070\n end\n\n it 'reads large (>12 char) integers as integers' do\n _(sheet.rows.slurp[1][1]).must_equal 1234567890123\n end\n end\n\n describe 'with mysteriously chunky UTF-8 text' do\n let(:chunky_utf8_path) do\n File.join(File.dirname(__FILE__), 'chunky_utf8.xlsx')\n end\n\n let(:sheet) { SimpleXlsxReader.open(chunky_utf8_path).sheets[0] }\n\n it 'reads the whole cell text' do\n _(sheet.rows.slurp[1]).must_equal(\n [\"sample-company-1\", \"Korntal-Münchingen\", \"Bronholmer straße\"]\n )\n end\n end\n\n describe 'when using percentages & currencies' do\n let(:pnc_path) do\n # This file provided by a GitHub user having parse errors in these fields\n File.join(File.dirname(__FILE__), 'percentages_n_currencies.xlsx')\n end\n\n let(:sheet) { SimpleXlsxReader.open(pnc_path).sheets[0] }\n\n it 'reads percentages as floats of the form 0.XX' do\n _(sheet.rows.slurp[1][2]).must_equal(0.87)\n end\n\n it 'reads currencies as floats' do\n _(sheet.rows.slurp[1][4]).must_equal(300.0)\n end\n end\nend\n" }, { "path": "test/styles.xml", "content": "\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\n" }, { "path": "test/test_helper.rb", "content": "# frozen_string_literal: true\n\ngem 'minitest'\nrequire 'minitest/autorun'\nrequire 'minitest/spec'\nrequire 'pry'\nrequire 'time'\nrequire 'test_xlsx_builder'\n\n$LOAD_PATH.unshift File.expand_path('lib')\nrequire 'simple_xlsx_reader'\n" }, { "path": "test/test_xlsx_builder.rb", "content": "# frozen_string_literal: true\n\nrequire 'nokogiri'\n\nTestXlsxBuilder = Struct.new(:shared_strings, :styles, :sheets, :workbook, :rels, keyword_init: true) do\n\n DEFAULTS = {\n workbook:\n Nokogiri::XML(\n <<-XML\n \n \n \n \n \n XML\n ).remove_namespaces!,\n\n styles:\n Nokogiri::XML(\n <<-XML\n \n \n \n \n \n XML\n ).remove_namespaces!,\n\n sheet:\n Nokogiri::XML(\n <<-XML\n \n \n \n \n \n Cell A\n \n \n Cell B\n \n \n Cell C\n \n \n \n \n XML\n ).remove_namespaces!\n }\n\n def initialize(*args)\n super\n\n self.workbook ||= DEFAULTS[:workbook]\n self.styles ||= DEFAULTS[:styles]\n self.sheets ||= [DEFAULTS[:sheet]]\n self.rels ||= []\n end\n\n def archive\n tmpfile = Tempfile.new(['workbook', '.xlsx'])\n tmpfile.binmode\n tmpfile.rewind\n\n Zip::File.open(tmpfile.path, create: true) do |zip|\n zip.mkdir('xl')\n\n zip.get_output_stream('xl/workbook.xml') do |wb_file|\n wb_file.write(workbook)\n end\n\n zip.get_output_stream('xl/styles.xml') do |styles_file|\n styles_file.write(styles)\n end\n\n if shared_strings\n zip.get_output_stream('xl/sharedStrings.xml') do |ss_file|\n ss_file.write(shared_strings)\n end\n end\n\n zip.mkdir('xl/worksheets')\n\n sheets.each_with_index do |sheet, i|\n zip.get_output_stream(\"xl/worksheets/sheet#{i + 1}.xml\") do |sf|\n sf.write(sheet)\n end\n\n if rels[i]\n zip.mkdir('xl/worksheets/_rels')\n zip.get_output_stream(\"xl/worksheets/_rels/sheet#{i + 1}.xml.rels\") do |rf|\n rf.write(rels[i])\n end\n end\n end\n end\n\n tmpfile\n end\nend\n\n" } ]